Chicken epidemic disease detection method and device based on audio data enhancement and feature fusion

Abstract

The invention discloses a chicken epidemic disease detection method and device based on audio data enhancement and feature fusion. The method comprises the steps that sound data of a chicken flock is collected and preprocessed; performing data enhancement on the preprocessed sound data to obtain a chicken flock audio training sample; frequency spectrum features and original audio features are extracted from the chicken flock audio training samples; splicing all the spectrum feature maps to obtain a spliced spectrum feature map; inputting the extracted original audio features into a network formed by combining two front-end acoustic models and a pre-training model to obtain a first semantic feature map and a second semantic feature map; fusing all the features to obtain a fused feature map; and inputting the fused feature map into a preset lightweight multilayer perceptron, and outputting a chicken epidemic disease detection result. By introducing data enhancement and feature fusion technologies, the limitation of a single feature extraction means is overcome, and the accuracy and robustness of sound signal processing are improved, so that the chicken epidemic disease is detected more effectively.

Description

Technical Field

[0001] The invention belongs to the technical field of chicken disease detection, and in particular relates to a chicken disease detection method and device based on audio data enhancement and feature fusion. Background Art

[0002] Traditional animal disease detection methods, especially for poultry disease monitoring, mainly rely on manual inspection and collection of blood or feces samples for laboratory testing. These traditional methods are not only time-consuming and labor-intensive, but also require operators to have considerable professional knowledge and skills. More importantly, diseases often begin to spread before showing obvious clinical symptoms, which poses a huge challenge to early detection and prevention of diseases.

[0003] At present, the detection of chicken diseases still mainly relies on manual observation, that is, farmers or veterinarians make a preliminary judgment on whether there is a disease by observing the behavior patterns, appearance and health status of the chickens. However, this method based on individual experience is highly subjective, and its accuracy is limited by the individual's professional level and experience accumulation, making it difficult to achieve ideal detection results. In view of the above problems, disease detection technology based on audio analysis has received widespread attention in recent years. Studies have shown that when chickens are sick, they make sounds that are significantly different from those in a healthy state. By capturing and analyzing these sound signals, early warning of the disease can be achieved. This method has significant advantages such as non-invasiveness, real-time monitoring and automated processing, which can significantly improve the efficiency and accuracy of disease diagnosis.

[0004] like Figure 1 As shown in the figure, this technology usually involves using a microphone array to collect sound data in the chicken house, and then using feature extraction methods such as Mel Frequency Cepstral Coefficient (MFCC) and Linear Predictive Coding (LPC) to process the sound signal. These features can effectively characterize the spectral and time domain characteristics of the sound signal. Furthermore, classification algorithms such as support vector machine (SVM) and random forest (RF) are used to analyze the extracted features to achieve the identification of the disease. Although audio-based disease detection technology has shown certain advantages, it still faces challenges in practical applications. On the one hand, most methods rely only on a single feature extraction technology, which has limited adaptability to complex and changeable environmental sounds and is easily affected by environmental noise, resulting in a decrease in recognition accuracy. On the other hand, current sound recognition algorithms are mainly based on traditional machine learning models, and their performance and efficiency need to be improved when processing large data sets or complex signals. In addition, the lack of data diversity and scale in existing technologies limits the generalization ability of the model and makes it easy to overfit in new environments. Finally, the high hardware cost and complex system deployment process also hinder the large-scale promotion and application of this technology.

[0005] In summary, although the chicken disease monitoring technology based on audio detection shows great potential, it still needs further research and optimization in terms of improving recognition accuracy, enhancing model generalization ability, reducing system costs, and simplifying deployment processes. Summary of the invention

[0006] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and provide a method and device for detecting chicken disease based on audio data enhancement and feature fusion. The present invention overcomes the limitations of a single feature extraction method by introducing data enhancement and feature fusion technology, improves the accuracy and robustness of sound signal processing, and thus more effectively detects chicken disease.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] In a first aspect, the present invention provides a method for detecting chicken disease based on audio data enhancement and feature fusion, comprising the following steps:

[0009] Collect the sound data of the chickens and pre-process the sound data;

[0010] Perform data enhancement on the preprocessed sound data to obtain chicken audio training samples;

[0011] Utilizing the distribution difference between the spectrogram and the original audio, the spectral features and original audio features are extracted from the chicken audio training samples respectively.

[0012] The extracted spectrum features are input into multiple preset models to further mine the features, thereby obtaining multiple spectrum feature graphs, and all the spectrum feature graphs are spliced ​​to obtain a spliced ​​spectrum feature graph;

[0013] Inputting the extracted original audio features into a network respectively composed of two front-end acoustic models and a pre-trained model to obtain a first semantic feature map and a second semantic feature map;

[0014] Performing feature fusion on the spliced ​​spectral feature map, the first semantic feature map and the second semantic feature map to obtain a fused feature map; the feature fusion includes fixed fusion, shared fusion and sampling fusion;

[0015] The fused feature map is input into a preset lightweight multi-layer perceptron to output a chicken disease detection result.

[0016] As a preferred technical solution, the data enhancement performed on the pre-processed sound data includes pitch up-shifting, pitch down-shifting, slow time stretching, fast time stretching and silence elimination.

[0017] As a preferred technical solution, the spectral features are extracted as follows:

[0018] Perform short-time Fourier transform on the audio signal of the chicken flock to extract the time-frequency domain features of the audio signal of the chicken flock;

[0019] Perform Mel frequency transformation on the audio signal of the chicken flock to generate a Mel spectrum graph;

[0020] Performing discrete cosine transform on the Mel-frequency spectrum to extract Mel-frequency cepstrum coefficients;

[0021] The time-frequency domain features, the Mel-spectrogram and the Mel-frequency cepstrum coefficients are all re-cropped to a set size, and then concatenated according to the channel dimension to obtain the spectrum features.

[0022] As a preferred technical solution, the original audio features are extracted as follows:

[0023] The acoustic word segmenter is used to learn the representation of the original audio signal of the chicken flock. The input original audio clip is first divided into multiple frames, and the multiple frames are represented as multiple vectors through the acoustic word segmenter. Then, a position code is added to each input vector; the vector with the added position code is input into the Transformer encoder, and the original audio features are output after encoding.

[0024] As a preferred technical solution, the multiple preset models include VGG13, ResNet50, DenseNet, and EfficientNet.

[0025] As a preferred technical solution, the feature fusion is specifically as follows:

[0026] Fixed fusion, connecting the concatenated spectral feature map, the first semantic feature map and the second semantic feature map, specifically: h = Concat[f spe , f sem ]

[0027] where f sem is the semantic feature map, f spe is the spectrum feature map. Concat is the connection function.

[0028] Shared fusion uses a method similar to the attention mechanism to train the fusion process simultaneously by conditionally weighting the features. Specifically:

[0029] First, a weighted matrix is ​​generated for the spectrum graph and a weighted matrix is ​​generated for the semantic feature, which are represented as W spe and W sem :

[0030] W spe =Sigmoid(Linear(f sem ))

[0031] W sem=Sigmoid(Linear(f spe ))

[0032] Linear represents the linear layer, and Sigmoid is the activation function. Then the weighted feature map is calculated, represented as h spe and h sem :

[0033] h spe =f spe ⊙W sem

[0034] h sem =f sem ⊙W spe

[0035] Finally, the obtained feature maps are fused together, and the formula is:

[0036] h=Concat[h spe ,h sem ]

[0037] Sampling fusion, specifically:

[0038] By learning a random function, sampling from the feature map is performed to achieve more robust feature fusion. The calculation formula is as follows:

[0039] f′ spe =f spe ⊙ReLU(Linear(f spe ))

[0040] f′ sem =f sem ⊙ReLU(Linear(f sem ))

[0041] Where ReLU is the activation function and ⊙ is the Hadamard product. Next, the original features are sampled using the Gumbel-Softmax distribution:

[0042] s spe ~p(s spe |f spe ,τ1)=GS(f spe )

[0043] s sem ~p(s sem |f sem ,τ2)=GS(f sem )

[0044] Finally, the new representation of the Gumbel-Softmax distribution domain is element-wise multiplied to obtain the final fusion feature representation:

[0045] h = Concat[f′ spe ⊙s spe ; f′ sem ⊙s sem ].

[0046] As a preferred technical solution, the fusion feature map is input into a preset lightweight multi-layer perceptron to output the chicken disease detection result, specifically:

[0047] First, the fused feature map h is globally averaged and pooled into a shape h that can be input into the linear layer. gap , and then input into the multi-layer perceptron, expressed as:

[0048]

[0049] in is the sigmoid function, which is used to convert the output into a probability value, indicating the probability that the input belongs to the positive class.

[0050] In a second aspect, the present invention provides a chicken disease detection system based on audio data enhancement and feature fusion, which is applied to the chicken disease detection method based on audio data enhancement and feature fusion, and includes a data acquisition module, a data enhancement module, a feature extraction module, a spectrum feature processing module, a semantic feature processing module, a feature fusion module and a detection module;

[0051] The data acquisition module is used to collect the sound data of the chickens and pre-process the sound data;

[0052] The data enhancement module is used to perform data enhancement on the pre-processed sound data to obtain a chicken flock audio training sample;

[0053] The feature extraction module is used to extract spectrum features and original audio features from the chicken audio training samples by utilizing the distribution difference between the spectrum graph and the original audio;

[0054] The spectrum feature processing module is used to input the extracted spectrum features into multiple preset models to further mine the features, obtain multiple spectrum feature graphs, and splice all the spectrum feature graphs to obtain a spliced ​​spectrum feature graph;

[0055] The semantic feature processing module is used to input the extracted original audio features into a network respectively composed of two front-end acoustic models and a pre-trained model to obtain a first semantic feature map and a second semantic feature map;

[0056] The feature fusion module is used to perform feature fusion on the spliced ​​spectrum feature map, the first semantic feature map and the second semantic feature map to obtain a fused feature map; the feature fusion includes fixed fusion, shared fusion and sampling fusion;

[0057] The detection module is used to input the fused feature map into a preset lightweight multi-layer perceptron and output a chicken disease detection result.

[0058] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0059] at least one processor; and,

[0060] a memory communicatively connected to the at least one processor; wherein,

[0061] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the chicken disease detection method based on audio data enhancement and feature fusion.

[0062] In a fourth aspect, the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the chicken disease detection method based on audio data enhancement and feature fusion.

[0063] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0064] 1. The present invention has significant advantages in multimodal feature fusion, data enhancement technology, deep learning algorithm, system optimization and low-cost deployment. By fusing frequency domain features, time domain features and time-frequency domain features, the ability to express complex sound signals is improved, and the interference of environmental noise is effectively reduced. A variety of feature extraction methods such as short-time Fourier transform (STFT), Mel-spectrogram and Mel-frequency cepstral coefficient (MFCC) are used, and comprehensive processing is performed through a feature fusion module.

[0065] 2. The present invention uses data enhancement technology to generate more diverse training samples, improve the generalization ability of the model, reduce overfitting, and make the model more stable and reliable in practical applications. Data enhancement methods include pitch upshift, pitch downshift, time stretching, and silence removal.

[0066] 3. This invention uses advanced deep learning algorithms to improve the processing efficiency of large-scale and highly complex sound data, ensuring the high efficiency of real-time detection. Combined with audio front-end modules such as BEATs and LEAF, the multi-layer Transformer encoder is used to further explore sound features, improving the accuracy of recognition and processing speed.

[0067] 4. The present invention reduces hardware requirements and simplifies the system deployment process by optimizing algorithms and system architecture design, achieving the goal of low cost and large-scale promotion and application. A lightweight multi-layer perceptron (MLP) is designed for classification, which improves the system's computing efficiency and resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0069] Figure 1 It is a flow chart of a chicken disease detection method in the prior art;

[0070] Figure 2 This is a schematic diagram of a chicken disease detection method based on audio data enhancement and feature fusion according to an embodiment of the present invention;

[0071] Figure 3 This is a schematic diagram of fixed fusion of features of an embodiment of the present invention;

[0072] Figure 4 This is a schematic diagram of feature sharing and fusion in an embodiment of the present invention;

[0073] Figure 5 This is a schematic diagram of feature sampling and fusion in an embodiment of the present invention;

[0074] Figure 6 The present invention is a block diagram of a chicken disease detection system based on audio data enhancement and feature fusion according to an embodiment of the present invention.

[0075] Figure 7 4 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0076] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0077] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0078] This embodiment provides a chicken disease detection method based on audio data enhancement and feature fusion, which collects and analyzes the sound signals of chickens to identify whether the chickens are sick. Figure 2 As shown, including the following steps

[0079] S1. Collect the sound data of the chickens and pre-process the sound data.

[0080] Furthermore, the sound data of the chickens is collected in real time through a microphone array installed in the chicken farm. These microphones can efficiently capture various sounds in the chicken house and convert them into digital audio signals for subsequent processing.

[0081] Furthermore, the preprocessing of the sound data is mainly to reduce noise, and the collected audio signals of the chicken flock are preprocessed by denoising, normalization, etc. to remove noise and interference signals in the chicken farm environment.

[0082] In a specific embodiment, the noise reduction process is specifically:

[0083] S11, input a random noise segment sample, and obtain the frequency spectrum of the noise segment by calculation;

[0084] S12, subtracting the characteristic frequency component corresponding to the frequency spectrum of the noise segment from the collected audio data of the chicken flock to complete the noise reduction process.

[0085] S2. Perform data enhancement on the preprocessed sound data to obtain chicken audio training samples.

[0086] Furthermore, more diverse chicken audio training samples are generated through data enhancement technology to improve the generalization ability of the model. The data enhancement methods include:

[0087] 1) Pitch Shift (PPS): Raise the pitch of an audio clip without changing its playback speed.

[0088] 2) NPS: Lowers the pitch of an audio clip without changing its playback speed.

[0089] 3) Slow Time Stretch (STS): Lengthens the duration of an audio clip.

[0090] 4) Fast Time Stretch (FTS): shortens the duration of an audio clip.

[0091] 5) Silence Removal (TS): Detects and removes quiet or silent parts of an audio clip.

[0092] S3. Utilizing the distribution difference between the spectrogram and the original audio, the spectrum features and the original audio features are extracted for the chicken audio training samples respectively.

[0093] Furthermore, step S3 is specifically as follows:

[0094] S31. The frequency spectrum characteristics of the chicken audio signal include the following:

[0095] S311, Short-time Fourier Transform (STFT): By performing short-time Fourier transform on the chicken audio signal, its time-frequency domain features are extracted. SIFT can provide local information of the audio signal in time and frequency, which helps to capture the dynamic change characteristics of the chicken audio signal. Its formula is:

[0096]

[0097] In the formula, τ is the integral variable used to scan the value of the signal x(τ) on the entire time axis. t is the current time point. (τ-t) is the window function, which depends on the difference between τ and t and is used to intercept a part of the signal. -jωτ It is the kernel function of Fourier transform, which is used to convert time domain signals into frequency domain.

[0098] S312, Mel-spectrogram: generate a Mel-spectrogram by performing a Mel-frequency transformation on the chicken audio signal. The formula is:

[0099] LogMel(t,m)=log(∑ ω |X(t,ω)| 2 H m (ω)+∈)

[0100] Where X(t, ω) is the STFT result, H m (ω) is the frequency response of the mth Mel filter. ∈ is a small constant used to prevent zero values ​​from appearing when taking the logarithm.

[0101] S313, Mel frequency cepstral coefficient (MFCC): extract the Mel frequency cepstral coefficient by performing discrete cosine transform on the Mel frequency spectrum in the above text. The formula is:

[0102]

[0103] Where n is the coefficient index of MFCC and M is the number of Mel filters.

[0104] S314, the obtained short-time Fourier transform, Mel-frequency spectrum and Mel-frequency cepstrum coefficients are all re-cropped to a size of 224×224, and then concatenated according to the channel dimension to a size of [3, 224, 224].

[0105] S32, the original audio features of the chicken flock are as follows:

[0106] S321, Bidirectional Encoder Representation of Audio Transformer (BEATs): Learn the raw audio signal representation of the chicken flock by using an acoustic word segmenter. The input audio clip is first segmented into small time segments (frames), which are represented as a series of vectors X1, X2...X by the acoustic word segmenter. T , and then add the position code P to each input vector:

[0107] H 0 =X+P

[0108] S322, through the L-layer Transformer encoder, the calculation of each layer is as follows:

[0109] H l =LayerNorm(H l-1 +MultiHeadAttention(H l-1 , H l-1 , H l-1 ))

[0110] H l =LayerNorm(H l +FFN(H l ))

[0111] Among them, MultiHeadAttention is a multi-head attention mechanism, and FFN is a feedforward neural network. After L layers of encoders, the final output Z = H L .

[0112] S323, Trainable Audio Front End (LEAF): Use machine learning models to learn and optimize the processing of chicken audio signals. The formula is:

[0113]

[0114] Where x(t) is the input audio signal, f is the center frequency of the trainable Gabor filter, σ is the standard deviation of the trainable Gabor filter, φ is the trainable nonlinear transformation function, N is the size of the pooling window, and * represents the convolution operation.

[0115] S4. Input the extracted spectrum features into multiple preset models to further mine the features, obtain multiple spectrum feature graphs, and splice all the spectrum feature graphs to obtain a spliced ​​spectrum feature graph.

[0116] Furthermore, step S4 is specifically as follows:

[0117] The extracted audio spectrogram features of the chicken flock are input into various models, including VGG13, ResNet50, DenseNet, and EfficientNet, to further mine the features and obtain the spectral feature map (Spectral feature map)f 1 , f 2 , f 3 , f 4 , and then concatenate all the spectral feature maps to obtain the concatenated spectral feature map f spe .

[0118] S5. Input the extracted original audio features into a network respectively composed of two front-end acoustic models and a pre-trained model to obtain a first semantic feature map and a second semantic feature map.

[0119] Furthermore, step S5 is specifically as follows:

[0120] The original audio signal of the chicken flock is input into the network which is composed of two front-end acoustic models and the pre-trained model ResNet18 to obtain the semantic feature map. (Choose one of the two semantic features).

[0121] S6. Perform feature fusion on the spliced ​​spectral feature map, the first semantic feature map and the second semantic feature map to obtain a fused feature map; the feature fusion includes fixed fusion, shared fusion and sampling fusion.

[0122] Furthermore, step S6 is specifically as follows:

[0123] S61. Use the following fusion strategies to perform feature fusion;

[0124] S611, Fixed Fusion: Connect feature maps together, such as Figure 3 The formula is as follows:

[0125] h=Concat[f spe , f sem ]

[0126] where f sem is the semantic feature map, f spe is the spectrum feature map, Concat is the connection function;

[0127] S612, Shared Fusion: Use a method similar to the attention mechanism to simultaneously train the fusion process by conditionally weighting the features, such as Figure 4 As shown. First, a weighting matrix is ​​generated for the spectrum graph (using the semantic feature graph) and a weighting matrix is ​​generated for the semantic feature (using the spectrum feature graph), denoted as W spe and W sem :

[0128] W spe =Sigmoid(Linear(f sem ))

[0129] W sem =Sigmoid(Linear(f spe ))

[0130] Linear represents the linear layer, and Sigmoid is the activation function. Then the weighted feature map is calculated, represented as h spe and h sem :

[0131] h spe =f spe ⊙W sem

[0132] h sem =f sem ⊙W spe

[0133] Finally, the obtained feature maps are fused together, and the formula is:

[0134] h=Concat[h spe ,h sem ]

[0135] S613, Sampling Fusion: Sampling fusion learns a random function and samples from the feature map to achieve more robust feature fusion, such as Figure 5 The following are the detailed steps and formulas:

[0136] f′ spe =f spe ⊙ReLU(Linear(f spe ))

[0137] f′ sem =f sem ⊙ReLU(Linear(f sem ))

[0138] Where ReLU is the activation function, ⊙ is the Hadamard product;

[0139] S62. Next, use Gumbel-Softmax (GS) distribution to sample the original features:

[0140] s spe ~p(s spe |f spe ,τ1)=GS(f spe )

[0141] s sem ~p(s sem |f sem ,τ2)=GS(f sem )

[0142] S63. Finally, perform element-wise multiplication on the new representation of the GS distribution domain to obtain the final fusion feature representation:

[0143] h = Concat[f′ spe ⊙s spe ; f′ sem ⊙s sem ]

[0144] S7, inputting the fused feature map into a preset lightweight multi-layer perceptron, and outputting the chicken disease detection result.

[0145] Furthermore, step S7 is specifically as follows:

[0146] The fused features are input into a lightweight multi-layer perceptron to output the binary classification result of chicken disease detection, that is, whether the chickens are sick or not. First, h is subjected to global average pooling (GAP) to a shape h that can be input into the linear layer. gap , and then input into the multi-layer perceptron, the formula is:

[0147]

[0148] in is the sigmoid function, which is used to convert the output into a probability value, indicating the probability that the input belongs to the positive class.

[0149] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0150] Based on the same idea as the chicken disease detection method based on audio data enhancement and feature fusion in the above embodiment, the present invention also provides a chicken disease detection system based on audio data enhancement and feature fusion, which can be used to execute the above chicken disease detection method based on audio data enhancement and feature fusion. For ease of explanation, in the structural schematic diagram of the chicken disease detection system embodiment based on audio data enhancement and feature fusion, only the parts related to the embodiment of the present invention are shown. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than the illustrated one, or combine certain components, or arrange the components differently.

[0151] See also Figure 6 In another embodiment of the present application, a chicken disease detection system 100 based on audio data enhancement and feature fusion is provided, the system comprising a data acquisition module 101, a data enhancement module 102, a feature extraction module 103, a spectrum feature processing module 104, a semantic feature processing module 105, a feature fusion module 106 and a detection module 107;

[0152] The data collection module 101 is used to collect the sound data of the chickens and pre-process the sound data;

[0153] The data enhancement module 102 is used to perform data enhancement on the pre-processed sound data to obtain a chicken flock audio training sample;

[0154] The feature extraction module 103 is used to extract the spectrum features and the original audio features of the chicken audio training samples respectively by utilizing the distribution difference between the spectrum graph and the original audio;

[0155] The spectrum feature processing module 104 is used to input the extracted spectrum features into multiple preset models to further mine the features, obtain multiple spectrum feature graphs, and splice all the spectrum feature graphs to obtain a spliced ​​spectrum feature graph;

[0156] The semantic feature processing module 105 is used to input the extracted original audio features into a network respectively composed of two front-end acoustic models and a pre-trained model to obtain a first semantic feature map and a second semantic feature map;

[0157] The feature fusion module 106 is used to perform feature fusion on the spliced ​​spectrum feature map, the first semantic feature map and the second semantic feature map to obtain a fused feature map; the feature fusion includes fixed fusion, shared fusion and sampling fusion;

[0158] The detection module 107 is used to input the fused feature map into a preset lightweight multi-layer perceptron and output a chicken disease detection result.

[0159] It should be noted that the chicken disease detection system based on audio data enhancement and feature fusion of the present invention corresponds one to one with the chicken disease detection method based on audio data enhancement and feature fusion of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the chicken disease detection method based on audio data enhancement and feature fusion are applicable to the embodiment of chicken disease detection based on audio data enhancement and feature fusion. For specific contents, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here. It is hereby declared.

[0160] In addition, in the implementation of the chicken disease detection system based on audio data enhancement and feature fusion in the above-mentioned embodiment, the logical division of each program module is only an example. In actual application, the above-mentioned functions can be assigned to different program modules as needed, for example, for the convenience of corresponding hardware configuration requirements or software implementation. That is, the internal structure of the chicken disease detection system based on audio data enhancement and feature fusion is divided into different program modules to complete all or part of the functions described above.

[0161] See also Figure 7 In one embodiment, an electronic device for implementing a chicken disease detection method based on audio data enhancement and feature fusion is provided. The electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as a chicken disease detection program 203 based on audio data enhancement and feature fusion.

[0162] Among them, the first memory 202 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart memory card (Sma rt Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 200. Further, the first memory 202 can also include both an internal storage unit of the electronic device 200 and an external storage device. The first memory 202 can not only be used to store application software and various types of data installed in the electronic device 200, such as the code of the chicken disease detection program 203 based on audio data enhancement and feature fusion, but also can be used to temporarily store data that has been output or is to be output.

[0163] In some embodiments, the first processor 201 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The first processor 201 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions and processes data of the electronic device 200 by running or executing programs or modules stored in the first memory 202, and calling data stored in the first memory 202.

[0164] Figure 7 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 7 The structure shown does not constitute a limitation on the electronic device 200 , and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0165] The chicken disease detection program 203 based on audio data enhancement and feature fusion stored in the first memory 202 in the electronic device 200 is a combination of multiple instructions. When running in the first processor 201, it can achieve:

[0166] Collect the sound data of the chickens and pre-process the sound data;

[0167] Perform data enhancement on the preprocessed sound data to obtain chicken audio training samples;

[0168] Utilizing the distribution difference between the spectrogram and the original audio, the spectral features and original audio features are extracted from the chicken audio training samples respectively.

[0169] The extracted spectrum features are input into multiple preset models to further mine the features, thereby obtaining multiple spectrum feature graphs, and all the spectrum feature graphs are spliced ​​to obtain a spliced ​​spectrum feature graph;

[0170] Inputting the extracted original audio features into a network respectively composed of two front-end acoustic models and a pre-trained model to obtain a first semantic feature map and a second semantic feature map;

[0171] Performing feature fusion on the spliced ​​spectral feature map, the first semantic feature map and the second semantic feature map to obtain a fused feature map; the feature fusion includes fixed fusion, shared fusion and sampling fusion;

[0172] The fused feature map is input into a preset lightweight multi-layer perceptron to output a chicken disease detection result.

[0173] Furthermore, if the module / unit integrated in the electronic device 200 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0174] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0175] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0176] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A chicken disease detection method based on audio data enhancement and feature fusion, characterized in that: Includes the following steps 2 Collect the sound data of the chickens and pre-process the sound data; Perform data enhancement on the preprocessed sound data to obtain chicken audio training samples; Utilizing the distribution difference between the spectrogram and the original audio, the spectral features and original audio features are extracted from the chicken audio training samples respectively. The extracted spectrum features are input into multiple preset models to further mine the features, thereby obtaining multiple spectrum feature graphs, and all the spectrum feature graphs are spliced ​​to obtain a spliced ​​spectrum feature graph; Inputting the extracted original audio features into a network respectively composed of two front-end acoustic models and a pre-trained model to obtain a first semantic feature map and a second semantic feature map; Performing feature fusion on the spliced ​​spectral feature map, the first semantic feature map and the second semantic feature map to obtain a fused feature map; the feature fusion includes fixed fusion, shared fusion and sampling fusion; The fused feature map is input into a preset lightweight multi-layer perceptron to output a chicken disease detection result.

2. The chicken disease detection method based on audio data enhancement and feature fusion according to claim 1 is characterized in that: The data enhancement performed on the pre-processed sound data includes pitch up-shifting, pitch down-shifting, slow time stretching, fast time stretching and silence elimination.

3. The chicken disease detection method based on audio data enhancement and feature fusion according to claim 1 is characterized in that: The specific extraction of spectrum features is as follows: Perform short-time Fourier transform on the audio signal of the chicken flock to extract the time-frequency domain features of the audio signal of the chicken flock; Perform Mel frequency transformation on the audio signal of the chicken flock to generate a Mel spectrum graph; Performing discrete cosine transform on the Mel-frequency spectrum to extract Mel-frequency cepstrum coefficients; The time-frequency domain features, the Mel-spectrogram and the Mel-frequency cepstrum coefficients are all re-cropped to a set size, and then concatenated according to the channel dimension to obtain the spectrum features.

4. The chicken disease detection method based on audio data enhancement and feature fusion according to claim 1 is characterized in that: Extracting original audio features is as follows: The acoustic word segmenter is used to learn the representation of the original audio signal of the chicken flock. The input original audio clip is first divided into multiple frames, and the multiple frames are represented as multiple vectors through the acoustic word segmenter. Then, a position code is added to each input vector; the vector with the added position code is input into the Transformer encoder, and the original audio features are output after encoding.

5. The chicken disease detection method based on audio data enhancement and feature fusion according to claim 1 is characterized in that: The multiple preset models include VGG13, ResNet50, DenseNet, and EfficientNet.

6. The chicken disease detection method based on audio data enhancement and feature fusion according to claim 1 is characterized in that: The feature fusion is specifically as follows: Fixed fusion connects the concatenated spectral feature map, the first semantic feature map and the second semantic feature map, specifically: h=Concat[f spe ,f sem ] where f sem is the semantic feature map, f spe is the spectrum feature map, Concat is the connection function; Shared fusion uses a method similar to the attention mechanism to simultaneously train the fusion process by conditionally weighting the features, specifically: First, a weighted matrix is ​​generated for the spectrum graph and a weighted matrix is ​​generated for the semantic feature, which are represented as W spe and W sem : W spe =Sigmoid(Linear(f sem )) W sem =Sigmoid(Linear(f spe )) Linear represents the linear layer, Sigmoid is the activation function; then the weighted feature map is calculated, represented as h spe and h sem : h spe =f spe ⊙W sem h sem =f sem ⊙W spe Finally, the obtained feature maps are fused together, and the formula is: h=Concat[h spe ,h sem ] Sampling fusion, specifically: By learning a random function, sampling from the feature map is performed to achieve more robust feature fusion. The calculation formula is as follows: f′ spe =f spe ⊙ReLU(Linear(f spe )) f′ sem =f sem ⊙ReLU(Linear(f sem )) Where ReLU is the activation function, ⊙ is the Hadamard product; Next, the original features are sampled using the Gumbel-Soffmax distribution: s spe ~p(s spe |f spe ,τ1)=GS(f spe ) s sem ~p(s sem |f sem ,τ2)=GS(f sem ) Finally, the new representation of the Gumbel-Softmax distribution domain is element-wise multiplied to obtain the final fused feature representation: h=Concat[f′ spe ⊙s spe ;f′ sem ⊙s sem ]。 7. The chicken disease detection method based on audio data enhancement and feature fusion according to claim 1 is characterized in that: The fusion feature map is input into a preset lightweight multi-layer perceptron to output the chicken disease detection result, specifically: First, the fused feature map h is globally averaged and pooled into a shape h that can be input into the linear layer. gap , and then input into the multi-layer perceptron, expressed as: in is the sigmoid function, which is used to convert the output into a probability value, indicating the probability that the input belongs to the positive class.

8. A chicken disease detection system based on audio data enhancement and feature fusion, characterized in that: The chicken disease detection method based on audio data enhancement and feature fusion applied to any one of claims 1-7 comprises a data acquisition module, a data enhancement module, a feature extraction module, a spectrum feature processing module, a semantic feature processing module, a feature fusion module and a detection module; The data acquisition module is used to collect the sound data of the chickens and pre-process the sound data; The data enhancement module is used to perform data enhancement on the pre-processed sound data to obtain a chicken flock audio training sample; The feature extraction module is used to extract spectrum features and original audio features from the chicken audio training samples by utilizing the distribution difference between the spectrum graph and the original audio; The spectrum feature processing module is used to input the extracted spectrum features into multiple preset models to further mine the features, obtain multiple spectrum feature graphs, and splice all the spectrum feature graphs to obtain a spliced ​​spectrum feature graph; The semantic feature processing module is used to input the extracted original audio features into a network respectively composed of two front-end acoustic models and a pre-trained model to obtain a first semantic feature map and a second semantic feature map; The feature fusion module is used to perform feature fusion on the spliced ​​spectrum feature map, the first semantic feature map and the second semantic feature map to obtain a fused feature map; the feature fusion includes fixed fusion, shared fusion and sampling fusion; The detection module is used to input the fused feature map into a preset lightweight multi-layer perceptron and output a chicken disease detection result.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the chicken disease detection method based on audio data enhancement and feature fusion as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the chicken disease detection method based on audio data enhancement and feature fusion described in any one of claims 1 to 7 is implemented.

Images