Method and device for detecting fowl disease based on audio data enhancement and feature fusion

Abstract

The application discloses a kind of based on audio data enhancement and feature fusion's chicken disease detection method and device, method includes: the sound data of chicken population is collected and pretreated;The data enhancement is carried out to the sound data after pretreatment, and chicken population audio training sample is obtained;The frequency spectrum feature and original audio feature are extracted to chicken population audio training sample respectively;All frequency spectrum feature maps are spliced to obtain spliced frequency spectrum feature map;The extracted original audio feature is input into the network combined by two kinds of front-end acoustic model and pre-training model respectively to obtain first semantic feature map and second semantic feature map;All features are fused to obtain fusion feature map;The fusion feature map is input into the preset lightweight multilayer perception, and the output chicken disease detection result is output.The application introduces data enhancement and feature fusion technology, overcomes the limitation of single feature extraction means, improves the accuracy and robustness of sound signal processing, so that chicken disease is more effectively detected.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of chicken disease detection, and particularly relates to a chicken disease detection method and device based on audio data enhancement and feature fusion. BACKGROUND

[0002] Traditional animal disease detection methods, especially for poultry disease monitoring, mainly rely on manual inspection and collection of blood or fecal samples for laboratory testing. These traditional methods are not only time-consuming and labor-intensive, but also require operators to have considerable professional knowledge and skills. More importantly, diseases often begin to spread before showing obvious clinical symptoms, which poses a great challenge to early detection and prevention of diseases.

[0003] Currently, the detection of chicken diseases still mainly relies on manual observation, that is, by breeders or veterinarians observing the behavior patterns, appearance and health status of the chicken flock to preliminarily judge whether there is a disease. However, this method based on individual experience is highly subjective, and the accuracy is subject to the professional level and experience accumulation of individuals, making it difficult to achieve ideal detection results. In view of the above problems, disease detection technology based on audio analysis has received extensive attention in recent years. Studies have shown that chickens will emit significantly different sounds when they are sick. By capturing and analyzing these sound signals, early warning of diseases can be achieved. This method has the advantages of non-invasiveness, real-time monitoring and automation, which can significantly improve the efficiency and accuracy of disease diagnosis.

[0004] As shown in Figure 1 , this technology usually involves using a microphone array to collect sound data inside the chicken coop, and then using feature extraction methods such as Mel Frequency Cepstral Coefficients (MFCC) and Linear Predictive Coding (LPC) to process the sound signals. These features can effectively represent the frequency and time domain characteristics of the sound signals. Further, classification algorithms such as Support Vector Machine (SVM) and Random Forest (RF) are used to analyze the extracted features, thereby realizing the recognition of diseases. Although the audio-based disease detection technology has shown certain advantages, it still faces challenges in practical application. On the one hand, most methods rely only on a single feature extraction technique, which has limited adaptability to complex and variable environmental sounds and is easily affected by environmental noise, resulting in a decrease in recognition accuracy. On the other hand, current sound recognition algorithms are mainly based on traditional machine learning models, and their performance and efficiency in processing large data sets or complex signals need to be improved. In addition, the lack of data diversity and scale in existing technologies limits the generalization ability of the model and makes it prone to overfitting in new environments. Finally, the high hardware cost and complex system deployment process hinder the large-scale promotion and application of this technology.

[0005] In summary, although the audio detection-based chicken disease monitoring technology has great potential, there is still room for further research and optimization in terms of improving recognition accuracy, enhancing model generalization ability, reducing system cost, and simplifying deployment process. SUMMARY

[0006] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and to provide a chicken disease detection method and device based on audio data enhancement and feature fusion. The present application overcomes the limitations of single feature extraction method by introducing data enhancement and feature fusion technology, improves the accuracy and robustness of sound signal processing, and thus more effectively detects chicken diseases.

[0007] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0008] In a first aspect, the present application provides a chicken disease detection method based on audio data enhancement and feature fusion, comprising the following steps:

[0009] Collecting sound data of the chicken flock and pre-processing the sound data;

[0010] Data enhancement is performed on the pre-processed sound data to obtain chicken flock audio training samples;

[0011] Spectrum features and original audio features are extracted from the chicken flock audio training samples using the distribution difference between the spectrum graph and the original audio;

[0012] The extracted spectrum features are input into multiple pre-set models to further extract features, obtaining multiple spectrum feature maps, and all spectrum feature maps are spliced to obtain a spliced spectrum feature map;

[0013] The extracted original audio features are input into a network combined by two kinds of front-end acoustic models and pre-training models to obtain a first semantic feature map and a second semantic feature map;

[0014] The spliced spectrum feature map, the first semantic feature map and the second semantic feature map are fused to obtain a fusion feature map; the feature fusion includes fixed fusion, shared fusion and sampling fusion;

[0015] The fusion feature map is input into a pre-set light multi-layer perception machine to output a chicken disease detection result.

[0016] As a preferred technical solution, the data enhancement of the pre-processed sound data includes pitch upshift, pitch downshift, slow time stretch, fast time stretch and silence elimination.

[0017] As a preferred technical solution, the extraction of spectrum features is as follows:

[0018] The chicken audio signal is subjected to short-time Fourier transform, and the time-frequency domain feature of the chicken audio signal is extracted;

[0019] The chicken audio signal is subjected to Mel frequency transform, and a Mel spectrogram is generated;

[0020] The Mel spectrogram is subjected to discrete cosine transform, and a Mel frequency cepstrum coefficient is extracted;

[0021] The time-frequency domain feature, the Mel spectrogram and the Mel frequency cepstrum coefficient are all re-clipped to a set size, and then spliced according to the channel dimension to obtain a spectrum feature.

[0022] As a preferred technical solution, the extraction of the original audio feature is specifically:

[0023] The original audio signal representation learning of the chicken is performed by an acoustic segmenter. The input original audio segment is first segmented into multiple frames, and the multiple frames are represented as multiple vectors by the acoustic segmenter. Then, a position encoding is added to each input vector. The vector with the added position encoding is input into a Transformer encoder, and the original audio feature is output after encoding.

[0024] As a preferred technical solution, the plurality of preset models include VGG13, ResNet50, DenseNet and EfficientNet.

[0025] As a preferred technical solution, the feature fusion is specifically:

[0026] Fixed fusion, the spliced spectrum feature map, the first semantic feature map and the second semantic feature map are connected, specifically: h = Concat [f spe , f sem ]

[0027] Where f sem is a semantic feature map, and f spe is a spectrum feature map. Concat is a connection function.

[0028] Shared fusion, using a method similar to the attention mechanism, the fusion process is trained simultaneously by conditionally weighting the features, specifically:

[0029] First, a weighting matrix is generated for the spectrum map and a weighting matrix is generated for the semantic feature, denoted as W spe and W sem :

[0030] W spe = Sigmoid (Linear (f sem ))

[0031] W sem= Sigmoid(Linear(f spe ))

[0032] where Linear denotes a linear layer and Sigmoid is an activation function. Then the weighted feature maps are calculated and denoted as h spe and h sem :

[0033] h spe = f spe ⊙W sem

[0034] h sem = f sem ⊙W spe

[0035] Finally, the resulting feature maps are fused together, and the formula is:

[0036] h = Concat[h spe , h sem ]

[0037] Sampling fusion, specifically:

[0038] By learning a random function, sampling is performed from the feature maps to achieve more robust feature fusion, and the calculation formula is as follows:

[0039] f′ spe = f spe ⊙ReLU(Linear(f spe ))

[0040] f′ sem = f sem ⊙ReLU(Linear(f sem ))

[0041] where ReLU is an activation function and ⊙ is Hadamard product. Next, the original features are sampled using Gumbel-Softmax distribution respectively:

[0042] s spe ~ p(s spe | f spe , τ1) = GS(f spe )

[0043] s sem ~ p(s sem | f sem , τ2) = GS(f sem )

[0044] Finally, the new representation of the Gumbel-Softmax distribution domain is obtained by element multiplication operation to obtain the final fused feature representation:

[0045] h = Concat [f′ spe ⊙s spe ; f′ sem ⊙s sem ].

[0046] As a preferred technical solution, the fusion feature map is input into a preset lightweight multi-layer perception, and a chicken disease detection result is output, specifically:

[0047] First, the fused feature map h is globally averaged pooled into a shape h that can be input into a linear layer gap , and then input into a multi-layer perception, represented as:

[0048]

[0049] wherein is a sigmoid function used to convert the output into a probability value, representing the probability of the input belonging to the positive class.

[0050] In a second aspect, the present application provides a chicken disease detection system based on audio data enhancement and feature fusion, applied to the chicken disease detection method based on audio data enhancement and feature fusion, comprising a data acquisition module, a data enhancement module, a feature extraction module, a spectral feature processing module, a semantic feature processing module, a feature fusion module and a detection module.

[0051] The data acquisition module is used to acquire sound data of a chicken flock and pre-process the sound data.

[0052] The data enhancement module is used to perform data enhancement on the pre-processed sound data to obtain chicken flock audio training samples.

[0053] The feature extraction module is used to extract spectral features and original audio features from the chicken flock audio training samples by utilizing the distribution difference between the spectral graph and the original audio.

[0054] The spectral feature processing module is used to input the extracted spectral features into a plurality of preset models to further mine features, obtain a plurality of spectral feature maps, and splice the spectral feature maps to obtain a spliced spectral feature map.

[0055] The semantic feature processing module is used to input the extracted original audio features into a network combined by two front-end acoustic models and pre-training models to obtain a first semantic feature map and a second semantic feature map.

[0056] The feature fusion module is used to fuse the spliced spectral feature map, the first semantic feature map and the second semantic feature map to obtain a fusion feature map; the feature fusion includes fixed fusion, shared fusion and sampling fusion.

[0057] The detection module is configured to input the fused feature map into a preset lightweight multi-layer perception, and output a chicken disease detection result.

[0058] In a third aspect, the present application provides an electronic device, which comprises:

[0059] at least one processor; and

[0060] a memory in communication with the at least one processor; wherein

[0061] The memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the chicken disease detection method based on audio data enhancement and feature fusion.

[0062] In a fourth aspect, the present application provides a computer readable storage medium storing a program, and the program is executed by a processor to implement the chicken disease detection method based on audio data enhancement and feature fusion.

[0063] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0064] 1. The present application has significant advantages in multi-modal feature fusion, data enhancement technology, deep learning algorithm, system optimization and low-cost deployment, etc. By fusing frequency domain features, time domain features and time-frequency domain features, the expression ability of complex sound signals is improved, and the interference of environmental noise is effectively reduced. A variety of feature extraction methods such as short-time Fourier transform (STFT), Mel-spectrogram and mel-frequency cepstral coefficient (MFCC) are adopted, and comprehensive processing is performed through a feature fusion module.

[0065] 2. The present application uses data enhancement technology to generate more diverse training samples, improves the generalization ability of the model, reduces the overfitting phenomenon, and makes the model perform more stably and reliably in actual application. The data enhancement method includes pitch up, pitch down, time stretching and removing silence, etc.

[0066] 3. The present application adopts advanced deep learning algorithm, improves the processing efficiency of large-scale and high-complexity sound data, and ensures the efficiency of real-time detection. Combined with BEATs and LEAF audio front-end modules, the sound features are further excavated by using multi-layer Transformer encoder, which improves the recognition accuracy and processing speed.

[0067] 4. The application optimizes the algorithm and system architecture design, reduces the hardware device requirements, simplifies the system deployment process, and realizes the goal of low cost and large-scale popularization and application. A light multi-layer perception (MLP) is designed for classification, which improves the computing efficiency and resource utilization of the system. BRIEF DESCRIPTION OF DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0069] Figure 1 Flowchart of the chicken disease detection method in the prior art;

[0070] Figure 2 Schematic diagram of the chicken disease detection method based on audio data enhancement and feature fusion in the embodiment of the present application;

[0071] Figure 3 Schematic diagram of feature fixed fusion in the embodiment of the present application;

[0072] Figure 4 Schematic diagram of feature sharing fusion in the embodiment of the present application;

[0073] Figure 5 Schematic diagram of feature sampling fusion in the embodiment of the present application;

[0074] Figure 6 Block diagram of the chicken disease detection system based on audio data enhancement and feature fusion in the embodiment of the present application.

[0075] Figure 7 Structure diagram of the electronic device in the embodiment of the present application. DETAILED DESCRIPTION

[0076] In order to make those skilled in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0077] Reference to an "embodiment" in this application means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. As will be apparent to those of ordinary skill in the art, embodiments described in this application can be combined with other embodiments in various ways.

[0078] The embodiment provides a chicken disease detection method based on audio data enhancement and feature fusion. By collecting and analyzing sound signals of the chicken group, whether the chicken group is diseased is identified, such as Figure 2 As shown, the method comprises the following steps two

[0079] S1, collecting sound data of the chicken group, and pre-processing the sound data.

[0080] Further, the sound data of the chicken group is collected by a microphone array installed in the chicken farm to collect the sound signals of the chicken group in real time. The microphones can efficiently capture various sounds in the chicken house and convert them into digital audio signals for subsequent processing.

[0081] Further, the pre-processing of the sound data mainly includes noise reduction, denoising, normalization and other pre-processing of the collected chicken audio signals, to remove noise and interference signals in the chicken farm environment.

[0082] In a specific embodiment, the noise reduction process specifically includes:

[0083] S11, input a random noise segment sample, and calculate the spectrum of the noise segment;

[0084] S12, subtract the characteristic frequency component corresponding to the spectrum of the noise segment from the collected chicken audio data to complete the noise reduction process.

[0085] S2, data enhancement is performed on the pre-processed sound data to obtain chicken audio training samples.

[0086] Further, more diverse chicken audio training samples are generated through data enhancement technology to improve the generalization ability of the model. The data enhancement methods include:

[0087] 1) Pitch shift up (PPS): increase the pitch of the audio segment without changing its playback speed.

[0088] 2) Pitch shift down (NPS): lower the pitch of the audio segment without changing its playback speed.

[0089] 3) Slow time stretch (STS): lengthen the duration of the audio segment.

[0090] 4) Fast time stretch (FTS): Shorten the duration of an audio segment.

[0091] 5) Silence removal (TS): Detect and remove quiet or silent parts of an audio segment.

[0092] S3, using the difference in distribution of the spectrogram and the original audio, extract the spectral features and the original audio features of the chicken audio training sample respectively.

[0093] Further, step S3 is specifically:

[0094] S31, the spectral features of the chicken audio signal include the following:

[0095] S311, Short-time Fourier transform (STFT): By performing short-time Fourier transform on the chicken audio signal, its time-frequency domain features are extracted. SIFT can provide local information of the audio signal in time and frequency, which helps to capture the dynamic change characteristics of the chicken audio signal. Its formula is:

[0096]

[0097] In the formula, τ is the integral variable, which is used to scan the value of signal x(τ) on the entire time axis. t is the current time point. (τ-t) is the window function, which depends on the difference between τ and t, and is used to intercept a part of the signal. e -jωτ is the kernel function of Fourier transform, which is used to convert the time domain signal to the frequency domain.

[0098] S312, Mel-spectrogram: By performing Mel frequency transform on the chicken audio signal, a Mel-spectrogram is generated. Its formula is:

[0099] LogMel(t,m)=log(∑ ω |X(t,ω)| 2 H m (ω)+∈)

[0100] In the formula, X(t,ω) is the result of STFT, H m (ω) is the frequency response of the mth Mel filter. ∈ is a small constant to prevent zero value when taking logarithm.

[0101] S313, Mel frequency cepstral coefficient (MFCC): By performing discrete cosine transform on the Mel-spectrogram in the above, the Mel frequency cepstral coefficient is extracted. Its formula is:

[0102]

[0103] In the formula, n is the coefficient index of MFCC, and M is the number of Mel filters.

[0104] S314, all the obtained short-time Fourier transform, mel-spectrogram and mel-frequency cepstral coefficients are re-clipped to a size of 224*224, and then spliced according to the channel dimension to a size of [3, 224, 224].

[0105] S32, the original audio features of the chicken group are as follows:

[0106] S321, audio Transformer bidirectional encoder representation (BEATs): original audio signal representation learning of the chicken group is performed by using an acoustic tokenizer. The input audio segment is first segmented into small time segments (frames), and the frames are represented as a series of vectors X1, X2...X T , and then a position encoding P is added to each input vector:

[0107] H 0 = X + P

[0108] S322, through an L-layer Transformer encoder, the calculation of each layer is as follows:

[0109] H l = LayerNorm(H l-1 + MultiHeadAttention(H l-1 , H l-1 , H l-1 ))

[0110] H l = LayerNorm(H l + FFN(H l ))

[0111] Wherein, MultiHeadAttention is a multi-head attention mechanism, and FFN is a feedforward neural network. After the L-layer encoder, the final output Z = H L is obtained.

[0112] S323, trainable audio front-end (LEAF): the processing process of the chicken audio signal is learned and optimized through a machine learning model. The formula is:

[0113]

[0114] Wherein, x(t) is the input audio signal, f is the center frequency of the trainable Gabor filter, σ is the standard deviation of the trainable Gabor filter, φ is the trainable nonlinear transformation function, N is the size of the pooling window, and * represents the convolution operation.

[0115] S4, input the extracted spectral features into multiple preset models to further mine the features, obtain multiple spectral feature maps, and splice all the spectral feature maps to obtain a spliced spectral feature map.

[0116] Further, step S4 is specifically:

[0117] The extracted chicken audio spectral feature map is input into multiple models, including VGG13, ResNet50, DenseNet, and EfficientNet, to further mine the features and obtain a spectral feature map (Spectral feature map) f 1 , f 2 , f 3 , f 4 Then, all the spectral feature maps are spliced to obtain a spliced spectral feature map f spe .

[0118] S5, input the extracted original audio features into networks combined by two front-end acoustic models and pre-training models respectively to obtain first and second semantic feature maps.

[0119] Further, step S5 is specifically:

[0120] The chicken original audio signal is input into networks combined by two front-end acoustic models and a pre-training model ResNet18 respectively to obtain a semantic feature map (Semantic feature map) (two semantic features are optional).

[0121] S6, perform feature fusion on the spliced spectral feature map, the first semantic feature map, and the second semantic feature map to obtain a fused feature map; the feature fusion includes fixed fusion, shared fusion, and sampling fusion.

[0122] Further, step S6 is specifically:

[0123] S61, use the following several fusion strategies for feature fusion;

[0124] S611, fixed fusion: connect the feature maps together, as shown in Figure 3 The formula is as follows:

[0125] h=Concat[f spe , f sem ]

[0126] Where f sem is a semantic feature map, f spe is a spectral feature map, and Concat is a connection function;

[0127] S612, Shared Fusion: Using a method similar to attention mechanism, the fusion process is trained simultaneously by conditionally weighting the features, as shown in Figure 4 . First, a weight matrix is generated for the spectrogram (using the semantic feature map) and for the semantic features (using the spectrogram feature map), denoted as W spe and W sem :

[0128] W spe = Sigmoid(Linear(f sem ))

[0129] W sem = Sigmoid(Linear(f spe ))

[0130] where Linear denotes a linear layer and Sigmoid is an activation function. Then the weighted feature maps are calculated, denoted as h spe and h sem :

[0131] h spe = f spe ⊙W sem

[0132] h sem = f sem ⊙W spe

[0133] Finally, the resulting feature maps are fused together, with the formula:

[0134] h = Concat[h spe , h sem ]

[0135] S613, Sampling Fusion: Sampling fusion learns a random function to sample from the feature maps to achieve more robust feature fusion, as shown in Figure 5 . The detailed steps and formulas are as follows:

[0136] f′ spe = f spe ⊙ReLU(Linear(f spe ))

[0137] f′ sem = f sem ⊙ReLU(Linear(f sem ))

[0138] where ReLU is an activation function and ⊙ is Hadamard product;

[0139] S62, next, the original features are sampled using a Gumbel-Softmax (GS) distribution respectively:

[0140] s spe ~p(s spe |f spe , τ1) = GS(f spe )

[0141] s sem ~p(s sem |f sem , τ2) = GS(f sem )

[0142] S63, finally, the new representation in the GS distribution domain is subjected to element multiplication to obtain the final fused feature representation:

[0143] h = Concat [f' s; f' s] spe spe sem sem

[0144] S7, input the fused feature map into a preset lightweight multilayer perceptron to output a chicken disease detection result.

[0145] Further, step S7 is specifically:

[0146] The fused features are input into the lightweight multilayer perceptron to output a binary classification result of chicken disease detection, i.e., whether the chicken population is sick or not. First, h is subjected to global average pooling (GAP) to a shape h that can be input into a linear layer gap , and then input into the multilayer perceptron, which is:

[0147]

[0148] wherein is a sigmoid function for converting the output into a probability value, indicating the probability of the input belonging to the positive class.

[0149] It should be noted that, for the foregoing method embodiments, in order to facilitate description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other orders or simultaneously.

[0150] ​​​​Based on the same idea as the chicken disease detection method based on audio data enhancement and feature fusion in the above embodiment, the application also provides a chicken disease detection system based on audio data enhancement and feature fusion, which can be used to execute the chicken disease detection method based on audio data enhancement and feature fusion. For the convenience of description, in the structural schematic diagram of the chicken disease detection system based on audio data enhancement and feature fusion, only the parts related to the embodiments of the application are shown, and those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and can include more or fewer components than the illustrated components, or combine certain components, or different component arrangements.

[0151] Please refer to Figure 6 In another embodiment of the present application, a chicken disease detection system 100 based on audio data enhancement and feature fusion is provided, which comprises a data acquisition module 101, a data enhancement module 102, a feature extraction module 103, a spectral feature processing module 104, a semantic feature processing module 105, a feature fusion module 106, and a detection module 107.

[0152] The data acquisition module 101 is configured to acquire sound data of a chicken flock and pre-process the sound data.

[0153] The data enhancement module 102 is configured to perform data enhancement on the pre-processed sound data to obtain chicken audio training samples.

[0154] The feature extraction module 103 is configured to extract spectral features and original audio features from the chicken audio training samples by utilizing the distribution difference between the spectral graph and the original audio.

[0155] The spectral feature processing module 104 is configured to input the extracted spectral features into a plurality of preset models to further mine features, obtain a plurality of spectral feature maps, and splice all the spectral feature maps to obtain a spliced spectral feature map.

[0156] The semantic feature processing module 105 is configured to input the extracted original audio features into a network combined by two front-end acoustic models and pre-training models to obtain a first semantic feature map and a second semantic feature map.

[0157] The feature fusion module 106 is configured to perform feature fusion on the spliced spectral feature map, the first semantic feature map, and the second semantic feature map to obtain a fusion feature map. The feature fusion includes fixed fusion, shared fusion, and sampling fusion.

[0158] The detection module 107 is configured to input the fusion feature map into a preset lightweight multilayer perception machine to output a chicken disease detection result.

[0159] It should be noted that the chicken disease detection system based on audio data enhancement and feature fusion of the present application corresponds to the chicken disease detection method based on audio data enhancement and feature fusion of the present application, and the technical features and advantages described in the above embodiment of the chicken disease detection method based on audio data enhancement and feature fusion are applicable to the embodiment of the chicken disease detection system based on audio data enhancement and feature fusion, and the specific content can be referred to the description in the method embodiment of the present application, which will not be described here again, and hereby declared.

[0160] In addition, in the embodiment of the chicken disease detection system based on audio data enhancement and feature fusion of the above embodiment, the logical division of each program module is only illustrative, and in actual application, the above function allocation can be completed by different program modules according to needs, for example, for the configuration requirements of corresponding hardware or the convenience of software implementation, that is, the internal structure of the chicken disease detection system based on audio data enhancement and feature fusion is divided into different program modules to complete all or part of the functions described above.

[0161] Please refer to Figure 7 In one embodiment, an electronic device implementing the chicken disease detection method based on audio data enhancement and feature fusion is provided, and the electronic device 200 can include a first processor 201, a first memory 202 and a bus, and can further include a computer program stored in the first memory 202 and executable on the first processor 201, such as a chicken disease detection program based on audio data enhancement and feature fusion 203.

[0162] The first memory 202 includes at least one type of readable storage medium, including flash memory, mobile hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The first memory 202 can be an internal storage unit of the electronic device 200 in some embodiments, such as the mobile hard disk of the electronic device 200. The first memory 202 can also be an external storage device of the electronic device 200 in other embodiments, such as the plug-in mobile hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the electronic device 200. Further, the first memory 202 can include both the internal storage unit and the external storage device of the electronic device 200. The first memory 202 can be used not only to store application software and various data installed on the electronic device 200, such as the code of the chicken disease detection program based on audio data enhancement and feature fusion 203, but also to temporarily store data that has been output or will be output.

[0163] The first processor 201 may, in some embodiments, be composed of integrated circuits, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple packaged integrated circuits of the same function or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The first processor 201 is the control unit of the electronic device, which connects various components of the entire electronic device through various interfaces and lines, and executes programs or modules stored in the first memory 202 and calls data stored in the first memory 202 to perform various functions and process data of the electronic device 200.

[0164] Figure 7 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 7 The structure shown does not constitute a limitation on the electronic device 200, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0165] The first memory 202 in the electronic device 200 stores the avian disease detection program 203 based on audio data enhancement and feature fusion, which is a combination of multiple instructions and can realize:

[0166] Collecting sound data of the chicken population and pre-processing the sound data;

[0167] Performing data enhancement on the pre-processed sound data to obtain chicken population audio training samples;

[0168] Using the distribution difference between the spectrogram and the original audio to extract the spectral features and the original audio features of the chicken population audio training samples, respectively;

[0169] Inputting the extracted spectral features into multiple preset models to further mine features, obtaining multiple spectral feature maps, and splicing all the spectral feature maps to obtain a spliced spectral feature map;

[0170] Inputting the extracted original audio features into a network respectively combined by two front-end acoustic models and pre-training models to obtain a first semantic feature map and a second semantic feature map;

[0171] Performing feature fusion on the spliced spectral feature map, the first semantic feature map, and the second semantic feature map to obtain a fused feature map; the feature fusion includes fixed fusion, shared fusion, and sampling fusion;

[0172] Inputting the fused feature map into a preset lightweight multilayer perceptron to output an avian disease detection result.

[0173] Further, the modules / units of the electronic device 200, if implemented in the form of software function units and sold or used as independent products, can be stored in a nonvolatile computer-readable storage medium. The computer-readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0174] Those of ordinary skill in the related art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. In the embodiments provided in the present application, any reference to memory, storage, database or other medium can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0175] The technical features of the above embodiments can be combined in any way. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0176] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations, simplifications made without departing from the spirit and principles of the present application should be considered as equivalent replacement methods, and are included in the protection scope of the present application.

Claims

1. A method for detecting fowl disease based on audio data enhancement and feature fusion, characterized in that, The method comprises the following steps: Collecting sound data of the chicken group and preprocessing the sound data; Performing data enhancement on the preprocessed sound data to obtain chicken group audio training samples; Extracting spectral features and original audio features from the chicken group audio training samples respectively by using the distribution difference between the spectral graph and the original audio; Inputting the extracted spectral features into multiple preset models to further mine the features, obtaining multiple spectral feature maps, and splicing all the spectral feature maps to obtain a spliced spectral feature map; Inputting the extracted original audio features into a network combined by two kinds of front-end acoustic models and pre-training models to obtain a first semantic feature map and a second semantic feature map; Performing feature fusion on the spliced spectral feature map, the first semantic feature map and the second semantic feature map to obtain a fused feature map; the feature fusion comprises fixed fusion, shared fusion and sampling fusion; Inputting the fused feature map into a preset lightweight multilayer perceptron to output a chicken disease detection result.

2. The method of claim 1, wherein the method is characterized by, The data enhancement on the preprocessed sound data comprises pitch upshift, pitch downshift, slow time stretching, fast time stretching and silence elimination. 3.The avian disease detection method based on audio data enhancement and feature fusion according to claim 1, characterized in that, The spectral feature extraction specifically comprises: Performing short-time Fourier transform on the chicken group audio signal to extract the time-frequency domain feature of the chicken group audio signal; Performing Mel frequency transform on the chicken group audio signal to generate a Mel spectrum graph; Performing discrete cosine transform on the Mel spectrum graph to extract Mel frequency cepstrum coefficients; Re-cropping the time-frequency domain feature, the Mel spectrum graph and the Mel frequency cepstrum coefficients to a set size, and then splicing them according to the channel dimension to obtain the spectral feature.

4. The method of claim 1, wherein the method further comprises: The original audio feature extraction specifically comprises: Learning the original audio signal representation of the chicken group through an acoustic segmenter; the input original audio segment is first segmented into multiple frames, and the multiple frames are represented as multiple vectors through the acoustic segmenter, and then position encoding is added to each input vector; the vector with position encoding is input into a Transformer encoder, and the original audio feature is output after coding.

5. The method of claim 1, wherein the method further comprises: The multiple preset models comprise VGG13, ResNet50, DenseNet and EfficientNet.

6. The method of claim 1, wherein the method further comprises: The feature fusion specifically comprises: Fixed fusion, connecting the spliced spectral feature map, the first semantic feature map and the second semantic feature map, specifically: h = Concat [f spe , f sem ] where f sem is a semantic feature map, f spe is a spectral feature map, and Concat is a concatenation function. Shared fusion, using a method similar to the attention mechanism to simultaneously train the fusion process by conditionally weighting the features, specifically: First, a weighting matrix is generated for the spectrogram and a weighting matrix is generated for the semantic features, respectively denoted as W spe and W sem : W spe = Sigmoid(Linear(f sem )) W sem = Sigmoid(Linear(f spe )) where Linear denotes a linear layer, and Sigmoid is an activation function; then the weighted feature map is calculated, denoted as h spe and h sem : h spe = f spe ⊙W sem h sem = f sem ⊙W spe Finally, the obtained feature maps are fused together, and the formula is: h = Concat [h spe , h sem ] Sampling fusion, specifically: Sampling from the feature map by learning a random function to achieve more robust feature fusion, and the calculation formula is as follows: f′ spe = f spe ⊙ ReLU(Linear(f spe )) f′ sem = f sem ⊙ ReLU(Linear(f sem )) Where ReLU is an activation function, and is a Hadamard product; next, the original features are sampled using a Gumbel-Soffmax distribution: s spe ~p(s spe |f spe , τ1) = GS(f spe ) s sem ~p(s sem |f sem , τ2) = GS(f sem ) Finally, the new representation of the Gumbel-Softmax distribution domain is subjected to element multiplication operation to obtain the final fused feature representation: h = Concat [f' (1), f' (2),..., f' (n) ]. spe s spe ; f' (i) = f (i). sem s sem ].

7. The method of claim 1, wherein the method further comprises: The input of the fused feature map into the preset lightweight multilayer perceptron to output the chicken disease detection result specifically comprises: The fused feature map h is first globally averaged pooled into a shape h that can be input into a linear layer gap and then input into a multi-layer perceptron, denoted as: wherein is a sigmoid function used to convert the output into a probability value representing the probability that the input belongs to the positive class.

8. A chicken disease detection system based on audio data enhancement and feature fusion, characterized in that, The method for detecting chicken disease based on audio data enhancement and feature fusion according to any one of claims 1-7 comprises a data acquisition module, a data enhancement module, a feature extraction module, a spectral feature processing module, a semantic feature processing module, a feature fusion module, and a detection module. The data acquisition module is configured to acquire sound data of a chicken group and pre-process the sound data. The data enhancement module is configured to perform data enhancement on the pre-processed sound data to obtain chicken group audio training samples. The feature extraction module is configured to extract spectral features and original audio features from the chicken group audio training samples by using the distribution difference between a spectral graph and the original audio. The spectral feature processing module is configured to input the extracted spectral features into a plurality of preset models to further mine features, obtain a plurality of spectral feature graphs, and splice all the spectral feature graphs to obtain a spliced spectral feature graph. The semantic feature processing module is configured to input the extracted original audio features into a network combined by two front-end acoustic models and pre-training models to obtain a first semantic feature graph and a second semantic feature graph. The feature fusion module is configured to perform feature fusion on the spliced spectral feature graph, the first semantic feature graph, and the second semantic feature graph to obtain a fusion feature graph, wherein the feature fusion comprises fixed fusion, shared fusion, and sampling fusion. The detection module is configured to input the fusion feature graph into a preset lightweight multi-layer perception machine to output a chicken disease detection result.

9. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the method for detecting chicken disease based on audio data enhancement and feature fusion according to any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that, The program is executed by the processor to implement the method for detecting chicken disease based on audio data enhancement and feature fusion according to any one of claims 1-7.

Images