Chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement

Abstract

The application discloses a kind of flock sound event detection methods based on multiscale perception and time domain dynamic enhancement, comprising: flock sound data is based on random screening and marking, and standard dataset with label is obtained;The sound signal in standard dataset is preprocessed, and spectrum diagram is obtained;Intercept the spectrum picture section containing flock sound in spectrum diagram, and the spectrum picture section is converted into mel spectrum diagram by mel filter bank;Depth feature extractor is constructed, and the sound depth feature of mel spectrum diagram is extracted using depth feature extractor;Frame level sound classifier is constructed, and the time domain distribution of flock sound event is obtained by processing and analyzing sound depth feature.The application can accurately and quickly extract the time domain distribution of flock sound event in long time, multi-noise breeding environment.

Description

Technical Field

[0001] The present invention belongs to the technical field of poultry health management, and in particular relates to a chicken flock sound event detection method based on multi-scale perception and time domain dynamic enhancement. Background Art

[0002] In modern agricultural farming, chicken health management and disease prevention are crucial, especially in large-scale farming environments, where the health of the flock directly impacts the production efficiency and economic benefits of the entire farm. Currently, chicken health monitoring relies primarily on manual observation and listening, with farmers assessing the health of the chickens by identifying their vocalizations, abnormal behavior, and changes in appearance.

[0003] However, this method, based on individual experience, is highly subjective, and its accuracy is limited by individual expertise and experience, making it difficult to achieve ideal detection results. Given these issues, disease detection technology based on audio analysis has garnered widespread attention in recent years. Research has shown that sick chickens emit distinctly different sounds than healthy ones. By capturing and analyzing these acoustic signals, early warning of diseases can be achieved. This method offers significant advantages, including non-invasiveness, real-time monitoring, and automated processing, significantly improving the efficiency and accuracy of disease diagnosis.

[0004] While audio-based chicken disease detection technology exhibits certain advantages, its practical application still faces challenges. For one thing, most methods rely solely on a single feature extraction technique, which has limited adaptability to complex and changing environmental sounds and is susceptible to environmental noise, resulting in reduced recognition accuracy. Furthermore, existing technologies focus solely on constructing global features when processing spectrograms, ignoring local details in the temporal domain. This leads to inaccurate localization of sound events. Finally, high hardware costs and complex system deployment processes hinder the large-scale promotion and application of this technology.

[0005] In summary, although chicken health monitoring technology based on audio detection shows great potential, further research and optimization are needed in terms of improving event classification accuracy, enhancing event location capabilities, and reducing software and hardware system costs. Summary of the Invention

[0006] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and provide a chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement. It uses a sound deep feature extractor and a frame-level sound classifier to form an end-to-end network model, which can process the sound signals of chickens in real time and automatically identify abnormal sound events.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] In a first aspect, the present invention provides a method for detecting chicken sound events based on multi-scale perception and time domain dynamic enhancement, comprising the following steps:

[0009] Randomly screening and labeling the chicken flock sound data to obtain a labeled standard data set; the labels include the chicken flock sound event category and start and end time;

[0010] Preprocess the sound signals in the standard data set to obtain a spectrogram;

[0011] Cut out the spectrum segment containing the chicken sounds in the spectrum graph, and convert the spectrum segment into a Mel spectrum graph through a Mel filter bank;

[0012] Constructing a deep feature extractor and using the deep feature extractor to extract the sound deep features of the Mel-spectrogram; the deep feature extractor includes a multi-scale perception module and a time domain dynamic enhancement module;

[0013] A frame-level sound classifier is constructed to process and analyze the deep features of the sound to obtain the time domain distribution of the chicken sound events; the frame-level sound classifier includes a bidirectional long short-term memory feedforward network.

[0014] As a preferred technical solution, the preprocessing of the sound signals in the standard data set includes:

[0015] Convert all sound signals to mono format and resample them to the set frequency;

[0016] The resampled sound signal is pre-emphasized, and the pre-emphasis processing is specifically performed as follows:

[0017] Compensate the high frequency components of the sound signal as follows:

[0018] y(n)=x(n)-αx(n-1)

[0019] a∈(0.9,1.0)

[0020] Where x(n) is the sound sample value at time n, the result after pre-emphasis processing is y(n), and a is the pre-emphasis coefficient;

[0021] The pre-emphasized sound signal is framed using a Hanning window and windowed using window shifting.

[0022] Performing a short-time discrete Fourier transform on the processed sound signal after windowing to obtain a spectrum of the sound signal;

[0023] Spectral subtraction is used to denoise the spectrogram.

[0024] As a preferred technical solution, the method of intercepting the spectrum segment containing the chicken sounds in the spectrum graph and converting the spectrum segment into a Mel spectrum graph through a Mel filter bank includes:

[0025] Calculate the short-time energy of each frame of sound signal to determine whether the frame contains chicken sounds. Based on the judgment result of the short-time energy, crop the spectrogram corresponding to the frame containing chicken sounds to obtain the spectrogram segment containing chicken sounds;

[0026] Pass the spectrogram segment through the Mel filter bank to obtain the Mel spectrogram.

[0027] As a preferred technical solution, for the vth frame sound signal y of the chicken sound signal y(n), v (n), the calculation formula of its short-time energy Ev is as follows:

[0028]

[0029] Among them, L represents the frame length, and n represents the sampling point number of the chicken sound sample;

[0030] The short-time energy Ev is judged based on the single-parameter dual-threshold endpoint detection method. When the short-time energy Ev exceeds the high threshold, it is marked as the beginning of the chicken sound frame. When the short-time energy Ev is lower than the low threshold, it is marked as the end of the chicken sound frame.

[0031] As a preferred technical solution, the multi-scale perception module includes three layers of cascade learning, each layer of cascade learning includes an overlapping block and a multi-scale perception self-attention block, and the overlapping block is embedded before the multi-scale perception self-attention block; the overlapping block of the first layer is an improved overlapping block, including a normal overlapping block, a 3×3 convolution, and a non-overlapping 2×2 convolution, the normal overlapping block includes a 7×7 convolution kernel and a convolution with a stride of 2; the overlapping blocks of other layers are normal overlapping blocks; the improved overlapping blocks are as follows:

[0032] X l,1 =Conv 2×2，2 (Conv 3×3，1 (Conv 7×7，2 (X l )))

[0033] The common overlapping blocks are as follows:

[0034] X l,i＝2，3 =Conv 7×7，2 (X l )

[0035] Where Conv 2×2，2 Indicates the use of a 2×2 convolution kernel with a step of 2; X l,iRepresents the feature map input to the i-th level;

[0036] The multi-scale perceptual self-attention block includes a multi-scale perceptual self-attention layer and an improved multi-layer perceptron, wherein the improved multi-layer perceptron includes a skip connection for supplementing cross-information between example features.

[0037] As a preferred technical solution, the method of extracting the sound depth features of the mel-spectrogram using a deep feature extractor includes:

[0038] The first step is to convert the Mel spectrum map X l Input to the multi-scale perception module, and obtain the local feature map X through overlapping blocks l,i ;

[0039] The second step is to transform the local feature map X l,i Input the multi-scale perceptual self-attention block and use the multi-scale perceptual self-attention layer to transform the local feature map X l,i Divide the feature X into h equal parts in the channel dimension i ={x1,x2,…,x h}, each equal part of the feature X i Mapped to query Q, key K and value V vectors as follows:

[0040]

[0041] where d k is the feature dimension of each attention head,

[0042] The key K and value V vectors are mapped to two different scales through convolution operations, as follows:

[0043] K′ i ,V′ i =Conv(K i ,r i ),Conv(V i ,r i )

[0044] where r i is the downsampling scale of the i-th self-attention mechanism head,

[0045] Calculate multi-head self-attention as follows:

[0046]

[0047] where d k is the feature dimension of each attention head;

[0048] The third step is to splice the calculation results of each head to obtain multi-scale information, and use the multi-layer perceptron to exchange and aggregate the multi-scale information to obtain the feature X∈RC×T×F , where C represents the number of channels of the feature, T represents the time dimension of the feature, and F represents the frequency dimension of the feature.

[0049] As a preferred technical solution, the time domain dynamic enhancement module includes a local branch and a global branch. The local branch includes multiple nonlinear one-dimensional convolution layers and ReLU activation functions to identify instantaneous information in short-term time domain changes; wherein the first one-dimensional convolution layer reduces the number of channels from C to The second one-dimensional convolutional layer includes a sigmoid activation function;

[0050] The global branch includes a depth-separable dynamic convolution for learning adaptive convolution kernels, as shown in the following formula:

[0051]

[0052] Where θ∈R C×K It is the convolution kernel aggregated from the time domain features, K is the size of the convolution kernel, and W1 and W2 are both linear layers.

[0053] As a preferred technical solution, the deep feature extractor is used to extract the sound depth features of the mel-spectrogram, specifically:

[0054] The features output by the multi-scale perception self-attention block are average-pooled along the frequency dimension. The formula is as follows:

[0055]

[0056] Where c, t, and f are the indices of channel, time domain, and frequency domain respectively. represents the frequency information of aggregation, X c,t,i represents the features output by the multi-scale perceptual self-attention block, and F represents the frequency dimension of the features;

[0057] The aggregated frequency information Input to the local branch and use the local branch to capture the instantaneous features as follows:

[0058]

[0059] Where ⊙ represents element-by-element multiplication, V is broadcast along the frequency dimension to adapt to the dimension of X, and the output dimension is Z∈R C×T×F ;

[0060] The aggregated frequency information Input to the global branch, model the context relationship through the adaptive convolution kernel, and obtain the aggregation weight θ; perform dynamic convolution operation on the aggregation weight θ and the output result Z of the local branch, as shown in the following formula:

[0061]

[0062] Among them, * represents the convolution operation.

[0063] As a preferred technical solution, the processing and analysis of the sound depth characteristics includes:

[0064] The sound depth features are input into the bidirectional long short-term memory feedforward network frame by frame for classification processing, and the label probability vector O is obtained at each moment. t And the time period corresponds to the chicken sound event label h t ;

[0065] The binary cross entropy is used to calculate the model training loss as follows:

[0066]

[0067] Among them, E is the binary cross entropy loss function, O k and P k They represent the estimated value and label value at sample index k respectively, and the total amount of data is K.

[0068] As a preferred technical solution, the sound depth features are input into the bidirectional long short-term memory feedforward network frame by frame for classification processing, specifically:

[0069] For the bidirectional long short-term memory feedforward network module at time t, forward calculation and backward calculation are performed simultaneously. For each time step t, the forward calculation from t = 1 to t = T is as follows:

[0070]

[0071] in, It is a complex computing unit inside the long short-term memory network module. is the weight matrix input to the hidden layer, is the recursive weight matrix from the previous hidden state to the current hidden state, is the bias term;

[0072] For each time step t, the backward calculation from t=T to t=1 is as follows:

[0073]

[0074] Concatenate the forward hidden state and the backward hidden state to form the comprehensive feature h at the current moment t ;

[0075] The comprehensive feature ht is input into a fully connected layer and the frame-level category probability is output. The formula is as follows:

[0076] O t=Softmax(W o h t +b o )

[0077] Where W o is the weight matrix of the classification layer, b o is the bias value of the classification layer, O t is the converted category probability.

[0078] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0079] (1) A sound deep feature extractor based on multi-scale perception and time domain dynamic enhancement is used to extract deep features of chicken sound data. Compared with the current Transformer-based sound deep feature extractor, on the one hand, the present invention combines a multi-scale feature extractor to structurally learn information of different scales and can extract more abundant and effective acoustic characteristics of chickens; on the other hand, the time domain dynamic enhancement module improves the model's ability to perceive time changes and uses instantaneous information and global information to generate position-invariant adaptive convolution kernels to dynamically aggregate time information.

[0080] (2) According to the deep features of the chicken sound data, a frame-level sound classifier based on a bidirectional long short-term memory feedforward network is used to perform frame-level classification on the deep feature map to obtain the sound event label of each frame, and finally output the time domain distribution result of the chicken sound event, which has the characteristics of high accuracy and fast speed.

[0081] (3) The present invention generates a time domain distribution diagram of chicken sound events based on the detection results of chicken sound events, which contains rich temporal evolution information of chicken sound events, can reflect the temporal evolution of the state of the chickens, and provide a reference for detecting the physiological state of the chickens.

[0082] (4) The present invention effectively enhances the unique acoustic features of the chicken sound signals by performing a series of preprocessing steps on the sound signals in the chicken sound dataset, including resampling, pre-emphasis, and short-time Fourier transform, combined with spectral subtraction denoising. Through these preprocessing steps, the acoustic signals originally in the time domain are converted into spectrograms, significantly reducing the noise interference in actual detection environments such as farms, and providing a clearer and more reliable data foundation for subsequent acoustic modeling of chicken sound events.

[0083] (5) The chicken flock sound event detection method based on multi-scale perception and time domain dynamic enhancement effectively solves the uncertainty of manually judging the health status of the chicken flock by listening to the chicken flock sounds, saves a lot of expert analysis time, reduces labor costs, and can monitor the abnormal status of the chicken flock in real time through the sound detection system. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0085] Figure 1 This is a flow chart of a chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to an embodiment of the present invention;

[0086] Figure 2 Schematic diagram of a multi-scale perception model according to an embodiment of the present invention;

[0087] Figure 3 Schematic diagram of a model of a multi-scale perception module according to an embodiment of the present invention;

[0088] Figure 4 A structural diagram of an improved overlapping block according to an embodiment of the present invention;

[0089] Figure 5 This is a schematic diagram of a multi-scale perception self-attention machine block according to an embodiment of the present invention;

[0090] Figure 6 A schematic diagram of a model for time domain dynamic enhancement according to an embodiment of the present invention;

[0091] Figure 7 Schematic diagram of an embodiment of the present invention for detecting the time domain distribution of sound events based on sound depth features based on a bidirectional long short-term memory feedforward network. DETAILED DESCRIPTION

[0092] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0093] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0094] See also Figure 1This embodiment provides a method for detecting chicken sound events based on multi-scale perception and time domain dynamic enhancement, comprising the following steps:

[0095] S1. Collect the sound data of chicken flocks in the farm, and label the chicken flock sound event categories and start and end times of a randomly selected portion of the sound data to form a labeled standard data set.

[0096] S2. Preprocessing the sound signals in the standard data set.

[0097] In this embodiment, the preprocessing of the sound signal is specifically as follows:

[0098] S21, converting all sound signals into a mono format and resampling them to 32kHz;

[0099] For general birds and poultry, a sampling rate of 16KHz is sufficient to capture key frequency features, while higher frequencies can also be better preserved. 32KHz is selected to obtain more audio features. Generally, the lowest sampling value is set to above 8KHz.

[0100] S22. Compensate the high-frequency component of the sound signal as follows:

[0101] y(n)=x(n)-αx(n-1)

[0102] a∈(0.9,1.0)

[0103] Among them, x(n) is the sound sampling value at time n, the result after pre-emphasis processing is y(n), and a is the pre-emphasis coefficient.

[0104] S23. The pre-emphasized sound signal is framed and windowed using a 1024-dimensional Hanning window and a 313-dimensional window shift. This process utilizes the short-term stationary nature of the audio signal to divide the audio signal into multiple short segments, each of which is called a frame. There is a certain overlap between adjacent frames to ensure the continuity of adjacent frames. The length of each frame is called the frame length, and the length of the non-overlapping portion between adjacent frames is called the window shift. Each frame is windowed using a Hanning window to reduce spectral leakage.

[0105] S24. Perform a short-time discrete Fourier transform on the framed and windowed sound signal to obtain a spectrum of the sound signal. The short-time discrete Fourier transform can provide local information about the audio signal in time and frequency, which helps capture the dynamic characteristics of the chicken audio signal. The formula is:

[0106]

[0107] Where τ is the integral variable used to scan the value of the signal x(τ) over the entire time axis. t is the current time point. (τ-t) is the window function that depends on the difference between τ and t and is used to intercept a portion of the signal. -jωτ It is the kernel function of Fourier transform, which is used to convert time domain signals into frequency domain.

[0108] S25. Use spectral subtraction to remove noise. This method can effectively remove additive noise and is suitable for the most common steady or slowly changing environmental noise in farms. The frequency domain distribution of the noise is quite different from the frequency domain distribution of the sounds made by chickens.

[0109] S3, extracting segments containing chicken sounds, and converting these segments into mel spectrograms through a mel filter bank; in this embodiment, S3 uses a single-parameter double-threshold endpoint detection method based on short-time energy, and crops the preprocessed spectrogram, specifically:

[0110] S31, a single parameter double threshold endpoint detection method based on short-time energy, which determines whether the frame contains the sound of chickens by calculating the short-time energy of each frame of sound signal, and extracts the segment containing the sound of chickens in the spectrum graph. v (n), the calculation formula of its short-time energy Ev is as follows:

[0111]

[0112] Where L represents the frame length, and n represents the sampling point number of the chicken sound sample. The single parameter of the single-parameter dual-threshold endpoint detection method refers to the amplitude-normalized short-time energy. The dual-threshold method sets two different thresholds as thresholds: a low threshold is used to determine whether the frame is a sound frame, and a high threshold is used to determine whether the frame contains chicken sounds. When the short-time energy exceeds the high threshold, it is marked as the beginning of the chicken sound frame; when the short-time energy falls below the low threshold, it is marked as the end of the chicken sound frame.

[0113] S32. In order to obtain a mel-spectrogram to adapt to the model input, the pre-processed spectrum segment containing the chicken sounds is passed through the mel filter bank. The conversion formula is:

[0114]

[0115] Where f represents the original frequency of the audio data, and m represents the converted Mel-frequency representation. The Mel-frequency spectrogram is sensitive to low-frequency changes in the spectrogram but insensitive to high-frequency changes, which aligns with the characteristics of the human auditory system.

[0116] S4, build Figure 2 and Figure 5 The deep feature extractor shown; Figure 2The multi-scale perception module is shown, Figure 5 The time domain dynamic enhancement module is shown, the multi-scale perception module combines multi-scale feature extraction, can learn information of different scales from structure, and can extract more rich and effective chicken acoustic characteristics; the time domain dynamic enhancement module improves the ability of the model to perceive time changes, generates a position-invariant adaptive convolution kernel using instantaneous information and global information to dynamically aggregate time information.

[0117] The specific operation is as follows:

[0118] S41, input the feature map X l to the feature extraction backbone network.

[0119] S42, as Figure 2 shown, the network includes three levels of cascaded learning, each level includes an overlapping block and a self-attention mechanism block, and the overlapping block is embedded before the self-attention module. The ordinary overlapping block embedding uses a 7x7 convolution kernel and a convolution operation with a step of 2. As Figure 4 shown, the improved overlapping block embedding not only uses a 7x7 convolution kernel and a convolution operation with a step of 2, but also adds a layer of 3x3 convolution to extract more fine-grained local details; finally, a non-overlapping 2x2 convolution operation is used for local feature fusion. In order to extract more robust local features of the spectrogram and accelerate the convergence of the model, the improved overlapping block embedding is used in the first stage, and the ordinary overlapping block embedding is used in the second and third stages. The embedding operation formula of each layer is as follows:

[0120] Improved overlapping block embedding:

[0121] X l,1 = Conv 2×2，2 (Conv 3×3，1 (Conv 7×7，2 (X l ))

[0122] Ordinary overlapping block embedding:

[0123] X l,i＝2，3 = Conv 7×7，2 (X l )

[0124] In the formula, Conv 2×2，2 represents using a 2x2 convolution kernel with a step of 2; X l,i represents the feature map input to the i-th stage.

[0125] S43, input the local feature map X l,i extracted in the overlapping block embedding to the multi-scale perception self-attention mechanism block. As Figure 3As shown in the figure, the scale-aware self-attention mechanism block is mainly composed of a multi-scale-aware self-attention mechanism layer and a multi-layer perceptron layer. The scale-aware self-attention mechanism block first divides the input feature X into h equal parts in the channel dimension X = {x1, x2, ..., x h}, then each copy is mapped to a query Q, key K, value V vector for linear mapping, calculated as follows:

[0126] Q i ,K i ,V i =W i q x i ,W i K x i ,W i V x i

[0127] Where W i Q ,W i K ,W i V is the linear mapping weight of the i-th self-attention mechanism head. Figure 4 As shown in Figure 2, the scale-aware attention mechanism maps K and V to two different scales through convolution operations, as shown in the following formula:

[0128] K′ i ,V′ i =Conv(K i ,r i ),Conv(V i ,r i )

[0129] where r i is the downsampling scale of the i-th self-attention mechanism head. Then the multi-head self-attention calculation is performed as shown below:

[0130]

[0131] where d k is the feature dimension of each attention head.

[0132] In S44, the calculation results of each head are spliced ​​together, and then the multi-layer perceptron is used to complete the exchange and aggregation of multi-scale information in the instance features, as shown in the following formula:

[0133] X′=MLP(concat(H1,H2,…,H h ))

[0134] In the multi-layer perceptron, in order to supplement the cross-information between the example features, a skip connection is added to the original multi-layer perceptron. The calculation of the multi-layer perceptron is as follows:

[0135] MLP(I)=GeLU(I+DWConv(I)W1)W2

[0136] Where I represents the input of the MLP layer, W1 and W2 are the parameters of the two linear layers, DWConv is the depth-wise separable convolution, and GeLU is the activation function.

[0137] S45, such as Figure 5 As shown, the time domain dynamic enhancement module includes: a local branch and a global branch; the execution steps of the time domain dynamic enhancement module are as follows:

[0138] S451、 Figure 5 As shown, the feature X∈R extracted from the multi-scale perception module C×T×F , where T and F represent the time and frequency dimensions of the feature, and average pooling is performed along the frequency dimension. The formula is as follows:

[0139]

[0140] Where c, t, and f are the indices of channel, time domain, and frequency domain respectively, and Indicates the frequency information of the aggregation.

[0141] S452, such as Figure 5 As shown, the aggregated frequency information The input is sent to the local branch, which is composed of a series of nonlinear one-dimensional convolutional layers combined with ReLU activation functions. Since the goal of the local branch is to capture instantaneous information and to control the complexity of the model, the first one-dimensional convolutional layer reduces the number of channels from C to Then, the second 1D convolutional layer with sigmoid activation function produces the temporal position sensitive importance weights V∈R C×T , the calculation summary of the local branch is as follows:

[0142]

[0143] Where ⊙ represents element-by-element multiplication, V is broadcast along the frequency dimension to adapt to the dimension of X, and the output dimension is Z∈R C×T×F .

[0144] S453, such as Figure 5 As shown, the aggregated frequency information The input is sent to the global branch, which uses depth-wise separable dynamic convolution. The learned adaptive convolution kernel can model contextual relationships without considering channel correlation. The learning process of the adaptive convolution kernel is as follows:

[0145]

[0146] where θ∈R C×K is a convolution kernel aggregated from temporal features, where K is the kernel size. Unlike the local branch, the global branch stacks two linear layers W1 and W2 and a ReLU activation function to learn a dynamic convolution kernel by scaling the feature map. Finally, the weights are normalized using a softmax function to produce an aggregate weight. The learned aggregate weight θ is used to perform a dynamic convolution operation on the local branch results, as shown below:

[0147]

[0148] Where * represents the convolution operation. The above calculation process can be integrated as follows:

[0149]

[0150] S5, such as Figure 6 As shown, the extracted deep feature sequence is input frame by frame into the frame-level sound classifier based on the bidirectional long short-term memory feedforward network to obtain the label probability vector of the sound at each moment and the chicken sound event label corresponding to the time period. The input sequence is X = {x1, x2, ..., x T ,}, where x t Represents the input feature vector at time t.

[0151] S51. For the bidirectional long short-term memory feedforward network module at time t, forward calculation and backward calculation are performed simultaneously. For each time step t, the forward calculation is as follows:

[0152]

[0153] in It is a complex computing unit inside the long short-term memory network module, including a combination of input gate, forget gate, and output gate, usually acted upon by an activation function. is the weight matrix input to the hidden layer, is the recursive weight matrix from the previous hidden state to the current hidden state, is the bias term. Repeat the above calculation until t = T, completing the forward calculation of the entire sequence. For each time step t, the backward calculation is as follows:

[0154]

[0155] The meaning of the parameters is the same as the forward calculation, but the direction is opposite, from t=T to t=1.

[0156] S52. At each time step t, concatenate the forward hidden state and the backward hidden state to form the comprehensive feature h at the current moment t , the formula is as follows:

[0157]

[0158] Here, the concatenation [;] operation is used to generate a feature vector containing bidirectional information.

[0159] S53, such as Figure 7 As shown, the comprehensive hidden state ht is input into a fully connected layer and the frame-level category probability is output. The formula is as follows:

[0160] O t =Softmax(W o h t +b o )

[0161] Where W o is the weight matrix of the classification layer, b o is the bias value of the classification layer, O t is the converted category probability.

[0162] S6. The sound depth feature extractor is connected to the frame-level sound classifier to form an end-to-end network model. The end-to-end network model training process uses binary cross entropy as the loss function, which is expressed as:

[0163]

[0164] Among them, E is the binary cross entropy loss function, O k and P k They represent the estimated value and label value at sample index k respectively, and the total amount of data is K.

[0165] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0166] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement, characterized in that: The steps include: Randomly screening and labeling the chicken flock sound data to obtain a labeled standard data set; the labels include the chicken flock sound event category and start and end time; Preprocess the sound signals in the standard data set to obtain a spectrogram; Cut out the spectrum segment containing the chicken sounds in the spectrum graph, and convert the spectrum segment into a Mel spectrum graph through a Mel filter bank; A deep feature extractor is constructed and used to extract the deep sound features of the Mel-spectrogram. The deep feature extractor includes a multi-scale perception module and a time domain dynamic enhancement module. The multi-scale perception module includes three layers of cascade learning, each layer of cascade learning includes an overlapping block and a multi-scale perception self-attention block, and the overlapping block is embedded before the multi-scale perception self-attention block. The overlapping blocks of the first layer are improved overlapping blocks, including ordinary overlapping blocks, 3×3 convolutions, and non-overlapping 2×2 convolutions. The ordinary overlapping blocks include a 7×7 convolution kernel and a convolution with a stride of 2. The overlapping blocks of other layers are ordinary overlapping blocks. The improved overlapping blocks are as follows: The common overlapping blocks are as follows: In the formula Indicates the use of a 2×2 convolution kernel with a stride of 2; Indicates input to Level feature map; The multi-scale perception self-attention block includes a multi-scale perception self-attention layer and an improved multi-layer perceptron, wherein the improved multi-layer perceptron includes a skip connection for supplementing the cross information between the example features; the time domain dynamic enhancement module includes a local branch and a global branch, wherein the local branch includes multiple nonlinear one-dimensional convolution layers and Activation function, used to identify instantaneous information in short-term time domain changes; the first one-dimensional convolution layer changes the number of channels from C Reduce to , the second one-dimensional convolutional layer includes a sigmoid activation function; The global branch includes a depth-separable dynamic convolution for learning adaptive convolution kernels, as shown in the following formula: in, is the convolution kernel aggregated from the time domain features, 𝐾 is the size of the convolution kernel, and W1 and W2 are both linear layers; A frame-level sound classifier is constructed to process and analyze the deep features of the sound to obtain the time domain distribution of the chicken sound events; the frame-level sound classifier includes a bidirectional long short-term memory feedforward network.

2. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 1 is characterized in that: The preprocessing of the sound signals in the standard data set includes: Convert all sound signals to mono format and resample them to the set frequency; The resampled sound signal is pre-emphasized, and the pre-emphasis processing is specifically performed as follows: Compensate the high frequency components of the sound signal as follows: in, for n The sound sampling value at the moment, after pre-emphasis processing, is , a is the pre-emphasis coefficient; The pre-emphasized sound signal is framed using a Hanning window and windowed using window shifting. Performing a short-time discrete Fourier transform on the processed sound signal after windowing to obtain a spectrum of the sound signal; Spectral subtraction is used to denoise the spectrogram.

3. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 1 is characterized in that: The intercepting of the spectrum segment containing the chicken sounds in the spectrum segment and converting the spectrum segment into a Mel spectrum segment through a Mel filter bank comprises: Calculate the short-time energy of each frame of sound signal to determine whether the frame contains chicken sounds. Based on the judgment result of the short-time energy, crop the spectrogram corresponding to the frame containing chicken sounds to obtain the spectrogram segment containing chicken sounds; Pass the spectrogram segment through the Mel filter bank to obtain the Mel spectrogram.

4. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 3 is characterized in that: Sound signals for chickens No. Frame sound signal , its short-term energy The calculation formula is as follows: in, L Indicates the frame length, n Represents the sampling point number of the chicken sound sample; Short-time energy detection based on single-parameter double-threshold endpoint detection method Make a judgment, when short-term energy When the high threshold is exceeded, it is marked as the beginning of the chicken sound frame. When it falls below the lower threshold, it marks the end of the chicken sound frame.

5. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 1 is characterized in that: The method of extracting the sound depth features of the mel-spectrogram using a deep feature extractor includes: The first step is to convert the Mel spectrum X l Input into the multi-scale perception module, and obtain the local feature map through overlapping blocks ; The second step is to convert the local feature map Input the multi-scale perceptual self-attention block and use the multi-scale perceptual self-attention layer to convert the local feature map Divided into channel dimensions h Equal-portion feature , each equal feature X i Mapping to query Q ,key K Sum V Vector, as follows: in is the feature dimension of each attention head, Press the key K Sum V The vector is mapped to two different scales through the convolution operation, as follows: in It is i The downsampling scale of the self-attention mechanism head, Calculate multi-head self-attention as follows: in is the feature dimension of each attention head; The third step is to splice the calculation results of each head to obtain multi-scale information, and use the multi-layer perceptron to exchange and aggregate the multi-scale information to obtain features. , where C represents the number of channels of the feature, T Represents the time dimension of the feature, F Represents the frequency dimension of the feature.

6. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 1 is characterized in that: The method of extracting the sound depth features of the Mel-spectrogram using a deep feature extractor is as follows: The features output by the multi-scale perception self-attention block are average-pooled along the frequency dimension. The formula is as follows: in c 、 t and f are the indexes of channel, time domain and frequency domain respectively, Represents the frequency information of the aggregation, Represents the features output by the multi-scale perceptual self-attention block, F The frequency dimension representing the feature; The aggregated frequency information Input to the local branch and use the local branch to capture the instantaneous features as follows: Where ⊙ represents element-wise multiplication, V Broadcast along the frequency dimension to accommodate X The dimension of the output is ; The aggregated frequency information Input to the global branch, model the context relationship through the adaptive convolution kernel, and obtain the aggregation weight 𝜃; the aggregation weight 𝜃 and the output result of the local branch are combined Z Perform dynamic convolution operation as follows: Among them, ∗ represents the convolution operation.

7. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 1 is characterized in that: The processing and analysis of the sound depth feature includes: Input the sound depth features frame by frame into the bidirectional long short-term memory feedforward network for classification processing to obtain the label probability vector at each moment And the chicken sound event label corresponding to the time period ; The binary cross entropy is used to calculate the model training loss as follows: Where E is the binary cross entropy loss function, and Represents the sample index The estimated value and label value at , the total amount of data is K .

8. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 7 is characterized in that: The sound depth features are input into the bidirectional long short-term memory feedforward network frame by frame for classification processing, specifically: for The bidirectional long short-term memory feedforward network module performs forward and backward calculations at the same time. For each time step ,from t =1 to t =T forward calculation is as follows: in, It is a complex computing unit inside the long short-term memory network module. is the weight matrix input to the hidden layer, is the recursive weight matrix from the previous hidden state to the current hidden state, is the bias term; For each time step ,from t =T to t =1 The backward calculation is as follows: Concatenate the forward hidden state and the backward hidden state to form the comprehensive features of the current moment ; The comprehensive features Input to a fully connected layer and output frame-level category probability, the formula is as follows: in is the weight matrix of the classification layer, is the bias value of the classification layer, is the converted category probability.

Images