Chicken flock sound event detection method based on multi-scale perception and time domain dynamic enhancement

Abstract

The invention discloses a chicken flock sound event detection method based on multi-scale perception and time domain dynamic enhancement, and the method comprises the steps: carrying out the random screening and marking based on chicken flock sound data, and obtaining a standard data set with a label; preprocessing the sound signals in the standard data set to obtain a spectrogram; intercepting a spectrum picture segment containing chicken flock sound in the spectrogram, and converting the spectrum picture segment into a Mel spectrogram through a Mel filter bank; constructing a depth feature extractor, and extracting sound depth features of the Mel spectrogram by using the depth feature extractor; and constructing a frame level sound classifier, processing and analyzing sound depth features, and obtaining time domain distribution of chicken flock sound events. According to the method, the time domain distribution of the sound events of the chicken flocks can be accurately and quickly extracted in a long-time and noisy breeding environment.

Description

Technical Field

[0001] The invention belongs to the technical field of poultry health management, and in particular relates to a chicken flock sound event detection method based on multi-scale perception and time domain dynamic enhancement. Background Art

[0002] In modern agricultural breeding, chicken health management and epidemic prevention are crucial, especially in large-scale breeding environments, where the health of the chicken group directly affects the production efficiency and economic benefits of the entire farm. At present, chicken health monitoring mainly relies on manual observation and listening. Breeders judge the health of chickens by identifying their calls, abnormal behaviors, and changes in appearance.

[0003] However, this method based on individual experience is highly subjective, and its accuracy is limited by the individual's professional level and experience accumulation, making it difficult to achieve ideal detection results. In view of the above problems, disease detection technology based on audio analysis has received widespread attention in recent years. Studies have shown that chickens make sounds that are significantly different from those in a healthy state when they are sick. By capturing and analyzing these sound signals, early warning of diseases can be achieved. This method has significant advantages such as non-invasiveness, real-time monitoring and automated processing, which can significantly improve the efficiency and accuracy of disease diagnosis.

[0004] Although audio-based chicken disease detection technology has shown certain advantages, it still faces challenges in practical applications. On the one hand, most methods rely only on a single feature extraction technology, which has limited adaptability to complex and changeable environmental sounds and is easily affected by environmental noise, resulting in reduced recognition accuracy. On the other hand, when processing spectrograms, existing technologies focus only on global feature construction and ignore local details in the time domain, resulting in inaccurate positioning of the time when sound events occur. Finally, high hardware costs and complex system deployment processes also hinder the large-scale promotion and application of this technology.

[0005] In summary, although the chicken health monitoring technology based on audio detection shows great potential, it needs further research and optimization in improving the accuracy of event classification, enhancing event location capabilities, and reducing the cost of software and hardware systems. Summary of the invention

[0006] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and provide a chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement. A sound deep feature extractor and a frame-level sound classifier are used to form an end-to-end network model, which can process the sound signals of chickens in real time and automatically identify abnormal sound events.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] In a first aspect, the present invention provides a method for detecting chicken sound events based on multi-scale perception and time domain dynamic enhancement, comprising the following steps:

[0009] Randomly screen and label the chicken sound data to obtain a standard data set with labels; the labels include the chicken sound event category and start and end time;

[0010] Preprocess the sound signal in the standard data set to obtain a spectrogram;

[0011] The spectrum image segment containing the chicken sounds in the spectrum image is intercepted, and the spectrum image segment is converted into a Mel spectrum image through a Mel filter bank;

[0012] Constructing a deep feature extractor, and using the deep feature extractor to extract the sound deep features of the Mel-spectrogram; the deep feature extractor includes a multi-scale perception module and a time domain dynamic enhancement module;

[0013] A frame-level sound classifier is constructed to process and analyze the deep features of the sound to obtain the time domain distribution of the sound events of the chicken flock; the frame-level sound classifier includes a bidirectional long short-term memory feedforward network.

[0014] As a preferred technical solution, the preprocessing of the sound signal in the standard data set includes:

[0015] Convert all sound signals to mono format and resample to the set frequency;

[0016] The resampled sound signal is pre-emphasized, and the pre-emphasis processing is performed on the resampled sound signal, specifically:

[0017] Compensate the high frequency component of the sound signal as follows:

[0018] y(n)=x(n)-αx(n-1)

[0019] a∈(0.9,1.0)

[0020] Where x(n) is the sound sampling value at time n, the result after pre-emphasis processing is y(n), and a is the pre-emphasis coefficient;

[0021] The pre-emphasized sound signal is framed using the Hanning window and windowed using window shifting.

[0022] Performing short-time discrete Fourier transform on the processed sound signal after windowing the sound signal to obtain a spectrum diagram of the sound signal;

[0023] Spectral subtraction is used to denoise the spectrogram.

[0024] As a preferred technical solution, the method of intercepting the spectrum segment containing the chicken sounds in the spectrum graph and converting the spectrum segment into a Mel spectrum graph through a Mel filter bank includes:

[0025] Calculate the short-time energy of each frame of sound signal to determine whether the frame contains the sound of chickens, and according to the judgment result of the short-time energy, cut the spectrum graph corresponding to the frame containing the sound of chickens to obtain the spectrum graph segment containing the sound of chickens;

[0026] Pass the spectrum graph segment through the Mel filter bank to obtain the Mel spectrum graph.

[0027] As a preferred technical solution, for the vth frame sound signal y of the chicken sound signal y(n), v (n), the calculation formula of its short-time energy Ev is as follows:

[0028]

[0029] Among them, L represents the frame length, and n represents the sampling point number of the chicken sound sample;

[0030] The short-time energy Ev is judged based on the single-parameter dual-threshold endpoint detection method. When the short-time energy Ev exceeds the high threshold, it is marked as the beginning of the chicken sound frame. When the short-time energy Ev is lower than the low threshold, it is marked as the end of the chicken sound frame.

[0031] As a preferred technical solution, the multi-scale perception module includes three layers of cascade learning, each layer of cascade learning includes an overlapping block and a multi-scale perception self-attention block, and the overlapping block is embedded before the multi-scale perception self-attention block; the overlapping block of the first layer is an improved overlapping block, including an ordinary overlapping block, a 3×3 convolution, and a non-overlapping 2×2 convolution, and the ordinary overlapping block includes a 7×7 convolution kernel and a convolution with a step of 2; the overlapping blocks of other layers are ordinary overlapping blocks; the improved overlapping blocks are as follows:

[0032] X l,1 =Conv 2×2，2 (Conv 3×3，1 (Conv 7×7，2 (X l )))

[0033] The common overlapping blocks are as follows:

[0034] X l,i＝2，3 =Conv 7×7，2 (X l )

[0035] Where Conv 2×2，2 Indicates the use of a 2×2 convolution kernel with a step of 2; X l,iRepresents the feature map input to the i-th level;

[0036] The multi-scale perceptual self-attention block includes a multi-scale perceptual self-attention layer and an improved multi-layer perceptron, wherein the improved multi-layer perceptron includes a skip connection for supplementing cross-information between example features.

[0037] As a preferred technical solution, the method of extracting the sound depth features of the Mel-spectrogram using a deep feature extractor includes:

[0038] The first step is to convert the Mel spectrum map X l Input to the multi-scale perception module, and obtain the local feature map X through overlapping blocks l,i ;

[0039] The second step is to transform the local feature map X l,i Input the multi-scale perceptual self-attention block, and use the multi-scale perceptual self-attention layer to transform the local feature map X l,i Divide the feature X into h equal parts in the channel dimension i ={x1,x2,…,x h}, each equal part of feature X i Mapped to query Q, key K and value V vectors as follows:

[0040]

[0041] where d k is the feature dimension of each attention head,

[0042] The key K and value V vectors are mapped to two different scales through convolution operations, as shown below:

[0043] K′ i ,V′ i =Conv(K i ,r i ),Conv(V i ,r i )

[0044] where r i is the downsampling scale of the ith self-attention mechanism head,

[0045] Calculate multi-head self-attention as follows:

[0046]

[0047] where d k is the feature dimension of each attention head;

[0048] The third step is to concatenate the calculation results of each head to obtain multi-scale information, and use the multi-layer perceptron to exchange and aggregate the multi-scale information to obtain the feature X∈RC×T×F , where C represents the number of channels of the feature, T represents the time dimension of the feature, and F represents the frequency dimension of the feature.

[0049] As a preferred technical solution, the time domain dynamic enhancement module includes a local branch and a global branch. The local branch includes multiple nonlinear one-dimensional convolutional layers and ReLU activation functions for identifying instantaneous information in short-term time domain changes. The first one-dimensional convolutional layer reduces the number of channels from C to The second one-dimensional convolutional layer includes a sigmoid activation function;

[0050] The global branch includes a depth-separable dynamic convolution for learning an adaptive convolution kernel, as shown in the following formula:

[0051]

[0052] Among them, θ∈R C×K It is the convolution kernel aggregated from the time domain features, K is the size of the convolution kernel, and W1 and W2 are both linear layers.

[0053] As a preferred technical solution, the deep feature extractor is used to extract the sound deep features of the Mel-spectrogram, specifically:

[0054] The features output by the multi-scale perceptual self-attention block are averaged and pooled along the frequency dimension. The formula is as follows:

[0055]

[0056] Where c, t and f are the indexes of channel, time domain and frequency domain respectively. represents the frequency information of the aggregation, X c,t,i represents the features output by the multi-scale perceptual self-attention block, and F represents the frequency dimension of the features;

[0057] The aggregated frequency information Input to the local branch and use the local branch to capture instantaneous features as follows:

[0058]

[0059] Where ⊙ represents element-by-element multiplication, V is broadcast along the frequency dimension to adapt to the dimension of X, and the output dimension is Z∈R C×T×F ;

[0060] The aggregated frequency information Input to the global branch, model the contextual relationship through the adaptive convolution kernel, and obtain the aggregation weight θ; perform dynamic convolution operation on the aggregation weight θ and the output result Z of the local branch, as shown in the following formula:

[0061]

[0062] Among them, * represents the convolution operation.

[0063] As a preferred technical solution, the processing and analysis of the sound depth feature includes:

[0064] The sound depth features are input frame by frame into the bidirectional long short-term memory feedforward network for classification processing to obtain the label probability vector O at each moment t And the time period corresponds to the chicken sound event label h t ;

[0065] The binary cross entropy is used to calculate the model training loss as follows:

[0066]

[0067] Among them, E is the binary cross entropy loss function, O k and P k They represent the estimated value and label value at sample index k respectively, and the total amount of data is K.

[0068] As a preferred technical solution, the sound depth features are input into a bidirectional long short-term memory feedforward network frame by frame for classification processing, specifically:

[0069] For the bidirectional long short-term memory feedforward network module at time t, forward calculation and backward calculation are performed simultaneously. For each time step t, the forward calculation from t = 1 to t = T is as follows:

[0070]

[0071] in, It is a complex computing unit inside the long short-term memory network module. is the weight matrix input to the hidden layer, is the recursive weight matrix from the previous hidden state to the current hidden state, is the bias term;

[0072] For each time step t, the backward calculation from t = T to t = 1 is as follows:

[0073]

[0074] Concatenate the forward hidden state and the backward hidden state to form the comprehensive feature h at the current moment t ;

[0075] The comprehensive feature ht is input into a fully connected layer and the frame-level category probability is output. The formula is as follows:

[0076] O t=Softmax(W o h t +b o )

[0077] Where W o is the weight matrix of the classification layer, b o is the bias value of the classification layer, O t is the converted category probability.

[0078] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0079] (1) A sound deep feature extractor based on multi-scale perception and time domain dynamic enhancement is used to extract deep features of chicken sound data. Compared with the current Transformer-based sound deep feature extractor, on the one hand, the present invention combines a multi-scale feature extractor to structurally learn information of different scales and can extract more abundant and effective acoustic characteristics of chickens; on the other hand, the time domain dynamic enhancement module improves the model's ability to perceive time changes and uses instantaneous information and global information to generate a position-invariant adaptive convolution kernel to dynamically aggregate time information.

[0080] (2) According to the deep features of the chicken sound data, a frame-level sound classifier based on a bidirectional long short-term memory feedforward network is used to perform frame-level classification on the deep feature map to obtain the sound event label of each frame, and finally output the time domain distribution result of the chicken sound event, which has the characteristics of high accuracy and fast speed.

[0081] (3) The present invention generates a time domain distribution diagram of chicken sound events based on the detection results of chicken sound events, which contains rich temporal evolution information of chicken sound events, can reflect the temporal evolution of the state of the chickens, and provide a reference for detecting the physiological state of the chickens.

[0082] (4) The present invention performs a series of preprocessing on the sound signals in the chicken sound data set, including resampling, pre-emphasis, short-time Fourier transform and other steps, and combines spectral subtraction denoising to effectively enhance the unique acoustic features in the chicken sound signals. Through these preprocessing, the acoustic signals originally in the time domain are converted into spectrograms, which greatly reduces the noise interference in actual detection environments such as farms, and provides a clearer and more reliable data basis for the subsequent acoustic modeling of chicken sound events.

[0083] (5) The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement effectively solves the uncertainty of manually judging the health status of the chicken flock by listening to the chicken flock sounds, saves a lot of expert analysis time, and reduces labor costs. The sound detection system can be used to monitor the abnormal status of the chicken flock in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0085] Figure 1 It is a flow chart of a method for detecting chicken sound events based on multi-scale perception and time domain dynamic enhancement according to an embodiment of the present invention;

[0086] Figure 2 A schematic diagram of a multi-scale perception model according to an embodiment of the present invention;

[0087] Figure 3 Schematic diagram of a model of a multi-scale perception module according to an embodiment of the present invention;

[0088] Figure 4 A structural diagram of an improved overlapping block according to an embodiment of the present invention;

[0089] Figure 5 This is a schematic diagram of a multi-scale perception self-attention machine block according to an embodiment of the present invention;

[0090] Figure 6 A schematic diagram of a model for time domain dynamic enhancement according to an embodiment of the present invention;

[0091] Figure 7 It is a schematic diagram of detecting the time domain distribution of sound events based on sound depth features based on a bidirectional long short-term memory feedforward network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0092] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0093] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0094] See also Figure 1This embodiment provides a method for detecting chicken sound events based on multi-scale perception and time domain dynamic enhancement, comprising the following steps:

[0095] S1. Collect the sound data of chicken flocks in the farm, and label the chicken flock sound event categories and start and end times of a randomly selected part of the chicken flock sound data to form a labeled standard data set.

[0096] S2. Preprocessing the sound signals in the standard data set.

[0097] In this embodiment, the preprocessing of the sound signal is specifically as follows:

[0098] S21, converting all sound signals into a mono format and resampling them to 32kHz;

[0099] For general birds and poultry, a sampling rate of 16KHz is sufficient to capture key frequency features, while higher frequencies can also be better preserved. 32KHz is selected to obtain more audio features. Generally, the lowest sampling value is set to above 8KHz.

[0100] S22, compensating the high frequency component of the sound signal, as shown in the following formula:

[0101] y(n)=x(n)-αx(n-1)

[0102] a∈(0.9,1.0)

[0103] Among them, x(n) is the sound sampling value at time n, the result after pre-emphasis processing is y(n), and a is the pre-emphasis coefficient.

[0104] S23, using a 1024-dimensional Hanning window and a 313-dimensional window shift to divide the pre-emphasized sound signal into frames and add windows; this process utilizes the short-term stability of the audio signal to divide the audio signal into multiple short segments, each of which is called a frame. There is a certain overlap between adjacent frames to ensure the continuity of adjacent frames; the length of each frame is called the frame length, and the length of the non-overlapping part between adjacent frames is called the window shift; a Hanning window is used to add windows to each frame to reduce spectrum leakage.

[0105] S24, perform short-time discrete Fourier transform on the framed and windowed sound signal to obtain a spectrum of the sound signal; short-time Fourier transform can provide local information of the audio signal in time and frequency, which helps to capture the dynamic change characteristics of the audio signal of the chicken flock. The formula is:

[0106]

[0107] In the formula, τ is the integral variable used to scan the value of the signal x(τ) on the entire time axis. t is the current time point. (τ-t) is the window function, which depends on the difference between τ and t and is used to intercept a part of the signal. -jωτ It is the kernel function of Fourier transform, which is used to convert time domain signals into frequency domain.

[0108] S25. Use spectral subtraction to remove noise. This method can effectively remove additive noise and is suitable for the most common steady or slowly changing environmental noise in farms. The frequency domain distribution of the noise is quite different from the frequency domain distribution of the sound emitted by the chickens.

[0109] S3, extracting segments containing chicken sounds, and converting these segments into Mel spectrograms through a Mel filter bank; in this embodiment, S3 uses a single-parameter double-threshold endpoint detection method based on short-time energy, and cuts the preprocessed spectrogram, specifically:

[0110] S31, a single parameter double threshold endpoint detection method based on short-time energy, by calculating the short-time energy of each frame of sound signal to determine whether the frame contains the sound of the chicken flock, and extracting the segment containing the sound of the chicken flock in the spectrum graph. v (n), the calculation formula of its short-time energy Ev is as follows:

[0111]

[0112] In the formula, L represents the frame length, and n represents the sampling point number of the chicken sound sample; the single parameter of the single-parameter dual-threshold endpoint detection method refers to the short-time energy after amplitude normalization; the dual threshold refers to setting two different thresholds as thresholds, where the low threshold is used to determine whether the frame is a sound frame, and the high threshold is used to determine whether the frame contains chicken sounds. When the short-time energy exceeds the high threshold, it is marked as the start of the chicken sound frame; when the short-time energy is lower than the low threshold, it is marked as the end of the chicken sound frame.

[0113] S32. In order to obtain a mel spectrum graph to adapt to the model input, the preprocessed spectrum graph segment containing the chicken sounds is passed through a mel filter bank, and the conversion formula is:

[0114]

[0115] In the formula, f represents the original frequency of the audio data, and m represents the Mel frequency obtained after conversion. The Mel spectrum can sensitively capture the changes in the low-frequency band of the spectrum, but is insensitive to the changes in the high-frequency band, which is in line with the characteristics of the human auditory system.

[0116] S4. Construct Figure 2 and Figure 5 The deep feature extractor shown; Figure 2The multi-scale perception module is shown. Figure 5 The figure shows a time domain dynamic enhancement module. The multi-scale perception module is combined with multi-scale feature extraction, which can structurally learn information of different scales and extract more abundant and effective acoustic characteristics of chicken flocks. The time domain dynamic enhancement module improves the model's ability to perceive time changes, and uses instantaneous information and global information to generate position-invariant adaptive convolution kernels to dynamically aggregate time information.

[0117] The specific operations are as follows:

[0118] S41, feature map X l Input to the feature extraction backbone network.

[0119] S42, such as Figure 2 As shown in , the network consists of three layers of cascade learning. Each layer contains an overlapping block and a self-attention mechanism block. The overlapping block is embedded before the self-attention module. The ordinary overlapping block embedding uses a 7×7 convolution kernel and a convolution operation with a stride of 2. Figure 4 As shown in the figure, the improved overlapping block embedding not only uses a 7×7 convolution kernel and a convolution operation with a stride of 2, but also adds a layer of 3×3 convolution to extract more fine-grained local details; finally, a non-overlapping 2×2 convolution operation is used for local feature fusion. In order to extract more robust local features of the spectrogram and accelerate the convergence of the model, the improved overlapping block embedding is used in the first level, and the ordinary overlapping block embedding is used in the second and third levels. The embedding operation formulas of each layer are as follows:

[0120] Improved overlapping block embedding:

[0121] X l,1 =Conv 2×2，2 (Conv 3×3，1 (Conv 7×7，2 (X l )))

[0122] Normal overlapping block embedding:

[0123] X l,i＝2，3 =Conv 7×7，2 (X l )

[0124] Where Conv 2×2，2 Indicates the use of a 2×2 convolution kernel with a step of 2; X l,i Represents the feature map input to the i-th level.

[0125] S43, embed the overlapping blocks into the extracted local feature map X l,i Input to the multi-scale perceptual self-attention mechanism block. Figure 3As shown in Figure 1, the scale-aware self-attention mechanism block is mainly composed of a multi-scale-aware self-attention mechanism layer and a multi-layer perceptron layer. The scale-aware self-attention mechanism block first divides the input feature X into h equal parts in the channel dimension X = {x1, x2, …, x h}, then each copy is mapped to a query Q, key K, value V vector for linear mapping, calculated as follows:

[0126] Q i ,K i ,V i =W i q x i ,W i K x i ,W i V x i

[0127] Where W i Q ,W i K ,W i V is the linear mapping weight of the i-th self-attention mechanism head. Figure 4 As shown in the figure, the scale-aware attention mechanism maps K and V to two different scales through convolution operations, as shown in the following formula:

[0128] K′ i ,V′ i =Conv(K i ,r i ),Conv(V i ,r i )

[0129] where r i is the downsampling scale of the ith self-attention mechanism head. Then the multi-head self-attention calculation is performed as shown in the following formula:

[0130]

[0131] where d k is the feature dimension of each attention head.

[0132] S44, the calculation results of each head are spliced ​​together, and then the multi-layer perceptron is used to complete the exchange and aggregation of multi-scale information in the instance features, as shown in the following formula:

[0133] X′=MLP(concat(H1,H2,…,H h ))

[0134] In the multi-layer perceptron, in order to supplement the cross-information between the example features, a skip connection is added to the original multi-layer perceptron. The calculation of the multi-layer perceptron is as follows:

[0135] MLP(I)=GeLU(I+DWConv(I)W1)W2

[0136] Where I represents the input of the MLP layer, W1 and W2 are the parameters of the two linear layers, DWConv is the depthwise separable convolution, and GeLU is the activation function.

[0137] S45, such as Figure 5 As shown, the time domain dynamic enhancement module includes: a local branch and a global branch; the execution steps of the time domain dynamic enhancement module are as follows:

[0138] S451, such as Figure 5 As shown, the feature X∈R extracted from the multi-scale perception module C×T×F , where T and F represent the time and frequency dimensions of the feature, and average pooling is performed along the frequency dimension. The formula is as follows:

[0139]

[0140] Where c, t, and f are the indices of the channel, time domain, and frequency domain, respectively, and Represents the frequency information of the aggregation.

[0141] S452, such as Figure 5 As shown, the aggregated frequency information The input is sent to the local branch, which consists of a series of nonlinear one-dimensional convolutional layers combined with ReLU activation functions. Since the goal of the local branch is to capture instantaneous information and to control the complexity of the model, the first one-dimensional convolutional layer reduces the number of channels from C to Then, the second 1D convolutional layer with sigmoid activation function produces the temporal position-sensitive importance weights V∈R C×T , the calculation summary of the local branch is as follows:

[0142]

[0143] Where ⊙ represents element-by-element multiplication, V is broadcast along the frequency dimension to adapt to the dimension of X, and the output dimension is Z∈R C×T×F .

[0144] S453, such as Figure 5 As shown, the aggregated frequency information The input is sent to the global branch, which uses a depth-separable dynamic convolution. The learned adaptive convolution kernel can model contextual relationships without considering channel correlation. The learning process of the adaptive convolution kernel is as follows:

[0145]

[0146] where θ∈R C×K is a convolution kernel aggregated from time-domain features, where K is the size of the convolution kernel. Unlike the local branch, the global branch stacks two linear layers W1, W2 and ReLU activation functions to learn dynamic convolution kernels by scaling feature maps. Finally, the weights are normalized using the softmax function to produce aggregate weights. The learned aggregate weight θ is used for the convolution kernel that performs dynamic convolution operations on the local branch results, as shown below:

[0147]

[0148] Where * represents the convolution operation. The above calculation process can be integrated as follows:

[0149]

[0150] S5, such as Figure 6 As shown in the figure, the extracted deep feature sequence is input frame by frame into the frame-level sound classifier based on the bidirectional long short-term memory feedforward network to obtain the label probability vector of the sound at each moment and the chicken sound event label corresponding to the time period. The input sequence is X = {x1, x2, ..., x T ,}, where x t Represents the input feature vector at time t.

[0151] S51, for the bidirectional long short-term memory feedforward network module at time t, forward calculation and backward calculation are performed simultaneously. For each time step t, the forward calculation is as follows:

[0152]

[0153] in It is a complex computing unit inside the long short-term memory network module, including a combination of input gate, forget gate, and output gate, usually acted upon by an activation function. is the weight matrix input to the hidden layer, is the recursive weight matrix from the previous hidden state to the current hidden state, is the bias term. Repeat the above calculation until t = T, completing the forward calculation of the entire sequence. For each time step t, the backward calculation is as follows:

[0154]

[0155] The meaning of the parameters is the same as the forward calculation, but the direction is opposite, from t=T to t=1.

[0156] S52, at each time step t, concatenate the forward hidden state and the backward hidden state to form the comprehensive feature h at the current moment t , the formula is as follows:

[0157]

[0158] Here, the concatenation [;] operation is used to generate a feature vector containing bidirectional information.

[0159] S53, such as Figure 7 As shown, the comprehensive hidden state ht is input into a fully connected layer, and the frame-level category probability is output. The formula is as follows:

[0160] O t =Softmax(W o h t +b o )

[0161] Where W o is the weight matrix of the classification layer, b o is the bias value of the classification layer, O t is the converted category probability.

[0162] S6. The sound depth feature extractor is connected to the frame-level sound classifier to form an end-to-end network model. The end-to-end network model training process uses binary cross entropy as the loss function, which is expressed as:

[0163]

[0164] Among them, E is the binary cross entropy loss function, O k and P k They represent the estimated value and label value at sample index k respectively, and the total amount of data is K.

[0165] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0166] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0167] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A method for detecting chicken sound events based on multi-scale perception and time domain dynamic enhancement, characterized in that: The steps include: Randomly screen and label the chicken sound data to obtain a standard data set with labels; the labels include the chicken sound event category and start and end time; Preprocess the sound signal in the standard data set to obtain a spectrogram; The spectrum image segment containing the chicken sounds in the spectrum image is intercepted, and the spectrum image segment is converted into a Mel spectrum image through a Mel filter bank; Constructing a deep feature extractor, and using the deep feature extractor to extract the sound deep features of the Mel-spectrogram; the deep feature extractor includes a multi-scale perception module and a time domain dynamic enhancement module; A frame-level sound classifier is constructed to process and analyze the deep features of the sound to obtain the time domain distribution of the sound events of the chicken flock; the frame-level sound classifier includes a bidirectional long short-term memory feedforward network.

2. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 1 is characterized in that: The preprocessing of the sound signal in the standard data set includes: Convert all sound signals to mono format and resample to the set frequency; The resampled sound signal is pre-emphasized, and the pre-emphasis processing is performed on the resampled sound signal, specifically: Compensate the high frequency component of the sound signal as follows: y(n)=x(n)-αx(n-1) a∈(0.9,1.0) Where x(n) is the sound sampling value at time n, the result after pre-emphasis processing is y(n), and a is the pre-emphasis coefficient; The pre-emphasized sound signal is framed using the Hanning window and windowed using window shifting. Performing short-time discrete Fourier transform on the processed sound signal after windowing the sound signal to obtain a spectrum diagram of the sound signal; Spectral subtraction is used to denoise the spectrogram.

3. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 1 is characterized in that: The intercepting of the spectrum segment containing the chicken sounds in the spectrum segment and converting the spectrum segment into a Mel spectrum segment through a Mel filter bank comprises: Calculate the short-time energy of each frame of sound signal to determine whether the frame contains the sound of chickens, and according to the judgment result of the short-time energy, cut the spectrum graph corresponding to the frame containing the sound of chickens to obtain the spectrum graph segment containing the sound of chickens; Pass the spectrum graph segment through the Mel filter bank to obtain the Mel spectrum graph.

4. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 3 is characterized in that: For the vth frame sound signal y of the chicken sound signal y(n) v (n), the calculation formula of its short-time energy Ev is as follows: Among them, L represents the frame length, and n represents the sampling point number of the chicken sound sample; The short-time energy Ev is judged based on the single-parameter dual-threshold endpoint detection method. When the short-time energy Ev exceeds the high threshold, it is marked as the beginning of the chicken sound frame. When the short-time energy Ev is lower than the low threshold, it is marked as the end of the chicken sound frame.

5. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 1 is characterized in that: The multi-scale perception module includes three layers of cascade learning, each layer of cascade learning includes an overlapping block and a multi-scale perception self-attention block, and the overlapping block is embedded before the multi-scale perception self-attention block; the overlapping block of the first layer is an improved overlapping block, including a common overlapping block, a 3×3 convolution and a non-overlapping 2×2 convolution, and the common overlapping block includes a 7×7 convolution kernel and a convolution with a stride of 2; the overlapping blocks of other layers are common overlapping blocks; the improved overlapping blocks are as follows: X l,1 =Conv 2×2，2 (Conv 3×3，1 (Conv 7×7，2 (X l ))) The common overlapping blocks are as follows: X l,i＝2，3 =Conv 7×7，2 (X l ) Where Conv 2×2，2 Indicates the use of a 2×2 convolution kernel with a step of 2; X l,i Represents the feature map input to the i-th level; The multi-scale perceptual self-attention block includes a multi-scale perceptual self-attention layer and an improved multi-layer perceptron, wherein the improved multi-layer perceptron includes a skip connection for supplementing cross-information between example features.

6. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 5 is characterized in that: The method of extracting the sound depth features of the Mel-spectrogram using a deep feature extractor includes: The first step is to convert the Mel spectrum map X l Input to the multi-scale perception module, and obtain the local feature map X through overlapping blocks l,i ; The second step is to transform the local feature map X l,i Input the multi-scale perceptual self-attention block, and use the multi-scale perceptual self-attention layer to transform the local feature map X l,i Divide the feature X into h equal parts in the channel dimension i ={x1,x2,…,x h }, each equal part of feature X i Mapped to query Q, key K and value V vectors as follows: where d k is the feature dimension of each attention head, The key K and value V vectors are mapped to two different scales through convolution operations, as shown below: K′ i ,V′ i =Conv(K i ,r i ),Conv(V i ,r i ) where r i is the downsampling scale of the ith self-attention mechanism head, Calculate multi-head self-attention as follows: where d k is the feature dimension of each attention head; The third step is to concatenate the calculation results of each head to obtain multi-scale information, and use the multi-layer perceptron to exchange and aggregate the multi-scale information to obtain the feature X∈T C×T×F , where C represents the number of channels of the feature, T represents the time dimension of the feature, and F represents the frequency dimension of the feature.

7. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 1 is characterized in that: The temporal dynamic enhancement module includes a local branch and a global branch. The local branch includes multiple nonlinear one-dimensional convolutional layers and ReLU activation functions, which are used to identify instantaneous information in short-term temporal changes. The first one-dimensional convolutional layer reduces the number of channels from C to The second one-dimensional convolutional layer includes a sigmoid activation function; The global branch includes a depth-separable dynamic convolution for learning an adaptive convolution kernel, as shown in the following formula: Among them, θ∈R C×K It is the convolution kernel aggregated from the time domain features, K is the size of the convolution kernel, and W1 and W2 are both linear layers.

8. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 7 is characterized in that: The method of extracting the sound depth features of the Mel-spectrogram using a deep feature extractor is specifically as follows: The features output by the multi-scale perceptual self-attention block are averaged and pooled along the frequency dimension. The formula is as follows: Where c, t and f are the indexes of channel, time domain and frequency domain respectively. represents the frequency information of the aggregation, X c,t,i represents the features output by the multi-scale perceptual self-attention block, and F represents the frequency dimension of the features; The aggregated frequency information Input to the local branch and use the local branch to capture instantaneous features as follows: Where ⊙ represents element-by-element multiplication, V is broadcast along the frequency dimension to adapt to the dimension of X, and the output dimension is Z∈R C×T×F ; The aggregated frequency information Input to the global branch, model the contextual relationship through the adaptive convolution kernel, and obtain the aggregation weight θ; perform dynamic convolution operation on the aggregation weight θ and the output result Z of the local branch, as shown in the following formula: Among them, * represents the convolution operation.

9. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 1 is characterized in that: The processing and analysis of the sound depth feature includes: The sound depth features are input frame by frame into the bidirectional long short-term memory feedforward network for classification processing to obtain the label probability vector O at each moment t And the time period corresponds to the chicken sound event label h t ; The binary cross entropy is used to calculate the model training loss as follows: Where E is the binary cross entropy loss function, O k and P k They represent the estimated value and label value at sample index k respectively, and the total amount of data is K.

10. The chicken sound event detection method based on multi-scale perception and time domain dynamic enhancement according to claim 9, characterized in that: The sound depth features are input into the bidirectional long short-term memory feedforward network frame by frame for classification processing, specifically: For the bidirectional long short-term memory feedforward network module at time t, forward calculation and backward calculation are performed simultaneously. For each time step t, the forward calculation from t = 1 to t = T is as follows: in, It is a complex computing unit inside the long short-term memory network module. is the weight matrix input to the hidden layer, is the recursive weight matrix from the previous hidden state to the current hidden state, is the bias term; For each time step t, the backward calculation from t = T to t = 1 is as follows: Concatenate the forward hidden state and the backward hidden state to form the comprehensive feature h at the current moment t ; The comprehensive feature ht is input into a fully connected layer and the frame-level category probability is output. The formula is as follows: O t =Softmax(W o h t +b o ) Where W o is the weight matrix of the classification layer, b o is the bias value of the classification layer, O t is the converted category probability.

Images