Laying hen voice recognition method and system fusing acoustic features and deep learning features

Abstract

The invention provides a laying hen voice recognition method and system fusing acoustic features and deep learning features. The method comprises the steps of obtaining a to-be-recognized original audio signal and a voice recognition model; wherein the voice recognition model comprises a feature extraction network, a feature fusion network and a classification recognition network; performing feature extraction on the original audio signal by using the feature extraction network to obtain a spectrogram feature, a Mel-frequency cepstrum coefficient feature and a deep speech feature; the feature fusion network performs feature fusion on the spectrogram features, the Mel-frequency cepstrum coefficient features and the deep speech features by using a collaborative attention mechanism or a multi-head attention mechanism to obtain fused features; and inputting the fused features into a classification recognition network to obtain a voice recognition result. According to the method, the advantages of various characteristics can be fully utilized, and the sound signals are described and analyzed from multiple angles, so that the voiceprint of the laying hen is more accurately recognized, and the voiceprint recognition accuracy of the laying hen is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of poultry farming monitoring technology, and in particular to a laying hen sound recognition method and system integrating acoustic features and deep learning features. Background Art

[0002] Intensive, digital, and intelligent farming, driven by animal welfare, are effective pathways to high-quality development in the laying hen industry. These initiatives require fully safeguarding animal welfare during the breeding process, ensuring that their physiological needs are met as much as possible, rationally raising and utilizing animals, ensuring their health and psychological well-being, minimizing suffering, and fostering harmonious coexistence between animals and humans. However, animals lack the ability to communicate directly with humans, making the assessment of their growth status, psychological state, and health needs a hot topic in animal science. Animal behavior can reflect their physical and psychological health, as well as their adaptability to the breeding environment. Animal behavior encompasses not only physical behavior but also vocalizations, a direct reflection of their adaptability to the environment and their physical condition. These vocalizations are often considered a key assessment method in animal welfare research. With the advancement of digital speech analysis technology, quantitative analysis and feature recognition of livestock and poultry vocalizations have become a reality. Therefore, analyzing and studying different types of livestock and poultry audio has important implications for achieving healthy livestock and poultry farming and improving production management efficiency.

[0003] With the continuous development of artificial intelligence, deep learning has become a key technology in scientific research and has been widely applied in acoustic studies of animals, livestock, and poultry. However, most existing research relies solely on single acoustic features or simply fuses multiple features into a network. For layer-laying hen voiceprint recognition, no research has yet combined traditional acoustic features with deep learning features. In particular, the recognition of layer-laying hen voices faces the challenge of comprehensively representing the multidimensional characteristics of the sound signal. Summary of the Invention

[0004] The present invention provides a method and system for identifying laying hen sounds that integrates acoustic features and deep learning features. This method addresses the existing problem of difficulty in fully characterizing the multi-dimensional features of sound signals in identifying laying hen sounds, thereby enabling accurate classification and identification of laying hen sounds. The technical solutions proposed by the present invention are as follows: In a first aspect, the present invention provides a method for identifying laying hen sounds by integrating acoustic features and deep learning features, comprising: Obtaining an original audio signal to be recognized and a sound recognition model; wherein the sound recognition model includes a feature extraction network, a feature fusion network, and a classification recognition network; Extracting features from the original audio signal using the feature extraction network to obtain spectrogram features, Mel-frequency cepstral coefficient features, and deep speech features; The feature fusion network uses a collaborative attention mechanism or a multi-head attention mechanism to fuse the spectrogram features, the Mel-frequency cepstral coefficient features, and the deep speech features to obtain fused features; The fused features are input into the classification and recognition network to obtain a sound recognition result.

[0005] Optionally, the feature extraction network includes a Mel filter bank and an improved ViT model; the improved ViT model replaces the standard global self-attention mechanism with a time-frequency dual-branch attention mechanism, and replaces the fully connected multi-layer perceptron in the ViT model with a convolutional feedforward network with time-frequency local perception; the spectrogram is extracted by: Denoising the original audio signal to obtain a denoised audio signal; Performing frame windowing on the denoised audio signal to obtain a windowed audio signal; Performing Fourier transform on each frame of the windowed audio signal to obtain a linear spectrum of each frame; arranging the linear spectrum of each frame in chronological order to form a spectrogram; The spectrogram is input into the improved ViT model to obtain spectrogram features.

[0006] Optionally, the mel-spectrogram is input into the improved ViT model to obtain mel-spectrogram features, including: Filling and normalizing the spectrum graph to obtain a processed spectrum graph; Scaling the processed spectrum graph, and performing a block operation on the scaled spectrum graph to obtain a plurality of spectrum blocks; Perform linear transformation on each spectrum block to obtain the corresponding spectrum block vector; Add spectrum embedding to the spectrum block vector to obtain a processed spectrum block vector; Spectrum position embedding is performed on the spectrum block vector, and each processed spectrum block vector is added to the spectrum position embedding vector of the corresponding position to obtain the spectrum graph feature.

[0007] Optionally, the feature extraction network includes a Mel filter bank and a bidirectional long short-term memory network; the Mel frequency cepstral coefficient feature is extracted by: Performing high-pass filtering on the original audio signal to obtain a preprocessed audio signal; Dividing the preprocessed audio signal into multiple frames, and performing smoothing processing on each frame using a preset window function to obtain a smoothed audio signal; wherein each frame includes N sampling points; Perform Fourier transform on each frame of smoothed audio signal to convert it into frequency domain signal to obtain amplitude spectrum; Input the amplitude spectrum into a Mel filter bank, and take the logarithm of the energy value output by each Mel filter; Perform discrete cosine transform on the logarithmic energy values output by all Mel filters, and retain the first N coefficients as the original Mel frequency cepstral coefficient features; The original Mel-frequency cepstral coefficient feature is input into a bidirectional long short-term memory network to obtain a Mel-frequency cepstral coefficient feature.

[0008] Optionally, the feature extraction network includes a pre-trained deep learning model, which includes a feature encoder, a quantization module, and a context network; and the deep speech features are extracted by: Preprocessing the original audio signal to obtain a preprocessed audio waveform; Converting the preprocessed audio waveform through a feature encoder to obtain a speech feature vector; Converting the speech feature vector into discrete speech units using a quantization module to obtain a quantized feature vector; Randomly masking some of the quantized feature vectors to obtain masked feature vectors; The masked feature vector is input into a context network including a Transformer encoder for processing to obtain the deep speech feature.

[0009] Optionally, the feature fusion network utilizes a collaborative attention mechanism to fuse the spectrogram features, the Mel-frequency cepstral coefficient features, and the deep speech features, including: Fusing the Mel-frequency cepstral coefficient feature and the Mel-frequency spectrogram feature as the attention weight of each frame in the deep speech feature; Weighting the features of the corresponding frame in the deep speech feature according to the attention weight of each frame to obtain a weighted deep speech feature; The mel-frequency cepstral coefficient feature, the mel-frequency spectrogram feature and the weighted deep speech feature are concatenated to obtain the fused feature.

[0010] In a second aspect, the present invention further provides a laying hen sound recognition system that integrates acoustic features and deep learning features, comprising the following modules: A signal acquisition module, configured to acquire an original audio signal to be recognized and a sound recognition model; wherein the sound recognition model includes a feature extraction network, a feature fusion network, and a classification recognition network; A feature extraction module is used to extract features from the original audio signal using the feature extraction network to obtain spectrogram features, Mel-frequency cepstral coefficient features and deep speech features; A feature fusion module is used for the feature fusion network to fuse the spectrogram features, the Mel-frequency cepstral coefficient features and the deep speech features using a collaborative attention mechanism or a multi-head attention mechanism to obtain fused features; The classification and recognition module is used to input the fused features into the classification and recognition network to obtain a sound recognition result.

[0011] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the laying hen sound recognition method that integrates acoustic features and deep learning features as described in the first aspect above.

[0012] In a fourth aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the egg-laying chicken sound recognition method that integrates acoustic features and deep learning features as described in the first aspect above.

[0013] In a fifth aspect, the present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the laying hen sound recognition method that integrates acoustic features and deep learning features as described in the first aspect above.

[0014] Based on the above technical solution, the present invention has the following beneficial effects compared with the prior art: The present invention provides a system for layer hen voice recognition that integrates acoustic and deep learning features. By fusing acoustic features (spectrogram features and Mel-frequency cepstral coefficient features) with deep learning features (i.e., deep speech features), the acoustic features provide the physical properties of the audio signal, offering clear interpretability and computational efficiency. Deep learning features provide high-level semantic information about the audio signal, capable of capturing complex patterns and context. Feature fusion, utilizing a collaborative attention mechanism or a multi-head attention mechanism, dynamically balances the contributions of acoustic and deep learning features, generating more expressive fused features. The combination of acoustic and deep learning features provides richer information and effectively improves the accuracy of layer hen voice recognition. By integrating traditional acoustic features with deep learning features, comprehensive multidimensional feature representation is achieved. The present method leverages the strengths of various features to describe and analyze sound signals from multiple perspectives, enabling more accurate recognition of layer hen voiceprints and significantly improving the accuracy of layer hen voiceprint recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 It is a flow chart of the laying hen sound recognition method provided by the present invention that integrates acoustic features and deep learning features.

[0017] Figure 2 It is a structural schematic diagram of the improved ViT model provided by the present invention.

[0018] Figure 3 It is a structural diagram of the Wav2vec2 model provided by the present invention.

[0019] Figure 4 It is a flow chart of the sound recognition model provided by the present invention.

[0020] Figure 5 It is a structural schematic diagram of the laying hen sound recognition system provided by the present invention that integrates acoustic features and deep learning features.

[0021] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0023] Reference Figure 1 As shown, the laying hen sound recognition method integrating acoustic features and deep learning features includes the following: S110 , obtaining an original audio signal to be recognized and a sound recognition model; wherein the sound recognition model includes a feature extraction network, a feature fusion network, and a classification recognition network.

[0024] Obtain the original audio signals to be recognized. These signals are recorded from the laying hen's environment using a microphone or other audio acquisition device. Furthermore, in the field of laying hen sound recognition, given that sound is a key information carrier and multiple features are crucial for accurate recognition, a multi-feature fusion neural network framework driven by sound is constructed, namely the aforementioned sound recognition model. Load the pre-trained sound recognition model. This model consists of three main components: a feature extraction network, a feature fusion network, and a classification and recognition network.

[0025] S120: Utilize the feature extraction network to perform feature extraction on the original audio signal to obtain spectrogram features, Mel-frequency cepstral coefficient features, and deep speech features.

[0026] The feature extraction network is responsible for extracting multiple features from the original audio signal. Specifically, it extracts the following three features: Spectrogram: A spectrogram is a time-frequency representation method based on a linear frequency scale. It extracts the amplitude spectrum of the audio signal through frame division and windowing, and performs short-time Fourier transform (STFT). It then generates a time-frequency energy distribution diagram through logarithmic compression.

[0027] Mel-Frequency Cepstral Coefficients (MFCC): MFCC is an acoustic feature that converts the audio signal's spectrum into the cepstral domain and extracts coefficients that represent the sound's characteristics. MFCC features typically consist of multiple cepstral coefficients and can effectively represent the spectral envelope and details of the sound.

[0028] Deep speech features are high-level features extracted from raw audio signals using deep learning models (such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs). These features capture complex patterns and contextual information in audio signals and are extracted using pre-trained deep learning models.

[0029] S130, the feature fusion network uses a collaborative attention mechanism or a multi-head attention mechanism to fuse the spectrogram features, the Mel-frequency cepstral coefficient features and the deep speech features to obtain fused features.

[0030] The feature fusion network fuses the spectrogram features, MFCC features, and deep speech features obtained from the feature extraction network. To effectively fuse these features, the method of the present invention employs a co-attention mechanism or multi-head attention mechanism. The co-attention mechanism simultaneously considers the relationships between multiple features. It dynamically adjusts the contribution of each feature in the fusion process by calculating attention weights between different features. Specifically, the co-attention mechanism calculates attention weights for the spectrogram features, MFCC features, and deep speech features, and then performs a weighted sum of these features to produce the fused features. This mechanism effectively captures the complementary information between different features, thereby improving the expressive power of the fused features.

[0031] S140: Input the fused features into the classification and recognition network to obtain a sound recognition result.

[0032] The fused features are input into a classification and recognition network, which can be a deep neural network (such as a fully connected neural network, a convolutional neural network, or a recurrent neural network). The classification and recognition network's task is to classify the sound based on the fused features and ultimately output the sound recognition result.

[0033] The classification and recognition network uses multiple layers of nonlinear transformations to map the fused features to different categories (such as the behavioral states and health status of laying hens). The final layer of the network can be a Softmax layer, which outputs a probability distribution for each category. Ultimately, the category with the highest probability is selected as the sound recognition result.

[0034] The present invention employs supervised learning to train the sound recognition model. The training data includes a large number of laying hen sound samples (e.g., typical audio data such as singing, egg-laying sounds, warning sounds, and distress sounds) and their corresponding labels (e.g., health status, behavioral category, etc.). The model's training objective is to minimize the difference between the predicted results and the true labels, using a cross-entropy loss function. Using a backpropagation algorithm, the model continuously adjusts network parameters to improve sound recognition accuracy.

[0035] This method can be applied to automated monitoring systems in laying hen farms to monitor the health and behavior of laying hens in real time. For example, by recognizing the sounds of laying hens, the system can determine whether the hens are stressed or sick, thereby helping farmers take timely management measures.

[0036] Acoustic features are features extracted from audio signals that reflect the physical properties of sound. In the method described in this paper, acoustic features primarily include spectrograms and Mel-Frequency Cepstral Coefficients (MFCCs). The spectrogram decomposes the audio signal into a linear amplitude spectrum of time-frequency units using a short-time Fourier transform (STFT), which is then logarithmically compressed to generate a time-frequency energy distribution. MFCCs perform cepstrum analysis on the audio signal's spectrum to extract coefficients that represent the sound's spectral envelope and details.

[0037] Deep learning features refer to high-level features automatically learned from raw audio signals using deep neural networks (such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs). These features are derived through a data-driven training process and can capture the complex structure and nonlinear relationships in audio signals. In the present method, deep learning features refer to the aforementioned deep speech features. Deep speech features are extracted from raw audio signals using pre-trained deep learning models (such as CNNs or RNNs). These features can represent high-level semantic information in audio signals, such as the context and pattern of sounds.

[0038] Acoustic features have limited expressive power and struggle to capture complex patterns and contextual information in audio signals. Deep learning features can automatically learn complex patterns in data, but their drawbacks are that they require large amounts of annotated data and computational resources, and their interpretability is poor. The method presented in this paper fuses acoustic features (such as spectrogram features and MFCC features) with deep learning features (such as deep speech features). Acoustic features provide the physical properties of audio signals, offering clear interpretability and computational efficiency. Deep learning features provide high-level semantic information about audio signals, enabling the capture of complex patterns and context. Feature fusion using a collaborative attention mechanism or a multi-head attention mechanism dynamically balances the contributions of acoustic and deep learning features, generating more expressive fused features. The combination of acoustic and deep learning features provides richer information and can effectively improve the accuracy of laying hen sound recognition. The fused features are more robust to noise and variability, adapting to diverse environments and scenarios. For complex laying hen sound recognition tasks, such as identifying different behavioral states or health conditions, fused features can better address these challenges. This method has broad application prospects in the automated monitoring of laying hen farms.

[0039] Most existing research relies solely on single acoustic features. While these features can capture the time and frequency domain characteristics of speech signals, they often fail to fully characterize the multidimensional characteristics of laying hen sound signals. This invention, by fusing multiple traditional acoustic features, such as spectrograms and Mel-Frequency Cepstral Coefficients (MFCCs), can extract features from different perspectives, thereby more comprehensively reflecting the characteristics of laying hen sounds. This fusion captures more detailed information in the sound signal, improving recognition accuracy. Furthermore, combining deep learning features with acoustic features leverages the powerful feature learning capabilities of deep learning technology to extract deeper features from the raw sound signal. These deep learning features can capture nonlinear relationships and complex patterns in the sound signal, thereby improving recognition robustness and accuracy.

[0040] The recognition of laying hen voices faces the problem of difficulty in comprehensively characterizing the multi-dimensional features of sound signals. Traditional methods can often only extract a limited number of features, and although deep learning technology can extract a large number of features, it may not be able to fully utilize these features. The present invention can achieve comprehensive characterization of multi-dimensional features by integrating traditional acoustic features and deep learning features. It can fully utilize the advantages of various features to describe and analyze sound signals from multiple angles, thereby more accurately identifying the voiceprints of laying hens and significantly improving the accuracy of laying hen voiceprint recognition. In practical applications, there are often various background noises in laying hen farms. The method of the present invention can utilize the robustness of traditional acoustic features to noise and the ability of deep learning features to recognize complex patterns, thereby enhancing the noise resistance of the system.

[0041] In an optional embodiment, ViT is a visual model based on the Transformer architecture for processing image data. It can be used to process different input resolutions and sizes, and has the ability to capture long-range dependencies and global context through a self-attention mechanism. However, when processing spectrograms, the standard global self-attention mechanism of the existing ViT model is difficult to effectively capture the frequency domain regularity of the harmonic structure and flattens the two-dimensional time-frequency features into one-dimensional processing, thereby destroying the inherent structural correlation of the time-frequency domain. To address these problems, the present invention proposes an improved ViT architecture (Time-Frequencyaware Vision Transformer, TF-ViT), hereinafter referred to as the improved ViT model. This architecture replaces the standard global self-attention mechanism with a time-frequency dual-branch attention mechanism and replaces the fully connected multi-layer perceptron in ViT with a convolutional feedforward network with time-frequency local perception, thereby further improving the feature extraction capability in the laying hen sound recognition task.

[0042] The core idea of the time-frequency dual-branch attention is to model attention in the time domain and frequency domain respectively, and combine harmonic bias and dynamic sparse attention to improve the representation ability of time-frequency features. The harmonic structure (i.e., the fundamental frequency and its multiples) in audio signals plays a vital role in classification tasks. However, the global attention mechanism of the existing ViT model fails to explicitly capture and model the intrinsic correlation between harmonics when processing the spectrum graph, which limits the model's perception of key features to a certain extent. To this end, the present invention introduces harmonic bias attention. By adding a learnable scalar parameter to the attention, each attention head h independently learns a scalar bias value , which is used to adjust the sensitivity of the head to the harmonic relationship when calculating the attention weight. The calculation formula is as follows: Where, The query matrix represents the information that the current word or sentence wants to focus on; is the key matrix, representing the features of all words; is a value matrix, which represents the final feature after attention weighting. is the dimension of each attention head, As a scaling factor to prevent gradients from disappearing or exploding, , represents the real number space, is the number of attention heads, The shape is . is the activation function.

[0043] At the same time, a dynamic sparse gating mechanism is used to process time domain and frequency domain information respectively. The time domain sparse mask is generated by the Sigmoid gating network. and frequency domain mask , can realize the dynamic allocation of attention in time domain and frequency domain, and can reduce redundant calculations compared to traditional ViT global attention. The attention formula is as follows: Where, is the time domain attention weight matrix, is the frequency domain attention weight matrix.

[0044] Finally, the fully connected Multi-Layer Perceptron (MLP) in ViT is replaced with a Time-Frequency Convolutional Feed-Forward Network (TFConvFFN), which is sensitive to local time-frequency patterns. MLPs primarily rely on global nonlinear transformations, making it difficult to effectively capture the detailed characteristics of local time-frequency patterns in audio signals. The proposed TFConvFFN, however, introduces two-dimensional convolutions that slide simultaneously on both the time and frequency axes, thereby more comprehensively modeling local correlations in the time-frequency domain. This design effectively enhances the ability to recognize dynamic changes in the time domain and harmonic features in the frequency domain.

[0045] The relevant parameter settings for spectral graph feature extraction based on the TF-ViT model are shown in Table 1 below: Table 1 The feature extraction network includes a Mel filter bank and an improved ViT model; the Mel spectrum graph is extracted by: S210: De-noise the original audio signal to obtain a de-noised audio signal.

[0046] First, the original audio signal is denoised to eliminate background noise and interference, improving the accuracy of subsequent feature extraction. This denoising method can employ filtering (such as low-pass or high-pass filtering) or a deep learning-based denoising model. The denoised audio signal is cleaner, more accurately reflecting the characteristics of laying hen sounds, and reducing noise interference with feature extraction.

[0047] The present invention performs an improved spectral subtraction denoising on the original audio signal. The improved spectral subtraction calculation formula is as follows: (1) in, (2) Where, is the power spectrum amplitude of the noisy signal, is the power spectrum amplitude of the noise, is the oversubtraction factor, .

[0048] S220 : Perform frame windowing on the denoised audio signal to obtain a windowed audio signal.

[0049] The denoised audio signal is subjected to frame-based windowing. Because audio signals are time-varying, directly processing the entire signal will result in information loss. Therefore, the signal must be segmented into short time frames (each frame can be 20-40 milliseconds long) and a window function (such as a Hamming window or Hanning window) is applied to each frame to reduce spectral leakage. Frame-based windowing converts a continuous audio signal into short, stationary segments, facilitating subsequent spectral analysis.

[0050] Specifically, the denoised audio signal is divided into multiple segments, and then the audio signal of each segment is preprocessed, including resampling, pre-emphasis, frame division and windowing. First, the audio signals of all segments are uniformly resampled to 16000 Hz, because the Wav2vec2 model requires that the sampling rate of all audio is 16000 Hz; then the resampled audio signal is pre-emphasized. Pre-emphasis is achieved through a first-order high-pass filter, and its differential equation can be expressed as: (3) Where, 、 They are nth, The audio signal of a fragment, is the pre-emphasized audio signal, It is the pre-emphasis coefficient, usually around 0.95.

[0051] The pre-emphasized audio signal is then framed and windowed. The frame length is selected as 25ms, and the frame shift between adjacent frames is 40%. To reduce the impact of spectrum leakage after Fourier transform, each frame of the audio signal is windowed after framing. The Hamming window is selected as the window function. The calculation formula is as follows: (4) Where, is the windowed audio signal, is the index of the sample point in the window, Window length.

[0052] S230 , performing Fourier transform on each frame of the windowed audio signal to obtain a linear spectrum of each frame.

[0053] A Fast Fourier Transform (FFT) is performed on each frame of the windowed audio signal, converting the time-domain signal into the frequency domain. This yields a linear spectrum for each frame. The linear spectrum represents the energy distribution of the signal at different frequencies. The Fourier transform reveals the frequency components of the audio signal, providing the basis for subsequent Mel-scale conversion.

[0054] Specifically, a fast Fourier transform (FFT) is performed on each frame of the windowed audio signal to obtain the spectrum information of each frame. Assume that the number of FFT points is , then the spectrum The calculation formula is as follows: (5) Where k represents the frequency index, is the first Sample values, is the index of the sample point in the window, Is an imaginary unit.

[0055] S240: Arrange the linear spectrum of each frame in chronological order to form a spectrogram. A spectrogram is a two-dimensional matrix with time on the horizontal axis and Mel-frequency on the vertical axis. Each value in the matrix represents the energy intensity at the corresponding time and frequency. The spectrogram can intuitively represent the time-frequency characteristics of the audio signal, providing input for subsequent feature extraction.

[0056] S250: Input the spectrogram into the improved ViT model to obtain spectrogram features.

[0057] The generated spectrogram is input into the improved Visual Transformer (ViT) model, and the spectrogram features are extracted by leveraging the powerful feature learning capabilities of the ViT model. The improved ViT model divides the spectrogram into multiple spectral patches and extracts high-level feature representations through a multi-layer Transformer encoder. The improved ViT model can capture the global relationship between different regions in the spectrogram through the self-attention mechanism, thereby better understanding the time-frequency structure of the audio signal. The improved ViT model can extract high-level semantic features from the spectrogram, which can represent the complex patterns and contextual information in the audio signal. The improved ViT model has strong generalization capabilities and can adapt to different audio tasks and data distributions.

[0058] Denoising the original audio signal can reduce noise interference and improve signal quality. Logarithmic compression and convolution operations are robust to noise and can suppress its impact to a certain extent. Logarithmic compression enhances the saliency of low-frequency components while suppressing the influence of high-frequency noise, while convolution integrates feature information from different frequency bands in the spectrogram, improving feature representation. The improved ViT model, through a self-attention mechanism and a Transformer encoder, captures global contextual information and further extracts high-level features, enhancing robustness and accuracy. The combination of the spectrogram and the improved ViT model can simultaneously capture local details and global structure of the audio signal. The spectrogram directly extracts the linear spectral energy distribution of the signal through the short-time Fourier transform (STFT), preserving complete frequency domain information. The high-level features extracted by the improved ViT model through the self-attention mechanism can represent complex patterns and semantic information in the audio signal. Denoising and frame windowing can reduce the impact of noise and signal variability on feature extraction, improving feature robustness. This method combines traditional acoustic features (spectrograms and Mel-frequency cepstral coefficients) with deep learning features (features extracted using an improved ViT model) to more comprehensively describe the characteristics of laying hen sound signals and improve recognition accuracy. This method is suitable for complex tasks such as laying hen sound recognition and provides high-quality feature input for subsequent classification and recognition.

[0059] Reference Figure 4 As shown in Figure 2, the improved ViT model enhances feature expression capabilities through the following key modules: Linear Embedding divides the input spectrogram into fixed-size image patches (Patches) and maps them to a high-dimensional feature space through linear projection (Linear layer) to form patch embeddings (PatchEmbeddings). This converts local regions of the original spectrogram (such as time-frequency units) into processable vector representations.

[0060] Positional Embedding adds a learnable positional encoding to each image block, preserving the temporal and frequency-domain positional information of the spectrogram. This overcomes the Transformer's inherent insensitivity to input order and ensures that the model can distinguish features across different time / frequency segments.

[0061] The Transformer Encoder uses self-attention and multi-head attention to globally model the relationships between regions in the spectrogram. It outputs high-level semantic features that can capture complex patterns in audio signals.

[0062] In an optional embodiment, the spectrogram is input into the improved ViT model through a series of operations such as padding, normalization, scaling, blocking, linear transformation, spectrum embedding, and spectrum position embedding, and finally high-level spectrogram features are extracted. These features can not only represent the time-frequency characteristics of the audio signal, but also capture local details and global structure, providing strong feature support for tasks such as laying hen sound recognition. The above-mentioned step S250 of inputting the spectrogram into the improved ViT model to obtain spectrogram features includes: S2501 : Fill and normalize the spectrum graph to obtain a processed spectrum graph.

[0063] To accommodate the input requirements of the improved ViT model, the spectrogram is padded to meet the model's size requirements. Padding can be performed by adding zeros or other fixed values to the edges of the spectrogram. The spectrogram is normalized, scaling each pixel value to a fixed range (such as [0, 1] or [-1, 1]). Normalization accelerates model convergence and improves training stability. Padding ensures that the spectrogram size is consistent with the improved ViT model's input requirements, avoiding size mismatch issues. Normalization eliminates bias in the data distribution, making it easier for the model to learn effective features.

[0064] S2502: Scale the processed spectrum graph, and perform a block operation on the scaled spectrum graph to obtain a plurality of spectrum blocks.

[0065] Scale the processed spectrogram and adjust the resolution of the spectrogram to a size suitable for the block operation. For example, adjust the resolution of the spectrogram from the original size to a fixed size (such as 224×224). Figure 2 As shown in Figure 1, the scaled spectrum graph is divided into multiple fixed-size spectrum blocks (Flattened Patches). Each spectrum block is a small local area, such as 16×16 or 32×32 pixels.

[0066] The scaling operation ensures that the resolution of the spectrogram is suitable for the subsequent blocking operation while reducing the computational complexity. The blocking operation converts the spectrogram into a series of local regions, making it easier for the improved ViT model to model local features.

[0067] S2503: Perform a linear transformation on each spectrum block to obtain a corresponding spectrum block vector.

[0068] A linear transformation is performed on each spectral block, mapping it to a high-dimensional vector space. Specifically, each spectral block (e.g., 16×16 pixels) is flattened into a one-dimensional vector (e.g., 256 dimensions), which is then mapped to a higher-dimensional space (e.g., 768 dimensions) using a linear transformation matrix. This linear transformation maps the spectral block from the original pixel space to a high-dimensional feature space, enabling each spectral block to represent richer feature information. This mapping provides the basis for subsequent feature extraction and fusion.

[0069] S2504: Add spectrum embedding to the spectrum block vector to obtain a processed spectrum block vector.

[0070] A spectrum embedding is added to each spectrum block vector. A spectrum embedding is a learnable vector that represents the position of the spectrum block in the spectrogram. Each spectrum block vector is added to its corresponding spectrum embedding vector. The spectrum embedding provides positional information for each spectrum block, enabling the improved ViT model to distinguish between spectrum blocks at different positions. This positional information is crucial for modeling the global structure of the spectrogram.

[0071] S2505 , performing spectrum position embedding on the spectrum block vector, adding each processed spectrum block vector to the spectrum position embedding vector at the corresponding position, to obtain the spectrum graph feature.

[0072] Position embedding is performed on the spectral block vector. A spectral position embedding is a learnable vector that represents the absolute position of the spectral block in the mel-spectrogram. Each processed spectral block vector is added to the spectral position embedding vector at the corresponding position. This spectral position embedding provides the improved ViT model with the absolute position information of the spectral block, enabling the model to capture the global context in the spectrogram. This position information is crucial for understanding the time-frequency structure of audio signals.

[0073] After the above operations, each spectrum block vector contains the local features, spectrum embedding and spectrum position embedding information of the spectrum block. Figure 2 As shown in Figure 2, these vectors are input into the Transformer Encoder of the improved ViT model, where high-level feature representations are extracted through a multi-layer self-attention mechanism and a feedforward neural network. Ultimately, the feature vectors output by the improved ViT model are spectrogram features. Spectrogram features can represent the time-frequency characteristics, local details, and global structure of the audio signal. These features contain high-level semantic information and provide strong support for subsequent classification and recognition tasks.

[0074] The present invention combines block operation and position embedding to enable the improved ViT model to simultaneously capture the local details and global structure of the mel-spectrogram. Through linear transformation and Transformer encoder, the extracted features have high-level semantic information and can represent complex patterns in audio signals. Spectral embedding and spectral position embedding provide the model with position information of spectral blocks, enabling the model to understand the time-frequency structure of audio signals. The improved ViT model has strong generalization ability and can adapt to different audio tasks and data distributions. Through block and linear transformation, the improved ViT model can efficiently process the mel-spectrogram and extract high-quality features. Through a series of operations such as padding, normalization, scaling, blocking, linear transformation, spectral embedding and spectral position embedding, the spectrogram is input into the improved ViT model, and finally high-level spectrogram features are extracted. These features can not only represent the time-frequency characteristics of the audio signal, but also capture local details and global structure, providing strong feature support for tasks such as laying hen sound recognition.

[0075] Specifically, the spectrogram is input into the improved ViT model as the input data of the improved ViT model. The spectrogram is first padded and normalized to a uniform size, and then the spectrogram is input into the improved ViT model. The input spectrogram is scaled and unified to a size of 224×224. The processed spectrogram is then divided into 16×16 spectrum blocks. Each spectrum block is then linearly transformed through a fully connected layer and stretched to a 1×768 vector. Spectrum_embedding (spectrum embedding, analogous to class_embedding) is then added to become a 1+(768+768) vector, and then merged with the 1×(768+768) pos_spectrum_embedding (spectrum position embedding, analogous to pos_embedding) vector to obtain the spectrogram feature.

[0076] In an optional embodiment, the feature extraction network includes a Mel filter bank and a bidirectional long short-term memory network. The present invention extracts Mel-frequency cepstral coefficient (MFCC) features from the original audio signal through a Mel filter bank and a bidirectional long short-term memory network (BiLSTM). By combining the Mel filter bank and BiLSTM, efficient extraction of high-level Mel-frequency cepstral coefficient features from the original audio signal is achieved. This method can not only capture the spectral characteristics of the audio signal, but also extract temporal dynamics and contextual information through BiLSTM, thereby providing strong feature support for tasks such as laying hen sound recognition. The Mel-frequency cepstral coefficient features are extracted in the following manner: S310: Perform high-pass filtering on the original audio signal to obtain a preprocessed audio signal.

[0077] High-pass filtering is performed on the original audio signal to emphasize high-frequency components. A high-pass filter with a cutoff frequency of several tens of hertz (such as 50Hz or 100Hz) can be used. The purpose of high-pass filtering is to remove low-frequency noise (such as power supply interference or environmental noise) from the audio signal. High-pass filtering improves the quality of the audio signal and reduces the interference of low-frequency noise on subsequent feature extraction.

[0078] S320 , dividing the preprocessed audio signal into multiple frames, and performing smoothing processing on each frame using a preset window function to obtain a smoothed audio signal; wherein each frame includes N sampling points.

[0079] The preprocessed audio signal is divided into multiple frames, each containing 20-40 milliseconds of audio data (for example, for audio with a 16kHz sampling rate, each frame contains 320-640 samples). Each frame is smoothed using a preset window function (such as a Hamming window or Hanning window) to reduce spectral leakage at frame edges. Framing converts the continuous audio signal into short, stationary segments, facilitating subsequent frequency domain analysis. Smoothing reduces discontinuities at frame edges and improves the accuracy of spectral analysis.

[0080] S330 , performing Fourier transform on each frame of the smoothed audio signal to convert it into a frequency domain signal to obtain an amplitude spectrum.

[0081] A Fast Fourier Transform (FFT) is performed on each frame of the smoothed audio signal, converting the time-domain signal into the frequency-domain signal to obtain the amplitude spectrum. The amplitude spectrum represents the energy distribution of the signal at different frequencies. The Fourier transform reveals the frequency components of the audio signal, providing the basis for subsequent Mel filter bank processing.

[0082] S340: Input the amplitude spectrum into a Mel filter bank, and take the logarithm of the energy value output by each Mel filter.

[0083] The magnitude spectrum is fed into a Mel filter bank. A Mel filter bank is a set of triangular filters whose center frequencies are distributed along the Mel scale, simulating the human ear's frequency perception. The logarithm of the energy output of each Mel filter is taken to compress the dynamic range and enhance low-frequency detail. This Mel filter bank better reflects the human ear's perception of sound while reducing redundant information in high-frequency regions. This logarithmic operation compresses the dynamic range of the energy values, making the features more stable.

[0084] S350 , performing discrete cosine transform on the logarithmic energy values output by all Mel filters, and retaining the first N coefficients as original Mel frequency cepstral coefficient features.

[0085] The logarithmic energy values of all Mel filter outputs are subjected to a discrete cosine transform (DCT) to obtain cepstral coefficients. The first N cepstral coefficients (e.g., 12-20) are retained as the Mel-Frequency Cepstral Coefficient (MFCC) feature vector. These coefficients represent the spectral envelope and details of the audio signal. The DCT transforms the output of the Mel filter bank into the cepstral domain, extracting coefficients that represent the characteristics of the sound. Retaining the first N coefficients reduces the feature dimensionality while retaining the most important information.

[0086] S360: Input the original Mel-frequency cepstral coefficient feature into a bidirectional long short-term memory network to obtain a Mel-frequency cepstral coefficient feature.

[0087] The MFCC feature vectors are input into a bidirectional long short-term memory (BiLSTM) network. BiLSTM is a deep learning model that can capture long-term dependencies in sequential data. It processes the input sequence through two LSTM layers (one forward and one backward). Each LSTM layer contains 128 neurons, the input feature dimension is 13, and the hidden layer dimension is set to 512. The BiLSTM model encodes the MFCC feature vectors and extracts high-level temporal features. BiLSTM can capture the temporal relationship between MFCC feature vectors, thereby better understanding the dynamic changes of audio signals. Through bidirectional processing, BiLSTM can simultaneously utilize past and future contextual information to extract more expressive features. BiLSTM can extract high-level semantic features from MFCC feature vectors. These features can represent complex patterns and contextual information in audio signals.

[0088] BiLSTM is a variant of recurrent neural network (RNN), which can consider contextual information when processing sequence data. BiLSTM consists of forward LSTM and backward LSTM. The forward LSTM processes the input sequence from front to back in chronological order to capture the positive dependency in the sequence; the backward LSTM processes the input sequence from back to front to capture the reverse dependency. The LSTM unit contains an input gate. , Forget Gate , output gate and cell status The corresponding formula is as follows: (6) (7) (8) (9) (10) Where, It is the current moment input, that is, the MFCC feature vector of a certain frame. is the hidden state at the previous moment, is the sigmoid function, .

[0089] is the weight, where the subscript express The corresponding weight, Indicates the hidden state at the last moment Corresponding weights. I, f, o, c represent the weights of the input gate, forget gate, output gate, and cell state, respectively. is the bias, which is used to adjust the input of the activation function. It is the hidden state at the current moment, which contains the sequence information up to the current moment.

[0090] The MFCC feature vector is input into BiLSTM, and the BiLSTM processing process is as follows: 1. Input sequence preparation: Convert the original speech signal into a sequence of MFCC feature vectors. Each frame of speech signal corresponds to one MFCC feature vector.

[0091] 2. Forward LSTM processing: Starting from the first MFCC feature vector of the sequence, it is input into the forward LSTM in sequence. According to formulas (7)-(11), the input gate at each moment is calculated , Forget Gate , output gate and cell status , and the hidden state . Hidden state Contains the forward sequence information up to the current moment.

[0092] 3. Backward LSTM processing: Starting from the last MFCC feature vector of the sequence, it is input into the backward LSTM in sequence. Also according to formulas (7)-(11), the gating mechanism and cell state at each moment, as well as the hidden state are calculated. , here is the hidden state at a different time from the forward LSTM). Hidden state Contains the reverse sequence information up to the current moment.

[0093] 4. Bidirectional merging: For each moment , the hidden states of the forward LSTM and the backward LSTM Merge (e.g., concatenate) the two-way hidden state at that moment. This two-way hidden state contains both forward and reverse sequence information and can be used for subsequent tasks (e.g., classification, regression, etc.).

[0094] Through the above process, BiLSTM can fully utilize the context information of sequence data and improve the performance of the model.

[0095] After processing by the BiLSTM model, the output feature vector is the Mel-frequency cepstral coefficient feature. These features contain not only the spectral information of the MFCC but also temporal and contextual information. Mel-frequency cepstral coefficient features can represent the spectral characteristics, temporal dynamics, and contextual information of the audio signal. These features contain high-level semantic information and provide strong support for subsequent classification and recognition tasks.

[0096] This method combines a Mel filter bank with a BiLSTM to simultaneously capture the spectral characteristics and temporal dynamics of audio signals. The Mel filter bank simulates the human ear's frequency perception, making the extracted features more consistent with human auditory perception. The BiLSTM can capture long-term dependencies in audio signals and extract temporal dynamic features. Through bidirectional processing, the BiLSTM can leverage past and future contextual information to enhance the expressive power of features. This method is suitable for complex tasks such as laying hen sound recognition, providing high-quality feature input for subsequent classification and recognition.

[0097] In an optional embodiment, the feature extraction network includes a pre-trained deep learning model, which includes a feature encoder, a quantization module, and a context network. Through the combination of the feature encoder, the quantization module, and the context network, efficient extraction of high-level deep speech features from the original audio signal is achieved. This method can not only capture the local details and spectral characteristics of the audio signal, but also extract global context information through the Transformer encoder, thereby providing strong feature support for tasks such as laying hen sound recognition. The deep speech features are extracted in the following way: S410: Preprocess the original audio signal to obtain a preprocessed audio waveform.

[0098] Preprocess the raw audio signal, including denoising, normalization, and resampling. The goal of preprocessing is to improve the quality of the audio signal and make it suitable for subsequent feature extraction. Preprocessing can reduce the interference of noise and signal variability on feature extraction and improve the stability and consistency of features.

[0099] S420: Convert the preprocessed audio waveform through a feature encoder to obtain a speech feature vector.

[0100] The preprocessed audio waveform is fed into a feature encoder. This encoder, a convolutional neural network (CNN) or recurrent neural network (RNN), extracts low-level speech features from the audio waveform. The feature encoder converts the audio waveform into a continuous sequence of speech feature vectors. Each feature vector represents a local feature of the audio signal over a specific time period. The feature encoder captures the local details and spectral characteristics of the audio signal, providing a foundation for subsequent feature extraction.

[0101] S430: Utilize a quantization module to convert the speech feature vector into discrete speech units to obtain a quantized feature vector.

[0102] The speech feature vector is converted into discrete speech units using a quantization module. The quantization module can be a vector quantization (VQ) layer, which discretizes the features by mapping the continuous speech feature vector to the nearest neighbor vector in a discrete codebook. The quantized feature vector consists of discrete speech units, which represent the basic building blocks of the audio signal. Quantization converts the continuous speech feature vector into a discrete representation, reducing feature redundancy. Discrete speech units better represent the basic structure and patterns of the audio signal.

[0103] S440 : Randomly mask some of the quantized feature vectors to obtain masked feature vectors.

[0104] Randomly mask some of the quantized feature vectors. Specifically, a certain percentage of feature vectors (e.g., 15%) is randomly selected and replaced with mask tokens. This masking operation simulates missing or noisy audio signals, increasing the robustness of the model. Random masking can enhance the model's generalization ability, enabling it to extract valid features even when some information is missing. Masking can also guide the model to learn contextual information, thereby improving the expressiveness of features.

[0105] S450: Input the masked feature vector into a context network including a Transformer encoder for processing to obtain the deep speech feature.

[0106] The masked feature vector is input into a context network consisting of a Transformer encoder for processing. The Transformer encoder encodes the masked feature vector using a multi-layer self-attention mechanism and a feedforward neural network to extract high-level contextual features. The self-attention mechanism captures global relationships between feature vectors, thereby better understanding the context of the audio signal. The Transformer encoder captures global dependencies in the audio signal and extracts high-level semantic features. Through the multi-layer Transformer encoder, the model extracts deep speech features from the masked feature vector. These features represent complex patterns and contextual information in the audio signal. The combination of masking and the context network makes the model more robust to noise and partial information loss. After processing by the context network, the output feature vector is the deep speech feature. These features contain not only local details of the audio signal but also global contextual information. Deep speech features represent the spectral characteristics, local details, and global contextual information of the audio signal. These features contain high-level semantic information and provide strong support for subsequent classification and recognition tasks.

[0107] The present invention can simultaneously capture local details and global contextual information of audio signals through the combination of a feature encoder, a quantization module, and a context network. The quantization module converts continuous speech feature vectors into discrete speech units, reducing feature redundancy. The Transformer encoder can capture global dependencies in audio signals and extract high-level semantic features. The combination of random masking operations and the context network makes the model more robust to noise and partial information loss. This method can adapt to complex tasks such as laying hen sound recognition and provide high-quality feature input for subsequent classification and recognition.

[0108] The Wav2Vec2.0 model is an audio pre-training model based on self-supervised learning. Through convolutional layers and Transformer structures, it can capture the temporal information and contextual semantics of audio. It is mainly used for speech recognition, speaker recognition, and emotion recognition. This paper uses the pre-trained Wav2Vec2.0 model for end-to-end fine-tuning for the laying hen audio classification task. First, the original audio signal is resampled to 16000Hz and input into the model to achieve classification and recognition of laying hen sounds. Figure 4As shown in Figure 2, the Wav2Vec2.0 model consists of a feature extractor based on a convolutional neural network (CNN) and a Transformer. During the feature extraction stage, the CNN is used for encoding, converting the input audio into a frame-level feature sequence. The VQ module discretizes each frame's features into a feature vector q, which serves as the target for self-supervised learning. Simultaneously, the frame-level feature sequence is masked and fed into the Transformer for contextual embedding, generating a representation C containing contextual information. During model training, a contrastive learning loss function is used to narrow the distance between the contextual representation at the masked position and the corresponding feature vector q, constructing positive sample pairs and achieving self-supervised optimization. The specific settings for the relevant parameters during fine-tuning are shown in Table 2 below.

[0109] Table 2 Taking the pre-trained model wav2vec2-base-960h as an example, the process of deep speech feature extraction is described as follows: Download the pre-trained model wav2vec2-base-960h from the Hugging Face website. It is based on the self-supervised learning framework. Figure 3 As shown, the raw audio signal is first normalized and converted into feature vectors using a feature encoder. The signal then passes through a quantization module to learn discrete speech units. Finally, a contextual network containing a Transformer encoder is used for processing. Some feature vectors are randomly masked, and the model is required to predict these masked vectors. This pre-trained model was pre-trained on 960 hours of LibriSpeech data, which provides the model with rich speech information, enabling it to learn common speech features. Feature extraction can be achieved by fine-tuning this pre-trained model using a custom dataset. Therefore, this paper uses a labeled laying hen audio dataset to fine-tune this pre-trained model to adapt it to its own dataset, using a contrastive loss for fine-tuning. After experimental research, it is found that when fine-tuning wav2vec2-base-960h, it is necessary to set the learning rate to 5e-5, the batch_size to 16, the epoch to 10 rounds, the optimizer to Adaw, and add a regularization term to the model weight with a Dropout of 0.1; then save the fine-tuned wav2vec2-base-960h model as finetuning-wav2vec2-base-chicken for subsequent extraction of deep speech features.

[0110] In an optional embodiment, a co-attention mechanism is a mechanism that simultaneously considers the relationships between multiple features. It dynamically adjusts the contribution of each feature in the fusion process by calculating attention weights between different features. In this invention, the co-attention mechanism is used to fuse spectrogram features, Mel-frequency cepstral coefficient features, and deep speech features to capture their complementary information.

[0111] The feature fusion network described in S130 utilizes a collaborative attention mechanism to fuse the mel-spectrogram feature, the mel-frequency cepstral coefficient feature, and the deep speech feature, including: S1301. Fusing the Mel-frequency cepstral coefficient feature and the spectrogram feature as the attention weight of each frame in the deep speech feature.

[0112] Specifically, an attention computation module (such as a fully connected layer or dot-product attention) calculates the correlation between MFCC features and spectrogram features to generate attention weights. These weights represent the importance of the deep speech features in each frame during the fusion process. Attention weights dynamically adjust the contribution of deep speech features, making the fusion process more flexible and adaptive. By fusing MFCC and spectrogram features, attention weights capture the spectral characteristics and time-frequency structure of the audio signal.

[0113] S1302. Weight the features of the corresponding frame in the deep speech features according to the attention weight of each frame to obtain weighted deep speech features.

[0114] Specifically, the attention weights are multiplied frame by frame with the deep speech features to produce weighted deep speech features. This weighting operation enhances the feature representation of important frames while suppressing unimportant frames, thereby improving the expressiveness of the features. The weighted deep speech features can better reflect the temporal dynamics and contextual information of the audio signal.

[0115] S1303: Concatenate the Mel-frequency cepstral coefficient feature, the spectrogram feature, and the weighted deep speech feature to obtain the fused feature.

[0116] The concatenation operation connects different features along the feature dimension to form a higher-dimensional feature vector. This concatenation operation preserves the original information of each feature while combining them to form a more comprehensive feature representation. The fused features can simultaneously represent the spectral characteristics, time-frequency structure, and temporal dynamics of the audio signal.

[0117] This invention represents audio signals from different perspectives using spectrogram features, MFCC features, and deep speech features: spectrogram features represent the time-frequency characteristics of audio signals, MFCC features represent the spectral envelope information of audio signals, and deep speech features represent the temporal dynamics and contextual information of audio signals. Through a collaborative attention mechanism and feature concatenation, these features complement each other to form a more comprehensive feature representation. The collaborative attention mechanism dynamically adjusts the contribution of each feature during the fusion process, making the fusion process more flexible and adaptive. Attention weights enhance the feature representation of important frames while suppressing unimportant frames, thereby improving the expressiveness of the features. The fused features not only incorporate the spectral characteristics and time-frequency structure of the audio signal, but also the temporal dynamics and contextual information. These features contain high-level semantic information, providing strong support for subsequent classification and recognition tasks. By fusing multiple features, the model becomes more robust to noise and signal variability. The collaborative attention mechanism dynamically adjusts feature weights based on the characteristics of the input signal, further improving the model's robustness. The fused features are suitable for complex audio processing tasks such as laying hen sound recognition and speech recognition.

[0118] Assume that the Mel frequency cepstral coefficient features, spectrogram and deep speech features of each audio signal are respectively , express, represents the number of frames of the original Mel-frequency cepstral coefficient feature, and Represents the dimension of the raw Mel-frequency cepstral coefficient features for each frame (i.e., the number of Mel-frequency cepstral coefficients). represents the number of frames of the spectrogram, and Indicates the dimension of the spectrogram per frame. Indicates the number of frames output by the wav2vec2 model.

[0119] The original Mel frequency cepstral coefficient features are input into BiLSTM: (11) in, , which represents the Mel-frequency cepstral coefficient feature obtained after BiLSTM processing. Represents the dimension of the Mel-frequency cepstral coefficient features obtained after processing. Represents the process of BiLSTM processing the original Mel-frequency cepstral coefficient features and outputting the processed feature vector.

[0120] The spectrogram is input into the improved ViT model: (12) in, , which represents the spectrum features obtained after processing by the improved ViT model. It represents the process of the improved ViT model processing the spectrogram features and outputting the processed feature vector. Indicates the dimension of the spectrogram features obtained after processing.

[0121] Then, the features are fused using the co-attention mechanism. First, the Mel-frequency cepstral coefficient features and the spectrogram features are fused as the attention weights for each frame in the wav2vec2 output: (14) in, , the attention weights calculated by the collaborative attention mechanism, Indicates the number of frames output by the wav2vec2 model. represents the collaborative attention mechanism function, Represents a concat operation.

[0122] Then the multi-frame output of wav2vec2 is linearly weighted.

[0123] (15) in, , to use attention weights right The result after linear weighting, that is, the above-mentioned weighted deep speech features, Represents the weighted feature dimension. Represents a dot product operation.

[0124] Finally, concatenate the three to make a prediction: The above formula represents the concatenation of MFCC features, spectrogram features and weighted deep speech features, and further processing through a function f (such as a fully connected layer, a nonlinear activation function, etc.) to obtain the final feature vector or matrix for prediction. .

[0125] The classification recognition network of the present invention includes a fully connected layer, which inputs the fused features into the collaborative attention mechanism, and then passes the accepted output through a fully connected layer through the cross entropy loss function and softmax to obtain the confidence of each class.

[0126] After processing by the feature fusion network, the resulting fused features are high-dimensional vectors that incorporate information from spectrogram features, Mel-Frequency Cepstral Coefficients (MFCCs), and weighted deep speech features. These fused features are then fed into the collaborative attention mechanism. The collaborative attention mechanism dynamically adjusts feature weights by calculating correlations between features to capture complementary information between them. Specifically, the collaborative attention mechanism generates an attention weight matrix, which is used to perform a weighted summation of the fused features to obtain an attention-weighted feature representation. The collaborative attention mechanism enhances the representation of important features while suppressing unimportant ones, thereby improving feature expressiveness. By capturing global relationships between features, the collaborative attention mechanism can better understand the context of the audio signal.

[0127] The features output by the collaborative attention mechanism are input into a fully connected layer. The function of a fully connected layer is to map high-dimensional features into a category space. Assuming there are C categories, the output of the fully connected layer is a C-dimensional vector, with each dimension corresponding to a category score (logits). The fully connected layer converts high-dimensional features into category scores, providing a foundation for subsequent classification tasks.

[0128] Apply the Softmax function to the C-dimensional vector output by the fully connected layer to convert the category score into a probability distribution. The softmax function is used to calculate the confidence of each class. The formula of the Softmax function is: in, is the output of the fully connected layer The scores of the categories, It is The probability of a category.

[0129] The output of the Softmax function is a probability distribution that represents the confidence level for each category. The Softmax function converts the category scores into probabilities, allowing the model to output the confidence level for each category. The probability distribution facilitates subsequent classification decisions and loss calculations.

[0130] The cross-entropy loss function is used to calculate the difference between the probability distribution predicted by the model and the true label. The calculation formula is as follows: in, The true label Class values are usually represented by one-hot encoding, that is, only the true class corresponds to is equal to 1, and the rest are 0. The model output is The probability of the class. Indicates the number of categories.

[0131] The cross-entropy loss function measures the difference between the probability distribution predicted by the model and the true distribution. Smaller values indicate more accurate model predictions. The cross-entropy loss function effectively guides model optimization, gradually bringing the model's predicted probability distribution closer to the true distribution. By minimizing cross-entropy loss, the model can improve classification accuracy.

[0132] During training, the backpropagation algorithm calculates the gradient of the cross-entropy loss function with respect to the model parameters, and an optimization algorithm (such as stochastic gradient descent (SGD) or Adam) is used to update the model parameters. Through multiple iterations of training, the model gradually learns how to extract useful information from the fused features and output accurate category confidence scores. This training process enables the model to gradually improve its classification performance, ultimately demonstrating good generalization on test data.

[0133] The present invention uses a collaborative attention mechanism to calculate the correlation between features and dynamically adjust the feature weights to capture the complementary information between different features. The model can effectively fuse spectrogram features, MFCC features, and deep speech features to form a comprehensive feature representation. The collaborative attention mechanism can dynamically adjust the feature weights to enhance the representation of important features. The features output by the collaborative attention mechanism are input into a fully connected layer. The fully connected layer and the Softmax function map the high-dimensional features to the category space and output the confidence score for each category to obtain the predicted category, such as laying sound, pain sound, and happy sound. The cross-entropy loss function can effectively guide model optimization and improve classification accuracy. This method is suitable for complex tasks such as laying hen sound recognition and provides strong support for classification tasks.

[0134] In addition to using the collaborative attention mechanism for feature fusion, the present invention can also use the multi-head attention mechanism to fuse the spectrogram features extracted by the improved ViT model with the MFCC features processed by BiLSTM. The multi-head attention mechanism assigns adaptive weights to features of different modalities. For example, high-frequency noise areas may rely more on the stable representation of MFCC, while harmonic structures may rely on local details of the spectrogram. By concatenating or summing the weighted features, a high-dimensional fusion vector is generated, i.e., the fused feature. Then, the output of wav2vec2 is used. The combined features are then input into the classification and recognition network for prediction to obtain the sound recognition results.

[0135] The multi-head attention formula is: = Where, is the query matrix, which represents the information that the current word or sentence wants to focus on; K is the key matrix, which represents the features of all words; V is the value matrix, which represents the final features after attention weighting. is the dimension of each attention head, Acts as a scaling factor to prevent gradients from vanishing or exploding.

[0136] Reference Figure 4 As shown in the figure, in the multi-head attention mechanism, Linear performs linear projection on the input features (spectrogram features and MFCC features) to generate Query (Q), Key (K), and Value (V). Query (Q) comes from the spectrogram features of ViT, while Key (K) and Value (V) come from the MFCC features. Self-Attention is used to calculate attention weights and weightedly fuse the Values. Concat is used to concatenate the outputs of multiple attention heads. Linear is used to perform the final projection of the concatenated features.

[0137] The above classification and recognition network in the present invention can also use XGBoost classifier. In this case, the sound recognition model architecture diagram is as follows: Figure 4 XGBoost combines a grid search algorithm to search for optimal hyperparameters related to model performance. The XGBoost parameters to be optimized include the number of weak learners N_estimators, the maximum tree depth max_depth, the number of leaf nodes num_leaves, the learning rate learning_rate, and the sample sampling ratio subsample. The optimized hyperparameters are determined based on the maximum F1 score. The optimization range and optimal hyperparameters for each parameter are shown in Table 3.

[0138] Table 3 The voice recognition model of the present invention operates as follows: a feature extraction network extracts features from the original audio signal to obtain spectrogram features and Mel-frequency cepstral coefficient features; a fine-tuned Wav2Vec2.0 model is used to classify the original audio signal to obtain predictions of deep speech features; a feature fusion network uses a multi-head attention mechanism to fuse the spectrogram features and Mel-frequency cepstral coefficient features to obtain fused features. The fused features are then combined with the predictions of the fine-tuned Wav2Vec2.0 model for the original audio signal (i.e., the deep speech features described above) and input into an optimization network, an XGBoost classifier, for optimization, ultimately achieving voice recognition of laying hens. This invention fully utilizes the advantages of various features to describe and analyze sound signals from multiple perspectives, thereby more accurately identifying the voiceprints of laying hens and significantly improving the accuracy of voiceprint recognition of laying hens.

[0139] In this embodiment, how to train relevant parameters of the sound recognition model will be specifically described.

[0140] A handheld audio information testing system was built to record audio data from laying hens using time-sharing and frequency conversion according to the rearing stage (brooding and laying). Preprocessing processes included A / D conversion, pre-emphasis, and resampling. During training, manual annotation was used to extract typical audio data, such as singing, laying, warning, and distress sounds from laying hens. This data was then compiled into a dataset of typical laying hen audio.

[0141] The convolution kernel length of the improved ViT model is set to 16, the pooling step size is set to 3, the dimension of BiLSTM is set to 128 dimensions, the number of layers is set to 1, and the relevant hyperparameters are set as described above. The attention heads are set to 8, the number of blocks is set to 2, the cross entropy loss function is used as the objective function for optimization, AdamW is used as the optimizer, the initial learning rate is set to 1e-5, the batch size is set to 32, the epoch is set to 100, the drop out rate is set to 0.5, and the learning rate is set to 0.001. After the model converges, the weights are saved in .pth format for later reasoning.

[0142] After model training is completed, diagnostic reasoning tests are performed. Initialize the pytorch model class and load the pre-trained model weights to perform forward propagation diagnostic prediction. In this embodiment, the egg-laying sounds, pain sounds, and singing sounds of laying hens are taken as examples. For these sounds, the acquisition module automatically collects audio and uploads it to the trained sound recognition model, which will automatically identify these types of sounds and make predictions. The specific embodiments are as follows: (1) Audio collection and pre-processing: The main purpose is to collect the egg-laying sounds, pain sounds, and singing sounds of laying hens in caged chicken houses. Pain sounds may be caused by external physical injuries to chickens, injuries caused by fighting with chickens, or when chickens are infected with various diseases. For example, when infected with infectious diseases such as avian influenza and Newcastle disease, there will be many uncomfortable symptoms such as fever, difficulty breathing, and physical weakness. In this painful state, chickens will frequently make abnormal sounds; or it may be due to the unsuitable ambient temperature in the breeding house. Focus on pain sounds. When frequent pain sounds occur, the breeder should pay more attention to the health status of the chickens and take timely adjustment measures.

[0143] (2) Upload the pre-processed laying hen audio to the sound recognition model, and the audio category will be predicted. For example, the audio is predicted to be a sound of pain.

[0144] Reference Figure 4 As shown in the figure, the preprocessed laying hen audio (Chicken sound) is input into the sound recognition model for acoustic feature extraction (Acoustic feature extraction) and deep learning feature extraction (Feature extraction in deep learning), and the extracted features are respectively input into the BiLSTM model (BiLSTM model), the improved ViT model (ViTmodel) and the Wav2vec2 model (Wav2vec2 model), and the features output by each model are fused. The fused features are input into the XGBoost classifier to obtain the predicted categories, such as laying sound (Laying sound), pain sound (Pain sound), and singing sound (Happy sound).

[0145] In order to illustrate the effect of the present invention, the performance of the present invention is compared with that of different acoustic features and the proposed feature fusion method on the constructed common dataset, as shown in Table 4.

[0146] Table 4 CQT features represent constant Q transform features, and Spectrogram features represent spectrogram features. Accuracy, Precision, Recall, and F1-score represent accuracy, precision, recall, and F1-score, respectively. It can be seen that compared to using a single feature or features in existing technologies (such as CQT features), the fusion of these features in this invention achieves more accurate classification and recognition of laying hen sounds.

[0147] The laying hen sound recognition system that integrates acoustic features and deep learning features provided by the present invention is described below. The laying hen sound recognition system that integrates acoustic features and deep learning features described below and the laying hen sound recognition method that integrates acoustic features and deep learning features described above can be referenced to each other.

[0148] The laying hen sound recognition system provided by the present invention integrates acoustic features and deep learning features, referring to Figure 5 Shown, including: The signal acquisition module 510 is used to obtain the original audio signal to be recognized and the sound recognition model; wherein the sound recognition model includes a feature extraction network, a feature fusion network and a classification recognition network; A feature extraction module 520 is configured to extract features from the original audio signal using the feature extraction network to obtain spectrogram features, Mel-frequency cepstral coefficient features, and deep speech features; A feature fusion module 530 is configured to fuse the spectrogram features, the Mel-frequency cepstral coefficient features, and the deep speech features using a collaborative attention mechanism or a multi-head attention mechanism in the feature fusion network to obtain fused features; The classification and recognition module 540 is used to input the fused features into the classification and recognition network to obtain a sound recognition result.

[0149] Figure 6 An example of a physical structure diagram of an electronic device is shown below. Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640. The processor 610, the communications interface 620, and the memory 630 communicate with each other via the communication bus 640. The processor 610 may call logic instructions in the memory 630 to execute a laying hen sound recognition method that integrates acoustic features and deep learning features.

[0150] Furthermore, the logic instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0151] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the laying hen sound recognition method that integrates acoustic features and deep learning features provided by the above methods.

[0152] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the laying hen sound recognition method that integrates acoustic features and deep learning features provided by the above methods.

[0153] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0154] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0155] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A laying hen sound recognition method integrating acoustic features and deep learning features, characterized in that: include: Obtaining an original audio signal to be recognized and a sound recognition model; wherein the sound recognition model includes a feature extraction network, a feature fusion network, and a classification recognition network; Extracting features from the original audio signal using the feature extraction network to obtain spectrogram features, Mel-frequency cepstral coefficient features, and deep speech features; The feature fusion network uses a collaborative attention mechanism or a multi-head attention mechanism to fuse the spectrogram features, the Mel-frequency cepstral coefficient features, and the deep speech features to obtain fused features; The fused features are input into the classification and recognition network to obtain a sound recognition result.

2. The laying hen sound recognition method integrating acoustic features and deep learning features according to claim 1 is characterized in that: The feature extraction network includes a Mel filter bank and an improved ViT model; the spectrogram is extracted by: Denoising the original audio signal to obtain a denoised audio signal; Performing frame windowing on the denoised audio signal to obtain a windowed audio signal; Obtain the linear spectrum of each frame for each frame of the windowed audio signal; Arrange the linear spectrum of each frame in time sequence to form a spectrum graph; The spectrogram is input into the improved ViT model to obtain spectrogram features.

3. The laying hen sound recognition method integrating acoustic features and deep learning features according to claim 2 is characterized in that: The improved ViT model replaces the standard global self-attention mechanism with a time-frequency dual-branch attention mechanism, and replaces the fully connected multi-layer perceptron in the ViT model with a convolutional feedforward network with time-frequency local perception; The spectrogram is input into the improved ViT model to obtain spectrogram features, including: Filling and normalizing the spectrum graph to obtain a processed spectrum graph; Scaling the processed spectrum graph, and performing a block operation on the scaled spectrum graph to obtain a plurality of spectrum blocks; Perform linear transformation on each spectrum block to obtain the corresponding spectrum block vector; Add spectrum embedding to the spectrum block vector to obtain a processed spectrum block vector; Spectrum position embedding is performed on the spectrum block vector, and each processed spectrum block vector is added to the spectrum position embedding vector of the corresponding position to obtain the spectrum graph feature.

4. The laying hen sound recognition method integrating acoustic features and deep learning features according to claim 1 is characterized in that: The feature extraction network includes a Mel filter bank and a bidirectional long short-term memory network; the Mel frequency cepstral coefficient feature is extracted by the following method: Performing high-pass filtering on the original audio signal to obtain a preprocessed audio signal; Dividing the preprocessed audio signal into multiple frames, and performing smoothing processing on each frame using a preset window function to obtain a smoothed audio signal; wherein each frame includes N sampling points; Perform Fourier transform on each frame of smoothed audio signal to convert it into frequency domain signal to obtain amplitude spectrum; Input the amplitude spectrum into a Mel filter bank, and take the logarithm of the energy value output by each Mel filter; Perform discrete cosine transform on the logarithmic energy values output by all Mel filters, and retain the first N coefficients as the original Mel frequency cepstral coefficient features; The original Mel-frequency cepstral coefficient feature is input into a bidirectional long short-term memory network to obtain a Mel-frequency cepstral coefficient feature.

5. The laying hen sound recognition method integrating acoustic features and deep learning features according to claim 1 is characterized in that: The feature extraction network includes a pre-trained deep learning model, which includes a feature encoder, a quantization module, and a context network; the deep speech features are extracted by: Preprocessing the original audio signal to obtain a preprocessed audio waveform; Converting the preprocessed audio waveform through a feature encoder to obtain a speech feature vector; Converting the speech feature vector into discrete speech units using a quantization module to obtain a quantized feature vector; Randomly masking some of the quantized feature vectors to obtain masked feature vectors; The masked feature vector is input into a context network including a Transformer encoder for processing to obtain the deep speech feature.

6. The laying hen sound recognition method integrating acoustic features and deep learning features according to claim 1 is characterized in that: The feature fusion network utilizes a collaborative attention mechanism to fuse the spectrogram features, the Mel-frequency cepstral coefficient features, and the deep speech features, including: Fusing the Mel-frequency cepstral coefficient feature and the spectrogram feature as the attention weight of each frame in the deep speech feature; Weighting the features of the corresponding frame in the deep speech feature according to the attention weight of each frame to obtain a weighted deep speech feature; The mel-frequency cepstral coefficient feature, the spectrogram feature and the weighted deep speech feature are concatenated to obtain the fused feature.

7. A laying hen sound recognition system integrating acoustic features and deep learning features, characterized in that: include: A signal acquisition module, configured to acquire an original audio signal to be recognized and a sound recognition model; wherein the sound recognition model includes a feature extraction network, a feature fusion network, and a classification recognition network; A feature extraction module is used to extract features from the original audio signal using the feature extraction network to obtain spectrogram features, Mel-frequency cepstral coefficient features and deep speech features; A feature fusion module is used for the feature fusion network to fuse the spectrogram features, the Mel-frequency cepstral coefficient features and the deep speech features using a collaborative attention mechanism or a multi-head attention mechanism to obtain fused features; The classification and recognition module is used to input the fused features into the classification and recognition network to obtain a sound recognition result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the laying hen sound recognition method that integrates acoustic features and deep learning features as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the laying hen sound recognition method that integrates acoustic features and deep learning features as described in any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the laying hen sound recognition method that integrates acoustic features and deep learning features as described in any one of claims 1 to 6 is implemented.

Images