Chicken sound recognition method under heat stress condition based on multi-domain feature fusion

Abstract

The invention discloses a chicken sound recognition method under a heat stress condition based on multi-domain feature fusion, and the method employs a chicken sound recognition model CVC-Fusion based on multi-domain feature fusion, and is used for recognizing the sound of a chicken flock under the heat stress condition and a normal condition. The CVC-Fusion model adopts a double-encoder architecture and comprises a frequency domain path and a time-frequency domain path. The frequency domain path performs modeling on the frequency domain features of the chick sound by using a module based on an inverse residual structure MV2, and the time-frequency domain path is integrated with a hybrid feature selection module HFS and is used for local and global characterization of a dynamic weighted time-frequency graph. A double-path feature fusion module at the tail end of the model is adopted for information exchange and complementation between frequency domain features and time-frequency domain features, and chicken sound signal recognition under the heat stress condition and the normal condition is achieved. The invention provides an efficient and reliable solution for automatic monitoring of poultry heat stress.

Description

Technical Field

[0001] The present invention relates to the field of chicken sound recognition methods, and in particular to a chicken call recognition method under heat stress conditions based on multi-domain feature fusion, which is used to identify whether a chicken flock is in a heat stress state. Background Art

[0002] As an important part of the poultry farming industry, broiler farming plays a key role in ensuring the supply of poultry meat and promoting the development of the agricultural economy. As the global average temperature rises year by year, the trend of global warming intensifies, and extreme high temperature events occur frequently, the threat of heat stress in chicken flocks to the farming industry becomes more and more serious. At present, China mainly adopts semi-open chicken houses combined with ground flat farming to raise broilers. Although this farming model has the characteristics of high breeding density and low cost, it lacks reasonable and efficient environmental control measures, resulting in uneven temperature distribution in the chicken house during high temperature weather in summer. The local temperature is higher than the suitable growth temperature of broilers, and the broilers themselves have a fast metabolism and generate a lot of heat. These factors will cause heat stress in broilers, which is not conducive to ensuring the production performance and welfare level of the flock. At present, the judgment of heat stress in chicken flocks mainly relies on manual observation, which is highly subjective, labor-intensive and inefficient. With the aging of the population and the increase in labor costs, an automated and intelligent method is needed to replace manual observation to meet the development needs of the modern poultry farming industry.

[0003] The sounds of broilers contain a variety of information reflecting their health, emotions and behavior. Acoustic analysis technology has the advantages of high efficiency, no stress and low cost, and has great potential in identifying the vocalizations of broilers under heat stress. Through acoustic analysis, we can gain a deep understanding of the causes of specific vocalizations of chickens, such as repetitive, high-energy and relatively loud calls made by chicks when they are in pain.

[0004] In recent years, there have been many studies on livestock and poultry vocalization analysis, mainly focusing on livestock and poultry stress identification, respiratory disease detection, etc. In livestock and poultry stress identification, the study found that the hen's call is significantly correlated with the environmental temperature and humidity index. The number of turkeys under heat stress is significantly higher than the number of turkeys under normal conditions. The decision tree algorithm combined with the time domain and frequency domain characteristics of pig calls can identify piglet stress calls. In the detection of respiratory diseases, the use of data mining methods to extract effective information in the time domain and frequency domain of livestock and poultry sound signals can realize the recognition of coughs or sneezes caused by diseases such as Newcastle disease, bronchitis and avian influenza.

[0005] However, the machine learning methods used in these studies usually rely on manually extracted features, and there is a lot of room for improvement in indicators such as the accuracy and precision of livestock and poultry sound recognition models. In contrast, deep learning methods have been proven to have broad application potential in the field of bioacoustic detection due to their advantages in end-to-end modeling. The input of deep learning models is usually 1D frequency domain data or 2D spectrograms of livestock and poultry sounds. The high-dimensional representation of the sound signal is extracted through convolution operations or multi-head self-attention mechanisms. After processing by classification heads, the final livestock and poultry stress recognition or respiratory disease detection is achieved. These studies have shown good recognition performance in specific tasks by extracting features from time domain and frequency domain data. However, the potential complementary relationship between multi-domain features of sound signals has not been fully explored, which limits the model's ability to enhance feature representation and effectively distinguish different sound categories. In addition, although existing studies have involved various stress types of livestock and poultry calls, there is still a lack of research on livestock and poultry calls under heat stress conditions, which hinders the research and application of acoustic technology in the direction of livestock and poultry heat stress recognition. Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention proposes a method for recognizing chicken calls under heat stress conditions based on multi-domain feature fusion, which is used for recognizing chicken calls under heat stress conditions and under normal conditions, so as to determine whether the chickens are in a state of heat stress.

[0007] A method for recognizing chicken calls under heat stress conditions based on multi-domain feature fusion, comprising the following steps:

[0008] 1) Obtaining continuous audio data of chicken house sounds and extracting chicken call segments, including chicken call segments under heat stress conditions and chicken calls under normal conditions;

[0009] 2) preprocessing the chicken call sound segments obtained in step 1) to form a chicken call sound recognition data set;

[0010] 3) Replace the MobileViTv3 block in the MobileViTv3 model with a hybrid feature selection module to obtain an improved MobileViTv3 model. The improved MobileViTv3 model is a time-frequency domain path, denoted as the CVC-2D path;

[0011] 4) Based on the 1D convolutional layer combined with the stacking of two layers of MobilenetV2 block, a one-dimensional convolutional neural network model is constructed. The one-dimensional convolutional neural network model is a frequency domain path, denoted as CVC-1D path;

[0012] 5) fusing the CVC-2D path obtained in step 3) and the CVC-1D path obtained in step 4) to obtain a multi-domain feature fusion model, and using the chicken call recognition data set obtained in step 2) to train and optimize the multi-domain feature fusion model to obtain a multi-domain feature fusion model with reasoning ability;

[0013] 6) A multi-domain feature fusion model with reasoning ability is used to identify the chicken calls that need to be identified, obtain the chicken call category, and determine whether the chickens are in a state of heat stress.

[0014] In step 1), the chicken calls under heat stress conditions refer to the chicken calls at an ambient temperature of 30 to 45°C, and the chicken calls under normal conditions refer to the chicken calls when the chicken enters the weaning period and is at an ambient temperature of 21 to 25°C.

[0015] In step 2), the chicken call sound segment obtained in step 1) is preprocessed, specifically comprising:

[0016] The chicken call sound segments are framed, windowed, and short-time Fourier transformed, and then converted into frequency domain data and time-frequency domain data, and the data of 1000-5500Hz is retained to form a chicken call recognition data set.

[0017] In step 3), the hybrid feature selection module includes: a local representation module, a global representation module and a hybrid representation module connected in sequence;

[0018] The hybrid characterization module comprises:

[0019] a first global average pooling layer connected to the global representation module;

[0020] a second global average pooling layer connected to the local representation module;

[0021] A shared fully connected layer connected to the output ends of the first global average pooling layer and the second global average pooling layer;

[0022] A convolutional layer connected to the output of the shared fully connected layer.

[0023] In step 5), the chicken call recognition dataset obtained in step 2) is used to train and optimize the multi-domain feature fusion model, specifically including:

[0024] The time-frequency domain data in the chicken call recognition dataset is input into the CVC-2D path of the multi-domain feature fusion model, and the local representation module outputs the local representation F local , the global representation module outputs the global representation F global , the local representation F local Input to the second global average pooling layer, the global representation F globalInput to the first global average pooling layer, and output weight W through the shared fully connected layer local and weight W global , through the weight W local , local representation F local , weight W global , global representation F global Calculate the mixed feature F mix , the mixed feature F mix The input is input into the convolution layer, and the output of the convolution layer is added to the input of the local representation module and output after the addition.

[0025] In step 5), the mixed feature F is calculated by the following formula: mix ;

[0026] F mix =W local F local +W global F global .

[0027] In step 5), the CVC-2D path obtained in step 3) and the CVC-1D path obtained in step 4) are fused to obtain a multi-domain feature fusion model, which specifically includes:

[0028] 5.1) The global representation of the frequency domain features output by the CVC-1D path is transformed through a linear layer to be consistent with the high-order representation dimension of the time-frequency domain features output by the CVC-2D path;

[0029] 5.2) Send the global representation of frequency domain features after linear layer transformation into the bidirectional linear cross attention module Q 1D The branch is linearly transformed and globally averaged pooled to obtain feature F Q ;

[0030] The high-order representation of time-frequency domain features is fed into the bidirectional linear cross attention module K 2D The branch undergoes linear transformation to obtain feature F k ;

[0031] The high-order representation of time-frequency domain features is fed into the bidirectional linear cross attention module V 2D The branch undergoes linear transformation and passes through the Relu activation function to obtain the feature F v ;

[0032] Will come from Q 1D The characteristic F of the branch Q The transpose of 2D The characteristic F of the branch k Multiply and pass through the Softmax function to get the attention score W 1D ;

[0033] The attention score W 1D With V 2D The characteristic F of the branch v Perform Hadamard product operation to obtain the time-frequency domain feature F after weighting based on the frequency domain feature spec ;

[0034] Send the time-frequency domain features into Q 2D Branch, sending the frequency domain features into the bidirectional linear cross attention module K 1D Branch and V 1D The branch performs calculations to obtain the frequency domain feature F based on the weighted time-frequency domain features. freq ;

[0035] The time-frequency domain features F based on the weighted frequency domain features spec and the frequency domain features F after weighting based on the time-frequency domain features freq Add them together to get the feature F after fusion based on the bidirectional linear attention module att , and then undergo a linear transformation to form a multi-domain feature fusion model.

[0036] The dual-path feature fusion module realizes the fusion and interaction of the frequency domain information and the time-frequency domain information of the chicken call signal. Compared with the square-level computational complexity of the traditional self-attention mechanism, it has the advantages of fast calculation and high efficiency.

[0037] Further preferably, the key to a method for identifying chicken calls under heat stress conditions based on multi-domain feature fusion is a model CVC-Fusion (Chicken Vocalizations Classification-Fusion) that integrates frequency domain and time-frequency domain features, which mainly includes the following steps:

[0038] The CVC-Fusion model adopts a dual-tower encoder architecture and simultaneously models the one-dimensional frequency domain information and two-dimensional time-frequency domain information of chicken call signals through two parallel feature extraction paths. It makes full use of the local feature extraction advantages of the convolution operator and the long-term modeling ability of the self-attention mechanism to deeply mine the frequency domain characteristics and spectral characteristics of chicken call signals. The dual-path feature fusion module at the end of the model is used to exchange and complement information between frequency domain features and time-frequency domain features, thereby realizing the recognition of chicken call signals under heat stress conditions and normal conditions.

[0039] Specifically, the two feature extraction paths in CVC-Fusion are the time-frequency domain path and the frequency domain path. For the time-frequency domain path, that is, the CVC-2D path, the present invention improves the MobileViTv3 model by replacing the MobileViTv3 block in the original model with the HSF block to enhance the modeling capability of the time-frequency graph features. CVC-2D completes the hierarchical construction of the input time-frequency graph features by stacking multiple MV2 (MobilenetV2) modules and HFS (Hybrid Feature Select) blocks, and compresses the feature map dimension to 256×1 through the global pooling layer to form a high-order representation of the time-frequency graph. For the frequency domain path, i.e., the CVC-1D path, given that the duration of the chicken call segment is short, the data dimension obtained after converting the signal to the frequency domain space is low. In order to reduce the risk of model overfitting and improve the effectiveness of feature extraction, the present invention designs this path as a shallow 1D convolutional feature extraction network, which performs a 1×3 convolution operation on the input data of length L to obtain a feature map with a shape of 16×L. Subsequently, features are further extracted through two consecutive layers of MV2 modules, and a feature map with a shape of 32×L / 2 is output. The number of channels of the feature map is upsampled to 64 through a 1×1 convolution, and then the feature map dimension is compressed to 64×1 through a pooling layer to form a global representation of the frequency domain features. For the dual-path feature fusion module at the end of the CVC-Fusion model, the spectrogram image representation received from the CVC-2D-HFS path and the frequency domain feature representation received from the CVC-1D path are respectively aligned in dimension through a linear layer, and then an operation based on the bidirectional linear cross-attention mechanism is performed to perform cross-domain fusion of dual-path features. Finally, a linear layer and a Softmax function are used to complete the chicken call signal classification task.

[0040] The CVC-Fusion model includes three main modules: MV2 module, HFS block and dual-path feature fusion module.

[0041] The MV2 module adopts an inverse residual structure. It first expands the channel dimension through 1×1 convolution to increase the feature expression ability, then extracts features in high-dimensional space through 3×3 depth-separable convolution, and finally compresses the features back to low-dimensional space through 1×1 convolution. For the MV2 module with a step size of 1, a jump connection is used to directly add the input and output, retaining the original information to enhance the stability of gradient propagation. For the MV2 module with a step size of 2, the output is directly output to achieve the purpose of feature map downsampling. The MV2 module has the advantages of low computational complexity and small number of parameters, and is suitable for efficient extraction of different levels of features of chicken call signals.

[0042] HFS block is an improvement on MobileViTv3 block. It replaces Fusion block in MobileViTv3 block with hybrid feature selection module to dynamically weight local representation and global representation, further enhancing feature expression capability. Its principle is as follows: HFS block receives C from the previous layer. in ×H in ×W in The feature map of size C is generated by extracting features through 3×3 depth-separable convolution and 1×1 convolution. local The local representation F of ×H0×W0 local (Local representation), the local representation F local Input linear transformer encoder, which consists of separable self-attention (SSA) and feed-forward network to model long-range dependencies between features, generating a size C global The global representation F of ×H0×W0 global (Global representation). local and the global representation F global Perform a global average pooling operation to obtain a size of C local ×1×1 feature vector V local and size C global ×1×1 feature vector V global . V local and V global The weight W of each channel is learned through a shared fully connected layer local and W global , and normalized by the Softmax function to ensure that the sum of the weights is 1. local and W global For the local representation F local and the global representation F global Perform weighted summation to obtain a value of C w ×H0×W0 mixed feature F mix , the calculation method is shown in formula (1). Finally, a 1×1 convolution is performed to adjust the channel dimension and generate a size of C out ×H out ×W out The output feature map of .

[0043] The dual-path feature fusion module is designed based on a bidirectional linear cross-attention mechanism. First, the frequency domain global representation obtained by the frequency domain path is dimensionally aligned with the time-frequency domain representation obtained by the time-frequency domain path through a linear layer transformation, and then the frequency domain feature global representation is sent to Q1D The branch is linearly transformed and globally averaged pooled to obtain a feature F of size k×1. Q . Send the high-order representation of time-frequency domain features into K 2D The branch undergoes linear transformation to obtain a feature F of size k×d k . Send the high-order representation of time-frequency domain features into V 2D The branch undergoes linear transformation and passes through the Relu activation function to obtain a feature F of size k×d. v . Will come from Q 1D The characteristic F of the branch Q The transpose of 2D The characteristic F of the branch k Multiply and pass through the Softmax function to get the attention score W of size 1×d 1D , attention score W 1D With V 2D The characteristic F of the branch v Perform Hadamard product operation to obtain the time-frequency domain feature F of size k×d after weighting based on the frequency domain feature. spec Similarly, the time-frequency domain features are fed into Q 2D Branch, send the frequency domain features into K 1D Branch and V 1D The branch performs the above operation to obtain the frequency domain feature F of size k×d after weighting based on the time-frequency domain features. freq The k×d time-frequency domain features weighted by the frequency domain features and the k×d frequency domain features weighted by the time-frequency domain features are added to obtain the feature F after fusion based on the bidirectional linear attention module. att , and then a linear transformation is performed to form the final multi-domain feature fusion model. The dual-path feature fusion module realizes the fusion and interaction of the frequency domain information and the time-frequency domain information of the chicken call signal. Compared with the square-level computational complexity of the traditional self-attention mechanism, it has the advantages of fast operation and high efficiency.

[0044] Compared with the prior art, the present invention has the following advantages:

[0045] The present invention proposes a multi-domain feature fusion model CVC-Fusion, which simultaneously models the frequency domain information and time-frequency domain information of chicken call signals through a dual-tower encoder architecture, fully utilizing the local feature extraction capability of the convolution operator and the long-range modeling capability of the self-attention mechanism, and realizing the deep fusion and complementarity between the frequency domain characteristics and the spectral characteristics. The improved HFS block is used to improve the feature modeling capability of the time-frequency domain path, and the expression capability of the model is enhanced by the local and global representation of the dynamic weighted chicken call time-frequency graph. The proposed dual-path feature fusion module based on the bidirectional linear cross-attention mechanism realizes the efficient interaction of the frequency domain and time-frequency domain features of the chicken call signal, significantly reduces the computational complexity compared to the traditional self-attention mechanism, and has a faster computing speed and higher computing efficiency. The chicken call recognition method under heat stress conditions based on multi-domain feature fusion proposed in the present invention has high accuracy and stability, and provides a new idea for the automated monitoring of poultry heat stress. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is the overall architecture diagram of the chicken call recognition model CVC-Fusion in a specific implementation manner of the present invention;

[0047] Figure 2 It is a diagram of the MV2block architecture of the chicken call recognition model CVC-Fusion in a specific embodiment of the present invention;

[0048] Figure 3 It is a HFS block architecture diagram of the chicken call recognition model CVC-Fusion in a specific implementation manner of the present invention;

[0049] Figure 4 It is a dual path feature fusion module architecture diagram of the chicken call recognition model CVC-Fusion in a specific embodiment of the present invention;

[0050] Figure 5 is a feature visualization diagram of the CVC-Fusion model in a specific embodiment of the present invention, Figure 5 (a) and Figure 5 (b) are the time-frequency diagrams of chicken calls after preprocessing under heat stress and normal conditions, Figure 5 (c) and Figure 5 (d) are the time-frequency domain heat maps of chicken calls under heat stress and normal conditions with superimposed weight information, Figure 5 (e) and Figure 5 (f) are frequency domain curves of chicken calls under heat stress conditions and normal conditions with weight information superimposed. DETAILED DESCRIPTION

[0051] The specific implementation modes of the present invention are described in detail below with reference to the accompanying drawings.

[0052] A method for recognizing chicken calls under heat stress conditions based on multi-domain feature fusion, comprising the following steps:

[0053] 1) Obtaining continuous audio data of chicken house sounds, and extracting chicken call segments therein, including chicken call segments under heat stress conditions and chicken call segments under normal conditions;

[0054] 2) preprocessing the chicken call sound segments obtained in step 1) to form a chicken call sound recognition data set;

[0055] 3) Replace the MobileViTv3 block in the MobileViTv3 model with a hybrid feature selection block (HFS block) to obtain an improved MobileViTv3 model to improve the modeling capability of the time-frequency domain features in the chicken call recognition dataset in step 2). The improved MobileViTv3 model is a time-frequency domain path, denoted as CVC-2D path;

[0056] 4) Based on the 1D convolutional layer combined with the stacking of two layers of MV2MobilenetV2 (MV2) blocks, a one-dimensional convolutional neural network model is constructed to extract the frequency domain features in the chicken call recognition dataset in step 2). The one-dimensional convolutional neural network model is a frequency domain path, denoted as CVC-1D path;

[0057] 5) The time-frequency domain representation of the CVC-2D path in step 3) and the frequency domain representation of the CVC-1D path in step 4) are fused through a dual-path feature fusion module to obtain a multi-domain feature fusion model CVC-Fusion. The chicken call recognition data set obtained in step 2) is used to train and optimize the multi-domain feature fusion model to obtain a multi-domain feature fusion model with reasoning ability;

[0058] 6) A multi-domain feature fusion model with reasoning ability is used to identify the chicken calls that need to be identified, obtain the chicken call category, and determine whether the chickens are in a state of heat stress.

[0059] In step 1), the chicken calls under heat stress conditions refer to chicken calls at an ambient temperature of 30-45°C, and the chicken calls under normal conditions refer to chicken calls when the chicken enters the weaning period and is at an ambient temperature of 21-25°C.

[0060] In step 2), the chicken call sound segment obtained in step 1) is preprocessed, specifically comprising:

[0061] After framing, windowing and short-time Fourier transform, the chicken call sound fragments are converted into frequency domain data and time-frequency domain data, and the data of 1000-5500Hz is retained to form a chicken call recognition data set.

[0062] In step 3), the hybrid feature selection module HFS block includes a local characterization module, a global characterization module and a hybrid characterization module connected in sequence:

[0063] HFS block receives the size of C from the upper layer in ×H in ×W in The feature map is sent to the local representation module, and features are extracted through 3×3 depth-separable convolution and 1×1 convolution to generate a size of C local The local representation F of ×H0×W0 local (Local representation);

[0064] The local representation F local The global representation module is composed of separable self-attention (SSA) and feedforward network, which is used to model the long-range dependency between features and generates a size of C. global The global representation C of ×H0×W0 global (Global representation);

[0065] The local representation F local and the global representation C global Send to the hybrid characterization module to respectively local and F global Perform a global average pooling operation to obtain a size of C local ×1×1 feature vector V local and size C global ×1×1 feature vector V global ;

[0066] V local and V global The weight W of each channel is learned through a shared fully connected layer local and W global , and normalized by the Softmax function to ensure that the sum of the weights is 1;

[0067] Using W local and W global For the local representation F local and the global representation F global Perform weighted summation to obtain a value of C w ×H0×W0 mixed feature Fmix , the calculation method is shown in the following formula;

[0068] Finally, a 1×1 convolutional layer is used to adjust the channel dimension and generate a size of C out ×H out ×W out The output feature map of the convolutional layer is connected to the input residual of the local representation module.

[0069] F mix =W local F local +W global F global

[0070] In step 3), the modeling process of the CVC-2D path includes:

[0071] The hierarchical construction of the time-frequency graph features is completed by stacking multiple MV2 modules and HSF blocks;

[0072] The feature map dimension is compressed to 256×1 through the global pooling layer to form a high-order representation of the time-frequency domain features.

[0073] In step 4), the modeling process of the CVC-1D path includes:

[0074] By performing a 1×3 convolution operation on the input data of length L, a feature map of shape 16×L is obtained;

[0075] After two consecutive layers of MV2 modules, features are further extracted and the output feature map is 32×L / 2;

[0076] The number of channels of the feature map is upsampled to 64 through a 1×1 convolution, and then the feature map dimension is compressed to 64×1 through a pooling layer to form a global representation of the frequency domain features.

[0077] In step 5), the dual-path feature fusion module is designed based on a bidirectional linear cross attention mechanism, including:

[0078] The global representation of the frequency domain features of size 64×1 described in step 4) is transformed through a linear layer to obtain a global representation of the frequency domain features of size 256×1, so that the dimension is consistent with the high-order representation of the time-frequency domain features of size 256×1 described in step 3);

[0079] Send the global representation of frequency domain features into Q 1D The branch is linearly transformed and globally averaged pooled to obtain a feature F of size k×1. Q ;

[0080] The high-order representation of time-frequency domain features is fed into K 2DThe branch undergoes linear transformation to obtain a feature F of size k×d k ;

[0081] Send the high-order representation of time-frequency domain features into V 2D The branch undergoes linear transformation and passes through the Relu activation function to obtain a feature F of size k×d. v ;

[0082] Will come from Q 1D The transpose of the features of the branch of size k×1 is combined with the features from K 2D The features of the branch size of k×d are multiplied and passed through the Softmax function to obtain the attention score W of size 1×d. 1D ;

[0083] Combine the attention scores of size 1×d with the 2D The branch size is k*d. The Hadamard product operation is performed to obtain the time-frequency domain feature F with a size of k×d after weighting based on the frequency domain feature. spec ;

[0084] Similarly, the time-frequency domain features are sent to Q 2D Branch, send the frequency domain features into K 1D Branch and V 1D The branch performs the above operation to obtain the frequency domain feature F of size k×d after weighting based on the time-frequency domain features. freq ;

[0085] The time-frequency domain features F based on the weighted frequency domain features spec and the frequency domain features F after weighting based on the time-frequency domain features freq Add them together to get the feature F after fusion based on the bidirectional linear attention module att , and then undergo a linear transformation to form the final multi-domain feature fusion model.

[0086] A method for recognizing chicken calls under heat stress conditions based on multi-domain feature fusion is proposed. The key is a model CVC-Fusion that integrates frequency domain and time-frequency domain features. Figure 1 The CVC-Fusion model adopts a dual-tower encoder architecture, and simultaneously models the one-dimensional frequency domain information and two-dimensional time-frequency domain information of the chicken call signal through two parallel feature extraction paths. It fully utilizes the local feature extraction advantages of the convolution operator and the long-term modeling ability of the self-attention mechanism to deeply mine the frequency domain characteristics and spectral characteristics of the chicken call signal, and uses the dual-path feature fusion module at the end of the model to exchange and complement the information between the frequency domain features and the time-frequency domain features, so as to realize the recognition of chicken call signals under heat stress conditions and normal conditions.

[0087] Specifically, the two feature extraction paths in CVC-Fusion are the time-frequency domain path and the frequency domain path. For the time-frequency domain path, that is, the CVC-2D path, the present invention improves the MobileViTv3 model by replacing the MobileViTv3 block in the original model with the HSF block to enhance the modeling capability of the time-frequency graph features. CVC-2D completes the hierarchical construction of the input time-frequency graph features by stacking multiple MV2 modules and HFS blocks, and compresses the feature map dimension to 256×1 through the global pooling layer to form a high-order representation of the time-frequency graph. For the frequency domain path, i.e., the CVC-1D path, given that the duration of the chicken call segment is short, the data dimension obtained after converting the signal to the frequency domain space is low. In order to reduce the risk of model overfitting and improve the effectiveness of feature extraction, the present invention designs this path as a shallow 1D convolutional feature extraction network, which performs a 1×3 convolution operation on the input data of length L to obtain a feature map with a shape of 16×L. Subsequently, features are further extracted through two consecutive layers of MV2 modules, and a feature map with a shape of 32×L / 2 is output. The number of channels of the feature map is upsampled to 64 through a 1×1 convolution, and then the feature map dimension is compressed to 64×1 through a pooling layer to form a global representation of the frequency domain features. For the dual-path feature fusion module at the end of the CVC-Fusion model, the spectrogram image representation received from the CVC-2D-HFS path and the frequency domain feature representation received from the CVC-1D path are respectively aligned in dimension through a linear layer, and then an operation based on the bidirectional linear cross-attention mechanism is performed to perform cross-domain fusion of dual-path features. Finally, a linear layer and a Softmax function are used to complete the chicken call signal classification task.

[0088] The CVC-Fusion model contains three main modules: MV2 module, HFS block and dual-path feature fusion module.

[0089] MV2 module architecture is as follows Figure 2 As shown in the figure, the module adopts an inverse residual structure. First, the channel dimension is expanded through 1×1 convolution to increase the feature expression ability, then the features are extracted in high-dimensional space through 3×3 depth-separable convolution, and finally the features are compressed back to low-dimensional space through 1×1 convolution. For the MV2 module with a step size of 1, a jump connection is used to directly add the input and output, retaining the original information to enhance the stability of gradient propagation. For the MV2 module with a step size of 2, the output is directly output to achieve the purpose of feature map downsampling. The MV2 module has the advantages of low computational complexity and small number of parameters, and is suitable for efficient extraction of different levels of features of chicken call signals.

[0090] HFS block architecture is as follows Figure 3As shown in Figure 1, HFS block is an improvement on MobileViTv3 block. It replaces Fusion block in MobileViTv3 block with hybrid feature selection module to dynamically weight local representation and global representation, further enhancing feature expression capability. The principle is as follows: HFS block receives C from the previous layer. in ×H in ×W in The feature map of size C is generated by extracting features through 3×3 depth-separable convolution and 1×1 convolution. local The local representation F of ×H0×W0 local (Local representation), the local representation F local Input linear transformer encoder, which consists of separable self-attention (SSA) and feed-forward network to model long-range dependencies between features, generating a size C global The global representation F of ×H0×W0 global (Globalrepresentation). local and the global representation Fg local Perform a global average pooling operation to obtain a size of C local ×1×1 feature vector V local and size C global ×1×1 feature vector V global . V local and V global The weight W of each channel is learned through a shared fully connected layer local and W global , and normalized by the Softmax function to ensure that the sum of the weights is 1. local and W global For the local representation F local and the global representation F global Perform weighted summation to obtain a value of C w ×H0×W0 mixed feature F mix , the calculation method is shown in formula (1). Finally, a 1×1 convolution is performed to adjust the channel dimension and generate a size of C out ×H out ×W out The output feature map of .

[0091] The dual-path feature fusion module is designed based on the bidirectional linear cross-attention mechanism, such as Figure 4First, the frequency domain global representation obtained by the frequency domain path is dimensionally aligned with the time-frequency domain representation obtained by the time-frequency domain path through a linear layer transformation, and then the frequency domain feature global representation is sent to Q 1D The branch is linearly transformed and globally averaged pooled to obtain a feature F of size k×1. Q . Send the high-order representation of time-frequency domain features into K 2D The branch undergoes linear transformation to obtain a feature F of size k×d k . Send the high-order representation of time-frequency domain features into V 2D The branch undergoes linear transformation and passes through the Relu activation function to obtain a feature F of size k×d. v . Will come from Q 1D The characteristic F of the branch Q The transpose of 2D The characteristic F of the branch k Multiply and pass through the Softmax function to get the attention score W of size 1×d 1D , attention score W 1D With V 2D The characteristic F of the branch v Perform Hadamard product operation to obtain the time-frequency domain feature F of size k×d after weighting based on the frequency domain feature. spec Similarly, the time-frequency domain features are fed into Q 2D Branch, send the frequency domain features into K 1D Branch and V 1D The branch performs the above operation to obtain the frequency domain feature F of size k×d after weighting based on the time-frequency domain features. freq The k×d time-frequency domain features weighted by the frequency domain features and the k×d frequency domain features weighted by the time-frequency domain features are added to obtain the feature F after fusion based on the bidirectional linear attention module. att , and then a linear transformation is performed to form the final multi-domain feature fusion model. The dual-path feature fusion module realizes the fusion and interaction of the frequency domain information and the time-frequency domain information of the chicken call signal. Compared with the square-level computational complexity of the traditional self-attention mechanism, it has the advantages of fast operation and high efficiency.

[0092] The present invention uses Grad-CAM technology to visualize and analyze the feature extraction process of the dual paths of the CVC-Fusion model. Figure 5 shown. Figure 5 (a) and Figure 5 (b) The images of chicken call samples under heat stress and normal conditions before input into the CVC-2D path of the CVC-Fusion model and after preprocessing, respectively. Figure 5 (c) and Figure 5(d) are heat maps generated by visualizing the output features of the last layer feature extraction module of the CVC-2D path. The high-attention areas in the heat map are highly consistent with the high-energy areas of the original time-frequency graph, which shows that the model can effectively capture the energy characteristics of the key frequency bands of the chicken call signal in the CVC-2D path, especially the energy-concentrated areas. Figure 5 (e) and Figure 5 (f) are the weight curves generated after visualizing the output features of the last layer feature extraction module of the CVC-1D path, where the curve is the frequency domain curve of the original sample and the solid points on the curve are the weights of the frequency domain data. Figure 5 In (e), except for the first three data points on the left end of the curve, which are assigned relatively high weight values, the weights are evenly distributed in the rest of the curve, which reflects that the CVC-1D pathway focuses on the global frequency domain characteristics of chicken calls under heat stress conditions. Figure 5 In (f), the CVC-1D path assigns higher weights to the data points at both ends of the curve (frames 4-12 and frames 27-41), while paying relatively less attention to the frequency domain features of the middle peak segment (corresponding to frames 13-26). However, it is worth noting that the middle peak segment (corresponding to frames 13-26) is exactly the energy-concentrated segment in the time-frequency diagram. This shows that for the chicken call signal under normal conditions, the features extracted by the CVC-2D path and the CVC-1D path form a good information complement: the CVC-2D path is responsible for focusing on the energy-concentrated area in the frequency domain, and the CVC-1D path is responsible for focusing on the relatively low amplitude area in the frequency domain.

[0093] The visualization results show that the CVC-Fusion model can effectively extract key features in the frequency domain and time-frequency domain through the synergy of dual paths, providing reliable decision support for the chicken call recognition task. This result further verifies the rationality of the model design and its adaptability to input signals, and demonstrates the interpretability and application potential of the CVC-Fusion model.

Claims

1. A method for recognizing chicken calls under heat stress conditions based on multi-domain feature fusion, characterized in that: The following steps are involved: 1) Obtaining continuous audio data of chicken house sounds and extracting chicken call segments, including chicken call segments under heat stress conditions and chicken calls under normal conditions; 2) preprocessing the chicken call sound segments obtained in step 1) to form a chicken call sound recognition data set; 3) Replace the MobileViTv3 block in the MobileViTv3 model with a hybrid feature selection module to obtain an improved MobileViTv3 model. The improved MobileViTv3 model is a time-frequency domain path, denoted as the CVC-2D path; 4) Based on the 1D convolutional layer combined with the stacking of two layers of MobilenetV2 block, a one-dimensional convolutional neural network model is constructed. The one-dimensional convolutional neural network model is a frequency domain path, denoted as CVC-1D path; 5) fusing the CVC-2D path obtained in step 3) and the CVC-1D path obtained in step 4) to obtain a multi-domain feature fusion model, and using the chicken call recognition data set obtained in step 2) to train and optimize the multi-domain feature fusion model to obtain a multi-domain feature fusion model with reasoning ability; 6) A multi-domain feature fusion model with reasoning ability is used to identify the chicken calls that need to be identified, obtain the chicken call category, and determine whether the chickens are in a state of heat stress.

2. The chicken call recognition method under heat stress conditions based on multi-domain feature fusion according to claim 1 is characterized in that: In step 1), the chicken calls under heat stress conditions refer to the chicken calls at an ambient temperature of 30 to 45°C, and the chicken calls under normal conditions refer to the chicken calls when the chicken enters the weaning period and is at an ambient temperature of 21 to 25°C.

3. The chicken call recognition method under heat stress conditions based on multi-domain feature fusion according to claim 1 is characterized in that: In step 2), the chicken call sound segment obtained in step 1) is preprocessed, specifically comprising: The chicken call sound segments are framed, windowed, and short-time Fourier transformed, and then converted into frequency domain data and time-frequency domain data, and the data of 1000-5500Hz is retained to form a chicken call recognition data set.

4. The chicken call recognition method under heat stress conditions based on multi-domain feature fusion according to claim 1 is characterized in that: In step 3), the hybrid feature selection module includes: a local representation module, a global representation module and a hybrid representation module connected in sequence; The hybrid characterization module comprises: a first global average pooling layer connected to the global representation module; a second global average pooling layer connected to the local representation module; A shared fully connected layer connected to the output ends of the first global average pooling layer and the second global average pooling layer; A convolutional layer connected to the output of the shared fully connected layer.

5. The chicken call recognition method under heat stress conditions based on multi-domain feature fusion according to claim 1 is characterized in that: In step 5), the CVC-2D path obtained in step 3) and the CVC-1D path obtained in step 4) are fused to obtain a multi-domain feature fusion model, which specifically includes: 5.1) The global representation of the frequency domain features output by the CVC-1D path is transformed through a linear layer to be consistent with the high-order representation dimension of the time-frequency domain features output by the CVC-2D path; 5.2) Send the global representation of frequency domain features after linear layer transformation into the bidirectional linear cross attention module Q 1D The branch is linearly transformed and globally averaged pooled to obtain feature F Q ; The high-order representation of time-frequency domain features is fed into the bidirectional linear cross attention module K 2D The branch undergoes linear transformation to obtain feature F k ; The high-order representation of time-frequency domain features is fed into the bidirectional linear cross attention module V 2D The branch undergoes linear transformation and passes through the Relu activation function to obtain the feature F v ; Will come from Q 1D The characteristic F of the branch Q The transpose of 2D The characteristic F of the branch k Multiply and pass through the Softmax function to get the attention score W 1D ; The attention score W 1D With V 2D The characteristic F of the branch v Perform Hadamard product operation to obtain the time-frequency domain feature F after weighting based on the frequency domain feature spec ; Send the time-frequency domain features into Q 2D Branch, sending the frequency domain features into the bidirectional linear cross attention module K 1D Branch and V 1D The branch performs calculations to obtain the frequency domain feature F based on the weighted time-frequency domain features. freq ; The time-frequency domain features F based on the weighted frequency domain features spec and the frequency domain features F after weighting based on the time-frequency domain features freq Add them together to get the feature F after fusion based on the bidirectional linear attention module att , and then undergo a linear transformation to form a multi-domain feature fusion model.

6. The chicken call recognition method under heat stress conditions based on multi-domain feature fusion according to claim 1 is characterized in that: In step 5), the chicken call recognition dataset obtained in step 2) is used to train and optimize the multi-domain feature fusion model, specifically including: The time-frequency domain data in the chicken call recognition dataset is input into the CVC-2D path of the multi-domain feature fusion model, and the local representation module outputs the local representation F local , the global representation module outputs the global representation F global , the local representation F local Input to the second global average pooling layer, the global representation F global Input to the first global average pooling layer, and output weight W through the shared fully connected layer local and weight W global , through the weight W local , local representation F local , weight W global , global representation F global Calculate the mixed feature F mix , the mixed feature F mix The input is input into the convolution layer, and the output of the convolution layer is added to the input of the local representation module and output after the addition.

7. The method for recognizing chicken calls under heat stress conditions based on multi-domain feature fusion according to claim 6, characterized in that: In step 5), the mixed feature F is calculated by the following formula: mix ; F mix =W local F local +W global F global 。

Images