An egg-laying hen behavior continuous recognition method, system, device and medium based on multi-modal acceleration feature fusion

Abstract

This invention relates to a method, system, device, and medium for continuous identification of laying hen behavior based on multimodal acceleration feature fusion, comprising: collecting triaxial acceleration signals of the laying hen to be identified; extracting features from the collected triaxial acceleration signals of the laying hen to obtain temporal modal features, spectral modal features, and statistical modal features; fusing the extracted temporal modal features and spectral modal features using a cross-modal acceleration feature fusion network; and concatenating the obtained multimodal fused features with the statistical modal features and inputting the result into a classifier to continuously predict the behavior of the laying hen to be identified, thereby obtaining the continuous identification result of the laying hen's behavior. This invention can be widely applied in the field of intelligent poultry farming and behavior monitoring technology.

Description

Technical Field

[0001] This invention relates to the field of intelligent poultry farming and behavior monitoring technology, specifically to a method, system, device, and medium for continuous identification of laying hen behavior based on multimodal acceleration feature fusion. Background Technology

[0002] Behavioral activity is a crucial indicator of the health and welfare of laying hens. The diverse behaviors, rapid transitions, and high frequency of these behaviors, coupled with the complex environment of the chicken coop—including varying lighting and dust—pose a significant challenge to the accurate identification of laying hen behavior. Recent research has shown that wearable accelerometers are an effective means of continuously monitoring laying hen behavior.

[0003] Many acceleration-based poultry behavior recognition methods have been proposed, primarily involving sampling parameter optimization and feature selection to improve classification performance. However, existing classification algorithms mainly rely on machine learning models, employing single-modal features and manually constructed discrete datasets for intelligent recognition of laying hen behaviors, targeting a few typical behaviors. However, in real-world scenarios, atypical behaviors are difficult to classify, and continuous behavior data suffers from poor recognition accuracy. Summary of the Invention

[0004] To address the challenges of continuous behavior recognition in individual laying hens under existing three-dimensional free-range environments, insufficient representation capabilities of single-modal features, and the inability to classify atypical behaviors, the present invention aims to provide a method, system, device, and medium for continuous behavior recognition of laying hens based on multimodal acceleration feature fusion.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion, comprising: Collect the three-axis acceleration signals of the laying hens to be identified; Feature extraction was performed on the collected triaxial acceleration signals of the laying hens to be identified, resulting in temporal modal features, spectral modal features, and statistical modal features. A cross-modal acceleration feature fusion network is used to fuse the extracted temporal modal features and spectral modal features. The resulting multimodal fusion features are then concatenated with statistical modal features and input into a classifier to continuously predict the behavior of the laying hens to be identified, thus obtaining continuous identification results of the laying hens' behavior.

[0006] Furthermore, the feature extraction of the collected triaxial acceleration signals of the laying hens to be identified yields temporal mode features, spectral mode features, and statistical mode features, including: Construct a multimodal acceleration feature extraction network; An acceleration dataset is constructed based on pre-collected historical triaxial acceleration data of laying hens. The multimodal acceleration feature extraction network is then trained using the acceleration dataset to obtain the trained multimodal acceleration feature extraction network. The collected triaxial acceleration signals of the laying hens to be identified are input into a trained multimodal acceleration signal feature extraction network to extract temporal modal features, spectral modal features, and statistical modal features.

[0007] Furthermore, the multimodal acceleration feature extraction network includes: a temporal modal encoder, a spectral modal encoder, and a statistical modal encoder; The temporal modal encoder uses a one-dimensional convolutional network and a bidirectional gated recurrent unit in series structure to learn the dynamic features of the original triaxial acceleration and obtain temporal modal features. The spectrum mode encoder uses a two-dimensional convolutional network to extract the energy change features in the local frequency feature domain of the Mel spectrogram to obtain the spectrum mode features; The statistical modal encoder extracts specific statistical features from the triaxial acceleration data to obtain statistical modal features.

[0008] Furthermore, the temporal modal encoder includes a first convolutional layer, a second convolutional layer, a BiGRU encoder layer, and a linear mapping layer; The first and second convolutional layers are both one-dimensional convolutions, used to extract the temporal features of the triaxial acceleration signal and perform distribution normalization and nonlinear transformation; The BiGRU encoder layer is provided with a first bidirectional cyclic structure and a second bidirectional cyclic structure, which are used to encode temporal features from both positive and negative time directions. The linear mapping layer is used to linearly map the output of the BiGRU encoder layer to output temporal modal features.

[0009] Furthermore, the spectral modal encoder includes a third convolutional layer, an adaptive average pooling layer, and a linear mapping layer; The third convolutional layer is a two-dimensional convolutional sampling layer, used to extract local features from the multi-channel Mel spectrogram, while capturing the energy distribution features in the frequency dimension and the energy change features in the time dimension. The adaptive average pooling layer is used to perform global average pooling operation on the frequency dimension of the spectral feature map. The linear mapping layer is used to output spectral modal features.

[0010] Furthermore, the statistical modal encoder includes a statistical feature calculation layer, a first fully connected mapping layer, and a second fully connected mapping layer; The statistical feature calculation layer is used to calculate 22-dimensional statistical features of triaxial acceleration data, including standard deviation, mean, maximum value, range, X-axis standard deviation, Y-axis standard deviation, Z-axis standard deviation, X-axis mean, Y-axis mean, Z-axis mean, Z-axis kurtosis, Z-axis skewness, Y-axis kurtosis, X-axis kurtosis, dominant frequency, spectral entropy, frequency standard deviation, correlation coefficient between X and Y axes, correlation coefficient between X and Z axes, correlation coefficient between Y and Z axes, XYZ standard deviation ratio, and Z-axis zero-crossing rate. The first fully connected mapping layer is used to perform linear dimensionality upscaling transformation on the output 22-dimensional statistical features, mapping the low-dimensional manual statistical features to the high-dimensional latent feature space, and completing the preliminary abstract expression of the statistical features; The second fully connected mapping layer is used to output statistical modal features.

[0011] Furthermore, the extracted temporal modal features and spectral modal features are fused using a cross-modal acceleration feature fusion network. The resulting multimodal fused features are then concatenated with statistical modal features and input into a classifier to continuously predict the behavior of the laying hens to be identified, resulting in continuous identification results of the laying hens' behavior, including: A cross-modal feature fusion network and a classifier are constructed; wherein, the cross-modal feature fusion network employs a multi-layer bidirectional interactive fusion unit to fuse temporal modal features and spectral modal features; An acceleration dataset is constructed based on pre-collected historical triaxial acceleration data of laying hens. The cross-modal feature fusion network and classifier are then trained using the acceleration dataset to obtain the trained cross-modal feature fusion network and classifier. Temporal modal features and spectral modal features are input into a trained cross-modal feature fusion network for feature fusion. The feature fusion result is then concatenated with statistical features and input into a classifier to predict the behavior of laying hens.

[0012] Secondly, the present invention provides a continuous recognition system for laying hen behavior based on multimodal acceleration feature fusion, comprising: The data acquisition unit is used to collect the three-axis acceleration signals of the laying hens to be identified; The feature extraction unit is used to extract features from the collected triaxial acceleration signals of the laying hens to be identified, and obtain time-series modal features, spectral modal features and statistical modal features; The behavior recognition unit is used to fuse extracted temporal modal features and spectral modal features using a cross-modal acceleration feature fusion network. The resulting multimodal fusion features are then concatenated with statistical modal features and input into a classifier to continuously predict the behavior of the laying hens to be identified, thus obtaining continuous behavior recognition results for the laying hens to be identified.

[0013] Thirdly, the present invention provides a computer-readable storage medium for storing one or more programs, said one or more programs including instructions that, when executed by a computing device, cause the computing device to perform any method.

[0014] Fourthly, the present invention provides a computing device comprising: one or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, the one or more programs including instructions for performing any method.

[0015] Due to the adoption of the above technical solutions, the present invention has the following advantages: The present invention constructs a multimodal acceleration data representation and fusion framework, which effectively improves the representation capability through the cross-fusion of time series features, spectral features and statistical features, and significantly improves the recognition accuracy of laying hen behavior on continuous datasets, especially the recognition capability of the mixed category of "other".

[0016] Therefore, this invention can be widely applied in the field of intelligent poultry farming and behavior monitoring technology. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts. In the drawings: Figure 1 This is a flowchart of the continuous recognition method for laying hen behavior based on multimodal acceleration feature fusion according to an embodiment of the present invention; Figure 2 This is a schematic diagram of how the accelerometer sensor is worn according to an embodiment of the present invention; Figure 3 This is a flowchart of the feature fusion module according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the device structure according to an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0019] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0020] Existing methods suffer from problems such as inability to classify atypical behaviors and poor performance in recognizing continuous data. In some embodiments of this invention, a method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion is provided. This method includes: collecting triaxial acceleration data of laying hens using a wing-mounted accelerometer; extracting temporal modal features, spectral modal features, and statistical modal features from the triaxial acceleration data; inputting the extracted temporal and spectral modal features into a cross-modal acceleration feature extraction network to extract multimodal fusion features; concatenating the multimodal fusion features with the statistical modal features; and using a classifier to predict laying hen behavior to complete continuous recognition of laying hen behavior. Therefore, this invention significantly improves the accuracy of recognizing laying hen resting, feeding, drinking, walking, flying, and six other behaviors in actual production environments, and has higher engineering application value.

[0021] Correspondingly, in other embodiments of the present invention, a continuous recognition system, device and medium for laying hen behavior based on multimodal acceleration feature fusion are provided.

[0022] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the invention and to fully convey the scope of the invention to those skilled in the art.

[0023] Example 1 like Figure 1 As shown in the figure, this embodiment provides a method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion, including the following steps: 1) Collect the triaxial acceleration signals of the laying hens to be identified; 2) Feature extraction was performed on the collected triaxial acceleration signals of the laying hens to be identified to obtain time-series modal features, spectral modal features, and statistical modal features; 3) The extracted temporal modal features and spectral modal features are fused using a cross-modal acceleration feature fusion network. The resulting multimodal fusion features are then concatenated with statistical modal features and input into a classifier to continuously predict the behavior of the laying hens to be identified, thus obtaining continuous identification results of the laying hens' behavior.

[0024] Furthermore, in step 1) above, such as Figure 2 As shown, the collection of triaxial acceleration signals from the laying hen to be identified can be achieved using a wing-mounted accelerometer. In this embodiment, a Fofia accelerometer is used, mounted on the wing of the laying hen, with a sampling frequency of 15Hz, and stored in txt format. This is just one example, and is not limited to this.

[0025] Furthermore, in step 2) above, when extracting features from the collected triaxial acceleration signals of the laying hens to be identified, the following steps are included: 2.1) Construct a multimodal acceleration feature extraction network.

[0026] In this embodiment, the constructed multimodal acceleration feature extraction network includes a temporal modal encoder, a spectral modal encoder, and a statistical modal encoder. The temporal modal encoder uses a one-dimensional convolutional network and a bidirectional gated recurrent unit (GRU) cascaded structure to learn the dynamic features of the original triaxial acceleration, obtaining temporal modal features. The spectral modal encoder uses a two-dimensional convolutional network to extract the energy change features of the local frequency feature domain of the Mel spectrogram, obtaining spectral modal features. The statistical modal encoder extracts specific statistical features from the triaxial acceleration data, obtaining statistical modal features.

[0027] Specifically, the temporal modality encoder includes a first convolutional layer, a second convolutional layer, a BiGRU encoder layer, and a linear mapping layer. Both the first and second convolutional layers are one-dimensional convolutions, with the first convolutional layer having a kernel size of 3 and a stride of 1; the second convolutional layer has a kernel size of 5 and a stride of 1. Both the first and second convolutional layers are followed by a batch normalization layer, a GELU activation function, and a Dropout regularization layer. These layers are used to normalize the distribution and perform nonlinear transformations on the temporal features of the convolutional output, mitigating internal covariate shift issues and accelerating network convergence. Simultaneously, random deactivation of neurons suppresses overfitting and improves the model's generalization performance. The BiGRU encoder layer contains a first bidirectional recurrent structure and a second bidirectional recurrent structure, with a hidden layer dimension of 128 in each direction. This is used to encode temporal features from both forward and reverse time directions, comprehensively capturing the long and short-term contextual dependencies of the acceleration signal, fully characterizing the transition and evolution features of continuous behavior, and enhancing the ability to represent subtle differences in actions. The linear mapping layer is used to linearly map the output of the BiGRU encoder layer, and the output is a 128-dimensional temporal modality feature.

[0028] The spectral modal encoder consists of a third convolutional layer, an adaptive average pooling layer, and a linear mapping layer. The third convolutional layer is a two-dimensional convolutional sampling layer with a kernel size of 3*3 and a stride of 1. It is used to extract local features from multi-channel Mel spectrograms, while capturing energy distribution features in the frequency dimension and energy change features in the time dimension, thereby uncovering time-frequency domain texture patterns and energy aggregation patterns corresponding to different laying hen behaviors. The third convolutional layer is followed by a batch normalization layer, a GELU activation function, and a Dropout regularization layer. This normalizes the extracted spectral feature map along the channel dimension, stabilizes the feature distribution, introduces a nonlinear transformation, amplifies the differences in spectral energy distribution across different behaviors, and reduces the risk of overfitting in frequency domain feature extraction through random channel deactivation. The adaptive average pooling layer performs global average pooling on the spectral feature map output by the Dropout regularization layer, compressing the frequency dimension of the feature map to 1. This aggregates global energy features across the entire frequency band while fully preserving the temporal dimension information, achieving dimensionality reduction and temporal length alignment of the frequency domain features. The linear mapping layer outputs a 128-dimensional spectral modality feature.

[0029] The statistical modal encoder comprises a statistical feature calculation layer, a first fully connected mapping layer, and a second fully connected mapping layer. The statistical feature calculation layer calculates 22-dimensional statistical features of the triaxial acceleration data, including standard deviation, mean, maximum value, range, X-axis standard deviation, Y-axis standard deviation, Z-axis standard deviation, X-axis mean, Y-axis mean, Z-axis mean, Z-axis kurtosis, Z-axis skewness, Y-axis kurtosis, X-axis kurtosis, dominant frequency, spectral entropy, frequency standard deviation, correlation coefficients between X and Y axes, X and Z axes, Y and Z axes, XYZ standard deviation ratio, and Z-axis zero-crossing rate. The first fully connected mapping layer performs a linear upscaling transformation on the output 22-dimensional statistical features, mapping the low-dimensional manual statistical features to a high-dimensional latent feature space, completing the initial abstract expression of the statistical features. The output of the statistical feature calculation layer is activated by ReLU and Dropout. The regularization layer performs nonlinear transformation and regularization to enhance the discriminative power of different behavior categories in statistical features, while reducing the risk of overfitting of statistical feature paths; the second fully connected mapping layer is used to output an 80-dimensional statistical modality feature.

[0030] 2.2) An acceleration dataset is constructed based on the pre-collected historical triaxial acceleration data of laying hens, and the constructed multimodal acceleration feature extraction network is trained using the acceleration dataset to obtain the trained multimodal acceleration feature extraction network.

[0031] Specifically, it includes the following steps: 2.2.1) Collect historical data on triaxial acceleration of laying hens.

[0032] 2.2.2) Preprocess and label the collected historical triaxial acceleration data of laying hens, construct an acceleration dataset and divide it into training, validation and test datasets according to a preset ratio (e.g., 8:1:1).

[0033] In this embodiment, the behavior categories of laying hens are labeled from the original triaxial acceleration historical data. The category labels include resting, drinking, feeding, walking, flying, and others. The number of each category is shown in Table 1, thus obtaining the acceleration dataset.

[0034] Table 1 Number of each behavior category

[0035] 2.2.3) Based on the training, validation and test datasets, the constructed multimodal acceleration feature extraction network is trained, validated and tested to obtain the trained multimodal acceleration feature extraction network.

[0036] In practice, the acceleration data in the training dataset is first sliced ​​using a sliding window of a preset size to form acceleration data segments. Then, the acceleration data segments are fed into the temporal modality encoder, spectrogram encoder, and statistical feature encoder for feature learning and extraction.

[0037] In this embodiment, the length of the sliding window is time 'a' seconds, the step size is 'b' seconds, and x fixed-length segments are extracted. For example, a 3-second sliding window is used to truncate the acceleration data, with a step size of 1.5 seconds, extracting 1690 fixed-length acceleration data segments.

[0038] 2.3) Input the triaxial acceleration signal of the laying hen to be identified collected in step 1) into the trained multimodal acceleration signal feature extraction network to extract the temporal modal features, spectral modal features and statistical modal features.

[0039] Furthermore, in step 3) above, such as Figure 3 As shown, in this embodiment, a cross-modal feature fusion network is used to fuse temporal modal features and spectral modal features, extract cross-modal correlation information from multimodal acceleration features, and fuse the complementary advantages of temporal features, frequency features and statistical features to construct a unified behavior representation space, providing highly discriminative feature inputs for subsequent behavior classifiers.

[0040] Specifically, it includes the following steps: 3.1) Construct a cross-modal feature fusion network and classifier.

[0041] like Figure 3As shown, in this embodiment, the cross-modal feature fusion network employs multi-layer bidirectional interactive fusion units to fuse modal features. Each bidirectional interactive fusion unit includes a feature concatenation operation, a multi-head self-attention mechanism, layer normalization, and a feedforward network. The feature concatenation operation concatenates the input temporal modal feature sequence and the spectral modal feature sequence along the feature dimension, constructing a cross-modal feature space that simultaneously contains information in the time and frequency domains. The multi-head self-attention mechanism uses multiple independent attention heads to perform global dependency modeling on the concatenated cross-modal feature sequence, automatically learning the contextual relationships within the temporal and spectral modalities, as well as the complementary correspondences between modalities, and uncovering the behavioral patterns of cross-modal collaborative representations. Layer normalization, combined with residual connections, standardizes the feature distribution and optimizes gradient propagation of the attention output, ensuring training stability when the multi-layer fusion units are stacked. The feedforward network performs position-wise nonlinear transformations on the attention features through two layers of linear transformation and activation functions, further enhancing the abstract representation capability of the cross-modal fused features. The enhanced temporal and spectral features are subjected to global average pooling, and then concatenated with the statistical features output by the statistical feature encoder to output a 336-dimensional feature. The output cross-modal feature is then input into the classifier for training and to achieve the prediction of laying hen behavior.

[0042] 3.2) Based on the pre-collected historical triaxial acceleration data of laying hens, an acceleration dataset is constructed, and the constructed cross-modal feature fusion network and classifier are trained using the acceleration dataset to obtain the trained cross-modal feature fusion network and classifier.

[0043] The method for collecting historical triaxial acceleration data of laying hens to construct an acceleration dataset is the same as the method in step 2.2) above, and will not be described again in this invention.

[0044] 3.3) Input the temporal modal features and spectral modal features into the trained cross-modal feature fusion network for feature fusion. Then, concatenate the feature fusion results with statistical features and input them into the classifier to predict the behavior of laying hens.

[0045] Example 2 The above-described embodiment 1 provides a method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion. Correspondingly, this embodiment provides a system for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion. The system provided in this embodiment can implement the method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion of embodiment 1. The system can be implemented through software, hardware, or a combination of both. For example, the system may include integrated or separate functional modules or units to execute the corresponding steps in the methods of embodiment 1. Since the system in this embodiment is basically similar to the method embodiment, the description process in this embodiment is relatively simple. Relevant details can be found in the description of embodiment 1. The system embodiment provided in this embodiment is merely illustrative.

[0046] The continuous recognition system for laying hen behavior based on multimodal acceleration feature fusion provided in this embodiment includes: The data acquisition unit is used to collect the three-axis acceleration signals of the laying hens to be identified; The feature extraction unit is used to extract features from the collected triaxial acceleration signals of the laying hens to be identified, and obtain time-series modal features, spectral modal features and statistical modal features; The behavior recognition unit is used to fuse extracted temporal modal features and spectral modal features using a cross-modal acceleration feature fusion network. The resulting multimodal fusion features are then concatenated with statistical modal features and input into a classifier to continuously predict the behavior of the laying hens to be identified, thus obtaining continuous behavior recognition results for the laying hens to be identified.

[0047] Example 3 This embodiment provides a processing device corresponding to the continuous recognition method for laying hen behavior based on multimodal acceleration feature fusion provided in Embodiment 1. The processing device can be a client-side processing device, such as a mobile phone, laptop, tablet computer, desktop computer, etc., to execute the method of Embodiment 1.

[0048] like Figure 4 As shown, the processing device includes a processor, a memory, a communication interface, and a bus. The processor, memory, and communication interface are connected via the bus to communicate with each other. The memory stores a computer program that can run on the processor. When the processor runs the computer program, it executes the continuous recognition method for laying hen behavior based on multimodal acceleration feature fusion provided in Embodiment 1.

[0049] Preferably, the memory may be high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device.

[0050] Preferably, the processor can be any type of general-purpose processor such as a central processing unit (CPU) or a digital signal processor (DSP), and there is no limitation herein.

[0051] Example 4 The continuous recognition method for laying hen behavior based on multimodal acceleration feature fusion in Embodiment 1 can be specifically implemented as a computer program product. The computer program product may include a computer-readable storage medium on which computer-readable program instructions for executing the continuous recognition method for laying hen behavior based on multimodal acceleration feature fusion described in Embodiment 1 are loaded.

[0052] A computer-readable storage medium can be a tangible device that holds and stores instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination thereof.

[0053] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes. These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion, characterized in that, include: Collect the three-axis acceleration signals of the laying hens to be identified; Feature extraction was performed on the collected triaxial acceleration signals of the laying hens to be identified, resulting in temporal modal features, spectral modal features, and statistical modal features. A cross-modal acceleration feature fusion network is used to fuse the extracted temporal modal features and spectral modal features. The resulting multimodal fusion features are then concatenated with statistical modal features and input into a classifier to continuously predict the behavior of the laying hens to be identified, thus obtaining continuous identification results of the laying hens' behavior.

2. The method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion as described in claim 1, characterized in that, The process of extracting features from the collected triaxial acceleration signals of the laying hens to be identified yields temporal modal features, spectral modal features, and statistical modal features, including: Construct a multimodal acceleration feature extraction network; An acceleration dataset is constructed based on pre-collected historical triaxial acceleration data of laying hens. The multimodal acceleration feature extraction network is then trained using the acceleration dataset to obtain the trained multimodal acceleration feature extraction network. The collected triaxial acceleration signals of the laying hens to be identified are input into a trained multimodal acceleration signal feature extraction network to extract temporal modal features, spectral modal features, and statistical modal features.

3. The method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion as described in claim 2, characterized in that, The multimodal acceleration feature extraction network includes: a temporal modal encoder, a spectral modal encoder, and a statistical modal encoder; The temporal modal encoder uses a one-dimensional convolutional network and a bidirectional gated recurrent unit in series structure to learn the dynamic features of the original triaxial acceleration and obtain temporal modal features. The spectrum mode encoder uses a two-dimensional convolutional network to extract the energy change features in the local frequency feature domain of the Mel spectrogram to obtain the spectrum mode features; The statistical modal encoder extracts specific statistical features from the triaxial acceleration data to obtain statistical modal features.

4. The method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion as described in claim 3, characterized in that, The temporal modal encoder includes a first convolutional layer, a second convolutional layer, a BiGRU encoder layer, and a linear mapping layer; The first and second convolutional layers are both one-dimensional convolutions, used to extract the temporal features of the triaxial acceleration signal and perform distribution normalization and nonlinear transformation; The BiGRU encoder layer is provided with a first bidirectional cyclic structure and a second bidirectional cyclic structure, which are used to encode temporal features from both positive and negative time directions. The linear mapping layer is used to linearly map the output of the BiGRU encoder layer to output temporal modal features.

5. The method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion as described in claim 3, characterized in that, The spectral modal encoder includes a third convolutional layer, an adaptive average pooling layer, and a linear mapping layer; The third convolutional layer is a two-dimensional convolutional sampling layer, used to extract local features from the multi-channel Mel spectrogram, while capturing the energy distribution features in the frequency dimension and the energy change features in the time dimension. The adaptive average pooling layer is used to perform global average pooling operation on the frequency dimension of the spectral feature map. The linear mapping layer is used to output spectral modal features.

6. The method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion as described in claim 3, characterized in that, The statistical modal encoder includes a statistical feature calculation layer, a first fully connected mapping layer, and a second fully connected mapping layer; The statistical feature calculation layer is used to calculate 22-dimensional statistical features of triaxial acceleration data, including standard deviation, mean, maximum value, range, X-axis standard deviation, Y-axis standard deviation, Z-axis standard deviation, X-axis mean, Y-axis mean, Z-axis mean, Z-axis kurtosis, Z-axis skewness, Y-axis kurtosis, X-axis kurtosis, dominant frequency, spectral entropy, frequency standard deviation, correlation coefficient between X and Y axes, correlation coefficient between X and Z axes, correlation coefficient between Y and Z axes, XYZ standard deviation ratio, and Z-axis zero-crossing rate. The first fully connected mapping layer is used to perform linear dimensionality upscaling transformation on the output 22-dimensional statistical features, mapping the low-dimensional manual statistical features to the high-dimensional latent feature space, and completing the preliminary abstract expression of the statistical features; The second fully connected mapping layer is used to output statistical modal features.

7. The method for continuous recognition of laying hen behavior based on multimodal acceleration feature fusion as described in claim 1, characterized in that, The method utilizes a cross-modal acceleration feature fusion network to fuse extracted temporal modal features and spectral modal features. The resulting multimodal fused features are then concatenated with statistical modal features and input into a classifier to continuously predict the behavior of the laying hens to be identified, yielding continuous behavior identification results for the laying hens, including: A cross-modal feature fusion network and a classifier are constructed; wherein, the cross-modal feature fusion network employs a multi-layer bidirectional interactive fusion unit to fuse temporal modal features and spectral modal features; An acceleration dataset is constructed based on pre-collected historical triaxial acceleration data of laying hens. The cross-modal feature fusion network and classifier are then trained using the acceleration dataset to obtain the trained cross-modal feature fusion network and classifier. Temporal modal features and spectral modal features are input into a trained cross-modal feature fusion network for feature fusion. The feature fusion result is then concatenated with statistical features and input into a classifier to predict the behavior of laying hens.

8. A continuous recognition system for laying hen behavior based on multimodal acceleration feature fusion, characterized in that, include: The data acquisition unit is used to collect the three-axis acceleration signals of the laying hens to be identified; The feature extraction unit is used to extract features from the collected triaxial acceleration signals of the laying hens to be identified, and obtain time-series modal features, spectral modal features and statistical modal features; The behavior recognition unit is used to fuse extracted temporal modal features and spectral modal features using a cross-modal acceleration feature fusion network. The resulting multimodal fusion features are then concatenated with statistical modal features and input into a classifier to continuously predict the behavior of the laying hens to be identified, thus obtaining continuous behavior recognition results for the laying hens to be identified.

9. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods described in claims 1 to 7.

10. A computing device, characterized in that, include: One or more processors and a memory, wherein the memory stores one or more programs and is configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described in claims 1 to 7.

Images