Deep learning-based cattle behavior classification and oestrus detection method

Abstract

The invention relates to a cattle behavior classification and oestrus detection method based on deep learning, and the method comprises the steps: S1) building an acceleration and sound data set for representing cattle behaviors, which comprises the steps: carrying out the collection and marking of acceleration data and sound data representing the daily behaviors and oestrus behaviors of cattle, forming acceleration data and sound data with annotation information; step S2), constructing a CLAT-MMNet model; s3) training the CLAT-MMNet model to an expected result by using a data set of acceleration data and sound data with annotation information; and S4) inputting the collected acceleration data and sound data representing cattle behaviors into the trained CLAT-MMNet model, and outputting a detection result of the cattle behaviors.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method for classifying bovine behavior and detecting estrus based on deep learning. Background Technology

[0002] In cattle farming, daily behaviors (such as lying down, standing, walking, and eating) are closely related to the cattle's health. When cattle exhibit health abnormalities (such as indigestion, lameness, postpartum inflammation, etc.), their daily behaviors will change significantly. For example, sick cattle may exhibit abnormally prolonged lying down time, increased standing time, decreased walking frequency, or reduced eating time and slower eating movements. By automatically identifying and statistically analyzing these daily behaviors, individual cattle whose behavior deviates from normal patterns can be detected in a timely manner, thereby achieving early disease warning and health status monitoring. Estrus in cattle is an important indicator of their reproductive status. Mounting behavior is one of the typical behaviors exhibited by cows during estrus, especially at the peak of estrus. Cows often actively mount other cattle or are mounted by other cattle. This behavior rarely occurs outside of estrus, thus having a strong indicative nature of estrus. By monitoring mounting behavior, the estrus status of cattle can be understood, which can help farmers accurately determine the timing of mating and improve reproductive efficiency.

[0003] Therefore, accurate and efficient monitoring of cattle behavior and estrus status helps to understand the physiological health and estrus timing of cattle, reducing economic losses on farms and effectively improving cattle reproductive efficiency. Thus, monitoring cattle behavior and estrus has become an important task in modern livestock production.

[0004] Traditional cattle farming methods primarily rely on manual inspections and body temperature monitoring. While these methods can monitor cattle behavior and determine estrus to some extent, they have several problems. First, manual inspections consume significant manpower and resources, and due to the lack of automated monitoring and diagnostic tools, relying solely on human experience is prone to human error, failing to accurately and promptly obtain information on cattle behavior and health status. Second, body temperature monitoring depends on the experience of the farmers, and its frequency and efficiency are limited, easily leading to missed detections and hindering timely detection of estrus, thus impacting cattle reproductive efficiency.

[0005] With the rapid development of multi-sensor and artificial intelligence technologies, deep learning-based cattle behavior classification and estrus detection technologies are gradually becoming a new solution. The YOLOv8 model, as an advanced object detection algorithm, can efficiently process image or video data from cattle farming scenarios, identifying individual cattle and various behaviors.

[0006] However, there are some drawbacks to using the YOLOv8 model: the cattle farming environment is mostly dark and damp, the interior of the cattle shed is complex and there are many interfering factors, the lighting conditions inside the cattle shed are variable and the light intensity varies greatly at different times of the day, and the fences, feed troughs, water troughs and other objects in the cattle shed may obstruct or confuse the images of the cattle, which will reduce the recognition accuracy of the YOLOv8 model and make it easy to make misjudgments or omissions, resulting in a series of incorrect measures.

[0007] Therefore, new methods for detecting bovine behavior and estrus are needed. Summary of the Invention

[0008] To address the aforementioned technical problems, this invention provides a deep learning-based method for detecting bovine behavior, comprising:

[0009] Step S1) Establish an acceleration and sound dataset to characterize cattle behavior, including: collecting and labeling acceleration and sound data that characterize the daily behavior and estrus behavior of cattle, forming acceleration and sound data with labeled information;

[0010] Step S2) Construct the CLAT-MMNet model;

[0011] Step S3) Train the CLAT-MMNet model to the expected result using a dataset of acceleration data and sound data with labeled information;

[0012] Step S4) The acceleration data and sound data representing the behavior of cattle collected are input into the trained CLAT-MMNet model, and the detection results of cattle behavior are output.

[0013] The beneficial effects of this invention are as follows:

[0014] This invention provides a deep learning-based method for classifying bovine behavior and detecting estrus. It innovatively designs a CLAT-MMNet (CNN-LSTM-Attention-Transformer Multi-ModalNetwork) model, enabling the detection of daily and estrus behaviors in cattle farms under complex conditions. This method accurately classifies these behaviors, avoiding the problems of late disease detection and delayed mating caused by inadequate manual inspections or missed temperature monitoring in traditional cattle farming, as well as the low accuracy and susceptibility to misjudgments or missed detections inherent in existing AI technologies. By collecting and analyzing data from multiple sensors, including acceleration and sound data, the method accurately monitors bovine behavior, helping farm staff to accurately assess the physiological health and estrus status of cattle. This allows for early detection of diseases or estrus, timely treatment of sick cattle, and timely mating of cattle in estrus, ultimately improving farm efficiency.

[0015] The CLAT-MMNet model was established to study the long-term behavioral trends of cattle, which enhanced the comprehensive understanding of cattle behavior and further improved the detection accuracy of daily behavior and estrus behavior, thereby increasing the efficiency and effectiveness of cattle farming. Attached Figure Description

[0016] Figure 1 This is a flowchart of the deep learning-based bovine behavior classification and estrus detection method of the present invention;

[0017] Figure 2 This is a schematic diagram of the structure of the CLAT-MMNet model designed in this invention;

[0018] Figure 3 This is a schematic diagram of the ResNet module structure;

[0019] Figure 4 This is a schematic diagram of the Softmax Attention module.

[0020] Figure 5 This is a schematic diagram of the LSTM module.

[0021] Figure 6 This is a schematic diagram of the Attention module.

[0022] Figure 7 This is a structural diagram of the Transformer module;

[0023] Figure 8 This is a schematic diagram of the MLP module. Detailed Implementation

[0024] The technical solutions in the embodiments of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Analysis of long-term collected experimental data revealed that the acceleration and vocal characteristics of bovine movements differ across various behavioral states. Placing an accelerometer, such as a triaxial accelerometer, in the bovine neck allows for the measurement of acceleration during lying down, standing, walking, eating, and mounting behaviors. This triaxial acceleration data captures the bovine's movement patterns and is a core signal source for identifying bovine behavior. Vocal data reflects the bovine's acoustic expression during specific behaviors, particularly indicating estrus. A sound sensor can be placed in the bovine neck to collect vocalization data. The combined use of these two methods facilitates high-precision identification of bovine daily behaviors such as lying down, standing, walking, and eating, as well as mounting and estrus behaviors.

[0026] In the daily behavior of cattle, lying down behavior is usually characterized by a low-amplitude stable state of acceleration signal. The X-axis or Z-axis data in the three-axis acceleration data show a static distribution close to gravitational acceleration for a long time, and the direction angle remains stable, accompanied by a steady and long breathing sound. The acceleration signal characteristics of standing behavior are similar to those of lying down, but the Z-axis direction is mostly vertical, and the overall tremor is slightly higher, accompanied by a stable breathing sound. Walking behavior will produce periodic high-amplitude vibration characteristics in the three acceleration axes, showing obvious rhythmicity, and accompanied by a stable and relatively heavy breathing sound. Feeding behavior usually occurs on the basis of standing, accompanied by small and frequent head shaking, with visible local high-frequency vibrations in the acceleration signal, and often accompanied by a stable low-intensity chewing sound or licking sound.

[0027] The most distinctive behavior in cattle estrus is mounting, which triggers strong acceleration signal changes, especially short-term violent vibrations in the Z-axis direction. At the same time, the sound signal is often accompanied by strong panting, mooing or impact sounds, which are high-energy spectrum bands with significant peaks, significantly different from the acoustic patterns of other behaviors.

[0028] Therefore, by extracting features such as amplitude variation, azimuth distribution, and frequency domain of acceleration data, as well as features such as Mel spectrum, energy envelope, and pitch variation of sound data, this invention uses time-series modeling methods and attention mechanisms to achieve accurate identification of daily behaviors of cattle, such as lying down, standing, walking, and eating, as well as mounting and estrus behaviors.

[0029] refer to Figure 1-8This invention proposes a method for cattle behavior classification and estrus detection based on deep learning, specifically including:

[0030] Step S1) Establish an acceleration and sound dataset to characterize cattle behavior, including: collecting and labeling acceleration and sound data that characterize daily and estrus behaviors of cattle, forming acceleration and sound data with labeled information, including: lying down, standing, walking, eating, and mounting.

[0031] Acceleration and sound data representing the behavior of cattle were collected using field data collection methods. Here, sound data and audio data are different ways of expressing the same meaning.

[0032] Accelerometer and sound sensors were used to collect acceleration and sound data representing cattle behavior and estrus behavior at the cattle breeding site, respectively. Video data of cattle behavior was also collected using a camera device. The acceleration and sound data reflect the cattle's daily behavior types and estrus behavior, serving as the dataset for subsequent model training and application. The video data visually records the cattle's daily and estrus behaviors and is used to annotate the acceleration and sound data for these behaviors. Considering the diversity of cattle behavior during field data collection, a camera device was used to capture video of various behaviors. For daily and estrus states, this invention focuses only on the following daily behaviors: lying down, standing, walking, and eating, which reflect the cattle's health status. For estrus behavior, only mounting behavior is considered, as this reflects the cattle's estrus state. The collected video data was used as the basis for labeling the daily and estrus behaviors of cattle. Based on the cattle behaviors identified through video, the collected acceleration and sound data of the cattle behaviors were manually labeled. The acceleration and sound data to be labeled were time-synchronized and labeled with the same content. The labeling content included: daily cattle behaviors: lying down, standing, walking, eating; and estrus behaviors: mounting. The specific method is as follows: When labeling, the collected video data was viewed, that is, the cattle behavior was observed in the video where the time point or time period of the acceleration and sound data synchronized with the time point or time period. The acceleration and sound data collected at that time point or time period were labeled with the following content: lying down, standing, walking, eating, mounting. In this way, corresponding acceleration and sound data were obtained for the cattle behaviors. The following explains in detail how acceleration, sound, and video data are selected at the same time point or time period that can characterize cattle behavior. By watching the video data, the behavior of the cattle can be learned, such as lying down, standing, walking, eating, or mounting. The corresponding acceleration and sound data are labeled with the corresponding behavioral information of the cattle. When selecting video data for a time period that can identify the behavior of cattle, multiple acceleration and sound data points can be selected within that time period, and the behavioral information of the cattle can be labeled for each of them. For example, if cattle are observed eating in the video, the acceleration and sound data collected at multiple synchronous times during the time period when the cattle are eating can be labeled with the behavioral information of "eating".

[0033] To ensure high data quality, this invention uses Label Studio to annotate acceleration data and Audacity to annotate sound data.

[0034] Furthermore, to enhance the diversity, robustness, and generalization ability of the dataset, this invention also performs data preprocessing on the collected acceleration and sound data.

[0035] Accelerometer data preprocessing methods include: random rotation enhancement, scaling and jitter enhancement, and data normalization.

[0036] Random rotation enhancement:

[0037] Because the orientation of collars or other ring components with accelerometers installed can be inconsistent or inverted during wear, the accelerometer readings may flip, but the behavior category label remains unchanged. To improve the robustness of the model under different wearing postures, a random rotation enhancement operation in three-dimensional space is applied to the acceleration data. Specifically, a rotation matrix is ​​constructed based on a three-axis coordinate system to rotate and transform the original acceleration vector, thereby expanding the spatial diversity of the dataset and enhancing the model's tolerance to changes in sensor installation orientation.

[0038] Scaling and jitter enhancement:

[0039] When cattle are in relatively static behaviors such as standing or lying down, the collars containing accelerometers around their necks can still generate interference signals of varying amplitudes due to slight swaying. This causes acceleration signals that should belong to the static category to be misrepresented as dynamic behaviors such as walking or eating, leading to increased intra-class variability and label confusion. To improve the robustness of the model under such boundary samples, scaling and jitter enhancement processing are applied to the acceleration data. Scaling is performed by multiplying by a randomly finely tuned scaling factor to simulate multiplicative noise caused by sensor sensitivity or calibration deviations. Jitter is performed by superimposing Gaussian white noise with a mean of 0 and an adjustable standard deviation to simulate additive noise caused by sensor environmental disturbances or minor hardware errors. This enhancement method not only improves the model's adaptability to multi-source noise but also further enriches the training sample space and reduces the risk of overfitting.

[0040] Data standardization:

[0041] To ensure comparability of data across different axes, the triaxial acceleration data were standardized. Standardization transforms the data into a dimensionless form, eliminating differences in numerical ranges between axes. This ensures that data from different axes have equal importance and weight during training and modeling. Therefore, all data were scaled to a form with a mean (μ) of 0 and a standard deviation (σ) of 1.

[0042] Its mathematical expression is:

[0043]

[0044] Where V represents the raw acceleration data collected by the collar, μ represents the population mean, σ represents the population standard deviation, and V′ represents the acceleration value after the raw acceleration data has been standardized.

[0045] Audio data preprocessing methods include: spectral subtraction noise reduction and wavelet filtering noise reduction.

[0046] Spectral subtraction noise reduction:

[0047] To remove relatively stable additive noise from audio data, such as white electrical noise and noise generated by mechanical equipment like manure trucks, a subtraction noise reduction method is used. The basic principle is to subtract the noise spectrum from the spectrum of the noisy frequency signal in the spectral space to obtain an estimated clean speech spectrum, which is then used to reconstruct a clear audio signal.

[0048] The specific process of spectral subtraction noise reduction mainly includes three steps: noise estimation, spectrum conversion and noise reduction, and clean audio reconstruction. First, noise estimation is performed by extracting background noise without behavioral activity from periods when the cow is still or silent, establishing a noise model, and extracting its spectral features as the noise baseline. Then, spectrum conversion and noise reduction are performed. The noisy audio signal is processed in frames, and a Fast Fourier Transform (FFT) is performed on each frame to obtain the spectrum. This spectrum is then subtracted from the pre-estimated noise spectrum frequency-by-frequency to obtain the denoised spectrum. To avoid negative spectrum values, a lower limit for spectral subtraction is introduced to ensure that the subtracted spectrum is non-negative. Finally, clean audio is reconstructed by performing an Inverse Fast Fourier Transform (IFFT) on the processed denoised spectrum, and then using methods such as overlapping windowing to restore the time-domain audio signal, ultimately obtaining the noise-suppressed audio data.

[0049] Wavelet filtering noise reduction:

[0050] Wavelet denoising is a signal processing technique suitable for non-stationary signals, and audio signals are indeed non-stationary. The basic principle of this method is to use wavelet decomposition to break down the signal into wavelet coefficients of different scales according to frequency bands. Taking into account the differences in the distribution of wavelet decomposition coefficients of signal and noise in different frequency bands, noisy wavelet coefficients are removed, while wavelet coefficients of the effective signal are retained. The clean signal is then reconstructed using wavelet decomposition.

[0051] The specific process of wavelet denoising mainly includes: first, inputting a noisy audio signal; then, selecting an appropriate wavelet basis function to perform multi-scale decomposition on the noisy signal. After decomposition, the corresponding wavelet coefficients are obtained. Among them, the detail coefficients represent the high-frequency information of the audio signal, and the approximation coefficients represent the low-frequency information of the audio signal. Subsequently, the wavelet coefficients are processed using the wavelet thresholding function denoising method, and finally, the thresholded wavelet coefficients are reconstructed to obtain the denoised audio data.

[0052] The wavelet threshold function denoising method is as follows:

[0053]

[0054] Where ω represents the wavelet coefficients of the original signal, λ is the preset threshold, and a and b are adjustment factors. These are the wavelet estimation coefficients for the denoised signal.

[0055] The wavelet threshold function denoising method has the advantages of continuity, flexibility, small constant deviation and high-order differentiability. To a certain extent, it can handle smaller signals and achieve better denoising effect.

[0056] Through these data preprocessing techniques, a diverse and representative dataset of acceleration and sound was ultimately generated.

[0057] S2) Construct the CLAT-MMNet model.

[0058] This invention designs a novel neural network model based on deep learning, namely, constructing the CLAT-MMNet model, as follows: Figure 2 As shown.

[0059] The CLAT-MMNet model consists of three parts: the first part is a multimodal time-frequency fusion unit, the second part is a multi-scale time-frequency feature analysis unit, and the third part is a classifier.

[0060] The multimodal time-frequency fusion unit includes a Conv module, a ResNet module, and a Softmax Attention module. Acceleration data with labeled content is input to the Conv module, and audio data with labeled content is input to the ResNet module. The outputs of the Conv module and the ResNet module are respectively connected to the input of the Softmax Attention module to achieve data fusion.

[0061] The Conv module extracts local time-frequency features of acceleration through sliding convolution, effectively characterizing short-term behavioral change patterns and improving the accuracy and robustness of early acceleration feature extraction. The ResNet module extracts deep speech features through a multi-layer residual convolution structure, effectively enhancing the ability to model the temporal structure and semantic information of speech signals while maintaining feature integrity. The Softmax Attention module normalizes the weights of acceleration and sound features to achieve weighted fusion of heterogeneous features, effectively improving the collaborative expression and discriminative performance of multi-source information in behavior classification and estrus detection tasks.

[0062] The multimodal time-frequency fusion unit further enhances the model's ability to capture weak but critical signals by fusing acceleration and sound data, which is of great significance for recognizing daily behavior and mounting estrus behavior in cattle.

[0063] The multi-scale time-frequency feature analysis unit includes a CNN module, an LSTM module, an Attention module, and a Transformer module. The input of the CNN module is connected to the output of the Softmax Attention module, the output of the CNN module is connected to the input of the LSTM module, the output of the LSTM module is connected to the input of the Attention module, and the output of the Attention module is connected to the input of the Transformer module.

[0064] The CNN module captures local temporal features through sliding convolution operations, improving its responsiveness to instantaneous dynamic information and providing high-quality initial feature representations for subsequent deep temporal modeling. The LSTM module captures medium- to long-term dependencies in sequence data through gated recurrent structures, enhancing the understanding of behavioral state change trends and improving the accuracy of temporal pattern recognition. The Attention module assigns weights to features at each time step in the time series, enhancing the discriminative features at key moments and suppressing redundant information, enabling accurate modeling of important behavioral segments in multimodal sequences. The Transformer module constructs global contextual semantic dependencies across time steps based on a self-attention mechanism, effectively mining the correlation between long-distance features in the sequence and improving the ability to model complex behavioral patterns.

[0065] The classifier employs an MLP module, whose input is connected to the output of the Transformer module. The MLP module outputs the detection results of cattle behavior, including the detection results of daily cattle behavior and estrus behavior. The MLP module decodes fused multimodal features through a fully connected structure to achieve the output task of cattle behavior detection results, including four daily cattle behaviors: "walking," "standing," "eating," and "lying down," and one estrus behavior: "mounting." The output of this MLP model addresses the needs of both daily behavior classification for understanding the physiological health of cattle and estrus behavior for understanding cattle estrus, thus improving the model's discriminative ability and application value.

[0066] Figure 3 The ResNet module structure is shown. Figure 4 The structure of the Softmax Attention module is shown. Figure 5 The LSTM module structure is shown. Figure 6 The structure of the Attention module is shown. Figure 7 The Transformer module structure is shown. Figure 8 The structure of the MLP module is shown. The Conv module and CNN module are publicly available in this field and will not be described in detail here.

[0067] The CLAT-MMNet model designed in this invention adopts the weighted cross-entropy loss function (WCE). This WCE loss function improves the model's attention to samples of a minority class by introducing a class weight adjustment mechanism, thereby alleviating the bias problem caused by uneven class distribution during training and improving the overall classification accuracy.

[0068] The WCE loss function formula is as follows:

[0069]

[0070] Among them, w i y represents the weight of the i-th class, which is set or calculated manually. i p represents the true class as the one-hot encoded form of class i. i This represents the model's predicted probability for class i; i represents the class, and C represents the number of classes.

[0071] S3) Train the CLAT-MMNet model to the expected result using a dataset of acceleration and sound data with labeled information.

[0072] The dataset established in step S1 is divided into a training dataset and a validation dataset in an 8:2 ratio. The training method can use any existing known method. A brief explanation follows.

[0073] The training process uses the Adam optimizer with an initial learning rate of 0.001, and the learning rate is dynamically adjusted using a known cosine annealing strategy.

[0074] The training settings are as follows:

[0075] Batch size: 32 sets of multimodal time series data, meaning the model processes 32 sets of multimodal time series data each time.

[0076] Training rounds: 300 rounds, with an early stopping mechanism to prevent overfitting.

[0077] Data preprocessing: All input acceleration and audio data are first normalized; acceleration data is truncated into a fixed-length time window (e.g., 5 seconds × 40Hz = 200 frames); audio data is extracted with the same number of frames within the time period corresponding to acceleration through parameter settings, and time-aligned with acceleration data.

[0078] Modal fusion processing: Acceleration data is processed by the Conv module to extract features, and audio data is processed by the ResNet module to extract features. Then, multimodal fusion is achieved through the Softmax Attention mechanism and the data is input to the multi-scale time-frequency feature analysis unit.

[0079] The CLAT-MMNet model is evaluated using metrics to determine if it has achieved the expected results. Specifically, the performance of the CLAT-MMNet model is assessed using a validation set. The CLAT-MMNet model is considered to have achieved the expected results when the mean average precision (mAP) is greater than 80%, the number of model parameters is less than 10M, the accuracy is approximately 93% or higher, and the recall is approximately 85% or higher. At this point, the training is considered complete.

[0080] S4) Input the collected acceleration data and sound data representing cattle behavior into the trained CLAT-MMNet model, and output the detection results of cattle behavior, including: daily behavior detection results and estrus behavior detection results.

[0081] Acceleration and sound data representing the daily and estrus behaviors of cattle, collected from the cattle farm, are input into the trained CLAT-MMNet model. The model outputs the detection results of cattle behaviors: walking, standing, eating, lying down, and mounting. Walking, standing, eating, and lying down are the classifications of daily behaviors of cattle, while mounting is the estrus behavior of cattle.

[0082] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, several simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered to fall within the scope of protection defined by the claims submitted herein.

Claims

1. A deep learning-based method for detecting cattle behavior, comprising: Step S1) Establish an acceleration and sound dataset to characterize cattle behavior, including: collecting and labeling acceleration and sound data that characterize the daily behavior and estrus behavior of cattle, forming acceleration and sound data with labeled information; Step S2) Construct the CLAT-MMNet model; Step S3) Train the CLAT-MMNet model to the expected result using a dataset of acceleration data and sound data with labeled information; Step S4) The acceleration data and sound data representing the behavior of cattle collected are input into the trained CLAT-MMNet model, and the detection results of cattle behavior are output.

2. The method for detecting cattle behavior based on deep learning according to claim 1, wherein, The labeling information includes: lying down, standing, walking, eating, and climbing.

3. The method for detecting cattle behavior based on deep learning according to claim 1, wherein, The method for annotating acceleration and sound data characterizing the daily and estrus behaviors of cattle includes: collecting video data of cattle behavior, using it as the basis for annotating acceleration and sound data characterizing the daily and estrus behaviors of cattle, watching the cattle behavior in video data where the time point or time period of the acceleration and sound data synchronized with the time of collection is the same, and annotating the acceleration and sound data collected at that time point or time period.

4. The method for detecting cattle behavior based on deep learning according to claim 1, wherein, Step S1 further includes: preprocessing the acceleration data, including: random rotation enhancement, scaling and jitter enhancement, and data normalization. The random rotation enhancement method includes: constructing a rotation matrix based on a three-axis coordinate system to perform a rotation transformation on the original acceleration vector, thereby expanding the spatial distribution diversity of the dataset. The scaling and jitter enhancement method includes: scaling by multiplying by a randomly fine-tuned scaling factor to simulate multiplicative noise caused by sensor sensitivity or calibration deviation; and jitter by superimposing Gaussian white noise with a mean of 0 and an adjustable standard deviation to simulate additive noise caused by sensor environmental disturbances or minor hardware errors. The data normalization method includes: Where V represents the raw acceleration data collected by the collar, μ represents the population mean, σ represents the population standard deviation, and V′ represents the acceleration value after the raw acceleration data has been standardized.

5. The method for detecting cattle behavior based on deep learning according to claim 1, wherein, Step S1 further includes: preprocessing the sound data, including: spectral subtraction noise reduction and wavelet filtering noise reduction. The spectral subtraction noise reduction method includes: firstly, noise estimation is performed, extracting background noise without behavior from when the cow is still or not vocalizing, establishing a noise model, extracting its spectral features as the noise baseline, and then performing spectral transformation and noise reduction operations. The noisy audio signal is processed in frames, and a fast Fourier transform is performed on each frame to obtain the spectrum. The spectrum is subtracted from the pre-estimated noise spectrum frequency by frequency and a lower limit value is introduced to obtain a non-negative denoised spectrum. An inverse fast Fourier transform is performed on the denoised spectrum, and then the time-domain audio signal is restored by the overlapping windowing method to finally obtain the noise-suppressed audio data. The wavelet filtering noise reduction method includes: inputting the noisy audio signal, then selecting wavelet basis functions to perform multi-scale decomposition on the noisy signal to obtain the corresponding wavelet coefficients, then processing the wavelet coefficients by the wavelet threshold function noise reduction method, and finally reconstructing the thresholded wavelet coefficients to obtain the denoised audio data. The wavelet threshold function denoising method is as follows: Where ω represents the wavelet coefficients of the original signal, λ is the preset threshold, and a and b are adjustment factors. These are the wavelet estimation coefficients for the denoised signal.

6. The method for detecting cattle behavior based on deep learning according to claim 1, wherein, The CLAT-MMNet model comprises three parts: the first part is a multimodal time-frequency fusion unit, the second part is a multi-scale time-frequency feature analysis unit, and the third part is a classifier, wherein: The multimodal time-frequency fusion unit includes a Conv module, a ResNet module, and a Softmax Attention module. Acceleration data with labeled content is input to the Conv module, and sound data with labeled content is input to the ResNet module. The outputs of the Conv module and the ResNet module are respectively connected to the input of the Softmax Attention module. The multi-scale time-frequency feature analysis unit includes: a CNN module, an LSTM module, an Attention module, and a Transformer module, wherein the input of the CNN module is connected to the output of the Softmax Attention module, the output of the CNN module is connected to the input of the LSTM module, the output of the LSTM module is connected to the input of the Attention module, and the output of the Attention module is connected to the input of the Transformer module. The classifier uses an MLP module, the input of which is connected to the output of the Transformer module, and the output of the MLP module is used to output the detection results of cattle behavior.

7. The method for detecting cattle behavior based on deep learning according to claim 1, wherein, The detection results of the cattle behavior include: daily behavior detection results and estrus behavior detection results, wherein daily behavior includes: lying down, standing, walking, and eating, and estrus behavior includes: mounting.

Images