A cow behavior recognition method based on improved YOLOv8

Abstract

The application discloses a kind of based on improved YOLOv8's cow behavior identification method, comprising: S01 obtains the video image of cow to be identified;S02 the video image of the cow is input into improved YOLOv8 model, obtains identification result;In the improved YOLOv8 model, main network introduces feature enhancement module FEM, separates and handles high-frequency information and low-frequency information to input feature map and fuses;Feature pyramid network introduces improved spatial pyramid pooling module SPPFCSPC-P and multi-scale feature aggregation module MSAA, respectively for expanding receptive field and retaining local fine-grained information, and using different scale convolution kernel parallel extraction multi-scale feature and fusion;Detection head includes classification branch and regression branch, respectively output class probability and boundary box coordinate;S03 output identification result and carry out visual display.The application realizes high-precision behavior identification on self-built data set by the synergistic effect of the above module, significantly improves the precision and robustness of cow behavior identification under complex environment.

Description

Technical Field

[0001] This invention belongs to the technical field of image recognition and analysis, specifically relating to a method for recognizing dairy cow behavior based on an improved YOLOv8. Background Technology

[0002] Cow behavior recognition has significant application value for cow health early warning, reproductive management, abnormal behavior monitoring, precision and efficient cow farming, and animal-computer interaction. Currently, cow behavior recognition mainly employs contact sensor-based methods and machine vision-based non-contact methods. While contact sensor-based methods for collecting cow behavior data are simple to acquire, less susceptible to target occlusion interference, highly stable, and easy to operate, the sensors are expensive and can easily cause stress responses in cows, making it impossible to accurately measure cow behavior data. For example, Deng Mingji proposed a sensor to collect and label various cow behavioral information, thereby effectively identifying multiple cow behaviors. Furthermore, manual feature extraction has drawbacks because it relies on expert experience and prior knowledge; different experts may choose different features.

[0003] With the development of high-definition cameras and deep learning technology, non-contact behavior recognition based on machine vision has become mainstream. Compared to human behavior recognition, cow behavior recognition has great potential for development. However, using deep learning for cow behavior recognition is more challenging than human behavior recognition. For example, cows only have black and white stripes, and their similar appearance makes computer recognition of cows too narrow, leading to model overfitting and the inability to generalize to cow behaviors only in specific situations. Secondly, cows' movements are relatively random compared to humans, making it difficult to collect data on certain behaviors, thus posing a significant challenge to data collection.

[0004] In the field of deep learning object detection, researchers have applied the YOLO series to dairy cow behavior recognition. For example, in 2022, Bai Qiang et al. proposed an improved YOLOv5s multi-scale behavior recognition algorithm for dairy cows, constructing SEPH to identify key features by adding a Transformer attention mechanism and squeezing and excitation modules. Also in 2022, Jessica et al. proposed replacing the YOLOv4 CSPDarknet backbone network with a DRNet structure to achieve the recognition of dairy cow feeding behavior.

[0005] However, the above method still has the following technical problems:

[0006] 1) The backbone network has limited ability to extract detailed features (such as edges and textures) in complex scenes and is easily affected by occlusion.

[0007] 2) Existing deep learning-based methods for recognizing dairy cow behavior suffer from insufficient data and low recognition accuracy, resulting in poor model generalization ability and poor recognition results in real dairy farm environments. Furthermore, there is a lack of relevant research.

[0008] 3) The feature pyramid network has a relatively simple way of fusing multi-scale features, making it difficult to effectively take into account both the global information of large targets and the detailed information of small targets.

[0009] Furthermore, in bounding box localization, traditional loss functions (such as CIoU) are prone to regression oscillations when dealing with large targets, affecting localization accuracy. The Wise-IoU (WIoU) loss function uses "outlier degree" instead of IoU to evaluate anchor box quality, improving detection performance by reducing harmful gradients of low-quality samples. It has been widely used in many fields such as vehicle detection and underwater target detection, but has not yet been applied to localization optimization in dairy cow behavior recognition.

[0010] Therefore, how to achieve high-precision and high-efficiency identification of six behaviors of dairy cows—feeding, drinking, licking, standing, walking, and lying down—in complex real-world environments remains a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0011] This invention aims to overcome the shortcomings of existing technologies and proposes a cow behavior recognition method based on an improved YOLOv8. It solves the problems of low accuracy and insufficient generalization ability of existing models in recognizing various cow behaviors in complex environments. The recognition method of this invention can not only accurately identify cow behaviors in complex scenarios, but also reduce the number of model parameters, providing research value for the subsequent deployment of the model on mobile devices.

[0012] This invention proposes a cow behavior recognition method based on an improved YOLOv8, applicable to cow behavior recognition in complex real-world environments. The recognition method includes the following steps:

[0013] Step (10). Data Acquisition

[0014] Video capture devices (such as cameras, drones, depth cameras, etc.) are used to collect videos of six behaviors of dairy cows at various time periods. These behaviors include eating, drinking, licking, standing, walking, and lying down, ensuring coverage of different lighting conditions, angles, and movement states.

[0015] Step (20). Data Preprocessing

[0016] The acquired video was divided into image frames, and the video image size was then uniformly set to 640×640. Data augmentation of the video frames was performed using techniques such as color perturbation, random rotation, and brightness enhancement to improve the generalization of the model.

[0017] Step (30). Model recognition

[0018] The pre-processed video images are input into a pre-trained improved YOLOv8 model to obtain cow behavior recognition results; wherein, the YOLOv8 model includes a backbone network, a feature pyramid network, and a detection head;

[0019] The backbone network incorporates a feature enhancement module that separates and fuses high-frequency and low-frequency information in the input feature map. The feature pyramid network incorporates an improved spatial pyramid pooling module and a multi-scale feature aggregation module. The improved spatial pyramid pooling module expands the receptive field while preserving local fine-grained information, and the multi-scale feature aggregation module extracts and fuses multi-scale features in parallel using convolutional kernels of different scales. The detection head includes a classification branch and a regression branch, which output class probabilities and bounding box coordinates, respectively.

[0020] Step (40). Use Non-Maximum Suppression (NMS) to filter redundant detection boxes, output cow behavior prediction boxes and category probabilities, and display them visually.

[0021] Preferably, the feature enhancement module FEM processes the high and low frequency information of the feature map separately through upsampling and downsampling.

[0022] Preferably, the feature pyramid network introduces cross-stage spatial pooling SPPFCSPC-P based on partial convolution.

[0023] Preferably, the multi-scale feature aggregation module uses convolutions with kernel sizes of 3*3, 5*5, and 7*7 to extract cow information at different scales and aggregate their features.

[0024] Preferably, the identification method incorporates the WIoU loss function.

[0025] Preferably, the visualization is displayed by drawing detection boxes, key points, and skeleton lines, which are then superimposed on the original image or video frame.

[0026] The technical solution of the present invention has at least the following technical effects:

[0027] 1. This invention systematically improves the YOLOv8 model by introducing a Feature Enhancement Module (FEM) into the backbone network and SPPFCSPC-P and MSAA modules into the feature pyramid network. It also employs the WIoU loss function. These improved modules work synergistically to significantly improve the recognition accuracy and localization accuracy of six cow behaviors in complex environments while maintaining real-time detection speed. Experimental results on a self-built dataset show that the mean accuracy (mAP) of this invention reaches 93.5%, an improvement of 5.4 percentage points compared to the original YOLOv8 (88.1%), and exhibits excellent robustness in complex scenarios such as occlusion and low light conditions.

[0028] 2. This invention embeds a Feature Enhancement Module (FEM) into the C2f module of the backbone network to separate and fuse high- and low-frequency information in the feature map: high-frequency information is preserved and enhanced, enabling the model to effectively identify cow behavioral features even under occlusion; low-frequency information captures long-range dependencies through depthwise separable convolutions, preserving global structural information. The introduction of the FEM significantly improves the model's recognition accuracy in occluded scenarios.

[0029] 3. This invention introduces the SPPFCSPC-P module into the feature pyramid network, which reduces gradient redundancy and improves parameter utilization efficiency through cross-stage partial connections; at the same time, it replaces traditional convolution with partial convolution, which significantly reduces the amount of computation and parameters while expanding the receptive field and capturing multi-scale contextual information.

[0030] 4. This invention only requires data acquisition devices such as cameras to obtain behavioral data, eliminating the need for manual installation of ear tags, greatly reducing manpower, alleviating stress on dairy cows, and significantly improving dairy farming efficiency, thus meeting the needs of dairy cow behavior monitoring in actual farming environments. Attached Figure Description

[0031] Figure 1 This is a flowchart of the identification method of the present invention;

[0032] Figure 2 This is a diagram of the improved YOLOv8n-detect model architecture according to an embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the improved module C2f-FEM structure according to an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram of the SPPFCSPC-P structure according to an embodiment of the present invention;

[0035] Figure 5 This is a schematic diagram of the MSAA module structure according to an embodiment of the present invention;

[0036] Figure 6 This is a visualization of cow behavior in an embodiment of the present invention. Detailed Implementation

[0037] This application provides a method for recognizing dairy cow behavior based on an improved YOLOv8, which solves the problems of low model recognition ability due to similar dairy cow behaviors or mutual occlusion within a dairy cow group, model complexity, insufficient model generalization and low recognition accuracy in complex environments.

[0038] The technical solution in this application aims to address the problem of low model recognition rate for cow behavior in complex environments. The overall solution approach is as follows:

[0039] We improved and optimized various modules of the YOLOv8n model. Specifically, we introduced cross-stage partial connections on top of YOLOv8's original Fast Spatial Pyramid Pooling (SPPF). This module effectively reduces gradient redundancy, improves parameter utilization efficiency, and maintains the stability of gradient flow. Simultaneously, to reduce module complexity, we introduced partial convolution (PConv). This module, combined with SPPFCSPC, can reduce module complexity while preserving the model's ability to extract fine-grained information about cow behavior, thereby enhancing feature representation power.

[0040] In the feature extraction stage, the C2f-FEM module is used to reshape the high and low frequency information of cow behavior features. This effectively integrates and aligns the texture of high-frequency information with the fine-grained information and the overall structure reflected by low-frequency information, thereby reducing the noise of the feature map and preserving the fine-grained information, thus improving the accuracy of the model in recognizing cow behavior in complex environments.

[0041] In the neck stage of YOLOv8, the Multi-Scale Feature Aggregation (MSAA) module is used. MSAA first feeds the feature maps into convolutions with learnable parameters of kernel sizes of 3*3, 5*5, and 7*7, respectively, so that the model learns behavioral information at different scales. At the same time, the learned behavioral information at different scales is fed into the spatial attention module to further refine the cow behavioral features.

[0042] Furthermore, given the relatively large size of the cows, traditional loss functions can lead to unstable bounding box regression, resulting in inaccuracies in the localization of the cows' behavior. Therefore, this embodiment employs a WIoU loss function that generates adaptive weights based on the target size, improving the accuracy of the bounding box and thus better locating the cows.

[0043] In summary, through the improvements and optimizations of the above modules, this embodiment enables the model to more effectively extract features of six dairy cow behaviors, reducing the risk of problems in complex environments. The model has low accuracy in recognizing cow behavior.Impact. In experiments on a self-built dataset, the model achieved better accuracy and recall for six behaviors of dairy cows: standing, walking, lying down, drinking, eating, and licking. It significantly reduced recognition errors caused by target occlusion and complex shapes, fundamentally reducing false positives and false negatives in behavior recognition tasks.

[0044] To better understand the above technical solutions, the following will describe them in detail with reference to the accompanying drawings and specific embodiments. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Example 1

[0046] This application provides a method for recognizing dairy cow behavior based on an improved YOLOv8, such as... Figure 1 As shown: The specific operation is as follows:

[0047] Step (1). Use video capture devices (such as cameras, drones, depth cameras, etc.) to capture videos of six behaviors of dairy cows at different time periods, under different lighting conditions, and from different angles, including eating, drinking, licking, standing, walking, and lying down, to ensure coverage of various movement states and complex scenes. The captured video data is then transmitted to a computer for further processing.

[0048] Step (2). Process the input data.

[0049] (2.1) Divide the video data into continuous image frames and perform data augmentation on the image frames, including flipping, translation, scaling, color adjustment, Mosaic enhancement and other methods. Through data augmentation, simulate new samples with different lighting conditions and backgrounds in different scenes, so that the model can be exposed to more diverse scenes during the training phase, reduce the risk of overfitting, and improve the model's generalization ability and scene adaptability.

[0050] (2.2) Normalize the image size and scale it to 640×640 pixels. At the same time, scale and translate the annotation box according to the same ratio / offset to ensure that the box is still aligned with the image.

[0051] (2.3) Perform pixel value normalization, scaling the pixel values ​​from the range of 0-255 to between 0-1, and standardize the image using the mean and standard deviation. This step can accelerate model convergence, reduce training time, and prevent gradient explosion.

[0052] Step (3). Input the preprocessed cow video images from step (2) into the pre-trained improved YOLOv8 model to obtain the cow behavior recognition results. The YOLOv8 model includes a backbone network, a feature pyramid network (Neck), and a detection head.

[0053] (3.1) Input a raw RGB image: The image is downsampled using a convolution with a stride of 2 and a kernel size of 3*3 to extract features, resulting in... :

[0054]

[0055] Subsequently, deep features of the image are extracted and the receptive field is expanded through multiple stacked convolutional modules and the feature enhancement module (FEM), thereby obtaining semantic information from the deep feature maps. The feature extraction module (C2f) extracts fine-grained features of the image through cross-stage local feature fusion and gradient splitting. The feature enhancement module (FEM) processes high- and low-frequency information of the feature maps separately through upsampling and downsampling, reducing spatial resolution while improving the model's ability to utilize detailed features.

[0056] Downsampling of the input feature map X is achieved using channel splitting and feature augmentation:

[0057]

[0058] FEM is the feature enhancement module. This indicates nonlinear activation.

[0059] The backbone part outputs three feature maps at different scales:

[0060]

[0061] The Feature Enhancement Module (FEM) utilizes upsampling, downsampling, and depthwise separable convolution to process high- and low-frequency information in the image separately. Because high-frequency information contains details such as feature map edges, corners, and textures, it can effectively identify cow behavior features even when the cows are occluded, which positively impacts training performance. Low-frequency information is processed using DWConv and global average pooling to capture long-range dependencies. The processed high- and low-frequency information is then fused, preserving both global information and fine-grained feature map details.

[0062] The overall steps are as follows: First, the input feature map is downsampled using a depthwise separable convolution (DWConv) with a stride of 2 to obtain the low-frequency feature FL. Then, FL is upsampled to the original feature map size Q using bilinear interpolation to preserve the low-frequency information of the original image. Next, Q is multiplied pixel-by-pixel with the input feature X to further refine the low-frequency feature. The high-frequency feature FH is obtained by subtracting the upsampled low-frequency feature Q from the original feature X. Finally, the low-frequency FL and high-frequency FH ​​are concatenated along the channel dimension, and then fused through a projection layer and an activation function to obtain the enhanced feature representation.

[0063] By introducing a depthwise separable convolution with a stride of 2 and a kernel size of 11, the receptive field is significantly expanded at the cost of a small number of parameters, improving the model's ability to recognize the overall feature map and thus enhancing the model's accuracy in recognizing cow behavior.

[0064] The YOLOv8 feature extraction network adopts the CSPDarknet53 network architecture. Based on this, our method incorporates a Feature Enhancement Module (FEM) into C2f to improve the model's feature extraction capabilities, enabling it to maintain high recognition accuracy even in complex scenes. Furthermore, the FEM module effectively enhances the feature representation of key regions by introducing an attention mechanism, while suppressing background noise interference. Experimental results show that the improved model exhibits superior performance in cow behavior recognition tasks, especially demonstrating significant advantages in occluded scenarios.

[0065] (3.2) The feature maps of different scales output by the backbone network are fed into the feature pyramid network, and multi-scale feature fusion is achieved using upsampling, convolution, and other methods. The fast spatial pyramid pooling (SPPFCSPC-P) module is used to capture long-distance contextual information and obtain the receptive fields of different receptive fields. Some convolutions in the module can reduce training time and the number of parameters. The specific operations are as follows:

[0066]

[0067] in This represents a partial convolution with a kernel size of 1*1.

[0068] The SPPFCSPC-P architecture further optimizes the classic Spatial Pyramid Pooling (SPP). Fast Spatial Pyramid Pooling (SPPF) enhances feature fusion by introducing a smaller pooling kernel, achieving a "secondary pooling" mechanism. It employs three identical lightweight pooling modules in parallel and replaces the traditional large kernel design with a small-scale kernel, significantly reducing computational cost and improving inference speed while maintaining accuracy.

[0069] Fast Spatial Pyramid Pooling with Cross-Stage Partial Connections (SPPFCSPC) adds cross-stage connections to SPPF, which reduces gradient flow redundancy, optimizes parameters, and improves utilization efficiency while ensuring the capture of multi-scale features and improving the model's adaptability to target size.

[0070] Finally, we replaced all the traditional convolutions in the SPPFCSPC module with partial convolutions (PConv). PConv performs convolutions only on a subset of the feature maps, leaving the remaining channels unchanged. After the replacement, the model significantly reduced computation and parameter count while maintaining detection performance.

[0071] (3.3) Align and aggregate features of different scales in the backbone network. In the process of upsampling deep features and concatenating shallow features, a multi-scale aggregation network (MSAA) is introduced. This allows the feature maps to pass through convolutional kernels of different sizes, and spatial attention and channel attention are used to focus features at different scales to achieve fusion. The specific operation is as follows:

[0072]

[0073] in Represents depthwise separable convolution. Represents spatial aggregation, This represents channel aggregation.

[0074] The multi-scale aggregation module MSAA utilizes depthwise separable convolutions with kernel sizes of 3*3, 5*5, and 7*7 to process features at different scales, and then fuses them using 1*1 convolutions. These three paths run in parallel, corresponding to small, medium, and large receptive fields, respectively. Spatial aggregation allows the model to focus on the edges, contours, and textures of the target while suppressing background or occluded areas. Channel aggregation enhances useful channels and suppresses redundant channels. A sigmoid function is added after channel aggregation to obtain the weights of each channel, which are then multiplied with the spatially aggregated features to obtain the final feature map. Compared to the simple addition of different scales, this module extracts and aligns key semantic information from features at different scales through channel weighting and spatial attention multiplication, effectively fusing both detailed and global information.

[0075] (3.4) Detector Head and Loss Function Section

[0076] The fused feature map is then fed into the YOLOv8 detection head. The detection head employs a parallel branching structure:

[0077] I. Classification Branch: Predicting the probability of each category;

[0078] II. Regression Branch: Predicts the coordinates of the bounding box [Δx, Δy, Δw, Δh];

[0079] In this embodiment, during the training of the deep learning model, the loss function defines the optimization objective, encouraging the model to learn specific types of features and ensuring that the training objective always moves towards the optimal parameters. The initial loss function of YOLOv8 consists of three parts: classification loss, confidence loss, and localization loss. The localization loss is calculated using CIoU. In this example, the CIoU loss function is replaced with the WIoU loss function to improve the accuracy of the bounding box and better locate the cow.

[0080] The inclusion of the WIoU loss function improves the quality of bounding box regression in cow behavior detection. Its core advantage lies in its dynamic, non-monotonic focusing mechanism, which significantly enhances the model's detection accuracy and generalization ability in complex scenes. Traditional loss functions like CIoU often degrade model performance by imposing excessively strong bounding box regression penalties when dealing with low-quality samples or targets with significant scale variations, which are common in behavior recognition tasks. WIOU replaces traditional IoU with "outlier" to evaluate anchor box quality and employs a dynamic gradient gain allocation strategy. This adaptively assigns larger gradients to high-quality anchor boxes while effectively suppressing harmful gradients from low-quality samples (such as blurred or heavily occluded behavioral targets). This mechanism allows the model to focus more on samples of average quality during training, thereby enhancing generalization performance and localization accuracy without affecting overall convergence speed.

[0081] (4) Post-processing and output

[0082] After the model outputs, non-maximum suppression (NMS) in the YOLOv8 algorithm is used to filter redundant detection boxes and retain the best recognition results:

[0083] (4.1) Detection results: bounding box coordinates ([Δx, Δy, Δw, Δh]);

[0084] (4.2) Visualization rendering: Draw detection boxes on the original image or video frame and output the probability of each behavior category above the detection boxes to realize the visualization rendering of the cow behavior recognition results.

[0085] Example 2

[0086] (1) The performance of the YOLOV8 algorithm in Embodiment 1 of this invention and the mainstream object detection algorithms on the dairy cow behavior recognition task were compared. The same dataset and evaluation index were used in the experiment to ensure the fairness of the comparison. Table 1 is obtained:

[0087] Table 1. Test results of different algorithms

[0088]

[0089] (2) The improved algorithm of Embodiment 1 of the present invention demonstrates the accuracy and recall of recognizing six specific behaviors of dairy cows. The experimental data is based on the test set of a self-built dataset, which includes samples under different lighting, angles and occlusion conditions.

[0090] Table 2. Results of identifying different behaviors in dairy cows

[0091]

[0092] The data in Table 2 show that: (a) walking behavior recognition performed best, with an accuracy of 91.2% and a recall of 87.4%, indicating that the model has a good ability to capture dairy cow behaviors with obvious movement; (b) feeding behavior recognition had an accuracy of 90.6% and a recall of 88.1%, the highest recall among the six behaviors, indicating that the model can effectively identify dairy cow feeding activities; (c) licking behavior recognition had an accuracy of 85.3%, mainly due to the small amplitude of this behavior and its high similarity to other head movements, but it still maintained a recall of 84.9% after introducing the FEM module. The average accuracy of the six behaviors was 88.8%, and the average recall was 86.0%, with overall balanced performance, meeting the monitoring needs of actual farming scenarios.

[0093] (3) This experiment uses the original YOLOv8 as the baseline (Index 1) and gradually introduces SPPFCSPC-P, FEM, WIoU and MSAA modules, and records the precision (P), recall (R) and mean precision (mAP) under each configuration.

[0094] Table 3. Contribution of different modules to the network

[0095]

[0096] The data in Table 3 verify the contribution of each improved module of this invention to the model performance. Overall, all four improved modules proposed in this invention contribute positively to the final performance, and there is good synergy among the modules. The final model (Index 5) achieved an mAP of 93.5%, an improvement of 5.4 percentage points compared to the baseline, with precision and recall reaching 92.4% and 91.9% respectively, verifying the superiority of the technical solution of this invention in the task of recognizing cow behavior in complex environments.

Claims

1. A method for recognizing dairy cow behavior based on an improved YOLOv8, characterized in that, Includes the following steps: S01 Acquire video images of the cows to be identified; S02 Input the cow video images into a pre-trained improved YOLOv8 model to obtain cow behavior recognition results; wherein, the YOLOv8 model includes a backbone network, a feature pyramid network, and a detection head; The backbone network incorporates a feature enhancement module, which separates and fuses high-frequency and low-frequency information from the input feature map. The feature pyramid network incorporates an improved spatial pyramid pooling module and a multi-scale feature aggregation module. The improved spatial pyramid pooling module expands the receptive field while preserving fine-grained local information, and the multi-scale feature aggregation module extracts and fuses multi-scale features in parallel using convolutional kernels of different scales. The detection head includes a classification branch and a regression branch, which output the class probability and bounding box coordinates, respectively. S03 Output the recognition result and display it visually.

2. The dairy cow behavior recognition method as described in claim 1, characterized in that, The feature enhancement module separates and processes the high- and low-frequency information of the feature map through upsampling and downsampling.

3. The dairy cow behavior recognition method as described in claim 1, characterized in that, The feature pyramid network introduces a cross-stage spatial pooling SPPFCSPC-P based on partial convolution.

4. The dairy cow behavior recognition method as described in claim 1, characterized in that, The multi-scale feature aggregation module uses convolutions with kernel sizes of 3*3, 5*5, and 7*7 to extract cow information at different scales and aggregate their features.

5. The method for recognizing dairy cow behavior as described in claim 1, characterized in that, The identification method introduces the WIoU loss function.

6. The method for recognizing dairy cow behavior as described in claim 1, characterized in that, The visualization is achieved by drawing detection boxes, key points, and skeleton lines, which are then superimposed on the original image or video frame.

Images