Deep learning-based sheep rumination behavior identification method and system

Abstract

The invention discloses a sheep rumination behavior recognition method and system based on deep learning, and belongs to the technical field of livestock and poultry healthy breeding, and the method comprises the steps: obtaining video data for sheep rumination behavior recognition, constructing a training set, and training a YOLOv8 target detection model; video data to be recognized are converted into video frames, the video frames are input into the trained YOLOv8 target detection model, a detection frame and features of a sheep head image with an ID mark frame in the video frames are obtained, the position and state of the next frame of a target are predicted through a Deepsort target tracking algorithm, optimal matching is carried out on the multiple targets of the front frame and the rear frame, and the target detection accuracy is improved. Obtaining a plurality of target updated detection frames and ID numbers; by judging that the interval between the current frame and the previous frame exceeds the preset frame number, when the interval exceeds the preset frame number, the current frame is reserved, all previous frames are removed, a sheep rumination classification video data set is constructed, and rumination behaviors are identified through a constructed neural network model. The method can accurately identify the ruminant behavior of the sheep.

Description

Technical Field

[0001] The present invention relates to the technical field of healthy livestock and poultry breeding, and more specifically to a method and system for identifying sheep rumination behavior based on deep learning. Background Art

[0002] With the development of scale and intensive sheep farming, accurate monitoring of sheep behavior information has become crucial for assessing their physiological health. Timely detection of abnormal sheep behavior and taking treatment measures can reduce the economic losses of farms to a certain extent. Rumination is a behavior unique to rumen animals such as cattle and sheep. Among them, sheep rumination monitoring is a key factor in health monitoring. Changes in rumination time are closely related to the health status and physiological state of sheep. Healthy sheep rumination is continuous and vigorous, while sick sheep rumination is slow and even stops in severe cases. After treatment, the gradual recovery of rumination is an important indicator of improvement.

[0003] At present, the traditional method of monitoring rumination behavior mainly relies on manual observation by farmers or contact devices equipped with sensors. The contact devices mainly include pressure sensors, acoustic sensors, and triaxial acceleration sensors. Although contact devices can accurately reflect rumination behavior information to a certain extent, wearing them on animals may induce stress reactions, which in turn has an adverse effect on animal welfare. In addition, the high cost and fragility of contact devices also limit their application in large-scale farming. In this context, it is more appropriate to use computer vision methods to automatically detect and identify the rumination behavior of sheep without contacting the animal's body.

[0004] However, the existing technology of using computer vision to study sheep rumination behavior mainly uses traditional computer vision methods, which have limitations in utilizing the rich information in large amounts of video data, cannot achieve automatic learning and optimization, have poor adaptability in different environments and conditions, and cannot accurately identify sheep rumination behavior. Summary of the invention

[0005] In response to the problems existing in the above-mentioned fields, the present invention proposes a sheep rumination behavior recognition method and system based on deep learning. By combining the YOLOv8 target detection model and the Deepsort target tracking algorithm, a sheep rumination classification video dataset is constructed, which can capture more temporal information, thereby being able to more accurately identify and distinguish rumination behavior.

[0006] In order to solve the above technical problems, the present invention discloses a sheep rumination behavior recognition method based on deep learning, comprising the following steps:

[0007] Obtain video data for sheep rumination behavior recognition, capture multiple sheep images in the video and select key frame images for annotation, construct a training set, train the YOLOv8 target detection model, and obtain the trained YOLOv8 target detection model;

[0008] The video data to be identified is converted into video frames, and the trained YOLOv8 target detection model is input to obtain the detection frame and features of the sheep head image with the ID mark frame in the video frame; based on the detection frame and features, the Deepsort target tracking algorithm is used to predict the next frame position and state of the target, and multiple targets in the previous and next frames are optimally matched, and the tracking frames and ID numbers of multiple targets are output and updated to obtain the updated detection frames and ID numbers of multiple targets;

[0009] According to the updated detection frame and ID number, by judging whether the interval between the current frame and the previous frame exceeds the preset number of frames, when the interval between the current frame and the previous frame exceeds the preset number of frames, the current frame is retained and all previous frames are cleared to construct a sheep rumination classification video dataset;

[0010] The sheep rumination classification video dataset is input into the constructed neural network model to identify the rumination behavior.

[0011] Preferably, the step of obtaining the detection frame and features of the sheep head image with the ID mark frame in the video frame comprises the following steps:

[0012] Convert the input video data into video frames through the video processing library;

[0013] The trained YOLOv8 target detection model is used to analyze the video frames, extract the detection frames and deep features of the sheep head images with ID mark frames in the video frames, and use the non-maximum suppression algorithm to remove similar detection frames.

[0014] Preferably, the outputting of the tracking frames and ID numbers of the multiple targets specifically includes:

[0015] The sheep head image with the ID mark frame in the extracted video frame is used as the detection target. The position and state of the detection target in the continuous video frames are tracked by Kalman filtering, and the detection frame provided by the detector is compared with the result predicted by the Kalman filtering model.

[0016] Match the targets in the previous frame and the current frame to obtain a matching combination, and calculate the matching cost through appearance information, Mahalanobis distance or IOU;

[0017] With the goal of minimizing the total cost, the optimal match is found through the Hungarian algorithm, the movement of the target is tracked, and the tracking box and ID number of each detected target are output.

[0018] Preferably, obtaining updated detection frames and ID numbers of multiple targets specifically includes:

[0019] When the target moves or is covered for a long time, the ID number of the detected target is updated through the re-identification model of the Deepsort target tracking algorithm;

[0020] By extracting the feature vector of the detection box of each detected target, the nearest L k = 100 frames of vector Calculate the minimum cosine distance between the vector and the feature vector of the i-th detection frame in the current frame, and update the detection frame and ID number of each detection target;

[0021] The calculation formula for the minimum cosine distance is:

[0022]

[0023] Preferably, constructing a sheep rumination classification video dataset specifically includes:

[0024] According to the updated detection frames and ID numbers of multiple targets, the sliding window method is used to set each window to 32 frames and slide forward at a certain step length to generate multiple overlapping frame sequences;

[0025] Based on the sheep head images continuously tracked by multiple overlapping frame sequences, when the interval between the current frame and the previous frame exceeds 3 frames, only the current frame is retained and all previous frames are cleared, and a continuous head motion video of each sheep is generated and adjusted to a fixed size of pixels;

[0026] Through manual annotation, these head movement videos are divided into rumination behavior and non-rumination behavior, and the videos are divided into video clips of 60 frames each. The head videos of these video clips are flipped left and right to expand the data set.

[0027] Preferably, the constructed neural network model is a C3D convolutional neural network model, which is used to perform real-time analysis on the behavior of sheep in the video. When the sheep is detected to be ruminating, a chewing label is added, and the rumination time is calculated by the frame difference method; when the sheep is detected to be in non-ruminating behavior, a normal label is added and no timing is performed.

[0028] Preferably, when the sheep is detected to be ruminating, the steps include:

[0029] Record the index of the current frame as the start time;

[0030] When the start time already exists, the duration of the current rumination behavior is obtained by calculating the difference between the current frame index and the start time and dividing it by the frame rate:

[0031]

[0032] Among them, Ruminating Time is the duration of rumination behavior, Current Frame Index is the frame number of the current frame, Starting Frame Index is the frame number of the starting frame, and FPS is the frame rate.

[0033] Preferably, when the sheep is detected to be in non-rumination behavior, the steps include:

[0034] When the start time of rumination has been recorded before, the current interval time is obtained by calculating the difference between the current frame index and the start time and dividing it by the frame rate, and the start time is cleared to prepare for the next timing and accumulate it with the subsequent interval time;

[0035] The interval time is:

[0036]

[0037] Among them, Interval Time is the interval time, CurrentFrame Index is the frame number of the current frame, StartingFrameIndex is the frame number of the starting frame, and FPS is the frame rate.

[0038] Preferably, the method further comprises adding the duration and interval time of the rumination behavior of each sheep to obtain the total behavior time of the sheep, and displaying the total behavior time on the sheep's mark frame in real time;

[0039] The total behavior time is:

[0040] Total Time=Ruminating Time+interval Time

[0041] Among them, Total Time is the total behavior time, Ruminating Time is the duration of rumination behavior, and Interval Time is the interval time.

[0042] Preferably, a sheep rumination behavior recognition system based on deep learning is also included, comprising:

[0043] The YOLOv8 target detection module is used to obtain video data for sheep rumination behavior recognition, capture multiple sheep images in the video and select key frame images for annotation, construct a training set, train the YOLOv8 target detection model, and obtain the trained YOLOv8 target detection model;

[0044] The Deepsort target tracking module is used to convert the video data to be identified into video frames, input the trained YOLOv8 target detection model, and obtain the detection frame and features of the sheep head image with the ID mark frame in the video frame; according to the detection frame and features, the Deepsort target tracking algorithm is used to predict the next frame position and state of the target, optimally match multiple targets in the previous and next frames, output the tracking frames and ID numbers of multiple targets, and update them to obtain the updated detection frames and ID numbers of multiple targets;

[0045] The sheep rumination classification video data set construction module is used to construct the sheep rumination classification video data set according to the updated detection frame and ID number by judging whether the interval between the current frame and the previous frame exceeds the preset number of frames. When the interval between the current frame and the previous frame exceeds the preset number of frames, the current frame is retained and all previous frames are cleared;

[0046] The sheep rumination behavior recognition module is used to input the sheep rumination classification video dataset into the constructed neural network model to identify the rumination behavior.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention proposes a method for identifying sheep rumination behavior based on deep learning. The sheep rumination behavior is mainly reflected in the movement of the sheep's jaw. Therefore, before identifying the rumination behavior, it is necessary to accurately track the sheep's head. This method combines the YOLOv8 target detection model with the Deepsort target detection algorithm, assigns a unique detection frame and ID number to the sheep's head image in the video and dynamically tracks it, crops the tracked sheep's head image, and updates the detection frame and ID number of each sheep's head image by optimally matching multiple targets in the previous and next frames of the video. The Deepsort target detection algorithm significantly improves the target recognition and data association performance in complex situations such as changes in viewing angles, changes in lighting conditions, and occlusion by using appearance descriptors generated by deep learning. After collecting the sheep head images continuously tracked by the YOLOv8 target detection model and the Deepsort target tracking algorithm, in order to ensure the real-time and accuracy of tracking, when the interval between the current frame and the previous frame exceeds the preset number of frames, only the current frame will be retained and all previous frames will be cleared to construct a rumination classification dataset. This can ensure that the frame sequence input to the model is continuous and accurate, avoid behavioral misjudgment caused by frame sequence interruption, and thus improve the accuracy of sheep rumination behavior recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of the sheep rumination behavior identification method based on deep learning proposed by the present invention;

[0050] Figure 2Public dataset images and video key frame images collected for the experiments set up by the present invention;

[0051] Figure 3 The effect diagram of labeling the collected sheep head images in the experiment set up for the present invention;

[0052] Figure 4 The detection effect diagram of the collected sheep head images in the experiment set up by the present invention;

[0053] Figure 5 A detection and tracking flow chart of the combination of the YOLOv8 target detection model and the Deepsort target tracking algorithm in the experimental process set up for the present invention;

[0054] Figure 6 The results of training the re-identification module of the Deepsort target tracking algorithm used in the experiment set by the present invention on the sheep head re-identification dataset;

[0055] Figure 7 Example diagrams of sheep head tracking at different times provided for the experiment set up by the present invention;

[0056] Figure 8 A network architecture diagram of a C3D neural network model provided for the experiment set up by the present invention;

[0057] Fig. 9 A line graph showing the accuracy of the C3D neural network model training in the experiment set up for the present invention;

[0058] Fig.10 A line graph of the loss value of the C3D neural network model training for the experiment set up for the present invention;

[0059] Fig.11 A video rumination behavior detection effect diagram obtained by the experimental process set up for the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the attached embodiment of the present invention Figure 1-Figure 11 , the technical solutions in the embodiments of the present invention are clearly and completely described. It should be understood that the terms described in the present invention are only used to describe specific implementation methods and are not used to limit the present invention.

[0061] like Figure 1 As shown, the present invention proposes a sheep rumination behavior recognition method based on deep learning, comprising the following steps:

[0062] Obtain video data for sheep rumination behavior recognition, capture multiple sheep images in the video and select key frame images for annotation, construct a training set, train the YOLOv8 target detection model, and obtain the trained YOLOv8 target detection model;

[0063] The video data to be identified is converted into video frames, and the trained YOLOv8 target detection model is input to obtain the detection frame and features of the sheep head image with the ID mark frame in the video frame; based on the detection frame and features, the Deepsort target tracking algorithm is used to predict the next frame position and state of the target, and multiple targets in the previous and next frames are optimally matched, and the tracking frames and ID numbers of multiple targets are output and updated to obtain the updated detection frames and ID numbers of multiple targets;

[0064] According to the updated detection frame and ID number, by judging whether the interval between the current frame and the previous frame exceeds the preset number of frames, when the interval between the current frame and the previous frame exceeds the preset number of frames, the current frame is retained and all previous frames are cleared to construct a sheep rumination classification video dataset;

[0065] The sheep rumination classification video dataset is input into the constructed neural network model to identify the rumination behavior.

[0066] Specifically, obtaining the detection frame and features of the sheep head image with the ID mark frame in the video frame includes the following steps:

[0067] Convert the input video data into video frames through the video processing library;

[0068] The trained YOLOv8 target detection model is used to analyze the video frames, extract the detection frames and deep features of the sheep head images with ID mark frames in the video frames, and use the non-maximum suppression algorithm to remove similar detection frames.

[0069] Output the tracking boxes and ID numbers of multiple targets, including:

[0070] The sheep head image with the ID mark frame in the extracted video frame is used as the detection target. The position and state of the detection target in the continuous video frames are tracked by Kalman filtering, and the detection frame provided by the detector is compared with the result predicted by the Kalman filtering model.

[0071] Match the targets in the previous frame and the current frame to obtain a matching combination, and calculate the matching cost through appearance information, Mahalanobis distance or IOU;

[0072] With the goal of minimizing the total cost, the optimal match is found through the Hungarian algorithm, the movement of the target is tracked, and the tracking box and ID number of each detected target are output.

[0073] Get the updated detection boxes and ID numbers of multiple targets, including:

[0074] When the target moves or is covered for a long time, the ID number of the detected target is updated through the re-identification model of the Deepsort target tracking algorithm;

[0075] By extracting the feature vector of the detection box of each detected target, the nearest L k = 100 frames of vector Calculate the minimum cosine distance between the vector and the feature vector of the i-th detection frame in the current frame, and update the detection frame and ID number of each detection target;

[0076] The calculation formula for the minimum cosine distance is:

[0077]

[0078] Construct a sheep rumination classification video dataset, including:

[0079] According to the updated detection frames and ID numbers of multiple targets, the sliding window method is used to set each window to 32 frames and slide forward at a certain step length to generate multiple overlapping frame sequences;

[0080] Based on the sheep head images continuously tracked by multiple overlapping frame sequences, when the interval between the current frame and the previous frame exceeds 3 frames, only the current frame is retained and all previous frames are cleared, and a continuous head motion video of each sheep is generated and adjusted to a fixed size of pixels;

[0081] Through manual annotation, these head movement videos are divided into rumination behavior and non-rumination behavior, and the videos are divided into video clips of 60 frames each. The head videos of these video clips are flipped left and right to expand the data set.

[0082] The constructed neural network model is a C3D convolutional neural network model, which is used to perform real-time analysis of the sheep behavior in the video. When the sheep is detected to be ruminating, it is labeled as chewing and the rumination time is calculated using the frame difference method. When the sheep is detected to be non-ruminating, it is labeled as normal and no timing is performed.

[0083] When sheep are detected to be ruminating, the following steps are included:

[0084] Record the index of the current frame as the start time;

[0085] When the start time already exists, the duration of the current rumination behavior is obtained by calculating the difference between the current frame index and the start time and dividing it by the frame rate:

[0086]

[0087] Among them, Ruminating Time is the duration of rumination behavior, CurrentFrameIndex is the frame number of the current frame, StartingFrameIndex is the frame number of the starting frame, and FPS is the frame rate.

[0088] When the sheep is detected to be in non-rumination behavior, the following steps are included:

[0089] When the start time of rumination has been recorded before, the current interval time is obtained by calculating the difference between the current frame index and the start time and dividing it by the frame rate, and the start time is cleared to prepare for the next timing and accumulate it with the subsequent interval time;

[0090] The interval time is:

[0091]

[0092] Among them, Interval Time is the interval time, CurrentFrame Index is the frame number of the current frame, StartingFrameIndex is the frame number of the starting frame, and FPS is the frame rate.

[0093] The method also includes adding the duration and interval time of the rumination behavior of each sheep to obtain the total behavior time of the sheep, and displaying it in real time on the marking box of the sheep;

[0094] The total behavior time is:

[0095] Total Time=Ruminating Time+interval Time

[0096] Among them, Total Time is the total behavior time, Ruminating Time is the duration of rumination behavior, and Interval Time is the interval time.

[0097] The present invention also proposes a sheep rumination behavior recognition system based on deep learning, comprising:

[0098] The YOLOv8 target detection module is used to obtain video data for sheep rumination behavior recognition, capture multiple sheep images in the video and select key frame images for annotation, construct a training set, train the YOLOv8 target detection model, and obtain the trained YOLOv8 target detection model;

[0099] The Deepsort target tracking module is used to convert the video data to be identified into video frames, input the trained YOLOv8 target detection model, and obtain the detection frame and features of the sheep head image with the ID mark frame in the video frame; according to the detection frame and features, the Deepsort target tracking algorithm is used to predict the next frame position and state of the target, optimally match multiple targets in the previous and next frames, output the tracking frames and ID numbers of multiple targets, and update them to obtain the updated detection frames and ID numbers of multiple targets;

[0100] The sheep rumination classification video data set construction module is used to construct the sheep rumination classification video data set according to the updated detection frame and ID number by judging whether the interval between the current frame and the previous frame exceeds the preset number of frames. When the interval between the current frame and the previous frame exceeds the preset number of frames, the current frame is retained and all previous frames are cleared;

[0101] The sheep rumination behavior recognition module is used to input the sheep rumination classification video dataset into the constructed neural network model to identify the rumination behavior.

[0102] This method uses a target detection and tracking method that combines the YOLOv8 target detection model with the Deepsort target tracking algorithm to identify the rumination behavior of sheep, thereby improving the recognition accuracy.

[0103] Experimental analysis

[0104] In order to verify the feasibility of the method proposed in the present invention, the video data of the breeding farms in Hohhot City, Inner Mongolia Autonomous Region was collected as an example to verify the sheep rumination behavior recognition method based on deep learning proposed in the present invention.

[0105] In order to meet the real farm environment, this experiment used screws to fix the monitor bracket on the wall to collect videos. The video frame rate was 30 frames / s, and 6 rumination videos were screened out. Among them, videos 1, 2, and 3 were taken during the day with sufficient light; videos 4, 5, and 6 were taken at night with dim light. In the screened videos, there were multiple sheep ruminating, and there were also sheep that did not ruminate (shaking their heads, staying still) for some time. The statistics of rumination information videos are shown in Table 1.

[0106] Table 1 Manual statistics of rumination video information

[0107]

[0108]

[0109] This experiment collects 3000 pictures, including video key frame images and public dataset pictures. Figure 2 As shown in Figure (a), the video key frame image is Figure 2 As shown in Figure (b).

[0110] Use the image labeling software labelImg to manually draw the sheep's head label box and generate the corresponding text file. The effect of labeling the collected sheep head images is shown in the figure below. Figure 3 As shown in Figure 2, the images are randomly divided into training set, validation set and test set in a ratio of 8:1:1. The YOLOv8 target detection model is used for training.

[0111] The structure diagram of the YOLOv8 target detection model is as follows Figure 1 The system environment for training the YOLOv8 target detection model is: Windows 10 operating system, using Python 3.9.16 and torch 2.0.0+cu118 deep learning framework, CPU is Intel(R) Core(TM) i5-13400F, graphics card is NVIDIA GeForceRTX 4060Ti, memory is 8GB. The detection effect of the sheep head image is shown in the figure below. Figure 4 shown.

[0112] The rumination behavior of sheep is mainly reflected in the movement of the sheep's jaw. Before identifying the rumination behavior, the sheep's head must be accurately tracked. The Deepsort target tracking algorithm significantly improves the accuracy and robustness of target tracking by introducing deep learning technology, combining deep feature extraction, Kalman filtering and Hungarian algorithm. By using the appearance descriptors generated by deep learning, the target recognition and data association performance are significantly improved in complex situations such as changes in viewing angles, changes in lighting conditions and occlusion.

[0113] In this experiment, the YOLOv8 target detection model and the Deepsort target tracking algorithm are combined to track the head of the sheep in the video. The detection and tracking flow chart of the combination of the YOLOv8 target detection model and the Deepsort target tracking algorithm is shown in the figure. Figure 5 As shown, it mainly includes the following steps:

[0114] (1) The input video is converted into video frames by the video processing library. The YOLOv8 target detection model is used to analyze the video frames, extract the deep features of the target to be detected, and use the non-maximum suppression algorithm NMS to remove similar detection frames to obtain the final detection results.

[0115] (2) Using the Kalman filter algorithm, the position and state of the object in the continuous video frames are tracked, and the detection frame provided by the detector is compared with the result predicted by the Kalman filter model to obtain the most accurate prediction result. During the processing, the detection frame and tracking model can be continuously updated to grasp the latest data changes and improve the accuracy of the prediction.

[0116] (3) Match the targets in the previous frame and the current frame to obtain possible matching combinations, and use the Mahalanobis distance indicator to calculate the matching cost.

[0117] The optimal match is found through the Hungarian algorithm to minimize the total cost and track the movement of the target.

[0118] When the target moves or is covered for a long time, the re-identification model plays an important role in reducing the occurrence of ID switching.

[0119] This experiment uses the re-identification module in the Deepsort target tracking algorithm to train the sheep head re-identification dataset. The training results are as follows: Figure 6 As shown in the figure, during the training process, the loss and error gradually decrease and tend to be stable as the number of training times increases, and finally a re-identification model is obtained to reduce the ID jump problem caused by target disappearance or coverage during tracking.

[0120] Experiments show that this method performs well in the sheep head tracking task. In Video 3, when the sheep numbered 1 moves briefly and shakes its head, the system can still accurately track its head, and the ID number does not jump. Figure 7 The figure also shows the head tracking effect of several frames during the process of the sheep turning its head, that is, the tracking effect diagram of the sheep's head at different times.

[0121] This experiment uses the C3D convolutional neural network model to identify the rumination behavior of sheep. The head motion images pre-processed by early tracking are input into the pre-trained C3D convolutional neural network model. The model analyzes the input video frames and combines its ability to extract spatiotemporal features to accurately predict whether the sheep is ruminating. When the model processes a sufficient number of image frames, it can generate high-precision recognition results for rumination based on the patterns and features learned during training.

[0122] The identification of sheep rumination behavior is carried out through the C3D convolutional neural network model, which includes the following steps:

[0123] Rumination classification dataset construction

[0124] In the selected videos, the head movement video segment of each sheep was captured and resized to a fixed size of 256x256 pixels, and each head movement video segment was saved separately. Through manual annotation, these head movement videos were divided into two categories: rumination behavior (chew) and non-rumination behavior (normal). In order to ensure the diversity of the dataset, the longer videos were divided into video segments of 60 frames each, and the head videos of these segments were flipped left and right to expand the rumination classification dataset.

[0125] A total of 27 sheep house videos were collected in this experiment. The final constructed rumination classification dataset contains 428 rumination behavior videos (chew) and 388 non-rumination behavior videos (normal), providing rich and diverse data support for subsequent model training and evaluation.

[0126] The C3D convolutional neural network model was proposed by the research team of Facebook Artificial Intelligence Research Institute FAIR and has a wide range of application prospects, including action recognition, scene recognition, video similarity analysis and other fields. The network structure of the C3D convolutional neural network model is as follows: Figure 8 As shown in the figure, it contains 5 convolutional layers and 5 pooling layers, 2 fully connected layers and 1 softmax loss layer. It is used to predict action labels. Each convolutional layer is followed by a pooling layer. The number of filters in the convolutional layer is 64, 128, 256, 512 and 512, respectively. A total of 8 convolution operations and 5 pooling operations are performed. The high-level spatiotemporal features of the video are gradually extracted through multi-layer convolution and pooling. The fully connected layer further processes these features and converts the output into a category probability distribution through the softmax layer to achieve accurate action label prediction.

[0127] C3D convolutional neural network model training was carried out under Windows 10 operating system. The hardware configuration was Intel(R) Core(TM) i5-13400F CPU, NVIDIA GeForce RTX 4060Ti graphics card and 8GB memory. 30 epochs were set during the training process, and the cross entropy loss function (Cross Entropy Loss) was used to measure the performance of the model. In order to accelerate convergence and prevent overfitting, the optimizer selected stochastic gradient descent (SGD) and combined with the StepLR learning rate scheduler to gradually reduce the learning rate. The training results include accuracy and loss rate, such as Fig. 9 The following is a line chart showing the accuracy of the C3D convolutional neural network model training. Fig.10 Shown is a line chart of the loss value of the C3D convolutional neural network model training.

[0128] In the field of behavior recognition, the C3D convolutional neural network model usually uses 16 frames of input to extract the spatiotemporal features of the video. However, in actual applications, it is found that 16 frames may not be enough for sheep rumination behavior recognition. In order to improve the accuracy and reliability of recognition, this experiment increases the number of C3D model input frames to 32 frames.

[0129] First, a 16-frame video sequence covers a time span of 0.53 seconds at a frame rate of 30 frames per second. For some behaviors, this time span is sufficient to capture key features. However, rumination behavior has obvious periodicity and complex temporal changes. A time span of only 0.53 seconds may not be able to fully capture the complete information of the entire rumination process, resulting in a decrease in recognition accuracy.

[0130] Secondly, the number of frames was increased to 32, extending the time span to 1.06 seconds. This adjustment has significant advantages in capturing the continuity and details of the behavior. Rumination behavior is manifested as a series of continuous actions in time, and a longer time span can capture more temporal information, thereby helping the model to more accurately identify and distinguish rumination behavior.

[0131] This experiment achieved accurate calculation of the sheep's rumination time by combining the sliding window strategy with the frame difference method. The combination of the sliding window strategy and the frame difference method enhances the detection capability of the system. Specifically, on the basis of using 32 frames, each window contains 32 frames and slides forward at a certain step length to generate a series of overlapping frame sequences. In this way, each behavior segment is detected multiple times, thereby reducing the possibility of missed detection.

[0132] The frame difference method can capture tiny motion changes in rumination behavior by calculating the pixel differences between consecutive frames, providing more accurate behavior detection and timing functions.

[0133] After collecting images continuously tracked by the YOLOv8 target detection model and the Deepsort target tracking algorithm, in order to ensure the real-time and accuracy of tracking, when the interval between the current frame and the previous frame exceeds 3 frames, only the current frame will be retained and all previous frames will be cleared. The advantage of this approach is that it can ensure that the frame sequence input to the model is continuous and accurate, avoiding misjudgment of behavior due to interrupted frame sequences.

[0134] When the 32-frame condition is met, the model will determine whether the sheep is ruminating. If rumination is detected, it will be marked as chew; if not, it will be marked as normal.

[0135] In order to ensure the continuity and real-time nature of behavior judgment, new image frames will continue to be collected and the first frame of image collected will be deleted, thus ensuring that there are always 32 frames for behavior judgment.

[0136] When the sheep is detected to be ruminating, the duration of the rumination behavior is calculated. For the interval timing of normal behavior, when the sheep is detected to be in non-rumination behavior, the interval time is calculated. The rumination time and the interval time of each sheep are added to obtain the total behavior time of the sheep.

[0137] This total time is displayed in real time on the sheep's tag frame for continuous monitoring and recording. This method ensures accurate capture and real-time feedback of behavioral data, providing a reliable basis for behavioral analysis.

[0138] Experimental results analysis

[0139] The effect diagram of video rumination behavior detection obtained in this experiment is as follows Fig.11 shown.

[0140] The rumination time error calculated in this experiment is:

[0141]

[0142] Among them, Total Time is the total rumination behavior time of the sheep in the video detection, and Observation Time is the total rumination behavior time of the sheep observed manually.

[0143] The calculation formula for the average error of rumination time is:

[0144]

[0145] In this experiment, the collected videos were tested to obtain the rumination time of each sheep, which was compared with the rumination time obtained by manual observation. The rumination time error data obtained are shown in Table 2.

[0146] Table 2 Statistics of rumination information errors

[0147]

[0148]

[0149] This experiment provides a sheep rumination behavior recognition method based on deep learning, and tests it in videos under different environments. The experimental results show that the average error of the rumination time under daytime and nighttime lighting conditions is 4.403% and 6.156% respectively, both showing high detection accuracy, proving that the method has high accuracy and good real-time performance when dealing with complex scenes, can effectively identify the rumination behavior of sheep, and can meet the requirements of automatic monitoring of sheep rumination behavior recognition in farms.

[0150] In summary, the recognition method proposed in the present invention shows that the rumination behavior of sheep is mainly reflected in the movement of the sheep's jaw. Therefore, before recognizing the rumination behavior, it is necessary to accurately track the sheep's head. This method combines the YOLOv8 target detection model with the Deepsort target detection algorithm, assigns a unique detection frame and ID number to the sheep's head image in the video and tracks it dynamically, crops the tracked sheep's head image, and updates the detection frame and ID number of each sheep's head image by optimally matching multiple targets in the previous and next frames of the video. The Deepsort target detection algorithm significantly improves the target recognition and data association performance in complex situations such as changes in viewing angles, changes in lighting conditions, and occlusion by using appearance descriptors generated by deep learning. After collecting the sheep head images continuously tracked by the YOLOv8 target detection model and the Deepsort target tracking algorithm, in order to ensure the real-time and accuracy of tracking, when the interval between the current frame and the previous frame exceeds the preset number of frames, only the current frame will be retained and all previous frames will be cleared to construct a rumination classification dataset. This can ensure that the frame sequence input to the model is continuous and accurate, avoid behavioral misjudgment caused by frame sequence interruption, and thus improve the accuracy of sheep rumination behavior recognition.

[0151] This method can reduce human resource costs to a certain extent, and help farmers detect abnormal behavior of sheep in a timely manner by accurately monitoring rumination behavior. It can improve animal welfare, provide reliable technical support for abnormal behavior detection and remote diagnosis of smart ranches, and promote the development of animal husbandry towards intelligence.

[0152] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

[0153] In addition, unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as commonly understood by those skilled in the art to which the present invention belongs. All documents mentioned in this specification are incorporated by reference to disclose and describe the methods related to the documents. In the event of any conflict with any incorporated document, the content of this specification shall prevail.

Claims

1. A method for identifying sheep rumination behavior based on deep learning, characterized in that: The following steps are involved: Obtain video data for sheep rumination behavior recognition, capture multiple sheep images in the video and select key frame images for annotation, construct a training set, train the YOLOv8 target detection model, and obtain the trained YOLOv8 target detection model; The video data to be identified is converted into video frames, and the trained YOLOv8 target detection model is input to obtain the detection frame and features of the sheep head image with the ID mark frame in the video frame; based on the detection frame and features, the Deepsort target tracking algorithm is used to predict the next frame position and state of the target, and multiple targets in the previous and next frames are optimally matched, and the tracking frames and ID numbers of multiple targets are output and updated to obtain the updated detection frames and ID numbers of multiple targets; According to the updated detection frame and ID number, by judging whether the interval between the current frame and the previous frame exceeds the preset number of frames, when the interval between the current frame and the previous frame exceeds the preset number of frames, the current frame is retained and all previous frames are cleared to construct a sheep rumination classification video dataset; The sheep rumination classification video dataset is input into the constructed neural network model to identify the rumination behavior.

2. The sheep rumination behavior recognition method based on deep learning according to claim 1 is characterized in that: The method of obtaining the detection frame and features of the sheep head image with the ID mark frame in the video frame includes the following steps: Convert the input video data into video frames through the video processing library; The trained YOLOv8 target detection model is used to analyze the video frames, extract the detection frames and deep features of the sheep head images with ID mark frames in the video frames, and use the non-maximum suppression algorithm to remove similar detection frames.

3. The sheep rumination behavior recognition method based on deep learning according to claim 2 is characterized in that: The output of the tracking frames and ID numbers of the multiple targets specifically includes: The sheep head image with the ID mark frame in the extracted video frame is used as the detection target. The position and state of the detection target in the continuous video frames are tracked by Kalman filtering, and the detection frame provided by the detector is compared with the result predicted by the Kalman filtering model. Match the targets in the previous frame and the current frame to obtain a matching combination, and calculate the matching cost through appearance information, Mahalanobis distance or IOU; With the goal of minimizing the total cost, the optimal match is found through the Hungarian algorithm, the movement of the target is tracked, and the tracking box and ID number of each detected target are output.

4. The sheep rumination behavior recognition method based on deep learning according to claim 3 is characterized in that: The obtaining of the updated detection frames and ID numbers of the multiple targets specifically includes: When the target moves or is covered for a long time, the ID number of the detected target is updated through the re-identification model of the Deepsort target tracking algorithm; By extracting the feature vector of the detection box of each detected target, the nearest L k = 100 frames of vector Calculate the minimum cosine distance between the vector and the feature vector of the i-th detection frame in the current frame, and update the detection frame and ID number of each detection target; The calculation formula for the minimum cosine distance is:

5. The sheep rumination behavior recognition method based on deep learning according to claim 4 is characterized in that: The step of constructing a sheep rumination classification video dataset specifically includes: According to the updated detection frames and ID numbers of multiple targets, the sliding window method is used to set each window to 32 frames and slide forward at a certain step length to generate multiple overlapping frame sequences; Based on the sheep head images continuously tracked by multiple overlapping frame sequences, when the interval between the current frame and the previous frame exceeds 3 frames, only the current frame is retained and all previous frames are cleared, and a continuous head motion video of each sheep is generated and adjusted to a fixed size of pixels; Through manual annotation, these head movement videos are divided into rumination behavior and non-rumination behavior, and the videos are divided into video clips of 60 frames each. The head videos of these video clips are flipped left and right to expand the data set.

6. The sheep rumination behavior recognition method based on deep learning according to claim 5, characterized in that: The constructed neural network model is a C3D convolutional neural network model, which is used to perform real-time analysis on the behavior of sheep in the video. When the sheep is detected to be ruminating, a chewing label is added, and the rumination time is calculated by the frame difference method; when the sheep is detected to be in non-ruminating behavior, a normal label is added and no timing is performed.

7. The sheep rumination behavior recognition method based on deep learning according to claim 6, characterized in that: When the sheep is detected to be ruminating, the method comprises the following steps: Record the index of the current frame as the start time; When the start time already exists, the duration of the current rumination behavior is obtained by calculating the difference between the current frame index and the start time and dividing it by the frame rate: Among them, Ruminating Time is the duration of rumination behavior, CurrentFrameIndex is the frame number of the current frame, StartingFrameIndex is the frame number of the starting frame, and FPS is the frame rate.

8. The sheep rumination behavior recognition method based on deep learning according to claim 7, characterized in that: When the sheep is detected to be in non-rumination behavior, the method comprises the following steps: When the start time of rumination has been recorded before, the current interval time is obtained by calculating the difference between the current frame index and the start time and dividing it by the frame rate, and the start time is cleared to prepare for the next timing and accumulate it with the subsequent interval time; The interval time is: Among them, Interval Time is the interval time, CurrentFrame Index is the frame number of the current frame, StartingFrameIndex is the frame number of the starting frame, and FPS is the frame rate.

9. The sheep rumination behavior recognition method based on deep learning according to claim 8, characterized in that: The method also includes adding the duration and interval time of the rumination behavior of each sheep to obtain the total behavior time of the sheep, and displaying it in real time on the marking box of the sheep; The total behavior time is: Total Time=Ruminating Time+Interval Time Among them, Total Time is the total behavior time, Ruminating Time is the duration of rumination behavior, and IntervalTime is the interval time.

10. A sheep rumination behavior recognition system based on deep learning, characterized in that: include: The YOLOv8 target detection module is used to obtain video data for sheep rumination behavior recognition, capture multiple sheep images in the video and select key frame images for annotation, construct a training set, train the YOLOv8 target detection model, and obtain the trained YOLOv8 target detection model; The Deepsort target tracking module is used to convert the video data to be identified into video frames, input the trained YOLOv8 target detection model, and obtain the detection frame and features of the sheep head image with the ID mark frame in the video frame; according to the detection frame and features, the Deepsort target tracking algorithm is used to predict the next frame position and state of the target, optimally match multiple targets in the previous and next frames, output the tracking frames and ID numbers of multiple targets, and update them to obtain the updated detection frames and ID numbers of multiple targets; The sheep rumination classification video data set construction module is used to construct the sheep rumination classification video data set according to the updated detection frame and ID number by judging whether the interval between the current frame and the previous frame exceeds the preset number of frames. When the interval between the current frame and the previous frame exceeds the preset number of frames, the current frame is retained and all previous frames are cleared; The sheep rumination behavior recognition module is used to input the sheep rumination classification video dataset into the constructed neural network model to identify the rumination behavior.

Images