Sheep behavior recognition method, system and device and storage medium

Abstract

The invention provides a sheep behavior recognition method, system and device and a storage medium, and belongs to the technical field of computer vision, and the method comprises the steps: firstly introducing an attention mechanism and an online mask strategy into a DeepSORT model, and constructing a sheep tracking model; and adding a multi-scale group convolution module and a self-adaptive down-sampling module in the YOLOv8 model, and constructing a sheep behavior recognition model. And after the images in the video data are subjected to frame extraction cutting, the cut images are input into a sheep tracking model to continuously track the individual sheep, and a plurality of tracked images with unique identifiers (ID) are input into a sheep behavior recognition model to recognize the behavior of the single sheep. And recording the occurrence frequency and duration of the morphological characteristics of the same sheep at different times and different positions, and classifying the behaviors of the sheep according to the occurrence frequency and duration of the morphological characteristics of the same sheep. According to the method, the limitation of single sheep processing is overcome, and the detection precision and the behavior recognition accuracy are greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and in particular relates to a sheep behavior recognition method, system, device and storage medium. Background Art

[0002] With the rapid advancement of science and technology, especially in the fields of information technology and automation, animal husbandry is undergoing a transformation. Large-scale and intelligent farming models are increasingly becoming a new development trend. This emerging farming model integrates advanced sensors, monitoring equipment and data analysis tools to improve farming efficiency and improve animal health and welfare management. The behavior patterns of animals are important indicators of their health status. Regular eating, activity and social behavior usually indicate that the animal is in good health; on the contrary, abnormal stillness, loss of appetite or aggressive behavior may indicate that health problems are emerging. The behavior of sheep can reflect their growth and health status.

[0003] However, the current method of sheep behavior recognition is generally to process and analyze the collected pasture images, and judge the sheep's behavior based on the movement, shape and other characteristics of a single sheep at a certain moment. The recognition result is relatively blind, and it is impossible to recognize sheep under occlusion. The accuracy of recognition needs to be improved. Summary of the invention

[0004] In view of the problem that the existing sheep behavior recognition has relatively blind recognition results and the accuracy needs to be improved, the present invention provides a sheep behavior recognition method.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] The Vision Permutator attention mechanism and online mask strategy are introduced into the DeepSORT model, and the improved DeepSORT model is trained to obtain the sheep tracking model ViTSORT. The multi-scale group convolution module MSGConv and the LAWDS adaptive downsampling module are added to the YOLOv8 model, and the improved YOLOv8 model is trained to obtain the sheep behavior recognition model YOLOv8-MS.

[0007] Obtain real-time video data of multiple sheep and their different behaviors in a sheep farm environment;

[0008] Extract multiple frames per second from real-time video data containing multiple sheep and different behaviors of the sheep to detect the sheep, generate a corresponding bounding box for the sheep detected in each frame, and crop the sheep image according to the generated bounding box;

[0009] The cropped image is input into the sheep tracking model ViTSORT, and the appearance features of the sheep in the cropped image are extracted through the Vision Permutator attention mechanism. A unique identifier ID is assigned to each sheep with the same appearance features, and the individual sheep is continuously tracked based on the ID.

[0010] The tracked multiple images with unique identifiers ID are input into the sheep behavior recognition model YOLOv8-MS. The multi-scale group convolution module MSGConv uses multiple convolution kernels of different sizes to process multiple images with unique identifiers ID in parallel to obtain multiple spatial detail features of the sheep. The LAWDS adaptive downsampling module generates an image sequence set based on the multiple spatial detail features of the sheep, and identifies the behavior of a single sheep based on the image sequence set; wherein the spatial detail features are the morphological features of the sheep at different times and locations.

[0011] The frequency and duration of the morphological characteristics of the same sheep at different times and locations were recorded, and the behavior of the sheep was classified according to the frequency and duration of the same morphological characteristics.

[0012] Preferably, multiple frames are extracted from the video data per second for sheep detection through the YOLOv5 model, and a corresponding bounding box is generated for the sheep detected in each frame.

[0013] Preferably, before training the improved DeepSORT model and the improved YOLOv8 model, it also includes constructing a training set, and the constructing of the training set specifically includes:

[0014] Acquire historical video data containing multiple sheep and different behaviors of the sheep in a sheep farm environment; the historical video data is overhead video data of multiple sheep simultaneously existing under the same lens;

[0015] Extract frames from historical video data to obtain video image data;

[0016] Performing random data enhancement on the video image data, wherein the random data enhancement includes flipping, rotating, cropping and deforming the image;

[0017] The labelme annotation software was used to manually classify different sheep behaviors in the enhanced image data and annotate the classified behaviors. The annotated image data constituted the training set.

[0018] Preferably, the improved DeepSORT model and the improved YOLOv8 model are trained by the training set, specifically including:

[0019] The image data in the training set is input into the improved DeepSORT model, the image data is evenly divided into multiple image blocks, the multiple image blocks are marked by linear projection, the marked image blocks are input into a series of permutation operators for feature encoding, the global average pooling layer and the fully connected layer are used to capture the detailed body features of the sheep in the encoded image blocks, a unique identifier ID is assigned to each sheep according to the detailed body features of the sheep, and the output is a feature map after the sheep is assigned an ID; the detailed body features of the sheep include horns, tails and body shape; the mask strategy is used to solve the problem of target loss or ID matching errors caused by occlusion of the sheep.

[0020] The feature map after ID assignment is input into the improved YOLOv8 model. The multi-scale group convolution module MSGConv uses multiple convolution kernels of different sizes to process the feature map in parallel, capturing spatial detail features from coarse to fine in the feature map. Then, 1X1 convolution is used to integrate the spatial detail features, and the input feature map is divided into g groups along the channel dimension, where g is the number of elements in the predefined kernel size set {1,3,5,7}. Each group of feature maps is processed by the corresponding convolution kernel to realize the parallel extraction of multi-scale features. The LAWDS adaptive downsampling module generates an image sequence set based on the parallel extracted features by combining average pooling and 1x1 convolution, and recognizes the behavior of a single sheep according to the image sequence set.

[0021] Preferably, the sheep's behaviors include normal behaviors and abnormal behaviors, the normal behaviors include walking, standing, eating and licking, and the abnormal behaviors include lameness, attacking and lying down.

[0022] Preferably, the pan-tilt camera is arranged at the top of the pasture passage, and the pan-tilt camera is used to obtain historical video data and real-time video data including multiple sheep and different behaviors of the sheep in the sheep farm environment.

[0023] The present invention also provides a sheep behavior recognition system, comprising:

[0024] The model building module is used to introduce the Vision Permutator attention mechanism and online mask strategy into the DeepSORT model, and train the improved DeepSORT model to obtain the sheep tracking model ViTSORT; add the multi-scale group convolution module MSGConv and the LAWDS adaptive downsampling module into the YOLOv8 model, and train the improved YOLOv8 model to obtain the sheep behavior recognition model YOLOv8-MS;

[0025] The data acquisition module is used to obtain real-time video data of multiple sheep and their different behaviors in the sheep farm environment;

[0026] An image cropping module is used to extract multiple frames per second from real-time video data containing multiple sheep and different behaviors of sheep for sheep detection, generate a corresponding bounding box for the sheep detected in each frame, and crop the sheep image according to the generated bounding box;

[0027] The tracking module is used to input the cropped image into the sheep tracking model ViTSORT, extract the appearance features of the sheep in the cropped image through the VisionPermutator attention mechanism, assign a unique identifier ID to each sheep with the same appearance features, and continuously track the individual sheep according to the identification ID;

[0028] The recognition module is used to input the tracked multiple images with unique identifiers ID into the sheep behavior recognition model YOLOv8-MS. The multi-scale group convolution module MSGConv uses multiple convolution kernels of different sizes to process multiple images with unique identifiers ID in parallel to obtain multiple spatial detail features of the sheep. The LAWDS adaptive downsampling module generates an image sequence set based on the multiple spatial detail features of the sheep, and recognizes the behavior of a single sheep based on the image sequence set; wherein the spatial detail features are the morphological features of the sheep at different times and locations;

[0029] The behavior classification module is used to record the frequency and duration of the morphological characteristics of the same sheep at different times and locations, and classify the behavior of the sheep according to the frequency and duration of the same morphological characteristics.

[0030] The present invention also provides a computer device, comprising a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement any one of the steps in the sheep behavior identification method.

[0031] The present invention also provides a computer-readable storage medium, on which a computer program is stored, characterized in that when the computer program is loaded by a processor, it can execute any one of the steps in the sheep behavior identification method.

[0032] The sheep behavior identification method provided by the present invention has the following beneficial effects:

[0033] The present invention introduces the Vision Permutator attention mechanism and online mask strategy into the DeepSORT model to construct a sheep tracking model ViTSORT. The attention mechanism can reduce the resolution of feature maps while retaining important information, thereby improving the model's representation ability. At the same time, the online mask strategy is added to solve the problem of target loss or ID matching errors caused by occlusion of sheep, thereby enhancing the robustness and accuracy of the model in complex scenarios. The sheep behavior recognition model YOLOv8-MS is used for behavior recognition, and the multi-scale group convolution and adaptive downsampling modules are combined to improve the model's feature extraction and representation capabilities. The sheep's behavior can be classified according to the frequency and duration of the morphological features of the same sheep at different times and locations, overcoming the limitations of single sheep processing and greatly improving the detection accuracy and the accuracy of behavior recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 This is a flow chart of model training of the sheep behavior recognition method according to Example 1 of the present invention.

[0036] Figure 2 Layout diagram for the data acquisition module.

[0037] Figure 3 This is a classification diagram for sheep behavior recognition, where Figure 3 (A) is standing and lying behavior, Figure 3 (B) is an aggressive behavior. Figure 3 (C) is walking behavior, Figure 3 (D) is eating behavior, Figure 3 (E) is limping behavior, Figure 3 (F) indicates licking behavior.

[0038] Figure 4 This is the structural diagram of the sheep tracking model and the sheep behavior recognition model.

[0039] Figure 5 There are three different levels of masking modes for the online masking strategy. DETAILED DESCRIPTION

[0040] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0041] At present, some progress has been made in sheep behavior detection, but there are still some problems to be solved. First, there are few studies on multi-behavior recognition of sheep, especially destructive behaviors such as attack and biting. In addition, the frequency and duration of various behaviors of sheep are also different. Healthy sheep have relatively regular daily behaviors, and the duration of various behaviors such as eating, standing, and lying down is relatively fixed. However, sick sheep often do not participate in eating, have poor appetite, stay away from the flock, stand still in the fence, by the wall, or lie on the ground. Existing methods have not yet paid attention to the statistics of the duration of various behaviors of individual sheep. Traditional behavior recognition methods may ignore abnormal behaviors, such as lying down for a long time or not eating. Abnormal behaviors of sick sheep, such as lying down for a long time or abnormal eating, cannot be discovered in time by manual observation and behavior recognition alone, but after adding time series features (i.e., the frequency and duration of different behaviors of the same sheep), sick sheep can be discovered more quickly and intuitively, and timely diagnosis and treatment can be carried out to reduce breeding losses. In order to count the duration and frequency of various behaviors of sheep every day, it is necessary to accurately track each sheep and identify its behavior and count the duration of the behavior.

[0042] Based on this, the present invention provides a method for identifying sheep behavior, specifically a method for identifying sheep behavior based on ViTSORT and YOLOv8-MS. The present invention aims to improve the accuracy and efficiency of statistical analysis of sheep behavior and abnormal behavior detection in a sheep farm environment on the basis of constructing a framework for sheep behavior identification. The framework includes the construction of a sheep behavior dataset, a sheep tracking model based on ViTSORT, and a sheep behavior identification model based on YOLOv8. Figure 1 As shown, the method specifically comprises the following steps:

[0043] Step 1: Obtain real-time video data of multiple sheep and their different behaviors in a sheep farm environment. Specifically, Figure 2 As shown in the figure, the PTZ camera is placed on the top of the pen passage, and the image data of multiple sheep is obtained by using the PTZ camera. This ensures that the top-view images of sheep of different growth status and body size are captured, providing high-quality data for subsequent analysis.

[0044] Step 2: extract multiple frames per second from the real-time video data containing multiple sheep and different behaviors of sheep for sheep detection, generate a corresponding bounding box for the sheep detected in each frame, and crop the sheep image according to the generated bounding box. In this embodiment, the YOLOv5 model is used to extract multiple frames per second from the video data for sheep detection, and a corresponding bounding box is generated for the sheep detected in each frame.

[0045] Step 3: Input the cropped image into the sheep tracking model ViTSORT, extract the appearance features of the sheep in the cropped image through the Vision Permutator attention mechanism, assign a unique identifier ID to each sheep with the same appearance features, and continuously track the individual sheep based on the identification ID.

[0046] Step 4. Input the tracked multiple images with unique identifiers ID into the sheep behavior recognition model YOLOv8-MS. The multi-scale group convolution module MSGConv uses multiple convolution kernels of different sizes to process multiple images with unique identifiers ID in parallel to obtain multiple spatial detail features of the sheep. The LAWDS adaptive downsampling module generates an image sequence set based on the multiple spatial detail features of the sheep, and identifies the behavior of a single sheep based on the image sequence set. Among them, the spatial detail features are the morphological features of the sheep at different times and locations.

[0047] Step 5: Record the frequency and duration of the morphological features of the same sheep at different times and locations, and classify the sheep's behavior according to the frequency and duration of the same morphological features. Figure 3 As shown, the sheep's behavior includes normal behavior and abnormal behavior, among which normal behavior includes Figure 3 (A) Standing behavior, Figure 3 (C) walking behavior, Figure 3 (D) Eating behavior and Figure 3 Abnormal behaviors include: Figure 3 (A) Lying down behavior, Figure 3 (B) Aggression and Figure 3 (E) Lameness behavior.

[0048] The present invention combines the YOLOv5 model, the sheep tracking model ViTSORT and the sheep behavior recognition model YOLOv8-MS, which increases the accuracy of behavior recognition on the basis of ensuring the feasibility of real-time detection, and can count the cumulative duration of different behaviors of sheep every day, so as to more quickly and intuitively discover sick sheep, and provide timely diagnosis and treatment to reduce breeding losses.

[0049] Specifically, in response to the occlusion problem of multiple sheep, the present invention proposes a tracking method based on DeepSORT, ViTSORT, designs a mask strategy to solve the problem of target loss or ID matching errors caused by occlusion of sheep, and introduces the Vision Permutator (ViP) attention mechanism to more accurately extract the appearance information of sheep and further enhance the tracking effect. The improved DeepSORT model is trained to obtain the sheep tracking model ViTSORT.

[0050] After sheep tracking is realized, the present invention proposes a sheep behavior recognition model Yolov8-MS based on the YOLOv8 model. Inspired by the idea of ​​grouped convolution, a multi-scale group convolution module (MSGConv) is added to the YOLOv8 model to generate more features with fewer parameters, and a lightweight adaptive downsampling module (LAWDS) is added to reduce the resolution of feature maps while retaining important information, thereby improving the representation ability of the model. The improved YOLOv8 model is trained to obtain the sheep behavior recognition model YOLOv8-MS.

[0051] Specifically, Figure 4 As shown in the figure, the behavior detection network is constructed by using the C5 convolution module, LeakyReLU layer, MSGConv module and LAWDS adaptive downsampling module of the YOLOv8-MS backbone network. The deep neural network model is constructed by combining the Vision Permutator attention mechanism and the small target detection layer. The YOLOv8-MS deep neural network model is trained using various labeled behavior data to obtain the sheep behavior recognition model. The MSGConv module is used to reduce the number of parameters as much as possible without losing accuracy, providing more possibilities for the model in actual deployment.

[0052] Experimental results show that the sheep behavior recognition model YOLOv8-MS achieves 96.9% mAP accuracy on the sheep behavior dataset, with a detection speed of 196FPS, which is significantly better than existing methods. This invention integrates multiple advanced deep learning technologies to achieve efficient recognition and time series statistics of sheep behavior, providing strong support for sheep farm management and the improvement of animal welfare.

[0053] In this embodiment, training the improved DeepSORT model and the improved YOLOv8 model specifically includes the following steps:

[0054] S1. Acquire historical video data containing multiple sheep and different behaviors of the sheep in a sheep farm environment; the historical video data is overhead video data of multiple sheep simultaneously existing under the same lens.

[0055] S2. Extract frames from the historical video data of multiple sheep to obtain video image data. In order to reduce the impact of training data on the model and improve the generalization of the model in actual deployment, one frame of the video stream data is extracted every 15 frames for processing.

[0056] S3. Perform data enhancement on the video image data, and annotate the various behaviors of the sheep in the enhanced images to obtain annotated behavior data. The annotated image data constitutes a training set, totaling 6212 images and 30528 annotated samples. Among them, the various behaviors of sheep include normal behaviors and abnormal behaviors. Normal behaviors include walking, standing, eating and licking, and abnormal behaviors include lameness, attacking and lying down. Use labelme annotation software to annotate the sheep behaviors in the image so that the model has the correct reference object during training. Through training with a large amount of annotated data, the model can accurately identify a variety of sheep behaviors.

[0057] S4. Extract multiple frames per second from the image data for sheep detection through the YOLOv5 model, generate a corresponding bounding box for the sheep detected in each frame, and crop the sheep image according to the generated bounding box.

[0058] The cropped image is input into the improved DeepSORT model, the image data is evenly divided into multiple image blocks, the multiple image blocks are marked by linear projection, the marked image blocks are input into a series of permutation operators for feature encoding, the global average pooling layer and the fully connected layer are used to capture the detailed body features of the sheep in the encoded image blocks, a unique identifier ID is assigned to each sheep according to the detailed body features of the sheep, and the output is a feature map after the sheep is assigned an ID; the detailed body features of the sheep include horns, tail and body shape.

[0059] The feature map after ID assignment is input into the improved YOLOv8 model. The multi-scale group convolution module MSGConv uses multiple convolution kernels of different sizes to process the feature map in parallel, capturing spatial detail features from coarse to fine in the feature map. Then, 1X1 convolution is used to integrate the spatial detail features, and the input feature map is divided into g groups along the channel dimension, where g is the number of elements in the predefined kernel size set {1,3,5,7}. Each group of feature maps is processed by the corresponding convolution kernel to realize the parallel extraction of multi-scale features. The LAWDS adaptive downsampling module generates an image sequence set based on the parallel extracted features by combining average pooling and 1x1 convolution, and recognizes the behavior of a single sheep according to the image sequence set.

[0060] During the training phase, 25% to 100% of the sheep image area was randomly intercepted for training to simulate the different occlusion states of sheep every day. 20 mask modes were designed for the test data set, and the multiple modes were divided into three different levels: simple, medium and difficult. At the end of each training round, the mask strategy was used to enhance the test data, and 20 enhanced images were obtained for each test image. The three different levels of mask modes of the online mask strategy are shown in Figure 2. Figure 5 shown.

[0061] In this embodiment, the cross entropy loss function is used to train the deep neural network model with the labeled behavior data to obtain a sheep behavior recognition model.

[0062] In this embodiment, the YOLOv5 model, the sheep tracking model ViTSORT and the sheep behavior recognition model YOLOv8-MS together constitute the following Figure 1 The behavior detection module shown in the embodiment can provide early warning of various emergencies in real time, thereby improving the management efficiency of the farm and the welfare of the animals.

[0063] S6. Input the real-time video data of multiple sheep into the trained behavior detection module, and output various behavior detection results of the sheep. Then optimize the parameters of each model according to the detection results to obtain the optimal detection model.

[0064] The present invention also provides a sheep behavior recognition system, which includes a model building module, a data acquisition module, an image cropping module, a tracking module, a recognition module and a behavior classification module.

[0065] Specifically, the model building module is used to introduce the VisionPermutator attention mechanism and online mask strategy into the DeepSORT model, and train the improved DeepSORT model to obtain the sheep tracking model ViTSORT; add the multi-scale group convolution module MSGConv and the LAWDS adaptive downsampling module into the YOLOv8 model, and train the improved YOLOv8 model to obtain the sheep behavior recognition model YOLOv8-MS. The data acquisition module is used to obtain real-time video data of multiple sheep and different behaviors in the sheep farm environment. The layout of the components in the data acquisition module is as follows: Figure 2As shown. The image cropping module is used to extract multiple frames per second from real-time video data containing multiple sheep and different behaviors of sheep for sheep detection, generate corresponding bounding boxes for the sheep detected in each frame, and crop the sheep image according to the generated bounding boxes. The tracking module is used to input the cropped image into the sheep tracking model ViTSORT, extract the appearance features of the sheep in the cropped image through the Vision Permutator attention mechanism, and assign a unique identifier ID to each sheep with the same appearance features extracted, and continuously track the individual sheep according to the identification ID. The recognition module is used to input the tracked multiple images with unique identifiers ID into the sheep behavior recognition model YOLOv8-MS, and the multi-scale group convolution module MSGConv uses multiple convolution kernels of different sizes to process multiple images with unique identifiers ID in parallel to obtain multiple spatial detail features of the sheep; the LAWDS adaptive downsampling module generates an image sequence set based on multiple spatial detail features of the sheep, and identifies the behavior of a single sheep based on the image sequence set; wherein, the spatial detail features are the morphological features of the sheep at different times and locations. The behavior classification module is used to record the frequency and duration of the morphological characteristics of the same sheep at different times and locations, and classify the behavior of the sheep according to the frequency and duration of the same morphological characteristics.

[0066] Each module in the above-mentioned sheep behavior recognition device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0067] The present invention also provides a computer device, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps in the sheep behavior recognition method embodiment. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0068] Furthermore, the present invention also provides a non-temporary computer-readable storage medium containing instructions, and a computer program is stored on the storage medium. For example, a memory containing instructions, the above instructions can be executed by a processor of a computer device to complete the above method. For example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device. When the computer program is executed by the processor, the steps in the embodiment of the sheep behavior identification method can be implemented. The specific implementation method can be found in the method embodiment, which will not be repeated here.

[0069] Example 1

[0070] Taking sheep as an example, the sheep behavior recognition method based on ViTSORT and YOLOv8-MS provided by the present invention is further described below. The method includes the following steps:

[0071] Step 1. In order to obtain overhead images of sheep of different growth states and sizes, a pan-tilt camera is placed on one side of the channel. The camera is about 4 meters from the ground and the sheep pen is 5 meters wide to ensure that the video image data of the sheep's overhead view is collected. In order to reduce the scene where the sheep's behavior is too single, try to set the collection scene and time after the sheep have eaten, and keep multiple sheep in the video at the same time to ensure that the trained model has practical deployment significance. The data collection of the sheep's overhead view in step 1 ensures that sheep of different sizes and behavioral states can be captured to ensure that the model can be applied to actual production.

[0072] Step 2: To reduce the impact of training data on the entire model and improve the generalization of the entire model in actual deployment, the present invention first extracts frames from the video stream data captured by the PTZ camera, extracting one frame every 15 frames, and then uses data enhancement methods such as flipping, rotation, cropping, and deformation to enrich the collected data as much as possible. The data preprocessing of the top view of the sheep in step 2 uses data enhancement methods to enrich the sheep images, which greatly enhances the generalization ability of the entire model and enhances the significance of deployment.

[0073] Step 3: Use labelme annotation software to annotate the various behaviors of the sheep in the image and obtain annotated sheep behavior data. There are seven categories, including four normal behaviors: walking, standing, eating, and licking, and three abnormal behaviors: lameness, attacking, and lying down. The sheep behavior annotation process in step 3 is the basis of model training, so that the model has correct reference objects during training.

[0074] Step 4: Sheep behavior detection requires semantic information of images with different resolutions in the image. At the same time, in order to improve the efficiency of sheep behavior detection, YOLOv8-MS is selected as the basic network of the deep neural network model. It extracts semantic information at different levels through branch detection heads with different resolutions. It also has lightweight internal modules, small parameters and high computational efficiency. A lightweight sheep behavior real-time monitoring deep neural network model YOLOv8-MS that combines different behaviors is constructed. The MSGConv module is used to reduce the number of parameters as much as possible without losing accuracy, providing more possibilities for the model in actual deployment.

[0075] Step 5: Use the cross entropy loss function to train the deep neural network model with the labeled sheep behavior data to obtain the final sheep behavior recognition model. Use a lightweight network for training from scratch to fully learn the characteristics of sheep behavior in the data.

[0076] Step 6: Deploy the final model obtained in step 4. For the target sheep house where the behavior needs to be detected, collect the video data of the sheep in real time according to step 1; then input the intercepted video image data into the final sheep behavior recognition model to obtain the behavior detection results of the corresponding sheep flock. The sheep behavior recognition model in step 6 uses a lightweight structure to detect sheep behavior in real time, and can promptly warn of various emergencies.

[0077] The advantages of the present invention are:

[0078] (1) Compared with traditional methods, the present invention adopts a "non-contact, high-yield, and high-efficiency" deep learning method, which is superior to traditional livestock behavior detection methods in terms of efficiency and accuracy.

[0079] (2) The present invention uses a pan-tilt camera to collect multiple sheep behavior videos. Through frame extraction and data enhancement methods, reliable and rich sheep behavior data can be obtained, thereby improving the model's generalization ability for sheep object characteristics.

[0080] (3) The present invention introduces the ViTSORT model for target tracking. By combining the Vision Permutator attention mechanism and the online mask strategy, the robustness and accuracy of the model in complex scenarios are enhanced.

[0081] (4) The present invention adopts the sheep behavior recognition model YOLOv8-MS for behavior recognition, adds a multi-scale group convolution module MSGConv and a LAWDS adaptive downsampling module, and reduces the number of parameters as much as possible without losing accuracy, thereby improving the feature extraction and representation capabilities of the model and providing more possibilities for the model in actual deployment.

[0082] (5) Through the analysis of real-time video data of pastures, the present invention can timely detect and deal with abnormal behaviors, reduce potential risks and losses, save the time cost of relevant personnel, significantly improve the efficiency of animal husbandry management, increase animal welfare, and provide new ideas for the practical application of relevant livestock behavior detection methods. It has broad application prospects.

[0083] It will be appreciated by those skilled in the art that embodiments of the present invention may provide methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0084] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0085] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0086] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0087] It should be pointed out that the specific implementation methods described above can enable those skilled in the art to understand the invention more comprehensively, but do not limit the invention in any way. Therefore, although the invention has been described in detail in this specification and embodiments, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved. Any simple change or equivalent replacement of the technical solution that can be obviously obtained by any technician familiar with the field within the technical scope disclosed in the present invention belongs to the protection scope of the present invention.

Claims

1. A sheep behavior recognition method, characterized in that: The following steps are involved: The attention mechanism and online mask strategy are introduced into the DeepSORT model, and the improved DeepSORT model is trained to obtain the sheep tracking model ViTSORT; the multi-scale group convolution module and adaptive downsampling module are added to the YOLOv8 model, and the improved YOLOv8 model is trained to obtain the sheep behavior recognition model YOLOv8-MS; Obtain real-time video data of multiple sheep and their different behaviors in a sheep farm environment; Extract multiple frames per second from real-time video data containing multiple sheep and different behaviors of the sheep to detect the sheep, generate a corresponding bounding box for the sheep detected in each frame, and crop the sheep image according to the generated bounding box; The cropped image is input into the sheep tracking model ViTSORT, and the appearance features of the sheep in the cropped image are extracted through the attention mechanism. A unique identifier ID is assigned to each sheep with the same appearance features, and the individual sheep is continuously tracked based on the identification ID. The tracked multiple images with unique identifiers are input into the sheep behavior recognition model YOLOv8-MS. The multi-scale group convolution module uses multiple convolution kernels of different sizes to process multiple images with unique identifiers in parallel to obtain multiple spatial detail features of the sheep. The adaptive downsampling module generates an image sequence set according to multiple spatial detail features of the sheep, and identifies the behavior of a single sheep according to the image sequence set; wherein the spatial detail features are morphological features of the sheep at different times and locations; The frequency and duration of the morphological characteristics of the same sheep at different times and locations were recorded, and the behavior of the sheep was classified according to the frequency and duration of the same morphological characteristics.

2. The sheep behavior identification method according to claim 1, characterized in that: The YOLOv5 model is used to extract multiple frames per second from the video data for sheep detection, and a corresponding bounding box is generated for the sheep detected in each frame.

3. The sheep behavior identification method according to claim 1, characterized in that: Before training the improved DeepSORT model and the improved YOLOv8 model, a training set is constructed. The construction of the training set specifically includes: Acquire historical video data containing multiple sheep and different behaviors of the sheep in a sheep farm environment; the historical video data is overhead video data of multiple sheep simultaneously existing under the same lens; Extract frames from historical video data to obtain video image data; Performing random data enhancement on the video image data, wherein the random data enhancement includes flipping, rotating, cropping and deforming the image; The labelme annotation software was used to manually classify different sheep behaviors in the enhanced image data and annotate the classified behaviors. The annotated image data constituted the training set.

4. The sheep behavior identification method according to claim 3, characterized in that: The improved DeepSORT model and the improved YOLOv8 model are trained through the training set, including: Extract multiple frames per second from the image data for sheep detection, generate a corresponding bounding box for the sheep detected in each frame, and crop the sheep image according to the generated bounding box; Input the cropped image into the improved DeepSORT model, evenly divide the image data to obtain multiple image blocks, mark the multiple image blocks using linear projection, input the marked image blocks into a series of permutation operators for feature encoding, use a global average pooling layer and a fully connected layer to capture the detailed features of the sheep's body in the encoded image blocks, assign a unique identifier ID to each sheep according to the detailed features of the sheep's body, and output a feature map after the sheep's ID is assigned; the detailed features of the sheep's body include horns, tails and body shape; The feature maps after ID assignment are input into the improved YOLOv8 model. The multi-scale group convolution module uses multiple convolution kernels of different sizes to process the feature maps in parallel, capturing spatial detail features from coarse to fine in the feature maps. Then, 1X1 convolution is used to integrate the spatial detail features, and the input feature maps are divided into g groups along the channel dimension, where g is the number of elements in the predefined kernel size set {1,3,5,7}. Each group of feature maps is processed by the corresponding convolution kernel to realize the parallel extraction of multi-scale features. The adaptive downsampling module generates an image sequence set based on the parallel extracted features by combining average pooling and 1x1 convolution, and recognizes the behavior of a single sheep according to the image sequence set.

5. The sheep behavior identification method according to claim 1, characterized in that: The sheep behaviors include normal behaviors and abnormal behaviors. Normal behaviors include walking, standing, eating and licking, and abnormal behaviors include lameness, attacking and lying down.

6. The sheep behavior identification method according to claim 1, characterized in that: The PTZ camera is placed on the top of the pasture passage, and the historical video data and real-time video data of multiple sheep and their different behaviors in the sheep farm environment are obtained by using the PTZ camera.

7. A sheep behavior recognition system, characterized in that: include: The model building module is used to introduce the attention mechanism and online mask strategy into the DeepSORT model, and train the improved DeepSORT model to obtain the sheep tracking model ViTSORT; add the multi-scale group convolution module and the adaptive downsampling module into the YOLOv8 model, and train the improved YOLOv8 model to obtain the sheep behavior recognition model YOLOv8-MS; The data acquisition module is used to obtain real-time video data of multiple sheep and their different behaviors in the sheep farm environment; An image cropping module is used to extract multiple frames per second from real-time video data containing multiple sheep and different behaviors of sheep for sheep detection, generate a corresponding bounding box for the sheep detected in each frame, and crop the sheep image according to the generated bounding box; The tracking module is used to input the cropped image into the sheep tracking model ViTSORT, extract the appearance features of the sheep in the cropped image through the attention mechanism, assign a unique identifier ID to each sheep with the same appearance features, and continuously track the individual sheep according to the identification ID; The recognition module is used to input multiple images with unique identifiers ID into the sheep behavior recognition model YOLOv8-MS. The multi-scale group convolution module uses multiple convolution kernels of different sizes to process multiple images with unique identifiers ID in parallel to obtain multiple spatial detail features of the sheep. The adaptive downsampling module generates an image sequence set according to multiple spatial detail features of the sheep, and identifies the behavior of a single sheep according to the image sequence set; wherein the spatial detail features are morphological features of the sheep at different times and locations; The behavior classification module is used to record the frequency and duration of the morphological characteristics of the same sheep at different times and locations, and classify the behavior of the sheep according to the frequency and duration of the same morphological characteristics.

8. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the sheep behavior identification method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is loaded by a processor, it can execute the steps of the sheep behavior identification method according to any one of claims 1 to 6.

Images