Pig behavior identification method and system

Abstract

The invention relates to a pig behavior identification method and system. The pig behavior identification method comprises the following steps: acquiring a pig behavior image, and preprocessing the image to obtain a pig behavior identification data set; carrying out lightweight improvement on an original Y0L0v8 model framework, recording the original Y0L0v8 model framework as a YOL0-PIG model, and training the YOL0-PIG model; and running the trained YOLO-PIG model on the fog computing node, and performing real-time target detection and tracking on the video data to obtain the position, the motion trail, the behavior type and the state information of the pig. According to the scheme provided by the invention, the position, the motion trail, the behavior type and the state information of the pig can be detected through the improved YOLO-PIG model, video data processing and analysis tasks are distributed at the edge end in cooperation with a fog computing technology, and real-time rapid identification of the behavior of the pig in the pig house is realized.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent pig farming, and in particular to a method and system for pig behavior recognition. Background Art

[0002] Pigs are one of the important livestock resources in my country. Their breeding efficiency and health level directly affect my country's food security and economic development. Therefore, effective monitoring and analysis of pig behavior is one of the key technologies to improve the quality and efficiency of pig breeding. Pig behavior can reflect information about the physiological health, growth and reproduction of pigs. By identifying and analyzing pig behavior, abnormal conditions of pigs, such as disease, injury, fighting, etc., can be discovered and handled in a timely manner, thereby improving the welfare and productivity of pigs.

[0003] At present, the main method for pig behavior recognition is computer vision technology based on video data, that is, collecting video data inside the pig house through cameras, and then processing and analyzing the video data through target detection, tracking, classification and other algorithms to obtain the behavior type and status of the pigs.

[0004] However, this method has the following problems: the large amount of video data and high network transmission delay lead to low efficiency of data processing and analysis, and cannot achieve real-time and reliability. The behavior recognition algorithm lacks a detailed and comprehensive classification and labeling system, resulting in inaccurate behavior recognition results that cannot reflect the true situation of the pigs. Summary of the invention

[0005] In order to solve or partially solve the problems existing in the related technology, the present application provides a method for pig behavior recognition. The method for pig behavior recognition can detect the pig's position, motion trajectory, behavior type and status information through an improved YOLO-PIG model, and cooperate with fog computing technology to distribute video data processing and analysis tasks at the edge, thereby realizing real-time and rapid recognition of pigs in the pig house.

[0006] The first aspect of the present application provides a method for pig behavior recognition, comprising:

[0007] Obtaining pig behavior images, and preprocessing the pig behavior images to obtain a pig behavior recognition data set, wherein the image preprocessing includes image enhancement, size scaling, grayscale change, channel merging, and data set division and labeling;

[0008] The original Y0L0v8 model framework is lightweighted and improved, recorded as the YOL0-PIG model, and the YOL0-PIG model is trained. The lightweight improvement of the original Y0L0v8 model framework includes: replacing some convolutions, adding the CBAM attention mechanism, and performing BN layer pruning;

[0009] The trained YOLO-PIG model is run on the fog computing node to perform real-time target detection and tracking on the video data to obtain pig behavior recognition results.

[0010] Optional, partial convolution replacements, including:

[0011] The 6X6 convolution in the first CBS module in the bottleneck structure of the original YOLOv8 model is deleted and replaced with a 3X3 partial convolution, and only 1 / 4 of the input channels are selected for convolution operation;

[0012] The 1×1 point-by-point convolution operation of the CBS module is used to expand the number of channels of the output feature map of the upper layer to twice the original number, and at the same time, the unprocessed channels of some of the above convolutional layers are utilized;

[0013] The next 1×1 convolutional layer is used to restore the output of the upper layer back to the number of channels before expansion.

[0014] Optionally, add the CBAM attention mechanism, including:

[0015] Connect the algorithm module of the CBAM attention mechanism to the backbone network of the YOLOv8 model.

[0016] Optionally, perform BN layer pruning, including:

[0017] The Loss function is used to add regular constraints to the BN layer, and a scaling factor is introduced. The scaling factor is regularized by sparsity to generate a pruned network. The pruned network is fine-tuned to generate an improved YOLOv8 model, which is denoted as the YOL0-PIG model.

[0018] Optionally, train the YOL0-PIG model, including:

[0019] Set the category, namely, Stand and Lying, enter the image size width and height, set the maximum number of iterations and learning rate, and use the Adam optimizer as the optimizer;

[0020] Input the training set into the initial weight training network to train the model until the maximum number of iterations is reached;

[0021] The trained model is validated using the test set. If the expected recognition requirements are met, the trained model is used as the pig behavior detection model. Otherwise, the pre-training steps are re-executed.

[0022] Optionally, run the trained YOLO-PIG model on the fog computing node to perform real-time object detection and tracking on video data, including:

[0023] The fog computing node analyzes and processes the real-time monitoring video data of the pig house, captures 2 video frames per minute, scales the image to be tested to a standardized size, and converts it into an RGB image;

[0024] Input the RGB image into the YOL0-PIG model for prediction, and output the behavior category, category confidence, prediction box coordinates, and border confidence;

[0025] Pig behavior analysis is performed based on the pig behavior identification results combined with daily body temperature data and behavioral history records.

[0026] Optionally, obtain images of pig behavior, including:

[0027] Collect and mix pig video images with open-source video images in an existing database to obtain a mixed image;

[0028] The initial pig behavior image is obtained by sampling the mixed image at intervals.

[0029] Optionally, perform image preprocessing, including:

[0030] The pig behavior recognition images were randomly divided into training sets and test sets at a ratio of 9:1;

[0031] Confirmed tag categories include pigs standing and lying;

[0032] According to the label category, use the labeling tool to label the target position coordinates and label category of the pig behavior recognition images in the training set, and generate the corresponding xml file with the labeling information.

[0033] In a second aspect, the present application provides a pig behavior recognition system, comprising:

[0034] The pig behavior recognition system includes:

[0035] Cameras, fog computing nodes, cloud servers;

[0036] The camera is used to obtain images of pig behavior;

[0037] The fog computing node is used to carry the embedded computing card of the trained YOLO-PIG model, and the pig behavior detection is performed through the embedded computing card;

[0038] The cloud server is used to store, analyze anomalies and issue early warnings for pig behavior recognition results.

[0039] The technical solution provided by this application may have the following beneficial effects:

[0040] The improved YOLO-PIG model is used to detect the location, movement trajectory, behavior type and status information of pigs, achieving high-speed target detection and tracking while ensuring high accuracy, thus adapting to the complex and changing pig house environment and realizing functions such as disease warning, health management and pig growth assessment for pigs;

[0041] With the help of fog computing technology, video data processing and analysis tasks are distributed at the edge, reducing network transmission delay and bandwidth consumption, and improving the real-time and reliability of data processing and analysis.

[0042] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The above and other objects, features and advantages of the present application will become more apparent through a more detailed description of exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0044] Figure 1 It is a flow chart of a method for pig behavior recognition shown in an embodiment of the present application;

[0045] Figure 2 It is a schematic diagram of the Faster Block structure block of the method for pig behavior recognition shown in the embodiment of the present application;

[0046] Figure 3 It is a schematic diagram of the CBAM attention mechanism module of the method for pig behavior recognition shown in the embodiment of the present application;

[0047] Figure 4 It is a schematic diagram of the model BN layer pruning process of the method for pig behavior recognition shown in the embodiment of the present application;

[0048] Figure 5 It is a structural diagram of a pig behavior recognition system shown in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0050] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0051] In the description of the present application, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0052] Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0053] At present, the main method for pig behavior recognition is computer vision technology based on video data. This method has the following problems: the amount of video data is large, the network transmission delay is high, resulting in low efficiency of data processing and analysis, and the inability to achieve real-time and reliability. The behavior recognition algorithm lacks a detailed and comprehensive classification and labeling system, resulting in inaccurate behavior recognition results that cannot reflect the true situation of the pigs.

[0054] In response to the above problems, the embodiments of the present application provide a method and system for pig behavior recognition, which can detect the pig's position, motion trajectory, behavior type and status information through an improved YOLO-PIG model, while ensuring high precision and achieving high-speed target detection and tracking, thereby adapting to the complex and changeable pig house environment, and realizing disease warning, health management, pig growth assessment and other functions for pigs; in conjunction with fog computing technology, video data processing and analysis tasks are distributed at the edge, reducing network transmission delay and bandwidth consumption, and improving the real-time and reliability of data processing and analysis.

[0055] The technical solution of the embodiments of the present application is described in detail below with reference to the accompanying drawings.

[0056] Figure 1 It is a flow chart of the method for pig behavior identification shown in the embodiment of the present application.

[0057] like Figure 1 As shown, the method for pig behavior recognition in the embodiment of the present application includes:

[0058] S101, obtaining pig behavior images, and preprocessing the pig behavior images to obtain a pig behavior recognition data set;

[0059] Specifically, in step S101, obtaining a set of pig behavior images includes:

[0060] Collect and mix pig video images with open-source video images in an existing database to obtain a mixed image;

[0061] The initial pig behavior recognition image is obtained by sampling the mixed image at intervals, and the initial pig behavior image is preprocessed to generate a behavior recognition data set.

[0062] Specifically, in step S101, preprocessing the image includes:

[0063] Among them, image preprocessing includes image enhancement, size scaling, grayscale change, channel merging, data set division and annotation.

[0064] The preprocessed initial pig behavior recognition images were randomly divided into training sets and test sets at a ratio of 9:1;

[0065] Determine the label categories as pigs standing and lying down. According to the label categories, use the LabelImg labeling tool to label the target position coordinates and label categories of each pig behavior recognition image in the training set, and generate the corresponding xml file with the labeling information. Store each image in the training set and the corresponding xml file according to the VOC data file structure.

[0066] S102. Make lightweight improvements to the original Y0L0v8 model framework, record it as the YOL0-PIG model, and train the YOL0-PIG model.

[0067] Specifically, in step S102, part of the convolution is replaced, and the bottleneck structure in the original YOLOv8 model is replaced with a Faster Block structure, such as Figure 2As shown in the figure, the 6X6 convolution in the first CBS module in the bottleneck structure of the original YOLOv8 model is deleted and replaced with a 3X3 partial convolution, and only 1 / 4 of the input channels are selected for convolution operation. Among them, the CBS module includes a basic convolution block consisting of convolution, batch normalization and activation function. The 1×1 point-by-point convolution operation of the CBS module is used to expand the number of channels of the output feature map of the upper layer to twice the original number, and the unprocessed channels of the above partial convolution layer are used. The next 1×1 convolution layer is used to restore the output of the upper layer to the number of channels before the expansion. The above structure is recorded as the Faster Block structure.

[0068] Specifically, in step S102, a CBAM (Convolutional Block Attention Module) attention mechanism module is added, such as Figure 3 As shown in the figure, the CBAM attention mechanism consists of two independent sub-modules, the Channel Attention Module (CAM) and the Spatial Attention Module (SAM), which combines channel and spatial attention, not only greatly improving the model performance, but also ensuring that it can be integrated into the YOLOv8 network architecture as a plug-and-play module.

[0069] First, open the tasks.py file, add CBAM in the import, and add CBAM-related calculation code blocks; then modify the configuration file, add the CBAM attention mechanism module after the c2f module, and connect it to the backbone network of the YOLOv8 model; finally, test whether the forward propagation of the modified model is successful.

[0070] Specifically, in step S102, BN layer pruning is performed, and the calculation formula of the BN layer is:

[0071]

[0072]

[0073] In formula (1) and (2), Z in Represents the channel output matrix of the convolutional layer; Z out Represents the corresponding output of the BN layer in this channel; σ B Represents the variance coefficient of the BN layer; μ B Represents the mean coefficient of the BN layer; It is usually 0 to avoid the denominator being zero; γ represents the scaling factor for normalization; β represents compensation for the deviation of the normalization operation.

[0074] From the formula, we can see that the activation size Z of each channel is out It is positively correlated with the coefficient γ. If γ is too small and close to 0, the corresponding activation value is also very small, regardless of Z. in What is the size of the corresponding BN layer output? in If the contribution of these channels to the model is irrelevant, then they can be considered to be very small, so they are redundant structures that can be pruned.

[0075] There is a scaling factor γ on each channel in the convolutional layer of the neural network. If the L1 regularization constraint is applied to these scaling factors and the network is trained, a model containing sparse parameters can be obtained. Then, the importance of the channels in the model is judged: channels with small γ values ​​can be judged as unimportant channels. These channels can be cut off to obtain a compact model. The calculation formula of the L1 regularization constraint is shown in formula (3):

[0076] L=∑ (x,y) l(f(x,W),y)+λ∑ γ∈T g(γ) (3)

[0077] In formula (3), (x, y) represents the input and target of the training; W represents the trainable weight; g represents the penalty term of the sparsity scaling factor; λ represents the regularization coefficient, that is, the sparsity rate.

[0078] According to the above BN layer pruning principle, the design is as follows Figure 4 The model pruning operation process shown is as follows: first, the CBAM-YOLOv8 model is input into the pruned network, then the appropriate sparsity rate is set, and then L1 sparse training is performed. After obtaining the sparse model, the channels with scaling factors less than the threshold are pruned to obtain a preliminary pruned model. Then, the preliminary pruned model is tested to check whether its detection accuracy has dropped significantly. If there is a significant drop, fine-tuning training is required. Finally, the final pruned model YOLOv8 model is output, which is recorded as the YOL0-PIG model.

[0079] Specifically, in step S102, the YOL0-PIG model is trained, and the categories classes=2 are set, i.e., two types of pigs: standing (Stand) and lying (Lying), the width and height of the input image size are both 640, the maximum number of iterations batches=8000, the learning rate is 0.001, and the optimizer uses Adam.

[0080] The training set is input into the initial weight training network for model training until the maximum number of iterations is reached.

[0081] The trained model is validated using the test set. If the expected recognition requirements are met, the trained model is used as the pig behavior detection model. Otherwise, the pre-training steps are re-executed.

[0082] S103. Run the trained YOLO-PIG model on the fog computing node to perform real-time target detection and tracking on the video data to obtain pig behavior recognition results.

[0083] Specifically, in step S103, the trained YOLO-PIG model is run on the fog computing node to perform real-time target detection and tracking on the video data, including:

[0084] Cameras are installed in the pig houses to collect images. After acquiring the image information, the data is transmitted to the relevant fog computing nodes through switches. The fog computing nodes are equipped with the trained YOLO-PIG model to perform behavior recognition detection on the real-time monitoring video data of the pig houses. The fog computing process occurs on the local area network (LAN) level network architecture, and the pig behavior detection calculation and processing is transferred from the cloud to the fog computing side, which not only speeds up the transmission speed of video data and reduces latency, but also speeds up the pig behavior recognition detection speed, while also reducing the burden on the server.

[0085] Before deploying the model on the fog computing node, the YOLO-PIG.pt model file trained on the server must be converted to an .onnx intermediate model, and then the obtained onnx model file is accelerated by TensorRT. After TensorRT quantization acceleration, the engine file corresponding to the model will be generated. Use this file on the fog computing device to infer different input formats such as pictures, videos, or streaming media, and output the detection results through the display.

[0086] The fog computing device uses the NVIDIA Jeston TX2 development board, which can meet the storage requirements of the YOLO-PIG model mentioned above and can detect at least one high-definition video in real time. TX2 can provide 1.3TOPS computing power and consumes only about 10W of energy on average.

[0087] When performing fog computing, 2 video frames are captured every minute, the image to be tested is standardized, scaled to 640X640 size, and converted into RGB images.

[0088] Input the RGB image into the YOL0-PIG model for prediction, and output the behavior category, category confidence, prediction box coordinates, and border confidence.

[0089] Through the results of pig behavior recognition, combined with daily body temperature data and behavioral history records, it is determined whether there is abnormal behavior, and primary response information is generated in real time, and sent to designated targets in a preset manner to remind attention.

[0090] Through pig behavior recognition, it can be determined whether the locked target has escalated abnormal behavior, and based on the determination result, the preset second-level response plan will be automatically executed, including alarms, notifying pig farm staff to go and check, etc.

[0091] Through information feedback from pig farm staff and pig behavior identification, it is determined whether the target's escalated abnormal behavior has been intervened, and based on whether intervention has occurred and the results of the intervention, the preset third-level response plan or the subsequent plan for successful intervention will be automatically executed.

[0092] The success of the intervention is assessed through information feedback from farm staff and identification of pig behavior. If unsuccessful, the information will be fed back to the farm veterinarian in real time and the platform will automatically enter the highest level of response.

[0093] Corresponding to the embodiment of the above-mentioned pig behavior identification method, the embodiment of the present application also provides a pig behavior identification system, which is as follows:

[0094] Figure 5 It is a structural diagram of a pig behavior recognition system shown in an embodiment of the present application.

[0095] like Figure 5 As shown, the pig behavior recognition system in the embodiment of the present application includes:

[0096] The overall architecture consists of three parts: cameras, fog computing nodes, and cloud servers.

[0097] The camera is used to obtain pig behavior images. The fog computing node is used to carry the embedded computing card of the trained YOLO-PIG model, and the pig behavior detection is performed through the embedded computing card. The cloud server is used to store the pig behavior recognition results, analyze anomalies and issue early warnings.

[0098] In the pig behavior recognition system, the monitoring terminal is composed of multiple cameras deployed in the pig house to ensure that there are no blind spots in the monitoring. The fog computing side selects multiple NVIDIA Jeston TX2 embedded computing cards according to the size of the monitoring range, and the cloud server is composed of cloud computing servers. The specific division of labor is as follows: the video data of pig activities within the monitoring range is collected through the camera, and transmitted to the fog computing node located in the same local area network in real time through the wireless network. The embedded computing card of the fog computing node has been equipped with a pre-trained pig behavior recognition model in advance. Next, the pig behavior detection is performed through the embedded computing card to realize the local processing of the data, and the pig behavior recognition results are transmitted to the cloud server through the wireless network for storage, abnormal analysis and early warning.

Claims

1. A method for pig behavior recognition, characterized in that: include: Obtaining pig behavior images, and preprocessing the pig behavior images to obtain a pig behavior recognition data set, wherein the preprocessing of the images includes image enhancement, size scaling, grayscale change, channel merging, and data set division and labeling; The original Y0L0v8 model framework is lightweighted and improved, recorded as the YOL0-PIG model, and the YOL0-PIG model is trained, wherein the lightweight improvement of the original Y0L0v8 model framework includes: replacing part of the convolution, adding the CBAM attention mechanism, and performing BN layer pruning; The trained YOLO-PIG model is run on the fog computing node to perform real-time target detection and tracking on the video data to obtain pig behavior recognition results.

2. The method according to claim 1, characterized in that The partial convolution replacement comprises: The 6X6 convolution in the first CBS module in the bottleneck structure of the original YOLOv8 model is deleted and replaced with a 3X3 partial convolution, and only 1 / 4 of the input channels are selected for convolution operation; The 1×1 point-by-point convolution operation of the CBS module is used to expand the number of channels of the output feature map of the upper layer to twice the original number, and at the same time, the unprocessed channels of some of the above convolutional layers are utilized; The next 1×1 convolutional layer is used to restore the output of the upper layer back to the number of channels before expansion.

3. The method according to claim 1, characterized in that The added CBAM attention mechanism includes: The algorithm module of the CBAM attention mechanism is connected to the backbone network of the original YOLOv8 model.

4. The method according to claim 1, characterized in that: The BN layer pruning includes: A regular constraint is added to the BN layer through the Loss function, a scaling factor is introduced, sparsity regularization is applied to the scaling factor, a pruned network is generated, and the pruned network is fine-tuned to generate an improved YOLOv8 model, which is recorded as the YOL0-PIG model.

5. The method according to claim 1, characterized in that The training of the YOL0-PIG model includes: Set the category, namely, Stand and Lying, enter the image size width and height, set the maximum number of iterations and learning rate, and use the Adam optimizer as the optimizer; Input the training set into the initial weight training network to train the model until the maximum number of iterations is reached; The trained model is validated using the test set. If the expected recognition requirements are met, the trained model is used as the pig behavior detection model. Otherwise, the pre-training steps are re-executed.

6. The method according to claim 1, characterized in that The trained YOLO-PIG model is run on the fog computing node to perform real-time target detection and tracking on the video data, including: The fog computing node analyzes and processes the real-time monitoring video data of the pig house, captures 2 video frames per minute, scales the image to be tested to a standardized size, and converts it into an RGB image; Input the RGB image into the YOL0-PIG model for prediction, and output the behavior category, category confidence, prediction box coordinates, and border confidence; Pig behavior analysis is performed based on the pig behavior identification results combined with daily body temperature data and behavioral history records.

7. The method according to claim 1, characterized in that The step of obtaining the pig behavior image comprises: Collect and mix pig video images with open-source video images in an existing database to obtain a mixed image; The initial pig behavior image is obtained by extracting frames from the mixed image at intervals.

8. The method according to claim 1 or 7, characterized in that: The preprocessing of the image comprises: The pig behavior recognition images are randomly divided into a training set and a test set at a ratio of 9:1; Determine the tag categories including pigs standing and lying; According to the label category, use the labeling tool to label the target position coordinates and label category of the pig behavior recognition images in the training set, and generate the corresponding xml file with the labeling information.

9. A pig behavior recognition system, characterized in that: The pig behavior recognition system comprises: Cameras, fog computing nodes, cloud servers; The camera is used to obtain images of pig behavior; The fog computing node is used to carry the embedded computing card of the trained YOLO-PIG model, and the pig behavior detection is performed through the embedded computing card; The cloud server is used to store, analyze anomalies and issue early warnings for pig behavior recognition results.

10. The system according to claim 9, characterized in that The system is used to execute a method for pig behavior recognition as described in any one of claims 2-8.

Images