A method and system for recognizing pig postures and behaviors.

Abstract

This invention relates to the field of smart animal husbandry technology, and discloses a method and system for recognizing pig postures and behaviors. This method, based on real-time video and / or images of pigs, employs an anchor-free convolutional neural network for model training and inference to achieve the recognition of pig postures and behaviors. This invention solves the problems of existing technologies that struggle to achieve pig activity and behavior recognition with low computational resource consumption and low cost.

Description

Technical Field

[0001] This invention relates to the field of smart farming technology, specifically a method and system for recognizing pig postures and behaviors. Background Technology

[0002] Swine health is one of the major challenges facing the livestock industry. Previously, monitoring swine health primarily relied on manual observation or invasive sensors. This resulted in delayed detection of anomalies, low efficiency, and potential harm to the pigs. With the rise of deep learning, a new approach exists that uses non-contact cameras to capture video images of pigs and then leverages deep learning technology for behavioral recognition. It's worth noting that the main target detection method used often only performs a rough selection and classification of target pigs; generating suggested anchor boxes requires significant computational resources, necessitating training and inference on GPU-equipped servers, which leads to high costs and hinders widespread adoption. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies, this invention provides a method and system for recognizing pig postures and behaviors, solving the problems of difficulty in recognizing pig activities and behaviors with low computational resource consumption and low cost.

[0004] The technical solution adopted by the present invention to solve the above problems is:

[0005] A method for recognizing pig postures and behaviors, based on real-time video and / or images of pigs, employs an anchor-free convolutional neural network for model training and inference to achieve recognition of pig postures and behaviors.

[0006] As a preferred technical solution, the steps include:

[0007] S1, Data Acquisition and Processing: Acquire images of pigs and obtain a dataset for training based on the images;

[0008] S2, Model Training: The dataset is trained using the SOLOv2 instance segmentation model without anchor boxes. After model conversion, a quantized rknn model file is obtained for use on edge computing devices with NPUs.

[0009] S3, Model Inference: Based on the RKNN model, model inference is performed to identify the pig's posture and behavior.

[0010] As a preferred technical solution, step S1 includes the following steps:

[0011] S11, Collect images of pigs inside the pigpen;

[0012] S12, Clean and filter the acquired images:

[0013] S13, annotate the images to form a dataset for training.

[0014] As a preferred technical solution, step S2 includes the following steps:

[0015] S21, Based on the SOLOv2 instance segmentation model training dataset without anchor boxes, the image feature information extracted by backbone and fpn is combined with the location information to directly predict the preliminary mask and convolution kernel. In the post-processing process, the predicted convolution kernel and the preliminary mask are further convolved to form the final segmentation mask. The training results are saved in the pth weight file.

[0016] S22 converts the pth weight file into a pt file with model structure, and then converts the pt file into a quantized RKNN model file that can be used on edge computing devices with NPUs.

[0017] As a preferred technical solution, in step S13, the image is labeled using the labelme annotation tool, and the dataset includes the labeled image, pig outline coordinates, and pose instance categories.

[0018] As a preferred technical solution, step S3 includes the following steps:

[0019] S31, use the rknn model file to segment the pig's edge contour and give the classification results of different poses of the same class to obtain different pose information of the pig;

[0020] S32, compare the pig outline segmented from the rknn model file with the pre-labeled region, and determine whether the pig has a certain behavior based on the region overlap degree and overlap duration obtained from the comparison.

[0021] As a preferred technical solution, step S3 further includes the following steps:

[0022] S33, based on the image recognition, and according to the number of pigs identified in each frame of the image, the duration information of the pigs' posture and / or behavior indicators in the pen is obtained, expressed as:

[0023]

[0024] Where D represents the duration information of posture and / or behavior indicators, and S n M represents the number of pigs in a certain action of the nth frame. n T represents the total number of pigs identified in the nth frame. total This indicates the total duration of each frame.

[0025] As a preferred technical solution, the duration information data is sent in JSON file format.

[0026] As a preferred technical solution, the posture includes standing posture and lying posture, and the behavior includes eating behavior and drinking behavior.

[0027] A pig posture and behavior recognition system, used to implement a pig posture and behavior recognition method, includes a camera, an edge computing device, a data center, a visualization platform, and a GPU. The camera is communicatively connected to the edge computing device, and both the camera and the edge computing device are located inside the pigsty. The edge computing device is connected to the data center via the Internet. The GPU is used for model training and transmits the trained model file to the edge computing device. The edge computing device is used for data acquisition and processing, and model inference. The data receiving center is used to receive the results output by the edge computing device in a timely manner, process the data, and then display the data processing results on the visualization platform.

[0028] Compared with the prior art, the present invention has the following advantages:

[0029] (1) The present invention uses an anchorless instance segmentation algorithm, which can relatively reduce the consumption of computing resources;

[0030] (2) The present invention calculates directly on the local edge computing device at the camera end, and only needs to send the result back.

[0031] (3) By recognizing real-time video, the standing activities, eating and drinking behaviors of pigs can be quantified into durations, which will help monitor the health status of pigs.

[0032] (4) The quantified duration information of standing activities, eating and drinking behaviors of the present invention can then be used for manual monitoring of pig health status, or can provide a basis for further design of automated pig health status monitoring.

[0033] (5) The overall solution of the present invention significantly reduces economic costs. It no longer requires a backend server with GPU, but can recognize the posture and behavior of pigs through the front-end camera. This will help to deploy a large number of monitoring systems for pig activities and behavior duration. Attached Figure Description

[0034] Figure 1 This is a schematic diagram illustrating the steps of a method for recognizing pig postures and behaviors according to the present invention;

[0035] Figure 2 This is a flowchart illustrating a method for recognizing pig postures and behaviors in a specific embodiment of the present invention.

[0036] Figure 3 This is a schematic diagram of the structure of a pig posture and behavior recognition system according to the present invention. Detailed Implementation

[0037] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0038] Example 1

[0039] like Figures 1 to 3 As shown, this invention provides a scheme for recognizing pig posture and behavior based on an anchor-free convolutional neural network, enabling 24 / 7 monitoring of activity levels and behaviors such as eating and drinking based on real-time video and / or images. Unlike other methods that use object detection, this system uses instance segmentation to accurately locate the pig's position and can distinguish its posture and behavior. Furthermore, the system's model inference process can run on edge devices equipped with NPUs, which will significantly reduce costs.

[0040] To achieve the above objectives, the present invention employs the following technical solutions:

[0041] This invention relates to a system for monitoring the activity and behavior duration of pigs based on an anchorless convolutional neural network. The system includes: several high-definition cameras, an edge computing device, a data receiving center, a visualization platform, and a GPU. The high-definition cameras and the edge computing device are locally connected, and the edge computing device is connected to a data center via the internet. The GPU is used for model training and transmits the trained model file to the edge computing device. The edge computing device is used for data acquisition and processing, and model inference. The edge computing device includes a pig image acquisition module, a pig model recognition and inference module, and a result transmission module. The data receiving center receives the results output by the edge computing device in a timely manner, processes them accordingly, and then displays the results on the visualization platform.

[0042] The image can be a photograph or an image extracted from a video.

[0043] This invention comprises three parts: data acquisition and processing, model training, and model inference.

[0044] The data collection and processing steps are as follows:

[0045] First, collect a large number of overhead images of pigs in the pigpen, including as many images as possible of pigs in various standing and lying positions;

[0046] The collected images are cleaned and filtered to select relatively clear images and remove those that do not meet the requirements.

[0047] The LabelMe annotation tool was used to annotate the images, including pig outlines and posture instance categories, to form a dataset;

[0048] The model training steps are as follows:

[0049] The dataset used for training the SOLOv2 instance segmentation model without anchor boxes is labeled. This model abandons the previous method of generating a large number of proposed candidate boxes. Instead, it directly combines the image feature information extracted by the backbone and FPN with the location information to directly predict the initial mask and convolution kernel. In the post-processing process, the predicted convolution kernel and the initial mask are further convolved to form the final segmentation mask. The training results are saved as a pth weight file. This step can be run on a server with a GPU to accelerate training.

[0050] Convert the pth weight file into a pt file with model structure, and then convert the pt file into a quantized RKNN model file that can be used by edge devices with NPUs;

[0051] Model inference:

[0052] The high-definition camera is fixed directly above the pigpen and is used to capture the various states and behaviors of the pigs in the pen in real time, and transmit the images to the edge computing device.

[0053] The images captured by the high-definition camera need to have the edge coordinates of the feeding and watering troughs marked in advance, which is used to identify the pigs' feeding or drinking behavior.

[0054] The RKNN model file needs to be stored on an edge device for recognizing the captured overhead images of pigs.

[0055] The RKNN model is used to accurately segment the pig's edge contours and to provide classification results for different poses of the same class, thus obtaining information on different pig poses (including standing and lying down).

[0056] The model accurately segments the pig's edge contour and generates an overlap (IoU) with the pre-labeled areas of the feeding troughs and drinking troughs in the pen. Based on the overlap duration, it is determined whether feeding and drinking behavior exists.

[0057] By identifying real-time video and counting the number of pigs in the pen exhibiting standing, lying, eating, and drinking behaviors in each frame, the duration of these activities can be determined, which can be expressed as:

[0058]

[0059] Where D represents the duration information of posture and / or behavior indicators, and S n Let T represent the number of pigs in a certain action in frame n, Mn represent the total number of pigs identified in frame n, and T represent the total number of pigs identified in frame n. total The total duration is represented by the total time spent, so we can determine the percentage of time spent standing, eating, drinking, etc., by pigs in a particular pen. These indicators are of great significance for monitoring the health status of pigs.

[0060] The duration of standing activities, eating, drinking, and other indicators are sent to the data center in JSON file format via HTTP protocol at a frequency of once per minute.

[0061] The data center receives JSON file data, performs statistical analysis on various indicators according to time periods such as per minute, per day, and per week, and stores the data.

[0062] The visualization platform can read and display the duration of standing activities, eating, drinking, and other indicators for different time periods based on different pens, which will help farmers to quantitatively monitor the pigs in the pens at all times.

[0063] The present invention has the following technical effects:

[0064] 1. Unlike other object detection algorithms that use anchor boxes, which require generating a large number of anchor boxes and consume computing resources, thus requiring GPU-equipped servers for training and inference, this invention uses an anchor-free instance segmentation algorithm, which can relatively reduce the consumption of computing resources.

[0065] 2. Existing solutions can move the model inference process for pig posture and behavior recognition forward, while this invention calculates directly on the local edge computing device at the camera end, and only needs to send the results back.

[0066] 3. By recognizing real-time video, the standing activities, eating, and drinking behaviors of pigs can be quantified in terms of duration, which will help monitor the health status of pigs.

[0067] 4. The quantified information on the duration of standing activities, eating and drinking behaviors can then be used for manual monitoring of pig health status, or to provide a basis for further designing automated pig health monitoring systems.

[0068] 5. The overall solution significantly reduces economic costs, eliminating the need for backend servers with GPUs. Instead, it enables the recognition of pigs' postures and behaviors through front-end cameras, which will facilitate the large-scale deployment of monitoring systems for pig activities and behavior duration.

[0069] Example 2

[0070] like Figures 1 to 3 As shown, as a further optimization of Embodiment 1, this embodiment also includes the following technical features based on Embodiment 1:

[0071] This application discloses a pig activity and behavior duration monitoring system based on an anchorless convolutional neural network. The system includes several components, including a high-definition camera, an edge computing device, a data processing center, a visualization platform, and a GPU.

[0072] A high-definition camera located directly above the pigsty captures real-time images of the pigs inside. Using a pre-trained quantized RKNN model in an edge computing device, the camera calculates the pigs' postures and behaviors. Based on the real-time sequence image recognition results, the camera calculates the percentage of time spent standing, eating, and drinking. The results are then transmitted to a data center in JSON format via HTTP protocol at a frequency of once per minute for processing and storage. The visualization platform displays information such as the percentage of time spent standing, eating, and drinking for the pigs in the pen at different times, based on the information stored in the data center. This information can be used as a reference for farm staff and can also be used to automatically identify abnormalities in pigs in the future.

[0073] To obtain a quantized RKNN model, the SOLOv2 instance segmentation model needs to be trained first using a server with a GPU. The SOLOv2 model includes a Backbone module and an FPN module. Then, the features of the FPN are divided into three branches for output: one for classification, one for prediction convolutional kernel, and the last for prediction of the initial mask. The trained model is saved as a pt weight file.

[0074] The pt file is then converted into a .pth file containing the model structure. The converted .pth file is then further quantized into a uint8 type quantized RKNN model file. The output of the RKNN model file requires further post-processing; the predicted convolutional kernels are convolved with the initial mask to obtain the final mask and standing / lying classifications. Based on the overlap duration between the segmented mask and the coordinates of the feeding and drinking troughs pre-marked on the pigpen, it is determined whether the pigs are exhibiting feeding or drinking behavior.

[0075] Due to the advantages of the model itself, this invention can not only monitor the duration of pig posture and behavior, but also count the pigs in the pen. This is because the model itself can obtain the outline and classification of the target pigs, and thus realize the counting of the number of pigs.

[0076] As described above, the present invention can be implemented well.

[0077] All features disclosed in all embodiments of this specification, or steps in all methods or processes implied in the disclosure, may be combined and / or extended or replaced in any way, except for mutually exclusive features and / or steps.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Based on the technical essence of the present invention, any simple modifications, equivalent substitutions, and improvements made to the above embodiments within the spirit and principles of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for recognizing pig postures and behaviors, characterized in that, Based on real-time video and / or images of pigs, an anchorless convolutional neural network is used for model training and inference to recognize the pigs' postures and behaviors; including the following steps: S1, Data Acquisition and Processing: Acquire images of pigs and obtain a dataset for training based on the images; S2, Model Training: The dataset is trained using the SOLOv2 instance segmentation model without anchor boxes. After model conversion, a quantized rknn model file is obtained for use on edge computing devices with NPUs. S3, Model Inference: Based on the RKNN model, model inference is performed to identify the pig's posture and behavior; Step S1 includes the following steps: S11, Collect images of pigs inside the pigpen; S12, Clean and filter the acquired images: S13, Annotate the images to form a dataset for training; Step S2 includes the following steps: S21, Based on the SOLOv2 instance segmentation model training dataset without anchor boxes, the image feature information extracted by backbone and fpn is combined with the location information to directly predict the preliminary mask and convolution kernel. In the post-processing process, the predicted convolution kernel and the preliminary mask are further convolved to form the final segmentation mask. The training results are saved in the pth weight file. S22, convert the pth weight file into a pt file with model structure, and convert the pt file into a quantized RKNN model file that can be used in edge computing devices with NPU; Step S3 includes the following steps: S31, use the rknn model file to segment the pig's edge contour and give the classification results of different poses of the same class to obtain different pose information of the pig; S32, compare the pig outline segmented from the rknn model file with the pre-labeled region, and determine whether the pig has a certain behavior based on the overlap degree and overlap duration obtained from the comparison. Step S3 also includes the following steps: S33, based on the image recognition, and according to the number of pigs identified in each frame of the image, the duration information of the pigs' posture and / or behavior indicators in the pen is obtained, expressed as: ； Where D represents the duration information of posture and / or behavior indicators, Represented as the first The number of pigs in a certain action of a frame. Represented as the first The total number of pigs was identified from the frames. This indicates the total duration of each frame.

2. The method for recognizing pig postures and behaviors according to claim 1, characterized in that, In step S13, the image is labeled using the labelme annotation tool. The dataset includes the labeled image, pig outline coordinates, and pose instance categories.

3. The method for recognizing pig postures and behaviors according to claim 1, characterized in that, Duration information data is sent via JSON file format.

4. A method for recognizing pig postures and behaviors according to any one of claims 1 to 3, characterized in that, The postures include standing postures and lying postures, and the behaviors include eating behaviors and drinking behaviors.

5. A system for recognizing pig postures and behaviors, characterized in that, A method for recognizing pig postures and behaviors as described in any one of claims 1 to 4 includes a camera, an edge computing device, a data center, a visualization platform, and a GPU; the camera is communicatively connected to the edge computing device, both the camera and the edge computing device are located inside the pigsty, and the edge computing device is connected to the data center via the Internet; the GPU is used for model training and transmits the trained model file to the edge computing device, the edge computing device is used for data acquisition and processing, and model inference, and the data receiving center is used to receive the results output by the edge computing device in a timely manner, process the data, and then display the data processing results on the visualization platform.

Images