Non-contact pig body size estimation method and device based on multi-position camera

Abstract

The invention relates to the technical field of live pig body size measurement, in particular to a non-contact pig body size estimation method and device based on a multi-position camera. The method comprises the following steps: firstly, acquiring back and side images of a pig through a multi-position camera, generating a standardized data set, then carrying out live pig attitude detection by utilizing an improved YOLOv11 model, screening out an image with an ideal attitude by introducing a deformable convolution and adaptive spatial feature fusion module, and carrying out real-time detection on the attitude of the live pig. Secondly, accurately positioning key point pixel coordinates related to the pig body size by using an HRNet key point positioning model, calibrating and correcting camera image distortion through internal and external parameters, calculating a conversion factor of pixels and actual size, and finally converting the pixel distance of the key points into actual body size data. According to the method, the multi-camera layout and the deep learning technology are combined, high-precision, automatic and stress-free pig body size measurement is realized in a complex breeding environment, and an efficient solution is provided for large-scale breeding and pig health management.

Description

Technical Field

[0001] The present invention relates to the technical field of pig body size measurement, and in particular to a non-contact pig body size estimation method and device based on a multi-position camera. Background Art

[0002] With the rapid development of modern breeding industry, the body size measurement of pigs is an important indicator for evaluating their health status, growth and development, and production performance. As an important phenotypic evaluation indicator of pigs, body size can not only evaluate the growth status of pigs, but also be used for breeding of breeding pigs. It occupies an important position in pig farm management and scientific research. However, traditional body size measurement methods mainly rely on manual operation and often require physical contact, which is not only inefficient and has large errors, but also easily causes stress reactions to pigs, affecting their normal growth and health. In addition, due to the complexity of the breeding environment, such as the large number of pigs, frequent movement behaviors, and changes in light and posture, the implementation of manual measurement faces many challenges. Therefore, it is urgent to solve the technical problems of breeding pig body size measurement. Summary of the invention

[0003] The purpose of the present invention is to address the problems existing in the background technology and to propose a non-contact pig body size estimation method and device based on a multi-position camera.

[0004] The technical solution of the present invention is a non-contact pig body size estimation method based on a multi-position camera, comprising the following specific implementation steps:

[0005] S1. Use a camera to collect images of the pig's side and back, and use the shadow difference method to remove background information and enhance target information; use dynamic threshold segmentation to process pig images, use median fuzzy filtering for multiple iterations to perform denoising, eliminate noise to the greatest extent, and construct a standardized image data set;

[0006] S2. The YOLOv11 target detection model is improved using three modules: Mosaic9 data enhancement, deformable convolution, and adaptive spatial feature fusion. The improved target detection model YOLOv11DA is then trained and its detectability is evaluated on a standardized image dataset to obtain a pig posture detection model.

[0007] S3. Use the key point positioning model HRNet to directly locate the key points of the body size of the live pig. Use the standardized image data set to train the HRNet model to obtain the key point positioning model, and input the collected pig images to output the image containing the key point positioning of the pig body size, and obtain the key point positioning coordinates of the pig body size;

[0008] S4. Use intrinsic parameters to correct the image distortion, and use extrinsic parameters to convert the pig body size key point positioning coordinates in the camera coordinate system to the world coordinate system, and convert the pixel distance of the key point into the actual body size result by using pixel conversion technology.

[0009] Preferably, the process of constructing the standardized image dataset is as follows:

[0010] S21, configure two sets of high-resolution cameras, one set for collecting images of the pig's back, and the other set for collecting images of the side;

[0011] S22, by establishing a background model and performing continuous frame comparison, the static background information in the image is accurately removed, and only the target part of the pig is retained;

[0012] S23, dynamically adjusting the threshold according to the gray value of the image, and separating the pig outline from the background through adaptive segmentation;

[0013] S24, multiple iterations of median filtering to remove random noise in the image, retain key structural information, and ensure clear pig outlines;

[0014] S25. Use the LabelMe annotation tool to annotate the images, ensure that the dataset contains the pixel coordinates of the key points of the pig's location, posture category, and body size measurement, and convert the annotated data into COCO format to obtain a standardized image dataset.

[0015] Preferably, the process of obtaining the positioning coordinates of the key points of the pig's body size is as follows:

[0016] S31, collecting pig images, inputting the collected pig images into the pig posture detection model YOLOv11DA, outputting the pig posture category, and obtaining the ideal posture image by screening the ideal posture, that is, the image of the back view and the side view;

[0017] S32, select the HRNet model, use the constructed standardized image data set for training, and obtain the key point positioning model;

[0018] S33, inputting the ideal posture image into the key point positioning model, and outputting the pixel coordinates of the key points of the pig's body size, including the key point coordinates of the back view and the key point coordinates of the side view, that is, obtaining the key point positioning coordinates of the pig's body size.

[0019] Preferably, the conversion process of converting the pixel distance of the key point into the actual body size result by using the pixel conversion technology is as follows:

[0020] S41, calibrating the camera to correct its distortion effect and ensure a linear relationship between the pixel distance in the image and the actual size;

[0021] S42, converting the pixel coordinates in the camera coordinate system into the world coordinate system through external calibration, thereby realizing the mapping between the body size data in the image and the actual body size;

[0022] S43, calculating a mapping scaling factor ρ between pixels and actual sizes by knowing the size of the calibration object and its pixel length in the image;

[0023] S44, using the mapping scaling factor ρ, according to the pig body size calculation rule, converting the pixel distance of the body size key point into the actual body size value.

[0024] Preferably, the construction process of the pig posture detection model is as follows:

[0025] S51, introduce deformable convolution, dynamically adjust the shape of the convolution kernel according to different image contents, and better adapt to the changes in pig posture;

[0026] S52, introduce adaptive spatial feature fusion module, combine multi-level feature information, and improve the ability of pig posture detection model to extract pig features under different viewing angles and ambient lighting conditions;

[0027] S53, the constructed standardized image dataset is divided into a training set and a test set in a ratio of 7:3, and the Mosaic9 data augmentation technology is introduced. First, four pictures are randomly selected from the training set, and then each picture is randomly scaled, cropped and arranged to adapt to the input size of the model. Then, the four pictures are spliced ​​together to form a new training sample, and finally the generated new sample is input into the model for training;

[0028] S54. Use the test set to test the model effect and obtain a pig posture detection model.

[0029] The technical solution of the present invention is as follows: a non-contact pig body size estimation device based on a multi-position camera, which is used for the above-mentioned non-contact pig body size estimation method based on a multi-position camera. The non-contact pig body size estimation device based on a multi-position camera is made of aluminum and acrylic plate, with dimensions of 1.5m in length, 0.5m in width, and 1m in height. The bottom steel structure is a slatted floor structure, a weighing sensor is installed at the bottom, and a camera is installed on the top and the side respectively, which are connected to a laptop computer. In the actual operation of collecting pig images, according to the image collection rules, the pig images are collected from different directions to obtain the pig images.

[0030] Preferably, the image acquisition rule is: a camera is placed at a height of 1.5 meters directly above the non-contact pig body size estimation device based on a multi-position camera for shooting, an image is captured every 1 second for 2 minutes, and the resolution of the captured image is 1280×720, and the format is jpg; and another camera is placed at a distance of 0.4 meters to the left of the non-contact pig body size estimation device based on a multi-position camera for shooting, an image is captured every 1 second for 2 minutes, and the resolution of the captured image is 1280×720, and the format is jpg.

[0031] Compared with the prior art, the above technical solution of the present invention has the following beneficial technical effects:

[0032] The present invention designs a non-contact pig body size estimation method and device based on a multi-camera, which provides a lightweight, easy-to-deploy, high-precision automatic body size measurement solution for large-scale pig farms. It can run smoothly on the edge of the farm and on terminal equipment, providing an intelligent solution for pig breeding:

[0033] (1) Improve measurement accuracy: Use high-resolution cameras to capture images of the pig's back and side to ensure that key body positions are fully captured to avoid blind spots that affect measurement accuracy. Use the shadow difference method to eliminate background information and enhance target information. Use dynamic threshold segmentation to process pig images, and use median fuzzy filtering for multiple iterations to perform denoising to eliminate noise to the greatest extent possible and build a standardized image data set. These steps help improve the accuracy of subsequent model training and testing;

[0034] (2) Improve the generalization ability of the model: The YOLOv11 target detection model is improved by introducing three modules: Mosaic9 data enhancement, deformable convolution, and adaptive spatial feature fusion. This improves the model's ability to extract pig features under different viewing angles and ambient lighting conditions, helps the model better adapt to various complex scenarios, and improves its generalization ability.

[0035] (3) Accurate key point positioning: Using the HRNet key point positioning model, image features are accurately extracted at multiple levels and scales, which is suitable for key point positioning tasks of pigs in different postures. This helps to improve the accuracy and reliability of body measurement;

[0036] (4) Non-contact measurement: The non-contact measurement method avoids the animal stress response and human errors that may be caused by traditional contact measurement, and improves the safety and comfort of measurement;

[0037] (5) Automation and efficiency: Through automated image acquisition, processing, and analysis processes, the need for manual operation is reduced, and measurement efficiency and consistency are improved;

[0038] (6) Flexibility and scalability: Adjust and optimize according to different needs, such as replacing different key point positioning models or introducing other advanced image processing technologies to further improve measurement performance;

[0039] (7) Cost-effectiveness: Compared with traditional contact measurement equipment and methods, the present invention may have lower costs and higher cost-effectiveness, especially in large-scale farming scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flow chart of a non-contact pig body size estimation method based on multiple cameras of the present invention;

[0041] Figure 2 A schematic diagram of key point coordinates from a back perspective output by a key point positioning model of a non-contact pig body size estimation method based on a multi-camera according to the present invention;

[0042] Figure 3 It is a schematic diagram of the key point coordinates from the side view output by the key point positioning model of the non-contact pig body size estimation method based on multiple cameras of the present invention;

[0043] Figure 4 It is a schematic diagram of experimental measurement results of a non-contact pig body size estimation method based on a multi-position camera of the present invention;

[0044] Figure 5 This is a schematic diagram of the application of a non-contact pig body size estimation device based on a multi-position camera proposed by the present invention in an actual production process. DETAILED DESCRIPTION

[0045] Embodiment 1: The present invention proposes a non-contact pig body size estimation method based on a multi-position camera, such as Figure 1 As shown, the specific implementation steps include the following:

[0046] S1. Dataset construction: Use a camera to collect images of the pig's side and back, and use the shadow difference method to remove background information and enhance target information; use dynamic threshold segmentation to process pig images, use median fuzzy filtering for multiple iterations to perform denoising, eliminate noise to the greatest extent, and construct a standardized image dataset;

[0047] S2. Construction of pig posture detection model: The YOLOv11 target detection model is improved using three modules: Mosaic9 data enhancement, deformable convolution, and adaptive spatial feature fusion. The improved target detection model YOLOv11DA is then trained and its detectability is evaluated on a standardized image dataset to obtain a pig posture detection model.

[0048] S3. Positioning of key points of pig body size: Use the key point positioning model HRNet to directly position the key points of pig body size, train the HRNet model, obtain the key point positioning model, and input the collected pig images to obtain an image containing the positioning of key points of pig body size;

[0049] S4. Pig body size estimation: Use intrinsic parameters to correct the distortion of the image containing the key points of pig body size, and use extrinsic parameters to convert the key points of pig body size in the camera coordinate system to the world coordinate system. For the key points of pig body size, the pixel distance of the key points is converted into the actual body size result by using pixel conversion technology.

[0050] In an optional embodiment, a standardized image dataset is constructed, and images of the pig's back and side are collected. The images are subjected to background removal, noise processing and formatting, and finally a high-quality standardized image dataset is constructed as the basis for subsequent model training and testing. The specific implementation process is as follows:

[0051] S11, configure two sets of high-resolution cameras, one set for collecting images of the pig's back, and the other set for collecting images of the side;

[0052] The camera’s installation position and angle must ensure that the key positions of the pigs can be fully captured to avoid blind spots that affect measurement accuracy. The acquisition environment must also have uniform lighting to avoid image quality issues caused by strong direct light or shadows, ensuring consistency and accuracy during image acquisition.

[0053] S12, by establishing a background model and performing continuous frame comparison, the static background information in the image is accurately removed, and only the target part of the pig is retained;

[0054] S13, dynamically adjusting the threshold according to the gray value of the image, and separating the pig outline from the background through adaptive segmentation;

[0055] S14, multiple iterations of median filtering to remove random noise in the image, retain key structural information, and ensure clear pig outlines;

[0056] S15. Use the LabelMe annotation tool to annotate the images, ensure that the dataset contains the pixel coordinates of the pig's location, posture category (including but not limited to standing, lying), and key points of body measurement (including but not limited to shoulder height, back length, and abdominal height), and convert the annotated data into COCO format.

[0057] In an optional embodiment, the pig posture detection is based on the constructed pig posture detection model, by improving the YOLOv11 target detection model, combining Mosaic9 data enhancement technology, adaptive spatial feature fusion module and deformable convolution technology, to accurately detect different pig postures and improve the accuracy of subsequent body measurement. The specific implementation process is as follows:

[0058] S21. Deformable Convolutional Networks (DCN) are introduced to enable the pig posture detection model to dynamically adjust the shape of the convolution kernel according to different image contents, so as to better adapt to the changes in pig posture (including but not limited to bowing, bending or standing);

[0059] It should be noted that Deformable Convolutional Networks (DCN) is a technology that enhances the conversion modeling capability of Convolutional Neural Networks (CNNs). When processing images, the convolution kernel of traditional convolutional neural networks operates on a fixed geometric structure, such as a 3×3 or 5×5 rectangular grid. This fixed sampling method limits the model's ability to model geometric transformations. Deformable convolution introduces an additional offset, allowing the convolution kernel to flexibly adjust its position and shape according to the characteristics of the input image during training, thereby better capturing the details in the image, especially the ability to adapt to deformation and distortion. Specifically, deformable convolution adds an additional direction parameter (or offset) to each convolution kernel element. This offset is obtained through parallel network learning. Deformable convolution can significantly improve the performance of the model for complex visual tasks, and significant performance improvements can be achieved by increasing the amount of calculation very little.

[0060] S22, introduce the adaptive spatial feature fusion module (ASFF), combine multi-level feature information, improve the pig posture detection model's ability to extract pig features under different viewing angles and ambient lighting conditions, and improve the accuracy of posture detection;

[0061] It should be noted that the adaptive spatial feature fusion module (ASFF) is an efficient and flexible feature fusion method that solves the problem of inconsistent feature pyramids in single-shot detectors by adaptively fusing multi-scale features. The core idea of ​​ASFF is to adaptively fuse multi-scale features by learning weights, so that the network can directly learn how to spatially filter features at other levels, thereby retaining only useful information for combination. For features at a certain level, features at other levels are first adjusted to the same resolution, and then trained to find the best fusion method. At each spatial position, features at different levels are adaptively fused together. ASFF consists of the following three components: a feature selection module for selecting features to be fused from features of different levels or scales, a feature fusion module responsible for fusing the selected features, and a feature scaling module for scaling the fused features to ensure their compatibility with other features or models;

[0062] S23, the constructed standardized image dataset is divided into a training set and a test set in a ratio of 7:3, and the Mosaic9 data augmentation technology is introduced. First, four pictures are randomly selected from the training set, and then each picture is randomly scaled, cropped, and arranged to adapt to the input size of the model. Then, the four pictures are spliced ​​together to form a new training sample, and finally the generated new sample is input into the model for training;

[0063] S24. Use the test set to test the model effect and obtain the pig posture detection model YOLOv11DA.

[0064] In an optional embodiment, the specific implementation steps of locating the key points of the pig's body size are as follows:

[0065] S31, collecting pig images, inputting the collected pig images into the pig posture detection model YOLOv11DA, outputting the pig posture category, and obtaining the ideal posture image by screening the ideal posture (including but not limited to the back perspective and the side perspective);

[0066] S32, select HRNet (High Resolution Network), use the constructed data set for training, and obtain the key point positioning model;

[0067] It should be noted that HRNet (High Resolution Network) is a high-resolution network that can accurately extract image features at multiple levels and scales. The model has high accuracy in key point positioning and can handle the key point positioning task of pigs in different postures.

[0068] S33, input the ideal posture image into the key point positioning model, and output the pixel coordinates of the key points of the pig's body size, including the key point coordinates of the back view (such as Figure 2 ) and the key point coordinates of the side view (as shown Figure 3 shown);

[0069] Among them, key points, i.e. key positions related to body measurement, include but are not limited to shoulder height, front end of spine, end of spine, and lowest point of abdomen.

[0070] In an optional embodiment, the pig body size estimation converts the pixel distance of key points into actual body size data to complete the body size estimation, maps the pixel data to the actual size through the camera calibration process, and outputs the accurate body size result. The specific implementation process is as follows:

[0071] S41, calibrating the camera to correct its distortion effect and ensure a linear relationship between the pixel distance in the image and the actual size;

[0072] S42, converting the pixel coordinates in the camera coordinate system into the world coordinate system through external calibration, thereby realizing the mapping between the body size data in the image and the actual body size;

[0073] S43, calculating a mapping scaling factor ρ between pixels and actual sizes by knowing the size of the calibration object and its pixel length in the image;

[0074] S44. Use the pixel conversion factor to convert the pixel distance of the key points of body size into the actual body size value. The calculation rules of the pig body size are shown in Table 1:

[0075] Table 1 Pig body size calculation rules

[0076]

[0077] Where ρ represents the mapping scaling factor, (X i , Y i ) represents the pixel coordinate point of body scale key point i.

[0078] In order to verify the effect of the non-contact pig body size estimation method based on multiple cameras provided in this embodiment (hereinafter referred to as the method of this embodiment), the following experiments were conducted:

[0079] Twenty black pigs with a body weight of about 100 kg were selected and their actual body size values ​​were measured manually. Each body size was measured three times and the measurement results are as follows: Figure 4 As shown;

[0080] The average relative errors of the body length, body width, hip width and body height measured by taking the average value of 3 times and the measured values ​​were 2.06%, 2.76%, 3.46% and 3.53%, respectively, which were within the set error threshold, that is, the measurement effect of body size data detected by the method of this embodiment was good and met the actual production requirements.

[0081] Embodiment 2, a non-contact pig body size estimation device based on a multi-position camera proposed by the present invention is used to execute a non-contact pig body size estimation method based on a multi-position camera proposed in embodiment 1. The non-contact pig body size estimation device based on a multi-position camera is made of aluminum and acrylic plate, with dimensions of 1.5m in length, 0.5m in width, and 1m in height. The bottom steel structure is a slatted floor structure, such as Figure 5 As shown, a weighing sensor is installed at the bottom, and a camera is installed at the top and side, which are connected to a laptop computer. In the actual operation of collecting pig images, according to the image collection rules, the images are collected from different directions, including but not limited to top view and side view;

[0082] The image acquisition rules are as follows:

[0083] A camera was placed at a height of about 1.5 meters directly above the non-contact pig body size estimation device based on a multi-camera for shooting, and an image was collected every 1 second for 2 minutes. The resolution of the collected image was 1280×720 and the format was jpg. Another camera was placed at a distance of 0.4 meters to the left of the non-contact pig body size estimation device based on a multi-camera for shooting, and an image was collected every 1 second for 2 minutes. The resolution of the collected image was 1280×720 and the format was jpg.

[0084] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited thereto, and various changes can be made within the knowledge scope of technicians in the relevant technical field without departing from the purpose of the present invention.

Claims

1. A non-contact pig body size estimation method based on multiple cameras, characterized in that: The specific implementation steps include the following: S1. Use a camera to collect images of the pig's side and back, and use the shadow difference method to remove background information and enhance target information; use dynamic threshold segmentation to process pig images, use median fuzzy filtering for multiple iterations to perform denoising, eliminate noise to the greatest extent, and construct a standardized image data set; S2. The YOLOv11 target detection model is improved using three modules: Mosaic9 data enhancement, deformable convolution, and adaptive spatial feature fusion. The improved target detection model YOLOv11DA is then trained and its detectability is evaluated on a standardized image dataset to obtain a pig posture detection model. S3. Use the key point positioning model HRNet to directly locate the key points of the body size of the live pig. Use the standardized image data set to train the HRNet model to obtain the key point positioning model, and input the collected pig images to output the image containing the key point positioning of the pig body size, and obtain the key point positioning coordinates of the pig body size; S4. Use intrinsic parameters to correct the image distortion, and use extrinsic parameters to convert the pig body size key point positioning coordinates in the camera coordinate system to the world coordinate system, and convert the pixel distance of the key point into the actual body size result by using pixel conversion technology.

2. The non-contact pig body size estimation method based on a multi-position camera according to claim 1 is characterized in that: The construction process of the standardized image dataset is as follows: S21, configure two sets of high-resolution cameras, one set for collecting images of the pig's back, and the other set for collecting images of the side; S22, by establishing a background model and performing continuous frame comparison, the static background information in the image is accurately removed, and only the target part of the pig is retained; S23, dynamically adjusting the threshold according to the gray value of the image, and separating the pig outline from the background through adaptive segmentation; S24, multiple iterations of median filtering to remove random noise in the image, retain key structural information, and ensure clear pig outlines; S25. Use the LabelMe annotation tool to annotate the images, ensure that the dataset contains the pixel coordinates of the key points of the pig's location, posture category, and body size measurement, and convert the annotated data into COCO format to obtain a standardized image dataset.

3. The non-contact pig body size estimation method based on multiple cameras according to claim 1 is characterized in that: The process of obtaining the positioning coordinates of the key points of the pig's body size is as follows: S31, collecting pig images, inputting the collected pig images into the pig posture detection model YOLOv11DA, outputting the pig posture category, and obtaining the ideal posture image by screening the ideal posture, that is, the image of the back view and the side view; S32, select the HRNet model, use the constructed standardized image data set for training, and obtain the key point positioning model; S33, inputting the ideal posture image into the key point positioning model, and outputting the pixel coordinates of the key points of the pig's body size, including the key point coordinates of the back view and the key point coordinates of the side view, that is, obtaining the key point positioning coordinates of the pig's body size.

4. The non-contact pig body size estimation method based on multi-position cameras according to claim 1 is characterized in that: The conversion process of converting the pixel distance of key points into the actual body size result by using pixel conversion technology is as follows: S41, calibrating the camera to correct its distortion effect and ensure a linear relationship between the pixel distance in the image and the actual size; S42, converting the pixel coordinates in the camera coordinate system into the world coordinate system through external calibration, thereby realizing the mapping between the body size data in the image and the actual body size; S43, calculating a mapping scaling factor ρ between pixels and actual sizes by knowing the size of the calibration object and its pixel length in the image; S44, using the mapping scaling factor ρ, according to the pig body size calculation rule, converting the pixel distance of the body size key point into the actual body size value.

5. The non-contact pig body size estimation method based on multi-position cameras according to claim 1 is characterized in that: The construction process of the pig posture detection model is as follows: S51, introduce deformable convolution, dynamically adjust the shape of the convolution kernel according to different image contents, and better adapt to the changes in pig posture; S52, introduce adaptive spatial feature fusion module, combine multi-level feature information, and improve the ability of pig posture detection model to extract pig features under different viewing angles and ambient lighting conditions; S53, the constructed standardized image dataset is divided into a training set and a test set in a ratio of 7:3, and the Mosaic9 data augmentation technology is introduced. First, four pictures are randomly selected from the training set, and then each picture is randomly scaled, cropped and arranged to adapt to the input size of the model. Then, the four pictures are spliced ​​together to form a new training sample, and finally the generated new sample is input into the model for training; S54. Use the test set to test the model effect and obtain a pig posture detection model.

6. A non-contact pig body size estimation device based on a multi-position camera, which is used to execute the non-contact pig body size estimation method based on a multi-position camera according to claims 1 to 5, characterized in that: The non-contact pig body size estimation device based on multiple cameras is made of aluminum and acrylic plate, with dimensions of 1.5m in length, 0.5m in width and 1m in height. The bottom steel structure is a slatted floor structure, with a weighing sensor installed at the bottom. A camera is installed on the top and side, which are connected to a laptop computer. In the actual operation of collecting pig images, the images are collected from different directions according to the image collection rules to obtain pig images.

7. The non-contact pig body size estimation device based on a multi-position camera according to claim 6, characterized in that: The image acquisition rule is as follows: a camera is placed 1.5 meters above the non-contact pig body size estimation device based on a multi-camera for shooting, and an image is collected every 1 second for 2 minutes. The resolution of the collected image is 1280×720 and the format is jpg. Another camera is placed 0.4 meters to the left of the non-contact pig body size estimation device based on a multi-camera for shooting, and an image is collected every 1 second for 2 minutes. The resolution of the collected image is 1280×720 and the format is jpg.

Images