Method for measuring body size and weight of live pig based on RGB-D

Abstract

The invention discloses a live pig body size and weight measuring method based on RGB-D. The method comprises the following steps: S1, obtaining a left vision rotation translation matrix and a right vision rotation translation matrix; s2, acquiring a three-view RGB image and a depth image of a single pig; s3, acquiring a body scale measurement key point; s4, acquiring a left viewpoint cloud, a right viewpoint cloud and an overlook point cloud, and performing point cloud registration to obtain a complete pig point cloud C1; projecting the body size measurement key points to the complete pig point cloud C1; s5, obtaining a final complete pig point cloud C2; s6, calculating the body length, the body width, the body height and the abdominal girth of the pig by using the final complete pig point cloud C2, and forming body size data; and S7, substituting the body size data of the pig into the multiple linear regression prediction model to predict the weight. According to the method for measuring the body size and the weight of the live pig, the accuracy and the stability of extracting the position of the body size measuring point are improved, and the accuracy of the body size and the weight measuring result of the pig is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of contactless pig body size and weight measurement, and in particular to a pig body size and weight measurement method based on RGB-D. Background Art

[0002] Pig farming is a vital component of my country's agricultural economy. Advances in machine vision and deep learning are driving the scale, integration, and intelligent development of the livestock industry. Pig body measurements are crucial indicators of growth. Traditionally, manual measurements of pig body measurements are inefficient and prone to causing stress reactions in pigs, which in turn increases measurement errors. Therefore, utilizing visual information to achieve contactless, rapid, and accurate pig body measurement can effectively improve production efficiency, reduce the workload of farmers and the stress reactions of pigs, and ultimately enhance the profitability of farms.

[0003] Patent CN113920453A discloses a deep learning-based method for estimating pig body size and weight. The method includes the following steps: acquiring pig images, detecting key points using Keypoint-RCNN, correcting the pig's body tilt, and calculating body size data using a ResNet-101 feature extraction network. This method uses only RGB images of the pig's back to estimate body size. The accuracy of this estimate is limited by the image quality and the pig's condition. Furthermore, it only calculates shoulder width, hip width, and body length, and cannot calculate three-dimensional body dimensions such as chest and abdominal circumference.

[0004] Patent CN114693711A discloses a pig weight estimation method based on Kinect v2. This method uses a camera to capture depth and color images of the pig's back. The color image is used to identify individual pigs and obtain their weight, while the depth image is used through a series of image processing steps to obtain the pig's body dimensions. This combined body dimension and weight data is then used to construct a pig weight prediction model using a regression model. This method uses the depth image to obtain the pig's outline and then measures its dimensions. However, the curvature of the pig's body affects the position of the shoulder width measurement point, which in turn affects shoulder width. The undulations of the pig's back can increase errors in length measurement.

[0005] Patent CN115546282A discloses a stereoscopic vision-based pig body measurement system. This system uses dual Kinectv2 cameras to capture bilateral point clouds of pigs. Pig images suitable for measurement are manually selected and the point clouds are stitched together using a calibrated rotation and translation matrix. The system then measures pig body parameters using projection, minimum bounding rectangle (MBR), and slices. In this pig body measurement system, the length and width of the BRR do not represent the pig's actual length and width. Even slight bending of the pig's body can cause the BRR and slice positions to shift, resulting in unstable measurement results. Summary of the Invention

[0006] In response to the problems of the prior art, the present invention provides a pig body size measurement method based on RGB-D.

[0007] Another object of the present invention is to provide a pig weight measurement method based on RGB-D.

[0008] The present invention is achieved through the following technical solutions.

[0009] A method for measuring pig body size based on RGB-D, comprising:

[0010] S1: Set an area as the area to be collected, set up an RGB-D camera directly above the area to be collected, and set up an RGB-D camera on the left and right sides of the area to be collected, with the lenses of all three RGB-D cameras aimed at the area to be collected; calibrate the three RGB-D cameras to obtain the left-view rotation and translation matrix and the right-view rotation and translation matrix;

[0011] S2, using three RGB-D cameras to simultaneously collect RGB images and depth images of three perspectives of a single pig in the area to be collected and use them as a sample. Each sample includes: a top-view RGB image, a left-view RGB image, a right-view RGB image, a top-view depth map, a left-view depth map, and a right-view depth map of a single pig collected simultaneously;

[0012] S3, including S3-1 to S3-3:

[0013] S3-1, inputting the top-view RGB image of the sample into the trained first measurement point extraction model, the first measurement point extraction model being used to predict n spinal points of the pig in the top-view RGB image to obtain n spinal feature points corresponding to the pig in the sample, wherein the mean square error between the predicted value output by the first measurement point extraction model and the true value is within 6 pixels, and the n spinal points are n points evenly distributed on the midline of the pig's back;

[0014] The first spinal feature point among the n spinal feature points close to the pig's head is used as the first body length measurement point of the pig, the last spinal feature point among the n spinal feature points is used as the second body length measurement point of the pig, the first body length measurement point and the second body length measurement point are used as body length measurement points respectively, and the second spinal feature point among the n spinal feature points close to the pig's head is used as the body height measurement point of the pig.

[0015] S3-2, check whether the relative offset angle between the n spine feature points in the top-view RGB image is within the range [θ1, θ2]. If the relative offset angle is not within the range [θ1, θ2], delete the sample and re-execute S2 to obtain the sample; if the relative offset angle is within the range [θ1, θ2], execute S3-3.

[0016] S3-3, inputting the left-view RGB image of the sample into the trained second measurement point extraction model, predicting the body width measurement points and the left abdominal circumference points on the left side of the pig, and obtaining the predicted points of the body width measurement points and the left abdominal circumference points on the left side of the pig; inputting the right-view RGB image of the sample into the trained third measurement point extraction model, predicting the body width measurement points and the right abdominal circumference points on the right side of the pig, and obtaining the predicted points of the body width measurement points and the right abdominal circumference points on the right side of the pig;

[0017] The left body width measurement point is the widest point on the back edge of the pig's left shoulder blade, and the right body width measurement point is the widest point on the back edge of the pig's right shoulder blade. The abdominal girth point is the point on the pig's abdominal girth line, which is the longest line around the pig's abdomen.

[0018] S4, converting the RGB images and depth images of the three perspectives of the sample into point clouds respectively, and obtaining the left view point cloud, right view point cloud and top view point cloud of the sample from the RGB images and depth images of the three perspectives; performing point cloud registration on the left view point cloud, right view point cloud and top view point cloud of the sample using the left view rotation and translation matrix and the right view rotation and translation matrix obtained in step S1, and obtaining the complete pig point cloud C1 corresponding to the sample;

[0019] The body length measurement point, body height measurement point, left body width measurement point prediction point, left abdominal circumference point prediction point, right body width measurement point prediction point and right abdominal circumference point prediction point of the pig are used to form the body size measurement key points; the body size measurement key points are projected onto the complete pig point cloud C1 to obtain the complete pig point cloud C1 containing the body size measurement key points;

[0020] S5, using straight-through filtering, statistical filtering, and random sampling consistency algorithm to segment and remove the background, ground, and outliers from the complete pig point cloud C1 containing the key points of body measurement, and obtain the final complete pig point cloud C2;

[0021] In the S5, S5 includes S5-1 to S5-3:

[0022] S5-1, use a straight-through filter to filter the complete pig point cloud C1 containing the key points of body measurement, remove the background and retain only the point cloud of the pig area in the area to be collected, and obtain the complete pig point cloud C1 after straight-through filtering.

[0023] S5-2, use a statistical filter to filter the complete pig point cloud C1 after straight-through filtering to obtain the complete pig point cloud C1 after statistical filtering:

[0024] S5-3, using a random sampling consistency algorithm to remove the ground in the complete pig point cloud C1 after statistical filtering, to obtain the final complete pig point cloud C2.

[0025] S6, using the first and second body length measurement points in the final complete pig point cloud C2 to make vertical slices, and calculating the pig's body length by quadratic B-spline curve fitting;

[0026] The pig's body height was calculated using the straight-line distance between the body height measurement point and the ground;

[0027] The distance between the predicted point of the body width measurement point on the left and the predicted point of the body width measurement point on the right is taken as the body width of the pig;

[0028] The abdominal circumference point cloud curve is obtained by making vertical slices using the abdominal circumference point prediction points on the left and the abdominal circumference point prediction points on the right, and the abdominal circumference of the pig is obtained by elliptical fitting of the abdominal circumference point cloud curve using the least squares method; the body length, body width, body height and abdominal circumference of the pig constitute the body size data.

[0029] In S1, the calibration method includes: placing a calibration rod in the area to be collected, obtaining RGB images and depth images of the calibration rod from three perspectives through three RGB-D cameras; converting the RGB image and depth image of each perspective of the calibration rod into a point cloud, and obtaining three sets of point clouds from the RGB images and depth images of the three perspectives: a left-view point cloud, a right-view point cloud, and a top-view point cloud;

[0030] The coordinate system of the top-view point cloud of the calibration rod is used as the reference coordinate system, and the left view point cloud and the right view point cloud of the calibration rod are respectively transformed into the reference coordinate system to obtain the left view rotation and translation matrix and the right view rotation and translation matrix.

[0031] In said S3, n≥3.

[0032] In S3, the starting point of the pig's back midline is the center position of the line connecting the midpoints of the pig's two ears, and the end point of the pig's back midline is the point at the first natural whorl at the pig's tail root.

[0033] In S3, the left view point cloud, right view point cloud and overhead view point cloud of the sample are aligned: the left view point cloud is multiplied by the left view rotation and translation matrix, so that the left view point cloud is converted to the coordinate system where the overhead view point cloud is located; the right view point cloud is multiplied by the right view rotation and translation matrix, so that the right view point cloud is converted to the coordinate system where the overhead view point cloud is located. At this time, the complete pig point cloud C1 is obtained in the coordinate system where the overhead view point cloud is located.

[0034] In the above technical solution, the first measurement point extraction model, the second measurement point extraction model and the third measurement point extraction model are constructed separately.

[0035] In the above technical solution, the first measurement point extraction model, the second measurement point extraction model and the third measurement point extraction model are the same and are all measurement point extraction models. Each sample input to the measurement point extraction model is a top-view RGB image, a left-view RGB image and a right-view RGB image. The measurement point extraction model predicts the spine point through the top-view RGB image, predicts the body width measurement point and the left abdominal circumference point of the pig through the left-view RGB image, and predicts the body width measurement point and the right abdominal circumference point of the pig through the right-view RGB image.

[0036] In the above technical solution, the measurement point extraction model adopts an improved ResNet50 network, and the improved ResNet50 network includes: 49 convolutional layers and 1 deconvolution layer connected in sequence.

[0037] A method for measuring pig weight based on RGB-D includes: obtaining the body length, body width, body height and abdominal circumference of the pig and substituting them into a multiple linear regression prediction model, and predicting the pig weight through the multiple linear regression prediction model.

[0038] In the above technical solution, the multiple linear regression prediction model is a relationship curve among the pig's body length, body width, body height, abdominal circumference and weight.

[0039] The present invention has the following advantages due to the adoption of the above technical solution:

[0040] 1. The pig body measurement method of the present invention uses a trained first measurement point extraction model to extract the body length measurement points in the top-view RGB image and determine whether it is an ideal posture frame, thereby avoiding the influence of pig body bending on the measurement results;

[0041] 2. The pig body size measurement method of the present invention extracts key points of body size measurement through the trained first measurement point extraction model to the third measurement point extraction model and projects them into the complete pig point cloud C1, thereby improving the accuracy and stability of the extracted body size measurement point positions, thereby improving the accuracy of the pig body size and weight measurement results. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a schematic diagram of collecting three-view RGB images and depth images of a pig according to the present invention;

[0043] Figure 2 Depth maps of three perspectives;

[0044] Figure 3 RGB images of three viewing angles;

[0045] Figure 4 The complete pig point cloud C1 containing key points of body measurement of the present invention;

[0046] Figure 5This is a slice of the final complete pig point cloud C2;

[0047] Figure 6 This is the improved ResNet50 network structure diagram of the present invention;

[0048] Figure 7 This is a schematic diagram of the deconvolution layer of the present invention. DETAILED DESCRIPTION

[0049] The following describes in detail a method for measuring pig body size and weight based on RGB-D according to the present invention in conjunction with the accompanying drawings.

[0050] Example 1

[0051] A method for measuring pig body size based on RGB-D, comprising:

[0052] S1, such as Figure 1 As shown, an area for placing pigs is used as the area to be collected, an RGB-D camera is set directly above the area to be collected, and an RGB-D camera is set on each of the left and right sides of the area to be collected (in this embodiment, the RGB-D camera uses a Kinect v2 camera. A computer is used as a host computer to connect the Kinect v2 camera and control the Kinect v2 camera to shoot. The distance between the RGB-D cameras on the left and right sides of the area to be collected is 2000 mm, and the distance between the RGB-D camera directly above the area to be collected and the ground is 1500 mm.), the lenses of the three RGB-D cameras are all aimed at the area to be collected, and an RFID wireless radio frequency device is set up on either side of the area to be collected; the three RGB-D cameras are calibrated to obtain the left-view rotation and translation matrix and the right-view rotation and translation matrix;

[0053] The calibration method includes: placing a calibration rod in the area to be collected, obtaining RGB images and depth images of the calibration rod from three perspectives through three RGB-D cameras; converting the RGB images and depth images of each perspective of the calibration rod into point clouds (Li Xin. Research on RGBD data processing methods for multi-target sorting tasks [D]. Changchun University of Technology, 2024. DOI: 10.27805 / d.cnki.gccgy.2024.000457). The RGB images and depth images of the three perspectives are used to obtain three sets of point clouds: left view point cloud, right view point cloud, and top view point cloud.

[0054] The coordinate system of the top-view point cloud of the calibration rod is used as the reference coordinate system, and the left view point cloud and the right view point cloud of the calibration rod are respectively transformed into the reference coordinate system (Cheng Xun, Zhou Lele, Qian Rong, et al. Three-dimensional pig model reconstruction technology based on point cloud map [J]. Journal of Anhui University of Science and Technology, 2024, 38(05): 58-64. DOI: 10.19608 / j.cnki.1673-8772.2024.0509.), and the left view rotation and translation matrix and the right view rotation and translation matrix are obtained. Among them, taking the transformation of the left view point cloud of the calibration rod into the reference coordinate system as an example, the left view rotation and translation matrix The calculation formula is as follows:

[0055]

[0056] Among them, (x c ,y c ,z c ) is the coordinate of any point in the left viewpoint cloud in the coordinate system of the left viewpoint cloud before transformation, (x w ,y w ,z w ) is the coordinate of the point in the left viewpoint cloud in the reference coordinate system after transformation, R = R x R y R z is the rotation matrix, R x 、R y 、R z They are the rotation matrices in the x-axis, y-axis, and z-axis directions between the coordinate system of the left viewpoint cloud and the reference coordinate system before transformation, respectively. x t y t z ] T is the translation matrix, (t x , t y , t z ) is the translation coordinate between the origin of the left viewpoint cloud coordinate system and the origin of the reference coordinate system before transformation;

[0057] S2, three RGB-D cameras are used to simultaneously collect RGB images and depth images of three perspectives of a single pig in the area to be collected and used as a sample, that is, each sample includes: the top view RGB image, left view RGB image, right view RGB image, top view depth map, left view depth map and right view depth map of a single pig collected at the same time. The depth maps of the three perspectives are as follows: Figure 2 As shown, the RGB images of the three viewing angles are as follows Figure 3 As shown;

[0058] Among them, when a single pig is guided into the area to be collected, the RFID wireless radio frequency device detects the pig's ear tag information, and sends a trigger signal and the pig's identity information to the host computer. The host computer controls each RGB-D camera to collect the RGB image and depth map of the single pig;

[0059] S3, including S3-1 to S3-3:

[0060] S3-1, inputting the top-view RGB image of the sample into the trained first measurement point extraction model, the first measurement point extraction model being used to predict n spinal points of the pig in the top-view RGB image to obtain n spinal feature points corresponding to the pig in the sample, wherein the mean square error between the predicted value output by the first measurement point extraction model and the true value is within 6 pixels, the n spinal points are n points evenly spaced and arranged in sequence on the pig's back midline, n ≥ 3, the starting point of the pig's back midline is the center position of the line connecting the midpoints of the pig's two ear roots, and the end point of the pig's back midline is the point at the first natural whorl at the base of the pig's tail;

[0061] The first spinal feature point among the n spinal feature points close to the pig's head is used as the first body length measurement point of the pig, the last spinal feature point among the n spinal feature points is used as the second body length measurement point of the pig, the first body length measurement point and the second body length measurement point are used as body length measurement points respectively, and the second spinal feature point among the n spinal feature points close to the pig's head is used as the body height measurement point of the pig.

[0062] In this embodiment, n=5, and the five spine feature points are: P bl1 、P bl2 、P bl3 、P bl4 and P bl5 , the positions of the five spine feature points in the top-view RGB image of the sample are as follows Figure 3 As shown in (c), the spine feature point P bl1 is the first body length measurement point, spine feature point P bl5 is the second body length measurement point, spine feature point P bl2 This is the point where the pig's height is measured.

[0063] S3-2, check whether the relative offset angle between the n spine feature points in the top-view RGB image is within the range of [θ1, θ2]. If the relative offset angle is not within the range of [θ1, θ2], the pig's spine is bent in the top-view RGB image. Delete the sample and re-execute S2 to obtain a sample. If the relative offset angle is within the range of [θ1, θ2], use the top-view RGB image of the pig as the ideal posture frame and execute S3-3.

[0064] In this embodiment, θ1 = -10°, θ2 = 10°.

[0065] S3-3: Input the sample's left-view RGB image into the trained second measurement point extraction model, and predict the pig's left body width measurement point and p left abdominal circumference points to obtain the pig's left body width measurement point prediction points and p left abdominal circumference prediction points. Input the sample's right-view RGB image into the trained third measurement point extraction model, and predict the pig's right body width measurement point and q right abdominal circumference points to obtain the pig's right body width measurement point prediction points and q right abdominal circumference prediction points. The left body width measurement point is the widest point at the posterior edge of the pig's left shoulder blade, and the right body width measurement point is the widest point at the posterior edge of the pig's right shoulder blade. The abdominal circumference point is a point on the pig's abdominal circumference line, which is the longest line around the pig's abdomen.

[0066] In this embodiment, p=q=4, the right view RGB image of the sample is as follows Figure 3 As shown in (a), the left-view RGB image of the sample is as follows Figure 3 As shown in (b), Figure 3 (a) and Figure 3 (b) P ac1 ~P ac8 All are predicted points of abdominal circumference, P bw1 and P bw2 All are body width measurement and prediction points;

[0067] S4, the body length measurement point, body height measurement point, left body width measurement point prediction point, left abdominal circumference point prediction point, right body width measurement point prediction point and right abdominal circumference point prediction point of the pig are composed of body measurement key points, and the RGB image and depth image of the three-view of the sample are converted into point clouds respectively (Li Xin. Research on RGBD data processing method for multi-target sorting tasks [D]. Changchun University of Technology, 2024. DOI: 10.27805 / d.cnki.gcc gy.2024.000457), the RGB images and depth images of the three perspectives are used to obtain the left view point cloud, right view point cloud and top view point cloud of the sample; the left view point cloud, right view point cloud and top view point cloud of the sample are registered by the left view rotation and translation matrix and the right view rotation and translation matrix obtained in step S1 to obtain the complete pig point cloud C1 corresponding to the sample; the body size measurement key points are projected onto the complete pig point cloud C1 to obtain the complete pig point cloud C1 containing the body size measurement key points, as shown in Figure 4 shown.

[0068] Perform point cloud registration on the left view point cloud, right view point cloud, and top view point cloud of the sample: multiply the left view point cloud with the left view rotation and translation matrix to transform the left view point cloud to the coordinate system of the top view point cloud; multiply the right view point cloud with the right view rotation and translation matrix to transform the right view point cloud to the coordinate system of the top view point cloud. At this time, the complete pig point cloud C1 is obtained in the coordinate system of the top view point cloud.

[0069] S5, through the straight-through filtering, statistical filtering, random sampling consistency algorithm segmentation to remove the background, ground and outliers in the complete pig point cloud C1 containing the key points of body measurement, including:

[0070] S5-1, use a straight-through filter to filter the complete pig point cloud C1 containing the key points of body measurement, remove the background and retain only the point cloud of the pig area in the area to be collected, and obtain the complete pig point cloud C1 after straight-through filtering.

[0071] The filtering parameters of the pass filter are calculated based on the distance between the two RGB-D cameras on both sides of the area to be collected and the width of the area to be collected.

[0072] S5-2, use the statistical filter (Wang Wanqi. Research on point cloud registration algorithm based on geometric features [D]. Changchun University of Science and Technology, 2023. DOI: 10.26977 / d.cnki.gccgc.2023.000763.) to filter the complete pig point cloud C1 after straight-through filtering to obtain the complete pig point cloud C1 after statistical filtering;

[0073] S5-3, using the random sampling consistency algorithm (Wu Benzhao. Research on online point cloud processing system for high-speed roll forming [D]. University of Electronic Science and Technology of China, 2024. DOI: 10.27005 / d.cnki.gdzku.2024.002626.) to remove the ground in the complete pig point cloud C1 after statistical filtering, and obtain the final complete pig point cloud C2.

[0074] S6, use the first body length measurement point and the second body length measurement point in the final complete pig point cloud C2 to make vertical slices, and calculate the pig's body length by fitting the quadratic B-spline curve (Wang Fang, Bai Genzhu. Quadratic B-spline curve and its application [J]. Journal of Hubei University for Nationalities (Natural Science Edition), 2020, 38(02): 209-213. DOI: 10.13501 / j.cnki.42-1908 / n.2020.06.019.); calculate the pig's body height by using the straight-line distance between the body height measurement point and the ground; and take the distance between the predicted point of the body width measurement point on the left and the predicted point of the body width measurement point on the right as the pig's body width;

[0075] The abdominal circumference point cloud curve is obtained by vertically slicing the abdominal circumference prediction points on the left and the right. The abdominal circumference point cloud curve is then elliptical fitted using the least squares method to calculate the abdominal circumference of the pig. The final slice of the complete pig point cloud C2 is obtained, as shown in the figure below: Figure 5 As shown; the calculated body length, body width, body height and abdominal circumference of the pig constitute the body size data;

[0076] Example 2

[0077] A pig weight measurement method based on RGB-D, comprising:

[0078] Construct a relationship curve of pig body length, body width, body height, abdominal circumference and weight as a multiple linear regression prediction model (Wang Fangke. Analysis of sow culling rules in large-scale pig farms and research on factors affecting lifelong reproductive performance [D]. Huazhong Agricultural University, 2023. DOI: 10.27158 / d.cnki.ghznu.2023.000624.), input the body size data obtained in Example 1 into the multiple linear regression prediction model to predict the weight of the pigs and obtain the weight data of the pigs;

[0079] Among them, the expression of the multiple linear regression prediction model is:

[0080] y=β0+β1x1+β2x2+β3x3+β4x4+ε

[0081] In the formula, the dependent variable y is the weight data of the pigs, the independent variable x1 is the body length, the independent variable x2 is the body width, the independent variable x3 is the body height, the independent variable x4 is the abdominal circumference, β0 is the constant term, β1, β2.....β4 are all partial regression coefficients, and ε is the random error.

[0082] Example 3

[0083] On the basis of Example 2 or Example 1, the first measurement point extraction model, the second measurement point extraction model, and the third measurement point extraction model can not only be constructed and predicted separately, but also set up the same measurement point extraction model. Each sample input to the measurement point extraction model is a top view RGB image, a left view RGB image, and a right view RGB image. The measurement point extraction model predicts the spine point through the top view RGB image, predicts the body width measurement point and the left abdominal circumference point of the pig through the left view RGB image, and predicts the body width measurement point and the right abdominal circumference point of the pig through the right view RGB image. For example, the measurement point extraction model uses an improved ResNet50 network, such as Figure 6 As shown in the figure, the improved ResNet50 network includes: 49 convolutional layers and 1 deconvolution layer connected in sequence. The convolution kernel of the deconvolution layer is 3 and the stride is 2. The deconvolution layer is as follows: Figure 7 As shown;

[0084] For the ResNet50 network structure, see: Wu Yuqiang, Sun Xun, Ji Chengming, et al. Identification of edible wild vegetables based on deep learning [J / OL]. Chinese Vegetables, 1-15 [2024-10-24]. https: / / doi.org / 10.16861 / j.cnki.zggc.2024.0325.

[0085] Method to obtain the trained measurement point extraction model:

[0086] S1: Continuously collect multiple samples in the area to be collected and form a data set D2; each sample is a top-view RGB image, a left-view RGB image, and a right-view RGB image; manually annotate all samples in data set D2 with key body measurement points; divide the annotated samples in data set D2 into a training data set and a test data set in an 8:2 ratio; the manually annotated key body measurement points include: spine point, left body width measurement point, left abdominal circumference point, right body width measurement point, and right abdominal circumference point.

[0087] S2, inputting the training data set into the measurement point extraction model for training until the number of training rounds reaches a maximum of 200 rounds, thereby obtaining a trained measurement point extraction model;

[0088] S3, testing the trained measurement point extraction model: Input the test data set into the trained measurement point extraction model for testing. The coordinates of the body measurement key points are output and annotated in the sample. The coordinates of the output body measurement key points are compared with the manually annotated body measurement key points. The mean square error between the coordinates of the manually annotated actual body measurement points and the coordinates of the body measurement key points output by the trained measurement point extraction model is calculated. If the mean square error is less than 6 pixels, it can be seen that the error of the trained measurement point extraction model is small.

[0089] Example 4

[0090] A method for measuring the body size of live pigs based on RGB-D is proposed. Based on Example 3, in order to improve the measurement efficiency of the body size data of pigs, samples of 103 pigs are continuously collected to form a data set D1, and the body size data of all pigs in the data set D1 are measured.

[0091] The average absolute error of the pig body size data obtained by the pig body size measurement method of Example 4 is in the range of 0.68 cm to 2.26 cm.

[0092] Example 5

[0093] A method for measuring pig body size based on RGB-D was used to measure the body size of 103 pigs using the method described in patent CN115546282A to obtain body size data. The average absolute error of the body size data obtained in Example 5 was within the range of 1.15 cm to 5.21 cm.

[0094] Comparing Example 5 with Example 4, Example 4 adds a pig posture recognition function, performs subsequent measurement functions for standard postures, eliminates the negative impact of other postures on measurement, and improves the body size measurement method. By detecting the body size measurement key points of the color image and projecting them onto the point cloud, it is more stable and accurate than using the geometric features of the point cloud to find the measurement points.

[0095] The above is an exemplary description of the present invention. It should be noted that, without departing from the core of the present invention, any simple deformation, modification or other equivalent replacement that can be made by other skilled in the art without expending creative labor falls within the scope of protection of the present invention.

Claims

1. A method for measuring pig body size based on RGB-D, characterized in that: include: S1: Set an area as the area to be collected, set up an RGB-D camera directly above the area to be collected, and set up an RGB-D camera on the left and right sides of the area to be collected, with the lenses of all three RGB-D cameras aimed at the area to be collected; calibrate the three RGB-D cameras to obtain the left-view rotation and translation matrix and the right-view rotation and translation matrix; S2, using three RGB-D cameras to simultaneously collect RGB images and depth images of three perspectives of a single pig in the area to be collected and use them as a sample. Each sample includes: a top-view RGB image, a left-view RGB image, a right-view RGB image, a top-view depth map, a left-view depth map, and a right-view depth map of a single pig collected simultaneously; S3, including S3-1 to S3-3: S3-1, inputting the top-view RGB image of the sample into the trained first measurement point extraction model, the first measurement point extraction model being used to predict n spinal points of the pig in the top-view RGB image to obtain n spinal feature points corresponding to the pig in the sample, wherein the mean square error between the predicted value output by the first measurement point extraction model and the true value is within 6 pixels, and the n spinal points are n points evenly distributed on the midline of the pig's back; The first spinal feature point among the n spinal feature points that is close to the pig's head is used as the first body length measurement point of the pig, the last spinal feature point among the n spinal feature points is used as the second body length measurement point of the pig, the first body length measurement point and the second body length measurement point are used as body length measurement points respectively, and the second spinal feature point among the n spinal feature points that is close to the pig's head is used as the body height measurement point of the pig; S3-2, check whether the relative offset angle between the n spine feature points in the top-view RGB image is within the range of [θ1, θ2]. If the relative offset angle is not within the range of [θ1, θ2], delete the sample and re-execute S2 to obtain the sample; if the relative offset angle is within the range of [θ1, θ2], execute S3-3. S3-3, inputting the left-view RGB image of the sample into the trained second measurement point extraction model, predicting the body width measurement points and the left abdominal circumference points on the left side of the pig, and obtaining the predicted points of the body width measurement points and the left abdominal circumference points on the left side of the pig; inputting the right-view RGB image of the sample into the trained third measurement point extraction model, predicting the body width measurement points and the right abdominal circumference points on the right side of the pig, and obtaining the predicted points of the body width measurement points and the right abdominal circumference points on the right side of the pig; The left body width measurement point is the widest point at the rear edge of the pig's left shoulder blade, and the right body width measurement point is the widest point at the rear edge of the pig's right shoulder blade; the abdominal girth point is the point on the pig's abdominal girth line, which is the longest line around the pig's abdomen; S4, converting the RGB images and depth images of the three perspectives of the sample into point clouds respectively, and obtaining the left view point cloud, right view point cloud and top view point cloud of the sample from the RGB images and depth images of the three perspectives; performing point cloud registration on the left view point cloud, right view point cloud and top view point cloud of the sample using the left view rotation and translation matrix and the right view rotation and translation matrix obtained in step S1, and obtaining the complete pig point cloud C1 corresponding to the sample; The body length measurement point, body height measurement point, left body width measurement point prediction point, left abdominal circumference point prediction point, right body width measurement point prediction point and right abdominal circumference point prediction point of the pig are used to form the body size measurement key points; the body size measurement key points are projected onto the complete pig point cloud C1 to obtain the complete pig point cloud C1 containing the body size measurement key points; S5, using straight-through filtering, statistical filtering, and random sampling consistency algorithm to segment and remove the background, ground, and outliers from the complete pig point cloud C1 containing the key points of body measurement, and obtain the final complete pig point cloud C2; S6, use the first body length measurement point and the second body length measurement point in the final complete pig point cloud C2 to make a vertical slice, and calculate the pig's body length by fitting the quadratic B-spline curve; use the straight-line distance between the body height measurement point and the ground to calculate the pig's body height; use the distance between the body width measurement point prediction point on the left and the body width measurement point prediction point on the right as the pig's body width; use the abdominal circumference point prediction point on the left and the abdominal circumference point prediction point on the right to make a vertical slice to obtain the abdominal circumference point cloud curve, and use the least squares method to perform ellipse fitting of the abdominal circumference point cloud curve to obtain the pig's abdominal circumference; make the pig's body length, body width, body height and abdominal circumference constitute the body size data.

2. The method for measuring pig body size based on RGB-D according to claim 1, characterized in that: In S1, the calibration method includes: placing a calibration rod in the area to be collected, obtaining RGB images and depth images of the calibration rod from three perspectives through three RGB-D cameras; converting the RGB image and depth image of each perspective of the calibration rod into a point cloud, and obtaining three sets of point clouds from the RGB images and depth images of the three perspectives: a left-view point cloud, a right-view point cloud, and a top-view point cloud; The coordinate system of the top-view point cloud of the calibration rod is used as the reference coordinate system, and the left view point cloud and the right view point cloud of the calibration rod are respectively transformed into the reference coordinate system to obtain the left view rotation and translation matrix and the right view rotation and translation matrix.

3. The method for measuring pig body size based on RGB-D according to claim 1, characterized in that: In said S3, n≥3.

4. The method for measuring pig body size based on RGB-D according to claim 1, characterized in that: In S3, the starting point of the pig's back midline is the center position of the line connecting the midpoints of the pig's two ears, and the end point of the pig's back midline is the point at the first natural whorl at the pig's tail root.

5. The method for measuring pig body size based on RGB-D according to claim 1, characterized in that: In S3, the left view point cloud, right view point cloud and overhead view point cloud of the sample are aligned: the left view point cloud is multiplied by the left view rotation and translation matrix, so that the left view point cloud is converted to the coordinate system where the overhead view point cloud is located; the right view point cloud is multiplied by the right view rotation and translation matrix, so that the right view point cloud is converted to the coordinate system where the overhead view point cloud is located. At this time, the complete pig point cloud C1 is obtained in the coordinate system where the overhead view point cloud is located.

6. The method for measuring pig body size based on RGB-D according to claim 1, characterized in that: The first measurement point extraction model, the second measurement point extraction model and the third measurement point extraction model are constructed respectively.

7. The method for measuring pig body size based on RGB-D according to claim 1, characterized in that: The first measurement point extraction model, the second measurement point extraction model and the third measurement point extraction model are the same and are all measurement point extraction models. Each sample input to the measurement point extraction model is a top-view RGB image, a left-view RGB image, and a right-view RGB image. The measurement point extraction model predicts the spine point through the top-view RGB image, predicts the body width measurement point and the left abdominal circumference point on the left side of the pig through the left-view RGB image, and predicts the body width measurement point and the right abdominal circumference point on the right side of the pig through the right-view RGB image.

8. The method for measuring pig body size based on RGB-D according to claim 7, characterized in that: The measurement point extraction model adopts an improved ResNet50 network, which includes 49 convolutional layers and 1 deconvolution layer connected in sequence.

9. A pig weight measurement method based on RGB-D, characterized in that: include: The body length, body width, body height and abdominal circumference of the pigs were obtained and substituted into the multiple linear regression prediction model, and the weight of the pigs was predicted by the multiple linear regression prediction model.

10. The method for measuring pig weight based on RGB-D according to claim 9, characterized in that: The multiple linear regression prediction model is a relationship curve among the pig's body length, body width, body height, abdominal circumference and weight.

Images