Sheep body size measurement method, device, medium and program product based on three-dimensional reconstruction and point cloud segmentation

Abstract

The present application provides a kind of based on three-dimensional reconstruction and point cloud segmentation sheep body size measurement method, equipment, medium and program product, it is related to computer vision technical field.The method comprises: collecting the sheep image of the sheep to be measured, and the sheep image is preprocessed, obtains the multi-view sheep target point cloud;Multi-view sheep target point cloud is registered and handled, and the three-dimensional model of sheep is obtained;Based on body size measurement key point and the three-dimensional model of sheep, point cloud segmentation processing is carried out, and key point area point cloud is obtained;Based on key point area point cloud, determine the region point cloud projection space, and posture normalization is carried out to interval point cloud, determine the posture normalized key point area point cloud;Based on posture normalized key point area point cloud, calculate the body size data of the sheep to be measured.The present application embodiment realizes the measurement of the body size data of sheep by three-dimensional reconstruction of the surface point cloud of sheep, improves the measurement efficiency and accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a sheep body size measurement method, device, medium and program product based on three-dimensional reconstruction and point cloud segmentation. BACKGROUND

[0002] In the field of animal husbandry, it is generally believed that the size of livestock body size and growth rate can reflect the excellent breed, so the accurate collection of body size measurement data at different stages plays a crucial role in evaluating the growth state, breeding value and mutton yield of mutton sheep.

[0003] In the related art, traditional body size measurement often uses a tape measure, a circular measuring device and the like for manual measurement, and requires the measured sheep to stand on the ground with a standard posture. The traditional measurement method is time-consuming and laborious, and has low efficiency, and is likely to have adverse effects on livestock.

[0004] Therefore, in the related art, relevant scholars have proposed a new body size measurement method, such as applying acoustic devices (ultrasonic waves), X-rays and the like to body size measurement, and promoting the conversion of livestock body size measurement from contact measurement to non-contact measurement. However, although ultrasonic waves have good directivity and high accuracy in distance measurement, ultrasonic imaging still requires direct human involvement; X-rays have good penetration of matter, but have the disadvantages of damaging cells and requiring a high test environment, which cannot meet the actual application scenarios.

[0005] Therefore, how to measure the body size of sheep is a technical problem to be solved at present. SUMMARY

[0006] The present application provides a sheep body size measurement method, device, medium and program product based on three-dimensional reconstruction and point cloud segmentation, to solve the defects of low efficiency of manual measurement and damage to sheep in the prior art, by reconstructing the surface point cloud of the sheep, the body size data of the sheep in a natural state is measured without contact, the measurement efficiency and accuracy are improved, and the generalization and robustness of the body size data of sheep of different breeds and different postures are improved.

[0007] In a first aspect, the present application provides a sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation, comprising the following steps:

[0008] Collecting a sheep image of a measured sheep, and pre-processing the sheep image to obtain a multi-view sheep target point cloud;

[0009] Performing point cloud registration processing on the multi-view sheep target point cloud to obtain a three-dimensional model of the sheep;

[0010] The point cloud segmentation processing is performed based on the body size measurement key points and the three-dimensional model of the sheep, to obtain key point region point clouds corresponding to the body size measurement key points; wherein the body size measurement key points at least include at least one of body height, chest width, hip height, hip width, body diagonal length, and chest circumference;

[0011] Based on the key point region point clouds, a region point cloud projection space is determined, and the interval point cloud is posture normalized according to the region point cloud projection space, to determine posture-normalized key point region point clouds; the key point region point clouds at least include interval point clouds.

[0012] Based on the posture-normalized key point region point clouds, the body size data of the sheep to be measured is calculated; wherein the body size data of the sheep to be measured includes at least one of body height data, chest width data, hip height data, hip width data, body diagonal length data, and chest circumference data.

[0013] In a second aspect, the present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method for measuring the body size of the sheep based on the three-dimensional reconstruction and the point cloud segmentation as described above when executing the program.

[0014] In a third aspect, the present application further provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method for measuring the body size of the sheep based on the three-dimensional reconstruction and the point cloud segmentation as described above.

[0015] In a fourth aspect, the present application further provides a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the method for measuring the body size of the sheep based on the three-dimensional reconstruction and the point cloud segmentation as described above.

[0016] The application provides a sheep body size measurement method, device, medium and program product based on three-dimensional reconstruction and point cloud segmentation. The method comprises the following steps: collecting a sheep image of a to-be-measured sheep, and pre-processing the sheep image to obtain a multi-view sheep target point cloud; performing point cloud registration processing on the multi-view sheep target point cloud to obtain a three-dimensional sheep model; performing point cloud segmentation processing based on a body size measurement key point and the three-dimensional sheep model to obtain a key point region point cloud corresponding to the body size measurement key point; wherein the body size measurement key point at least comprises at least one of body height, chest width, hip height, hip width, body diagonal length and chest circumference; determining a region point cloud projection space based on the key point region point cloud, and performing posture normalization on the region point cloud according to the region point cloud projection space to determine a posture-normalized key point region point cloud; the key point region point cloud at least comprises a region point cloud; and calculating body size data of the to-be-measured sheep based on the posture-normalized key point region point cloud; wherein the body size data of the to-be-measured sheep at least comprises at least one of body height data, chest width data, hip height data, hip width data, body diagonal length data and chest circumference data. The method is used to solve the defects of low efficiency of artificial measurement and damage to sheep caused by X-ray measurement in the prior art, realizes three-dimensional reconstruction of the body surface point cloud of the sheep, contactless measurement of the body size data of the sheep in a natural state, improves the measurement efficiency and accuracy, and improves the generalization and robustness of the body size measurement of sheep of different breeds and different postures. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative effort.

[0018] Figure 1 is one of the flowcharts of the sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation provided by the application.

[0019] Figure 2 is the second schematic diagram of the sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation provided by the application.

[0020] Figure 3 is a schematic diagram of collecting sheep images on a farm provided by the application.

[0021] Figure 4 is a schematic diagram of collecting sheep images of a to-be-measured sheep provided by the application.

[0022] Figure 5 is a schematic diagram of three-view point cloud images of a to-be-measured sheep provided by the application.

[0023] Figure 6 is a schematic view of the extraction of a side view target point cloud provided by the present application.

[0024] Figure 7 is a schematic view of a top view target point cloud and a ground point cloud provided by the present application.

[0025] Figure 8 is a schematic view of a three-dimensional model of a sheep provided by the present application.

[0026] Figure 9 is a schematic view of a top view target point cloud segmentation provided by the present application.

[0027] Figure 10 is a schematic view of determining a three-dimensional model of a sheep provided by the present application.

[0028] Figure 11 is a schematic view of interval division and segmentation prediction model provided by the present application.

[0029] Figure 12 is a schematic view of B segmentation model abnormal identification result provided by the present application.

[0030] Figure 13 is a schematic view of the maximum point of the head of a sheep provided by the present application.

[0031] Figure 14 is a schematic view of the improved B label provided by the present application.

[0032] Figure 15 is a schematic view of the prediction result schematic view of the improved B standard A segmentation model and B segmentation model provided by the present application.

[0033] Figure 16 is a schematic view of calculating the body size data of the sheep to be measured provided by the present application.

[0034] Figure 17 is a schematic view of calculating the body height type body size parameter provided by the present application.

[0035] Figure 18 is a schematic view of establishing a first projection coordinate system provided by the present application.

[0036] Figure 19 is a schematic view of calculating the chest width data provided by the present application.

[0037] Figure 20 is a schematic view of establishing a hip region projection coordinate system provided by the present application.

[0038] Figure 21 is a schematic view of the scapula front edge measurement point interval segmentation provided by the present application.

[0039] Figure 22 is a schematic diagram of the process of locating the anterior glenoid rim measurement point provided by the present application.

[0040] Figure 23 is a schematic diagram of the process of locating the posterior greater trochanter measurement point provided by the present application.

[0041] Figure 24 is a schematic diagram of the body length measurement point provided by the present application.

[0042] Figure 25 is a schematic diagram of the process of segmenting the bust region provided by the present application.

[0043] Figure 26 is a schematic diagram of calculating the bust data provided by the present application.

[0044] Figure 27 is a schematic diagram of the bust point cloud and fitting result provided by the present application.

[0045] Figure 28 is a schematic diagram of the structure of an electronic device provided by the present application. DETAILED DESCRIPTION

[0046] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0047] The present application will be described below in conjunction with Figures 1-28 A sheep body size measurement method, device, medium and program product based on three-dimensional reconstruction and point cloud segmentation are described, which are used to solve the defects of low efficiency of artificial measurement and harm to sheep in the prior art, realize three-dimensional reconstruction of the body surface point cloud of the sheep, non-contact measurement of the body size data of the sheep in a natural state, improve the measurement efficiency and accuracy, and improve the generalization and robustness of the measurement of the body size of sheep of different breeds and different postures.

[0048] In conjunction with Figure 1 and Figure 2 It is shown that the sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation provided by the present application can include but is not limited to steps S100 to S500:

[0049] S100, acquiring a sheep image of a to-be-measured sheep, and pre-processing the sheep image to obtain a multi-view sheep target point cloud;

[0050] S200, performing point cloud registration processing on the multi-view sheep target point cloud to obtain a three-dimensional model of the sheep;

[0051] S300, performing point cloud segmentation processing based on the body size measurement key points and the three-dimensional model of the sheep to obtain key point region point clouds corresponding to the body size measurement key points; wherein the body size measurement key points at least include at least one of body height, chest width, hip height, hip width, body diagonal length, and chest circumference;

[0052] S400, determining a region point cloud projection space based on the key point region point clouds, and performing posture normalization on the interval point cloud according to the region point cloud projection space to determine posture-normalized key point region point clouds; the key point region point clouds at least include interval point clouds;

[0053] S500, calculating body size data of the to-be-measured sheep based on the posture-normalized key point region point clouds; wherein the body size data of the to-be-measured sheep includes at least one of body height data, chest width data, hip height data, hip width data, body diagonal length data, and chest circumference data.

[0054] In step S100 of some embodiments, a sheep image of a to-be-measured sheep is collected, and the sheep image is preprocessed to obtain a multi-view sheep target point cloud.

[0055] It can be understood that in some embodiments of the present application, an image collection device is first configured, and the image collection device is used to collect a sheep image of a to-be-measured sheep.

[0056] As shown in Figure 3 , the image collection device is composed of a ground scale, three KinectV2 cameras, three notebook computers, and an arch-shaped support. The frame rate of the camera is 30 frames per second, the depth image resolution is 512╳424 pixels, and the accuracy of the ground scale is ±0.5 kg, located in the central corridor of the sheep pen exit. There is a passageway in the sheep pen that can help the sheep pass through the image collection area of the image collection device alone. The overhead KinectV2 camera is installed at the center of the weighing scale 2±0.1 meters from the ground, and the side-view KinectV2 is installed at 0.8±0.1 meters from the ground and 1.5±0.1 meters from the center of the weighing scale. Figure 3 To collect a sheep image in a real field of a farm, Figure 4 To collect a sheep image of a to-be-measured sheep.

[0057] In the image collection device of the embodiment of the present application, KinectV2 cameras are selected as specific image collection equipment.

[0058] Further, the sheep image at least includes: a left view, an overhead view, and a right view of the to-be-measured sheep;

[0059] The multi-view sheep target point cloud at least comprises: a left-view target point cloud, an overhead-view target point cloud and a right-view target point cloud.

[0060] The sheep image is preprocessed to obtain the multi-view sheep target point cloud, comprising:

[0061] A camera intrinsic parameter of a collection device for collecting the sheep image is determined, wherein the camera intrinsic parameter represents projection parameter information for mapping a depth image to a three-dimensional point cloud;

[0062] Based on the camera intrinsic parameter, two-dimensional pixel coordinates of a left view, an overhead view and a right view of a sheep to be tested are respectively converted into three-dimensional camera coordinates to obtain corresponding left-view point cloud images, overhead-view point cloud images and right-view point cloud images;

[0063] The left-view point cloud images and the right-view point cloud images are respectively subjected to straight-through filtering processing to obtain left-view point cloud images and right-view point cloud images with background noise removed, and foreground images of the left-view point cloud images and the right-view point cloud images are respectively extracted, and the foreground images of the left-view point cloud images and the right-view point cloud images extracted are respectively subjected to statistical filtering processing to remove outliers to obtain corresponding left-view target point clouds and right-view target point clouds; and

[0064] A point cloud height distribution graph of the overhead-view point cloud images is drawn, and a segmentation threshold of an overhead-view point cloud and a ground point cloud is determined according to the point cloud height distribution graph;

[0065] The filtered overhead-view point cloud images are traversed to find a maximum depth value point and a minimum depth value point, and a height difference between the maximum depth value point and the minimum depth value point is calculated;

[0066] Point clouds with a depth value less than one-fourth of the height difference are taken as ground point clouds, and point clouds with a depth value greater than or equal to one-fourth of the height difference are taken as the overhead-view target point clouds.

[0067] In the embodiment of the application, the two-dimensional depth image is converted into a three-dimensional point cloud according to the camera intrinsic parameter, and the pixel coordinates (u, v) are converted to the camera coordinates (x, y, z) by applying formulas (1-1)-(1-3).

[0068] (1-1)

[0069] (1-2)

[0070] (1-3)

[0071] where d is the depth value of the depth map, depth_scale is the scaling factor of the depth value (usually taken as 1000), (c x ,c y ,f x ,f y ) is the camera intrinsic parameter.

[0072] Based on the camera intrinsic parameter, the two-dimensional pixel coordinates of the left view, the top view and the right view of the sheep to be measured are respectively converted into three-dimensional camera coordinates to obtain the corresponding left view point cloud image, the top view point cloud image and the right view point cloud image. The converted point cloud is as shown in Figure 5

[0073] The left view point cloud image and the right view point cloud image are respectively subjected to straight-through filtering processing to obtain the left view point cloud image and the right view point cloud image with background noise removed, and the foreground images of the left view point cloud image and the right view point cloud image are respectively extracted, and the foreground images of the extracted left view point cloud image and the right view point cloud image are respectively subjected to statistical filtering processing to remove outliers to obtain the corresponding left view target point cloud and the right view target point cloud. The steps are as follows:

[0074] Firstly, the background noise of the left view point cloud image and the right view point cloud image is removed by using the straight-through filtering method; in order to avoid the interference of the ground information in three-dimensional reconstruction, the normal vector of each point in the point cloud is estimated, and then the angle between the normal vector of each point and the horizontal coordinate axis is calculated, and the ground background is removed according to the statistical data to set a threshold value. That is, the ground is removed by applying formula (1-4) to retain the target point cloud. The foreground images of the extracted left view point cloud image and the right view point cloud image are subjected to statistical filtering processing to remove outliers to obtain the corresponding left view target point cloud and the right view target point cloud. The left view target point cloud and the right view target point cloud are as shown in Figure 6

[0075] The formula (1-4) for removing the ground background is as follows:

[0076] <cos(threshold) (1-4)

[0077] Where normal is the normal vector of the point, axis_direction is the direction of the horizontal coordinate axis, threshold is the set threshold value, and dot() function is used to calculate the inner product, and abs() function is used to calculate the absolute value.

[0078] ​​In some embodiments, a point cloud height distribution map of the top-view point cloud image is drawn, and a segmentation threshold between the top-view point cloud and the ground point cloud is determined based on the point cloud height distribution map; the filtered top-view point cloud image is traversed to find the maximum depth point and the minimum depth point, and the height difference between the maximum depth point and the minimum depth point is calculated; the steps of taking point clouds with depth values ​​less than one-quarter of the height difference as ground point clouds and point clouds with depth values ​​greater than or equal to one-quarter of the height difference as the target point cloud of the top view are as follows:

[0079] It should be noted that the top-view point cloud image contains foreground information of the sheep and ground information. The ground information is used to calculate the body height and body size parameters. Therefore, it is necessary to extract the top-view image information of the sheep and the ground information from the top-view point cloud image respectively.

[0080] First, a pass-through filter is used to roughly remove the background. Then, the distribution of point cloud height is statistically analyzed, and a point cloud height distribution map of the top-view point cloud image is drawn based on the statistical results.

[0081] Based on statistical results, the bimodal method can be used to determine the segmentation threshold between the top-view point cloud and the ground point cloud. The bimodal method refers to the method where, if the statistical histogram exhibits a clear bimodal characteristic, the valley between the two peaks can be selected as the segmentation threshold between the foreground and background. Therefore, the boundary between the top-view point cloud and the ground point cloud is approximately at about 1 / 4 of their height difference.

[0082] Therefore, the top-view point cloud image after background filtering is traversed to find the maximum depth point max_z_point and the minimum depth point min_z_point, and the height difference between them is calculated (the calculation formula is shown in formula (1-5)). The depth value is then set. The point cloud is the target point cloud in top view. The point cloud is the ground point cloud. Finally, statistical filtering is performed on both the target point cloud and the ground point cloud to remove outliers. The top-down view of the target point cloud and the ground point cloud is shown below. Figure 7 As shown.

[0083] (1-5)

[0084] Where d is the height difference between the maximum depth point and the minimum depth point, max_z_point is the maximum depth point and min_z_point is the minimum depth point.

[0085] This embodiment mainly establishes a walking sheep image acquisition device to acquire depth image data of the sheep. The acquired depth images are converted into 3D point clouds, and the target point cloud of the sheep is extracted based on the original point cloud, laying the foundation for subsequent 3D reconstruction of the point cloud.

[0086] In step S200 of some embodiments, point cloud registration processing is performed on the point cloud of the sheep target from multiple perspectives to obtain a three-dimensional model of the sheep.

[0087] It is understandable that multi-view point clouds of sheep targets are point cloud data obtained from different perspectives within the same field of view, namely, left-view target point clouds, top-view target point clouds, and right-view target point clouds. To transfer point clouds from different perspectives to the same perspective and coordinate system, thereby obtaining a complete 3D point cloud of the livestock, multi-view point cloud registration technology is needed. This technology involves obtaining livestock point clouds from different cameras and then using point cloud registration algorithms and calibration object registration methods to perform stereo matching of the point clouds from different perspectives, ultimately obtaining a 3D reconstructed model of the livestock's body surface. Therefore, how to accurately register multi-view point clouds is currently a key focus in the research of 3D reconstruction of livestock body surface point clouds.

[0088] In some embodiments of the present invention, coarse registration processing can first be performed on the target point cloud in the left view, the target point cloud in the top view, and the target point cloud in the right view to obtain coarse registration results. Fine registration processing is then performed based on the coarse registration results to obtain a three-dimensional model of the sheep.

[0089] Specifically, coarse registration refers to registering point clouds when their relative poses are completely unknown, finding a rotation and translation transformation matrix that brings the two point clouds relatively close, transforming the point cloud data from different coordinate systems to a unified coordinate system, providing a good initial position for fine registration. Fine registration, based on coarse registration, minimizes the spatial differences between point clouds to obtain a more accurate rotation and translation transformation matrix (for the 3D sheep model).

[0090] To address this issue, this invention employs a coarse registration strategy based on Random Sample Consensus (RANSAC). Its core advantages are: (1) by iteratively sampling the minimum point set to estimate the candidate transformation model, the interference of outliers on parameter estimation is effectively reduced; (2) by dynamically optimizing the model parameters using the maximum interior point set criterion, high-confidence matching pairs can be adaptively selected, significantly improving the robustness of registration in scenarios with fewer overlapping parts; (3) by combining a fast feature matching pre-screening mechanism, the computational resource overhead of exhaustive search is avoided while ensuring algorithm efficiency, thus improving computational efficiency.

[0091] The RANSAC algorithm improves the accuracy and robustness of coarse registration by using multiple iterations of random sampling and inlier model validation. Its core steps are as follows: First, a minimum subset is randomly extracted from the initially matched feature point pairs to estimate the transformation model parameters, such as the rotation matrix and translation vector. Then, "inliers" that conform to the current model are selected based on a preset threshold, and their number is counted. Through multiple iterations, the model with the most inliers and the smallest error is retained. This process effectively avoids interference from outliers and anomalies in the initial matching, ensuring that the coarse registration result provides an accurate and reliable initial position for subsequent fine registration.

[0092] For fine registration, the ICP (Iterative Closest Point) algorithm is computationally simple and highly accurate, making it a mainstream algorithm in fine registration applications. Given an initial pose, its core logic iteratively improves the matching accuracy of two point clouds: First, based on the current pose, it dynamically searches for the nearest neighbor in the target point cloud for each point in the source point cloud, establishing a temporary correspondence; second, based on these corresponding point pairs, it calculates the optimal transformation parameters (i.e., rotation and translation matrices) that align the source point cloud to the target point cloud, essentially adjusting the point cloud pose to minimize the overall distance deviation between all corresponding points; finally, it applies the calculated transformation parameters to the source point cloud and updates its pose, entering the next iteration. As iterations proceed, the correspondence becomes increasingly accurate due to pose optimization, and the calculation of transformation parameters gradually converges until the termination condition is met, such as when the error no longer decreases significantly or the preset number of iterations is reached. In summary, the prerequisite for the ICP algorithm to achieve good registration results is that both point clouds must have good initial positions.

[0093] In this embodiment, the point clouds of the left view, top view, and right view are used to reconstruct the sheep from three perspectives using the RANSAC algorithm and the ICP algorithm, respectively, to obtain a schematic diagram of the sheep's three-dimensional model, as shown below. Figure 8 As shown.

[0094] This invention proposes an algorithm for reconstructing a 3D model of sheep based on point cloud registration. The specific execution steps are as follows:

[0095] The target point cloud in the top view is segmented to obtain a first top view point cloud and a second top view point cloud;

[0096] The first top view point cloud and the left view target point cloud are registered to obtain a first registered image, and the second top view point cloud and the right view target point cloud are registered to obtain a second registered image.

[0097] The first registered image and the second registered image are merged to obtain a merged registered image;

[0098] The merged and registered image and the ground point cloud are merged to obtain the three-dimensional model of the sheep.

[0099] Furthermore, traverse the top-view target point cloud of the sheep and find the point P with the maximum value in the Y direction. max_y and the minimum point P min_y With a direction perpendicular to the XY plane, parallel to the X-axis, and passing through P max_y P min_y The plane at the center point divides the top-view point cloud image into two parts: PointCloud_left and PointCloud_right. Therefore, the more stable PointCloud_right part is selected to find the line of symmetry in the Y-direction of the point cloud. That is, the PointCloud_right point cloud is evenly divided into three parts along the Y-axis, and the centers of each part (Pcenter1, Pcenter2, Pcenter3) are found. A fitting plane parallel to the YZ plane is fitted along the three points Pcenter1, Pcenter2, and Pcenter3 to divide the top view of the sheep into two parts: PointCloud_front (first top view point cloud) and PointCloud_back (second top view point cloud). A schematic diagram of the top view target point cloud segmentation is shown below. Figure 9 As shown.

[0100] After the top-view point cloud is divided into left and right parts (first top-view point cloud and second top-view point cloud), the first top-view point cloud and the left-view target point cloud are registered to obtain the first registered image, and the second top-view point cloud and the right-view target point cloud are registered to obtain the second registered image. The first and second registered images are then merged to obtain the merged registered image, completing the initial reconstruction of the sheep's 3D model. Finally, the merged registered image and the ground point cloud are merged to obtain the sheep's 3D model. The specific process is illustrated below. Figure 10 As shown.

[0101] In step S300 of some embodiments, point cloud segmentation is performed based on the body size measurement key points and the three-dimensional model of the sheep to obtain a key point region point cloud corresponding to the body size measurement key points; wherein, the body size measurement key points include at least one of body height, chest width, hip height, hip width, body slant length, and chest circumference.

[0102] Understandably, when conducting body size measurements, it is necessary to first locate the key points for body size measurement, and then calculate the distance or perimeter to obtain body size data.

[0103] In existing technologies, directly locating and measuring key points on the overall point cloud of a sheep involves a large amount of computation. Sheep's bodies are flexible and varied, and the measurement points shift around their theoretical positions as their posture changes. Directly locating key points on the overall point cloud often results in positioning errors, affecting measurement accuracy. Furthermore, directly locating key points on the overall point cloud also makes the key point localization algorithm complex and lacks robustness.

[0104] Based on this, the embodiments of the present invention first perform point cloud region segmentation according to the probability interval of the body size measurement key points in the overall point cloud (three-dimensional model of sheep) to obtain the key point region point cloud.

[0105] In this embodiment of the invention, the PointNet++ model is selected for point cloud region segmentation.

[0106] Specifically, the key points of the body size measurement and the three-dimensional model of the sheep are input into the trained segmentation prediction model to predict the point cloud of the key point region.

[0107] The trained segmentation prediction model includes at least a first segmentation prediction model and a second segmentation prediction model. The first segmentation prediction model is characterized as an interval recognition model for measuring the body width, body height, hip height, and hip width of the sheep to be tested. The second segmentation prediction model is characterized as an interval recognition model for measuring the chest circumference and body oblique length of the sheep to be tested. The first segmentation prediction model is trained using chest region point cloud samples containing body height and chest width measurement points and hip region point cloud samples containing hip height and hip width measurement points. The second segmentation prediction model is trained using chest region point cloud samples containing anterior scapular border measurement points and chest circumference point clouds, and hip region point cloud samples containing posterior ischial tuberosity measurement points. Both the first and second segmentation prediction models are trained based on the PointNet++ model.

[0108] Understandably, based on the distribution of body size key points in the overall point cloud of the sheep, and to avoid challenges to the robustness of the key point detection strategy caused by changes in key point intervals and positional features due to diverse sheep postures, the overall point cloud of the sheep (3D sheep model) is subdivided into: a chest region (Ac) containing body height and chest width measurements; a hip region (Ar) containing hip height and hip width measurements; a chest region (Bc) containing measurements of the anterior edge of the scapula and chest circumference; and a hip region (Br) containing measurements of the posterior edge of the ischial tuberosity. Then, based on the overlap of these regions and the needs of body size measurement, interval recognition models are constructed for measuring body width, body height, hip height, and hip width, denoted as the first segmentation prediction model (A segmentation model); and an interval recognition model is constructed for measuring chest circumference and body oblique length, denoted as the second segmentation prediction model (B segmentation model). Examples of interval division and segmentation prediction models are shown below. Figure 11 As shown.

[0109] The key points of the body measurement and the three-dimensional model of the sheep are input into the trained segmentation prediction model. The inference steps for predicting the point cloud of the key point region include sampling grouping, feature extraction, and feature fusion to obtain higher-dimensional features. Finally, the obtained high-dimensional features are used to predict the category of each point in the point cloud to obtain the key point region point cloud of the corresponding key point.

[0110] Key point regions include, but are not limited to, key point regions of the head and neck, chest, buttocks, back, and regions including the anterior edge of the scapula.

[0111] However, in the prediction results of the B-segmentation model (second segmentation prediction model), due to the very frequent movement of the sheep's head and neck, the chest area recognition results may show instances where the head and neck are identified, such as... Figure 12 As shown in part a, the degree to which the head and neck are identified varies among different sheep. Furthermore, there are instances where the identified chest region does not include the area measured at the anterior border of the scapula, such as... Figure 12 As shown in part b.

[0112] Because sheep move their heads and necks very frequently, the two situations mentioned above are difficult to avoid. To address this issue using the B-segmentation model, we consider identifying the largest point P on the sheep's head. max ,like Figure 13 As shown. Statistical analysis was used to determine the region where the anterior edge of the scapula was located and the maximum point P on the head. max The positional relationship between the point clouds in the chest region is used to segment the point cloud interval that includes the area of ​​the anterior edge of the scapula.

[0113] Further, this invention includes a point cloud region extraction step based on PointNet++:

[0114] like Figure 11 As described in the previous section, the overall point cloud of the sheep is subdivided into the chest region (Ac) containing body height and width measurements, the rump region (Ar) containing rump height and width measurements, the head region (Bh), the chest region (Bc) containing the anterior edge of the scapula and the chest circumference point cloud, and the rump region (Br) containing the posterior edge of the ischial tuberosity. The improved B annotation is shown in [reference needed]. Figure 14 We construct interval recognition models for measuring body height, chest width, hip height, and hip width, denoted as the A-segmentation model; and an interval recognition model for measuring chest circumference and body oblique length, denoted as the B-segmentation model.

[0115] CloudCompare software was used to annotate the 3D reconstructed point cloud.

[0116] And train the PointNet++ model to construct a sheep body point cloud segmentation model. For example... Figure 15As shown, this diagram illustrates the prediction results of the A segmentation model and the B segmentation model after the improved B-labeling. Based on the prediction results of the B segmentation model, the prediction results are further improved according to the maximum head point P. max Point cloud data of the chest region was used to obtain the positional relationship of the area containing the measuring points at the anterior border of the scapula through statistical analysis. Testing with the experimental data showed that the area containing the measuring points at the anterior border of the scapula could be accurately segmented.

[0117] PointNet++'s innovation lies in its hierarchical feature aggregation strategy. After extracting local features from each domain, the model propagates these features across the entire point cloud through upsampling. During this process, high-level features are concatenated with low-level features to form a high-dimensional feature representation. This hierarchical feature aggregation not only enhances the model's ability to capture local details but also enables it to understand point cloud data from a global perspective. In the segmentation stage, PointNet++ uses the learned high-dimensional features to classify each point. Through a simple multilayer perceptron (MLP) layer, the model outputs a class label for each point. Because PointNet++ considers both the local structure and global relationships of the point cloud during feature extraction, it exhibits high accuracy and robustness in segmentation tasks.

[0118] In step S400 of some embodiments, the interval point cloud includes at least the chest region point cloud;

[0119] The steps of determining the projection space of the region point cloud based on the key point region point cloud, and performing pose normalization on the interval point cloud according to the region point cloud projection space to determine the pose-normalized key point region point cloud include:

[0120] Obtain the chest region point cloud from the point cloud of the key point region;

[0121] Traverse the point cloud of the chest region, find the point with the maximum and minimum distance from the ground plane, and extract the point cloud located in the interval between one-third and two-thirds of the distance between the maximum and minimum points as the point cloud to be projected.

[0122] Starting from the default coordinate system XZ plane, rotate around the Z-axis and project the point cloud to be projected onto the rotating plane to obtain the projection center. Calculate the sum of distances from all points of the point cloud to be projected to the projection center. When the sum of distances is the minimum, take the projection surface corresponding to the point cloud to be projected as the target projection surface.

[0123] Using the maximum depth point of the chest region point cloud as the origin, the normal vector of the target projection plane passing through the origin as the Y-axis, the normal vector of the Y-axis passing through the origin and parallel to the ground as the X-axis, and the straight line perpendicular to the XY plane and passing through the origin as the Z-axis, the chest region point cloud is normalized in attitude, and the key point region point cloud of attitude normalization is determined based on the attitude-normalized chest region point cloud.

[0124] In step S500 of some embodiments, the body size data of the sheep to be tested is calculated based on the point cloud of the key point region with pose normalization; wherein, the body size data of the sheep to be tested includes at least one of body height data, chest width data, hip height data, hip width data, body length data, and chest circumference data.

[0125] Understandably, body measurements include height, chest width, hip height, hip width, body length, and chest circumference. Height: The vertical distance from the highest point of the scapula to the ground. Chest width: The width between the left and right ribs at the posterior end of the scapula. Hip height: The vertical distance from the highest point of the sacrum to the ground. Hip width: The length at the widest point of the outer edge of the hip. Body length: The straight-line distance from the anterior end of the scapula to the posterior end of the ischial tuberosity. Chest circumference: The length around the chest at the posterior end of the scapula.

[0126] A schematic diagram illustrating the calculation of the body size data of the sheep under test based on the key point cloud of pose normalization is shown below. Figure 16 As shown. Based on the point cloud of the key point region of the sheep identified by the segmentation prediction model (PointNet++ network model) in step S300, the measurement points are extracted according to the distribution characteristics of the measurement points, and the body size data of the sheep to be measured are calculated.

[0127] It should be noted that the first step is to perform ground plane fitting. Specifically, in body size measurement, the model needs to be attitude normalized, where the ground serves as the reference surface for attitude normalization and also as the reference surface for body height parameters. Therefore, a ground plane equation needs to be obtained. The ground plane equation is obtained by fitting the point cloud coordinates using the Random Sample Consensus (RANSAC) algorithm, based on the point cloud coordinate normalization.

[0128] In some embodiments of the present invention, the step of calculating the body height data of the sheep to be tested includes:

[0129] Obtain the chest region point cloud from the point cloud of the key point region;

[0130] The highest point of the chest is found from the chest region point cloud based on the ground plane equation; wherein, the ground plane equation is represented by fitting the ground point cloud using a random consensus algorithm;

[0131] In the point cloud of the chest region, a neighborhood of the chest region is constructed with the highest point as the center and a preset length as the radius;

[0132] Calculate the distance from each point in the neighborhood of the chest region to the ground plane to obtain multiple chest region heights;

[0133] The maximum and minimum chest region heights are removed from multiple chest region heights, and the average value of the remaining chest region heights is calculated. The average value of the remaining chest region heights is used as the body height data of the sheep to be tested.

[0134] It should be noted that, referring to Figure 17 As shown, based on the recognition results labeled by PointNet++ model A, the point clouds of the chest region and the rump region are read separately, and statistical filtering is performed to remove outliers. Then, based on the obtained ground plane equation, the points in the chest and rump region point clouds that are highest from the ground plane are found respectively. A neighborhood with a radius of 1cm is constructed with this point as the center. The distances from points within the neighborhood to the ground plane are calculated, the maximum and minimum values ​​are removed, and the average distance is taken as the sheep's body height and rump height. The preset length can be 1cm.

[0135] The steps for calculating the hip height data of the sheep to be tested include:

[0136] Obtain the hip region point cloud from the point cloud of the key point region;

[0137] The highest point of the hip is found in the hip region point cloud based on the ground plane equation; wherein, the ground plane equation is represented by fitting the ground point cloud using a random consistency algorithm;

[0138] In the point cloud of the buttock region, a neighborhood of the buttock region is constructed with the highest point as the center and a preset length as the radius;

[0139] Calculate the distance from each point in the neighborhood of the hip region to the ground plane to obtain multiple hip region heights;

[0140] The maximum and minimum hip heights are removed from multiple hip heights, and the average value of the remaining hip heights is calculated. The average value of the remaining hip heights is used as the hip height data of the sheep to be tested.

[0141] It should be noted that the steps for automatically measuring chest width and other body size parameters are as follows:

[0142] Since the experimental data was obtained from sheep in a free state, the diverse postures of the sheep necessitate determining their projected coordinate system based on their spatial position when performing body size measurements. Taking chest width measurement as an example, the point cloud of the chest region is read based on the recognition results labeled by the A-segmentation model. The point cloud of the chest region is then traversed to find the points with the maximum and minimum distances from the ground plane. , Point cloud extraction is performed using formulas (2-1)-(2-5). .

[0143] ) / 3 (2-1)

[0144] = + (2-2)

[0145] (2-3)

[0146] < (2-4)

[0147] = [0]< (2-5)

[0148] Here, `points` represents the chest point cloud array. Considering that the symmetry plane of the chest point cloud is close to the YZ plane of the original coordinate system, the point cloud to be projected is obtained. Then, using the default coordinate system XZ plane as the starting plane, rotate around the Z-axis to project the point cloud. Project onto the plane of rotation, obtain the projection center, and calculate the point cloud to be projected. The sum of distances dsum from all points to the projection center after projection, when the sum of distances d sum At its minimum, this is the point cloud to be projected. The correct projection plane, i.e., the correct projection plane q of the chest region point cloud. c .

[0149] The method for establishing the projection coordinate system of the chest region point cloud is as follows: The maximum depth point of the chest region point cloud is used as the reference point. Let q be the origin of the coordinate system and the projection plane be q. c The normal vector passing through the origin is the Y-axis, the normal vector of the Y-axis passing through the origin and parallel to the ground is the X-axis, and the line perpendicular to the XY plane and passing through the origin is the Z-axis. The establishment of the first projected coordinate system is as follows: Figure 18 As shown.

[0150] Based on the obtained projected coordinate system, the chest region point cloud is sliced ​​along the Y-axis with a step size of 0.001 using a plane perpendicular to the ground plane and parallel to the X-axis. Then, each slice is sliced ​​again along the Z-axis with a step size of 0.001. The distance between the centers of two point cloud clusters is calculated; the maximum distance is the widest point in the chest region, i.e., the chest width. The measurement process is as follows: Figure 19 As shown.

[0151] During hip width measurement, point clouds of the hip region are read, and statistical filtering is performed to remove outliers. To avoid some point clouds identifying the posterior edge of the ischium, point cloud P is extracted.middle Then, a projection coordinate system for the gluteal region is established, following the same process as for the chest region. The specific steps for establishing the projection coordinate system for the gluteal region are as follows: Figure 20 As shown.

[0152] Furthermore, the calculation steps for measuring the body's oblique length and other body size parameters are as follows:

[0153] This embodiment measures the oblique length parameter of a sheep's body. Based on the B-interval recognition results of the PointNet++ model, point clouds of the head, chest, rump, and the complete sheep point cloud (3D model of the sheep) are read, and statistical filtering is performed to remove outliers.

[0154] (1) Steps for extracting the measuring points of the anterior border of the scapula:

[0155] To extract the measurement points along the anterior edge of the scapula, the region containing these measurement points needs to be segmented. Based on the projected coordinate system established from the chest region point cloud in chest width measurement, the segmentation process for the anterior edge measurement point region is described below.

[0156] 1. Point clouds and arrays after statistical filtering: Chest point cloud and array: chest_pcd, chest_points. Head point cloud and array: head_pcd, head_points. Complete sheep point cloud: complete_pcd.

[0157] 2. Obtain key points: the minimum and maximum indices of the chest point cloud array in the Y direction and the corresponding points P1 and P2. According to Mid_P = (P2 - P1) / 2, obtain half of the chest point cloud in the direction close to the tail, obtain the maximum index in the Z direction and its corresponding point P3, and calculate the distance from P3 to the ground plane.

[0158] 3. First segmentation preparation: Based on the Y coordinates in P1 and P2, segment the lower boundary min_z_boundary = P3[2] - (2 / 3) The first segmentation is performed using distance.

[0159] 4. First segmentation: Set the bounding box bbox_first for the first segmentation, and use bbox_first to clip complete_pcd to obtain the point cloud after the first segmentation, the array first_seg_complete_pcd, and first_seg_complete_points.

[0160] 5. Obtain key points after the first segmentation: Find the minimum index in the Z direction and its corresponding point P4 in first_seg_complete_points, and the maximum index in the Y direction of the head point cloud array and its corresponding point P5.

[0161] 6. Second segmentation preparation: Calculate average_y = (P5[1] - P4[1]) / 10, Y_min = P4[1] + average_y, Y_max = P4[1] + average_y 3. Perform a second segmentation based on Y_min, Y_max, and min_z_boundary.

[0162] 7. Second segmentation: Set the bounding box `bbox_second` for the second segmentation, and crop `complete_pcd` using `bbox_second` to obtain the point cloud after the second segmentation, the arrays `second_seg_complete_pcd` and `second_seg_complete_points`, which represent the region where the anterior edge of the scapula is located. 8. Segmentation complete.

[0163] Reference Figure 21 The diagram illustrates the process of segmenting the measurement area of ​​the anterior scapula. After obtaining the region containing the anterior scapula measurement points, a plane q1 is constructed perpendicular to the ground plane and passing through the Y-axis of the projection coordinate system. Plane q1 is the bilateral symmetric plane of the anterior scapula measurement point region. The anterior scapula measurement point region is then projected onto the bilateral symmetric plane q1. The lowest point is extracted for each Y-coordinate, thus obtaining the lower contour of the projected anterior scapula measurement point region. Statistical filtering is then applied to the obtained contour to remove outliers.

[0164] After obtaining the above contour, the 3D data needs to be reduced in dimensionality for curve fitting. This study uses a quaternion to rotate the contour to the YZ plane. A quaternion is an extended complex number that can be used to represent a rotation of a certain angle (theta) around a specified axis. It consists of one real part and three imaginary parts, in the form of... Where a is the real part, b, c, and d are the imaginary parts, and i, j, and k are the imaginary units of the quaternion. The specific calculation principle of the quaternion q is given in formulas (2-6)-(2-9):

[0165]

[0166] in, It is the angle between the normal vector of the YZ plane and the normal vector of the plane containing the projected profile. It is the axis of rotation, which is the cross product of the normal vector of the YZ plane and the normal vector of the plane containing the projected contour.

[0167] After reducing the dimensionality of the contour point cloud using quaternions, the coordinate values ​​in the point cloud data may decrease. To facilitate curve fitting, the coordinate values ​​of all points after rotation can be multiplied. Then, the contour is fitted using a quadratic polynomial with the least squares method (LSM) to find the point p with the maximum curvature. c3 Then, after scaling down the coordinates of this point by the corresponding factor, point p is determined in reverse within the point cloud containing the measurement points of the anterior edge of the scapula. c3 Corresponding point p c p c This refers to the measuring point on the anterior border of the scapula, which is obliquely long. See the location diagram below. Figure 22 As shown.

[0168] (2) Locating the measuring point at the posterior border of the ischial tuberosity:

[0169] Similarly, based on the projected coordinate system established from the hip width measurement point cloud, the extraction method for the posterior edge measurement point of the ischial tuberosity is similar to the method for obtaining the anterior edge measurement point of the scapula. It also involves projecting and reducing the dimensionality of the hip points, then using a least-squares quadratic polynomial to fit the rotated contour to a curve. For the fitted curve, the minimum point in the Y direction is searched, and the point p with the maximum curvature is found within a neighborhood of that point with a radius of 10cm. h3 And correspond to the point cloud of the buttocks, corresponding to point p. h This is the measuring point at the posterior border of the ischial tuberosity. Refer to [link / reference] for the specific procedure. Figure 23 As shown.

[0170] The extraction results of the anterior border measuring points of the scapula and the posterior border measuring points of the ischial tuberosity are referenced. Figure 24 As shown, after obtaining their coordinates, the Euclidean distance between them is calculated, which is the body slant length of the sheep. This embodiment chooses quaternion dimensionality reduction.

[0171] Furthermore, the steps for measuring body circumference parameters are as follows: In this embodiment of the invention, the measurement steps for calculating the chest circumference parameter are as follows: Based on the PointNet++ model B annotation recognition results, the chest point cloud and the complete point cloud are read respectively, and statistical filtering is performed to remove outliers. Similarly, in order to extract the chest circumference, the area containing the chest circumference needs to be segmented. Based on the projected coordinate system established from the chest area point cloud in the chest width calculation, the specific segmentation process is as follows:

[0172] 1. Point cloud and array after statistical filtering: Chest point cloud and array: chest_pcd, chest_points. Complete sheep point cloud: complete_pcd.

[0173] 2. Obtain key points: the minimum and maximum indices of the chest point cloud array in the Y direction and their corresponding points P1 and P2. According to Mid_P = (P2 - P1) / 2, obtain half of the chest point cloud in the direction close to the tail, obtain the maximum index in the Z direction and its corresponding point P3, and calculate the distance from P3 to the ground plane.

[0174] 3. First segmentation preparation: Based on the Y coordinates in P1 and P2, segment the lower boundary min_z_boundary = P3[2] - (2 / 3) The first segmentation is performed using distance.

[0175] 4. First segmentation: Set the bounding box bbox_first for the first segmentation, and use bbox_first to clip complete_pcd to obtain the point cloud after the first segmentation, the array first_seg_complete_pcd, and first_seg_complete_points.

[0176] 5. Obtain key points after the first segmentation: Find the minimum index in the Z direction and its corresponding point P4 in first_seg_complete_points.

[0177] 6. Second Segmentation Preparation: Based on the Y coordinates in P1 and P4, segment the lower boundary min_z_boundary for the second segmentation. 7. Second Segmentation: Set the bounding box bbox_second for the second segmentation, and clip complete_pcd using bbox_second to obtain the point cloud after the second segmentation, the arrays second_seg_complete_pcd and second_seg_complete_points, which represent the area containing the chest circumference. 8. Segmentation Complete.

[0178] Furthermore, a flowchart illustrating the process of dividing the bust area is shown below. Figure 25 As shown.

[0179] After segmenting the point cloud to obtain the area containing the bust circumference, similarly, the segmented point cloud is projected onto a bilateral symmetric plane to obtain the lower contour of the projected point cloud containing the bust circumference. Then, statistical filtering is performed on the obtained contour to remove outliers. The maximum value P1 in the Z-direction of the connected point cloud is then determined. 1 Point and minimum point P4 4 Let the straight line be line. Traverse the contour to find the point P that is farthest from the line. Point P is a point on the chest circumference. The intersection of the plane passing through point P and parallel to the XZ plane with the 3D point cloud of the sheep is the chest circumference contour.

[0180] After obtaining the 3D chest contour point cloud, it is necessary to reduce the dimensionality of the 3D point cloud for fitting. The dimensionality reduction process is as follows: First, a 2D completely black image (all pixel values ​​are 0) that completely overlaps with the chest contour point cloud is created as the background image. The position of each point in the chest contour point cloud in the background image is calculated, and then the corresponding pixel value is set to 255. By this calculation, the 3D point cloud can be reduced to the 2D completely black background image.

[0181] Then, all pixels with a value of 255 are extracted to obtain the two-dimensional outline of the bust. To prevent the influence of the front legs, the upper three-quarters of all pixels with a value of 255 are fitted with an ellipse using the least squares method.

[0182] Finally, calculate the perimeter of the ellipse in the background image, multiply it by the length represented by a single pixel in the point cloud, and you will get the bust circumference. See below for the detailed calculation process of the bust circumference. Figure 26 As shown, the chest circumference point cloud and fitting results are as follows: Figure 27 As shown.

[0183] Compared with manual measurement data, the relative error of the body size data proposed based on this invention is within 5%, which shows high accuracy.

[0184] This invention primarily defines six body measurement parameters for sheep: height, chest width, hip height, hip width, body length, and chest circumference. An automatic measurement algorithm based on 3D point clouds for these six parameters is also designed. Experimental results show that the average relative errors for height, chest width, hip height, hip width, body length, and chest circumference are 2.15%, 8.62%, 2.16%, 5.96%, 1.74%, and 1.8%, respectively. The relative errors in the data measurements are all within 5%, demonstrating high accuracy.

[0185] In some embodiments of the present invention, the method for measuring the body size of sheep based on three-dimensional reconstruction and point cloud segmentation provided by the present invention may include, but is not limited to, the following:

[0186] (1) Data acquisition platform setup. Based on the actual measurement environment and requirements, a 3-meter-wide and 2-meter-high through-arch support structure was erected. A scale was placed directly below the support structure to measure the sheep's body weight. A KinectV2 camera was then installed on the left, top, and right sides for image acquisition. Each KinectV2 camera was connected to a laptop computer for camera control and image storage. Using the established data acquisition platform, data was collected from 35 Dorper sheep aged 3 years. The acquired depth images were then converted into point clouds, and the point clouds of the target sheep were extracted from the point clouds.

[0187] (2) 3D Reconstruction of Sheep Body Surface Point Clouds. To address the registration error between the abdomen and back encountered during 3D reconstruction, a method was proposed that divide the back point cloud into left and right parts along the midline plane and register them with the corresponding side view point clouds. The improved 3D reconstruction method achieved a 100% success rate in the sheep point clouds used in the experiment, indicating that the improved method has good generalization and robustness.

[0188] (3) Segmentation of sheep point cloud regions. Due to the diverse postures and frequent activities of sheep, and the dense and numerous point clouds resulting from 3D reconstruction, directly extracting measurement points from the overall point cloud is difficult to guarantee accuracy. To address this issue, this embodiment of the invention proposes using the PointNet++ model to segment point cloud regions, thereby improving the accuracy of measurement point extraction. Experimental results show that the point cloud region segmentation method proposed in this embodiment can effectively reduce the dependence of sheep body size measurement on posture and improve measurement accuracy.

[0189] (4) Automatic detection of sheep body size. Based on the geometric features and positional constraints of the body size measurement points on the sheep's phenotypic contour, an algorithm was designed to automatically determine the measurement points and calculate the body size parameters. Experimental results show that when the sheep is in a standard posture, the relative errors of body height, chest width, hip height, hip width, body length, and chest circumference are ≤5%; when the average of 5 measurements of the same sheep is taken, the average relative errors of body height, chest width, hip height, hip width, body length, and chest circumference are 2.1%, 2.84%, 2.97%, 0.43%, 0.98%, and 0.15%, respectively, with a relative error of ≤3%; when the sheep is in a non-standard standing posture, the error of the body size data is larger.

[0190] (5) Design of a sheep body size measurement system. To facilitate the implementation of the embodiments of the present invention, the present invention also provides a sheep body size measurement system based on 3D reconstruction and point cloud segmentation. This system is written in Python and visualized using the PyQt5 framework. Based on the automated sheep body size parameter measurement process proposed in this paper, automatic body size measurement software was developed. This software automates a series of processes from camera calibration to body size measurement.

[0191] This invention provides a method, device, medium, and program product for measuring the body size of sheep based on 3D reconstruction and point cloud segmentation. The method involves acquiring images of the sheep to be measured and preprocessing these images to obtain multi-view point clouds of the sheep target; performing point cloud registration on the multi-view point clouds to obtain a 3D model of the sheep; and performing point cloud segmentation based on key points for body size measurement and the 3D model of the sheep to obtain a point cloud of key point regions corresponding to the key points for body size measurement. The key points for body size measurement include at least body height, chest width, hip height, hip width, and body slant length. The method includes at least one of the following: chest circumference; determining the projection space of the region point cloud based on the key point region point cloud, and performing posture normalization on the interval point cloud according to the region point cloud projection space to determine the posture-normalized key point region point cloud; the key point region point cloud includes at least the interval point cloud; calculating the body size data of the sheep to be measured based on the posture-normalized key point region point cloud; wherein the body size data of the sheep to be measured includes at least one of body height data, chest width data, hip height data, hip width data, body slope length data, and chest circumference data. This method addresses the shortcomings of existing technologies, such as low efficiency of manual measurement and injury to sheep caused by X-ray measurement. It achieves non-contact measurement of sheep's body size data in a natural state by reconstructing the sheep's body surface point cloud in three dimensions, improving measurement efficiency and accuracy, and enhancing the generalization and robustness of measuring the body size of sheep of different breeds and postures.

[0192] Figure 28 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 28As shown, the electronic device may include: a processor 2710, a communication interface 2720, a memory 2730, and a communication bus 2740. The processor 2710, communication interface 2720, and memory 2730 communicate with each other via the communication bus 2740. The processor 2710 can call logical instructions in the memory 2730 to execute a sheep body size measurement method based on 3D reconstruction and point cloud segmentation. This method includes: acquiring sheep images of the sheep to be measured and preprocessing the sheep images to obtain multi-view sheep target point clouds; performing point cloud registration processing on the multi-view sheep target point clouds to obtain a 3D sheep model; and performing point cloud segmentation processing based on body size measurement key points and the 3D sheep model to obtain key point region point clouds corresponding to the body size measurement key points. The body size measurement key points include at least body height, chest width, hip height, hip width, body slant length, etc. At least one of the following: chest circumference; based on the key point region point cloud, determine the region point cloud projection space, and perform posture normalization on the interval point cloud according to the region point cloud projection space to determine the posture-normalized key point region point cloud; the key point region point cloud includes at least the interval point cloud; based on the posture-normalized key point region point cloud, calculate the body size data of the sheep to be tested; wherein, the body size data of the sheep to be tested includes at least one of body height data, chest width data, hip height data, hip width data, body diagonal length data, and chest circumference data.

[0193] Furthermore, the logical instructions in the aforementioned memory 2730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0194] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation provided by the above methods. The method includes: acquiring sheep images of the sheep to be measured and preprocessing the sheep images to obtain multi-view sheep target point clouds; performing point cloud registration processing on the multi-view sheep target point clouds to obtain a three-dimensional sheep model; performing point cloud segmentation processing based on body size measurement key points and the three-dimensional sheep model to obtain key point region point clouds corresponding to the body size measurement key points; wherein, the body size measurement key points include at least body height, chest width, hip height, hip width, body oblique length, etc. At least one of the following: chest circumference; based on the key point region point cloud, determine the region point cloud projection space, and perform posture normalization on the interval point cloud according to the region point cloud projection space to determine the posture-normalized key point region point cloud; the key point region point cloud includes at least the interval point cloud; based on the posture-normalized key point region point cloud, calculate the body size data of the sheep to be tested; wherein, the body size data of the sheep to be tested includes at least one of body height data, chest width data, hip height data, hip width data, body diagonal length data, and chest circumference data.

[0195] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for measuring the body size of sheep based on 3D reconstruction and point cloud segmentation, as provided by the methods described above. The method includes: acquiring sheep images of the sheep to be measured and preprocessing the sheep images to obtain multi-view sheep target point clouds; performing point cloud registration processing on the multi-view sheep target point clouds to obtain a 3D model of the sheep; performing point cloud segmentation processing based on body size measurement key points and the 3D model of the sheep to obtain key point region point clouds corresponding to the body size measurement key points; and calculating the body size data of the sheep to be measured based on the pose-normalized key point region point clouds.

[0196] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0197] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for measuring the body size of sheep based on three-dimensional reconstruction and point cloud segmentation, characterized in that, The method comprises the following steps: Collecting a sheep image of a to-be-measured sheep, and pre-processing the sheep image to obtain a multi-view sheep target point cloud; Performing point cloud registration processing on the multi-view sheep target point cloud to obtain a three-dimensional sheep model; Based on the body size measurement key points and the three-dimensional sheep model, performing point cloud segmentation processing to obtain key point region point clouds corresponding to the body size measurement key points; wherein the body size measurement key points at least include at least one of body height, chest width, hip height, hip width, body oblique length, and chest circumference; Based on the key point region point cloud, a region point cloud projection space is determined, and the interval point cloud is posture-normalized according to the region point cloud projection space to determine the posture-normalized key point region point cloud; the key point region point cloud at least includes an interval point cloud, and the interval point cloud at least includes a chest region point cloud; The step of determining the region point cloud projection space based on the key point region point cloud and posture-normalizing the interval point cloud according to the region point cloud projection space to determine the posture-normalized key point region point cloud comprises: Obtaining the chest region point cloud from the key point region point cloud; Traversing the chest region point cloud to find the maximum and minimum points of the distance from the ground level, and extracting the point cloud located between one-third and two-thirds of the interval between the maximum and minimum points as the to-be-projected point cloud; Rotating the to-be-projected point cloud around the Z-axis with the default coordinate system XZ plane as the starting plane, projecting the to-be-projected point cloud onto the rotation plane, obtaining the projection center, calculating the sum of the distances of all points of the to-be-projected point cloud after projection to the projection center, and when the sum is the smallest, taking the projection plane corresponding to the to-be-projected point cloud as the target projection plane; Taking the chest region point cloud depth maximum point as the coordinate origin, the normal vector of the target projection plane passing through the coordinate origin as the Y-axis, the Y-axis normal vector passing through the coordinate origin and parallel to the ground as the X-axis, and the straight line perpendicular to the XY plane and passing through the coordinate origin as the Z-axis, the chest region point cloud is posture-normalized to determine the posture-normalized key point region point cloud according to the posture-normalized chest region point cloud; Based on the posture-normalized key point region point cloud, the body size data of the to-be-measured sheep is calculated; wherein the body size data of the to-be-measured sheep includes at least one of body height data, chest width data, hip height data, hip width data, body oblique length data, and chest circumference data.

2. The sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation according to claim 1, wherein The step of calculating the body size data of the to-be-measured sheep based on the posture-normalized key point region point cloud reduces the dependence on posture in the body size measurement process.

3. The sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation according to claim 1, wherein The sheep image at least includes a left view, a top view, and a right view of the to-be-measured sheep; The multi-view sheep target point cloud at least includes a left view target point cloud, a top view target point cloud, and a right view target point cloud; The pre-processing of the sheep image to obtain a multi-view sheep target point cloud comprises: Determine the camera internal parameter of the acquisition device for acquiring the sheep image; wherein the camera internal parameter is characterized as projection parameter information for mapping a depth image into a three-dimensional point cloud; Based on the camera internal parameter, the two-dimensional pixel coordinates of the left view, the top view and the right view of the sheep to be measured are respectively converted into three-dimensional camera coordinates to obtain the corresponding left view point cloud image, the top view point cloud image and the right view point cloud image; The left view point cloud image and the right view point cloud image are respectively subjected to straight-through filtering processing to obtain the left view point cloud image and the right view point cloud image with background noise removed, and the foreground images of the left view point cloud image and the right view point cloud image are respectively extracted, and the foreground images of the left view point cloud image and the right view point cloud image extracted are respectively subjected to statistical filtering processing to remove outliers to obtain the corresponding left view target point cloud and right view target point cloud; and Draw a point cloud height distribution graph of the top view point cloud image, and determine a segmentation threshold of the top view point cloud and the ground point cloud according to the point cloud height distribution graph; Iterate through the filtered top view point cloud image to find the maximum depth value point and the minimum depth value point, and calculate the height difference between the maximum depth value point and the minimum depth value point; The point cloud with a depth value less than one-fourth of the height difference is taken as the ground point cloud, and the point cloud with a depth value greater than or equal to one-fourth of the height difference is taken as the top view target point cloud.

4. The sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation according to claim 3, wherein The point cloud registration processing of the multi-view sheep target point cloud is performed to obtain a three-dimensional sheep model, comprising: segmenting the top view target point cloud to obtain a first top view point cloud and a second top view point cloud; performing registration processing on the first top view point cloud and the left view target point cloud to obtain a first registration image, and performing registration processing on the second top view point cloud and the right view target point cloud to obtain a second registration image; performing merging processing on the first registration image and the second registration image to obtain a merged registration image; performing merging processing on the merged registration image and the ground point cloud to obtain the three-dimensional sheep model.

5. The sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation according to claim 1, wherein The point cloud segmentation processing based on the body size measurement key point and the three-dimensional sheep model is performed to obtain a key point region point cloud corresponding to the body size measurement key point, comprising: inputting the body size measurement key point and the three-dimensional sheep model into a trained segmentation prediction model to predict the key point region point cloud; The trained segmentation prediction model at least includes a first segmentation prediction model and a second segmentation prediction model; the first segmentation prediction model is an interval identification model for measuring the chest width, body height, hip height and hip width of the to-be-measured sheep; the second segmentation prediction model is an interval identification model for measuring the chest girth and body slant length of the to-be-measured sheep; the first segmentation prediction model is obtained by training using chest region point cloud samples containing body height and chest width measuring points and hip region point cloud samples containing hip height and hip width measuring points; and the second segmentation prediction model is obtained by training using chest region point cloud samples containing scapula front edge measuring points and chest girth point cloud and hip region point cloud samples containing ischium tuberosity rear edge measuring points.

6. The sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation according to claim 4, characterized in that, The step of calculating the body height data of the to-be-measured sheep comprises: obtaining chest region point cloud from the key point region point cloud; finding the point farthest from the ground plane in the chest region point cloud as the highest point of the chest based on a ground plane equation; wherein the ground plane equation is obtained by fitting the ground point cloud using a random consistency algorithm; constructing a chest region neighborhood with the highest point as the center and a preset length as the radius in the chest region point cloud; calculating the distance of each point in the chest region neighborhood to the ground plane to obtain a plurality of chest region heights; excluding the maximum chest region height and the minimum chest region height from the plurality of chest region heights, and calculating the average of the remaining chest region heights, and taking the average of the remaining chest region heights as the body height data of the to-be-measured sheep.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the program to realize the sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation according to any one of claims 1 to 6.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to realize the sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation according to any one of claims 1 to 6.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to realize the sheep body size measurement method based on three-dimensional reconstruction and point cloud segmentation according to any one of claims 1 to 6.

Images