Cow body size measuring method based on colorized point cloud multi-view key point positioning

Abstract

The invention provides a colorization point cloud multi-view key point positioning-based cattle body size measurement method, which comprises the following steps of: acquiring cattle 3D point cloud to construct an initial point cloud data set, acquiring a three-view picture set by utilizing a spatial point colorization model, and dividing the three-view picture set into a training set, a verification set and a test set; constructing a key point identification model, performing training by using the training set, performing verification by using the verification set, performing key point identification on the test set after obtaining the key point identification model with the optimal performance, and obtaining two-dimensional plane coordinates of a plurality of key points; converting the two-dimensional plane coordinates into three-dimensional space coordinates by using a different-dimensional key point coordinate conversion algorithm, and calculating the body size parameters of the cattle by using the three-dimensional space coordinates of all key points; according to the invention, accurate positioning of the measurement key points of the cattle 3D point cloud body scale is realized, and the measurement precision is improved; meanwhile, based on a plurality of key points which are accurately positioned, various cattle body size parameters can be accurately calculated, and complete parameter calculation is realized.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional point cloud processing and deep learning technology, and more specifically, to a method for measuring cattle body size based on multi-view key point positioning of colorized point clouds. Background Art

[0002] Body measurement is an accurate method for directly evaluating the health status of cattle in modern animal husbandry. Measuring various indicators of cattle body size can help breeders evaluate the growth and development status, physical quality and production performance of cattle. Based on these data, breeders can select suitable breeding objects for the selection and reproduction of excellent cattle, improve the genetic quality of cattle and improve the production performance and economic benefits of cattle.

[0003] However, traditional manual measurement methods can no longer meet the needs of modernization, and are prone to cause stress reactions in livestock, affecting the accuracy of the results. In addition, the livestock industry has long faced problems such as insufficient technological innovation, low resource utilization efficiency, unstable product quality, low value conversion rate, and difficulty in achieving environmental benefits, which have restricted the process of agricultural modernization. The use of non-contact technologies such as automated measurement can not only improve measurement efficiency and reduce the consumption of human resources, but also reduce the probability of stress reactions in cattle due to tension, thereby more accurately assessing health conditions. Therefore, in the process of improving the level of cattle breeding technology and selecting superior varieties to increase production, automated morphological measurement technology is essential.

[0004] In recent years, non-contact automated body measurement has developed rapidly. For example, an existing patent document discloses a method for measuring cattle body size based on complementary dual-camera information. Two cameras set up in the channel, one directly above and one on the side, are used to collect images of the same cattle at different angles. When the side image of the cattle is incomplete, the cattle shoulder, tail and rump measurement point extraction algorithm based on yolov5s and envelope corner point detection is first used to extract measurement points from the cattle back image. Then, target detection is performed in the side camera image through an auxiliary ruler to obtain the parameters required for height calculation. Then, the proposed visual triangulation height measurement algorithm based on shoulder measurement point extraction can be used to calculate The height of the cow's shoulder blade measuring point from the ground is obtained, that is, the cow's height; then the cow's height is subtracted from the height of the camera directly above to obtain the depth of field of the cow in the camera image directly above, and finally the straight length data of the cow's body is obtained by using the proposed similar triangle straight length measurement algorithm based on the extraction of shoulder blade tail rump measuring points; however, most of the non-contact automatic body size measurement of cattle, including this scheme, faces the following challenges: First, the external morphological data features of the cow after 3D reconstruction are not obvious, which makes it difficult to accurately locate the key points of the cow's body size, thereby affecting the accuracy of the measurement results; second, the cow is large, and it is impossible to calculate all body size parameters under a single or non-full coverage perspective, resulting in incomplete measurement parameters. Summary of the invention

[0005] In order to overcome the defects of the above-mentioned prior art in inaccurate positioning of body size measurement key points in cattle 3D point clouds and inability to calculate all body size parameters, the present invention provides a cattle body size measurement method based on multi-view key point positioning of colorized point clouds. Starting from the data annotation stage of deep learning, a spatial point colorization model based on global spatial features and local image features is constructed, and a hetero-dimensional key point coordinate conversion algorithm based on depth-free camera intrinsic parameters is used to achieve accurate positioning of key points for body size measurement of cattle 3D point clouds, thereby improving the accuracy of body size measurement. At the same time, based on the accurately positioned multiple key points, various cattle body size parameters can be accurately calculated, and complete parameter calculation is achieved.

[0006] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0007] A method for measuring cattle body size based on multi-view key point positioning of colorized point cloud, comprising the following steps:

[0008] S1: Collect cattle 3D point clouds and construct the initial point cloud dataset;

[0009] S2: inputting the initial point cloud data set into a pre-built spatial point colorization model for processing, obtaining a three-view image set and dividing it into a training set, a validation set and a test set, and annotating key points of the training set and the validation set;

[0010] The three-view picture set includes a plurality of two-dimensional pictures of the cow in three views: side view, top view and rear view;

[0011] S3: Build a key point recognition model based on the deep learning algorithm, and use the training set after key point annotation to train the key point recognition model, and use the verification set after key point annotation to verify the trained key point recognition model to obtain the key point recognition model with the best performance;

[0012] S4: inputting the test set into the key point recognition model with the best performance to perform key point recognition, and obtaining the two-dimensional plane coordinates of several key points;

[0013] S5: The two-dimensional plane coordinates of all key points are converted into three-dimensional space coordinates using a preset hetero-dimensional key point coordinate conversion algorithm; the body size parameters of the cattle are calculated using the three-dimensional space coordinates of all key points to complete the body size measurement of the cattle.

[0014] Preferably, in step S2, the initial point cloud data set is input into a pre-built spatial point colorization model for processing, and obtaining a three-view picture set includes:

[0015] The pre-built spatial point colorization model performs the following operations on each cow 3D point cloud in the initial point cloud dataset:

[0016] The 3D point cloud of the cow is recorded as point set P, and the 3D spatial coordinate set of point set P is expressed as: p = {(x 1 ,y 1 ,z 1 ),(x 2 ,y 2 ,z 2 ),..,(x n ,y n ,z n )}, where n is the number of points in the 3D point cloud of the cow, (x i ,y i ,z i ) represents the 3D spatial coordinates of the i-th point in the 3D point cloud;

[0017] Adjust the direction of the 3D point cloud of the cow so that the head direction of the cow, the longitudinal direction of the cow and the transverse direction of the cow are parallel to the X-axis, Y-axis and Z-axis directions respectively;

[0018] The normal vectors of the three viewing planes of the cow's 3D point cloud, namely the side view, top view and rear view, are estimated based on the local surface fitting method.

[0019] Color mapping of the 3D point cloud of the cow: For the three views of the cow, the top view and the rear view, the Z-axis coordinate set Z = {z 1 ,z 2 ,z 3 ,...,z n}、Y axis coordinate set Y={y 1 ,y 2 ,y 3 ,...,y n} and the X-axis coordinate set X = {x 1 ,x 2 ,x 3 ,...,x n}, normalize the three coordinate sets respectively, map them to the interval [0,1] respectively, and obtain the three normalized coordinate sets;

[0020] According to the three normalized coordinate sets, the terrain color band in the Matplotlib data visualization library is used to perform color mapping on the 3D point cloud of the cow in three dimensions, and 3D color point clouds corresponding to the three views of the cow, namely the side view, top view and rear view, are generated respectively;

[0021] The three groups of 3D color point clouds were visualized using Open3D's physical lighting engine, and the initial two-dimensional images of the cow's side view, top view, and rear view were extracted. By calculating the normal vectors of the three views, the rendering effect can be better.

[0022] The initial two-dimensional images of the three views of the cow are cropped and sharpened to obtain the three-view images corresponding to the 3D point cloud of the cow;

[0023] The three-view images corresponding to all the cattle 3D point clouds are saved together as a three-view image set.

[0024] Preferably, the estimating the normal vector of the 3D point cloud of the cattle based on the local surface fitting method comprises:

[0025] A point in the 3D point cloud of the cattle is selected, and a hybrid search strategy is used to search for a number of points within a spherical range of a preset radius around the selected point, to construct a local neighborhood of the selected point and to fit a plane using the local neighborhood;

[0026] Construct the following problem and solve it:

[0027]

[0028] sA 2 +B 2 +C 2 =1

[0029] Where A, B, C and D are the first, second, third and fourth parameters of the fitting plane respectively;

[0030] Save the solution results of A, B and C as the normal vector of the cow's 3D point cloud Specifically:

[0031]

[0032] Among them, n x 、n y and n z They are normal vectors The three-dimensional coordinates of 0 , B 0 and C 0 ) are the solutions for the positive and negative terms of A, B, and C respectively; T represents the transpose of the matrix.

[0033] Preferably, the following normalization processing is performed on the three coordinate sets respectively:

[0034]

[0035] Among them, x i,n0rm ,y i,norm and z i,norm are the normalized X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively; max ,y max and z max are the maximum values ​​of the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively;min ,y min and z min are the minimum values ​​of the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively.

[0036] Preferably, in step S2, the key points of the training set and the validation set are annotated using the Labelme data annotation tool.

[0037] Preferably, in step S3, the key point recognition model constructed based on the deep learning algorithm is specifically a YOLOv8-pose neural network model.

[0038] Preferably, in step S5, the two-dimensional plane coordinates of all key points are converted into three-dimensional space coordinates using a preset different-dimensional key point coordinate conversion algorithm:

[0039] Obtain the size of the two-dimensional image for key point recognition in the test set; construct an AABB bounding box of the 3D point cloud of the cow corresponding to the two-dimensional image, and obtain the coordinates of eight vertices of the AABB bounding box;

[0040] Detect whether the pixel color of each key point identified in step S3 is white. If so, the key point is located outside the cow body, and the two-dimensional plane coordinates of the key point are migrated to the nearest color pixel for position correction; otherwise, the key point is located inside the cow body, and the next key point is detected until all key points are detected;

[0041] According to the size of the two-dimensional image, the vertex coordinates of the AABB bounding box, and the two-dimensional plane coordinates of all key points after detection, the plane approximate coordinates of each key point corresponding to two coordinate axes in the three-dimensional space are calculated, and the local area of ​​interest of each key point in the 3D point cloud of the cattle is obtained according to the plane approximate coordinates;

[0042] The local area of ​​interest contains several point cloud slices distributed along the third coordinate axis. The center point of the point cloud slice closest to the AABB bounding box interface is used as the key point after mapping, and the three-dimensional space coordinates of the center point are obtained and used as the three-dimensional space coordinates of the key point after mapping.

[0043] Preferably, the size of the two-dimensional image is recorded as a×b, where a and b are the length and width of the two-dimensional image respectively; the two-dimensional plane coordinates of the key points after detection are recorded as (p, q);

[0044] Starting from the vertex of the upper right corner of the upper surface of the AABB bounding box, the coordinates of the four vertices on the upper surface of the AABB bounding box are recorded as (x 1 ,y 1 ,z 1 )、(x2 ,y 2 ,z 2 )、(x 3 ,y 3 ,z 3 ) and (x 4 ,y 4 ,z 4 ), the vertex coordinates corresponding to the lower surface of the AABB bounding box are recorded as (x 5 ,y 5 ,z 5 )、(x 6 ,y 6 ,z 6 )、(x 7 ,y 7 ,z 7 ) and (x 8 ,y 8 ,z 8 ); the size of the AABB bounding box is l×w×h, where l, w and h are the length, width and height of the AABB bounding box respectively;

[0045] The three-dimensional space coordinates of the mapped key points are marked as (x 0 ,y 0 ,z 0 ),but:

[0046] If the 2D image is a side view, calculate the approximate coordinates of the key point (p, q) on the X-axis and Y-axis planes according to the following formula:

[0047]

[0048] At this time, the obtained point cloud slices are distributed along the Z axis. 0 The value of is determined by the coordinates of the center point of the point cloud slice closest to the AABB bounding box interface;

[0049] If the 2D image is a top view, calculate the approximate plane coordinates of the key point (p, q) on the X-axis and Z-axis according to the following formula;

[0050]

[0051] At this time, the obtained point cloud slices are distributed along the Y axis, y 0 The value of is determined by the coordinates of the center point of the point cloud slice closest to the AABB bounding box interface;

[0052] If the 2D image is a rear view, calculate the approximate plane coordinates of the key point (p, q) on the Y and Z axes according to the following formula;

[0053]

[0054]

[0055] At this time, the obtained point cloud slices are distributed along the X axis. 0 The value of is determined by the coordinates of the center point of the point cloud slice that is closest to the AABB bounding box interface.

[0056] Preferably, in step S5, the body size parameters of the cattle include: body oblique length, chest depth, hip height, buttocks length, body height, body straight length, abdominal circumference, cross height, chest circumference, chest width, waist angle width, hip width and ischial width.

[0057] The present invention also provides a cattle body size measurement system based on color point cloud multi-view key point positioning, and the cattle body size measurement method based on color point cloud multi-view key point positioning is applied, comprising:

[0058] Point cloud acquisition unit: used to collect 3D point clouds of cattle and construct the initial point cloud data set;

[0059] A data set construction unit is used to input the initial point cloud data set into a pre-constructed spatial point colorization model for processing, obtain a three-view picture set and divide it into a training set, a validation set and a test set, and annotate key points of the training set and the validation set;

[0060] The three-view picture set includes a plurality of two-dimensional pictures of the cow in three views: side view, top view and rear view;

[0061] Model training unit: used to build a key point recognition model based on a deep learning algorithm, and train the key point recognition model using a training set with key point annotations, and verify the trained key point recognition model using a verification set with key point annotations, to obtain a key point recognition model with the best performance;

[0062] Key point recognition unit: used for inputting the test set into the key point recognition model with the best performance to perform key point recognition and obtain the two-dimensional plane coordinates of several key points;

[0063] Body size measurement unit: used to convert the two-dimensional plane coordinates of all key points into three-dimensional space coordinates using a preset hetero-dimensional key point coordinate conversion algorithm; calculate the body size parameters of the cattle using the three-dimensional space coordinates of all key points to complete the body size measurement of the cattle.

[0064] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0065] The present invention provides a method for measuring the body size of cattle based on multi-view key point positioning of colorized point cloud. First, a 3D point cloud of cattle is collected and an initial point cloud data set is constructed. The initial point cloud data set is input into a pre-constructed spatial point colorization model for processing, a three-view picture set is obtained and divided into a training set, a verification set and a test set, and key points are annotated for the training set and the verification set. Then, a key point recognition model is constructed based on a deep learning algorithm, and the key point recognition model is trained by using the training set after the key points are annotated, and the trained key point recognition model is verified by using the verification set after the key points are annotated, so as to obtain the key point recognition model with the best performance. Then, the test set is input into the key point recognition model with the best performance for key point recognition, and the two-dimensional plane coordinates of several key points are obtained. Finally, the two-dimensional plane coordinates of all key points are converted into three-dimensional space coordinates by using a preset hetero-dimensional key point coordinate conversion algorithm. The body size parameters of cattle are calculated by using the three-dimensional space coordinates of all key points, and the body size measurement of cattle is completed.

[0066] In the prior art, it is common to improve the deep learning network to improve the accuracy of key point positioning, but the accuracy of data annotation in deep learning is often ignored. Therefore, the present invention starts from the data annotation stage of deep learning, and by constructing a spatial point colorization model based on global spatial features and local image features, a strategy of combining geographical terrain color bands with cattle point clouds is adopted to convert the cattle phenotype point cloud colors into terrain-like colors under different orthogonal perspectives, and coordinate point cloud physical rendering to highlight the surface texture effect and generate two-dimensional images under different perspectives, converting the 3D point cloud data annotation problem into a 2D image data annotation problem, reducing the difficulty of data annotation and improving the accuracy of key point positioning; secondly, the present invention and the hetero-dimensional key point coordinate conversion algorithm based on the intrinsic parameters of the depth-free camera can realize the mutual conversion of 3D point cloud coordinates and two-dimensional image coordinates, realize the precise positioning of the key points of the 3D point cloud body size measurement of cattle, thereby improving the accuracy of body size measurement; in addition, based on the precisely positioned multiple key points, the present invention can accurately calculate various cattle body size parameters and realize complete parameter calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a flow chart of a cattle body size measurement method based on multi-view key point positioning of colorized point cloud provided in Example 1.

[0068] Figure 2 This is an architecture diagram of a cattle body size measurement method based on multi-view key point positioning of colorized point cloud provided in Example 2.

[0069] Figure 3 This is an example diagram of the 3D point cloud of a cow provided in Example 2.

[0070] Figure 4This is a flow chart of the spatial point colorization model data processing provided in Example 2.

[0071] Figure 5 This is a schematic diagram of estimating possible locations of key points based on the statistical method provided in Example 2.

[0072] Figure 6 Schematic diagram of the data annotation heat map auxiliary mechanism provided in Example 2.

[0073] Figure 7 This is a schematic diagram of key point marking provided in Example 2.

[0074] Figure 8 This is a schematic diagram of the AABB bounding box provided in Example 2.

[0075] Fig. 9 This is a schematic diagram of the calculation of the key point mapping of the cow body provided in Example 2.

[0076] Fig.10 This is a schematic diagram of the three-dimensional spatial coordinates of the key points after mapping provided in Example 2.

[0077] Fig.11 This is a schematic diagram of the final mapping effect of the key points of the top view mapping provided in Example 2.

[0078] Fig.12 This is a schematic diagram of the measurement of various parameters of cattle body size provided in Example 2.

[0079] Fig.13 This is a comparison diagram of the original color image and the colorized image of the cow point cloud provided in Example 2.

[0080] Fig.14 This is a comparison chart of the training results of the original color image and the colorized image of the cow point cloud provided in Example 2.

[0081] Fig.15 This is a box plot of the various body size indicators of cattle provided in Example 2.

[0082] Fig.16 This is a structural diagram of a cattle body size measurement system based on multi-view key point positioning of colorized point cloud provided in Example 3. DETAILED DESCRIPTION

[0083] The drawings are for illustrative purposes only and should not be construed as limiting the present patent;

[0084] In order to better illustrate the present embodiment, some parts in the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product;

[0085] It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0086] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0087] Example 1

[0088] like Figure 1 As shown, this embodiment provides a method for measuring cattle body size based on multi-view key point positioning of colorized point cloud, comprising the following steps:

[0089] S1: Collect cattle 3D point clouds and construct the initial point cloud dataset;

[0090] S2: inputting the initial point cloud data set into a pre-built spatial point colorization model for processing, obtaining a three-view image set and dividing it into a training set, a validation set and a test set, and annotating key points of the training set and the validation set;

[0091] The three-view picture set includes a plurality of two-dimensional pictures of the cow in three views: side view, top view and rear view;

[0092] S3: Build a key point recognition model based on the deep learning algorithm, and use the training set after key point annotation to train the key point recognition model, and use the verification set after key point annotation to verify the trained key point recognition model to obtain the key point recognition model with the best performance;

[0093] S4: inputting the test set into the key point recognition model with the best performance to perform key point recognition, and obtaining the two-dimensional plane coordinates of several key points;

[0094] S5: The two-dimensional plane coordinates of all key points are converted into three-dimensional space coordinates using a preset hetero-dimensional key point coordinate conversion algorithm; the body size parameters of the cattle are calculated using the three-dimensional space coordinates of all key points to complete the body size measurement of the cattle.

[0095] In the specific implementation process, the 3D point cloud of cattle is first collected and the initial point cloud dataset is constructed;

[0096] The initial point cloud data set is input into the pre-built spatial point colorization model for processing, and the three-view image set is obtained and divided into a training set, a validation set, and a test set, and key points of the training set and the validation set are annotated;

[0097] Then, a key point recognition model is constructed based on a deep learning algorithm. The key point recognition model is trained using the training set after key point annotation, and the trained key point recognition model is verified using the verification set after key point annotation to obtain the key point recognition model with the best performance.

[0098] Then, the test set is input into the key point recognition model with the best performance to perform key point recognition and obtain the two-dimensional plane coordinates of several key points;

[0099] Finally, the preset hetero-dimensional key point coordinate conversion algorithm is used to convert the two-dimensional plane coordinates of all key points into three-dimensional space coordinates; the body size parameters of the cattle are calculated using the three-dimensional space coordinates of all key points to complete the body size measurement of the cattle;

[0100] This method starts from the data annotation stage of deep learning. By constructing a spatial point colorization model based on global spatial features and local image features, and a heterogeneous key point coordinate conversion algorithm based on depth-free camera intrinsic parameters, it achieves the precise positioning of key points in the 3D point cloud body size measurement of cattle, thereby improving the accuracy of body size measurement. At the same time, based on multiple precisely positioned key points, various cattle body size parameters can be accurately calculated, realizing complete parameter calculation.

[0101] Example 2

[0102] This embodiment provides a method for measuring cattle body size based on multi-view key point positioning of colorized point cloud, comprising the following steps:

[0103] S1: Collect cattle 3D point clouds and construct the initial point cloud dataset;

[0104] S2: inputting the initial point cloud data set into a pre-built spatial point colorization model for processing, obtaining a three-view image set and dividing it into a training set, a validation set and a test set, and annotating key points of the training set and the validation set;

[0105] The three-view picture set includes a plurality of two-dimensional pictures of the cow in three views: side view, top view and rear view;

[0106] S3: Build a key point recognition model based on the deep learning algorithm, and use the training set after key point annotation to train the key point recognition model, and use the verification set after key point annotation to verify the trained key point recognition model to obtain the key point recognition model with the best performance;

[0107] S4: inputting the test set into the key point recognition model with the best performance to perform key point recognition, and obtaining the two-dimensional plane coordinates of several key points;

[0108] S5: using a preset hetero-dimensional key point coordinate conversion algorithm to convert the two-dimensional plane coordinates of all key points into three-dimensional space coordinates; using the three-dimensional space coordinates of all key points to calculate the body size parameters of the cattle, and complete the body size measurement of the cattle;

[0109] In step S2, the initial point cloud data set is input into a pre-built spatial point colorization model for processing, and obtaining a three-view picture set includes:

[0110] The pre-built spatial point colorization model performs the following operations on each cow 3D point cloud in the initial point cloud dataset:

[0111] The 3D point cloud of the cow is recorded as point set P, and the 3D spatial coordinate set of point set P is expressed as: P = {(x 1 ,y 1 ,z 1 ),(x 2 ,y 2 ,z 2 ),..,(x n ,y n ,z n )}, where n is the number of points in the 3D point cloud of the cow, (x i ,y i ,z i ) represents the 3D spatial coordinates of the i-th point in the 3D point cloud;

[0112] Adjust the direction of the 3D point cloud of the cow so that the head direction of the cow, the longitudinal direction of the cow and the transverse direction of the cow are parallel to the X-axis, Y-axis and Z-axis directions respectively;

[0113] The normal vectors of the three viewing planes of the cow's 3D point cloud, namely the side view, top view and rear view, are estimated based on the local surface fitting method.

[0114] Color mapping of the 3D point cloud of the cow: For the three views of the cow, the top view and the rear view, the Z-axis coordinate set Z = {z 1 ,z 2 ,z 3 ,...,z n}、Y axis coordinate set Y={y 1 ,y 2 ,y 3 ,...,y n} and the X-axis coordinate set X = {x 1 ,x 2 ,x 3 ,...,x n}, normalize the three coordinate sets respectively, map them to the interval [0,1] respectively, and obtain the three normalized coordinate sets;

[0115] According to the three normalized coordinate sets, the terrain color band in the Matplotlib data visualization library is used to perform color mapping on the 3D point cloud of the cow in three dimensions, and 3D color point clouds corresponding to the three views of the cow, namely the side view, top view and rear view, are generated respectively;

[0116] The three sets of 3D color point clouds were visualized using the Open3D physical lighting engine, and the initial two-dimensional images of the cow's side, top and rear views were extracted;

[0117] The initial two-dimensional images of the three views of the cow are cropped and sharpened to obtain the three-view images corresponding to the 3D point cloud of the cow;

[0118] The three-view images corresponding to all the cattle 3D point clouds are saved together as a three-view image set;

[0119] The method for estimating the normal vector of the 3D point cloud of the cattle based on the local surface fitting method includes:

[0120] A point in the 3D point cloud of the cattle is selected, and a hybrid search strategy is used to search for a number of points within a spherical range of a preset radius around the selected point, to construct a local neighborhood of the selected point and to fit a plane using the local neighborhood;

[0121] Construct the following problem and solve it:

[0122]

[0123] sA 2 +B 2 +C 2 =1

[0124] Where A, B, C and D are the first, second, third and fourth parameters of the fitting plane respectively;

[0125] Save the solution results of A, B and C as the normal vector of the cow's 3D point cloud Specifically:

[0126]

[0127] Among them, n x 、n y and n z They are normal vectors The three-dimensional coordinates of 0 , B 0 and C 0 ) are the solutions for the positive and negative terms of A, B and C respectively; T represents the transpose of the matrix;

[0128] For the three coordinate sets, the following normalization processing is performed respectively:

[0129]

[0130] Among them, x i,norm ,y i,norm and z i,norm are the normalized X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively;max ,y max and z max are the maximum values ​​of the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively; min ,y min and z min are the minimum values ​​of the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively;

[0131] In step S2, the training set and the validation set are labeled with key points using the Labelme data labeling tool;

[0132] In step S3, the key point recognition model constructed based on the deep learning algorithm is specifically a YOLOv8-pose neural network model;

[0133] In step S5, the two-dimensional plane coordinates of all key points are converted into three-dimensional space coordinates using a preset different-dimensional key point coordinate conversion algorithm:

[0134] Obtain the size of the two-dimensional image for key point recognition in the test set; construct an AABB bounding box of the 3D point cloud of the cow corresponding to the two-dimensional image, and obtain the coordinates of eight vertices of the AABB bounding box;

[0135] Detect whether the pixel color of each key point identified in step S3 is white. If so, the key point is located outside the cow body, and the two-dimensional plane coordinates of the key point are migrated to the nearest color pixel for position correction; otherwise, the key point is located inside the cow body, and the next key point is detected until all key points are detected;

[0136] According to the size of the two-dimensional image, the vertex coordinates of the AABB bounding box, and the two-dimensional plane coordinates of all key points after detection, the plane approximate coordinates of each key point corresponding to two coordinate axes in the three-dimensional space are calculated, and the local area of ​​interest of each key point in the 3D point cloud of the cow is obtained according to the plane approximate coordinates; the local area of ​​interest includes a number of point cloud slices distributed along the third coordinate axis;

[0137] The size of the two-dimensional image is recorded as a×b, where a and b are the length and width of the two-dimensional image respectively; the two-dimensional plane coordinates of the key points after detection are recorded as (p, q);

[0138] Starting from the vertex of the upper right corner of the upper surface of the AABB bounding box, the coordinates of the four vertices on the upper surface of the AABB bounding box are recorded as (x 1 ,y 1 ,z 1 )、(x 2 ,y 2 ,z2 )、(x 3 ,y 3 ,z 3 ) and (x 4 ,y 4 ,z 4 ), the vertex coordinates corresponding to the lower surface of the AABB bounding box are recorded as (x 5 ,y 5 ,z 5 )、(x 6 ,y 6 ,z 6 )、(x 7 ,y 7 ,z 7 ) and (x 8 ,y 8 ,z 8 ); the size of the AABB bounding box is l×w×h, where l, w and h are the length, width and height of the AABB bounding box respectively;

[0139] The three-dimensional space coordinates of the mapped key points are marked as (x 0 ,y 0 ,z 0 ),but:

[0140] If the 2D image is a side view, calculate the approximate coordinates of the key point (p, q) on the X-axis and Y-axis planes according to the following formula:

[0141]

[0142] At this time, the obtained point cloud slices are distributed along the Z axis. 0 The value of is determined by the coordinates of the center point of the point cloud slice closest to the AABB bounding box interface;

[0143] If the 2D image is a top view, calculate the approximate plane coordinates of the key point (p, q) on the X-axis and Z-axis according to the following formula;

[0144]

[0145] At this time, the obtained point cloud slices are distributed along the Y axis, y 0 The value of is determined by the coordinates of the center point of the point cloud slice closest to the AABB bounding box interface;

[0146] If the 2D image is a rear view, calculate the approximate plane coordinates of the key point (p, q) on the Y and Z axes according to the following formula;

[0147]

[0148] At this time, the obtained point cloud slices are distributed along the X axis.0 The value of is determined by the coordinates of the center point of the point cloud slice closest to the AABB bounding box interface;

[0149] The center point of the point cloud slice closest to the AABB bounding box interface is used as the key point after mapping, and the three-dimensional space coordinates of the center point are obtained and used as the three-dimensional space coordinates of the key point after mapping;

[0150] In step S5, the body size parameters of the cattle include: body oblique length, chest depth, hip height, rump length, body height, body straight length, abdominal circumference, cross height, chest circumference, chest width, waist angle width, hip width and ischial width.

[0151] In the specific implementation process, Figure 2 As shown in the architecture diagram, first collect the 3D point cloud of the cow and build the initial point cloud dataset, such as Figure 3 As shown, Figure 3 (a) and (b) are examples of 3D point clouds of dairy buffalo and dairy cow, respectively;

[0152] After collecting the point cloud data set, a spatial point colorization model is constructed. The spatial color mapping uses the spatial distribution characteristics of the reconstructed 3D point cloud to map specific colors to the 3D point cloud, covering the original body color and performing open3d lighting physical model rendering based on this. After determining the overall color characteristics of the point cloud, a two-dimensional view of the point cloud is obtained, and the two-dimensional view is sharpened to highlight local image details. In this embodiment, the model is used to convert the initial point cloud data set into a training data set for the neural network model. In this embodiment, the initial point cloud data set is input into a pre-constructed spatial point colorization model for processing, and a three-view picture set is obtained and divided into a training set, a validation set, and a test set according to a ratio of 7:2:1.

[0153] Specifically, in this embodiment, if Figure 4 The figure shows the data processing flow of the spatial point colorization model. The following operations are performed for each cow 3D point cloud:

[0154] The 3D point cloud of the cow is recorded as point set P, and the 3D spatial coordinate set of point set P is expressed as: P = {(x 1 ,y 1 ,z 1 ),(x 2 ,y 2 ,z 2 ),..,(x n ,y n ,z n )}, where n is the number of points in the 3D point cloud of the cow, (x i ,y i ,z i ) represents the 3D spatial coordinates of the i-th point in the 3D point cloud;

[0155] Adjust the direction of the 3D point cloud of the cow so that the head direction of the cow, the longitudinal direction of the cow and the transverse direction of the cow are parallel to the X-axis, Y-axis and Z-axis directions respectively;

[0156] The normal vectors of the three view planes of the cow's 3D point cloud, namely the side view, top view and rear view, are estimated based on the local surface fitting method. This method fits a plane based on the points in the neighborhood of a certain point, and the normal direction of the plane is the normal vector of the point.

[0157] The steps of constructing the normal vector in this embodiment are:

[0158] A point in the 3D point cloud of the cattle is selected, and a hybrid search strategy is used to search for up to 30 points in a spherical range with a radius of 0.1 around the selected point, to construct a local neighborhood of the selected point and fit a plane using the local neighborhood;

[0159] Construct the following problem and solve it:

[0160]

[0161] sA 2 +B 2 +C 2 =1

[0162] Where A, B, C and D are the first, second, third and fourth parameters of the fitting plane respectively;

[0163] Save the solution results of A, B and C as the normal vector of the cow's 3D point cloud Specifically:

[0164]

[0165] Among them, n x 、n y and n z They are normal vectors The three-dimensional coordinates of 0 , B 0 and C 0 ) are the solutions for the positive and negative terms of A, B and C respectively; T represents the transpose of the matrix;

[0166] Color mapping of the 3D point cloud of the cow: For the three views of the cow, the top view and the rear view, the Z-axis coordinate set Z = {z 1 ,z 2 ,z 3 ,...,z n}、Y axis coordinate set Y={y 1 ,y 2 ,y 3 ,...,y n} and the X-axis coordinate set X = {x 1 ,x 2 ,x 3 ,...,x n}, normalize the three coordinate sets respectively, map them to the interval [0,1] respectively, and obtain the three normalized coordinate sets;

[0167] For the three coordinate sets, the following normalization processing is performed respectively:

[0168]

[0169] Among them, x i,norm ,y i,norm and z i,norm are the normalized X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively; max ,y max and z max are the maximum values ​​of the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively; min ,y min and z min are the minimum values ​​of the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively;

[0170] According to the three normalized coordinate sets, the terrain color band in the Matplotlib data visualization library is used to perform color mapping on the 3D point cloud of the cow in three dimensions, and 3D color point clouds corresponding to the three views of the cow, namely the side view, top view and rear view, are generated respectively;

[0171] Matplotlib is a widely used data visualization library in Python. It provides a rich variety of color bands (colormaps) for generating high-quality charts. Color bands are arrays used to represent the mapping relationship between data values ​​and colors. There are five major categories of color bands, which are suitable for different digital scenarios. Sequential color bands are suitable for scenarios where data values ​​are sequential and change continuously from low to high or from high to low; discrete color bands are suitable for scenarios where data has no order and only represents different categories or discrete values; divergent color bands are suitable for scenarios where data has a center point and colors diverge from the center point to both sides; cyclic color bands are suitable for scenarios where data changes periodically; miscellaneous color bands are suitable for user-defined scenarios. Therefore, combined with the distribution characteristics of the cow point cloud, this method uses the "terrain_r" color band, "ter "Rain" is a special color band, usually used to represent terrain and altitude data. This color band simulates the color changes of natural terrain, from blue and green to low altitude areas, and brown and white to high altitude areas, which is similar to the color distribution scheme of topographic maps. This color band combines the point cloud of the concave and convex changes on the surface of the cow to make the color distribution of the cow point cloud and the concave and convex changes of the point cloud consistent, enhancing the visualization of the data and making it easier for researchers to quickly identify and analyze the surface features of the cow based on color differences. In this method, in order to achieve the best visual effect, the "terrain" color band is inverted, resulting in the bluer color of the point cloud at higher altitudes and the whiter color of the point cloud at lower altitudes. This processing method causes color differences in the concave and convex parts of the cow point cloud, helping annotators to identify the joint positions of the cow and determine the positions of key points.

[0172] The three groups of 3D color point clouds are visualized separately using the physical lighting engine of Open3D, so that the point cloud of the cow is closer to the physical representation, and the initial two-dimensional images of the cow's side view, top view and rear view are extracted according to three orthogonal perspectives; in this embodiment, the orthogonal side view, top view and rear view of the cow's point cloud are captured by setting the "camera orientation" of the physical lighting engine;

[0173] The initial two-dimensional images of the three views of the cow are cropped and sharpened to obtain the three-view images corresponding to the 3D point cloud of the cow;

[0174] The purpose of cropping is to crop the redundant background pixels so that the edge of the cow's body is close to the edge of the image, so as to achieve the same encirclement effect as the point cloud AABB bounding box, and to do preliminary work for the coordinate conversion of points in different dimensional space; the main process is to convert the image into a grayscale image, search for non-white pixels in the image, and intercept the minimum area of ​​the cow's color image based on the boundary coordinates of the non-white pixels, and save the intercepted image;

[0175] The purpose of image sharpening is to enhance its contrast. Let the filter matrix K be:

[0176]

[0177] This filter is an edge detection filter that can detect edges in the diagonal direction. It will respond to changes along a specific diagonal direction. Applying this filter in the image processing process can detect and extract edge features in the cow image, making the contours of various parts of the cow clearer. Assume that the image matrix I is (i, j), which represents the pixel value of the image at position (i, j), and (m, n) is the coordinate convolution result of the output matrix O. The O matrix can be expressed as:

[0178]

[0179] Then, all the three-view images corresponding to the 3D point clouds of the cattle are saved together as a three-view image set and the data set is further divided. After the division is completed, it needs to be labeled before it can be used for neural network model training;

[0180] This embodiment completes the data annotation work with the assistance of the key point heat map combined with statistical laws, uses the statistical laws of the 3D point cloud of the cattle to predict the key point area of ​​the cattle body size, combines the heterodimensional key point coordinate conversion method to map the 3D key point area to the 2D image, and generates a "key point area heat map" with the mapped two-dimensional coordinate information to provide a reference for data annotation work; specifically, the cattle point cloud is horizontally sliced ​​along the point cloud space, and the distance between the slices is 1 cm. The number of point clouds in each slice is counted, such as Figure 5 As shown in the figure, the possible areas where the key points of measurement such as body height, oblique length, chest circumference, and abdominal circumference of the cow are located are located according to the difference in the number of point clouds of the front and back slices, and the possible areas where the key points of the 3D point cloud exist are statistically analyzed (for Figure 5 The three vertical lines in the lower right figure) are converted into the two-dimensional image key point coordinates by combining the heterodimensional key point coordinate conversion algorithm proposed in this method. Finally, the key point coordinates of the two-dimensional image are used to form a thermal auxiliary data annotation map, such as Figure 6 As shown, in Figure 6 In the right figure, the three vertical lines indicate where the key measurement points may be located;

[0181] Although the generated heat map can provide information tips for the labeled area, it does not represent the actual manual labeling, but only provides a reference for the location of the data labeling. This is because the key point positioning algorithm based on the statistical point cloud itself also has the problem of insufficient robustness, which will cause the key point area positioning deviation. At this time, it is necessary to manually correct the labeling position in the data labeling stage. In this embodiment, with the assistance of the heat map, the Labelme data labeling tool is finally used according to Figure 7 Label the key points of the training set and validation set; English and numbers represent the codes of the key points;

[0182] Then, a key point recognition model is constructed based on a deep learning algorithm. In this embodiment, the key point recognition model is specifically a YOLOv8-pose neural network model.

[0183] The YOLOv8-pose neural network model is trained using the training set after key point annotation, and the trained YOLOv8-pose neural network model is verified using the verification set after key point annotation to obtain the YOLOv8-pose neural network model with the best performance.

[0184] Then the test set is input into the YOLOv8-pose neural network model with the best performance to identify key points and obtain the two-dimensional plane coordinates of several key points;

[0185] The two-dimensional plane coordinates of all key points are converted into three-dimensional space coordinates using a preset different-dimensional key point coordinate conversion algorithm;

[0186] Specifically, in this embodiment, the size of the two-dimensional image for key point recognition in the test set is obtained; the AABB bounding box of the 3D point cloud of the cow corresponding to the two-dimensional image is constructed, and the eight vertex coordinates of the AABB bounding box are obtained; the AABB bounding box is a commonly used geometric representation method used to enclose and approximate the boundary of point cloud data, which is characterized by a cube or a cuboid whose six faces are parallel to the point cloud coordinate axis, so it is called an axis-aligned bounding box, and its boundary is aligned with the minimum and maximum coordinate values ​​of the point cloud data;

[0187] The size of the two-dimensional image is recorded as a×b, where a and b are the length and width of the two-dimensional image respectively; the two-dimensional plane coordinates of the key points after detection are recorded as (p, q);

[0188] like Figure 8 As shown, starting from the vertex at the upper right corner of the upper surface of the AABB bounding box, the coordinates of the four vertices on the upper surface of the AABB bounding box are recorded as (x 1 ,y 1 ,z 1 )、(x 2 ,y 2 ,z 2 )、(x 3 ,y 3 ,z 3 ) and (x 4 ,y 4 ,z 4 ), and record the vertex coordinates corresponding to the lower surface of the AABB bounding box as (x 5 ,y 5 ,z 5 )、(x 6 ,y 6 ,z6 )、(x 7 ,y 7 ,z 7 ) and (x 8 ,y 8 ,z 8 ); The size of the AABB bounding box is l×w×h, where l, w, and h are the length, width, and height of the AABB bounding box, respectively;

[0189] Because deep learning directly participates in reasoning, and the correct positions of many key points are at the edge of the cow's body, the key points may be mistakenly located outside the cow's body. However, the colorization technology makes each pixel of the cow's body in the two-dimensional image have different colors, and compared with the pure white background, it can detect whether the key point is located inside the cow's body. This method detects whether the pixel color of each key point recognized by the YOLOv8-pose neural network model with the best performance is white. If so, the key point is outside the cow's body, and the two-dimensional plane coordinates of the key point are migrated to the nearest color pixel for position correction; otherwise, the key point is inside the cow's body, and the next key point is detected until all key points are detected.

[0190] According to the size of the two-dimensional image, the vertex coordinates of the AABB bounding box, and the two-dimensional plane coordinates of all key points after detection, the plane approximate coordinates of each key point corresponding to two coordinate axes in the three-dimensional space are calculated, and the local area of ​​interest of each key point in the 3D point cloud of the cow is obtained according to the plane approximate coordinates; the local area of ​​interest includes a number of point cloud slices distributed along the third coordinate axis;

[0191] The three-dimensional space coordinates of the mapped key points are marked as (x 0 ,y 0 ,z 0 ),but:

[0192] If the 2D image is a side view, calculate the approximate coordinates of the key point (p, q) on the X-axis and Y-axis planes according to the following formula:

[0193]

[0194] At this time, the obtained point cloud slices are distributed along the Z axis. 0 The value of is determined by the coordinates of the center point of the point cloud slice closest to the AABB bounding box interface;

[0195] If the 2D image is a top view, calculate the approximate plane coordinates of the key point (p, q) on the X-axis and Z-axis according to the following formula;

[0196]

[0197] At this time, the obtained point cloud slices are distributed along the Y axis, y 0 The value of is determined by the coordinates of the center point of the point cloud slice closest to the AABB bounding box interface;

[0198] If the 2D image is a rear view, calculate the approximate plane coordinates of the key point (p, q) on the Y and Z axes according to the following formula;

[0199]

[0200] At this time, the obtained point cloud slices are distributed along the X axis. 0 The value of is determined by the coordinates of the center point of the point cloud slice closest to the AABB bounding box interface;

[0201] like Fig. 9 The figure shows a schematic diagram of key point mapping of the top view. The left figure is a two-dimensional image after key point detection, and the right figure is a top view of the 3D point cloud. The outer solid line represents the point cloud AABB bounding box; the mapping process of the side view and the rear view is the same as that of the top view;

[0202] Taking the top view as an example, the coordinate values ​​of the key point on the X-axis and Z-axis in the point cloud space are determined by two-dimensional coordinate mapping. These coordinates further define the local region of interest (ROI) of the key point in the 3D point cloud. Thanks to the high-resolution sampling capability of the depth camera, the single pixel point extracted from the two-dimensional view by this method corresponds to a small and complex region of interest in the 3D point cloud; Fig.10 As shown in FIG. 1 , the region of interest is composed of two or more point cloud blocks, which are arranged at a certain interval in the 3D space. After determining the 3D key point, this method starts from the top view and adopts a top-down search strategy to search along the Y axis direction from the maximum Y value (i.e., the top of the point cloud bounding box) in the local region of interest. In this process, the regional discrete point cloud group closest to the direction of the mapping line is retrieved along the direction of travel, and the center point of the regional point cloud is selected as the actual coordinate of the key point. The final mapping effect is shown in FIG. Fig.11 As shown;

[0203] Finally, the body size parameters of the cattle are calculated using the three-dimensional spatial coordinates of all key points to complete the body size measurement of the cattle; Fig.12 The measurement diagram shown in the figure can divide the body size information of cattle into three types: point-to-plane distance, point-to-point distance and point-to-circle distance, including: body oblique length, chest depth, hip height, buttocks length, body height, body straight length, abdominal circumference, cross height, chest circumference, chest width, waist angle width, hip width and ischial width; the specific definitions of each body size parameter are shown in Table 1:

[0204] Table 1 Definition of cattle body measurement indicators

[0205] Body size name Code explain body height WH Vertical distance from the highest point of the withers to the ground Deep chest CD The vertical distance from the withers to the sternum along the posterior edge of the scapula chest circumference CC At the back of the shoulder blade, around the chest Abdominal circumference AC Length around the widest part of the abdomen Cross height HH The vertical height from the center (cross) of the two waist angles to the ground Body obliquely long BDL The straight-line distance from the front of the scapula to the rear of the ischial tuberosity on the same side Hip high H Vertical distance from ischial tuberosity to the ground Straight body DL The horizontal distance from the shoulder to the vertical line of the posterior edge of the ischial end Chest width CB Width of the chest at the widest part of the posterior border of the shoulder blade Butt Length CL The straight-line distance from the front edge of the hip angle to the rear edge of the buttocks Waist width WW The maximum width of the left and right waist angles (hip tuberosities) Hip width HW The maximum width of the left and right hips (hip joints) Width of ischial end IW Width between the outermost protuberances of the left and right ischial tuberosities

[0206] The body height, hip height, and cross height of the cow can be converted into the calculation of the distance from the point to the plane. For this type of body size parameter, first capture the ground point cloud plane, and then calculate the distance from the three-dimensional space coordinates of the key points to the ground point cloud plane;

[0207] The body measurement calculations of the oblique length, chest depth, rump length, straight length, chest width, waist angle width, hip width, and ischial width of the cattle can be converted into point-to-point distance calculations, directly using the Euclidean distance between the three-dimensional spatial coordinates of two key points;

[0208] The waist circumference and chest circumference of cattle can be converted into point-type arc-type distance calculations. For this type of body size parameters, the polar coordinate point method is used for calculation. This method takes into account that the circular curve in the rectangular coordinate system is a straight line perpendicular to the polar coordinate system. It is simpler to calculate the waist circumference in the polar coordinate system than to calculate the body size in the rectangular coordinate system. In addition, the method of converting the point set of the perimeter shape into a one-dimensional straight line is more conducive to filling the data loss caused by the void, so as to achieve a measurement effect close to the actual body size of the cattle. Because both are calculations of arc length, this repair method is applicable to the above two body size calculations.

[0209] In order to verify the practicality and effectiveness of spatial color mapping, this method also designed a key point recognition comparison experiment between the original point cloud color image and the color-mapped point cloud image. Both were trained under the key point detection network under the same YOLOv8-pose.

[0210] Visual annotation comparison experiment analysis: In order to scientifically and accurately determine the position of the key points of the cow body during data annotation, it is necessary to rely on the detailed contour information of the cow body surface; however, there are differences between the cow body point cloud without color mapping and the point cloud image after color mapping in the legs and waist corners; Fig.13 As shown in the figure, the texture details of the cow body after color mapping are richer; for example, when determining the key points near the cow's legs, the color map can accurately mark the key points, avoiding marking the points on the cow's legs, while the boundary between the cow's legs and the body is difficult to see in the original image, and the waist outline can be identified in the waist area of ​​the cow; therefore, the color mapping scheme provides convenience in the data annotation stage;

[0211] Set up a comparison experiment of annotation points: annotate the key points of the image without the color mapping model. The number of annotated cow key points is the same as that of this method. After the annotation work is completed, the data is input into the YOLOv8-pose key point detection network for training and prediction. The training process of the two models is as follows: Fig.14 shown by Fig.14It can be seen that in this method, the mAP value is relatively high from the beginning, basically maintained above 0.8, showing a high accuracy from the beginning; although there are a few slight fluctuations, the mAP value is maintained around 0.95 overall, showing extremely high stability and superiority; combined with the analysis, the results of this method are significantly better than the original color results, and a higher mAP value is almost maintained throughout the training process; in terms of training stability, the method in this paper also performs better, and its mAP value fluctuates less, which is closer to the most ideal performance;

[0212] Therefore, the model using color mapping processing in the key point annotation stage facilitates the work of the annotators and makes the data annotation process more efficient and accurate. Secondly, from the perspective of deep learning training, by combining spatial point cloud color technology, the model shows advantages in the recognition of key points of the cow body.

[0213] Secondly, this example also evaluates the performance of the YOLOv8-pose network in the key point detection task; the final results show that the average precision (mAP50-95) value of target detection is 99.5%, and the average precision of key points is 96.2%;

[0214] In addition, in order to count the errors of the body size indicators of cattle and verify the repeatability and reliability of the method, this embodiment randomly selected 4 cattle in the experiment to conduct multiple repeatability error experiments; when each cattle passed through the acquisition channel, the front railing was used to control the cattle body in the acquisition channel, and the acquisition was performed every 10 seconds. Finally, the 4 cattle were photographed 10 times for point cloud data, a total of 40 point cloud data, and 120 key point detection data sets were generated using the color mapping model of this article; under the guidance of professional surveyors of the farm, the 4 cattle were also measured repeatedly using a soft ruler, and the 13 body size lengths of the 4 cattle were measured, and the average value of each body size length was taken as the true value of the body size of the cattle; the key points of the two-dimensional image of the cattle body that had been detected by key points were converted into 3D actual space point coordinates, and the actual 3D coordinates of the cattle body were used to calculate the body size of the cattle; finally, the measurement results and the actual values ​​were obtained for error analysis, which can effectively evaluate the accuracy of the method; the example of the average error of the final cattle body size data is shown in Fig.15 ; The relative error calculation of the overall body size data is shown in Table 2. From the table, it can be seen that the overall average error of body size measurement is 3.61%, and the measurement is relatively accurate;

[0215] Table 2 MAE and MAPE of body size measurement

[0216]

[0217]

[0218] This method starts from the data annotation stage of deep learning. By constructing a spatial point colorization model based on global spatial features and local image features, and a heterogeneous key point coordinate conversion algorithm based on depth-free camera intrinsic parameters, it achieves the precise positioning of key points in the 3D point cloud body size measurement of cattle, thereby improving the accuracy of body size measurement. At the same time, based on multiple precisely positioned key points, various cattle body size parameters can be accurately calculated, realizing complete parameter calculation.

[0219] Example 3

[0220] like Fig.16 As shown, this embodiment provides a cattle body size measurement system based on color point cloud multi-view key point positioning, and applies the cattle body size measurement method based on color point cloud multi-view key point positioning described in embodiment 1 or 2, including:

[0221] Point cloud acquisition unit 301: used to collect 3D point clouds of cattle and construct an initial point cloud data set;

[0222] Dataset construction unit 302: used to input the initial point cloud dataset into a pre-constructed spatial point colorization model for processing, obtain a three-view picture set and divide it into a training set, a validation set and a test set, and annotate key points of the training set and the validation set;

[0223] The three-view picture set includes a plurality of two-dimensional pictures of the cow in three views: side view, top view and rear view;

[0224] Model training unit 303: used to build a key point recognition model based on a deep learning algorithm, and train the key point recognition model using a training set after key point annotation, and verify the trained key point recognition model using a verification set after key point annotation, so as to obtain a key point recognition model with the best performance;

[0225] Key point recognition unit 304: used for inputting the test set into the key point recognition model with the best performance to perform key point recognition and obtain the two-dimensional plane coordinates of several key points;

[0226] The body size measurement unit 305 is used to convert the two-dimensional plane coordinates of all key points into three-dimensional space coordinates using a preset hetero-dimensional key point coordinate conversion algorithm; calculate the body size parameters of the cattle using the three-dimensional space coordinates of all key points to complete the body size measurement of the cattle.

[0227] In the specific implementation process, firstly, the point cloud acquisition unit 301 acquires the 3D point cloud of the cattle and constructs an initial point cloud data set;

[0228] The data set construction unit 302 inputs the initial point cloud data set into the pre-constructed spatial point colorization model for processing, obtains the three-view image set and divides it into a training set, a validation set and a test set, and annotates the key points of the training set and the validation set;

[0229] Then, the model training unit 303 constructs a key point recognition model based on the deep learning algorithm, and trains the key point recognition model using the training set after the key points are annotated, and verifies the trained key point recognition model using the verification set after the key points are annotated, so as to obtain the key point recognition model with the best performance;

[0230] Afterwards, the key point recognition unit 304 inputs the test set into the key point recognition model with the best performance to perform key point recognition and obtain the two-dimensional plane coordinates of several key points;

[0231] Finally, the body size measurement unit 305 uses a preset hetero-dimensional key point coordinate conversion algorithm to convert the two-dimensional plane coordinates of all key points into three-dimensional space coordinates; the body size parameters of the cattle are calculated using the three-dimensional space coordinates of all key points to complete the body size measurement of the cattle;

[0232] This system starts from the data annotation stage of deep learning. By constructing a spatial point colorization model based on global spatial features and local image features, as well as a heterogeneous key point coordinate conversion algorithm based on depth-free camera intrinsic parameters, it achieves the precise positioning of key points in the 3D point cloud body size measurement of cattle, thereby improving the accuracy of body size measurement. At the same time, based on multiple precisely positioned key points, it can accurately calculate various cattle body size parameters and realize complete parameter calculation.

[0233] The same or similar reference numerals correspond to the same or similar components;

[0234] The terms used in the drawings to describe positional relationships are only used for illustrative purposes and should not be construed as limiting this patent;

[0235] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for measuring cattle body size based on multi-view key point positioning of colorized point cloud, characterized in that: The following steps are involved: S1: Collect cattle 3D point clouds and construct the initial point cloud dataset; S2: inputting the initial point cloud data set into a pre-built spatial point colorization model for processing, obtaining a three-view image set and dividing it into a training set, a validation set and a test set, and annotating key points of the training set and the validation set; The three-view picture set includes a plurality of two-dimensional pictures of the cow in three views: side view, top view and rear view; S3: Build a key point recognition model based on the deep learning algorithm, and use the training set after key point annotation to train the key point recognition model, and use the verification set after key point annotation to verify the trained key point recognition model to obtain the key point recognition model with the best performance; S4: inputting the test set into the key point recognition model with the best performance to perform key point recognition, and obtaining the two-dimensional plane coordinates of several key points; S5: The two-dimensional plane coordinates of all key points are converted into three-dimensional space coordinates using a preset hetero-dimensional key point coordinate conversion algorithm; the body size parameters of the cattle are calculated using the three-dimensional space coordinates of all key points to complete the body size measurement of the cattle.

2. The method for measuring cattle body size based on multi-view key point positioning of colorized point cloud according to claim 1, characterized in that: In step S2, the initial point cloud data set is input into a pre-built spatial point colorization model for processing, and obtaining a three-view picture set includes: The pre-built spatial point colorization model performs the following operations on each cow 3D point cloud in the initial point cloud dataset: The 3D point cloud of the cow is recorded as point set P, and the 3D spatial coordinate set of point set P is expressed as: P = {(x1, y1, z1), (x2, y2, z2), .., (x n ,y n ,z n )}, where n is the number of points in the 3D point cloud of the cow, (x i ,y i ,z i ) represents the 3D spatial coordinates of the i-th point in the 3D point cloud; Adjust the direction of the 3D point cloud of the cow so that the head direction of the cow, the longitudinal direction of the cow and the transverse direction of the cow are parallel to the X-axis, Y-axis and Z-axis directions respectively; The normal vectors of the three viewing planes of the cow's 3D point cloud, namely the side view, top view and rear view, are estimated based on the local surface fitting method. Color mapping of the 3D point cloud of the cow: For the three views of the cow, the top view and the rear view, the Z-axis coordinate set Z = {z1,z2,z3,...,z n }, Y axis coordinate set Y = {y1,y2,y3,...,y n } and the X-axis coordinate set X={x1,x2,x3,...,x n }, normalize the three coordinate sets respectively, map them to the interval [0,1] respectively, and obtain the three normalized coordinate sets; According to the three normalized coordinate sets, the terrain color band in the Matplotlib data visualization library is used to perform color mapping on the 3D point cloud of the cow in three dimensions, and 3D color point clouds corresponding to the three views of the cow, namely the side view, top view and rear view, are generated respectively; The three sets of 3D color point clouds were visualized using the Open3D physical lighting engine, and the initial two-dimensional images of the cow's side, top and rear views were extracted; The initial two-dimensional images of the three views of the cow are cropped and sharpened to obtain the three-view images corresponding to the 3D point cloud of the cow; The three-view images corresponding to all the cattle 3D point clouds are saved together as a three-view image set.

3. The method for measuring cattle body size based on multi-view key point positioning of colorized point cloud according to claim 2, characterized in that: The method for estimating the normal vector of the 3D point cloud of the cattle based on the local surface fitting method includes: A point in the 3D point cloud of the cattle is selected, and a hybrid search strategy is used to search for a number of points within a spherical range of a preset radius around the selected point, to construct a local neighborhood of the selected point and to fit a plane using the local neighborhood; Construct the following problem and solve it: s.t.A 2 +B 2 +C 2 =1 Where A, B, C and D are the first, second, third and fourth parameters of the fitting plane respectively; Save the solution results of A, B and C as the normal vector of the cow's 3D point cloud Specifically: Among them, n x 、n y and n z They are normal vectors The three-dimensional coordinates of ; ±(A0, B0 and C0) are the positive and negative solution results of A, B and C respectively; T represents the transpose of the matrix.

4. The method for measuring cattle body size based on multi-view key point positioning of colorized point cloud according to claim 2, characterized in that: For the three coordinate sets, the following normalization processing is performed respectively: Among them, x i,norm ,y i,norm and z i,norm are the normalized X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively; max ,y max and z max are the maximum values ​​of the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively; min ,y min and z min are the minimum values ​​of the X-axis coordinate, Y-axis coordinate, and Z-axis coordinate of the i-th point respectively.

5. The method for measuring cattle body size based on multi-view key point positioning of colorized point cloud according to claim 1, characterized in that: In step S2, the Labelme data labeling tool is used to label the key points of the training set and the validation set.

6. The method for measuring cattle body size based on multi-view key point positioning of colorized point cloud according to claim 1, characterized in that: In step S3, the key point recognition model constructed based on the deep learning algorithm is specifically a YOLOv8-pose neural network model.

7. The method for measuring cattle body size based on multi-view key point positioning of colorized point cloud according to claim 2, characterized in that: In step S5, the two-dimensional plane coordinates of all key points are converted into three-dimensional space coordinates using a preset different-dimensional key point coordinate conversion algorithm: Obtain the size of the two-dimensional image for key point recognition in the test set; construct an AABB bounding box of the 3D point cloud of the cow corresponding to the two-dimensional image, and obtain the coordinates of eight vertices of the AABB bounding box; Detect whether the pixel color of each key point identified in step S3 is white. If so, the key point is located outside the cow body, and the two-dimensional plane coordinates of the key point are migrated to the nearest color pixel for position correction; otherwise, the key point is located inside the cow body, and the next key point is detected until all key points are detected; According to the size of the two-dimensional image, the vertex coordinates of the AABB bounding box, and the two-dimensional plane coordinates of all key points after detection, the plane approximate coordinates of each key point corresponding to two coordinate axes in the three-dimensional space are calculated, and the local area of ​​interest of each key point in the 3D point cloud of the cattle is obtained according to the plane approximate coordinates; The local area of ​​interest contains several point cloud slices distributed along the third coordinate axis. The center point of the point cloud slice closest to the AABB bounding box interface is used as the key point after mapping, and the three-dimensional space coordinates of the center point are obtained and used as the three-dimensional space coordinates of the key point after mapping.

8. The method for measuring cattle body size based on multi-view key point positioning of colorized point cloud according to claim 6, characterized in that: The size of the two-dimensional image is recorded as a×b, where a and b are the length and width of the two-dimensional image respectively; the two-dimensional plane coordinates of the key points after detection are recorded as (p, q); Starting from the vertex at the upper right corner of the upper surface of the AABB bounding box, the coordinates of the four vertices on the upper surface of the AABB bounding box are recorded as (x1, y1, z1), (x2, y2, z2), (x3, y3, z3) and (x4, y4, z4) in a clockwise direction, and the coordinates of the vertices corresponding to the lower surface of the AABB bounding box are recorded as (x5, y5, z5), (x6, y6, z6), (x7, y7, z7) and (x8, y8, z8); the size of the AABB bounding box is l×w×h, where l, w and h are the length, width and height of the AABB bounding box respectively; The three-dimensional space coordinates of the mapped key points are marked as (x0, y0, z0), then: If the 2D image is a side view, calculate the approximate coordinates of the key point (p, q) on the X-axis and Y-axis planes according to the following formula: At this time, the obtained point cloud slices are distributed along the Z axis, and the value of z0 is determined by the coordinates of the center point of the point cloud slice closest to the AABB bounding box interface; If the 2D image is a top view, calculate the approximate plane coordinates of the key point (p, q) on the X-axis and Z-axis according to the following formula; At this time, the obtained point cloud slices are distributed along the Y axis, and the value of y0 is determined by the coordinates of the center point of the point cloud slice closest to the AABB bounding box interface; If the 2D image is a rear view, calculate the approximate plane coordinates of the key point (p, q) on the Y and Z axes according to the following formula; At this time, the obtained point cloud slices are distributed along the X axis, and the value of x0 is determined by the coordinates of the center point of the point cloud slice closest to the AABB bounding box interface.

9. A method for measuring cattle body size based on multi-view key point positioning of colorized point cloud according to any one of claims 1 to 8, characterized in that: In step S5, the body size parameters of the cattle include: body oblique length, chest depth, hip height, rump length, body height, body straight length, abdominal circumference, cross height, chest circumference, chest width, waist angle width, hip width and ischial width.

10. A cattle body size measurement system based on multi-view key point positioning of colorized point cloud, using the cattle body size measurement method based on multi-view key point positioning of colorized point cloud as claimed in any one of claims 1 to 9, characterized in that: include: Point cloud acquisition unit: used to collect 3D point clouds of cattle and construct the initial point cloud data set; A data set construction unit is used to input the initial point cloud data set into a pre-constructed spatial point colorization model for processing, obtain a three-view picture set and divide it into a training set, a validation set and a test set, and annotate key points of the training set and the validation set; The three-view picture set includes a plurality of two-dimensional pictures of the cow in three views: side view, top view and rear view; Model training unit: used to build a key point recognition model based on a deep learning algorithm, and train the key point recognition model using a training set with key point annotations, and verify the trained key point recognition model using a verification set with key point annotations, to obtain a key point recognition model with the best performance; Key point recognition unit: used for inputting the test set into the key point recognition model with the best performance to perform key point recognition and obtain the two-dimensional plane coordinates of several key points; Body size measurement unit: used to convert the two-dimensional plane coordinates of all key points into three-dimensional space coordinates using a preset hetero-dimensional key point coordinate conversion algorithm; calculate the body size parameters of the cattle using the three-dimensional space coordinates of all key points to complete the body size measurement of the cattle.

Images