Single-pc-based multi-view automatic measurement method for cattle body size
By using a single PC to connect to a multi-view automatic cow body ruler measurement method, and by employing point cloud registration, background removal, color fusion, and simplification processing, the problems of low efficiency, large error, and stress in traditional cow body ruler measurement are solved, thus achieving efficient and accurate cow body ruler measurement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-29
- Publication Date
- 2026-03-27
AI Technical Summary
Existing methods for measuring bovine body size suffer from low measurement efficiency, high workload, stress on cattle, inability to obtain comprehensive point clouds, and "pseudo-synchronization" issues. In particular, when multiple PCs are connected to multiple depth cameras, it is difficult to achieve true synchronization, resulting in large data errors.
A multi-view automatic measurement method for cow body size based on a single PC is adopted. By constructing an automatic point cloud acquisition system, using regular cube box calibration, point cloud registration, background removal, noise reduction, color fusion and simplification are performed. Combined with the octree algorithm, the three-dimensional reconstruction of the cow body is realized and the body size data is calculated.
It enables efficient and accurate measurement of cattle body size, reduces measurement costs and power consumption, is easy to deploy and maintain, improves measurement accuracy and efficiency, and reduces stress on cattle.
Smart Images
Figure CN115272237B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of bovine body length measurement technology, and in particular to an automatic method for measuring bovine body length from multiple perspectives based on a single PC. Background Technology
[0002] In recent years, with the improvement of living standards in my country, the overall consumption of beef has shown an upward trend. In animal husbandry, precision farming and genetic improvement are becoming increasingly prominent. Body size parameters not only directly reflect the growth and development of cattle, but are also important indicators for cattle selection and meat quality evaluation in precision farming. As the scale of cattle breeding and farming grows larger, there is an urgent need to establish new, more efficient, accurate, and stress-free methods.
[0003] Existing technologies for measuring livestock body size parameters mainly include the following methods:
[0004] Manual measurement, in the traditional sense, involves direct contact with humans to obtain parameters such as body size and weight. In cattle, body size is mostly measured manually using instruments like calipers, measuring tapes, or by direct observation and scoring by experts. This method is very labor-intensive, easily causing stress to the cattle, affecting the quality of livestock growth, and may also result in injury to workers.
[0005] A single depth camera collects point cloud data, and then the body size is calculated based on the point cloud data from a single viewpoint. This method cannot comprehensively acquire information about the animal's body; it can only rely on data from one angle to calculate body size, which is prone to significant errors.
[0006] Multiple PCs are connected to multiple depth cameras to collect point clouds, and the point clouds from multiple perspectives are combined before body size measurement. This method can only achieve "pseudo-synchronization" through computer communication, making true synchronization difficult. Different depth cameras have time differences. However, the animal is moving during measurement, and due to these time differences, depth cameras from different perspectives may collect point clouds of the animal in different postures. Such point clouds are difficult to register and analyze. Summary of the Invention
[0007] To address the problems of low measurement efficiency, high workload, and stress on cattle caused by traditional manual measurement, as well as the inability to obtain comprehensive point clouds and "pseudo-synchronization" in traditional machine perspectives, this invention proposes a multi-view automatic measurement method for cattle body size based on a single PC. This method enables non-contact automated measurement of cattle body size and has many advantages such as low cost, low power consumption, easy deployment, easy maintenance, and easy promotion.
[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:
[0009] A single PC-based multi-view cattle body size automatic measurement method, comprising the following steps:
[0010] S1: Construct a point cloud automatic acquisition system, use a regular cubic box as a calibration object, acquire the point cloud of the cubic box, and realize system registration by splicing the point cloud of the cubic box;
[0011] S2: Use the registered point cloud automatic acquisition system to acquire initial cattle body size color point cloud data;
[0012] S3: Adopt a hybrid strategy to remove point cloud background and denoise the initial cattle body size color point cloud data, and obtain optimized cattle body size color point cloud data;
[0013] S4: For the overlapping part of the optimized cattle body size color point cloud data, the point cloud color is calculated by weighting the point cloud color of the top and side parts, the fusion of the overlapping part point cloud color is completed, and the weight coefficient is determined by the angle alpha between the normal of the point cloud on the cattle body surface and the upper part, and the angle beta between the normal and the side part;
[0014] S5: Based on the octree, the cattle body size color point cloud data obtained in step S4 is simplified to obtain the simplified cattle body size color point cloud data;
[0015] S6: According to the simplified cattle body size color point cloud data, the feature points of the cattle body point cloud are extracted, and the body size data of the cattle body is calculated.
[0016] The working principle of the present application is as follows:
[0017] The present application uses a single PC to connect five depth cameras, acquires cattle body point cloud from five different angles, completes three-dimensional reconstruction of the cattle body through registration, background removal, denoising, color fusion, simplification and structuring, and then locates the feature key points of the body size on the reconstructed three-dimensional point cloud of the cattle body, and calculates the cattle size.
[0018] Preferably, the point cloud automatic acquisition system acquisition process in S1 is as follows:
[0019] S11: Set the depth camera directly above as the reference camera, and acquire the distance from the reference camera to the ground;
[0020] S12: Set two rows of pixel points in the depth image of the reference camera, and continuously compare the distance from the points on the two rows of pixel points to the ground;
[0021] S13: When it is checked that the distance from the points on the two rows of pixel points to the ground is greater than 40cm, that is, when the cattle pass through the two lines at the same time, color point cloud acquisition is triggered;
[0022] S14: Acquire the depth image, generate a color image corresponding to the depth image, and generate a point cloud corresponding to the depth image, where each point in the point cloud corresponds to a pixel in the color image; finally, obtain the color point cloud P = {p1, p2, p3, ..., p...} n}, p i ={X,Y,Z,R,G,B}, where X, Y, and Z are the three-dimensional coordinates of the point, and R, G, and B are the color values of the point. The value of i ranges from 1 to n.
[0023] Preferably, the specific steps of S2 are as follows:
[0024] S21: First, using a regular cube box as a calibration object, five depth cameras acquire the point cloud of the box. Assume point p's coordinates in the first coordinate system are (x1, y1, z1), and its coordinates in the second coordinate system are (x2, y2, z2). Then the relationship between the two coordinate systems is:
[0025]
[0026] In the formula, R is the spatial transformation matrix, [t x t y t z ] T It is a translation vector;
[0027] S22: Since the color of the cube calibration object is different from the color of the surrounding environment, the point cloud of the calibration object is extracted using a color-based region growing segmentation method;
[0028] S23: After extracting the point cloud of the calibration object, use the random sampling consensus algorithm to obtain the surfaces of the calibration object that are directly above, to the left, and to the right of the three depth cameras.
[0029] S24: Based on the surface of the found calibration object, determine the four corner points on the surface as key points; the point farthest from the origin is the required key point; the four points of the depth camera directly to the left are the top left corner A. l B in the upper right corner l bottom right corner C l D in the bottom left corner l The four points of the depth camera on the right are the top left corner A. r B in the upper right corner r bottom right corner C r D in the bottom left corner r The four points of the depth camera directly above are the top left corner A. t B in the upper right corner t bottom right corner C t D in the bottom left corner t ;
[0030] First, find a common point, such as A.l and A t , B r and B t , translation matrix:
[0031]
[0032] point cloud of the upward depth camera plus translation matrix, A l and A t become the same point, straight line A l B l and A t D t form an angle, which is decomposed into three angles of rotation around the XYZ three axes, angles α, β, γ respectively, then the corresponding transformation matrix is:
[0033]
[0034]
[0035] point cloud of the upward depth camera after conversion, straight line A l B l and A t D t are collinear, ∠B t A t D l is an angle different from a right angle by θ, the point cloud of the upward depth camera is rotated by θ around straight line A l B l , the registration of the positive left point cloud and the positive merchant point cloud is completed, and the registration of the positive right point cloud is similar to the left side.
[0036] After the point cloud registration of the positive left depth camera, the positive right depth camera and the positive upward depth camera is completed, there are enough overlapping point clouds, and the ICP algorithm is used to complete the registration of the front upward point cloud and the rear upward point cloud.
[0037] S25: After the preliminary registration is completed, in order to further improve the registration reliability, the cylinder data is collected, the registration result is corrected by extracting the registration circular section, so that the point cloud registration and splicing are more accurate; all the point clouds of the cow body of different angles are uniformly registered into the coordinate system of the point cloud collected by the positive left depth camera, as a global coordinate system; wherein, the x axis is along the body length direction of the cow body, the y axis is perpendicular to the ground, and the z axis is along the body width direction of the cow body.
[0038] Further, the specific steps of S22 are as follows:
[0039] S221: Sort in ascending order according to the color RGB value;
[0040] S222: Select the initial seed point p with the lowest color RGB value, and compare the neighboring points q around the seed with the seed point;
[0041] S223: Determine whether the difference D of the color RGB value is less than 30, and the calculation formula of the difference D is:
[0042] D = |Ra-Rp| + |Bq-Bp| + |Gq-Gb|
[0043] S224: If the difference D of the color RGB value is less than 30, then the point is used as a seed point;
[0044] S225: From a certain seed, the "sub-seed" no longer appears, and this type of clustering is completed;
[0045] S226: Remove the points that have completed clustering, and continue steps S221-S225 from the remaining points until the number of remaining points is less than the set value count, and the value of count is set to 100;
[0046] S227: Select the cluster closest to the color of the calibration object, which is the point cloud of the calibration object.
[0047] Further, the specific steps of S23 are as follows:
[0048] S231: Randomly select three points p1, p2, p3 in the point cloud; the three points are called inner points, and the other points are called outer points;
[0049] S232: Calculate two vectors a[3] and b[3] according to the three points
[0050] a = p2-p1, b = p3-p1
[0051] According to the two vectors, a spatial plane equation is obtained:
[0052] Ax+By+Cz+D=0
[0053] The parameters A, B, C, and D are:
[0054] A = a[1]b[2]-a[2]b[1]
[0055] B = a[2]b[0]-a[0]b[2]
[0056] C = a[0]b[1]-a[1]b[0]
[0057] D = -(A*p1 x +B*p1 y +C*p1 z )
[0058] S233: Determine whether the plane is perpendicular to the Z axis, that is, whether it is in the front of the depth camera; if so, proceed to the next step, otherwise, go to step S232;
[0059] S234: Scan all other outer points, if the outer point q satisfies the distance to this plane is less than a set value k, then the outer point is classified as a new inner point, otherwise it is classified as a new outer point; k is the minimum distance to be classified as a certain plane, set to 0.02 cm;
[0060] After obtaining all the inner points, go to step S232, if the inner points are not changed or have been iterated steps S232 to S234n times, end the iteration; the set value of n is 10;
[0061] S235: The final inner points obtained are the point cloud of the surface of the calibration object facing the depth camera.
[0062] Preferably, the mixing strategy in S3 includes: removing ground background and surrounding environment noise data by bounding box culling, removing limit bar and worker, tool point cloud noise based on back point cloud boundary, removing residual environment and human noise based on distance-based clustering, removing outer points floating on the surface of the cattle by neighborhood filtering, and removing color noise caused by environmental reflection by mean filtering.
[0063] Further, the specific steps of removing limit bar and worker, tool point cloud noise based on back point cloud boundary are as follows:
[0064] S301: Project the point cloud of the depth camera directly above onto the XOY plane, i.e. z = 0;
[0065] S302: For any point p(x, y), find all points within a distance of 2r from p in the projected point cloud according to a pre-set rolling circle radius r, and record them as Q;
[0066] S303: Take any point q(x1, y1) on Q, and according to the rolling circle radius, find the centers of the two circles passing through points p and q, o1 and o2, respectively, and the calculation formula of o1 and o2 is:
[0067]
[0068]
[0069]
[0070]
[0071]
[0072] S304: For all points in Q except q, calculate the distance d1, d2 from the center o1, o2 respectively; if d1 and d2 are both greater than r, then p is a boundary point, continue to step S302 for the remaining points; if d1 and d2 are not both greater than r, then select another point in Q as q, and continue step S303; repeat this step until there are no remaining points;
[0073] S305: After obtaining the boundary points, the point cloud of the overhead depth camera is registered to the global coordinate system (X, Y, Z) by the overhead depth camera coordinate system (X1, Y1, Z1), and the coordinate transformation relationship is:
[0074] (X, Y, Z) = (X1, -Z1, Y1)
[0075] Remove points greater than max(X) or less than min(X) or greater than max(Z) or less than min(Z);
[0076] The specific steps of removing the points floating on the surface of the cattle by neighborhood filtering are as follows: a threshold std_mul which is a multiple of standard deviation is specified, the neighborhood of each point is statistically analyzed, the average distance of all neighboring points is calculated, if the obtained distribution is Gaussian distribution, a mean μ and a standard deviation σ are calculated, then all points in the neighborhood point set whose distance from the neighborhood is greater than μ+std_mul*σ are regarded as outliers, and are removed from the point cloud data;
[0077] The specific steps of removing color noise caused by environmental reflection by mean filtering are as follows: a mean area is set, a filtering radius R is defined, the larger the radius, the more blurred, the center point is selected, the average value of the color of all points in the area of the point is calculated, and the average value is used to replace the color of the point.
[0078] Preferably, the specific steps of the fusion of the overlapping point cloud color in S4 are as follows:
[0079] The overlapping part of the point cloud color is calculated by weighting the top and side point cloud colors, the top point cloud color is R1, G1, B1, the side point cloud color is R2, G2, B2, the color value of the new point after calculation is R, G, B, and the weight coefficient is determined by the angle α between the normal vector of the point cloud on the animal body surface and the y axis and the angle β between the corresponding axis of the side depth camera; the calculation formula is:
[0080] R = R1cos 2 α + R2cos 2 β
[0081] G = G1cos 2 α + G2cos 2 β
[0082] B = B1cos 2 a + B2cos 2 b
[0083] After the color is calculated, the coordinates of the overlapping points are assigned to the new points, and the new points replace the two overlapping points, completing the color fusion of the point cloud; the above top point cloud is the point cloud of the depth camera directly above, and the side point cloud is the point cloud of the depth camera directly to the right, directly to the left, front and above, and behind and above.
[0084] The point cloud simplification and structuring in S5 are specifically as follows:
[0085] S51: Set a minimum size m of a cube, find the maximum size of the point cloud, and establish a first cube with the size;
[0086] S52: Set an array for the cube, and put the points in the cube into the array;
[0087] S53: If the edge length of the cube is greater than the minimum size m, continue to subdivide it into eight equal parts, and then distribute all the points of the cube to the eight child cubes, and set an array for each child cube, and put the points in the child cube into the corresponding array; if there is no point in the child cube, delete the corresponding array;
[0088] S54: Repeat S52 until the edge length of the bottom layer cube is less than or equal to the minimum size m;
[0089] S55: Traverse all the arrays, and each array represents a minimum size cube, and replace all the points in the cube with the center point of the cube, and the coordinates X, Y and Z of the center point are the average values of the coordinates of all the points in the cube, that is,
[0090]
[0091] In the formula, n is the total number of all the points in the cube;
[0092] The color of the center point is the color of the nearest point to the center point and the average color of all the points in the box, and the calculation method and the coordinates are similar.
[0093] The feature points of the point cloud of the cow body extracted in S6 are mainly the body height points; the specific steps of extraction are as follows:
[0094] Project the back point cloud of the cow body to the XOZ plane, extract the boundary contour line of the cow, form a point cloud H, search for the spatial neighbor points of each point in the point set H using the K-neighbor search method, and compare the Z values of the current point and each point in the neighborhood; when the Z value of a point in the point cloud H is less than the Z values of all its spatial neighbor points, the point is the minimum value point of the contour point cloud along the Z axis direction; the first extreme point obtained by the above method is the body height point;
[0095] The specific steps for calculating the body size of the cow are as follows: performing least square fitting on the ground point cloud to obtain a plane equation of the ground, substituting the body height point into a distance formula of a point to a plane equation, and calculating the body height.
[0096] Compared with the prior art, the method has the following beneficial effects:
[0097] The method can effectively solve the problems in the conventional manual measurement, such as low measurement efficiency, heavy workload, stress caused to the cow, and the problems in the conventional machine vision, such as the inability to obtain comprehensive point cloud and pseudo-synchronization, and effectively improve the accuracy and efficiency of the measurement of the body size of the cow. BRIEF DESCRIPTION OF DRAWINGS
[0098] Figure 1 A step flowchart of the method for automatically measuring the body size of a cow based on a single PC and multiple perspectives is shown.
[0099] Figure 2 A position diagram of five depth cameras is shown.
[0100] Figure 3 A 3D coordinate system diagram of the depth cameras is shown.
[0101] Figure 4 A denoising effect diagram of the point cloud background removal and denoising using a hybrid strategy is shown.
[0102] Figure 5 A color fusion effect diagram of the overlapping point cloud is shown.
[0103] Figure 6 A logic diagram of the octree algorithm is shown.
[0104] Figure 7 A diagram showing the influence of different minimum sizes on the point cloud of the cow is shown. DETAILED DESCRIPTION
[0105] The application will be described in detail below with reference to the drawings and specific embodiments.
[0106] Embodiment 1
[0107] In this embodiment, as shown in the accompanying drawings, a method for automatically measuring the body size of a cow based on a single PC and multiple perspectives includes the following steps: Figure 1
[0108] S1: Construct a point cloud automatic acquisition system, use a regular cubic box as a calibration object, acquire the point cloud of the cubic box, and realize system registration by splicing the point cloud of the cubic box.
[0109] S2: automatically collecting initial cow body size color point cloud data by using the registered point cloud automatic collection system;
[0110] S3: removing point cloud background and denoising the initial cow body size color point cloud data by using a hybrid strategy to obtain optimized cow body size color point cloud data;
[0111] S4: for the overlapping part in the optimized cow body size color point cloud data, the point cloud color is calculated by weighting the point cloud color of the top and side parts, the fusion of the overlapping part point cloud color is completed, and the weight coefficient is determined by the angle alpha between the normal of the point cloud on the cow body surface and the upper part and the angle beta between the normal and the side part;
[0112] S5: based on the octree, the cow body size color point cloud data obtained in step S4 is simplified to obtain the simplified cow body size color point cloud data;
[0113] S6: extracting the feature points of the cow body point cloud according to the simplified cow body size color point cloud data, and calculating the body size data of the cow body.
[0114] The working principle of the present application is as follows:
[0115] The present application uses a single PC to connect five depth cameras to collect cow body point cloud from five different angles, and through registration, background removal, denoising, color fusion, simplification and structuring operations, the three-dimensional reconstruction of the cow body is completed, and then the feature key points of the body size are positioned on the reconstructed three-dimensional point cloud of the cow body, and the cow size is calculated.
[0116] In this embodiment, as shown in Figure 2 、 Figure 3 Before constructing the point cloud automatic collection system, a limiting collection device is built; the limiting collection device is divided into a mobile support and a device; the device comprises a PC and five depth cameras; the five depth cameras are fixed by using the mobile support, and then the five depth cameras are connected to a single PC; the positions of the depth cameras are: directly above, directly to the right, directly to the left, front and above, and rear and above; each camera has its own coordinate system, and the depth camera can be selected as an Azure Kinect depth camera.
[0117] In this embodiment, the point cloud automatic collection system in S1 collects the process as follows:
[0118] S11: setting the depth camera directly above as a reference camera to obtain the distance from the reference camera to the ground;
[0119] S12: setting two rows of pixel points in the depth image of the reference camera, and constantly comparing the distance from the points on the two rows of pixel points to the ground;
[0120] S13: When it is detected that the distance from the point to the ground of two rows of pixels is greater than 40cm, that is, when the cow passes through these two lines at the same time, the color point cloud acquisition is triggered.
[0121] S14: Acquire the depth image, generate a color image corresponding to the depth image, and generate a point cloud corresponding to the depth image, where each point in the point cloud corresponds to a pixel in the color image; finally, obtain the color point cloud P = {p1, p2, p3, ..., p...} n}, p i ={X,Y,Z,R,G,B}, where X, Y, and Z are the three-dimensional coordinates of the point, and R, G, and B are the color values of the point. The value of i ranges from 1 to n.
[0122] In this embodiment, the specific steps of S2 are as follows:
[0123] S21: First, using a regular cube box as a calibration object, five depth cameras acquire the point cloud of the box. Assume point p's coordinates in the first coordinate system are (x1, y1, z1), and its coordinates in the second coordinate system are (x2, y2, z2). Then the relationship between the two coordinate systems is:
[0124]
[0125] In the formula, R is the spatial transformation matrix, [t x t y t z ] T It is a translation vector;
[0126] S22: Since the color of the cube calibration object is different from the color of the surrounding environment, the point cloud of the calibration object is extracted using a color-based region growing segmentation method;
[0127] S23: After extracting the point cloud of the calibration object, use the random sampling consensus algorithm to obtain the surfaces of the calibration object that are directly above, to the left, and to the right of the three depth cameras.
[0128] S24: Based on the surface of the found calibration object, determine the four corner points on the surface as key points; the point farthest from the origin is the required key point; the four points of the depth camera directly to the left are the top left corner A. l B in the upper right corner l bottom right corner C l D in the bottom left corner l The four points of the depth camera on the right are the top left corner A. r B in the upper right corner r bottom right corner C r D in the bottom left corner r The four points of the depth camera directly above are the top left corner A. t B in the upper right corner t, right lower corner C t , left lower corner D t ;
[0129] First find a common point, such as A l and A t , B r and B t , translation matrix:
[0130]
[0131] The point cloud of the upward depth camera plus the translation matrix, A l and A t become the same point, the straight line A l B l and A t D t form an angle, which is decomposed into three angles of rotation around the XYZ three axes, angles α, β, γ, respectively, then the corresponding transformation matrix:
[0132]
[0133]
[0134] After conversion, the point cloud of the upward depth camera, the straight line A l B l and A t D t are collinear, and ∠B t A t D l is an angle of θ from a right angle, and the point cloud of the upper depth camera is rotated by θ around the straight line A l B l , completing the registration of the front-left point cloud and the front-up point cloud. The registration of the front-right point cloud is similar to the left side.
[0135] After the point cloud registration of the front-left depth camera, the front-right depth camera, and the front-up depth camera is completed, there are enough overlapping point clouds, and the ICP algorithm is used to complete the registration of the front-up and back-up point clouds.
[0136] S25: After the preliminary registration is completed, in order to further improve the registration reliability, the cylinder data is collected, the registration result is corrected by extracting the registration circular cross section, so that the point cloud registration and splicing are more accurate; all the view angle of the cow body point cloud is uniformly registered in the coordinate system of the point cloud collected by the front-left depth camera as the global coordinate system; wherein, the x-axis is along the body length direction of the cow body, the y-axis is perpendicular to the ground, and the z-axis is along the body width direction of the cow body.
[0137] More specifically, the specific steps of S22 are as follows:
[0138] S221: Sort the color RGB values in ascending order;
[0139] S222: Select the color RGB value with the lowest value as the initial seed point p, and compare the neighboring points q around the seed point with the seed point;
[0140] S223: Determine whether the difference D of the color RGB value is less than 30, where the value of the difference D can be adjusted according to the actual situation, and the calculation formula of the difference D is:
[0141] D = |Rq-Rp| + |Bq-Bp| + |Gq-Gb|
[0142] S224: If the difference D of the color RGB value is less than 30, then the point is used as a seed point;
[0143] S225: Starting from a certain seed, the "sub-seed" no longer appears, and this type of clustering is complete;
[0144] S226: Remove the points that have completed clustering, and continue steps S221-S225 from the remaining points until the number of remaining points is less than a set value count, where the value of count is set to 100; the value of count can be adjusted according to the size of the point cloud, and the value of count can be appropriately reduced for a small point cloud and appropriately increased for a large point cloud;
[0145] S227: Select the cluster closest to the color of the calibration object, which is the point cloud of the calibration object.
[0146] More specifically, the specific steps of S23 are as follows:
[0147] S231: Randomly select three points p1, p2, p3 in the point cloud; the three points are called inner points, and the other points are called outer points;
[0148] S232: Calculate two vectors a[3] and b[3] based on the three points
[0149] a = p2-p1, b = p3-p1
[0150] According to the two vectors, a spatial plane equation is obtained:
[0151] Ax + By + Cz + D = 0
[0152] The parameters A, B, C, and D are:
[0153] A = a[1]b[2] - a[2]b[1]
[0154] B = a[2]b[0] - a[0]b[2]
[0155] C = a[0]b[1] - a[1]b[0]
[0156] D = -(A * p1 x +B * p1 y +C * p1 z )
[0157] S233: Determine whether the plane is perpendicular to the Z axis, that is, whether it is in the front of the depth camera; if so, proceed to the next step, otherwise, go to step S232;
[0158] S234: Scan all other outliers, if the outlier q satisfies the distance to this plane is less than a set value k, then the outlier is classified as a new inner point, otherwise it is classified as a new outlier; k is the minimum distance to be classified as a certain plane, set to 0.02 cm;
[0159] After obtaining all the inner points, go to step S232, if the inner points are not changed or have been iterated steps S232 to S234n times, end the iteration; the set value of n is 10, the larger n is, the higher the accuracy and the slower the efficiency;
[0160] S235: The final inner points obtained are the point cloud of the surface of the calibration object facing the depth camera.
[0161] Example 2
[0162] In this embodiment, as shown in Figure 4 , the mixed strategy in S3 includes: removing ground background and surrounding environment noise data by bounding box, removing limit bar and worker, tool point cloud noise based on back point cloud boundary, removing residual environment and human noise based on distance-based clustering, removing outliers floating on the surface of the cattle by neighborhood filtering, and removing color noise caused by environmental reflection by mean filtering.
[0163] More specifically, the steps of removing ground background and surrounding environment noise data by bounding box are as follows: before the point cloud starts to be collected, the collection range of the depth camera is set artificially, that is, minX, maxX, minY, maxY, minZ, and maxZ are set, a cuboid of appropriate size is selected to enclose the cattle point cloud, and then the point cloud outside the cuboid is removed.
[0164] The specific steps of removing limit bar and worker, tool point cloud noise based on back point cloud boundary are as follows:
[0165] S301: Project the point cloud of the depth camera directly above onto the XOY plane, that is, z = 0;
[0166] S302: For any point p(x, y), find all points within a distance of 2r from p in the projected point cloud according to a pre-set rolling circle radius r, and record them as Q.
[0167] S303: Take a point q (x1, y1) on Q, and calculate the centers o1 and o2 of the two circles passing through p and q according to the rolling circle radius, the calculation formula of o1 and o2 is:
[0168]
[0169]
[0170]
[0171]
[0172]
[0173] S304: For all points in Q except q, calculate the distances d1 and d2 from the centers o1 and o2 respectively; if both d1 and d2 are greater than r, then p is a boundary point, continue to step S302 for the remaining points; if not, select another point in Q as q and continue step S303; repeat this step until there are no remaining points;
[0174] S305: After obtaining the boundary points, the point cloud of the overhead depth camera is registered to the global coordinate system (X, Y, Z) by the overhead depth camera coordinate system (X1, Y1, Z1), and the coordinate transformation relationship is:
[0175] (X, Y, Z) = (X1, -Z1, Y1)
[0176] Remove points greater than max(X) or less than min(X) or greater than max(Z) or less than min(Z);
[0177] The specific steps of removing the points floating on the surface of the cattle by neighborhood filtering are as follows: a threshold std_mul which is a multiple of standard deviation is specified, the neighborhood of each point is statistically analyzed, the average distance of all neighboring points is calculated, if the obtained distribution is Gaussian distribution, a mean μ and a standard deviation σ are calculated, then all points in the neighborhood point set whose distance from the neighborhood is greater than μ+std_mul*σ are regarded as outliers, and are removed from the point cloud data;
[0178] The step of removing residual environmental and artificial noise based on distance-based clustering is as follows: a threshold is set, if the distance between the current scanning point and the previous scanning point is within the preset threshold range, the current point is clustered into the class of the previous scanning point; otherwise, the current scanning point is set as a new cluster seed, and the next scanning point is determined whether it belongs to the same class as the seed according to the preset threshold; the above steps are repeated until all points are clustered into different classes, and the classes with small size are filtered out.
[0179] The specific steps of removing color noise caused by environmental reflection by mean-based filtering are as follows: a mean area is set, a filtering radius R is defined, the larger the radius, the more blurred, the center point is selected, the average value of the colors of all points in the area of the point is calculated, and the average value is used to replace the color of the point.
[0180] Embodiment 3
[0181] In this embodiment, as shown in Figure 5 , the specific steps of the fusion of the color of the overlapping point cloud in S4 are as follows:
[0182] The color of the overlapping part of the point cloud is calculated by weighting the color of the top and side point clouds, the color of the top point cloud is R1, G1, B1, the color of the side point cloud is R2, G2, B2, the color value of the new point after calculation is R, G, B, and the weight coefficient is determined by the angle α between the normal vector of the point cloud on the animal surface and the y axis and the angle β between the corresponding axis of the side depth camera; the calculation formula is:
[0183] R=R1cos 2 α+R2cos 2 β
[0184] G=G1cos 2 α+G2cos 2 β
[0185] B=B1cos 2 α+B2cos 2 β
[0186] After the color is calculated, the coordinates of the overlapping points are assigned to the new point, and the new point replaces the two overlapping points, and the color fusion of the point cloud is completed; the above top point cloud is the point cloud of the top depth camera, and the side point cloud is the point cloud of the right, left, front and back depth cameras.
[0187] In this embodiment, as shown in Figure 6 , Figure 7 , the specific steps of the point cloud simplification and structuring in S5 are as follows:
[0188] S51: Set a minimum size m of a cube, find the maximum size of the point cloud, and establish a first cube with the size.
[0189] S52: Set an array for the cube, and put the points in the cube into the array;
[0190] S53: If the edge length of the cube is greater than the minimum size m, continue to subdivide it into eight equal parts, and then distribute all the points of the cube to the eight sub-cubes, set an array for each sub-cube, and put the points in the sub-cube into the corresponding array; if there is no point in the sub-cube, delete the corresponding array;
[0191] S54: Repeat S52 until the edge length of the bottom layer cube is less than or equal to the minimum size m;
[0192] S55: Traverse all the arrays, each array represents a minimum size cube, and replace all the points in the cube with the center point of the cube, the coordinates X, Y and Z of the center point are the average values of the coordinates of all the points in the cube, that is
[0193]
[0194] In the formula, n is the total number of all points in the cube;
[0195] The color of the center point is the color of the nearest point to the center point and the average color of all points in the box, and the calculation method is similar to the coordinates.
[0196] In the embodiment, the feature points of the cow body point cloud extracted in S6 are mainly body height points, and the specific steps are as follows:
[0197] The back point cloud of the cow body is projected onto the XOZ plane, the boundary contour line of the cow is extracted, a point cloud H is formed, the K-neighborhood search method is used to search the spatial neighborhood points of each point in the point set H, and the Z values of the current point and each point in the neighborhood are compared; when the Z value of a point in the point cloud H is less than the Z values of all its spatial neighborhood points, the point is the minimum value point of the contour point cloud along the Z axis direction; the first extreme value point obtained by the above method is the body height point;
[0198] The specific steps of calculating the body size of the cow are as follows: the least square fitting plane of the ground point cloud is obtained, the plane equation of the ground is obtained, the body height point is substituted into the distance formula from the point to the plane equation, and the body height is obtained.
[0199] Obviously, the above embodiments of the present application are only examples for clearly illustrating the present application, and are not intended to limit the embodiments of the present application. Any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the claims of the present application.
Claims
1. A single PC-based multi-view automatic measurement method for cattle body size, characterized in that, The method comprises the following steps: S1: constructing a point cloud automatic acquisition system, using a regular cubic box as a calibration object, acquiring the point cloud of the cubic box, and realizing system registration by splicing the point cloud of the cubic box; S2: using the registered point cloud automatic acquisition system to acquire initial cow body size color point cloud data; S3: using a hybrid strategy to remove the point cloud background and denoise the initial cow body size color point cloud data, and obtaining optimized cow body size color point cloud data; The hybrid strategy comprises: removing the ground background and surrounding environmental noise data by bounding box culling, removing the limit fence and worker and tool point cloud noise based on the back point cloud boundary, removing the residual environmental and artificial noise based on distance-based clustering, removing the out points floating on the surface of the cow based on neighborhood filtering, and removing the color noise caused by environmental reflection based on mean filtering; S4: for the overlapping part of the optimized cow body size color point cloud data, the point cloud color is calculated by weighting the point cloud color of the top and side parts, the color fusion of the overlapping part point cloud is completed, and the weight coefficient is determined by the angle α between the normal vector of the point cloud on the surface of the cow body and the upper part and the angle β between the normal vector and the side part depth camera corresponding axis; the specific steps of the color fusion of the overlapping point cloud are as follows: The overlapping part point cloud color is calculated by weighting the top and side part point cloud color, the top part point cloud color is R1, G1 and B1, the side part point cloud color is R2, G2 and B2, the color value of the new point after calculation is R, G and B, and the weight coefficient is determined by the angle α between the normal vector of the point cloud on the surface of the animal body and the y axis and the angle β between the normal vector and the side part depth camera corresponding axis; the calculation formula is: After the color is calculated, the coordinates of the overlapping points are assigned to the new point, the new point replaces the two overlapping points, and the color fusion of the point cloud is completed; the above top part point cloud is the point cloud of the uppermost depth camera, and the side part point cloud is the point cloud of the rightmost, leftmost, front uppermost and back uppermost depth cameras; S5: based on the octree, the cow body size color point cloud data obtained in step S4 is simplified to obtain simplified cow body size color point cloud data; S6: extracting the feature points of the cow body point cloud according to the simplified cow body size color point cloud data, and calculating the body size data of the cow.
2. The single PC-based multi-view automatic measurement method for cattle body size according to claim 1, characterized in that, The point cloud automatic acquisition system in S1 comprises the following steps: S11: setting the uppermost depth camera as the reference camera, and acquiring the distance from the reference camera to the ground; S12: setting two rows of pixel points in the depth image of the reference camera, and continuously comparing the distance from the points on the two rows of pixel points to the ground; S13: when it is checked that the distance from the points on the two rows of pixel points to the ground is greater than 40 cm, that is, when the cow passes through the two lines at the same time, color point cloud acquisition is triggered; S14: Obtain a depth image, generate a color image corresponding to the depth image, generate a point cloud corresponding to the depth image, each point of the point cloud corresponding to a pixel of the color image; finally obtain a color point cloud P={p1, p2, p3, …, pn}, pi={X, Y, Z, R, G, B}, i=1, 2, 3, …, n. n},p i ={X, Y, Z, R, G, B}, X, Y, Z are three-dimensional coordinates of the point, R, G, B are color values of the point, wherein the value range of i is 1~n.
3. The single PC-based multi-view automatic measurement method for cattle body size according to claim 1, characterized in that, The specific steps of S2 are as follows: S21: first, using a regular cubic box as a calibration object, five depth cameras acquire the point cloud of the box, assuming that the coordinates of point p in the first coordinate system are (x1, y1, z1), and the coordinates of the point in the second coordinate system are (x2, y2, z2), then the change relationship between the two coordinate systems is: where R is a spatial transformation matrix, is a translation vector; S22: Since the color of the cube calibration object is different from the color of the surrounding environment, the point cloud of the calibration object is extracted by using a color-based region growing segmentation method; S23: After the point cloud of the calibration object is extracted, a random sample consensus algorithm is used to obtain the faces of the calibration object directly opposite the three depth cameras; S24: find the four corners' points on the found plane as key points; the one farthest from the origin is the required key point; the four points of the front left depth camera are the upper left corner A l , the upper right corner B l , the lower right corner C l , and the lower left corner D l , the four points of the front right depth camera are the upper left corner A r , the upper right corner B r , the lower right corner C r , and the lower left corner D r , the four points of the front up depth camera are the upper left corner A t , the upper right corner B t , the lower right corner C t , and the lower left corner D t ; Find a common ground first, A l and A t , B r and B t , translation matrix: Point cloud of the top depth camera plus translation matrix, A l and A t becomes the same point, straight line A l B l and A t D t form an angle, decompose it into three angles of rotation around the XYZ three axes, angles α, β, γ respectively, then the corresponding transformation matrix: The point cloud of the upper front depth camera is rotated around the line A l B l and A t D t are collinear, and the angle between B t A t D l is The point cloud of the upper front depth camera is rotated around the line A l B l is rotated The registration of the front right point cloud is similar to the left one. After the point cloud registration of the left, right and top depth cameras is completed, there are enough overlapping point clouds, and the ICP algorithm is used to complete the registration of the front and rear point clouds; S25: After the preliminary registration is completed, in order to further improve the registration reliability, the cylinder data is collected, the registration result is corrected by extracting the cross section of the registered cylinder, so that the point cloud registration and splicing are more accurate; all the point clouds of the cow body are uniformly registered into the coordinate system of the point cloud collected by the left depth camera as the global coordinate system; wherein the x-axis is along the length direction of the cow body, the y-axis is perpendicular to the ground, and the z-axis is along the width direction of the cow body.
4. The single PC-based multi-view automatic measurement method for cattle body size according to claim 3, characterized in that, The specific steps of S22 are as follows: S221: Sort the color RGB value in ascending order; S222: Select the lowest color RGB value as the initial seed point p, and compare the adjacent point q around the seed point with the seed point; S223: Determine whether the difference D of the color RGB value is less than 30, and the calculation formula of the difference D is: S224: If the difference D of the color RGB value is less than 30, the point is used as a seed point; S225: Starting from a certain seed, the "sub-seed" no longer appears, and then this type of clustering is completed; S226: Remove the points that have completed clustering, and continue steps S221-S225 from the remaining points until the number of remaining points is less than the set value count, and the value of count is set to 100; S227: Select the cluster with the smallest difference of the calibration object color RGB value, which is the point cloud of the calibration object.
5. The single PC-based multi-view automatic measurement method for cattle body size according to claim 3, characterized in that, The specific steps of S23 are as follows: S231: Randomly select three points p1, p2, p3 in the point cloud; the three points are called inner points, and the other points are called outer points; S232: Two vectors a[3] and b[3] are obtained according to the three points; According to the two vectors, a spatial plane equation is obtained: Parameters A, B, C, D are: S233: Determine whether the plane is perpendicular to the Z-axis, that is, whether it is directly opposite the face of the depth camera; if yes, proceed to the next step, otherwise, go to step S232; S234: Scan all other outer points, if the distance from the outer point q to the plane is less than the set value k, the outer point is classified as a new inner point, otherwise it is classified as a new outer point; k is the minimum distance for classification to a certain plane, and is set to 0.02 cm; After obtaining all the inner points, go to step S232, if the inner points are not changed or have been iterated for n times from step S232 to S234, end the iteration; the set value of n is 10; S235: The finally obtained inner points are the point clouds of the face directly opposite the calibration object of the depth camera.
6. The single PC-based multi-view automatic measurement method for a cow body size according to claim 5, characterized in that, The specific steps of removing the limit bar and worker and tool point cloud noise based on the back point cloud boundary are as follows: S301: Project the point cloud of the top depth camera onto the XOY plane, that is, z=0; S302: For any point p (x, y), find all points in the projected point cloud with a distance less than 2r from p according to the pre-set rolling circle radius r, and mark them as Q; S303: Take a point on Q, denoted as q (x1, y1), according to the rolling circle radius, the centers of the two circles passing through points p and q are o1 and o2, respectively, o1 and o2 2， The calculation formula is: S304: For all points in Q except q, calculate the distances d1 and d2 from the centers o1 and o2 respectively; if both d1 and d2 are greater than r, then p is a boundary point, and continue with step S302 for the remaining points; if not, select another point in Q as q and continue with step S303; repeat this step until there are no remaining points; S305: After the boundary points are acquired, the point cloud of the directly above depth camera is fitted by the directly above depth camera coordinate system (X, Y, Z) X 1 ，Y 1 ， Z 1) After registration to the global coordinate system (X, Y, Z) X, Y, Z ), the transformation relationship of the coordinates is: Remove points greater than max (X) or less than min (X) or greater than max (Z) or less than min (Z); The specific steps of removing the points floating on the surface of the cattle through neighborhood-based filtering are as follows: a threshold std_mul is specified, which is a multiple of the standard deviation; for each point, statistical analysis is performed on its neighborhood, the average distance of all neighboring points is calculated, and if the obtained distribution is a Gaussian distribution, a mean μ and a standard deviation σ are calculated; all points in this neighborhood point set whose distance from their neighborhood is greater than μ+std_mul*σ are considered outliers and are removed from the point cloud data; The specific steps of removing color noise caused by environmental reflection through mean-based filtering are as follows: set a mean area, define a filtering radius R, and gradually select a center point; calculate the average color of all points in the area, and replace the color of this point with the average color.
7. The single PC-based multi-view automatic measurement method of a cow body size according to claim 1, characterized in that, The specific steps of point cloud simplification and structuring in S5 are as follows: S51: Set a minimum size m of a cube, find the maximum size of the point cloud, and establish the first cube with this size; S52: Set an array for the cube and put the points in the cube into this array; S53: If the edge length of the cube is greater than the minimum size m, continue to subdivide it into eight equal parts, and distribute all points of the cube to the eight sub-cubes; set an array for each sub-cube and put the points in the sub-cube into the corresponding array; if there are no points in the sub-cube, delete the corresponding array; S54: Repeat S52 until the edge length of the bottommost cube is less than or equal to the minimum size m; S55: Traverse all arrays, each array represents a minimum size cube, replace all points in the cube with the center point of the cube, and the coordinates X, Y, Z of the center point are the average values of the coordinates of all points in the cube, i.e. Where n is the total number of points in the cube; The color of the center point is the color of the nearest point to the center point and the average color of all points in the box, and the calculation method is similar to the coordinates.
8. The single PC-based multi-view automatic measurement method of a cow body size according to claim 1, characterized in that, The feature points of the cattle body point cloud extracted in S6 are mainly the body height points, and the specific steps of extraction are as follows: The back point cloud of the cow body is projected to XOZ plane, the boundary contour line of the cow is extracted, a point cloud H is constituted, a K neighborhood search method is used to search the spatial neighborhood points of each point in the point set H, and the Z values of the current point and each point in the neighborhood are compared; when the Z value of a point in the point cloud H is less than the Z values of all the spatial neighborhood points, the point is a minimum point of the contour point cloud along the Z axis direction; the first extreme point obtained by the above method is the body height point. The specific steps for calculating the body size of the cow are as follows: the ground point cloud is least square fitted to a plane, the plane equation of the ground is obtained, the body height point is substituted into the distance formula of the point to the plane equation, and the body height is obtained.