Method and device for modeling three-dimensional point cloud of live pig based on nearest point iteration and landmark point matching

Abstract

The application discloses a live pig three-dimensional point cloud modeling method and device based on nearest point iteration and landmark point matching, and the method comprises the following steps: obtaining point cloud data of a target pig body from four view angles of the upper left, the lower left, the upper right and the lower right; taking the point cloud data of the upper left and the upper right of the target pig body as source point cloud, and taking the point cloud data of the lower left and the lower right as target point cloud, and calculating the nearest neighbor point of each point in the source point cloud in the target point cloud; obtaining point cloud data from two view angles of the left and the right; identifying four circular landmark points around the live pig based on RGB information, and obtaining the center coordinates of the landmark points in the point cloud in the two view angles of the left and the right through center fitting; transforming the point cloud data of the live pig from the front and back view angles to the same coordinate system for registration, and obtaining a live pig three-dimensional point cloud model. The application is based on nearest point iteration registration, and then fuses a landmark point registration method to obtain a whole live pig three-dimensional point cloud model, improves the span of the live pig point cloud model of each angle, effectively reduces the number of three-dimensional depth cameras, and meanwhile ensures the modeling precision.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of three-dimensional modeling, and particularly relates to a live pig three-dimensional point cloud modeling method and device based on nearest point iteration and landmark point matching. BACKGROUND

[0002] The phenotype data of live pigs is an important indicator (body length, body width, body height) of live pigs, and timely and accurate acquisition of the phenotype data of live pigs can immediately evaluate the growth and health conditions of live pigs, and detect the feed absorption rate of live pigs or separately raise live pigs with different nutritional conditions to achieve the maximum utilization of feed and the best growth control. The acquisition of the phenotype data of live pigs in the traditional way often needs to directly contact the live pigs, and manually measure, the whole process is not only time-consuming and laborious, but also has a large error in manual measurement, and the contact between humans and animals can easily cause the spread of epidemic diseases; sometimes, sedatives and other drugs need to be used for assistance, which brings great pressure to live pigs, affects daily activities such as eating and mating, and even causes sudden death of pigs, causing great economic losses. Therefore, point cloud modeling of live pigs has become a major direction to solve this problem.

[0003] In the traditional application, the point cloud modeling of an object adopts ICP point cloud registration, which usually needs more shooting angles for registration, resulting in the need for more depth cameras, high requirements for the production of the shooting device, high cost, and high failure rate caused by multiple depth cameras, which are difficult to solve. Based on the ICP point cloud registration technology, the present application adds a landmark point registration method to realize accurate point cloud modeling of the target pig body only by four shooting angles, and obtain the phenotype point cloud model of the live pig. SUMMARY

[0004] The main purpose of the present application is to overcome the shortcomings and deficiencies of the prior art, and provide a live pig three-dimensional point cloud modeling method and device based on nearest point iteration and landmark point matching. The present application is based on nearest point iteration registration, and further obtains the whole three-dimensional point cloud model of the live pig by fusing the landmark point registration method, improves the span of the live pig point cloud model at each angle, solves the small span defect in the traditional nearest point registration, effectively reduces the number of three-dimensional depth cameras, and ensures the modeling accuracy.

[0005] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0006] The present application provides a live pig three-dimensional point cloud modeling method based on nearest point iteration and landmark point matching, which includes the following steps:

[0007] (1) Obtain the point cloud data of the live pig from four perspectives of the upper left, lower left, upper right and lower right, denoise the point cloud data, cut the target pig body from the background, remove the ground from the point cloud data, and obtain the point cloud data of the target pig body from four perspectives of the upper left, lower left, upper right and lower right;

[0008] (2) Take the point cloud data of the target pig body from the upper left and upper right perspectives as the source point cloud, and the point cloud data from the lower left and lower right perspectives as the target point cloud, and calculate the nearest neighbor point of each point in the source point cloud in the target point cloud;

[0009] (3) Take the point cloud data from the upper left perspective as the source point cloud, and the point cloud data from the lower left perspective as the target point cloud, convert both point clouds to the center of mass coordinate system, calculate the optimal rotation matrix and optimal translation matrix of the current target point cloud rotation to the source point cloud coordinate system based on SVD decomposition, apply the rotation matrix and translation matrix to the target point cloud to obtain the rotated and translated target point cloud, and replace the original target point cloud; calculate the rotation error and translation error of the target point cloud and the source point cloud, and determine whether the stop iteration condition is reached, otherwise repeat steps (2) to (3), and finally obtain the point cloud data of the upper left perspective and the upper right perspective in the same coordinate system; the matching of the point cloud data from the upper right perspective and the lower right perspective is the same, and finally the point cloud data from the left and right perspectives is obtained;

[0010] (4) Identify the four circular marker points around the live pig based on RGB information, and obtain the center coordinates of the marker points in the respective point clouds in the left and right perspectives through center fitting;

[0011] (5) Calculate the rotation transformation matrix required for converting the same set of marker points in the left and right perspective point cloud data to the same coordinate system based on the three-point method, and use the rotation transformation matrix to transform the point cloud data from the front and back perspectives of the live pig to the same coordinate system for registration, and obtain the three-dimensional point cloud model of the live pig.

[0012] As a preferred technical solution, in step (1), the Ransac algorithm is used to remove the ground data of the point cloud, which is as follows:

[0013] (1-1) Take the area where the target pig body is located as the center, expand k coordinates in the up, down, left and right directions as the boundary for cutting, and obtain the point cloud data without background;

[0014] (1-2) Median filter the point cloud data without background to remove discrete points;

[0015] (1-3) Randomly select three points from the point cloud data without background to form a plane, calculate the distance of the remaining points in the remaining point cloud data to the plane, and if it is less than the threshold T, it is considered to be in the same plane; if the number of points in the same plane exceeds n, save the plane and mark all points of the plane as ground point cloud for removal, and obtain the point cloud data without ground.

[0016] As a preferred technical solution, in step (2), the KD-Tree algorithm is adopted to calculate the nearest neighbor point of each point in the source point cloud in the target point cloud, specifically:

[0017] (2-1) For the comparison dimension of the KD-Tree, the three axes of XYZ are selected as the comparison dimension in turn with the progress of the depth of the KD-Tree;

[0018] (2-2) A point in the target point cloud data is randomly selected as the root node of the KD-Tree, and the KD-Tree is constructed in accordance with the rule that the left child node is smaller than the parent node and the right child node is larger than the parent node;

[0019] (2-3) A current global best point and a shortest distance are set in the target point cloud, and the initial values are the root node of the KD-Tree and infinity;

[0020] (2-4) The distance between the current node in the target point cloud and the searched point in the source point cloud is calculated, and if it is smaller than the current shortest distance, the current node is updated as the global best point and the shortest distance;

[0021] (2-5) If the division scale of the searched point in the source point cloud is smaller than the division scale of the current node in the target point cloud, the left child node of the current node is set as the new current node, otherwise the right child node of the current node is set as the new current node; if the distance between the searched point in the source point cloud and the division scale of the current node in the target point cloud is smaller than the global shortest distance, the root node of the other subtree of the current node in the target point cloud is set as the new current node; the division scale refers to the distance scale of the left and right subtrees of the searched point;

[0022] (2-6) Steps (2-4) and (2-5) are repeated until the current node is empty, and the nearest neighbor point of each point in the source point cloud in the target point cloud is obtained; the same method is used to calculate the nearest neighbor points of the right upper view and the right lower view.

[0023] As a preferred technical solution, step (3) is specifically:

[0024] (3-1) Continue to take the point cloud data of the left upper view as the source point cloud and the point cloud data of the left lower view as the target point cloud, calculate the centroids of the source point cloud and the target point cloud, and convert the source point cloud and the target point cloud to the centroid coordinate system;

[0025] Centroid:

[0026]

[0027] Convert the coordinate system:

[0028] Wherein represents the value of the coordinate point of the source point cloud and the target point cloud on each axis, represents the center of mass of the source point cloud and the target point cloud, represents the coordinate point of the source point cloud and the target point cloud, represents the coordinate point of the source point cloud and the target point cloud after the transformation of the coordinate system;

[0029] (3-2) Let singular value decomposition of H = U∑V T , get the optimal rotation of the current target point cloud to the source point cloud coordinate system: R * = VU T and the optimal translation U is the left singular vector, V is the right singular vector, R * is the current optimal rotation matrix, t * is the current optimal translation matrix;

[0030] (3-3) Let the rotation error be: the translation error is The optimal rotation matrix and the optimal translation matrix R * ,t * obtained in each iteration are applied to the current target point cloud for rotation and translation, and the rotated and translated target point cloud data is taken as the original target point cloud data. Repeat steps (2-1) to (3-3) to obtain new optimal rotation matrix R * and transformation matrix t * until the set condition is met.

[0031] (3-4) Finally, the source point cloud data and the target point cloud data are successfully registered in the same coordinate system. The matching of the upper right view point cloud data and the lower right view point cloud data is the same. Finally, the left view point cloud data and the right view point cloud data are matched successfully.

[0032] As a preferred technical solution, the set condition is met, which means that one of the following conditions is met:

[0033] The change of R * ,t * is less than a set value;

[0034] The rotation error and the translation error are less than a set value;

[0035] The number of repetitions reaches a set value.

[0036] As a preferred technical solution, in step (4), the center coordinates of the identification mark point are specifically:

[0037] (4-1) According to the RGB information, the point cloud data of each different color circular marker point around the live pig is screened from the point cloud data of the left and right view angles;

[0038] (4-2) For each circular marker point, the three-dimensional coordinate point of the center of the circle in the point cloud is calculated;

[0039]

[0040] wherein, is the coordinate value of the center of the circle, x i , y i , and z i are the coordinate values of all points in the circular marker point in the point cloud, and n is the number of points of a single circular marker point.

[0041] As a preferred technical solution, in step (5), the point cloud data of the front and rear view angles of the live pig is transformed into the same coordinate system for registration, specifically:

[0042] (5-1) The left view point cloud data is taken as the source point cloud, and the right view point cloud data is taken as the target point cloud, and the orthogonal coordinate vector of the marker center of the source point cloud and the target point cloud is calculated:

[0043]

[0044]

[0045] wherein is the orthogonal coordinate vector of the marker center of the source point cloud, is the orthogonal coordinate vector of the marker center of the target point cloud, x p and x q are the coordinate values of the marker center of the source point cloud and the target point, respectively;

[0046] (5-2) After unitizing the orthogonal coordinate vector, the coordinate system with the marker center as the origin is obtained:

[0047]

[0048] wherein and are the coordinate systems of the source point cloud and the target point cloud with the marker center as the origin, respectively;

[0049] (5-3) According to the coordinate systems of the source point cloud and the target point cloud with the marker center as the origin, the rotation matrix and the translation matrix of the target point cloud transformed to the source point cloud are calculated;

[0050] Rotation matrix:

[0051] Translation matrix: T=p-qR

[0052] (5-4) Obtain the coordinate transformation matrix according to the obtained rotation matrix R and translation matrix T:

[0053] The target point cloud is converted to the source point cloud coordinate system by the transformation formula PF=Q to realize registration of the target point cloud and the source point cloud, and finally, a three-dimensional point cloud model of a live pig is obtained, wherein F is a transformation matrix, P is a target point cloud coordinate, and Q is a source point cloud coordinate.

[0054] Another aspect of the application provides a live pig three-dimensional point cloud modeling system based on nearest point iteration and landmark point matching, comprising a data acquisition module, a nearest neighbor point calculation module, a nearest neighbor point iteration module, a landmark point identification module and a landmark point registration module.

[0055] The data acquisition module is used to acquire point cloud data of four views of the upper left, lower left, upper right and lower right of the live pig, denoises the point cloud data, cuts the target pig body from the background, removes the ground in the point cloud data, and obtains the point cloud data of the upper left, lower left, upper right and lower right of the target pig body.

[0056] The nearest neighbor point calculation module is used to take the point cloud data of the upper left and upper right views of the target pig body as source point clouds, and take the point cloud data of the lower left and lower right views as target point clouds, and calculate the nearest neighbor point of each point in the source point cloud in the target point cloud.

[0057] The nearest neighbor point iteration module is used to take the point cloud data of the upper left view as a source point cloud, and take the point cloud data of the lower left view as a target point cloud, and convert both point clouds to the center of mass coordinate system, calculate the optimal rotation matrix and the optimal translation matrix of the current target point cloud rotation to the source point cloud coordinate system based on SVD decomposition, apply the rotation matrix and the translation matrix to the target point cloud to obtain the target point cloud after rotation and translation, and replace the original target point cloud, calculate the rotation error and the translation error of the target point cloud and the source point cloud, judge whether the stop iteration condition is reached, otherwise repeat the processing process of the nearest neighbor point calculation module and the nearest neighbor point iteration module, and finally obtain the point cloud data of the upper left view and the upper right view registered in the same coordinate system; the matching of the point cloud data of the upper right view and the lower right view is the same, and finally, the point cloud data of the left and right two views is obtained.

[0058] The landmark point identification module is used to identify four circular landmark points around the live pig based on RGB information, and obtain the center coordinates of the landmark points in the respective point clouds in the left and right two views through center fitting.

[0059] The landmark point registration module is used to calculate the rotation transformation matrix required to convert the same set of landmark points in the point cloud data of different left and right perspectives into the same coordinate system based on the three-point method. Using this rotation transformation matrix, the point cloud data of the front and back perspectives of the live pig are transformed into the same coordinate system for registration to obtain a three-dimensional point cloud model of the live pig.

[0060] Another aspect of the present invention provides an electronic device, comprising:

[0061] at least one processor; and,

[0062] a memory communicatively connected to the at least one processor; wherein,

[0063] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the living pig three-dimensional point cloud modeling method based on closest point iteration and landmark point matching.

[0064] On the other hand, the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the live pig three-dimensional point cloud modeling method based on closest point iteration and landmark point matching.

[0065] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0066] (1) The present invention is based on the iterative closest point registration and then integrates the landmark point registration method to obtain the entire three-dimensional point cloud model of the live pig, thereby improving the span of the live pig point cloud model at various angles. The present invention only requires four shooting angles and a small number of depth cameras, which effectively reduces the number of three-dimensional depth cameras while ensuring modeling accuracy.

[0067] (2) The four shooting angles of the present invention have relatively low requirements, the device is easy to design, and the model is shot at four angles at the same time, and there is no requirement for the movement state of the sow. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0069] Figure 1 This is a flow chart of the live pig three-dimensional point cloud modeling method based on closest point iteration and landmark point matching of the present invention.

[0070] Fig. 2(a), 2(b) are the original point cloud images taken by the left upper and left lower view depth cameras.

[0071] Fig. 2(c), 2(d) are the original point cloud images taken by the right upper and right lower view depth cameras.

[0072] Fig. 3(a), 3(b) are the left upper and left lower view point cloud images after removing the ground data.

[0073] Fig. 3(c), 3(d) are the right upper and right lower view point cloud images after removing the ground data.

[0074] Fig. 4(a) is the left upper and left lower view point cloud image after completing the nearest point iterative matching.

[0075] Fig. 4(b) is the right upper and right lower view point cloud image after completing the nearest point iterative matching.

[0076] Fig. 5(a) is the point cloud image (front view) after completing the matching of the left and right view landmark points.

[0077] Fig. 5(b) is the point cloud image (back view) after completing the matching of the left and right view landmark points.

[0078] Fig. 5(c) is the point cloud image (side view) after completing the matching of the left and right view landmark points.

[0079] Figure 6 Fig. 6 is the structure schematic diagram of the live pig three-dimensional point cloud modeling system based on the nearest point iteration and landmark point matching of the present application.

[0080] Figure 7 Fig. 7 is the structure schematic diagram of the electronic device of the present application. DETAILED DESCRIPTION

[0081] In order to make the personnel in the technical field better understand the scheme of the present application, the technical scheme in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the person skilled in the art without making creative efforts are within the protection scope of the present application.

[0082] In the present application, the phrase "embodiments" means that the specific features, structures or characteristics described in combination with the embodiments can be contained in at least one embodiment of the present application. The appearance of this phrase at various places in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. The person skilled in the art explicitly and implicitly understands that the embodiments described in the present application can be combined with other embodiments.

[0083] AsFigure 1 The embodiment shown is based on a live pig three-dimensional point cloud modeling method based on the nearest point iteration and landmark point matching, including the following steps:

[0084] (1) Based on the Kinect depth camera, the point cloud data of the live pig from the left upper, left lower, right upper and right lower four perspectives is obtained, the point cloud data is denoised, and the target and background are cut, the ground in the point cloud data is removed based on the Ransac algorithm, and the point cloud data of the target pig body from the four perspectives is obtained.

[0085] Further, the removal of the point cloud ground data based on the Ransac algorithm is specifically:

[0086] (1-1) Taking the initial live pig target area as the center, expanding k coordinates in the up, down, left and right four directions as the boundary for cutting to obtain point cloud data without background, such as Figures 2(a)-2(d) .

[0087] (1-2) The point cloud data is median filtered to remove discrete points.

[0088] (1-3) Three points are randomly selected from the point cloud data to form a plane, the distances of the remaining points in the point cloud data to the plane are calculated, and if the distance is less than the threshold T, it is considered that the points are in the same plane; if the points in the same plane exceed n, the plane is saved, and the points in the plane are all recorded as ground point clouds, which are removed to obtain point cloud data without ground, such as Figures 3(a)-3(d) .

[0089] (1-4) If the number of points in the plane found after N iterations is less than n, repeat step (1-3) to finally obtain point cloud data without ground.

[0090] (2) The point cloud data from the left upper perspective is taken as the source point cloud, and the point cloud data from the left lower perspective is taken as the target point cloud, and the nearest neighbor point of each point in the source point cloud in the target point cloud is calculated based on the KD-Tree.

[0091] Further, the calculation of the nearest neighbor point of each point in the source point cloud in the target point cloud is specifically:

[0092] (2-1) The point cloud data from the left upper perspective is taken as the source point cloud, and the point cloud data from the left lower perspective is taken as the target point cloud, and for the comparison dimension of the KD-Tree, the three axes of XYZ are selected as the comparison dimension in turn with the depth of the KD-Tree.

[0093] (2-2) A point in the target point cloud data is randomly selected as the root node of the KD-Tree, and the KD-Tree is constructed according to the rule that the left child node is less than the parent node and the right child node is greater than the parent node.

[0094] (2-3) Set a current global optimal point and shortest distance in the target point cloud, initially as the root node of the KD-Tree and infinity.

[0095] (2-4) Calculate the distance from the current node of the target point cloud (initially the root node) to the searched point in the source point cloud. If it is less than the root node, update the current node to the global optimal point and the shortest distance.

[0096] (2-5) If the partition scale of the searched point in the source point cloud (the distance scale of the left and right subtrees of the point) is smaller than the partition scale of the current node in the target point cloud, then the left child of the current node is set as the new current node, otherwise the right child of the current node is set as the new current node; if the partition scale distance from the searched point in the source point cloud to the current node in the target point cloud is smaller than the global shortest distance, then the root node of the other subtree of the current node of the target point cloud is set as the new current node. Repeat steps (2-4) and (2-5) until the current node is empty, and obtain the nearest neighbor of each point in the source point cloud in the target point cloud. The calculation of the nearest neighbor points of the upper right and lower right perspectives is similar.

[0097] (3) The point cloud data of the upper left perspective is used as the source point cloud, and the point cloud data of the lower left perspective is used as the target point cloud. Both point clouds are converted to the centroid coordinate system. Based on SVD decomposition, the optimal rotation matrix and optimal translation matrix of the current target point cloud are calculated to transform the rotation of the current target point cloud to the source point cloud coordinate system. The rotation matrix and translation matrix are applied to the target point cloud to obtain the target point cloud after rotation and translation, replacing the original target point cloud. The rotation error and translation error of the target point cloud and the source point cloud are calculated to determine whether the conditions for stopping iteration are met. Otherwise, repeat steps (2) to (3) to finally obtain the point cloud data of the upper left perspective point cloud and the upper right perspective point cloud in the same coordinate system. The matching of the upper right perspective and the lower right perspective point cloud data is similar, and finally the point cloud data of the left and right perspectives are obtained, as shown in Figure 4 (a) and Figure 4 (b).

[0098] Furthermore, the step (3) is specifically as follows:

[0099] (3-1) Continue to use the point cloud data of the upper left perspective as the source point cloud and the point cloud data of the lower left perspective as the target point cloud, calculate the center of mass of the source point cloud and the target point cloud, and convert the source point cloud and the target point cloud to the center of mass coordinate system.

[0100] Center of mass:

[0101]

[0102] Convert coordinate system:

[0103] in etc. represent the values ​​of the coordinate points of the source point cloud and the target point cloud on each axis. representing the center of mass of the source point cloud and the target point cloud, representing the coordinate points of the source point cloud and the target point cloud, representing the coordinate points of the source point cloud and the target point cloud after the transformation of the coordinate system.

[0104] (3-2) Let SVD decomposition of H = U∑V T , get the optimal rotation of the current target point cloud to the source point cloud coordinate system: R * = VU T and the optimal translation R * is the current optimal rotation matrix, t * is the current optimal translation matrix.

[0105] (3-3) Let the rotation error be: The translation error is The optimal rotation matrix and the optimal translation matrix R * ,t * obtained in each iteration are applied to the current target point cloud for rotation and translation, and the rotated and translated target point cloud data is taken as the original target point cloud data. Repeat steps (2-1) to (3-3) to obtain new optimal rotation matrix R * and transformation matrix t * until one of the following conditions is met:

[0106] A, the change of R * ,t * is less than a set value.

[0107] B, the rotation error and the translation error are less than a set value.

[0108] C, the number of repetitions reaches a set value.

[0109] Finally, the source point cloud data and the target point cloud data are successfully registered in the same coordinate system. The matching of the right upper view point cloud data and the right lower view point cloud data is the same. Finally, the left view point cloud data and the right view point cloud data are successfully matched respectively.

[0110] (4) Identify the four circular marker points around the live pig based on the RGB information, and obtain the center coordinates of the marker points in the respective point clouds through circle center fitting.

[0111] Further, the identification of the marker point center coordinates is as follows:

[0112] (4-1) According to the RGB information, screen the point cloud data of each different color circular marker point around the live pig from the point cloud data of the left and right views.

[0113] (4-2) For each circular marker point, calculate the three-dimensional coordinate point of its center in the point cloud.

[0114]

[0115] wherein, is the coordinate value of the center of the circle, x i , y i , and z i are the coordinate values of all points in the point cloud in the circular marker point, and n is the number of points in a single circular marker point.

[0116] (5) Based on the three-point method, calculate the rotation transformation matrix required for the same group of marker points to be converted to the same coordinate system in the left and right different angle point cloud data. Using the rotation transformation matrix, the front and rear angle point cloud data of the live pig is transformed to the same coordinate system for registration, and the three-dimensional point cloud model of the live pig is obtained, as shown in Figures 5(a), 5(b), and 5(c).

[0117] Further, based on the marker point registration, specifically:

[0118] (5-1) Taking the left angle point cloud data as the source point cloud and the right angle point cloud data as the target point cloud, calculate the orthogonal coordinate vector of the marker point center in the source point cloud and the target point cloud:

[0119]

[0120] wherein is the orthogonal coordinate vector of the marker point center in the source point cloud, is the orthogonal coordinate vector of the marker point center in the target point cloud, x p and x q are the coordinate values of the marker point center of the source point cloud and the target point, respectively.

[0121] (5-2) After unitizing the orthogonal coordinate vector, the coordinate system with the marker point center as the origin is obtained:

[0122]

[0123] wherein and are the coordinate systems with the marker point center as the origin of the source point cloud and the target point cloud, respectively.

[0124] (5-3) According to the coordinate systems with the marker point center as the origin of the source point cloud and the target point cloud, calculate the rotation matrix and translation matrix of the target point cloud transformed to the source point cloud.

[0125] Rotation matrix:

[0126] Translation matrix: T = p - qR

[0127] (5-4)According to the obtained rotation matrix R and translation matrix T, a coordinate transformation matrix is obtained:

[0128] The target point cloud is converted to the source point cloud coordinate system by the transformation formula PF=Q through the transformation matrix, registration of the target point cloud and the source point cloud is realized, and finally a live pig three-dimensional point cloud model is obtained. Wherein F is a transformation matrix, P is a target point cloud coordinate, and Q is a source point cloud coordinate.

[0129] It should be noted that, for the foregoing method embodiments, in order to facilitate description, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited by the action sequence described, because according to the present application, certain steps can be performed in other sequences or simultaneously.

[0130] Based on the same idea as the live pig three-dimensional point cloud modeling method based on nearest point iteration and landmark point matching in the above embodiment, the present application also provides a live pig three-dimensional point cloud modeling system based on nearest point iteration and landmark point matching, which can be used to execute the live pig three-dimensional point cloud modeling method based on nearest point iteration and landmark point matching. In order to facilitate description, in the structural schematic diagram of the live pig three-dimensional point cloud modeling system embodiment based on nearest point iteration and landmark point matching, only the parts related to the embodiments of the present application are shown, and those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and can include more or fewer components than the illustrated, or combine certain components, or different component arrangements.

[0131] Please refer to Figure 6 In another embodiment of the present application, a live pig three-dimensional point cloud modeling system 100 based on nearest point iteration and landmark point matching is provided, which comprises a data acquisition module 101, a nearest neighbor point calculation module 102, a nearest neighbor point iteration module 103, a landmark point identification module 104 and a landmark point registration module 105;

[0132] The data acquisition module 101 is used to acquire point cloud data of the live pig from four perspectives of the upper left, lower left, upper right and lower right, denoises the point cloud data, cuts the target pig body from the background, removes the ground in the point cloud data, and obtains point cloud data of the target pig body from four perspectives of the upper left, lower left, upper right and lower right.

[0133] The nearest neighbor point calculation module 102 is used to take the point cloud data of the target pig body from the upper left and upper right perspectives as the source point cloud, and take the point cloud data from the lower left and lower right perspectives as the target point cloud, and calculate the nearest neighbor point of each point in the source point cloud in the target point cloud.

[0134] The nearest neighbor point iteration module 103 is configured to take the point cloud data of the upper left view as source point cloud data and the point cloud data of the lower left view as target point cloud data, convert both of the point cloud data to a centroid coordinate system, calculate an optimal rotation matrix and an optimal translation matrix of the current target point cloud rotation to the source point cloud coordinate system based on SVD decomposition, apply the rotation matrix and the translation matrix to the target point cloud to obtain the target point cloud after rotation and translation, and replace the original target point cloud; calculate the rotation error and the translation error of the target point cloud and the source point cloud, determine whether a stop iteration condition is reached, and if not, repeat the processing of the nearest neighbor point calculation module and the nearest neighbor point iteration module, and finally obtain the point cloud data of the upper left view and the upper right view registered in the same coordinate system; the matching of the point cloud data of the upper right view and the lower right view is the same, and finally the point cloud data of the left view and the right view is obtained.

[0135] The landmark point identification module 104 is configured to identify four circular landmark points around the live pig based on RGB information, and obtain the center coordinates of the landmark points in the respective point cloud in the left view and the right view through center fitting.

[0136] The landmark point registration module 105 is configured to calculate a rotation transformation matrix required for converting the same set of landmark points in the left view and the right view point cloud data to the same coordinate system based on a three-point method, and use the rotation transformation matrix to transform the point cloud data of the front view and the rear view of the live pig to the same coordinate system for registration, and obtain a three-dimensional point cloud model of the live pig.

[0137] It should be noted that the live pig three-dimensional point cloud modeling system based on nearest point iteration and landmark point matching of the present application corresponds to the live pig three-dimensional point cloud modeling method based on nearest point iteration and landmark point matching of the present application, and the technical features and advantages described in the above embodiment of the live pig three-dimensional point cloud modeling method based on nearest point iteration and landmark point matching are applicable to the embodiment of the live pig three-dimensional point cloud modeling system based on nearest point iteration and landmark point matching, and the specific content can be referred to the description in the method embodiment of the present application, which will not be repeated here, and hereby declared.

[0138] In addition, in the embodiment of the live pig three-dimensional point cloud modeling system based on nearest point iteration and landmark point matching of the above embodiment, the logical division of each program module is only an example, and in actual application, the above functions can be completed by different program modules according to the needs, for example, the configuration requirements of the corresponding hardware or the convenience of software implementation, that is, the internal structure of the live pig three-dimensional point cloud modeling system based on nearest point iteration and landmark point matching is divided into different program modules to complete all or part of the functions described above.

[0139] Please refer to Figure 7In one embodiment, an electronic device implementing a live pig three-dimensional point cloud modeling method based on nearest point iteration and landmark point matching is provided. The electronic device 200 can include a first processor 201, a first memory 202, and a bus. The electronic device 200 can also include a computer program, such as a live pig three-dimensional point cloud modeling program 203, stored in the first memory 202 and executable on the first processor 201.

[0140] The first memory 202 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the first memory 202 can include both an internal storage unit and an external storage device of the electronic device 200. The first memory 202 can be used to store application software installed in the electronic device 200 and various data, such as the code of the live pig three-dimensional point cloud modeling program 203, and can also be used to temporarily store data that has been output or will be output.

[0141] The first processor 201 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, combinations of various control chips, etc. The first processor 201 is the control unit of the electronic device, which connects various components of the electronic device through various interfaces and lines, and executes programs or modules stored in the first memory 202 and data stored in the first memory 202 to perform various functions and process data of the electronic device 200.

[0142] Figure 7 Only an electronic device with components is shown, and those skilled in the art can understand that, Figure 7 The structure shown does not limit the electronic device 200, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0143] The live pig three-dimensional point cloud modeling program 203 stored in the first memory 202 in the electronic device 200 is a combination of a plurality of instructions, which, when running in the first processor 201, can realize:

[0144] (1) Obtain the point cloud data of the live pig from the left upper, left lower, right upper and right lower four views, denoise the point cloud data, cut the target pig body from the background, remove the ground in the point cloud data, and obtain the point cloud data of the target pig body from the left upper, left lower, right upper and right lower four views;

[0145] (2) Take the left upper and right upper view point cloud data of the target pig body as the source point cloud, and take the left lower and right lower view point cloud data as the target point cloud. Calculate the nearest neighbor point of each point in the source point cloud in the target point cloud;

[0146] (3) Take the point cloud data of the left upper view as the source point cloud, and take the point cloud data of the left lower view as the target point cloud. Convert both point clouds to the center of mass coordinate system. Calculate the optimal rotation matrix and optimal translation matrix of the current target point cloud rotation transformation to the source point cloud coordinate system based on SVD decomposition. Apply the rotation matrix and translation matrix to the target point cloud to obtain the rotated and translated target point cloud, which replaces the original target point cloud. Calculate the rotation error and translation error of the target point cloud and the source point cloud. Determine whether the stop iteration condition is reached. If not, repeat steps (2) to (3). Finally, the point cloud data of the left upper view and the right upper view is registered in the same coordinate system. The matching of the right upper view and the right lower view point cloud data is the same. Finally, the point cloud data of the left and right two views is obtained.

[0147] (4) Based on the RGB information, identify the four circular marker points around the live pig, and obtain the center coordinates of the marker points in the respective point clouds in the left and right two views through center fitting;

[0148] (5) Based on the three-point method, calculate the rotation transformation matrix required for the same group of marker points to be converted to the same coordinate system in the left and right different view point cloud data. Use the rotation transformation matrix to transform the front and rear view point cloud data of the live pig to the same coordinate system for registration, and obtain the live pig three-dimensional point cloud model.

[0149] Further, the modules / units of the electronic device 200, if realized in the form of software function units and sold or used as independent products, can be stored in a non-volatile computer readable storage medium. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM).

[0150] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware. The program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, storage, database or other medium used in the embodiments provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0151] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0152] The above embodiments are the preferred embodiments of the present application, but the embodiments of the present application are not limited to the above embodiments, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principles of the present application shall be equivalent replacement methods and shall be within the scope of protection of the present application.

Claims

1. A method for 3D point cloud modeling of live pigs based on nearest point iteration and landmark point matching, characterized in that, The method comprises the following steps: (1) acquiring point cloud data of a live pig from four perspectives of upper left, lower left, upper right and lower right, denoising the point cloud data, cutting the target pig body from the background, removing the ground from the point cloud data, and obtaining point cloud data of the target pig body from four perspectives of upper left, lower left, upper right and lower right; (2) taking the point cloud data of the target pig body from the perspectives of upper left and upper right as source point cloud data, and taking the point cloud data from the perspectives of lower left and lower right as target point cloud data, and calculating the nearest neighbor point of each point in the source point cloud in the target point cloud; (3) taking the point cloud data from the perspective of upper left as source point cloud data, and taking the point cloud data from the perspective of lower left as target point cloud data, converting both point clouds to a centroid coordinate system, calculating an optimal rotation matrix and an optimal translation matrix of the current target point cloud rotation to the source point cloud coordinate system based on SVD decomposition, and obtaining the target point cloud after rotation and translation by applying the rotation matrix and the translation matrix to the target point cloud, and replacing the original target point cloud; calculating the rotation error and translation error of the target point cloud and the source point cloud, and judging whether the stop iteration condition is reached, otherwise repeating steps (2) to (3), and finally obtaining the point cloud data of the upper left perspective and the upper right perspective in the same coordinate system; the matching of the point cloud data from the perspectives of upper right and lower right is the same, and finally the point cloud data from the left and right perspectives is obtained; (4) identifying four circular marker points around the live pig based on RGB information, and obtaining the center coordinates of the marker points in the respective point clouds by fitting the center; (5) calculating the rotation transformation matrix required for converting the same group of marker points in the left and right point cloud data in different perspectives to the same coordinate system based on the three-point method, and transforming the point cloud data from the front and rear perspectives of the live pig to the same coordinate system for registration by using the rotation transformation matrix, and obtaining a three-dimensional point cloud model of the live pig.

2. The method according to claim 1, wherein, In step (1), the Ransac algorithm is used to remove the ground data of the point cloud, which is as follows: (1-1) taking the area where the target pig body is located as the center, expanding k coordinates in the four directions of up, down, left and right as boundaries for cutting, and obtaining point cloud data without background; (1-2) performing median filtering on the point cloud data without background to remove discrete points; (1-3) randomly selecting three points from the point cloud data without background to form a plane, calculating the distances of the remaining points in the point cloud data to the plane, and if the distance is less than a threshold T, it is considered that the points are in the same plane; if the points in the same plane exceed n, the plane is saved, and the points of the plane are recorded as ground point cloud and removed to obtain point cloud data without ground.

3. The method according to claim 1, wherein, In step (2), the KD-Tree algorithm is used to calculate the nearest neighbor point of each point in the source point cloud in the target point cloud, which is as follows: (2-1) for the comparison dimension of the KD-Tree, the three axes of XYZ are selected as the comparison dimension in turn with the depth of the KD-Tree; (2-2) randomly selecting a point in the target point cloud data as the root node of the KD-Tree, and constructing the KD-Tree according to the rule that the left child node is smaller than the parent node and the right child node is larger than the parent node. (2-3) Set a current global best point and the shortest distance in the target point cloud, which is initially the root node of the KD-Tree and infinity; (2-4) Calculate the distance from the current node of the target point cloud to the searched point in the source point cloud, and if the distance is less than the current shortest distance, update the current node as the global best point and the shortest distance; (2-5) If the division scale of the searched point in the source point cloud is less than the division scale of the current node in the target point cloud, set the left child node of the current node as a new current node, otherwise set the right child node of the current node as the new current node; if the distance from the searched point in the source point cloud to the division scale of the current node in the target point cloud is less than the global shortest distance, set the root node of the other child tree of the current node in the target point cloud as the new current node; the division scale refers to the distance scale of the left and right child trees of the searched point; (2-6) Loop steps (2-4) and (2-5) until the current node is empty, and obtain the nearest neighbor point of each point in the source point cloud in the target point cloud; use the same method to calculate the nearest neighbor points of the upper right and lower right perspectives.

4. The method according to claim 1, wherein, The step (3) is specifically: (3-1) Continue to take the point cloud data of the upper left perspective as the source point cloud, and the point cloud data of the lower left perspective as the target point cloud, calculate the centroids of the source point cloud and the target point cloud, and convert the source point cloud and the target point cloud to the centroid coordinate system; centroid: Transforming coordinate systems: wherein denote the values of the coordinate points of the source point cloud and the target point cloud on the axes, denote the centroids of the source point cloud and the target point cloud, denote the coordinate points of the source point cloud and the target point cloud, denote the coordinate points of the source point cloud and the target point cloud after transforming the coordinate system; (3-2) Let Singular value decomposition of H gives H = U∑V T , the optimal rotation of the current target point cloud to the source point cloud coordinate system: R * = VU T and the optimal translation U is the left singular vector, V is the right singular vector, R * is the current optimal rotation matrix, t * is the current optimal translation matrix; (3-3) Set the rotation error as: The translation error is F(t * ) The optimal rotation matrix and the optimal translation matrix R * ,t * are obtained in each iteration, and the rotation and translation are performed on the current target point cloud, and the rotated and translated target point cloud data is taken as the original target point cloud data. Steps (2-1) to (3-3) are repeated to calculate new optimal rotation matrix R * and translation matrix t * until the set condition is met. (3-4) Finally, obtain the successfully registered point cloud data of the source point cloud data and the target point cloud data in the same coordinate system; the matching of the upper right perspective point cloud data and the lower right perspective point cloud data is the same, and finally the successfully matched point cloud data of the left perspective point cloud data and the right perspective point cloud data is obtained.

5. The method according to claim 4, wherein, The set condition is satisfied, which means that one of the following conditions is satisfied: R * t * a change amount of the temperature is less than a set value; The rotation error and the translation error are less than a set value; The number of repetitions reaches a set value.

6. The method of claim 1, wherein, In the step (4), the identification of the center coordinates of the marker points is specifically: (4-1) According to the RGB information, screen the point cloud data of each different color circular marker point around the live pig from the point cloud data of the left and right perspectives; (4-2) For each circular marker point, calculate the three-dimensional coordinate point of the center of the circular marker point in the point cloud; wherein, is the center coordinate value, x i ,y i ,z i is the coordinate value of all points in the point cloud in the circular marker point, and n is the number of points in a single circular marker point.

7. The method according to claim 1, wherein, In step (5), the point cloud data of the live pig from the front and rear perspectives is transformed to the same coordinate system for registration, which is specifically: (5-1) Take the left perspective point cloud data as the source point cloud, and the right perspective point cloud data as the target point cloud, and calculate the orthogonal coordinate vectors of the marker point centers in the source point cloud and the target point cloud: wherein is a center orthogonal coordinate vector of a landmark point of the source point cloud, is a center orthogonal coordinate vector of a landmark point of the target point cloud, x p and x q are center coordinate values of the landmark points of the source point cloud and the target point, respectively; (5-2) After the orthogonal coordinate vector is unitized, a coordinate system with the center of the marker point circle as the origin is obtained: wherein and respectively are the coordinate systems of the source point cloud and the target point cloud with the center of the landmark circle as the origin. (5-3) According to the coordinate system with the marker point center as the origin of the source point cloud and the target point cloud, calculate the rotation matrix and the translation matrix of the target point cloud transformed to the source point cloud; Rotation matrix: Translation matrix: T = p - qR (5-4) Obtain the coordinate transformation matrix according to the obtained rotation matrix R and translation matrix T: The target point cloud is converted to the source point cloud coordinate system by a transformation formula PF=Q through a transformation matrix to realize registration of the target point cloud and the source point cloud, and finally obtain a three-dimensional point cloud model of the live pig, wherein F is the transformation matrix, p is the target point cloud coordinate, and Q is the source point cloud coordinate.

8. A live pig three-dimensional point cloud modeling system based on the nearest point iteration and landmark point matching, characterized in that, The system comprises a data acquisition module, a nearest neighbor point calculation module, a nearest neighbor point iteration module, a marker point identification module and a marker point registration module. The data acquisition module is used for acquiring the point cloud data of the live pig from the left upper, left lower, right upper and right lower perspectives, performing noise reduction on the point cloud data, cutting the target pig body from the background, removing the ground in the point cloud data, and obtaining the point cloud data of the target pig body from the left upper, left lower, right upper and right lower perspectives. The nearest neighbor point calculation module is configured to take the left upper and right upper perspective point cloud data of the target pig as source point cloud data and the left lower and right lower perspective point cloud data as target point cloud data, and calculate the nearest neighbor point of each point in the source point cloud in the target point cloud; The nearest neighbor point iteration module is configured to take the point cloud data of the left upper perspective as source point cloud data and the point cloud data of the left lower perspective as target point cloud data, convert both point clouds to a centroid coordinate system, calculate an optimal rotation matrix and an optimal translation matrix of the current target point cloud rotation transformation to the source point cloud coordinate system based on SVD decomposition, apply the rotation matrix and the translation matrix to the target point cloud to obtain the target point cloud after rotation and translation, and replace the original target point cloud; The rotation error and the translation error of the target point cloud and the source point cloud are calculated, and it is determined whether a stop iteration condition is reached, and if not, the processing process of the nearest neighbor point calculation module and the nearest neighbor point iteration module is repeated, and finally the point cloud data of the left upper perspective point cloud and the right upper perspective point cloud registered in the same coordinate system is obtained; the matching of the right upper perspective point cloud data and the right lower perspective point cloud data is the same, and finally the point cloud data of the left and right perspectives is obtained. The landmark point identification module is configured to identify four circular landmark points around the live pig based on RGB information, and obtain the center coordinates of the landmark points in the respective point clouds in the left and right perspectives through center fitting. The landmark point registration module is configured to calculate the rotation transformation matrix required for converting the same set of landmark points in the left and right perspective point cloud data to the same coordinate system based on a three-point method, and use the rotation transformation matrix to transform the point cloud data of the front and rear perspectives of the live pig to the same coordinate system for registration, and obtain a three-dimensional point cloud model of the live pig.

9. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the live pig three-dimensional point cloud modeling method based on nearest point iteration and landmark point matching according to any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that: The program is executed by the processor to implement the live pig three-dimensional point cloud modeling method based on nearest point iteration and landmark point matching according to any one of claims 1-7.

Images