Orchid 3D modeling method and device based on SAC-IA and ICP registration

Abstract

The present invention discloses a method and device for three-dimensional modeling of orchids based on SAC-IA and ICP registration. The method includes the following steps: first, acquiring and collecting orchid point cloud data; then, clustering the orchid point cloud data from each perspective to separate landmark points from the orchid body; then, extracting the internal shape descriptor (ISS) and fast point feature histogram (FPFH) descriptor from the orchid body point cloud on the same side, and performing sampling consistency initial registration (SAC-IA) and iterative closest point (ICP) algorithm fine registration; finally, identifying the landmark points on the left and right sides based on color information, and registering and merging the point clouds on both sides. The method of the present invention is based on point cloud features such as the internal shape descriptor (ISS) and fast point feature histogram (FPFH) descriptor, combined with the sampling consistency initial registration (SAC-IA) algorithm and the iterative closest point (ICP) algorithm, and assisted by landmark points to achieve semi-automatic orchid point cloud modeling, thereby improving the efficiency of orchid modeling and enhancing the accuracy and stability of orchid modeling.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and point cloud three-dimensional reconstruction, and specifically relates to a three-dimensional modeling method and device for orchids based on SAC-IA and ICP registration. Background Art

[0002] Point cloud 3D modeling, which uses point cloud data collected by depth cameras, radar, and other devices to create a three-dimensional model of an object or its surroundings, is a research hotspot in fields such as computer vision, robotics, and military applications, and has attracted widespread attention in recent years. This technology has broad application prospects and plays an important role in numerous fields, including autonomous driving, environmental perception, industrial and crop monitoring. Although various point cloud 3D reconstruction methods exist, these methods often fail to provide ideal registration results and are unable to fully model orchids due to the complex phenotype and occlusion of orchids. Therefore, proposing an effective 3D orchid modeling method based on landmark recognition has important application value and practical significance.

[0003] Although many methods have been proposed to assist in modeling by attaching landmarks to the surface of objects, it is difficult to attach landmarks to the surfaces of objects such as orchids and crops. Developing a modeling method that can handle complex orchids without auxiliary landmarks on their surfaces remains a challenging problem. Therefore, a modeling method that does not directly attach landmarks to the orchid surface is essential for 3D reconstruction tasks. The orchid phenotypic 3D modeling method based on landmark recognition has good modeling results when the orchid leaves occlude each other or deform. Summary of the Invention

[0004] The primary purpose of this invention is to overcome the shortcomings and deficiencies of existing technologies by providing a method and apparatus for 3D orchid modeling based on SAC-IA and ICP registration. This method incorporates landmark recognition into the point cloud registration process, using landmark coordinates and colors to match orchids from multiple viewpoints for precise registration, aiming to enhance the accuracy and stability of 3D orchid modeling.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a method for three-dimensional modeling of orchids based on SAC-IA and ICP registration, comprising the following steps:

[0007] (1) Obtain orchid point cloud data from four perspectives: the left front side, the left back side, the right front side, and the right back side, and pre-process the orchid point cloud data to reduce the number of orchid point cloud points;

[0008] (2) Based on the random sampling consistency RANSAC algorithm, the ground in the orchid point cloud data is removed, the orchid and the background are cut, and the orchid point cloud data of each perspective is divided into two parts: "orchid main point cloud" and "landmark point cloud" using a clustering algorithm;

[0009] (3) The “orchid main body point cloud data” from two perspectives on the same side are set as the “source point cloud” and “target point cloud” respectively. The internal shape descriptor ISS is extracted from the “source point cloud” and “target point cloud” as key points, and their fast point histogram features FPFH are calculated. The two point clouds are subjected to the sampling consistency initial alignment SAC-IA operation to achieve preliminary alignment based on key points, and the orchid point cloud that has been preliminarily aligned on one side is obtained.

[0010] (4) The orchid point cloud that has been preliminarily registered is finely registered using the trimmed closest point iterative TRICP algorithm to obtain an orchid point cloud with color-precise registration of the landmark points on the same side; after processing one side, the same steps are followed for the other side to obtain an orchid point cloud with color-precise registration of the landmark points on the other side;

[0011] (5) The color information of the four landmark points in the point clouds on both sides of the orchid is extracted respectively, and their center coordinates are calculated; the rotation transformation matrix required to transform the landmark points in the point clouds on both sides into the same coordinate system is calculated, and the point cloud data on both sides are merged using the transformation matrix to obtain a complete orchid model.

[0012] As a preferred technical solution, in step (1), the orchid point cloud data is preprocessed to reduce the number of orchid point cloud points, specifically:

[0013] (1-1) Perform statistical analysis and filtering on the orchid point cloud. For each point in the orchid point cloud, calculate its distance to the nearby n points. s The statistical parameters of the average distance and variance of neighboring points are: s times the variance is considered an outlier and needs to be removed; where n s ,s is the preset threshold;

[0014] (1-2) Use a three-dimensional voxel grid to downsample the orchid point cloud, calculate the average value of the coordinates of all points in the voxel grid, and use the coordinates as the centroid coordinates. Finally, all points in the voxel grid are represented by one centroid point, thereby reducing the amount of point cloud data while maintaining the shape characteristics of the point cloud.

[0015] As a preferred technical solution, step (2) is specifically as follows:

[0016] (2-1) Randomly select a plane from the orchid point cloud data and calculate the distance from each point in the orchid point cloud data to the plane. If the distance is less than a threshold d, the point is considered to be within the plane. If there are more than s points in the same plane, the plane is recorded as a ground point cloud and removed. Where s and d are pre-set parameters.

[0017] (2-2) For a point p in the orchid point cloud e , search for the point n through KD-Tree algorithm e Nearby points, judge this n e Point to p e The Euclidean distance is less than the threshold r. e The points are grouped into the same category; where n e , r e is a pre-set parameter;

[0018] (2-3) If the number of points in a class meets the set value range, the class is valid, otherwise it is discarded;

[0019] (2-4) Repeat the same method for the remaining points in the cloud until the entire point cloud has been searched.

[0020] As a preferred technical solution, in step (3), the internal shape descriptor ISS is extracted from the "source point cloud" and the "target point cloud" as a key point, and its fast point histogram feature FPFH is calculated, and the two point clouds are subjected to the sampling consistency initial alignment SAC-IA operation, specifically:

[0021] (3-1) For each point p in the orchid point cloud i , establish a sphere with a radius of r with this point as the center. s spherical neighborhood; for each p in the neighborhood j Point, calculate p j To the center point p i The distance d ij , and assign each point its weight

[0022] (3-2) Calculate each point p i The covariance matrix of

[0023]

[0024] And record each point p i The three eigenvalues ​​of the covariance matrix are and For thresholds ∈1, ∈2, if the condition is satisfied Point p i This is the key point, denoted as p s ; where rs ,∈1,∈2 are pre-set parameters;

[0025] (3-3) Calculate the FPFH features of the key point cloud data composed of key points. For each key point p in the point cloud s , at its radius r f Search k in the spherical neighborhood of f Point p t , calculate the SPFH feature

[0026] w=n s ×(p t -p s )×n s

[0027] where n s For p s Normal vector of a point

[0028] α=v·n t

[0029]

[0030] θ=arctan(w·n t ,n s ·n t )

[0031] where n t For p t Normal vector of a point

[0032]

[0033] where k f ,r f Parameters that need to be set for the program

[0034] (3-4) Randomly select matching points based on FPFH features in the orchid point cloud;

[0035] (3-5) In this matching case, singular value decomposition (SVD) is used to obtain the transformation matrix required for the solution;

[0036] (3-6) Calculate the matching error after transformation, repeat (3-4), (3-5), and (3-6) until the matching error meets the set conditions, and retain the transformation matrix with the smallest error.

[0037] As a preferred technical solution, in step (4), the specific operation of using the trimmed nearest point iterative algorithm for precise registration is:

[0038] (4-1) Calculate the centroid p of the source point cloud and target point cloud after rough registrations , p t :

[0039]

[0040] where n s ,n t is the number of points in the source point cloud and the target point cloud;

[0041] Decentralize the point cloud to obtain

[0042]

[0043] (4-2) SVD solves the rotation matrix R * , translation matrix T *

[0044]

[0045] T * =p s -R * p s

[0046] (4-3) The obtained rotation matrix R * , translation matrix T * Acting on the source point cloud, for each point p of the point cloud after rotation and translation i Use KD-Tree to search for the nearest point in the target point cloud Form a corresponding relationship and calculate the square of the distance d between them i 2 , sort all corresponding relationships according to the distance between their point pairs, and take the first n u (n u ≤n s ) The distances of the corresponding relationships are summed up. If the termination condition is met, the program is exited, otherwise (4-1)(4-2)(4-3) are repeated, where n u are pre-set parameters.

[0047] As a preferred technical solution, the termination condition is to meet one of the following conditions:

[0048] (1) The maximum number of iterations is reached and the iteration is terminated;

[0049] (2) The error is less than the set value;

[0050] (3) The change in the rotation and translation matrix generated by the previous iteration is small enough.

[0051] As a preferred technical solution, in step (5), the specific operations of identifying the landmark points and merging the point clouds on both sides are as follows:

[0052] (5-1) According to the color information of the marker points, the point cloud data of the marker points of different colors are filtered out from the point cloud data on both sides;

[0053] (5-2) For each landmark point, calculate its center coordinates;

[0054]

[0055] Among them, c is the center coordinate value, m i are all the points that make up the landmark point, and n is the number of points in the point cloud of a single landmark point;

[0056] (5-3) The point cloud models on the left and right sides are set as "source point cloud" and "target point cloud" respectively. By using SVD to solve the coordinates of the landmark points on both sides in the two coordinate systems, the rotation matrix and translation matrix of the source point cloud transformed to the target point cloud coordinate system are obtained; the rotation matrix and translation matrix are applied to the source point cloud to align it with the target point cloud, and finally a complete point cloud model is obtained after the two sides are merged.

[0057] In a second aspect, the present invention provides a three-dimensional orchid modeling system based on SAC-IA and ICP registration, which is applied to the three-dimensional orchid modeling method based on SAC-IA and ICP registration, including a data acquisition module, a data partitioning module, a coarse registration module, a fine registration module, and a data merging module;

[0058] The data acquisition module is used to acquire orchid point cloud data from four perspectives: the left front side, the left rear side, the right front side, and the right rear side of the orchid, and pre-process the orchid point cloud data to reduce the number of orchid point cloud points;

[0059] The data segmentation module is used to remove the ground from the orchid point cloud data based on the random sampling consistency RANSAC algorithm, cut the orchid from the background, and use the clustering algorithm to divide the orchid point cloud data of each perspective into two parts: "orchid main point cloud" and "mark point cloud";

[0060] The coarse registration module is used to set the "orchid main body point cloud data" from two perspectives on the same side as the "source point cloud" and the "target point cloud" respectively, extract the internal shape descriptor ISS from the "source point cloud" and the "target point cloud" as key points, and calculate their fast point histogram features FPFH. The two point clouds are subjected to a sampling consistency initial registration (SAC-IA) operation to achieve preliminary registration based on key points, thereby obtaining a preliminary registered orchid point cloud on one side;

[0061] The fine registration module is used to finely register the preliminarily registered orchid point cloud using the trimmed closest point iterative TRICP algorithm to obtain an orchid point cloud with color-precise registration of the marker points on the same side; after processing one side, the same steps are followed for the other side to obtain an orchid point cloud with color-precise registration of the marker points on the other side;

[0062] The data merging module is used to extract the color information of the four marker points in the point clouds on both sides of the orchid and calculate their center coordinates; calculate the rotation transformation matrix required to convert the marker points in the point clouds on both sides to the same coordinate system, and use the transformation matrix to merge the point cloud data on both sides to obtain a complete orchid model.

[0063] In a third aspect, the present invention further provides an electronic device, comprising:

[0064] at least one processor; and,

[0065] a memory communicatively connected to the at least one processor; wherein,

[0066] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the orchid three-dimensional modeling method based on SAC-IA and ICP registration.

[0067] In a fourth aspect, the present invention further provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the orchid three-dimensional modeling method based on SAC-IA and ICP registration.

[0068] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0069] 1. The present invention aims at targets that are complex and inconvenient to add markers on the surface, such as orchids. It uses external independent marker points for assistance and combines multiple perspectives to shoot the target, so that the acquisition range can well cover the entire target, while reducing the impact of target swing and overlap on the modeling effect.

[0070] 2. This invention incorporates the Sampled Consistency Initial Registration (SAC-IA) algorithm into its registration mechanism, enabling automatic keypoint extraction and coarse registration of point clouds. Optimizing the automatic coarse registration mechanism reduces initial errors during the registration process, enabling faster convergence of the Iterative Closest Point (ICP) fine registration algorithm.

[0071] 3. The present invention can more accurately collect the phenotypic characteristics of orchids; it can realize continuous iterative registration, and the registration result with the smallest error is regarded as the more accurate result.

[0072] 4. The present invention combines ground and background removal algorithms, discards point cloud data in the background area, and improves modeling quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0074] Figure 1 It is a schematic diagram of the execution steps of the method of the present invention.

[0075] FIG2(a) and FIG2(b) are schematic diagrams of clustering the “left front” perspective and separating the landmarks from the orchid body according to the present invention.

[0076] FIG2(c) and FIG2(d) are schematic diagrams of clustering the “left rear” perspective and separating the landmarks from the orchid body according to the present invention.

[0077] Figure 3(a) is a schematic diagram of extracting ISS key points in the present invention. The red point cloud is the orchid target point cloud, the green point cloud is the orchid source point cloud, and the blue points are the ISS key points in the point cloud.

[0078] Figure 3(b) is a schematic diagram of the initial registration using the SAC-IA algorithm in the present invention. The red point cloud is the orchid target point cloud, the green point cloud is the orchid source point cloud, and the blue point cloud is the orchid source point cloud after SAC-IA transformation.

[0079] Figure 4 This is a schematic diagram of the orchid after fine registration using the ICP algorithm. The red point cloud is the orchid target point cloud, and the green point cloud is the orchid source point cloud after ICP fine registration.

[0080] FIG5(a) and FIG5(b) are respectively the point cloud data diagrams of the two sides of the orchid after the registration of the present invention is completed.

[0081] Figure 6 This is the complete orchid point cloud data after landmark point registration in the present invention.

[0082] Figure 7 This is a block diagram of an orchid 3D modeling system based on SAC-IA and ICP registration according to an embodiment of the present invention.

[0083] Figure 8 2 is a structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0084] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0085] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0086] The Sample Consensus Initial Alignment (SAC-IA) algorithm relies on point feature histograms. The ultimate goal of SAC-IA registration is to find the optimal set of transformations among all possible transformations that minimize the error function. This transformation becomes the final registration transformation matrix, which is then used to obtain the registration result. However, the transformation matrix obtained by SAC-IA is imprecise, so it can only be used for coarse registration. For more accurate registration results, a fine registration algorithm is required.

[0087] Iterative Closest Point (ICP) algorithm is an algorithm based on data registration method and the closest point search method to solve the problem based on free-form surfaces.

[0088] like Figure 1 As shown, this embodiment provides a method for three-dimensional modeling of orchids based on SAC-IA and ICP registration, comprising the following steps:

[0089] (1) First, the depth camera is used to obtain orchid point cloud data from four perspectives: the left front side, the left back side, the right front side, and the right back side. Then, the orchid point cloud data is subjected to denoising and outlier removal operations. Next, downsampling is performed to reduce the number of orchid point cloud points to improve processing efficiency.

[0090] Furthermore, in this step, the specific operations of denoising the orchid point cloud data to remove outliers, downsampling, and removing the ground point cloud are as follows:

[0091] (1-1) Perform statistical analysis and filtering on the orchid point cloud. For each point in the orchid point cloud, calculate its distance to the nearby n points. sThe average distance, variance and other statistical parameters of the neighboring points. If a point deviates from the average value by more than k s Times the variance is considered an outlier and needs to be removed. s ,k s Parameters that need to be set for the program.

[0092] (1-2) Downsample the orchid point cloud using a 3D voxel grid. Calculate the average coordinates of all points within the voxel grid and use this average as the centroid coordinates. Ultimately, all points within the voxel grid can be represented by a single centroid, reducing the amount of point cloud data while preserving its shape.

[0093] (2) Based on the Random Sample Consensus (RANSAC) algorithm, the ground is removed from the point cloud data. The orchid is cut from the background, and the Euclidean distance clustering algorithm is used to divide the point cloud data of each view into two parts: the "orchid main point cloud" and the "mark point cloud". Figure 2(a)-Figure 2(d) The orchid body is shown isolated from the left front and left rear views.

[0094] Furthermore, in this step, the specific operations for removing the ground from the point cloud data and separating the orchid body and the landmarks are as follows:

[0095] (2-1) Randomly select a plane from the point cloud data and calculate the distance from each point in the point cloud to the plane. If the distance is less than a threshold d, the point is considered to be within the plane. If there are more than s points in the same plane, the plane is recorded as a ground point cloud and removed. s and d are parameters that need to be set in the program.

[0096] (2-2) For a point p in the point cloud e , search for the point n through KD-Tree algorithm e Nearby points. e Point to p e The Euclidean distance is less than the threshold r. e The points are grouped into the same category. e , r e Parameters that need to be set for the program.

[0097] (2-3) If the number of points in a class meets the set numerical range, the class is valid, otherwise it is discarded.

[0098] (2-4) Repeat (2-2) and (2-4) for the remaining points in the cloud until the entire point cloud has been searched.

[0099] (3) The “orchid main point cloud data” from two perspectives on the same side are set as the “source point cloud” and “target point cloud” respectively, and the internal shape descriptor ISS (Intrinsic Shape Signatures) is extracted as the key points. The extracted key points are shown in Figure 3(a), and their FPFH (Fast Point Feature Histograms) features are calculated. The two point clouds are subjected to the SAC-IA (Sample Consensus Initial Alignment) operation to achieve preliminary alignment based on key points, and the orchid point cloud that has been preliminarily aligned on one side is obtained, as shown in Figure 3(b). The red point cloud is the orchid target point cloud, the green point cloud is the orchid source point cloud, and the blue point cloud is the orchid source point cloud after SAC-IA transformation.

[0100] In this step, the specific operations of extracting ISS key points, calculating their FPFH features, and performing preliminary registration in step (3) are as follows:

[0101] (3-1) For each point p in the orchid point cloud i , establish a sphere with a radius of r with this point as the center. s spherical neighborhood of .

[0102] For each point p in the neighborhood j , calculate its distance to the center point p i The distance d ij , and assign each point its weight

[0103] (3-2) Calculate each point p i The covariance matrix of

[0104]

[0105] And record each point p i The three eigenvalues ​​of the covariance matrix are and For thresholds ∈1, ∈2, if the condition is satisfied Point p i This is the key point, denoted as p s . Where r s ,∈1,∈2 are parameters that need to be set by the program

[0106] (3-3) Calculate the FPFH features of the key point cloud data composed of key points. For each key point p in the point cloud s , at its radius r f Search k in the spherical neighborhood of f Point p t Calculate the SPFH feature

[0107] w=n s ×(p t -p s )×n s

[0108] where n s For p s Normal vector of a point

[0109] α=v·n t

[0110]

[0111] θ=arctan(w·n t ,n s ·n t )

[0112] where n t For p t Normal vector of a point

[0113]

[0114] where k f ,r f Parameters that need to be set for the program

[0115] (3-4) Randomly select n (n>=3) pairs of matching points in the orchid point cloud based on FPFH features;

[0116] (3-5) In this matching case, SVD (Singular Value Decomposition) is used to obtain the transformation matrix required for the solution;

[0117] (3-6) Calculate the matching error after transformation, repeat (3-4), (3-5), and (3-6) until the matching error meets the set conditions, and retain the transformation matrix with the smallest error.

[0118] (4) The orchid point cloud that has been preliminarily registered in step (3) is finely registered using the trimmed iterative closest point (TRICP) algorithm to obtain a finely registered point cloud on the same side (including landmark points).

[0119] After one side is processed, perform the same (2)(3)(4) operations on the other side.

[0120] In step (4), the specific operation of using the trimmed closest point iteration (trimmed ICP) algorithm for fine registration is:

[0121] (4-1) Calculate the centroid p of the source point cloud and target point cloud after rough registration s , p t :

[0122]

[0123] where n s ,n t is the number of points in the source point cloud and the target point cloud.

[0124] Decentralize the point cloud to obtain

[0125]

[0126] (4-2) SVD solves the rotation matrix R * , translation matrix T *

[0127]

[0128] T * =p s -R * p s

[0129] (4-3) The obtained rotation matrix R * , translation matrix T * Acting on the source point cloud, for each point p of the point cloud after rotation and translation i Use KD-Tree to search for the nearest point (corresponding point) of the point in the target point cloud Form a corresponding relationship. And calculate the square of the distance d between them. i 2 Sort all corresponding relationships according to the distance between their point pairs and take the first n u (n u ≤n s ) correspondences are summed up. If the termination condition is met, the program is exited, otherwise (4-1)(4-2)(4-3) is repeated. u Parameters that need to be set for the program

[0130] Termination conditions:

[0131] (1) When the maximum number of iterations is reached, the iteration is terminated.

[0132] (2) The error is less than the set value.

[0133] (3) The change in the rotation and translation matrix generated by the previous iteration is small enough.

[0134] Finally, the transformation matrix with the smallest registration error is obtained. Apply the transformation matrix and rotation matrix to the source point cloud to obtain the orchid point cloud model on one side after registration. The same is true for the other side. Figure 4 ,Figure 5(a) and 5(b) show the point cloud data on both sides after ,registration is completed.

[0135] (5) Extract the color information of the four landmark points in the point clouds on both sides of the orchid and calculate their center coordinates. Calculate the rotation transformation matrix required to transform the landmark points in the point clouds on both sides to the same coordinate system. Use the transformation matrix to merge the point cloud data on both sides to obtain a complete orchid model.

[0136] The specific operations of identifying landmark points and merging the point clouds on both sides are:

[0137] (5-1) According to the color information of the marker points, the point cloud data of the marker points of different colors are filtered out from the point cloud data on both sides.

[0138] (5-2) For each landmark point, calculate its center coordinates.

[0139]

[0140] Among them, c is the center coordinate value, m i are all the points that make up the landmark point, and n is the number of points in the point cloud of a single landmark point.

[0141] (5-3) The point cloud models on the left and right sides are set as "source point cloud" and "target point cloud" respectively. By using SVD to solve the coordinates of the landmark points on both sides in the two coordinate systems, the rotation matrix and translation matrix of the source point cloud transformed to the target point cloud coordinate system are obtained. The rotation matrix and translation matrix are applied to the source point cloud to align it with the target point cloud, and finally the complete point cloud model is obtained after the two sides are merged, as shown in the figure below. Figure 6 shown.

[0142] The method of the present invention is based on point cloud features such as the internal shape descriptor (ISS) and the fast point feature histogram (FPFH) descriptor, combined with the sampling consistency initial registration (SAC-IA) algorithm and the iterative closest point (ICP) algorithm, and assisted by landmark points to achieve semi-automatic orchid point cloud modeling, thereby improving the efficiency of orchid modeling and enhancing the accuracy and stability of orchid modeling.

[0143] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.

[0144] Based on the same concept as the orchid 3D modeling method based on SAC-IA and ICP registration in the above-mentioned embodiment, the present invention also provides an orchid 3D modeling system based on SAC-IA and ICP registration, which can be used to execute the above-mentioned orchid 3D modeling method based on SAC-IA and ICP registration. For ease of explanation, the structural diagram of the embodiment of the orchid 3D modeling system based on SAC-IA and ICP registration only shows the parts related to the embodiment of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation of the device, and it may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0145] See also Figure 7 In another embodiment of the present application, a three-dimensional orchid modeling system 100 based on SAC-IA and ICP registration is provided, the system comprising a data acquisition module 101, a data partitioning module 102, a coarse registration module 103, a fine registration module 104, and a data merging module 105;

[0146] The data acquisition module 101 is used to acquire orchid point cloud data from four perspectives: the left front side, the left rear side, the right front side, and the right rear side of the orchid, and pre-process the orchid point cloud data to reduce the number of orchid point cloud points;

[0147] The data segmentation module 102 is used to remove the ground from the orchid point cloud data based on the random sampling consensus RANSAC algorithm, cut the orchid from the background, and use a clustering algorithm to divide the orchid point cloud data of each perspective into two parts: "orchid main point cloud" and "mark point cloud";

[0148] The coarse registration module 103 is used to set the "orchid main body point cloud data" from two perspectives on the same side as the "source point cloud" and the "target point cloud" respectively, extract the internal shape descriptor ISS from the "source point cloud" and the "target point cloud" as key points, and calculate their fast point histogram features FPFH, and perform a sampling consistency initial registration (SAC-IA) operation on the two point clouds to achieve preliminary registration based on the key points, thereby obtaining a preliminary registered orchid point cloud on one side;

[0149] The fine registration module 104 is configured to finely register the preliminarily registered orchid point cloud using the Trimmed Closest Point Iterative TRICP algorithm to obtain an orchid point cloud with color-precise registration of the marker points on the same side; after processing one side, the same steps are performed on the other side to obtain an orchid point cloud with color-precise registration of the marker points on the other side;

[0150] The data merging module 105 is used to extract the color information of the four marker points in the point clouds on both sides of the orchid respectively, and calculate their center coordinates; calculate the rotation transformation matrix required to convert the marker points in the point clouds on both sides to the same coordinate system, and use the transformation matrix to merge the point cloud data on both sides to obtain a complete orchid model.

[0151] It should be noted that the orchid 3D modeling system based on SAC-IA and ICP registration of the present invention corresponds one to one with the orchid 3D modeling method based on SAC-IA and ICP registration of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the orchid 3D modeling method based on SAC-IA and ICP registration are applicable to the embodiment of orchid 3D modeling based on SAC-IA and ICP registration. For specific contents, please refer to the description in the embodiment of the method of the present invention. No further details will be given here. This is hereby declared.

[0152] In addition, in the implementation of the orchid 3D modeling system based on SAC-IA and ICP registration in the above embodiment, the logical division of each program module is only an example. In actual application, the above functions can be assigned to different program modules as needed, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation. That is, the internal structure of the orchid 3D modeling system based on SAC-IA and ICP registration is divided into different program modules to complete all or part of the functions described above.

[0153] See also Figure 8 In one embodiment, an electronic device for implementing a three-dimensional orchid modeling method based on SAC-IA and ICP registration is provided. The electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as a three-dimensional orchid modeling program 203 based on SAC-IA and ICP registration.

[0154] The first memory 202 includes at least one type of readable storage medium, including a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the first memory 202 may be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 may also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 200. Furthermore, the first memory 202 may include both an internal storage unit of the electronic device 200 and an external storage device. The first memory 202 can be used not only to store application software and various types of data installed in the electronic device 200, such as the code of the orchid 3D modeling program 203 based on SAC-IA and ICP registration, but can also be used to temporarily store data that has been output or is about to be output.

[0155] In some embodiments, the first processor 201 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The first processor 201 is the control core (Control Unit) of the electronic device, connecting the various components of the entire electronic device using various interfaces and lines, and executing or executing programs or modules stored in the first memory 202, as well as calling data stored in the first memory 202, to perform various functions of the electronic device 200 and process data.

[0156] Figure 8 Only the electronic device with components is shown, and it can be understood by those skilled in the art that Figure 8 The structure shown does not constitute a limitation on the electronic device 200 , and the electronic device 200 may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0157] The orchid 3D modeling program 203 based on SAC-IA and ICP registration stored in the first memory 202 of the electronic device 200 is a combination of multiple instructions. When executed in the first processor 201, it can achieve the following:

[0158] Obtain orchid point cloud data from four perspectives: the left front side, the left back side, the right front side, and the right back side of the orchid, and pre-process the orchid point cloud data to reduce the number of orchid point cloud points;

[0159] Based on the random sampling consistency RANSAC algorithm, the ground is removed from the orchid point cloud data, the orchid is cut from the background, and the clustering algorithm is used to divide the orchid point cloud data of each perspective into two parts: "orchid main point cloud" and "landmark point cloud";

[0160] The "orchid point cloud data" from two perspectives on the same side are set as the "source point cloud" and "target point cloud" respectively. The internal shape descriptors (ISS) are extracted from the "source point cloud" and "target point cloud" as key points, and their fast point histogram features (FPFH) are calculated. The two point clouds are then subjected to a sampling consistency initial registration (SAC-IA) operation to achieve a preliminary registration based on key points, resulting in a preliminarily registered orchid point cloud on one side.

[0161] The orchid point cloud that has been preliminarily registered is finely registered using the Trimmed Closest Point Iterative TRICP algorithm to obtain an orchid point cloud with color-fine registration of landmark points on the same side; after processing one side, the same steps are followed for the other side to obtain an orchid point cloud with color-fine registration of landmark points on the other side;

[0162] The color information of the four landmark points in the point clouds on both sides of the orchid is extracted respectively, and their center coordinates are calculated; the rotation transformation matrix required to convert the landmark points in the point clouds on both sides to the same coordinate system is calculated, and the point cloud data on both sides are merged using the transformation matrix to obtain a complete orchid model.

[0163] Furthermore, if the modules / units integrated in the electronic device 200 are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0164] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0165] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A 3D modeling method for orchids based on SAC-IA and ICP registration, characterized by: The steps include: (1) Obtain orchid point cloud data from four perspectives: the left front side, the left back side, the right front side, and the right back side, and pre-process the orchid point cloud data to reduce the number of orchid point cloud points; (2) Based on the random sampling consistency RANSAC algorithm, the ground is removed from the orchid point cloud data, the orchid is cut from the background, and the orchid point cloud data of each perspective is divided into two parts: "orchid main point cloud" and "landmark point cloud" using a clustering algorithm; (3) The "orchid main body point cloud data" from two perspectives on the same side are set as the "source point cloud" and "target point cloud" respectively. The internal shape descriptor ISS is extracted from the "source point cloud" and "target point cloud" as key points, and their fast point histogram features FPFH are calculated. The two point clouds are subjected to the sampling consistency initial alignment SAC-IA operation to achieve preliminary alignment based on key points, and the orchid point cloud that has been preliminarily aligned on one side is obtained. (4) The orchid point cloud that has been preliminarily registered is finely registered using the trimmed closest point iterative TRICP algorithm to obtain an orchid point cloud with color-precise registration of the landmark points on the same side; after processing one side, the same steps are followed for the other side to obtain an orchid point cloud with color-precise registration of the landmark points on the other side; (5) The color information of the four landmark points in the point clouds on both sides of the orchid is extracted respectively, and their center coordinates are calculated; the rotation transformation matrix required to transform the landmark points in the point clouds on both sides into the same coordinate system is calculated, and the point cloud data on both sides are merged using the transformation matrix to obtain a complete orchid model.

2. The orchid 3D modeling method based on SAC-IA and ICP registration according to claim 1, characterized in that: In step (1), the orchid point cloud data is preprocessed to reduce the number of orchid point cloud points, specifically: (1-1) Perform statistical analysis and filtering on the orchid point cloud. For each point in the orchid point cloud, calculate its distance to the nearby n points. s The statistical parameters of the average distance and variance of neighboring points are: s times the variance is considered an outlier and needs to be removed; where n s ,k s is a pre-set threshold; (1-2) Use a three-dimensional voxel grid to downsample the orchid point cloud, calculate the average value of the coordinates of all points in the voxel grid, and use the coordinates as the centroid coordinates. Finally, all points in the voxel grid are represented by one centroid point, thereby reducing the amount of point cloud data while maintaining the shape characteristics of the point cloud.

3. The orchid 3D modeling method based on SAC-IA and ICP registration according to claim 1, characterized in that: Step (2) is specifically as follows: (2-1) Randomly select a plane from the orchid point cloud data and calculate the distance from each point in the orchid point cloud data to the plane. If the distance is less than the threshold d, the point is considered to be in the plane; if there are more than s points in the same plane, the plane is recorded as a ground point cloud and removed; Where s and d are pre-set parameters; (2-2) For a point p in the orchid point cloud e , search for the point n through KD-Tree algorithm e Nearby points, judge this n e Point to p e The Euclidean distance is less than the threshold r. e The points are grouped into the same category; where n e , r e is a pre-set parameter; (2-3) If the number of points in a class meets the set value range, the class is valid, otherwise it is discarded; (2-4) Repeat the same method for the remaining points in the cloud until the entire point cloud has been searched.

4. The orchid 3D modeling method based on SAC-IA and ICP registration according to claim 1, characterized in that: In step (3), the internal shape descriptor ISS is extracted from the "source point cloud" and the "target point cloud" as the key point, and its fast point histogram feature FPFH is calculated, and the two point clouds are subjected to the sampling consistency initial alignment SAC-IA operation, specifically: (3-1) For each point p in the orchid point cloud i , establish a sphere with a radius of r with this point as the center. s spherical neighborhood; for each p in the neighborhood j Point, calculate p j To the center point p i The distance d ij , and assign each point its weight (3-2) Calculate each point p i The covariance matrix of And record each point p i The three eigenvalues ​​of the covariance matrix are and For thresholds ∈1, ∈2, if the condition is satisfied Point p i This is the key point, denoted as p s ; where r s ,∈1,∈2 are pre-set parameters; (3-3) Calculate the FPFH features of the key point cloud data composed of key points. For each key point p in the point cloud s , at its radius r f Search k in the spherical neighborhood of f Point p t , calculate the SPFH feature w=n s ×(p t -p s )×n s where n s For p s Normal vector of a point α=v·n t θ=arctan(w·n t ,n s ·n t ) where n t For p t Normal vector of a point where k f ,r f Parameters that need to be set for the program (3-4) Randomly select n pairs of matching points in the orchid point cloud based on FPFH features; (3-5) In this matching case, singular value decomposition (SVD) is used to obtain the transformation matrix required for the solution; (3-6) Calculate the matching error after transformation, repeat (3-4), (3-5), and (3-6) until the matching error meets the set conditions, and retain the transformation matrix with the smallest error.

5. The orchid 3D modeling method based on SAC-IA and ICP registration according to claim 1, characterized in that: In step (4), the specific operation of using the trimmed nearest point iterative algorithm to perform precise registration is as follows: (4-1) Calculate the centroid p of the source point cloud and target point cloud after rough registration s , p t : where n s ,n t is the number of points in the source point cloud and the target point cloud; Decentralize the point cloud to obtain (4-2) SVD solves the rotation matrix R * , translation matrix T * T * =p s -R * p s (4-3) The obtained rotation matrix R * , translation matrix T * Acting on the source point cloud, for each point p of the point cloud after rotation and translation i Use KD-Tree to search for the nearest point in the target point cloud Form a corresponding relationship and calculate the square of the distance d between them i 2 , sort all corresponding relationships according to the distance between their point pairs, and take the first n u (n u ≤n s ) The distances of the corresponding relationships are summed up. If the termination condition is met, the program is exited, otherwise (4-1)(4-2)(4-3) are repeated, where n u are pre-set parameters.

6. The orchid 3D modeling method based on SAC-IA and ICP registration according to claim 5, characterized in that: The termination condition is to meet one of the following conditions: (1) The maximum number of iterations is reached and the iteration is terminated; (2) The error is less than the set value; (3) The change in the rotation and translation matrix generated by the previous iteration is small enough.

7. The orchid 3D modeling method based on SAC-IA and ICP registration according to claim 1, characterized in that: In step (5), the specific operations of identifying the landmark points and merging the point clouds on both sides are as follows: (5-1) According to the color information of the marker points, the point cloud data of the marker points of different colors are filtered out from the point cloud data on both sides; (5-2) For each landmark point, calculate its center coordinates; Among them, c is the center coordinate value, m i are all the points that make up the landmark point, and n is the number of points in the point cloud of a single landmark point; (5-3) The point cloud models on the left and right sides are set as "source point cloud" and "target point cloud" respectively. By using SVD to solve the coordinates of the landmark points on both sides in the two coordinate systems, the rotation matrix and translation matrix of the source point cloud transformed to the target point cloud coordinate system are obtained; the rotation matrix and translation matrix are applied to the source point cloud to align it with the target point cloud, and finally a complete point cloud model is obtained after the two sides are merged.

8. The orchid 3D modeling system based on SAC-IA and ICP registration is characterized by: The orchid three-dimensional modeling method based on SAC-IA and ICP registration applied to any one of claims 1 to 7 comprises a data acquisition module, a data partitioning module, a coarse registration module, a fine registration module, and a data merging module; The data acquisition module is used to acquire orchid point cloud data from four perspectives: the left front side, the left rear side, the right front side, and the right rear side of the orchid, and pre-process the orchid point cloud data to reduce the number of orchid point cloud points; The data segmentation module is used to remove the ground from the orchid point cloud data based on the random sampling consensus RANSAC algorithm, cut the orchid from the background, and use the clustering algorithm to divide the orchid point cloud data of each perspective into two parts: "orchid main point cloud" and "mark point cloud"; The coarse registration module is used to set the "orchid main body point cloud data" from two perspectives on the same side as the "source point cloud" and "target point cloud" respectively, extract the internal shape descriptor ISS from the "source point cloud" and "target point cloud" as key points, and calculate their fast point histogram features FPFH. The two point clouds are subjected to a sampling consistency initial registration (SAC-IA) operation to achieve preliminary registration based on key points, thereby obtaining a preliminary registered orchid point cloud on one side; The fine registration module is used to finely register the preliminarily registered orchid point cloud using the trimmed closest point iterative TRICP algorithm to obtain an orchid point cloud with color-precise registration of the marker points on the same side; after processing one side, the same steps are followed for the other side to obtain an orchid point cloud with color-precise registration of the marker points on the other side; The data merging module is used to extract the color information of the four marker points in the point clouds on both sides of the orchid and calculate their center coordinates; calculate the rotation transformation matrix required to convert the marker points in the point clouds on both sides to the same coordinate system, and use the transformation matrix to merge the point cloud data on both sides to obtain a complete orchid model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the orchid three-dimensional modeling method based on SAC-IA and ICP registration as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the orchid three-dimensional modeling method based on SAC-IA and ICP registration described in any one of claims 1 to 7 is implemented.

Images