Semantic information-based iterative nearest point methods, apparatus, devices, and media

Abstract

This invention discloses an iterative closest point method, apparatus, computer device, and storage medium based on semantic information. The method includes two parts: finding the nearest corresponding point and aligning the point cloud. It improves the implementation of the nearest corresponding point finding part in the iterative closest point (ICP) method by using the semantic information of the point cloud to segment the sub-point clouds sequentially to find the nearest corresponding point, thereby improving the accuracy and efficiency of finding the nearest corresponding point. In the point cloud alignment part, it uses the nearest corresponding points segmented by semantic information for registration, solving the problem of getting trapped in local optima during the registration process, and improving the registration success rate when the rotation angle between point clouds is large.

Description

Technical Field

[0001] This invention relates to the field of point cloud registration technology, and specifically to an iterative nearest point method, apparatus, computer device, and storage medium based on semantic information. Background Technology

[0002] In recent years, 3D point cloud registration technology has attracted widespread interest and research in many fields. 3D point clouds are spatial data composed of a large number of discrete points, which can accurately describe the 3D shape and geometric features of an object. However, limitations in point cloud data acquisition methods, such as the common use of lasers to acquire point clouds of target objects, mean that a single acquisition can only obtain a portion of the object's point cloud within the laser-visible area, and cannot directly obtain the complete 3D point cloud of the object. To achieve target object reconstruction and localization, it is necessary to register point clouds acquired multiple times.

[0003] Currently, a common scenario for point cloud registration is to acquire point clouds of the same target object at nearby locations, obtaining point cloud data of the object from two different angles. Registration methods are then used to find the overlapping portion between these two point cloud datasets and calculate their 3D spatial information, thus completing the point cloud registration process. Researchers have proposed the Iterative Closest Point (ICP) method to implement this registration process and have continuously improved it in subsequent research to enhance the speed and accuracy of registration.

[0004] However, the ICP method and its improved methods achieve good registration results when the rotation angle between point clouds is small. But when the rotation angle between point clouds is large (more than 90 degrees), it often gets trapped in local optima, leading to registration errors.

[0005] In summary, existing registration methods such as ICP still have certain limitations when handling point cloud registration with large rotation angles. The main problem is that these methods cannot fully utilize the semantic information of the point cloud, causing the registration process to stop when a local optimum is reached, thus yielding erroneous results. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing point cloud registration methods for handling large rotation angles, and to provide an iterative nearest point method, apparatus, computer device, and storage medium based on semantic information.

[0007] The first objective of this invention can be achieved by adopting the following technical solution:

[0008] An iterative nearest-point method based on semantic information is proposed. This method includes two parts: finding the nearest corresponding point and aligning the point cloud. It improves the implementation of the nearest corresponding point finding part in the ICP method, utilizes the semantic information of the point cloud for registration, solves the problem of getting trapped in local optima during the registration process, and improves the registration success rate when the rotation angle between point clouds is large. The method includes the following steps:

[0009] S1. Denote the two point clouds of the target object as source point cloud P and target point cloud Q. Preprocess the two point clouds by segmenting them using semantic information. Sub-point clouds with the same semantic information are given the same index k, resulting in a total of K semantically segmented sub-source point clouds P. k Hide target point cloud Q k Let P be the k-th sub-source point cloud. k The number of points is N k Initialize the optimal rotation matrix R as a 3x3 identity matrix, initialize the optimal translation matrix t as a 3x1 zero vector, and initialize the iteration counter n to 1;

[0010] S2, Find the k-th sub-source point cloud P k and the k-th sub-target point cloud Q k The nearest corresponding point: for the k-th sub-source point cloud P k The i-th point p in k,i ∈P k Perform the transformation of the current optimal rotation matrix R and optimal translation matrix t, and then find the k-th sub-target point cloud Q. k The middle satisfies the condition of this point. The nearest corresponding point q k,i Finally, N will be found k q k,i Composition of sub-source point cloud P k The nearest corresponding point target point cloud Q k ;

[0011] S3. Repeat step S2 for a total of K times to obtain the K semantically segmented sub-source point clouds P in the first iteration. k The nearest corresponding point cloud Q after transformation by the current optimal rotation matrix R and optimal translation matrix t k ;

[0012] S4. Update the optimal rotation and translation matrices: For the K semantically segmented sub-source point clouds P k The target point cloud Q with the nearest corresponding point k Solve using SVD decomposition. Calculate and update the current optimal rotation matrix R and translation matrix t, and calculate the current registration error.

[0013] S5. Repeat S2-S4, updating the iteration calculator n = n+1 each time. When the registration error E calculated in step S4 is less than the set threshold or the number of iterations n exceeds the set maximum value during the nth iteration, the registration ends and the optimal rotation matrix R and translation matrix t are output.

[0014] Further, in step S2, the nearest corresponding point in the sub-target point cloud with the same semantic information is found by storing the coordinates of each point in the target point cloud using a kd-tree data structure. The advantage of kd-trees is that they use spatial binary search to store points in the point cloud, reducing the average time complexity of searching for the nearest point from O(n) to O(logn), significantly improving the speed of searching for the nearest point when the number of points in the point cloud is large. The disadvantage of kd-trees is their slow generation speed, but since the target point cloud remains fixed during registration, storing it using a kd-tree data structure will not negatively impact registration efficiency. For each point in the source point cloud after transformation by the current optimal rotation matrix R and optimal translation matrix t, the nearest corresponding point is quickly searched in the kd-tree to form the nearest corresponding point target point cloud.

[0015] Furthermore, in step S4 This is a nonlinear optimization problem, which can be solved using SVD decomposition. The solution method is as follows: Let Perform SVD decomposition on W: W = UΣV T Solve for U and V, where U and V are 3x3 unitary matrices and Σ is a 3x3 non-negative real diagonal matrix. Calculate the optimal rotation and translation matrix R = VU for the nth iteration. T , in p′ k,i The transpose of the matrix R. If the determinant of the rotation matrix R is negative, then the current rotation transformation is a mirror transformation, and the third column element of the rotation matrix R needs to be changed to its opposite to revert to the rotation transformation.

[0016] The second objective of this invention can be achieved by adopting the following technical solution:

[0017] An iterative nearest point apparatus for executing the above-described semantic information-based iterative nearest point method includes a preprocessing module, a search and transformation module, a first iteration module, an update module, and a second iteration module.

[0018] The preprocessing module is used to denote the two point clouds of the target object as source point cloud P and target point cloud Q. It preprocesses the two point clouds by using their semantic information to segment them. Sub-point clouds with the same semantic information use the same index k, resulting in a total of K semantically segmented sub-source point clouds P.k Hide target point cloud Q k Let P be the k-th sub-source point cloud. k The number of midpoints is N k Initialize the optimal rotation matrix R as a 3x3 identity matrix, initialize the optimal translation matrix t as a 3x1 zero vector, and initialize the iteration counter n to 1;

[0019] The search and transformation module is used to find the k-th sub-source point cloud P. k and the k-th sub-target point cloud Q k The nearest corresponding point: for the k-th sub-source point cloud P k The i-th point p in k,i ∈P k Perform the transformation of the current optimal rotation matrix R and optimal translation matrix t, and then transform the k-th sub-target point cloud Q. k For each q∈Q k The point, find with p k,i satisfy The nearest corresponding point q k,i Finally, N will be found k q k,i Composition of sub-source point cloud P k The nearest corresponding point target point cloud Q k ;

[0020] The first iteration module repeatedly calls the lookup and transformation module K times to obtain K semantically segmented sub-source point clouds P in the first iteration. k The nearest corresponding point cloud Q after transformation by the current optimal rotation matrix R and optimal translation matrix t k ;

[0021] The update module is used to update the optimal rotation and translation matrices for the K semantically segmented sub-source point clouds P. k The target point cloud Q with the nearest corresponding point k Solve using SVD decomposition. Calculate and update the current optimal rotation matrix R and translation matrix t, and calculate the current registration error.

[0022] The second iteration module repeatedly calls the search transformation module, the first iteration module, and the update module. Each time, the iteration calculator n = n + 1 is updated. When the registration error E calculated in the update module is less than the set threshold or the number of iterations n exceeds the set maximum value during the nth iteration, the registration ends and the optimal rotation matrix R and translation matrix t are output.

[0023] A third objective of the present invention is to provide a computer device including a processor and a memory for storing a processor-executable program, wherein when the processor executes the program stored in the memory, it implements the above-described iterative nearest point method based on semantic information.

[0024] A fourth objective of the present invention is a storage medium storing a program that, when executed by a processor, implements the above-described iterative nearest point method based on semantic information.

[0025] The present invention has the following advantages and effects compared with the prior art:

[0026] 1. This invention proposes an iterative nearest-point method based on semantic information. This method searches for sub-source point clouds and sub-target point clouds with the same semantic information after semantic segmentation when searching for the nearest point, and uses the data results of a kd-tree to store each sub-target point cloud, thereby improving search speed and finding a more matching nearest corresponding point.

[0027] 2. The present invention proposes an iterative nearest point method based on semantic information. This method jointly optimizes the corresponding point clouds with different semantic information when updating the optimal rotation matrix and translation matrix. The optimal solution is obtained by using the SVD decomposition algorithm proposed in this patent. This improves the registration speed when solving the registration problem with large rotation angles between point clouds. Attached Figure Description

[0028] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0029] Figure 1 This is a flowchart of an iterative nearest point method based on semantic information disclosed in this invention;

[0030] Figure 2 This is a schematic diagram of the sub-point cloud after semantic segmentation of the Stanford rabbit head point cloud in Embodiment 1 of the present invention;

[0031] Figure 3 This is a comparison image of the registration results of the Stanford rabbit head point cloud in Embodiment 1 of the present invention;

[0032] Figure 4 This is a schematic diagram of the sub-point cloud after semantic segmentation of the point cloud of the monkey head of Susanna in Embodiment 2 of the present invention;

[0033] Figure 5 This is a comparison image of the registration results of the point cloud of Susanna's monkey head in Embodiment 2 of the present invention;

[0034] Figure 6This is a structural block diagram of the iterative nearest point device in Embodiment 3 of the present invention;

[0035] Figure 7 This is a structural block diagram of the computer device in Embodiment 4 of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Example 1

[0038] Existing point cloud registration methods still have certain limitations when dealing with point cloud registration with large rotation angles. This embodiment provides an iterative nearest point method based on semantic information, which uses the semantic information of the point cloud for registration to solve the problem of getting trapped in local optima during the registration process and improve the registration accuracy.

[0039] Figure 1 This is a flowchart of an iterative nearest point method based on semantic information disclosed in this embodiment. The specific steps are as follows:

[0040] S1. Perform semantic segmentation on the point cloud of an arbitrary rabbit head named "Stanford" (hereinafter referred to as: Stanford rabbit head), segmenting it into a sub-source point cloud P1 of "Stanford rabbit head left ear" (derived from point set p). 1,i ∈P1(i=1,...,N1)), “Stanford Rabbit Head and Face” sub-source point cloud P2 (composed of point set p) 2,i ∈P2 (i=1,...,N2)), “Stanford rabbit head right ear” sub-source point cloud P3 (composed of point set p) 3,i The sub-target point cloud is composed of three parts (i = 1, ..., N3). The corresponding sub-target point cloud is formed by rotating the sub-source point cloud through the simulated rotation matrix R. GT After transformation, the sub-target point cloud Q1 is obtained from R. GT P1 transformation yields the sub-target point cloud Q2, which is obtained from R. GT P2 transformation is obtained, and the sub-target point cloud Q3 is obtained from R. GT The P3 transformation is obtained. The optimal rotation matrix R is initialized as a 3x3 identity matrix, the optimal translation matrix t is initialized as a 3x1 zero vector, and the iteration counter n is initialized as 1.

[0041] S2, Find the k-th sub-source point cloud P k and the k-th sub-target point cloud Q kThe nearest corresponding point: for the k-th sub-source point cloud P k The i-th point p in k,i ∈P k The optimal rotation matrix R and the optimal translation matrix t are transformed to obtain the transformed point cloud RP. k +t. Move the subtarget point cloud Q. k The coordinates of each point in the transformed sub-source point cloud (RP) are stored using a kd-tree data structure. k Rp at each point +t k,i +t∈RP k +t(i=1,...,N k ), quickly search for Q in the kd-tree k The middle satisfies the condition of this point. The nearest corresponding point q k,i The sub-source point cloud P is composed of k The nearest corresponding point target point cloud Q k .

[0042] S3. Repeat step S2 for a total of 3 times to obtain the nearest corresponding target point clouds Q1, Q2 and Q3 after the transformation of the three semantically segmented sub-source point clouds P1, P2 and P3 in the first iteration by the current optimal rotation matrix R and the optimal translation matrix t.

[0043] S4. Update the optimal rotation and translation matrices: For each semantic segmentation sub-source point cloud P k The target point cloud Q with the nearest corresponding point k Solve using SVD decomposition. Calculate and update the current optimal rotation matrix R and translation matrix t, and calculate the current registration error.

[0044] S5. Repeat steps S2-S4, updating the iterative calculator n = n + 1 each time. When the registration error E calculated in step S4 is less than 10 during the nth iteration... -10 If the number of iterations n exceeds 100, the registration ends and the optimal rotation matrix R and translation matrix t are output.

[0045] In step S1 of this embodiment, the schematic diagram of the sub-point cloud after semantic segmentation of the Stanford rabbit head point cloud is shown below. Figure 2 As shown. The total number of points in the Stanford rabbit head point cloud is 711. After semantic segmentation, the number of points for "Stanford rabbit head left ear" is 167, the number of points for "Stanford rabbit head face" is 418, and the number of points for "Stanford rabbit head right ear" is 186. The "Stanford rabbit head" in this embodiment is only one of the test data of this invention, and this invention is effective in point cloud registration with semantic information.

[0046] In step S1 of this embodiment, R GTIt can be generated from Euler angles with 3 degrees of freedom, ranging from 0 to 2π. R GT The generating expression is as follows:

[0047]

[0048] Where θ is the roll angle about the x-axis, α is the pitch angle about the y-axis, and β is the yaw angle about the z-axis. In step S1 of this embodiment, the point cloud is processed by a simulated rotation matrix R. GT The transformation, specifically, involves multiplying the column vector coordinates of each point in the point cloud by the simulated rotation matrix R. GT The new point cloud formed by the calculated new point coordinates is the point cloud after being rotated by the simulated rotation matrix R. GT The transformed point cloud.

[0049] In step S2 of this embodiment, the transformation of the current optimal rotation matrix R and the optimal translation matrix t is specifically performed by multiplying the column vector coordinates of each point in the point cloud by the rotation matrix R on the left, and then adding the translation matrix t to the new column vector. The new point cloud formed by the calculated new point coordinates is the point cloud after the point cloud has been transformed by the rotation matrix R and the translation matrix t.

[0050] The number of point cloud semantic segmentation categories in step S1 of this embodiment can vary according to the semantic information of the point cloud, and the number of repetitions in step S3 will also change accordingly.

[0051] The SVD decomposition solution method in step S4 of this embodiment is as follows: Let Perform SVD decomposition on W: W = UΣV T Solve for U and V, and calculate the current optimal rotation matrix and translation matrix R = VU. T , in If the determinant of the rotation matrix R is negative, change the third column element of the rotation matrix R to its opposite.

[0052] In step S1 of this embodiment, R GT We can start from 0 in each of the three degrees of freedom θ, α, and β. Taking 10 points as the step size, we combine them into 1000 simulated rotation matrices R. GT This is used to verify the performance of the method proposed in this patent. The verification result shows that the iterative nearest point method based on semantic information proposed in this patent achieves a registration accuracy of 91.8% for the Stanford rabbit head point cloud used in this embodiment, while the registration accuracy of the ordinary iterative nearest point algorithm is 9%. Where θ = 0, Simulation results at that time are as follows Figure 3As shown, from left to right, are: a point cloud diagram before registration, the registration result using the method of this patent, and the registration result using existing methods. Simulation experiments verify that the performance of the method proposed in this patent is far superior to existing methods.

[0053] Example 2

[0054] This embodiment discloses a flowchart of an iterative nearest point method based on semantic information, and the specific steps are as follows:

[0055] S1. Perform semantic segmentation on the point cloud of any rabbit head named "Susanna" (hereinafter referred to as: Susanna rabbit head), segmenting it into a sub-source point cloud P1 of "Susanna monkey head left ear" (derived from point set p). 1,i ∈P1 (i=1,...,N1)), “Susanna monkey head and face” sub-source point cloud P2 (composed of point set p 2,i ∈P2 (i=1,...,N2)), “Susanna monkey head right ear” sub-source point cloud P3 (composed of point set p) 3,i The process consists of three parts: ∈P3 (i = 1, ..., N3). The optimal rotation matrix R is initialized as a 3x3 identity matrix, the optimal translation matrix t is initialized as a 3x1 zero vector, and the iteration counter n is initialized to 1.

[0056] S2. Following step S2 in Example 1, perform each process to find the kth sub-source point cloud P obtained from semantic segmentation in step S1. k and the k-th sub-target point cloud Q k The nearest corresponding point is used to obtain the sub-source point cloud P. k The nearest corresponding point target point cloud Q k .

[0057] S3. Perform each process according to step S3 in Example 1 to obtain the nearest corresponding target point cloud Q1, Q2 and Q3.

[0058] S4. Perform each process according to step S4 in Example 1, calculate and update the current optimal rotation matrix R and translation matrix t, and calculate the current registration error.

[0059] S5. Execute each process according to step S5 in Example 1, and output the optimal registration rotation matrix R and translation matrix t.

[0060] Except for the input target object point cloud in step S1, which differs from that in Example 1, all other steps in this embodiment are identical to those in Example 1. A schematic diagram of the sub-point cloud after semantic segmentation of the Susanna monkey head point cloud in step S1 of this embodiment is shown below. Figure 4As shown, the total number of points in the Susanna monkey head point cloud is 7958. After semantic segmentation, the number of points for "Stanford rabbit head left ear" is 978, the number of points for "Stanford rabbit head face" is 6085, and the number of points for "Stanford rabbit head right ear" is 954.

[0061] The performance verification method in this embodiment is the same as in Embodiment 1. It uses 1000 simulated rotation matrices R... GT Testing and verification showed that the iterative nearest point method based on semantic information proposed in this patent achieved a registration accuracy of 82.8% for the Susannah monkey head point cloud used in this embodiment, while the registration accuracy of the ordinary iterative nearest point algorithm was 4.6%. Where θ = 0, Simulation results at that time are as follows Figure 5 As shown, from left to right, are: a point cloud diagram before registration, the registration result using the method of this patent, and the registration result using existing methods. Simulation experiments verify that the performance of the method proposed in this patent is far superior to existing methods.

[0062] In summary, existing Iterative Closest Point (ICP) methods suffer from the problem of easily getting trapped in local optima and leading to registration errors when dealing with registration problems involving large rotation angles between point clouds. This embodiment utilizes the semantic information of the point cloud for registration. The sub-point clouds segmented using this semantic information are used to sequentially search for the nearest corresponding point, improving the accuracy and efficiency of the nearest corresponding point search. Furthermore, the nearest corresponding point segmented using semantic information is used for registration in the aligned point cloud portion, resolving the problem of getting trapped in local optima and improving registration success even with large rotation angles between point clouds.

[0063] Example 3

[0064] like Figure 6 As shown, based on the semantic information-based iterative nearest point method disclosed in Embodiment 1 above, this embodiment further provides an iterative nearest point apparatus for executing the semantic information-based iterative nearest point method. This apparatus includes a preprocessing module 601, a search and transformation module 602, a first iteration module 603, an update module 604, and a second iteration module 605. The specific functions of each module are as follows:

[0065] The preprocessing module 601 is used to denote the two point clouds of the target object as source point cloud P and target point cloud Q, preprocess the two point clouds, and segment the point clouds using semantic information. Sub-point clouds with the same semantic information use the same index k, resulting in a total of K semantically segmented sub-source point clouds P. k Hide target point cloud Q k Let P be the k-th sub-source point cloud. k The number of midpoints is N kInitialize the optimal rotation matrix R as a 3x3 identity matrix, initialize the optimal translation matrix t as a 3x1 zero vector, and initialize the iteration counter n to 1;

[0066] The transformation module 602 is used to find the k-th sub-source point cloud P. k and the k-th sub-target point cloud Q k The nearest corresponding point: for the k-th sub-source point cloud P k The i-th point p in k,i ∈P k Perform the transformation of the current optimal rotation matrix R and optimal translation matrix t, and then transform the k-th sub-target point cloud Q. k For each q∈Q k The point, find with p k,i satisfy The nearest corresponding point q k,i Finally, N will be found k q k,i Composition of sub-source point cloud P k The nearest corresponding point target point cloud Q k ;

[0067] The first iteration module 603 repeatedly calls the lookup and transformation module K times to obtain K semantically segmented sub-source point clouds P in the first iteration. k The nearest corresponding point cloud Q after transformation by the current optimal rotation matrix R and optimal translation matrix t k ;

[0068] Update module 604 is used to update the optimal rotation and translation matrices for the K semantically segmented sub-source point clouds P. k The target point cloud Q with the nearest corresponding point k Solve using SVD decomposition. Calculate and update the current optimal rotation matrix R and translation matrix t, and calculate the current registration error.

[0069] The second iteration module 605 repeatedly calls the search transformation module, the first iteration module, and the update module. Each time, it repeatedly updates the iteration calculator n = n + 1. When the registration error E calculated in the update module is less than the set threshold or the number of iterations n exceeds the set maximum value during the nth iteration, the registration ends and the optimal rotation matrix R and translation matrix t are output.

[0070] The specific implementation of each module in this embodiment can be found in Embodiment 1 above, and will not be repeated here. It should be noted that the device provided in this embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0071] Example 4

[0072] This embodiment provides a computer device, which can be a computer, such as... Figure 7 As shown, the processor 702, memory, input device 703, display 704, and network interface 705 are connected via system bus 701. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium 706 and internal memory 707. The non-volatile storage medium 706 stores the operating system, computer programs, and database. The internal memory 707 provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. When the processor 702 executes the computer program stored in the memory, it implements the iterative nearest point method based on semantic information proposed in Embodiment 1 above. This method includes the following steps:

[0073] S1. Denote the two point clouds of the target object as source point cloud P and target point cloud Q. Preprocess the two point clouds by segmenting them using semantic information. Sub-point clouds with the same semantic information are given the same index k, resulting in a total of K semantically segmented sub-source point clouds P. k Hide target point cloud Q k Let P be the k-th sub-source point cloud. k The number of points is N k Initialize the optimal rotation matrix R as a 3x3 identity matrix, initialize the optimal translation matrix t as a 3x1 zero vector, and initialize the iteration counter n to 1;

[0074] S2, Find the k-th sub-source point cloud P k and the k-th sub-target point cloud Q k The nearest corresponding point: for the k-th sub-source point cloud P k The i-th point p in k,i ∈P k Perform the transformation of the current optimal rotation matrix R and optimal translation matrix t, and then find the k-th sub-target point cloud Q. k The middle satisfies the condition of this point. The nearest corresponding point q k,i Finally, N will be found k q k,i Composition of sub-source point cloud P k The nearest corresponding point target point cloud Q k;

[0075] S3. Repeat step S2 for a total of K times to obtain the K semantically segmented sub-source point clouds P in the first iteration. k The nearest corresponding point cloud Q after transformation by the current optimal rotation matrix R and optimal translation matrix t k ;

[0076] S4. Update the optimal rotation and translation matrices: For the K semantically segmented sub-source point clouds P k The target point cloud Q with the nearest corresponding point k Solve using SVD decomposition. Calculate and update the current optimal rotation matrix R and translation matrix t, and calculate the current registration error.

[0077] S5. Repeat S2-S4, updating the iteration calculator n = n+1 each time. When the registration error E calculated in step S4 is less than the set threshold or the number of iterations n exceeds the set maximum value during the nth iteration, the registration ends and the optimal rotation matrix R and translation matrix t are output.

[0078] Example 5

[0079] This embodiment provides a storage medium, which is a computer-readable storage medium, storing a computer program. When the computer program is executed by a processor, it implements an iterative nearest point method based on semantic information as described in Embodiment 1 above. The method includes the following steps:

[0080] S1. Denote the two point clouds of the target object as source point cloud P and target point cloud Q. Preprocess the two point clouds by segmenting them using semantic information. Sub-point clouds with the same semantic information are given the same index k, resulting in a total of K semantically segmented sub-source point clouds P. k Hide target point cloud Q k Let P be the k-th sub-source point cloud. k The number of points is N k Initialize the optimal rotation matrix R as a 3x3 identity matrix, initialize the optimal translation matrix t as a 3x1 zero vector, and initialize the iteration counter n to 1;

[0081] S2, Find the k-th sub-source point cloud P k and the k-th sub-target point cloud Q k The nearest corresponding point: for the k-th sub-source point cloud P k The i-th point p in k,i ∈P k Perform the transformation of the current optimal rotation matrix R and optimal translation matrix t, and then find the k-th sub-target point cloud Q. k The middle satisfies the condition of this point. The nearest corresponding point q k,i Finally, N will be found k q k,i Composition of sub-source point cloud P k The nearest corresponding point target point cloud Q k ;

[0082] S3. Repeat step S2 for a total of K times to obtain the K semantically segmented sub-source point clouds P in the first iteration. k The nearest corresponding point cloud Q after transformation by the current optimal rotation matrix R and optimal translation matrix t k ;

[0083] S4. Update the optimal rotation and translation matrices: For the K semantically segmented sub-source point clouds P k The target point cloud Q with the nearest corresponding point k Solve using SVD decomposition. Calculate and update the current optimal rotation matrix R and translation matrix t, and calculate the current registration error.

[0084] S5. Repeat S2-S4, updating the iteration calculator n = n+1 each time. When the registration error E calculated in step S4 is less than the set threshold or the number of iterations n exceeds the set maximum value during the nth iteration, the registration ends and the optimal rotation matrix R and translation matrix t are output.

[0085] The storage medium described in this embodiment can be a disk, optical disk, computer memory, random access memory (RAM), USB flash drive, portable hard drive, etc.

[0086] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. An iterative nearest-point method based on semantic information, characterized in that, The iterative nearest point method includes the following steps: S1. Denote the two point clouds of the target object as source point cloud P and target point cloud Q. Preprocess the two point clouds by segmenting them using semantic information. Sub-point clouds with the same semantic information are given the same index k, resulting in a total of K semantically segmented sub-source point clouds P. k Hide target point cloud Q k Let P be the k-th sub-source point cloud. k The number of midpoints is N k Initialize the optimal rotation matrix R as a 3x3 identity matrix, initialize the optimal translation matrix t as a 3x1 zero vector, and initialize the iteration counter n to 1; S2, Find the k-th sub-source point cloud P k and the k-th sub-target point cloud Q k The nearest corresponding point: for the k-th sub-source point cloud P k The i-th point p in k,i ∈P k Perform the transformation of the current optimal rotation matrix R and optimal translation matrix t, and then transform the k-th sub-target point cloud Q. k For each q∈Q k The point, find with p k,i satisfy The nearest corresponding point q k,i Finally, N will be found k q k,i Composition of sub-source point cloud P k The nearest corresponding point target point cloud Q k ; S3. Repeat step S2 for a total of K times to obtain the K semantically segmented sub-source point clouds P in the first iteration. k The nearest corresponding point cloud Q after transformation by the current optimal rotation matrix R and optimal translation matrix t k ; S4. Update the optimal rotation and translation matrices: For the K semantically segmented sub-source point clouds P k The target point cloud Q with the nearest corresponding point k Solve using SVD decomposition. Calculate and update the current optimal rotation matrix R and translation matrix t, and calculate the current registration error. S5. Repeat steps S2-S4, updating the iteration calculator n = n+1 each time. When the registration error E calculated in step S4 is less than the set threshold or the number of iterations n exceeds the set maximum value during the nth iteration, the registration ends and the optimal rotation matrix R and translation matrix t are output.

2. The iterative nearest point method based on semantic information according to claim 1, characterized in that, In step S2, the nearest corresponding point in the sub-target point cloud with the same semantic information is found. The coordinates of each point in the target point cloud are stored using a kd-tree data structure. For each point in the source point cloud after transformation by the current optimal rotation matrix R and the optimal translation matrix t, the nearest corresponding point is quickly searched in the kd-tree to form the nearest corresponding point target point cloud.

3. The iterative nearest point method based on semantic information according to claim 1, characterized in that, The SVD decomposition solution method in step S4 is as follows: Let Perform SVD decomposition on W: W = UΣV T Solve for U and V, where U and V are 3x3 unitary matrices and Σ is a 3x3 non-negative real diagonal matrix. Calculate the optimal rotation and translation matrix R = VU for the nth iteration. T , in p′ k,i If the determinant of the rotation matrix R is negative, the third column element of the rotation matrix R is changed to its opposite.

4. An iterative nearest point apparatus for implementing the iterative nearest point method based on semantic information as described in any one of claims 1 to 3, characterized in that, The iterative nearest point device includes: The preprocessing module is used to denote the two point clouds of the target object as source point cloud P and target point cloud Q. It preprocesses the two point clouds by using their semantic information to segment them. Sub-point clouds with the same semantic information use the same index k, resulting in a total of K semantically segmented sub-source point clouds P. k Hide target point cloud Q k Let P be the k-th sub-source point cloud. k The number of midpoints is N k Initialize the optimal rotation matrix R as a 3x3 identity matrix, initialize the optimal translation matrix t as a 3x1 zero vector, and initialize the iteration counter n to 1; The search and transformation module is used to find the k-th sub-source point cloud P. k and the k-th sub-target point cloud Q k The nearest corresponding point: for the k-th sub-source point cloud P k The i-th point p in k,i ∈P k Perform the transformation of the current optimal rotation matrix R and optimal translation matrix t, and then transform the k-th sub-target point cloud Q. k For each q∈Q k The point, find with p k,i satisfy The nearest corresponding point q k,i Finally, N will be found k q k,i Composition of sub-source point cloud P k The nearest corresponding point target point cloud Q k ; The first iteration module repeatedly calls the lookup and transformation module K times to obtain K semantically segmented sub-source point clouds P in the first iteration. k The nearest corresponding point cloud Q after transformation by the current optimal rotation matrix R and optimal translation matrix t k ; The update module is used to update the optimal rotation and translation matrices for the K semantically segmented sub-source point clouds P. k The target point cloud Q with the nearest corresponding point k Solve using SVD decomposition. Calculate and update the current optimal rotation matrix R and translation matrix t, and calculate the current registration error. The second iteration module repeatedly calls the search transformation module, the first iteration module, and the update module. Each time, the iteration calculator n = n + 1 is updated. When the registration error E calculated in the update module is less than the set threshold or the number of iterations n exceeds the set maximum value during the nth iteration, the registration ends and the optimal rotation matrix R and translation matrix t are output.

5. A computer device, comprising a processor and a memory for storing a processor-executable program, characterized in that, When the processor executes the program stored in the memory, it implements the iterative nearest point method based on semantic information as described in any one of claims 1 to 3.

6. A storage medium storing a program, characterized in that, When the program is executed by the processor, it implements the iterative nearest point method based on semantic information as described in any one of claims 1 to 3.

Images