A local 3D point cloud registration system and method
By using a local 3D point cloud registration system, which combines image color and point cloud geometric information with an encoder and SVD decomposition module, the difficulty of matching key feature points in local point cloud registration is solved, and efficient registration in complex scenes is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- GUANGZHOU UNIVERSITY
- Filing Date
- 2023-06-20
- Publication Date
- 2026-05-29
AI Technical Summary
Existing local point cloud registration methods have difficulty finding discriminative point features, leading to the appearance of false corresponding points. This is especially true when noisy points or non-overlapping points are present, and they cannot effectively solve the problem of matching key overlapping feature points.
A local 3D point cloud registration system is adopted. The feature vectors of point cloud and image data are extracted by the encoder module, the overlap score is calculated by the mapping module, and the rigid transformation is calculated by the SVD decomposition module. The cross-entropy loss based on the overlap score and the auxiliary loss function of the rigid transformation are constructed to achieve point cloud registration.
It improves the accuracy and efficiency of point cloud registration in low-overlap and complex noise scenarios, and enhances the detection capability of key overlapping points by combining image color and point cloud geometric information to predict key feature points.
Smart Images
Figure CN116823904B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of point cloud data processing, and mainly to a local three-dimensional point cloud registration system and method. Background Technology
[0002] Robotic platforms equipped with modern depth cameras often generate partially overlapping 3D point clouds. In 3D reconstruction tasks, to generate a complete 3D scene, it is necessary to capture overlapping point clouds from different shooting angles, and then stitch these point clouds together to form a more complete and accurate scene point cloud from all angles. This task is called point cloud registration, which has developed rapidly due to its wide range of applications. Locally overlapping point cloud registration, that is, registration of local point clouds with local point clouds, is a common task in the real world. Rigid point cloud registration is a fundamental task in computer vision and robotics, aiming to find a rigid transformation that aligns a pair of point clouds. It has many important applications in scene reconstruction, SLAM, target pose estimation, and other fields.
[0003] There are three main existing technical solutions. One is to use traditional optimization algorithms, such as ICP, to estimate the optimal transform using mathematical methods. The second is to perform inter-frame matching based on artificially constructed feature descriptors, such as SIFT features. The third is to use deep learning methods, such as the DCP algorithm, to encode point cloud features and then predict the transform. This method is more robust and has greater development potential as computing power improves.
[0004] To date, local-to-local point cloud registration remains poorly solved. While finding a few precise corresponding points is sufficient to address the partial-to-partial registration problem, discriminative points are consistently difficult to find, and spurious correspondences may still exist due to noise or non-overlapping points. Therefore, discriminative point features are extremely important. Advanced feature extraction methods provide more comprehensive and accurate feature information for subsequent feature matching. Recent approaches to local point cloud registration, such as overlap score prediction, have addressed this issue to some extent, and the multimodal two-stage local registration combining 2D image color information and point cloud geometric information has also provided some relief. However, the improvements are minimal and cannot truly solve the critical problem of matching overlapping feature points in local point cloud registration. Summary of the Invention
[0005] To address the aforementioned problems, this invention aims to propose a local 3D point cloud registration system and method. This invention solves the key feature point matching problem in local registration to a certain extent and improves the registration capability of 3D local point clouds.
[0006] The objective of this invention is achieved through the following technical solution:
[0007] A local 3D point cloud registration system includes: an encoder module, a mapping module, and an SVD decomposition module;
[0008] The encoder module receives input point cloud data and image data, obtains the feature vector of the image data, extracts the features, and then transmits the features to the mapping module.
[0009] The mapping module assigns feature vectors to the corresponding 3D point cloud as overlap scores, then uses the calculated overlap scores and feature vectors of the points to calculate the similarity matrix, and then sends the calculation results to the SVD decomposition module.
[0010] The SVD decomposition module receives the calculation results of the similarity matrix and outputs the rigid transformation required for registration, ultimately achieving local point cloud registration.
[0011] Furthermore, the encoder is constructed using convolutional kernels and activation functions.
[0012] Furthermore, the encoder module includes:
[0013] The point cloud encoding submodule is configured to encode the input point cloud data values as feature vectors;
[0014] The image encoding submodule is configured to encode input image data values as image features.
[0015] A local 3D point cloud registration method includes the following steps:
[0016] S100: Input the source point cloud and the target point cloud to the encoder. The encoder will map the 3D point cloud data from the high-dimensional sample space to the low-dimensional feature space.
[0017] Simultaneously input the two-dimensional images corresponding to the source point cloud and the target point cloud into the encoder to encode image features;
[0018] S200. The extracted two-dimensional image feature vector is mapped onto the corresponding three-dimensional point cloud space as the overlap score of the corresponding three-dimensional point cloud points, and the obtained overlap score of the three-dimensional point cloud is input into the three-dimensional feature vector to calculate the similarity matrix by matrix multiplication.
[0019] S300, then calculate the rigid transformation matrix R and translation vector t by decomposing the similarity matrix H using SVD:
[0020] R, t = SVD(H)
[0021] And construct the loss function;
[0022] The loss function includes
[0023] The auxiliary loss function for rigid transformations R and t can be expressed mathematically as follows:
[0024]
[0025] The mathematical expression for the cross-entropy loss function based on overlap scores is as follows:
[0026]
[0027] The cross-entropy loss function is used to constrain the model's ability to predict the overlap score of key points.
[0028] S400 Finally, each pair of point cloud data and the corresponding image data are input into the neural network for training.
[0029] Furthermore, the rigid transformation satisfies the requirement of 6 degrees of freedom for variables.
[0030] Furthermore, after performing the rigid transformation, the registration data after the rigid transformation is updated in the source point cloud.
[0031] The present invention also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, is used to implement a local three-dimensional point cloud registration method.
[0032] The present invention also provides an electronic device for implementing a local three-dimensional point cloud registration method.
[0033] The beneficial effects of this invention are as follows:
[0034] This method proposes a local 3D point cloud registration approach that selects key overlapping points by image color and combines them with point cloud geometric information. By extracting features from the point cloud geometric data and combining this with the color information of the corresponding image, the overlap score of key feature points is predicted, resulting in more effective and accurate key point features. Constraints are achieved through cross-entropy loss based on the overlap score and auxiliary loss based on rigid transformation.
[0035] Compared to other advanced local point cloud registration methods, which have lower accuracy in predicting key points in overlapping regions, the local point cloud registration method based on this method deeply integrates image color information and point cloud geometric information for feature encoding, thereby improving the ability to detect key overlapping points in low-overlap scenes. Attached Figure Description
[0036] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.
[0037] Figure 1This is a flowchart of the local 3D point cloud registration method described in this invention, which selects key overlapping points by image color and combines them with point cloud geometric information.
[0038] Figure 2 This is a schematic diagram of the module connections of the electronic device provided by the present invention. Detailed Implementation
[0039] The present invention will be further described in conjunction with the following embodiments.
[0040] This invention provides a local 3D point cloud registration method that selects key overlapping points by image color and combines them with point cloud geometric information. To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0041] The present invention discloses a local three-dimensional point cloud registration system, comprising: an encoder module, a mapping module, and an SVD decomposition module;
[0042] The encoder module receives input point cloud data and image data, obtains the feature vector of the image data, extracts the features, and then transmits the features to the mapping module.
[0043] The mapping module assigns feature vectors to the corresponding 3D point cloud as overlap scores, then uses the calculated overlap scores and feature vectors of the points to calculate the similarity matrix, and then sends the calculation results to the SVD decomposition module.
[0044] The SVD decomposition module receives the calculation results of the similarity matrix and outputs the rigid transformation required for registration, ultimately achieving local point cloud registration.
[0045] Furthermore, the encoder, constructed using convolutional kernels and activation functions, is capable of extracting local and global features of point cloud geometric information. The mathematical expression for the encoder is as follows:
[0046] z1=Encoder(y1)z2=Encoder(y2).
[0047] Furthermore, the encoder module includes:
[0048] The point cloud encoding submodule is configured to encode the input point cloud data values as feature vectors;
[0049] The image encoding submodule is configured to encode input image data values as image features, as shown in the following mathematical expression:
[0050] q1=Encoder(y1)q2=Encoder(y2).
[0051] A local 3D point cloud registration method includes the following steps:
[0052] S100: Input the source point cloud and the target point cloud to the encoder. The encoder will map the 3D point cloud data from the high-dimensional sample space to the low-dimensional feature space.
[0053] Simultaneously input the two-dimensional images corresponding to the source point cloud and the target point cloud into the encoder to encode image features;
[0054] S200. The extracted two-dimensional image feature vectors are mapped onto the corresponding three-dimensional point cloud space as the overlap score of the corresponding three-dimensional point cloud points. The obtained overlap score of the three-dimensional point cloud is then input into the three-dimensional feature vectors, and the similarity matrix is calculated by matrix multiplication. The mathematical expression is as follows:
[0055] H = f(z1·o1, z2·o2);
[0056] S300, then calculate the rigid transformation matrix R and translation vector t by decomposing the similarity matrix H using SVD:
[0057] R, t = SVD(H)
[0058] And construct the loss function;
[0059] The loss function includes
[0060] The auxiliary loss function for rigid transformations R and t can be expressed mathematically as follows:
[0061]
[0062] The mathematical expression for the cross-entropy loss function based on overlap scores is as follows:
[0063]
[0064] The cross-entropy loss function is used to constrain the model's ability to predict the overlap score of key points.
[0065] S400 Finally, each pair of point cloud data and the corresponding image data are input into the neural network for training.
[0066] Furthermore, the rigid transformation satisfies the requirement of 6 degrees of freedom for variables.
[0067] Furthermore, after performing the rigid transformation, the registration data after the rigid transformation is updated in the source point cloud.
[0068] Combination Figure 2As shown, this disclosure provides an electronic device for a local 3D point cloud registration system and method, including a processor 30 and a memory 31. Optionally, the electronic device may further include a communication interface 32 and a bus 33. The processor 30, communication interface 32, and memory 31 can communicate with each other via the bus 33. The communication interface 32 can be used for information transmission. The processor 30 can call logical instructions in the memory 31 to execute the local 3D point cloud registration system and method of the above embodiments.
[0069] This disclosure provides a storage medium storing computer-executable instructions configured to execute the aforementioned local 3D point cloud registration system and method.
[0070] The aforementioned storage media can be either transient computer-readable storage media or non-transitory computer-readable storage media. Non-transitory storage media include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, and can also be transient storage media.
[0071] In summary, the local 3D point cloud registration method provided by this invention predicts the overlap score of key feature points by performing feature extraction on the geometric data of the point cloud and combining the color information of the corresponding 2D image, finds the partially overlapping areas between point clouds, and then uses the key overlapping feature points to calculate a more accurate similarity matrix and rigid transformation, ensuring accurate and fast local point cloud registration capability in low overlap scenes and complex multi-noise scenes.
[0072] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the essence and scope of the technical solutions of the present invention.
Claims
1. A local 3D point cloud registration system, comprising: Encoder module, mapping module, SVD decomposition module; The encoder module receives input point cloud data and image data, obtains the feature vector of the image data, extracts the features, and then transmits the features to the mapping module. The mapping module maps the feature vectors of the image data to each point of the 3D point cloud to obtain the overlap score corresponding to each point; then, it uses the overlap score and the feature vectors of the point cloud data to calculate the similarity matrix, and then sends the calculation result to the SVD decomposition module. The SVD decomposition module receives the calculation results of the similarity matrix and outputs the rigid transformation required for registration, ultimately achieving local point cloud registration. The SVD decomposition module receives the similarity matrix calculation results and outputs the rigid transformation required for registration, ultimately achieving local point cloud registration, including: The rigid transformation matrix R and translation vector t are calculated by decomposing the similarity matrix H using SVD: And construct the loss function; The loss function includes The auxiliary loss function for rigid transformations R and t can be expressed mathematically as follows: The mathematical expression for the cross-entropy loss function based on overlap scores is as follows: ； The cross-entropy loss function is used to constrain the model's ability to predict the overlap score of key points. Finally, each pair of point cloud data and the corresponding image data are input into the neural network for training. The encoder is constructed using convolutional kernels and activation functions; The encoder module includes: The point cloud encoding submodule is configured to encode the input point cloud data values as feature vectors; The image encoding submodule is configured to encode input image data values as image features; The encoder, constructed using convolutional kernels and activation functions, is capable of extracting local and global features of point cloud geometric information. The mathematical expression for the encoder is as follows: z1=Encoder(y1)z2=Encoder(y2); The image encoding submodule is configured to encode input image data values as image features, as shown in the following mathematical expression: q1=Encoder(y1)q2=Encoder(y2).
2. A method for local 3D point cloud registration, characterized in that, Includes the following steps: S100: Input the source point cloud and the target point cloud to the encoder. The encoder will map the 3D point cloud data from the high-dimensional sample space to the low-dimensional feature space. Simultaneously input the two-dimensional images corresponding to the source point cloud and the target point cloud into the encoder to encode image features; S200. The extracted two-dimensional image feature vectors are mapped onto each point of the three-dimensional point cloud to obtain the overlap score corresponding to each point; then, the similarity matrix is calculated using the overlap score and the feature vectors of the point cloud data. S300, then calculate the rigid transformation matrix R and translation vector t by decomposing the similarity matrix H using SVD: And construct the loss function; The loss function includes The auxiliary loss function for rigid transformations R and t can be expressed mathematically as follows: The mathematical expression for the cross-entropy loss function based on overlap scores is as follows: ； The cross-entropy loss function is used to constrain the model's ability to predict the overlap score of key points. S400. Finally, each pair of point cloud data and the corresponding image data are input into the neural network for training. The encoder is constructed using convolutional kernels and activation functions; The S100 further includes: The point cloud data values configured as coded inputs are feature vectors; The image data values configured as encoded inputs are image features; The encoder, constructed using convolutional kernels and activation functions, is capable of extracting local and global features of point cloud geometric information. The mathematical expression for the encoder is as follows: z1=Encoder(y1)z2=Encoder(y2); The image data values configured as the encoding input are image features, expressed mathematically as follows: q1=Encoder(y1)q2=Encoder(y2).
3. The local three-dimensional point cloud registration method according to claim 2, characterized in that, The rigid transformation satisfies the requirement of 6 degrees of freedom for variables.
4. The local three-dimensional point cloud registration method according to claim 2, characterized in that, After performing the rigid transformation, the registration data after the rigid transformation is updated in the source point cloud.
5. A storage medium storing a computer program, characterized in that, When the computer program is executed by a computer, it is used to implement the method described in any one of claims 2-4.
6. An electronic device for performing the method of any one of claims 2-4.