Local-to-global point cloud registration method and system based on internal and external point classification
Through a point cloud registration network based on internal and external point classification, local to global point cloud registration problems under the influence of low overlap rate and noise are solved, and efficient and accurate point cloud registration is achieved, which is suitable for multiple application scenarios.
Patent Information
- Application Number
- CN202410508125.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-25
- Publication Date
- 2024-11-12
AI Technical Summary
In the presence of low overlap, noise and outliers, the local to global point cloud registration effect is poor, especially when the overlap rate is 10%, it is difficult to accurately establish point cloud feature correspondence, affecting the accuracy of registration. Sex and efficiency.
Using a point cloud registration network (IOPCNet) based on internal and external point classification, through random sampling, local coordinate system alignment, feature extraction and similarity matrix enhancement, combined with multi-layer perceptron and ICP algorithm, points in non-overlapping areas are eliminated and matching is improved. Accurate accuracy and robustness.
Effectively dealing with noise, outliers and low overlap rates improves the accuracy and efficiency of point cloud registration, especially in the case of low overlap rates, achieving higher accuracy and robustness, suitable for multiple application scenarios such as Autonomous driving and cultural relics restoration.
Smart Images

Figure CN118941599A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer graphics and image processing, and in particular relates to a local to global point cloud registration method based on internal and external point classification. Background Art
[0002] Local to global point cloud registration is to align the reference frame of a local point cloud with a global point cloud. A local point cloud is usually obtained by scanning or capturing a portion of the target scene, while a global point cloud captures the entire target scene. Due to the different techniques used to capture local and global point clouds, as well as the diversity of application areas, several typical challenging situations arise, including noise, outliers, and low overlap.
[0003] Traditional point cloud registration methods usually involve computing descriptors of key points in the source and target point clouds and establishing correspondences between the two point clouds. Therefore, the quality of the solution depends largely on the accuracy of corresponding point pairs on the target and source point clouds. However, for point cloud registration with low overlap, algorithms such as RANSAC have been proposed to eliminate non-corresponding point pairs, but it is often affected by noise and outliers, resulting in failure and unnecessary consumption of computational resources.
[0004] Recently, several methods have emerged that use deep learning techniques to compute features and establish soft correspondences, such as simulated annealing and transformers. These methods have greatly enhanced the robustness of the model to noise. However, they have encountered difficulties in local-to-global point cloud registration with low overlap, such as point clouds with only 10% overlap. Constructing robust point cloud features and establishing correct correspondences between these features remains a very challenging task. This is because, in the case of low overlap, most points in the source point cloud have no corresponding points in the target point cloud, resulting in a high probability of false correspondence. In addition, the presence of a large number of outliers leads to inaccurate initial similarity matrices between the source and target point clouds. In addition, robust features should be rotationally and translationally invariant in Euclidean space, but some recent end-to-end deep learning-based methods ignore this, compromising the generalization of the model. Local-to-global point cloud registration means that the shape of the target point cloud is a subset of the shape of the source point cloud. Therefore, by identifying points in the overlapping area, local-to-global point cloud registration can be transformed into global-to-global point cloud registration. For low overlap point cloud registration, such as 10% overlap rate, identifying points in overlapping areas can still be effective. Therefore, in the present invention, points in overlapping areas and points in non-overlapping areas are first distinguished, where points in overlapping areas are called inner points, and points in non-overlapping areas are considered outer points. Overlapping areas, local to global point cloud registration can be converted into global to global point cloud registration. For low overlap point cloud registration, such as 10% overlap rate, identifying points in overlapping areas can still be effective.
[0005] Based on the above motivations, this paper proposes a low-overlap local-to-global point cloud registration network (IOPCNet) based on inside-outside point classification, which takes an unordered and undirected point cloud as input and outputs a binary mask of the source point cloud, indicating whether each point in the source point cloud is an internal point or not.
[0006] The significance of solving the above problems and defects is that local to global point cloud registration is widely adopted as a subroutine in many applications. In the field of earth sciences, local to global point cloud registration can be used to analyze geological faults, geological structures, and surface deformations. This is essential for studying earthquakes, volcanic activity, and other phenomena. In the field of autonomous driving, aligning the local point cloud around the vehicle with the global environment point cloud can improve the safety of autonomous vehicles. In the field of cultural heritage protection, registering local point clouds acquired in different time periods with global point cloud data can establish a historical record of archaeological sites and monitor their changes over time. Through the above analysis,
[0007] The problems and defects of the existing technology are as follows: the existing technology relies on a high overlap rate and is prone to failure in the case of low overlap, a large number of outliers, and local to global point cloud registration. Summary of the invention
[0008] In view of the problems existing in the prior art, the present invention provides a local to global point cloud registration method based on internal and external point classification.
[0009] The present invention is implemented as follows: a local to global point cloud registration method based on internal and external point classification includes:
[0010] Step 1: Randomly sample each point cloud to reduce the overall time complexity by reducing the number of points in the two point clouds;
[0011] Step 2: Calculate the local coordinate system of the sampling point and its K nearest neighbors, and align the local coordinates of each point with the global coordinates of the point cloud;
[0012] Step 3: The converted sampling points and their K nearest neighbors are used as the input of the network framework to map the coordinates to high dimensions, that is, to calculate the local features of each point;
[0013] Step 4: Calculate the similarity matrix of the local features of the two point clouds;
[0014] Step 5, using an external storage unit to enhance the similarity matrix;
[0015] Step 6: Concatenate the similarity matrix and the local feature matrix to form a global feature;
[0016] Step 7: Decode the global features to obtain the masks of the source point cloud and the target point cloud, and remove the points in the non-overlapping area;
[0017] Step 8: Train the network framework to obtain the points in the overlapping area, and then use ICP to align the points in the overlapping area.
[0018] Furthermore, the local coordinate system of the calculated sampling point and its K nearest neighbors in step 2, aligning the local coordinates of each point with the global coordinates of the point cloud, includes the following steps:
[0019] The source point cloud and the target point cloud are and Xi and Yj represent the i-th point of the source point cloud X and the j-th point of the target point cloud Y respectively. The K nearest neighbors of each point in the source point cloud and the target point cloud are calculated; the covariance matrix is estimated using the k nearest neighbors of Xi, and the covariance matrix is decomposed using the singular value decomposition method. Its eigenvalue is taken as 1 as the corresponding local coordinate system, and finally its local coordinates are aligned with the global coordinates of the point cloud.
[0020] Furthermore, the step 3 uses the converted sampling points and their K nearest neighbors as inputs of the network framework to map the coordinates to a high dimension, including the following steps:
[0021] The dimension of the converted sampling points and their K nearest neighbors is increased to 64 dimensions using a multi-layer perceptron, and the edge convolutional network is used to further extract features to obtain the local features F of the source point cloud and the target point cloud respectively. x and F y .
[0022] Further, the calculation of the similarity matrix of the local features of the two point clouds in step 4 includes the following steps:
[0023] Use Fx multiplied by the transposed matrix of Fy to approximate the similarity matrix H between the source point cloud and the target point cloud.
[0024] Further, the step of enhancing the similarity matrix by using an external storage unit in step 5 includes the following steps:
[0025] First, initialize the external storage unit M k and M v , use M k Multiply by H and use scale normalization to get S, and use the external memory unit M v Multiply it with S to get the enhanced similarity matrix F out .
[0026] Furthermore, the step six of concatenating the similarity matrix and the local feature matrix to form a global feature includes the following steps:
[0027] Use max pooling to solve F separately x and F y The global feature g xand g y ; The global feature g x and g y According to the second dimension, it is copied N times and the enhanced similarity matrix F after enhancement is obtained. out Concatenate in the first dimension to get the final global features;
[0028] The step 7 of decoding the global features to obtain masks of the source point cloud and the target point cloud, and removing points in non-overlapping areas, comprises the following steps:
[0029] MLP is applied to decode the above cascade features. The decoding layer passes through five MLP layers with sizes of 1024, 512, 256, 128 and 2 respectively. The last layer uses the Average-Pooling function to increase the robustness of the network. The Sigmoid activation function is used after each layer to ensure that the output value is between 0 and 1, and the mask of the target point cloud is obtained. For values greater than 0.5 in the mask, the value is set to 1, otherwise it is set to 0. Points with a value of 0 are outliers and are removed. The point cloud after removing the outliers uses the ICP-based method to solve the rotation and translation matrix.
[0030] The training network framework in step eight obtains points in the overlapping area, and then aligns the points in the overlapping area using ICP, including the following steps:
[0031] During the training process, the cross entropy error between the predicted mask and the true mask in step 7 is used as the loss function. The Adam optimizer is used to optimize the network. The initial learning rate is 0.001, and 500 epoches are trained. The learning rate is reduced to 0.1 times the original at the 10th and 300th training times, respectively.
[0032] A local to global point cloud registration system based on internal and external point classification includes:
[0033] Point cloud sampling module, used to randomly sample each point cloud, reducing the overall time complexity by reducing the number of points in the two point clouds;
[0034] The local coordinate system module is used to calculate the local coordinate system of each point in the two point clouds, eliminate the sign ambiguity of the local coordinate system in a simple and effective way, and finally align it with the global coordinate system;
[0035] Local feature calculation module, used to calculate the local features and global features of the aligned point cloud;
[0036] Similarity matrix enhancement module, which is used to calculate the similarity matrix of local features of point cloud and use external memory units for enhancement;
[0037] Feature aggregation module, used to concatenate and aggregate the enhanced similarity matrix and global features;
[0038] A feature decoding module is used to decode the above cascade features using a multi-layer perceptron;
[0039] An outlier removal module is used to apply the obtained decoding to the source point cloud so as to remove non-corresponding points in the source point cloud;
[0040] The registration module is used to train the network framework and perform registration using an ICP-based registration method.
[0041] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the local to global point cloud registration method based on internal and external point classification.
[0042] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the local-to-global point cloud registration method based on internal and external point classification.
[0043] An information data processing terminal is used to implement a local to global point cloud registration system based on internal and external point classification.
[0044] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:
[0045] Second, the present invention can effectively deal with the challenges of noise, outliers and low overlap rate in local to global point cloud registration through the point cloud registration network framework of deep learning internal and external point classification; using the local coordinate system introduced by the present invention before feature extraction, the local reference system can be aligned with the local area of the key points in the source point cloud and the target point cloud to improve the accuracy of the model; using the similarity matrix enhancement module introduced by the present invention, the initial erroneous correspondence can be corrected through gradient calculation and back propagation to achieve point cloud registration with low overlap rate.
[0046] The present invention is used for completing cultural relics in cultural relic restoration. The three-dimensional cultural relic to be restored is registered with the existing complete three-dimensional cultural relic. The area without matching in the complete cultural relic is the part to be restored. Compared with the point cloud generation method, the result obtained by the present invention is more in line with reality. After the virtual restoration is completed, it is extracted and printed at 1:1, and spliced with the restored cultural relic for physical restoration.
[0047] The technical solution of the present invention solves a technical problem that people have been eager to solve but have never succeeded in solving: compared with the existing 12 mainstream registration methods, the present invention has a higher accuracy rate, especially when the overlap rate is only 10%, the experimental accuracy rate of the present invention is almost unaffected, and the present invention has been verified to have good robustness under multiple data sets.
[0048] Third, the present invention effectively solves several key problems faced by the prior art in point cloud data processing and registration by providing a local to global point cloud registration system based on internal and external point classification:
[0049] Technical problems of existing technologies:
[0050] 1. High time complexity: Traditional point cloud registration methods often involve a large amount of point cloud data processing. Especially when processing high-density point clouds, the time complexity is high and it is difficult to implement quickly.
[0051] 2. Insufficient local features: Existing methods have limited effectiveness in extracting local features of point clouds and find it difficult to accurately capture detailed local information of point clouds, which is particularly evident in complex or partially occluded environments.
[0052] 3. External point interference: In the existing point cloud registration methods, the processing of external points (noise or non-corresponding points) is not effective enough, which can easily affect the accuracy of the final registration.
[0053] 4. Lack of global consistency: Although point clouds can be locally aligned, ensuring global consistency remains a challenge, especially when processing large-scale spatial point clouds.
[0054] Technical advances of the present invention:
[0055] 1. Reduce time complexity: Through the point cloud sampling module, the number of points processed is effectively reduced, thereby reducing the time complexity of calculation, making the point cloud registration process more efficient.
[0056] 2. Enhanced local feature recognition: The local coordinate system module and the local feature calculation module can accurately calculate and utilize the local features of the point cloud, improving the accuracy of registration, especially in scenes with rich details or partial occlusion.
[0057] 3. Effective processing of outliers: The outlier removal module specifically processes non-corresponding points, significantly improving the quality and reliability of the registration results.
[0058] 4. Effective integration of global and local: Through the design of feature aggregation module and feature decoding module, the effective integration of local features and global information is achieved, ensuring global consistency, thereby improving the performance of overall registration.
[0059] These advances not only improve the efficiency and accuracy of point cloud registration, but also expand the application potential of this technology in areas such as autonomous driving, robot navigation, and 3D modeling. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] Figure 1 It is a flow chart of a local to global point cloud registration method for internal and external point classification provided in an embodiment of the present invention.
[0061] Figure 2 It is a structural schematic diagram of a local to global point cloud registration system for internal and external point classification provided by an embodiment of the present invention;
[0062] Figure 3 It is a flow chart for implementing the local to global point cloud registration method for internal and external point classification provided by an embodiment of the present invention.
[0063] Figure 4 It is a schematic diagram of the registration network structure provided by an embodiment of the present invention.
[0064] Figure 5 Schematic diagrams of some registration results on the Terracotta Warriors and Stanford datasets provided by the embodiments of the present invention. (a) Source point cloud and target point cloud, the source point cloud is red, and the target point cloud is blue. (b) Predicted unstable points of the source point cloud, red is the unstable point of the source point cloud. (c) Point cloud registration result. The source point cloud is red, and the target point cloud is blue.
[0065] Figure 6 Schematic diagrams of some registration results on the Terracotta Warriors and Stanford datasets provided by the embodiments of the present invention. (a) Source point cloud and target point cloud, the source point cloud is red, and the target point cloud is blue. (b) Predicted unstable points of the source point cloud, red is the unstable point of the source point cloud. (c) Point cloud registration result. The source point cloud is red, and the target point cloud is blue.
[0066] Figure 2 In: 1. Point cloud sampling module; 2. Local coordinate system module; 3. Local feature calculation module; 4. Similarity matrix enhancement module; 5. Feature aggregation module; 6. Feature decoding module; 7. Outlier removal module; 8. Registration module. DETAILED DESCRIPTION
[0067] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0068] like Figure 1 As shown, a local to global point cloud registration method based on internal and external point classification provided by an embodiment of the present invention includes the following steps:
[0069] S101: Randomly sample each point cloud to reduce the overall time complexity by reducing the number of points in the two point clouds;
[0070] S102: The two sampled point clouds are used as inputs of the network framework, the local coordinate system of each point in the two point clouds is calculated, and the sign ambiguity of the local coordinate system is eliminated in a simple and effective way, and finally it is aligned with the global coordinate system;
[0071] S103: Calculate local features and global features of the aligned point cloud;
[0072] S104: Calculate the similarity matrix of the local features of the point cloud and use an external memory unit for enhancement;
[0073] S105; concatenating and aggregating the enhanced similarity matrix and the global features;
[0074] S106: Decode the cascaded features obtained in S105 using a multi-layer perceptron;
[0075] S107: applying the obtained decoding to the source point cloud, so as to remove non-corresponding points in the source point cloud;
[0076] S108: Train the network framework and perform registration using an ICP (Iterative Closest Point)-based registration method.
[0077] The local to global point cloud registration method based on internal and external point classification provided by the present invention can also be implemented by ordinary technicians in the industry using other steps. Figure 1 The local to global point cloud registration method based on internal and external point classification provided by the present invention is only a specific embodiment.
[0078] like Figure 2 As shown, the local to global point cloud registration system based on internal and external point classification provided by the present invention includes:
[0079] Point cloud sampling module 1 is used to randomly sample each point cloud and reduce the overall time complexity by reducing the number of two point cloud points;
[0080] The local coordinate system module 2 is used to take the two sampled point clouds as the input of the network framework, calculate the local coordinate system of each point in the two point clouds, eliminate the sign ambiguity of the local coordinate system in a simple and effective way, and finally align it with the global coordinate system;
[0081] Local feature calculation module 3, calculates the local features and global features of the aligned point cloud;
[0082] Similarity matrix enhancement module 4 calculates the similarity matrix of local features of point cloud and uses external memory unit for enhancement;
[0083] Feature aggregation module 5, concatenating and aggregating the enhanced similarity matrix and global features;
[0084] Feature decoding module 6, decoding the cascaded features obtained above using a multi-layer perceptron;
[0085] An outlier removal module 7 is used to apply the obtained decoding to the source point cloud so as to remove non-corresponding points in the source point cloud;
[0086] The registration module 8 is used to train the network framework and perform registration using an ICP-based registration method.
[0087] The point cloud registration system of the present invention effectively realizes accurate point cloud registration from local to global through the collaborative work of multiple modules. The following is the detailed working principle of each module:
[0088] 1. Point cloud sampling module:
[0089] -This module is responsible for randomly sampling from the input point cloud data to reduce the complexity and computation time of data processing. By randomly selecting a subset of the point cloud, the time complexity of subsequent calculations can be effectively reduced while retaining enough information for accurate registration.
[0090] 2. Local coordinate system module:
[0091] -The core task of this module is to define a local coordinate system for each point. This step is achieved by analyzing the local geometric characteristics of the point cloud, ensuring that the coordinate system of each point is consistent with the geometric layout of its neighborhood. In addition, the module also solves the problem of symbol ambiguity in the local coordinate system, ensuring the consistency and correctness of the coordinate system.
[0092] 3. Local feature calculation module:
[0093] -After the local coordinate system is determined, the module calculates the local features of each point and the overall global features. These features capture the geometric and topological properties of the point cloud and provide the necessary information for subsequent registration.
[0094] 4. Similarity matrix enhancement module:
[0095] -Using local features, this module builds a similarity matrix to describe the similarity of points in different point clouds. Through the enhancement of external memory units, the similarity matrix can more accurately reflect the correspondence between point clouds.
[0096] 5. Feature aggregation module:
[0097] -This module merges and aggregates the enhanced similarity matrix with the global features to form a comprehensive feature description, providing comprehensive data support for the final registration process.
[0098] 6. Feature decoding module:
[0099] -The aggregated features are decoded by a multi-layer perceptron to convert the high-dimensional features into an expression form that is more suitable for the registration task.
[0100] 7. Outlier removal module:
[0101] -During the registration process, this module is responsible for identifying and removing non-corresponding points (i.e., outliers) in the source point cloud, which are generated by noise, occlusion or other factors and interfere with the accuracy of the registration.
[0102] 8. Registration module:
[0103] -Finally, the registration module combines all the previous processing results and uses the registration technology based on the ICP (Iterative Closest Point) algorithm to accurately align the two point clouds. This step takes into account the local to global features and ensures the accuracy and stability of the registration.
[0104] Through this strategy of comprehensive use of local and global information, the point cloud registration system of the present invention can achieve high-precision point cloud registration in complex environments and is suitable for a variety of application scenarios, such as robot navigation, three-dimensional reconstruction and other fields.
[0105] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.
[0106] The present invention is based on the local to global point cloud registration network framework of internal and external point classification. The network structure diagram is as follows: Figure 3 As shown. Specifically including the following steps:
[0107] Step 1 is used to randomly sample each point cloud, reducing the overall time complexity by reducing the number of points in the two point clouds;
[0108] Step 2: Take the two sampled point clouds as the input of the network framework and use the k nearest neighbors of x to estimate the covariance matrix:
[0109]
[0110] The local coordinate system LRF is calculated by decomposing the covariance matrix using the singular value decomposition (SVD) method:
[0111]
[0112] Where U and V are third-order orthogonal matrices, and ∑ is a diagonal matrix arranged in descending order. The eigenvectors in U are orthogonal to each other and are arranged in descending order of the corresponding eigenvalues. For LRF, the x-axis, y-axis, and z-axis are used to mark the three axes of the three eigenvectors corresponding to the maximum, middle, and minimum eigenvalues. Since the negative value of the eigenvector is still an eigenvector, there is a sign ambiguity problem with the eigenvector. To solve this problem, the point cloud S is projected onto the x-axis, y-axis, and z-axis, and then the sum of the projections on each axis is calculated separately. If the sum is greater than 0, the axis remains unchanged, otherwise the corresponding axis takes a negative value. After calculating the local coordinate system, align its local coordinates with the global coordinates;
[0113] Step 3: For the i-th point of point cloud X, first calculate the Euclidean distance between its features to determine the set of k nearest neighbors of the feature, denoted as k xi Then, we use MLP to transform feature k xi Embedded into a high-dimensional feature space, and use the ReLU activation function after each MLP layer to increase the nonlinearity of the network, and then perform MaxPooling to obtain a new feature representation and input it to the next EdgeConv layer. The local feature calculation module first passes through four EdgeConv layers with sizes of 64, 64, 128, and 256. In order to retain the feature information of each layer, the features of the previous layers are concatenated to generate 512-dimensional features, and then MLP is used to map the features to 1024 dimensions, and the point-by-point features F of the source point cloud and the target point cloud are obtained respectively. x and F y Through the maximum pooling function, the global feature vector of the source point cloud and the target point cloud can be obtained as g x and g y .
[0114] Step 4: Correct the erroneous correspondence in the initial similarity matrix through an external storage unit:
[0115] H=H i,j =F x ×F y #(3)
[0116] S=S i,j =Norm(H×M k )#(4)
[0117] F out =S×M v #(5)
[0118] Where M is a learnable parameter that is independent of the input and acts as a storage unit for the entire training data set. The similarity matrix is sensitive to the scale of the input features. To avoid this problem, the similarity matrix is normalized alternately by rows and columns.
[0119] In step 5, the global features of the source point cloud and the target point cloud are copied 1024 times each, and they are concatenated with the enhanced similarity matrix as the input for the next step.
[0120] Step 6. After obtaining the connection features of the source point cloud and the target point cloud, MLP is applied to decode the above cascade features. The decoding layer passes through five MLP layers with sizes of 1024, 512, 256, 128 and 2 respectively. The last layer uses the Average-Pooling function to increase the robustness of the network. The Sigmod activation function is used after each layer to ensure that the output value is between 0 and 1. Normally, the mask should be 0-1 binary, but the discrete output of the neural network during training will cause discontinuity during backpropagation and prevent the generation of a binary vector as output. During the calculation process, the binary vector is calculated by applying the threshold e:
[0121]
[0122] Step 7: Apply the obtained decoding to the source point cloud to remove non-corresponding points in the source point cloud:
[0123]
[0124] Step 8: training the network. During the training process, the present invention uses the cross entropy error between the predicted mask and the true mask in step 7 as the loss function:
[0125] loss=-P gt log P+(1-P gt )log(1-P)#(8)
[0126] X1 and Y are input into the ICP-based pipeline to solve the rotation and translation matrix. Note that it can also be used as an upstream task for other registration methods (such as FGR, PointNetLK, etc.). ICP is used in this invention.
[0127] The present invention is tested on various datasets, such as KITTI, 3Dmatch, ModelNet40, Stanford and Terracotta Warriors data. Experimental results show that IOPCNet outperforms all baseline methods. In particular, IOPCNet still achieves high performance even when the overlap between two point clouds is low (only 10%). IOPCNet is mainly used for local to global point cloud registration, and can also be used as an upstream task for other point cloud registration algorithms to remove non-corresponding points.
[0128] Figure 4 It is a schematic diagram of the registration network structure provided by an embodiment of the present invention.
[0129] Figure 5Schematic diagrams of some registration results on the Terracotta Warriors and Stanford datasets provided by the embodiments of the present invention. (a) Source point cloud and target point cloud, the source point cloud is red, and the target point cloud is blue. (b) Predicted unstable points of the source point cloud, red is the unstable point of the source point cloud. (c) Point cloud registration result. The source point cloud is red, and the target point cloud is blue.
[0130] Figure 6 Schematic diagrams of some registration results on the Terracotta Warriors and Stanford datasets provided by the embodiments of the present invention. (a) Source point cloud and target point cloud, the source point cloud is red, and the target point cloud is blue. (b) Predicted unstable points of the source point cloud, red is the unstable point of the source point cloud. (c) Point cloud registration result. The source point cloud is red, and the target point cloud is blue.
[0131] A computer device is provided in an embodiment of the present invention, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the local to global point cloud registration method based on internal and external point classification.
[0132] A computer-readable storage medium provided by an embodiment of the present invention stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the local to global point cloud registration method based on internal and external point classification.
[0133] An embodiment of the present invention provides an information data processing terminal, which is used to implement a local to global point cloud registration system based on internal and external point classification.
[0134] Architecture and urban planning provided by the embodiments of the present invention: In architecture and urban planning, point cloud registration can be used to accurately model buildings and terrain in three dimensions, which helps designers, urban planners, and engineers to better understand the existing environment and to plan and design.
[0135] The geological exploration provided by the embodiment of the present invention: In geological exploration, local to global point cloud registration can be used to analyze geological structures, topography and underground resources. This is helpful for the development of oil and mineral resources, as well as the prediction and prevention of geological disasters.
[0136] Medical image processing provided by an embodiment of the present invention: In medical image processing, point cloud registration can be used to align medical images obtained at different time points or with different devices for lesion monitoring, surgical planning, etc.
[0137] The cultural heritage protection provided by the embodiment of the present invention is as follows: the three-dimensional cultural relic to be restored is aligned with the existing complete three-dimensional cultural relic, and the area without a match in the complete cultural relic is the part to be restored. Compared with the point cloud generation method, the result obtained by the present invention is more practical.
[0138] Compared with the existing 12 mainstream registration methods, the present invention has a higher accuracy rate, especially when the overlap rate is only 10%, the experimental accuracy of the present invention is almost unaffected. The present invention has been verified to have good robustness under multiple data sets. Figure 5 and Figure 6 The qualitative results of some experiments are shown. Figure 6 This is an application example in virtual restoration of cultural relics.
[0139] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.
[0140] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.
Claims
1. A local to global point cloud registration method based on internal and external point classification, characterized in that: The local to global point cloud registration method based on internal and external point classification comprises the following steps: Step 1: Randomly sample each point cloud to reduce the overall time complexity by reducing the number of points in the two point clouds; Step 2: Calculate the local coordinate system of the sampling point and its K nearest neighbors, and align the local coordinates of each point with the global coordinates of the point cloud; Step 3: Use the converted sampling points and their K nearest neighbors as the input of the network framework to map the coordinates to high dimensions, that is, calculate the local features of each point; Step 4: Calculate the similarity matrix of the local features of the two point clouds; Step 5, using an external storage unit to enhance the similarity matrix; Step 6: Concatenate the similarity matrix and the local feature matrix to form a global feature; Step 7: Decode the global features to obtain the masks of the source point cloud and the target point cloud, and remove the points in the non-overlapping area; Step 8: Train the network framework to obtain the points in the overlapping area, and then use ICP to align the points in the overlapping area.
2. The local to global point cloud registration method for internal and external point classification as claimed in claim 1, characterized in that: The local coordinate system where the sampling point and its K nearest neighbors are calculated in step 2, and the local coordinates of each point are aligned with the global coordinates of the point cloud, including the following steps: The source point cloud and the target point cloud are and Xi and Yj represent the i-th point of the source point cloud X and the j-th point of the target point cloud Y respectively. The K nearest neighbors of each point in the source point cloud and the target point cloud are calculated; the covariance matrix is estimated using the k nearest neighbors of Xi, and the covariance matrix is decomposed using the singular value decomposition method. Its eigenvalue is taken as 1 as the corresponding local coordinate system, and finally its local coordinates are aligned with the global coordinates of the point cloud.
3. The local to global point cloud registration method based on internal and external point classification according to claim 1, characterized in that: The step 3 uses the converted sampling points and their K nearest neighbors as the input of the network framework, and maps the coordinates to a high dimension, including the following steps: The dimension of the converted sampling points and their K nearest neighbors is increased to 64 dimensions using a multi-layer perceptron, and the edge convolutional network is used to further extract features to obtain the local features F of the source point cloud and the target point cloud respectively. x and F y .
4. The local to global point cloud registration method based on internal and external point classification according to claim 1, characterized in that: The calculation of the similarity matrix of the local features of the two point clouds in step 4 includes the following steps: Use Fx multiplied by the transposed matrix of Fy to approximate the similarity matrix H between the source point cloud and the target point cloud.
5. The local to global point cloud registration method based on internal and external point classification according to claim 1, characterized in that: The step 5 of enhancing the similarity matrix by using an external storage unit comprises the following steps: First, initialize the external storage unit M k and M v , use M k Multiply by H and use scale normalization to get S, and use the external memory unit M v Multiply it with S to get the enhanced similarity matrix F out .
6. The local to global point cloud registration method based on internal and external point classification according to claim 1, characterized in that: The step six of concatenating the similarity matrix and the local feature matrix to form a global feature includes the following steps: Use max pooling to solve F separately x and F y The global feature g x and g y ; The global feature g x and g y According to the second dimension, it is copied N times and the enhanced similarity matrix F after enhancement is obtained. out Concatenate in the first dimension to get the final global features; The step 7 of decoding the global features to obtain masks of the source point cloud and the target point cloud, and removing points in non-overlapping areas, includes the following steps: MLP is applied to decode the above cascade features. The decoding layer passes through five MLP layers with sizes of 1024, 512, 256, 128 and 2 respectively. The last layer uses the Average-Pooling function to increase the robustness of the network. The Sigmod activation function is used after each layer to ensure that the output value is between 0 and 1, and the mask of the target point cloud is obtained. For values greater than 0.5 in the mask, the value is set to 1, otherwise it is set to 0. Points with a value of 0 are outliers and are removed. The point cloud after removing the outliers uses the ICP-based method to solve the rotation and translation matrix. The training network framework in step eight obtains points in the overlapping area, and then aligns the points in the overlapping area using ICP, including the following steps: During the training process, the cross entropy error between the predicted mask and the true mask in step 7 is used as the loss function. The Adam optimizer is used to optimize the network. The initial learning rate is 0.001, and 500 epoches are trained. The learning rate is reduced to 0.1 times the original at the 10th and 300th training times, respectively.
7. A local to global point cloud registration system based on internal and external point classification, which implements the local to global point cloud registration method based on internal and external point classification as claimed in any one of claims 1 to 6, characterized in that: The local to global point cloud registration system based on internal and external point classification includes: Point cloud sampling module, used to randomly sample each point cloud, reducing the overall time complexity by reducing the number of points in the two point clouds; The local coordinate system module is used to calculate the local coordinate system of each point in the two point clouds, eliminate the sign ambiguity of the local coordinate system in a simple and effective way, and finally align it with the global coordinate system; Local feature calculation module, used to calculate the local features and global features of the aligned point cloud; Similarity matrix enhancement module, which is used to calculate the similarity matrix of local features of point cloud and use external memory units for enhancement; Feature aggregation module, used to concatenate and aggregate the enhanced similarity matrix and global features; A feature decoding module is used to decode the above cascade features using a multi-layer perceptron; An outlier removal module is used to apply the obtained decoding to the source point cloud so as to remove non-corresponding points in the source point cloud; The registration module is used to train the network framework and perform registration using an ICP-based registration method.
8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the local to global point cloud registration method based on internal and external point classification as described in any one of claims 1-6.
9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the local to global point cloud registration method based on internal and external point classification as described in any one of claims 1 to 6.
10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the local to global point cloud registration system based on internal and external point classification as described in claim 7.
Citation Information
Cited By
Geographic surveying and mapping data processing analysis method and system based on big data
CN119862519A
