A three-dimensional point cloud registration method based on geometric transformer and Gaussian mixture model
By combining geometric Transformer and Gaussian mixture model, the problem of registration difficulties caused by noise and outliers in point clouds is solved, and efficient point cloud registration in complex environments is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-02-23
- Publication Date
- 2026-03-24
AI Technical Summary
Existing technologies struggle to accurately register point clouds containing significant noise and outliers, and commonly used methods often prove ineffective.
A method based on geometric Transformer and Gaussian mixture model is adopted. Point cloud features are extracted through PointNet network, overlapping regions are identified by Transformer network with geometric relationship encoding, matching parameters are calculated by Gaussian mixture model, and finally registration is performed by solving the transformation matrix through weighted singular value decomposition.
Even with noise and outliers in the point cloud, it can quickly and accurately complete 3D point cloud registration, improving robustness and accuracy.
Smart Images
Figure CN116363177B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a three-dimensional point cloud model registration technology, in particular to a three-dimensional point cloud registration method based on geometric Transformer and Gaussian mixture model. BACKGROUND
[0002] Point cloud registration is a basic problem in the fields of robots, computer vision, cultural relic restoration, etc., and its purpose is to find the best transformation (rotation, translation and scaling) of aligning two point clouds.
[0003] When the real correspondence between point clouds is known, and the points are affected by zero-mean Gaussian noise, the registration problem is easy to solve. However, in practice, the correspondence is either unknown or contains many outliers, resulting in poor registration results of the solver of point cloud registration. For registration of unknown or uncertain correspondence, the commonly used method either depends on the availability of initial guess of unknown transformation (for example, iterative closest point method ICP), or assumes that there is a small set of outliers (for example, RANSAC method). These algorithms may not work well in the case of large amount of outliers and noise in point clouds. SUMMARY
[0004] In view of the deficiencies of the prior art, the purpose of the present application is to provide a three-dimensional point cloud registration method based on geometric Transformer and Gaussian mixture model, which can still be accurately processed in the case of a large amount of noise and outliers in point clouds.
[0005] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:
[0006] A three-dimensional point cloud registration method based on geometric Transformer and Gaussian mixture model, comprising the following steps:
[0007] Step 1, obtaining multiple three-dimensional point clouds of different angles by a three-dimensional scanner;
[0008] Step 2, first extracting the local features of the two point clouds by using PointNet network, and then fusing the local features of a single point cloud with the global features of the two point clouds;
[0009] Step 3, using a Transformer network with geometric relationship coding to identify the overlapping area between the two point clouds;
[0010] Step 4, using Gaussian mixture model (Gaussian Mixture Models, GMM) to represent the input point cloud, and calculating the matching parameters by using the point cloud features, the overlap score and the probability matrix;
[0011] Step 5, calculating a matching matrix from matched Gaussian mixture model (GMM) parameters to complete registration of the three-dimensional point cloud.
[0012] The application also has the following technical features:
[0013] Preferably, the PointNet network in step 2 extracts global features of the point cloud through a multi-layer perception and maximum pooling, and then connects the 1024-dimensional global features and the 64-dimensional local features through a concat operation and outputs the features of each point through a fully connected layer.
[0014] Further, the PointNet network has the following feature dimensions of each layer output: 64, 64, 64, 128, and 1024.
[0015] The fully connected layer has the following feature dimensions of each layer output: 1024, 1024, and 512.
[0016] Preferably, the geometric relationship in step 3 is composed of three handcrafted features through a concat operation.
[0017] The three handcrafted features are the distance between point pairs in the point cloud, the ternary angle, and the difference value between the local triangle perimeter.
[0018] Further, the distance between point pairs in the point cloud is denoted as ρ i,j , which is defined as: ρ i,j =||p i -p j ||2, where p i and p j are two points in a three-dimensional space.
[0019] For a point p i , its two nearest neighbors are denoted as and These three points form a ternary group, and the ternary angle is defined as Therefore, the ternary angle is denoted as
[0020] For a point p i , its two nearest neighbors are denoted as and These three points form a triangle, and the difference in the triangle perimeter is denoted as η i,j , which is expressed as:
[0021]
[0022] Therefore, the final geometric relationship g i,j is expressed as: g i,j =concate[ρ i,j ,αi,j ,η i,j ], where concat denotes the concatenation operation.
[0023] Preferably, the Transformer network in step 3 is composed of several stacked self-attention and cross-attention layers.
[0024] The self-attention is used to learn the global contextual information of each point cloud in the feature space and the geometric space.
[0025] For point P, the feature representation is d t , where d represents the dimension of the hidden feature, and the output feature is the weighted sum of all projected input features, denoted as:
[0026]
[0027] where a i,j is the weight coefficient, calculated by the self-attention score .
[0028]
[0029] g i,j represents the geometric relation embedding, and respectively represent the weight of the query vector, the key vector, and the value vector. is the projection weight of the geometric relation embedding, and f i represents the i-th feature of point P, and f j represents the j-th feature of point P.
[0030] The cross-attention is used to interact the features of two point clouds; the cross-attention score is denoted as:
[0031]
[0032] where represents the i-th feature of point P, and f represents the j-th feature of point Q.
[0033] Preferably, the Gaussian Mixture Model in step 4 establishes a multi-modal generative probability distribution in three-dimensional space as a weighted sum of L Gaussian densities, in the form of:
[0034]
[0035] Each density N(x∣μ j ,Σ j ) is called a component of the Gaussian Mixture Model, and uses the mean μ j and the covariance Σj Parameterization is performed, N(x | μ j ,Σ j ) represents the Gaussian distribution density function, i.e. probability, of the jth sub-model, and x represents the point cloud data.
[0036] Further, the point cloud feature set P is represented by a Gaussian mixture model as follows:
[0037]
[0038] wherein represents the probability of the point cloud P constituting the jth sub-model; represents the Gaussian distribution density function of the jth sub-model constituted by the input point cloud data;
[0039] The parameters of the Gaussian mixture model are represented as follows:
[0040]
[0041] wherein, the constant ∈ = 1e-4, s p and s q are probability matrices, represents the ith row and jth column of the probability matrix; o p is an overlap score, represents a scalar mixing weight, is a 3*1 average vector, is the jth component of a 3*3 covariance matrix.
[0042] Preferably, the matching matrix calculated in step 5 is calculated by optimal transport of the matching Gaussian mixture model (GMM) parameters and the feature centroid, and then the transformation matrix between the two point clouds is solved by a weighted SVD singular value decomposition method.
[0043] The matching matrix is calculated Γ The process is represented as:
[0044]
[0045] s.t.,Γ1 M = π p ,Γ · 1 N = π q ,Γ ij ∈ [0, 1]
[0046] wherein and represent the ith point of the point cloud P and the jth point of the point cloud Q, respectively.
[0047] After obtaining the matching matrix, the final transformation matrix is calculated by the following formula:
[0048]
[0049] is a 3*1 average vector belonging to the point cloud P, is a 3*1 average vector belonging to the point cloud q. ij represents the i-th row and j-th column of the matching matrix.
[0050] Compared with the prior art, the present application has the following technical effects:
[0051] The three-dimensional point cloud registration network model proposed by the present application is composed of five parts: point cloud feature extraction, point cloud feature fusion, a Transformer network embedding geometric relationship for aggregating features, GMM for solving matching parameters, and weighted singular value decomposition (SVD); specifically, the global features of the points are extracted using the PointNet network, then the 64-dimensional local features of a single point cloud and the 1024-dimensional global features of two point clouds are subjected to a concate operation and fused into 2112-dimensional features, then the Transformer network with geometric relationship coding is used to aggregate the features of the point clouds, helping the network to identify the overlapping areas between the point clouds; in addition, the Gaussian Mixture Models (GMM) are used to represent the input point clouds, and the matching parameters are calculated using the point cloud features, the overlap score, and the probability matrix; finally, the matching matrix is calculated from the matching GMM parameters, and the three-dimensional point cloud registration is completed; in order to speed up the robustness of the point cloud registration model, the local features and the global features of the point cloud are fused in the feature extraction part; in addition, the Transformer network is used to aggregate the features, improving the robustness of the features, and finally the GMM model is used to calculate the matching matrix; this enables the point cloud registration network constructed by the present application to quickly and accurately complete the three-dimensional point cloud registration even when the point cloud contains a large amount of noise and outliers. BRIEF DESCRIPTION OF DRAWINGS
[0052] Figure 1 is a flowchart of the present application;
[0053] Figure 2 is a point cloud registration network model diagram;
[0054] Figure 3 is a source point cloud effect diagram of the terracotta warriors;
[0055] Figure 4 is a target point cloud effect diagram of the terracotta warriors;
[0056] Figure 5 is an effect diagram of the terracotta point cloud after registration. Detailed Implementation
[0057] The specific content of the present invention will be further explained in detail below with reference to the embodiments.
[0058] Reference Figures 1-2 This embodiment provides a 3D point cloud registration method based on feature interaction and reliable correspondence estimation, including the following steps:
[0059] Step 1: Obtain 3D point clouds from multiple perspectives using a 3D scanner;
[0060] Step 2: First, the PointNet network is used to extract local features from two point clouds. Then, the local features of a single point cloud are fused with the global features of the two point clouds. PointNet consists of a weighted MLP, and the output feature dimensions are 64, 64, 64, 128, and 1024, respectively. The 1024-dimensional feature is then max-pooled to obtain global features. Next, the 64-bit local features of a single point cloud are fused with the 1024-dimensional global features of the two point clouds to obtain 2112-bit features. Finally, the final features of the point cloud are output through a fully connected layer. The feature dimensions output by each fully connected layer are 1024, 1024, and 512.
[0061] Step 3: Then, the Transformer network with geometric relationship encoding is used to identify the overlapping region between the two point clouds; the geometric relationship is composed of three handmade features through the concatenation operation. The three handmade features are the distance between point cloud pairs, the ternary angle, and the difference in the perimeter of the triangle.
[0062] The distance ρ between point cloud pairs i,j Defined as: ρ i,j =||p i -p j ||2, where p i and p j Let them be two points in three-dimensional space;
[0063] The definition of a three-dimensional angle is: for point p i The two nearest neighbors are represented as and These three points form a triple; we define Then the three angles are represented as
[0064] The difference in the perimeters of triangles is defined as: for point p i The two nearest neighbors are represented as and These three points form a triangle; then the difference η in the perimeter of the triangle is... i,j Represented as:
[0065]
[0066] So the final geometric relation g i,j is represented as: g i,j = concate [p i,j , a i,j , h i,j ]; where concate represents the concatenation operation;
[0067] And the geometric Transformer is stacked by multiple self-attention and cross-attention, where the self-attention aims to learn the global context information of each point cloud in the feature space and the geometric space;
[0068] For point P, the feature is represented as d t , where d represents the dimension of the hidden feature, and the output feature is the weighted sum of all the projected input features, and the weighted sum Z i is represented as:
[0069]
[0070]
[0071] where a i,j is the weight coefficient, which is calculated by the self-attention score ;
[0072]
[0073] g i,j represents the geometric relation embedding, and respectively represent the weight of the query vector, the key vector and the value vector; is the projection weight of the geometric relation embedding; f i represents the i-th feature of point P, and f j represents the j-th feature of point P.
[0074] And the cross-attention aims to interact the features of two point clouds, helping the network better identify the overlapping area of the two point clouds; the cross-attention is different from the self-attention in the solution of the attention score, and the cross-attention score is represented as:
[0075]
[0076] where represents the i-th feature of point P, represents the j-th feature of point Q.
[0077] Step 4, using Gaussian Mixture Models (GMM) to represent the input point cloud, using point cloud features, overlap score, probability matrix to calculate matching parameters; wherein the Gaussian Mixture Models (GMM) establishes a multi-modal probability distribution in three-dimensional space as a weighted sum of L Gaussian densities, which is in the form of:
[0078]
[0079] Each density N(x|μ j ,Σ j ) is called a component of a GMM, and is parameterized by the mean μ j and the covariance Σ j ; N(x|μ j ,Σ j ) represents the Gaussian distribution density function of the jth sub-model, i.e., the probability; x represents the point cloud data; the set P of point cloud features is represented by GMM as:
[0080]
[0081] And the parameters of GMM are:
[0082]
[0083] Wherein, the constant ∈ = 1e-4, s p and s q are probability matrices, represent the i-th row and j-th column of the probability matrix; o p is the overlap score, represents the scalar mixing weight, is a 3*1 average vector, is the jth component of the 3*3 covariance matrix.
[0084] Step 5, the matching GMM parameters and the centroid of the feature are calculated by the optimal transport to calculate the matching matrix, and then the transformation matrix between the two point clouds is solved by the SVD singular value decomposition method with weighting, completing the registration of the three-dimensional point cloud; the matching matrix Γ Process is represented as:
[0085]
[0086] s.t.,Γ1 M = π p ,Γ · 1 N = π q ,Γ ij ∈ [0,1]
[0087] Wherein and Pi and Pj represent the i-th point of point cloud P and the j-th point of point cloud Q, respectively;
[0088] After obtaining the matching matrix, the final transformation matrix is calculated by the following formula:
[0089]
[0090] is a 3*1 average vector belonging to point cloud P, is a 3*1 average vector belonging to point cloud q, Γ ij Pi,j represents the i-th row and the j-th column of the matching matrix.
[0091] Figure 5 is a point cloud registration result after registration by the method of the present application, and it can be seen from the figure that the method of the present application can still be robustly processed when a large amount of noise and abnormal points are contained between point clouds.
Claims
1. A 3D point cloud registration method based on geometric Transformer and Gaussian mixture model, characterized in that, Includes the following steps: Step 1: Obtain 3D point clouds from multiple perspectives using a 3D scanner; Step 2: First, use the PointNet network to extract local features from the two point clouds, and then fuse the local features of a single point cloud with the global features of the two point clouds. Step 3: Then, use a Transformer network with geometric relationship encoding to identify the overlapping region between the two point clouds; Step 4: Use a Gaussian mixture model to represent the input point cloud, and calculate the matching parameters using point cloud features, overlap score, and probability matrix. Step 5: Calculate the matching matrix from the matched Gaussian mixture model parameters to complete the registration of the 3D point cloud; The geometric relationship in step 3 is composed of three hand-crafted features through a concaten operation; The three handcrafted features are the distance between point cloud point pairs, the difference between ternary angles, and the difference between the perimeters of local triangles.
2. The 3D point cloud registration method based on geometric Transformer and Gaussian mixture model as described in claim 1, characterized in that, In step 2, the PointNet network extracts global features of the point cloud through a multilayer perceptron and max pooling. Then, the 1024-dimensional global features and 64-dimensional local features are concatenated through a concatenation operation, and the features of each point are output through a fully connected layer.
3. The 3D point cloud registration method based on geometric Transformer and Gaussian mixture model as described in claim 2, characterized in that, The dimensions of the output features of each layer of the PointNet network are 64, 64, 64, 128, and 1024, respectively. The dimensions of the output features of each layer of the fully connected layer are 1024, 1024, and 512, respectively.
4. The 3D point cloud registration method based on geometric Transformer and Gaussian mixture model as described in claim 1, characterized in that, The distance between the point cloud point pairs Defined as: ,in and Let them be two points in three-dimensional space; For point The two nearest neighbors are represented as and These three points form a triple, defined as follows: The three angles mentioned above are represented as ; For point The two nearest neighbors are represented as and These three points form a triangle, then the difference in the perimeter of the triangle is... Represented as: , , , , Therefore, the final geometric relationship Represented as: , where concate represents a join operation.
5. The 3D point cloud registration method based on geometric Transformer and Gaussian mixture model as described in claim 1, characterized in that, The Transformer network in step 3 consists of several stacked self-attention and cross-attention layers; The self-attention mentioned above is used to learn the global context information of each point cloud in the feature space and geometric space; For point P, the feature is represented as: , The output features represent the feature dimensions of each point. The weighted sum of all projected input features is represented as: in These are weighting coefficients, derived from self-attention scores. calculate; This represents the embedding of geometric relations, while These represent the query vectors respectively. The weights of the key vector K and the value vector V. Projection weights embedded for geometric relations. The first point P represents the first... h One characteristic, The first point P represents the first... m One characteristic, h and m These represent the coordinate indices of a point in three dimensions; The cross-attention is used to interact with the features of two point clouds; cross-attention score Represented as: in, The first point P represents the first... h One characteristic, Represents the point cloud Q's first... m One characteristic, h and These represent the coordinate indices of a point in three dimensions.
6. The 3D point cloud registration method based on geometric Transformer and Gaussian mixture model as described in claim 1, characterized in that, In step 4, the Gaussian mixture model establishes the multimodal generation probability distribution in three-dimensional space as a weighted sum of L Gaussian densities, in the form of: Each density The components are called the means of a Gaussian mixture model. Covariance Perform parameterization; This indicates that the input data constitutes the first... l The probability of each sub-model; Indicates the first l The Gaussian distribution density function of each sub-model is the probability; This represents point cloud data.
7. The 3D point cloud registration method based on geometric Transformer and Gaussian mixture model as described in claim 6, characterized in that, The point cloud P of the aforementioned point cloud features is represented by a Gaussian mixture model as follows: in, This represents the first point cloud data constructed from the input point cloud data. b Gaussian distribution density function of each sub-model; The parameters of the Gaussian mixture model are expressed as follows: in, , It is a probability matrix. Let represent the a-th row and b-th column of the probability matrix corresponding to point cloud P; These are overlapping fractions.
8. The 3D point cloud registration method based on geometric Transformer and Gaussian mixture model as described in claim 1, characterized in that, In step 5, the matching matrix is calculated by using the centroids of the matched Gaussian mixture model parameters and features through optimal transmission to calculate the matching matrix, and then the transformation matrix between the two point clouds is solved by the weighted SVD singular value decomposition method. Calculate the matching matrix The process is represented as follows: in , and Let i and j represent the i'th point in point cloud P and the j'th point in point cloud Q, respectively. After obtaining the matching matrix, the final transformation matrix is calculated using the following formula: in, It is a 3*1 average vector belonging to the point cloud P. It is a 3*1 average vector belonging to the point cloud q. This represents the i'th row and j'th column of the matching matrix.