A point cloud registration method based on neural network reconstruction of Gaussian mixture model

Abstract

The application provides a point cloud registration method based on a neural network reconstruction Gaussian mixture model, comprising obtaining two point cloud description information of the same object or scene in a real scene under noise and density difference, removing pose difference information between the point clouds by using a three-dimensional space transformation module; in a feature space, fusing the point cloud data by using a stacked information interaction module; based on the updated features, designing a clustering network to predict the posterior probability of each point belonging to each component in the Gaussian mixture model; combining the two point cloud data by a non-iterative method to restore the Gaussian mixture model of the target object, obtaining a transformation matrix between the two point cloud data, and applying the transformation matrix to complete the registration of the two point cloud data. The application has the advantages of robust algorithm performance and low running time in a real scene with noise and density difference.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional point cloud data processing, specifically to a point cloud registration method based on neural network reconstruction of Gaussian mixture models. Background Technology

[0002] In recent years, with the increasingly widespread application of 3D sensors such as LiDAR and Kinect, the registration problem of native point cloud data of the same object or scene acquired at different angles and distances has attracted increasing attention from companies and researchers. In fields such as 3D reconstruction and autonomous driving, due to the deviation in angle and distance when data is acquired at different times, two point clouds describing the same object or scene not only have rotation and translation errors, but also differences in density and noise intensity. Therefore, many applications require transforming point cloud data of the same object or scene in different coordinate systems to the same coordinate system. The point cloud registration task achieves this goal by estimating the rotation and translation transformation matrices.

[0003] However, due to the disordered, low-texture, and environmentally sensitive nature of point cloud data, the transformation matrix estimation problem, which is the core of point cloud registration, is extremely challenging and easily leads to local optima. To improve algorithm performance, researchers have proposed two main types of methods: methods based on correspondence search and methods that do not require correspondence search. Methods based on correspondence search estimate the transformation matrix between two point clouds by obtaining accurate point-to-point correspondences. Conversely, methods that do not require correspondence search do not need to estimate any correspondences; they use deep learning to iteratively optimize and minimize the differences between the two point clouds in the feature space, and then predict the corresponding transformation matrix.

[0004] Correspondence search-based methods have been around for some time and include both traditional and deep learning-based approaches. Traditional methods mainly consist of two processes: correspondence search and transformation estimation. Correspondence search aims to find a corresponding point in another paired point cloud for each point; transformation estimation calculates the transformation matrix based on these corresponding points. In optimization-based traditional methods, these two processes are performed iteratively, continuously improving the accuracy of obtaining corresponding points. In addition, there are registration methods based on point-to-Gaussian distribution correspondence search. These methods are based on Gaussian mixture models, iteratively finding the correspondence between points and distributions, constructing Gaussian mixture models based on two point cloud datasets respectively, and estimating the transformation matrix between the point clouds. Although traditional methods do not require training data, they require numerous special strategies for suppressing noise and density differences, significantly reducing the algorithm's execution efficiency. Furthermore, due to the lack of global information guidance, these methods are prone to getting trapped in local optima during iteration. In recent years, with the development of deep learning, machine learning-based point cloud descriptors have gradually become mainstream in registration tasks. These algorithms acquire local features with rigid transformation invariance for each point through supervised learning. Unlike traditional methods, registration methods based on these descriptors do not require an iterative process; they often directly estimate the transformation matrix based on the correspondence obtained from feature matching. Thanks to the powerful representation capabilities of deep learning, these methods have significantly improved accuracy and efficiency compared to traditional methods. However, these methods still have the following drawbacks: first, training the feature extraction model requires extensive manual annotation; second, their performance drops sharply when there is a significant difference between the input point cloud distribution and the training data distribution; and third, because the descriptor training phase and the transformation matrix estimation phase are completed independently, end-to-end training cannot be used to optimize descriptor extraction and improve registration accuracy.

[0005] Another type of method, which does not require a correspondence search, relies on neural networks. It extracts high-dimensional global features for each point cloud, obtains a transformation matrix based on the differences in these features, and iteratively minimizes the projection error between two features to optimize the transformation matrix. These neural networks employ an end-to-end training mode, using native point cloud data as input and the transformation matrix between the two point clouds as output, which helps optimize the accuracy of the registration task. However, this type of method has two drawbacks: first, it only uses the global features of the two point cloud data sets, ignoring the crucial local structural information of the point cloud data; second, the iterative optimization process reduces the algorithm's execution efficiency. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention combines the advantages of deep neural networks and traditional Gaussian mixture model algorithms, proposing a point cloud registration method based on neural network reconstruction of Gaussian mixture models. This method is based on a single Gaussian mixture model and employs a novel stacked information interaction neural network module to fuse information from two point clouds in the feature domain, reconstructing the Gaussian mixture model representation of the target object, and calculating the transformation matrix between the two point clouds based on this model representation.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a point cloud registration method based on neural network reconstruction of Gaussian mixture models, comprising: acquiring two point cloud descriptions of the same object or scene in a real scene under noise and density differences; removing pose difference information between point clouds using a three-dimensional spatial transformation module; fusing point cloud data in feature space using a stacked information interaction module; designing a clustering network based on updated features to predict the posterior probability of each point belonging to each component in the Gaussian mixture model; reconstructing the Gaussian mixture model of the target object by combining the two point cloud data in a non-iterative manner; obtaining the transformation matrix between the two point cloud data; and applying the transformation matrix to complete the registration of the two point cloud data.

[0008] Furthermore, obtaining the transformation matrix between two point cloud data sets includes obtaining the rotation transformation matrix and the translation transformation matrix between the two point cloud data sets.

[0009] Furthermore, obtaining two point cloud descriptions of the same object in a real scene under noise and density differences includes: based on the 3D object model data, performing at least two random samplings on each target object, normalizing the point cloud data obtained from the random sampling, and scaling the 3D coordinates of the two point cloud data to the same range.

[0010] Furthermore, after scaling the three-dimensional coordinates of the two point cloud datasets to the same range, random data augmentation was performed on the two point cloud datasets to obtain training data.

[0011] Further random augmentation of point cloud data includes: random rotation within a range of 0 to 45 degrees; and / or random translation within a range of 0 to 50 centimeters.

[0012] Furthermore, the method of removing large pose differences between point clouds using a 3D spatial transformation module includes: inputting the description information of the two point clouds into a 3D spatial transformation network with shared parameters, regressing the transformation matrix using a learning-based approach, and performing geometric transformation on the description information of the two point clouds to remove pose differences.

[0013] Furthermore, the use of stacked information interaction modules to fuse point cloud data includes: designing a mutual attention subnetwork within the information interaction module, obtaining weighted global information of each point on another point cloud based on feature similarity, and updating the features of each point by combining the initial point features.

[0014] Furthermore, the design of the clustering network to predict the posterior probability of each point belonging to each component in the Gaussian mixture model includes: performing max pooling on the feature maps of the two point cloud data to obtain the corresponding global feature vectors, and merging the global feature vectors with the local features of each point in the corresponding point cloud data in the channel dimension and inputting them into a clustering network with shared parameters to predict the posterior probability of the Gaussian components of all points.

[0015] Furthermore, based on the posterior probabilities predicted by the clustering network, all parameters of the Gaussian mixture model are constructed in a non-iterative manner, and the transformation matrix representing the rotation and translation between the two point cloud data is indirectly obtained.

[0016] Compared with the prior art, the beneficial effects of the present invention are:

[0017] This invention presents a point cloud registration method based on neural network-reconstructed Gaussian mixture models. It employs a mutual attention mechanism to fuse information between two point clouds, directly reconstructs the Gaussian mixture model of the target object using the updated fusion features, and estimates the transformation relationship between the two point clouds based on this model to complete the registration task. Quantitative and qualitative experimental analysis shows that, compared with existing technologies, this invention exhibits robust algorithmic performance and lower runtime in handling real-world scenes with noise and density differences. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an embodiment of the point cloud registration method for reconstructing Gaussian mixture models based on neural networks according to the present invention.

[0019] Figure 2 This is a network topology diagram of an embodiment of the point cloud registration method based on neural network reconstruction of Gaussian mixture models according to the present invention.

[0020] Figure 3 This is a diagram of the mutual attention subnetwork topology of an embodiment of the point cloud registration method based on neural network reconstruction of Gaussian mixture models according to the present invention.

[0021] Figure 4 This is a registration effect diagram of an embodiment of the point cloud registration method based on neural network reconstruction of Gaussian mixture model according to the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] See Figure 1 The point cloud registration method based on neural network reconstruction of Gaussian mixture models of the present invention first performs step S1 to obtain two point cloud descriptions of the same object or scene in a real scene under noise and density differences. A three-dimensional spatial transformation module is then used to remove large pose differences between the point cloud descriptions. Specifically, this embodiment uses existing three-dimensional object model data to perform two random samplings on each target object to simulate random noise and density differences. Subsequently, the point cloud data is normalized, scaling the three-dimensional coordinates of the two point cloud data sets to the same range, and random data augmentation is performed to obtain training data. Random augmentation of the point cloud data includes random rotation within a range of 0 to 45 degrees and / or random translation within a range of 0 to 50 centimeters. Furthermore, removing large pose differences between point clouds using a three-dimensional spatial transformation module can be achieved by inputting the two point cloud data sets into a three-dimensional spatial transformation network with shared parameters, regressing the transformation matrix through model training, and performing geometric transformations on the two point cloud data sets to remove large pose differences, thereby improving the robustness of convolutional network feature extraction.

[0024] Then, step S2 is executed. In the feature space, the stacked information interaction module is used to fully fuse point cloud information to update the features of each point. Specifically, this embodiment designs a mutual attention subnetwork within the information interaction module to obtain the weighted global information of each point on another point cloud description information based on feature similarity, and combines it with the initial point features to complete the update of the features of each point. By using this module multiple times on different feature dimensions, full interaction between the two point cloud description information is achieved.

[0025] Next, step S3 is executed: based on the updated features, a clustering network is designed to predict the posterior probability of each point belonging to each component of the Gaussian mixture model. Specifically, the feature maps of the two point cloud datasets are max-pooled to obtain the corresponding global feature vectors. These global feature vectors are then merged with the local features of each point in the corresponding point cloud dataset along the channel dimension and input into a clustering network with shared parameters to predict the posterior probability of the Gaussian components of all points.

[0026] Then, step S4 is executed to construct a Gaussian mixture model using the predicted probabilities and point cloud data, and indirectly obtain the transformation matrix between the two point clouds. Specifically, based on the posterior probabilities predicted by the clustering network, the Gaussian mixture model of the target object is reconstructed in a non-iterative manner by combining the information from the two point clouds, and the geometric relationship between the two point cloud data is indirectly obtained, that is, the rotation transformation matrix and translation transformation matrix are obtained to complete the point cloud registration task.

[0027] Finally, step S5 is executed to apply the transformation matrix to complete the registration of the two point clouds.

[0028] The following is combined Figure 2 This embodiment describes the topology of the point cloud registration method based on neural network reconstruction of Gaussian mixture models. This embodiment uses source point cloud 1 and target point cloud 2 as inputs. Specifically, source point cloud 1 can be represented by the following formula: The target point cloud 2 is represented by the following formula: Transform matrix T from X to Y gt The method employs supervised learning of a clustering function to predict the posterior probability of each Gaussian component corresponding to each point in the point cloud data. This is combined with two point cloud descriptions to reconstruct a Gaussian mixture model of the target object. The predicted rotation and translation matrices are indirectly obtained during this process. Preferably, the topology structure in this embodiment mainly includes two parts: clustering and parameter solving. Since each module contained within is differentiable, this method can be trained end-to-end.

[0029] Specifically, the clustering component used in this embodiment includes a three-dimensional spatial transformation network 10, an information interaction module 20, an information interaction module 50, and a clustering prediction module 80.

[0030] The source point cloud 1 and the target point cloud 2 are respectively input into the three-dimensional spatial transformation network 10 with shared parameters. The features of each point are extracted step by step through the convolutional layers 11, 12 and 13. Then, the global features are extracted using the pooling layer 14, and the preliminary predicted transformation relationship is obtained through three fully connected layers 15, 16 and 17. The corresponding transformations are applied to the two point cloud data respectively to eliminate their large pose differences.

[0031] The information interaction module 20 consists of a local feature extraction unit 30 and a mutual attention unit 40. First, local geometric features are extracted for each point in this stage. Then, in the feature dimension, these features are fused through mutual attention to achieve interactive feature information fusion. The local feature extraction unit 30 takes the point cloud data output from the 3D spatial transformation network 10 as input and includes two ReLU-activated convolutional layers 31 and 32. Within this unit, local geometric features F are progressively extracted from point cloud data X and point cloud data Y using convolutional layers with shared parameters. x and F yThey are then input together into the mutual attention unit 40 to achieve information interaction in the feature space. F is updated using the mutual attention unit. x For example, let F x With F y Simultaneously input mutual attention unit 40, at which point, as follows: Figure 3 Q is F x K and V are both F y After passing Q, K, and V through their respective fully connected layers 101, 102, and 103, the weighted global information of V corresponding to each feature in Q is obtained by multiplying the similarity between Q and K with V. This weighted global information is then concatenated with the corresponding features in Q and fed into fully connected layers 104 and 105 to obtain the fused features. The calculation formula for the mutual attention unit is as follows:

[0032]

[0033] in, F represents x The information obtained by weighting the feature of the i-th point in the c-dimensional feature space is given by A, where A is the similarity between Q and K, and softmax is the normalization exponential function. Subsequently, this information is compared with F... x Connect them along the feature dimensions at the corresponding positions to complete the connection of F. x The update. Similarly, F y Update in the same way. (F) x With F y After passing through information interaction module 20 and information interaction module 50, F x With F y Full information exchange was achieved across different dimensions, and the corresponding features were updated. and Where C represents the local feature dimension of each point after the information exchange is completed.

[0034] In this embodiment, the two information interaction modules 20 and 50 have the same structure. Information interaction module 50 consists of a local feature extraction unit 60 and a mutual attention unit 70. The local feature extraction unit 60 includes a convolutional layer 61 and a convolutional layer 62. The difference between information interaction module 50 and information interaction module 20 is that the local feature extraction units 30 and 60 in the two modules extract features of different dimensions. At the same time, the mutual attention units 40 and 70 perform information interaction in feature spaces of different dimensions.

[0035] The clustering prediction module 80 consists of a pooling layer 81 and several fully connected layers 82, 83, 84, 85, and 86. It takes a feature map of a point cloud as input each time and predicts the posterior probability of each point belonging to each component of a single Gaussian mixture model. The clustering prediction module 80 shares parameters when computing two input feature maps. Specifically, it first processes the input F...x Max pooling is used to obtain the global features of point cloud X. F x The local features of each point are merged with the global feature along the channel dimension to obtain... This ensures that each point possesses both local and global information. This feature is then processed through several fully connected layers to predict the posterior probability of the Gaussian component corresponding to the point. Where L represents the number of Gaussian components in the Gaussian mixture model. Similarly, the posterior probability of each point in the point cloud Y can be obtained.

[0036] The topological structure of the point cloud registration method based on neural network-reconstructed Gaussian mixture model also includes a parameter solving part, which comprises a Gaussian mixture model parameter solving module 90 and a transformation matrix solving module 110. Specifically, the Gaussian mixture model parameter solving module 90 calculates all parameters of the corresponding Gaussian mixture model based on the posterior probability predicted by the neural network and the original point cloud data. Firstly, based on the two input point clouds... and The posterior probability obtained by solving with a neural network and Calculate the weighted average of the two point cloud datasets for each Gaussian component. The calculation formula is shown in Equation 2:

[0037]

[0038] in This represents the l-th Gaussian component. and and represent the weighted average values ​​of point cloud X and point cloud Y in l, respectively.

[0039] Meanwhile, the mean μ of this Gaussian mixture model can be obtained by the following formula:

[0040]

[0041] In a Gaussian mixture model, the covariance matrix σ 2 The solution process is as follows:

[0042]

[0043] in

[0044] Furthermore, the weight π of each Gaussian component in the Gaussian mixture model is:

[0045]

[0046] Therefore, the parameters {π, μ, σ} of the Gaussian mixture model are used. 2} can represent the distribution information of the target object described by point cloud data X and point cloud data Y. Since neural networks can accurately predict the posterior probability of each point, compared with the traditional Gaussian mixture model registration method, this embodiment can construct an accurate Gaussian mixture model without iteration.

[0047] The transformation matrix solving module 110 first calculates the transformation relationship between point cloud data X and point cloud data Y to the distribution S described by the Gaussian mixture model, and indirectly obtains the transformation matrix between point cloud data X and point cloud data Y. Taking the solution of the transformation matrix from point cloud data X to distribution S as an example, its rotation matrix R... X It can be solved by the following formula:

[0048]

[0049] Among them U X and V X For matrix W X Λ X P X Λ X The left and right matrices obtained after singular value decomposition of μ, Λ X It is by The resulting diagonal matrix is ​​of dimension L×L, P X Is the value The projection matrix, e is a vector with a value of 1. After obtaining the rotation matrix from the point cloud data X to the Gaussian mixture model, the translation matrix t... X It can be calculated using the following formula:

[0050]

[0051] Therefore, the transformation matrix from point cloud data X to distribution S has been solved. Similarly, the transformation matrix from point cloud data Y to distribution S is shown below:

[0052]

[0053] From the equation relating X, Y, and S in Equation 8, we can obtain:

[0054]

[0055] Where R XY and t XY Let X and Y represent the rotation and translation transformation matrices, respectively, for transforming point cloud data X into point cloud data Y. Figure 2 The final predicted transformation matrix 3 is T. XY =[R XY , t XY ].

[0056] from Figure 4As can be seen, the point cloud registration method based on neural network reconstruction of Gaussian mixture model of the present invention can effectively register two different point cloud data and achieve good registration results.

[0057] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0058] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A point cloud registration method based on neural network reconstruction of Gaussian mixture model, characterized in that, The method comprises the following steps: Obtaining two point cloud description information of the same object or scene in the real scene under the noise and density difference, and removing the pose difference information between the point clouds by using a three-dimensional space transformation module; In the feature space, the point cloud data is fused by using a stacked information interaction module; Based on the updated features, a clustering network is designed to predict the posterior probability of each point belonging to each component in the Gaussian mixture model; The transformation matrix between the two point cloud data is obtained by restoring the Gaussian mixture model of the target object in a non-iterative manner, and the registration of the two point cloud data is completed by applying the transformation matrix. Wherein, The three-dimensional space transformation module is used to remove the large pose difference information between the point clouds, which comprises the following steps: The two point cloud description information is input into the three-dimensional space transformation network with shared parameters, and the transformation matrix is regressed by using a learning-based method to remove the pose difference between the two point cloud description information. Obtaining two point cloud description information of the same object in the real scene under the noise and density difference comprises the following steps: Based on the three-dimensional object model data, at least two random samplings are performed on each target object, and the three-dimensional coordinates of the two point cloud data are uniformly scaled to the same range after normalization processing; After the three-dimensional coordinates of the two point cloud data are uniformly scaled to the same range, random data augmentation is also performed on the two point cloud data to obtain training data; The random augmentation of the point cloud data comprises the following steps: Randomly rotating within 0 to 45 degrees; and / or randomly translating within 0 to 50 centimeters; The stacked information interaction module is used to fuse the point cloud data, which comprises the following steps: An attention subnetwork is designed in the information interaction module to obtain the weighted global information of each point on the other point cloud according to the feature similarity, and the initial point feature is updated; The clustering network is designed to predict the posterior probability of each point belonging to each component in the Gaussian mixture model, which comprises the following steps: The feature maps of the two point cloud data are respectively maximum-pooled to obtain corresponding global feature vectors, which are combined with the local features of each point in the corresponding point cloud data in the channel dimension and then input into the clustering network with shared parameters to predict the Gaussian component posterior probability of all points; The transformation matrix between the two point cloud data comprises the following steps: The rotation transformation matrix and the translation transformation matrix between the two point cloud data are obtained.

2. The point cloud registration method for reconstructing a Gaussian mixture model based on a neural network according to claim 1, wherein: Based on the posterior probability predicted by the clustering network, the parameters of the Gaussian mixture model are constructed in a non-iterative manner, and the transformation matrix representing the rotation and translation between the two point cloud data is indirectly obtained.

Images