Point cloud registration method and system based on feature extraction module and dual quaternion

By introducing local feature units and dual quaternions in point cloud registration, the problem of insufficient local features in point cloud registration is solved, the accuracy and robustness of point cloud registration are improved, and more efficient Feature extraction and noise resistance.

CN114638867BActive Publication Date: 2024-11-12XIDIAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202210317913.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-25
Publication Date
2024-11-12
Estimated Expiration
2042-03-25

AI Technical Summary

Technical Problem

The prior art lacks consideration of local features in point cloud registration, resulting in missing information during global feature extraction and insufficient noise robustness.

Method used

Using point cloud registration method based on feature extraction module and dual quaternary quaternary, we make up for the shortcomings of local features through local feature units, and use dual quaternary estimation to estimate rigid transformation, making full use of local information, double quaternary estimation represents rotation and translation for compact and precise transformation.

Benefits of technology

It effectively makes up for the local feature loss in the global feature extraction stage in point cloud registration, improves the accuracy and noise robustness of parameter estimation, and achieves higher registration accuracy and calculation efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114638867B_ABST
    Figure CN114638867B_ABST
Patent Text Reader

Abstract

The present invention discloses a point cloud registration method and system based on a feature extraction module and dual quaternion, which reads 3D point cloud data from a data set to obtain the three-dimensional coordinates and category labels of the points contained in each sample; randomly samples the points contained in each sample to obtain a template point cloud; translates and rotates the template point cloud to obtain a source point cloud, and divides the source point cloud and the template point cloud into a training set and a test set according to the category labels of the samples; constructs a point cloud registration model based on a feature extraction module and dual quaternion, and constructs a loss function of the point cloud registration model; trains the point cloud registration model with the divided training set, and performs point cloud registration on the divided test set using the trained point cloud registration model. The present invention fully exploits the global and local information in the point cloud, and effectively makes up for the lack of local features in the global feature extraction stage of point cloud registration.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a point cloud registration method and system based on a feature extraction module and a dual quaternion. Background Art

[0002] With the rapid development of 3D data acquisition technology, point cloud data collected by LiDAR, structured light sensors, and stereo cameras have been widely used. Therefore, this has attracted considerable attention in the development of algorithms for direct point cloud registration, classification, segmentation, tracking, etc. 3D rigid point cloud registration is a key task in many important applications in computer vision and robotics, including autonomous driving, surgical navigation, and simultaneous localization and mapping (SLAM). The goal of point cloud registration is to find a rigid transformation that aligns one point cloud with another. However, the inherent structural defects make it very difficult to use point clouds directly in deep learning architectures. The emergence of PointNet and its variants provides a structured representation of point clouds, which can overcome the difficulties of using point clouds directly in deep learning and improve processing speed.

[0003] Many advanced point cloud registration methods are described in the prior art. One of the most classic point cloud registration methods is the iterative closest point (ICP) algorithm, which estimates rigid body motion based on corresponding points and iteratively modifies corresponding points to minimize the distance from point to corresponding point. ICP and its extensions have been widely used in many applications. However, due to sensitivity to initialization, registration often fails due to being trapped in local minima. In recent years, learning-based methods have been designed to provide accurate alignment and speed up processing time when using point nets to extract global descriptors. Among these methods, PointNetLK and PCRNetar are typical deep learning-based registration techniques using PointNet. PointNetLK improves the classic LK algorithm and uses a gradient method to align the features extracted by PointNet. This method has significant advantages in initialization robustness and computational efficiency, but is susceptible to noise. PCRNet uses point nets as encoding functions to extract global features of point clouds and uses data-driven techniques for feature alignment. It can directly process point cloud data for registration without hand-crafted features, voxelization, or mesh generation.

[0004] However, based on the feature extraction without correspondence method proposed recently, the usual process is to extract the global features of the point cloud through the point network, and then perform a series of transformations on the global features to obtain the final registration result; however, one of the biggest drawbacks of using the point network is that these forget to consider the local features of the point cloud and do not make full use of the point cloud information. Therefore, the extracted global features will have a large amount of point information missing. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a point cloud registration method and system based on feature extraction module and dual quaternion in view of the deficiencies in the above-mentioned prior art, which makes up for the deficiencies of local features by using local feature units, and estimates the rigid transformation in feature alignment by using dual quaternion; the local feature unit feeds back the local information of each dimension to each point of the next layer, making full use of the local information; when extracting global features, more local information of the point cloud can be obtained by using the local feature units between adjacent layers; in addition, the dual quaternion can simultaneously represent rotation and translation in a common framework, and obtain a compact and accurate representation for rigid transformation. Compared with the existing free point cloud registration method, the method of the present invention does not need to find the corresponding point cloud, and has strong robustness to noise.

[0006] The present invention adopts the following technical solutions:

[0007] A point cloud registration method based on feature extraction module and dual quaternion reads 3D point cloud data from the data set to obtain the three-dimensional coordinates and category labels of the points contained in each sample; randomly samples the points contained in each sample to obtain a template point cloud; translates and rotates the template point cloud to obtain a source point cloud, and divides the source point cloud and the template point cloud into a training set and a test set according to the category labels of the samples; constructs a point cloud registration model based on feature extraction module and dual quaternion, and constructs a loss function of the point cloud registration model; trains the point cloud registration model with the divided training set, and uses the trained point cloud registration model to perform point cloud registration on the divided test set.

[0008] Specifically, each 3D point cloud data contains 2048 points, each point consists of 3D coordinates (x, y, z), and the 3D coordinates of the points contained in each sample and the category label of the sample are obtained.

[0009] Specifically, the points contained in each sample are randomly sampled to obtain 1024 points.

[0010] Specifically, the template point cloud is translated and rotated to obtain the source point cloud, and the source point cloud is divided into a training set and a test set as follows:

[0011] The data read from the original data set is set as the template point cloud; each template point cloud is rotated at a random angle and translated at a random distance, with the random angle being [-45°, 45°] and the random distance being [-1, 1], to obtain the source point cloud; the rotation angle and translation distance of each source point cloud corresponding to the template point cloud are set as the true value of the rigid transformation; 70% of the samples from different categories of source point clouds and template point clouds are selected as the training set, and the remaining 30% of the samples are used as the test set.

[0012] Specifically, the point cloud registration model based on feature extraction module and dual quaternion is constructed as follows:

[0013] The source point cloud and template point cloud are sent as input to the GLFNet feature extraction network, and the N*1152 feature matrix is ​​obtained after splicing. The feature matrices obtained from the source point cloud and template point cloud are respectively subjected to maximum pooling to obtain global features containing local information;

[0014] The extracted global features are concatenated to obtain features of size 1*2304; then the fully connected layer is input to extract elements of size N*8, and the 8-dimensional elements are defined as 8 parameters of the dual quaternion. The final rotation matrix and translation matrix are predicted through the 8 parameters to obtain the rigid transformation G. Finally, the source point cloud is transformed iteratively, and the updated source point cloud is used as the input of the GLFNet feature extraction network; after performing n iterations, the overall transformation G between the source point cloud and the template point cloud is obtained by combining all the postures in each iteration. est .

[0015] Furthermore, the GLFNet feature extraction network includes five MLPs of sizes 64, 64, 64, 128, and 1024, respectively. The LF unit is used to extract local features from the output features of each layer, and the features output from the shared MLP are used as the input of the LF unit of each layer; the output of the LF unit is concatenated with the output of the next layer of MLP and then used as the input of the next layer of MLP. The features extracted in each layer fuse local information, and the feature size of each layer is expanded to 64, 128, 128, 192, and 1152, respectively. The source point cloud and the template point cloud contain global features of local information. and They are:

[0016]

[0017]

[0018] Among them, MP is the maximum pooling operation, To expand to 1152-dimensional source point cloud features, The template point cloud features are expanded to 1152 dimensions.

[0019] Furthermore, the overall transformation G between the source point cloud and the template point cloud est for:

[0020] G est =G(n)×G(n-1)×…×G(1)

[0021] Among them, G(n) is the rotation matrix obtained by the nth iteration, and G(n-1) is the rotation matrix obtained by the n-1th iteration.

[0022] Specifically, the loss function L of the point cloud registration model is 3 Specifically:

[0023] L 3 =L 1 +0.007×L 2

[0024] Among them, L 1 is the chamfer loss, L 2 The loss is measured as the angle of the global transformation.

[0025] Specifically, the point cloud registration model is trained using the divided training set as follows:

[0026] The sample pairs of the training set are used as the input of the point cloud registration model. The source point cloud and the template point cloud are simultaneously input into the two branches of the point cloud registration model. The predicted rigid transformation of the two point clouds is used as the output of the point cloud registration model. The error between the predicted rigid transformation and the true rigid transformation and the error between the corresponding points are solved and the error is back-propagated to optimize the network parameters of the point cloud registration model and obtain a trained point cloud registration model.

[0027] Another technical solution of the present invention is a point cloud registration system based on a feature extraction module and a dual quaternion, comprising:

[0028] The reading module reads 3D point cloud data from the data set and obtains the three-dimensional coordinates and category labels of the points contained in each sample;

[0029] The sampling module randomly samples the points contained in each sample obtained by the reading module to obtain a template point cloud;

[0030] The division module translates and rotates the template point cloud obtained by the sampling module to obtain the source point cloud, and divides the source point cloud and the template point cloud into a training set and a test set according to the category label of the point cloud;

[0031] A construction module is used to construct a point cloud registration model based on a feature extraction module and a dual quaternion, and a loss function of the point cloud registration model is constructed;

[0032] The registration module uses the training set divided by the division module to train the point cloud registration model constructed by the construction module, and uses the trained point cloud registration model to perform point cloud registration on the test set divided by the division module.

[0033] Compared with the prior art, the present invention has at least the following beneficial effects:

[0034] The present invention is based on a point cloud registration method of a feature extraction module and a dual quaternion. The dual quaternion is used to estimate the rigid transformation. The dual quaternion represents the rotation and translation at the same time, and a compact and accurate rigid body transformation representation is obtained. A correspondence-free method is used. Without the need to extract the correspondence between points, the calculation cost is greatly reduced. The rigid transformation of the global features of the point cloud is directly estimated to obtain the rotation matrix and translation matrix required for the point cloud registration. A feature extraction network is designed to make up for the deficiency of local features, and the point cloud information is fully utilized to extract the global features of the point cloud containing local information, so as to fully explore the global and local information in the point cloud and effectively make up for the lack of local features in the global feature extraction stage of the point cloud registration.

[0035] Furthermore, the information required for point cloud registration mainly comes from the location information of the point cloud. The dataset provides the original coordinates of the point cloud for registration, so as to facilitate the registration task. At the same time, the setting of multiple categories in the dataset enables our model to realize the point cloud registration task of different shapes well, thus enhancing the fitting ability of the model.

[0036] Furthermore, all point clouds are sampled to 1024 points to unify the input size of the network and avoid model training difficulties caused by inconsistent input point cloud sizes.

[0037] Furthermore, the template point cloud is rotated and translated to obtain the source point cloud, which can ensure the integrity of the template point cloud and the source point cloud. The source point cloud and the template point cloud are essentially from the same point cloud, which can avoid the influence of noise points. Then we divide the source point cloud and the template point cloud into training set and test set according to the category for model training.

[0038] Furthermore, the feature extraction module can make up for the shortcomings of local features, fully explore the global and local information in the point cloud, effectively make up for the lack of local features in the global feature extraction stage of point cloud registration, and provide accurate feature representation for subsequent rigid body transformation estimation to improve the accuracy of parameter estimation; at the same time, the use of dual quaternions can simultaneously represent rotation and translation. Relying on the dual quaternions, we can unify translation and rotation into the same framework, so that the extracted global feature information can be fully utilized.

[0039] Furthermore, in the GLFNet feature extraction network, we extract features for each point in the point cloud. When extracting features for each point, the information between points is independent of each other and cannot interact with each other. For this reason, we designed a local feature unit to input low-dimensional local features to the next layer. The unit feeds back the local information of each dimension to each point in the next layer, making full use of the local information of each dimension, and each point will have the feature information of other points. When extracting global features, using units between adjacent layers will obtain more low-dimensional local information of the point cloud and strengthen information interaction, thereby achieving information fusion.

[0040] Furthermore, the rotation and translation are unified into the overall transformation G est Therefore, the model performance can be considered from the perspective of overall rigid transformation.

[0041] Furthermore, a large number of point cloud registration models use chamfer loss to minimize the distance between the template point cloud and the source point cloud. This method does not consider the rotation and translation of rigid changes from a global perspective. On this basis, we add a global transformation G est The loss is reduced by optimizing the model from two perspectives: point pair and transformation matrix. This makes the rotation matrix and translation matrix finally fitted by the model closer to the true value.

[0042] Furthermore, a network that can achieve the registration task is trained based on the divided training set, and then the generalization ability of the network is verified through the test set.

[0043] In summary, the present invention fully exploits the global and local information in the point cloud, effectively making up for the lack of local features in the global feature extraction stage of point cloud registration, and uses a deep neural network to fit the rigid body transformation parameters of the dual quaternion in the rigid body transformation estimation stage, which can simultaneously represent rotation and translation in a common framework, providing a compact and precise representation for posture estimation.

[0044] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flow chart for realizing the present invention;

[0046] Figure 2 This is a result diagram of registering a data set using the method of the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0049] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0050] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0051] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0052] The present invention provides a point cloud registration method based on feature extraction module and dual quaternion, designs a feature extraction network of global local fusion network (GLFNet feature extraction network) to make up for the defect of insufficient local features, and uses dual quaternion to estimate rigid transformation in the posture estimation stage. Dual quaternion can simultaneously represent rotation and translation in a common framework, and obtain compact and accurate representation for rigid transformation. In order to fully optimize the network, a new objective function is defined, and the influence of rigid transformation and corresponding points is taken into account; different coefficients are set for the objective function, so that the network converges more quickly. The method of the present invention improves the performance of point cloud registration; a large number of experiments are carried out on the data set, and the experimental results show that the method of the present invention has good competitive performance and can obtain higher registration accuracy.

[0053] See also Figure 1 The present invention provides a point cloud registration method based on a feature extraction module and a dual quaternion. The method comprises the following steps: reading 3D point cloud data from a data set; sampling the data; translating and rotating the data to obtain a source point cloud and a template point cloud and dividing the data into a training set and a test set; constructing a point cloud registration model based on a feature extraction module and a dual quaternion; training the registration model with the training set; and performing point cloud registration on the test set using the trained registration model. The specific steps are as follows:

[0054] S1. Read in 3D point cloud data:

[0055] Get 3D point cloud data from the data set. Each 3D point cloud data contains 2048 points. Each point consists of 3D coordinates (x, y, z). Get the 3D coordinates of the points contained in each sample and the category label of the sample.

[0056] S2. Sample the points contained in each sample in the data set.

[0057] For the samples contained in the data set, each sample contains 2048 points, and 1024 points need to be randomly sampled for each sample.

[0058] S3. Generate source point cloud and template point cloud

[0059] S301, setting the read data as a template point cloud.

[0060] S302 , performing a random angle rotation and a random distance translation on each template point cloud, wherein the range of the random angle is set to [-45°, 45°], and the range of the random distance is set to [-1, 1], to obtain a source point cloud.

[0061] S303 , setting the rotation angle and translation distance of each source point cloud corresponding to the template point cloud as the true value of the rigid transformation.

[0062] S304 , selecting a training set and a test set, using 70% of the source point clouds and template point clouds of different categories as the training set and 30% as the test set.

[0063] S4. Construct a point cloud registration network based on feature extraction module and dual quaternion

[0064] S401, for the source point cloud and the template point cloud, extract features of the source point cloud and the template point cloud respectively;

[0065] A global feature extraction module that integrates local features is used, and a local feature extraction unit is used to extract local features of each layer. Both the source point cloud and the template point cloud branch first pass through a GLFNet feature extraction network, with the input being N*3 point cloud coordinates and the output being N*1152 point cloud global features.

[0066] The GLFNet feature extraction network consists of five MLPs of sizes 64, 64, 64, 128, and 1024. LF units are used to extract local features from the output features of each layer, and the features output from the shared MLP are used as the input of each layer of LF units. In order to fuse local information, the output of the LF unit is concatenated with the output of the next layer of MLP and then used as the input of the next layer of MLP. In this way, the features extracted in each layer will fuse local information, and the feature size of each layer is expanded to 64, 128, 128, 192, and 1152 respectively.

[0067] The source point cloud and the template point cloud are first fed into the first layer of the GLFNet feature extraction network as input. The first layer is an MLP with a 3-dimensional input and a 64-dimensional output. The source point cloud and the template point cloud will pass through this layer and their size will change from N*3 to N*64.

[0068]

[0069]

[0070] The N*64 features of the source point cloud and the template point cloud are used as the input of the second layer MLP and the first LF unit respectively. The input of the second layer MLP is 64-dimensional, and the output is also 64-dimensional. After passing through the second layer MLP, the size of the feature is still N*64.

[0071]

[0072]

[0073] The LF unit is used to extract local information. This unit consists of three MLPs of size 64, 128, and D, where D is the dimension of the input feature. When the feature is input, it will pass through 3 MLPs to obtain a feature matrix of size N*D. Then the feature vector of size 1*D is extracted through the maximum pooling function. Finally, the feature vector is repeated N times and used in the next layer of the GLFNet feature extraction network. In this way, the input dimension and output dimension of the LF unit are both N*D.

[0074] After the N*64 feature passes through the LF unit, the feature size is still N*64. The feature output by the first LF unit is concatenated with the feature output by the second MLP to obtain a feature size of N*128.

[0075]

[0076]

[0077] The N*128-dimensional feature is used as the input of the third MLP. The input dimension of the third MLP is 128, and the output dimension is 64. The feature of size N*64 is obtained through the third MLP.

[0078]

[0079]

[0080] The N*64-dimensional features output by the second MLP are used as the input of the second LF unit, and an N*64 feature matrix is ​​obtained through the second LF unit.

[0081]

[0082]

[0083] Concatenate the two to get an N*128 feature matrix.

[0084]

[0085]

[0086] The N*128-dimensional feature is used as the input of the fourth MLP. The input dimension of the fourth MLP is 128, and the output dimension is 128. The feature of size N*128 is obtained through the fourth MLP.

[0087]

[0088]

[0089] The N*64-dimensional features output by the third MLP are used as the input of the third LF unit, and an N*64 feature matrix is ​​obtained through the third LF unit.

[0090]

[0091]

[0092] Concatenate the two to get the N*192 feature matrix.

[0093]

[0094]

[0095] The N*192-dimensional feature is used as the input of the fifth MLP. The input dimension of the fifth MLP is 192, and the output dimension is 1024. The fifth MLP obtains a feature of size N*1024.

[0096]

[0097]

[0098] The N*128-dimensional features output by the fourth MLP are used as the input of the fourth LF unit, and an N*128 feature matrix is ​​obtained through the fourth LF unit.

[0099]

[0100]

[0101] Concatenate the two to get the feature matrix of N*1152.

[0102]

[0103]

[0104] After obtaining N*1152 features for both the source point cloud and the template point cloud, we perform maximum pooling on the two features to obtain global features containing local information, both of which have a size of 1*1152.

[0105]

[0106]

[0107] S402, rigid body transformation estimation

[0108] Based on the principle of dual quaternion, the rigid body transformation is estimated. For the global features extracted in step S401, they are first concatenated to obtain features of size 1*2304;

[0109]

[0110] It is then used as input to a fully connected layer. The fully connected layer consists of 6 layers, and the size of each layer is 1024, 1024, 512, 512, 256 and 8 respectively.

[0111] F dual =FC(F f )

[0112] Through the fully connected layer, we can extract elements of size N*8, define the 8-dimensional elements as the 8 parameters of the dual quaternion (r0, r1, r2, r3, A, B, C, M), and then use these 8 parameters to predict the final rotation matrix and translation matrix.

[0113] How to find the rotation matrix:

[0114]

[0115] The method of finding the translation matrix:

[0116]

[0117] After the rotation matrix R and translation matrix T are obtained, the final rigid transformation G is expressed by these two parameters.

[0118]

[0119] Finally, the source point cloud is continuously transformed in an iterative manner to make it closer and closer to the template point cloud. The updated source point cloud will be used as the input of the GLFNet feature extraction network. After performing n iterations, the overall transformation Gest between the source point cloud and the template point cloud is obtained by combining all the poses in each iteration:

[0120] G est =G(n)×G(n-1)×…×G(1)

[0121] S5. Constructing the loss function of the point cloud registration model

[0122] The loss function is set from two parts. The first part sets the loss by the corresponding points in the source point cloud and the template point cloud, and the second part sets the loss by rigid transformation.

[0123] The first part of the loss L 1 Using the classic Chamfer Distance loss, L 1 To prevent the source point cloud and the template point cloud from losing the correspondence between point pairs during the alignment process, the distance loss between the two point sets is calculated.

[0124]

[0125] Among them, p s represents the source point cloud, p t Represents the template point cloud.

[0126] The second part of the loss L 2 The loss is measured from the perspective of global transformation, with the aim of reducing the gap between the model's predicted value Gest and the true value Gt of the rigid transformation.

[0127] L 2 =||(G est ) -1 G gt -I 4 || F

[0128] In order to consider the impact of transformation and corresponding points on the registration results during training, a new objective function L is used 3 .

[0129] L 3 =L 1 +λL 2

[0130] The optimal value of λ is 0.007 through experimental testing, and the final loss function is:

[0131] L 3 =L 1 +0.007×L 2

[0132] S6. Use the training set to train the point cloud registration model to obtain a trained registration model;

[0133] The sample pairs of the training set are used as the input of the point cloud registration model, where the source point cloud and the template point cloud are simultaneously input into the two branches of the point cloud registration model. The predicted rigid transformation of the two point clouds will be used as the output of the registration model. The network parameters of the point cloud registration model are optimized by solving the error between the predicted rigid transformation and the true rigid transformation and the error between the corresponding points and back-propagating the error to obtain a trained point cloud registration model.

[0134] S7. Use the trained point cloud registration model to register the test set and obtain the registration results of each group of samples in the test set.

[0135] The samples of the test set are used as the input of the trained registration model, and the output of the trained registration model is the rigid transformation estimate of each pair of source point cloud and template point cloud in the test set.

[0136] In yet another embodiment of the present invention, a point cloud registration system based on a feature extraction module and a dual quaternion is provided. The system can be used to implement the above-mentioned point cloud registration method based on a feature extraction module and a dual quaternion. Specifically, the point cloud registration system based on a feature extraction module and a dual quaternion includes a reading module, a sampling module, a division module, a construction module and a registration module.

[0137] Among them, the reading module reads 3D point cloud data from the data set to obtain the three-dimensional coordinates and category labels of the points contained in each sample;

[0138] The sampling module randomly samples the points contained in each sample obtained by the reading module to obtain a template point cloud;

[0139] The division module translates and rotates the template point cloud obtained by the sampling module to obtain the source point cloud, and divides the source point cloud and the template point cloud into a training set and a test set according to the category label of the point cloud;

[0140] A construction module is used to construct a point cloud registration model based on a feature extraction module and a dual quaternion, and a loss function of the point cloud registration model is constructed;

[0141] The registration module uses the training set divided by the division module to train the point cloud registration model constructed by the construction module, and uses the trained point cloud registration model to perform point cloud registration on the test set divided by the division module.

[0142] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0143] The effect of the present invention can be further illustrated by the following simulation experiment:

[0144] 1. Simulation conditions

[0145] The hardware platform is: HP-Z840 workstation, TITAN-X-12GB-GPU, 64GB RAM.

[0146] The software platform is: Python, Pytorch.

[0147] 2. Simulation content and results

[0148] See also Figure 2 The dataset for the simulation experiment of the present invention is the ModelNet40 point cloud dataset. It includes various point clouds, a total of 9840 models, divided into 40 different categories, such as airplanes, cars, tables, etc. Since CAD vertices are non-uniformly distributed, a version with 10,000 uniformly sampled surface points and normals for each model is provided.

[0149] The simulation experiment of the present invention is to register the rotated and translated point cloud with the original point cloud, and compare them with the true value to compare the performance of different methods.

[0150] Table 1 shows six methods that use the traditional ICP method to start from an initial transformation and iteratively solve two trivial sub-problems alternately: find the closest point as the corresponding point under the current transformation, and calculate the optimal transformation from the corresponding point cloud through singular value decomposition), the present invention 1 (does not use LF units in the feature extraction stage and does not fuse local information), the present invention 2 (based on the feature extraction module to fuse LF units, but does not use dual quaternions to calculate rigid transformation), the present invention 3 (only uses L1 loss), the present invention 4 (only uses L2 loss), the present invention 5 (uses both feature extraction modules and dual quaternions to calculate rigid transformation, and uses L3 loss). The mean square error (RMSE) of the six methods for rotation and translation is also counted.

[0151] Table 1 Performance comparison between the present invention (using different strategies and structures) and the traditional method

[0152] method RMSE(R) RMSE(T) ICP 45.8016 0.2873 The present invention 1 2.6360 0.0029 The present invention 2 2.7815 0.0509 Present invention 3 2.6415 0.0015 The present invention 4 3.6045 0.0693 The present invention 5 2.5257 0.0014

[0153] It can be seen from Table 1 that the method of the present invention has greatly improved the registration accuracy compared with the traditional method. Using LF unit and dual quaternion for registration will further improve the accuracy, and combining the two loss functions will significantly improve the accuracy compared with a single loss function.

[0154] In summary, the point cloud registration method and system based on feature extraction module and dual quaternion of the present invention have the following characteristics:

[0155] In the pose estimation stage corresponding to the free point cloud registration, the dual quaternion is used to estimate the rigid transformation. The dual quaternion can represent both rotation and translation and obtain a compact and accurate rigid body transformation representation using only 8 parameters.

[0156] A new feature extraction network GLFNet is designed, which includes a local feature extraction unit LF unit. This network can make up for the shortcomings of local features and make full use of point cloud information to extract global features of point clouds containing local information.

[0157] A new objective function is defined to comprehensively optimize the network, which takes into account the influence of rigid transformation and each point of the point cloud on the registration result. A large number of experiments have shown that the performance is better than other methods and is robust to noise.

[0158] The present invention can be used in autonomous driving, robotics, large-scale three-dimensional reconstruction of real-time monitoring of underground mining, providing high-precision positioning services for unmanned driving, attitude estimation and other related fields.

[0159] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0160] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0161] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0162] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0163] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A point cloud registration method based on feature extraction module and dual quaternion, characterized in that: Read 3D point cloud data from the data set to obtain the 3D coordinates and category labels of the points contained in each sample; randomly sample the points contained in each sample to obtain a template point cloud; translate and rotate the template point cloud to obtain a source point cloud, and divide the source point cloud and the template point cloud into a training set and a test set according to the category labels of the samples; construct a point cloud registration model based on a feature extraction module and dual quaternions, and construct a loss function for the point cloud registration model; train the point cloud registration model with the divided training set, and use the trained point cloud registration model to perform point cloud registration on the divided test set; The point cloud registration model based on feature extraction module and dual quaternion is as follows: The source point cloud and template point cloud are sent as input to the GLFNet feature extraction network, and the N*1152 feature matrix is ​​obtained after splicing. The feature matrices obtained from the source point cloud and template point cloud are respectively subjected to maximum pooling to obtain global features containing local information; The extracted global features are concatenated to obtain features of size 1*2304; then the fully connected layer is input to extract elements of size N*8, and the 8-dimensional elements are defined as the 8 parameters of the dual quaternion. The final rotation matrix and translation matrix are predicted through the 8 parameters to obtain the rigid transformation. Finally, the source point cloud is transformed iteratively and the updated source point cloud is used as the input of the GLFNet feature extraction network; after performing n iterations, the overall transformation between the source point cloud and the template point cloud is obtained by combining all the postures in each iteration. ; The GLFNet feature extraction network includes five MLPs of sizes 64, 64, 64, 128, and 1024 respectively. The LF unit is used to extract local features from the output features of each layer, and the features output from the shared MLP are used as the input of each layer of LF units; the output of the LF unit is concatenated with the output of the next layer of MLP and then used as the input of the next layer of MLP. The features extracted in each layer fuse local information, and the feature size of each layer is expanded to 64, 128, 128, 192, and 1152 respectively. The source point cloud and template point cloud contain global features of local information and They are: in, is the maximum pooling operation, To expand to 1152-dimensional source point cloud features, The template point cloud features are expanded to 1152 dimensions.

2. The point cloud registration method based on feature extraction module and dual quaternion according to claim 1, characterized in that: Each 3D point cloud data contains 2048 points, each point consists of 3D coordinates (x, y, z), and the 3D coordinates of the points contained in each sample and the category label of the sample are obtained.

3. The point cloud registration method based on feature extraction module and dual quaternion according to claim 1, characterized in that: The points contained in each sample are randomly sampled to obtain 1024 points.

4. The point cloud registration method based on feature extraction module and dual quaternion according to claim 1, characterized in that: The template point cloud is translated and rotated to obtain the source point cloud, and the source point cloud is divided into a training set and a test set as follows: The data read from the original data set is set as the template point cloud; each template point cloud is rotated at a random angle and translated at a random distance, with the random angle being [-45°, 45°] and the random distance being [-1, 1], to obtain the source point cloud; the rotation angle and translation distance of each source point cloud corresponding to the template point cloud are set as the true value of the rigid transformation; 70% of the samples from different categories of source point clouds and template point clouds are selected as the training set, and the remaining 30% of the samples are used as the test set.

5. The point cloud registration method based on feature extraction module and dual quaternion according to claim 1, characterized in that: The overall transformation between the source point cloud and the template point cloud for: in, is the rotation matrix obtained in the nth iteration, is the rotation matrix obtained at the n-1th iteration.

6. The point cloud registration method based on feature extraction module and dual quaternion according to claim 1, characterized in that: Loss function of point cloud registration model Specifically: in, is the chamfer loss, The loss is measured as the angle of the global transformation.

7. The point cloud registration method based on feature extraction module and dual quaternion according to claim 1, characterized in that: The specific training of the point cloud registration model using the divided training set is as follows: The sample pairs of the training set are used as the input of the point cloud registration model. The source point cloud and the template point cloud are simultaneously input into the two branches of the point cloud registration model. The predicted rigid transformation of the two point clouds is used as the output of the point cloud registration model. The error between the predicted rigid transformation and the true rigid transformation and the error between the corresponding points are solved and the error is back-propagated to optimize the network parameters of the point cloud registration model and obtain a trained point cloud registration model.

8. A point cloud registration system based on feature extraction module and dual quaternion, characterized in that: include: The reading module reads 3D point cloud data from the data set and obtains the three-dimensional coordinates and category labels of the points contained in each sample; The sampling module randomly samples the points contained in each sample obtained by the reading module to obtain a template point cloud; The division module translates and rotates the template point cloud obtained by the sampling module to obtain the source point cloud, and divides the source point cloud and the template point cloud into a training set and a test set according to the category label of the point cloud; Construction module, constructing a point cloud registration model based on feature extraction module and dual quaternion, and constructing the loss function of the point cloud registration model; the point cloud registration model based on feature extraction module and dual quaternion is specifically: The source point cloud and template point cloud are sent as input to the GLFNet feature extraction network, and the N*1152 feature matrix is ​​obtained after splicing. The feature matrices obtained from the source point cloud and template point cloud are respectively subjected to maximum pooling to obtain global features containing local information; The extracted global features are concatenated to obtain features of size 1*2304; then the fully connected layer is input to extract elements of size N*8, and the 8-dimensional elements are defined as the 8 parameters of the dual quaternion. The final rotation matrix and translation matrix are predicted through the 8 parameters to obtain the rigid transformation. Finally, the source point cloud is transformed iteratively and the updated source point cloud is used as the input of the GLFNet feature extraction network; after performing n iterations, the overall transformation between the source point cloud and the template point cloud is obtained by combining all the postures in each iteration. ; The GLFNet feature extraction network includes five MLPs of sizes 64, 64, 64, 128, and 1024 respectively. The LF unit is used to extract local features from the output features of each layer, and the features output from the shared MLP are used as the input of each layer of LF units; the output of the LF unit is concatenated with the output of the next layer of MLP and then used as the input of the next layer of MLP. The features extracted in each layer fuse local information, and the feature size of each layer is expanded to 64, 128, 128, 192, and 1152 respectively. The source point cloud and template point cloud contain global features of local information and They are: in, is the maximum pooling operation, To expand to 1152-dimensional source point cloud features, To expand the template point cloud features to 1152 dimensions; The registration module uses the training set divided by the division module to train the point cloud registration model constructed by the construction module, and uses the trained point cloud registration model to perform point cloud registration on the test set divided by the division module.