A point cloud registration method and system based on local feature learning
Through the point cloud registration method based on local feature learning, the local feature extraction network and maximum pooling operation are used to generate global features, and an end-to-end point cloud registration network is built, which solves the initial position and noise-sensitive in the existing technology The problem is that efficient and robust point cloud registration effect is achieved.
Patent Information
- Application Number
- CN202210301546.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-25
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-03-25
AI Technical Summary
The existing point cloud registration methods are sensitive to initial locations, easily fall into local optimality, and are highly complex, making it difficult to process noise data.
Using the point cloud registration method based on local feature learning, the local feature extraction network and maximum pooling operation generates global features, and the end-to-end point cloud registration network is built, and the CDLoss function is used for training to avoid calculating the correspondence between point pairs. .
It improves the accuracy and speed of point cloud registration, enhances the robustness to noise, avoids local optimal traps, and simplifies the registration network structure.
Smart Images

Figure CN114638866B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer vision and three-dimensional image processing, and in particular relates to a point cloud registration method and system based on local feature learning. Background Art
[0002] With the rapid development of high-precision sensors such as LiDAR and Kinect cameras, point cloud has become the main data format for describing three-dimensional images. Since sensors can only scan and acquire data within their limited field of view, a registration algorithm is needed to generate a complete three-dimensional scene. Point cloud registration is a process of estimating the transformation matrix between two frames of scanned point clouds. According to the transformation matrix, partial scanned point clouds of the same three-dimensional scene or object can be merged into a complete three-dimensional point cloud. Point cloud registration technology is widely used in many computer vision applications such as three-dimensional reconstruction, three-dimensional positioning, and posture estimation, and plays a unique and critical role.
[0003] The existing point cloud registration methods are mainly divided into two categories: one is the traditional registration method, and the other is the registration method based on deep learning. Among the traditional algorithms, the iterative closest point algorithm and its variants have been widely used due to their simplicity and efficiency. However, this method is very sensitive to the initial position of the point cloud and is prone to fall into local optimality. With the great success of deep learning in two-dimensional image processing, three-dimensional point cloud processing based on deep learning has also received great attention. However, most of the point cloud registration methods based on deep learning adopt a two-step approach, that is, first find the correspondence between point pairs, and then use the correspondence to solve the transformation matrix, so these methods are relatively complex. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide a point cloud registration method and system based on local feature learning in response to the above-mentioned deficiencies in the prior art, so as to solve the problems that the prior art is sensitive to the initial position of the point cloud, is sensitive to noise, and is prone to falling into local optimality, and can perform end-to-end registration without calculating the correspondence between point pairs.
[0005] The present invention adopts the following technical solutions:
[0006] A point cloud registration method based on local feature learning, including
[0007] S1, read point cloud data and pre-process the point cloud data;
[0008] S2, applying transformation to the point cloud data preprocessed in step S1 to create a training set and a test set;
[0009] S3, construct a local feature extraction network, obtain the local features of the point cloud to be registered through the local feature extraction network, and obtain the global features of the point cloud to be registered through the maximum pooling operation;
[0010] S4, constructing a point cloud registration network based on the local features of the point cloud to be registered obtained in step S3 and the global features of the point cloud to be registered;
[0011] S5, using the training set obtained in step S2 to train the point cloud registration network constructed in step S4;
[0012] S6. Use the point cloud registration network trained in step S5 to register the test set selected in step S2, obtain transformation parameters and evaluate the registration results.
[0013] Specifically, in step S1, the point cloud data is preprocessed as follows:
[0014] Point cloud data is read from the ModelNet40 dataset. Each point in the point cloud data includes three coordinate values: x, y, and z. 1024 points are collected from each point cloud through farthest point sampling. In each point cloud data, the coordinate value of each point is subtracted from the mean value of all coordinates in the point cloud.
[0015] Specifically, step S2 is as follows:
[0016] A random transformation is applied to the point cloud data preprocessed in step S1. The random transformation includes rotation transformation and translation transformation. The range of rotation transformation is -45 to +45°, and the range of translation transformation is -1 to +1. The point cloud before transformation is used as the template point cloud, and the point cloud after transformation is used as the source point cloud. The 5112 point cloud data contained in the first 20 object categories in the ModelNet40 dataset and the corresponding source point clouds are used as training sets, and the 1266 point cloud data contained in the last 20 object categories in the ModelNet40 dataset and the corresponding source point clouds are used as test sets.
[0017] Specifically, step S3 is as follows:
[0018] The point cloud is passed through the first layer of multi-layer perceptron and the second layer of multi-layer perceptron to map the 3D coordinate features to a 64-dimensional high-dimensional space; the source point cloud is sampled using the farthest point sampling to obtain 512 points (512×64);
[0019] Taking each point as the center and the distance as the scale, find the 32 neighboring points closest to each point among the original 1024 points. Subtract the center point feature from the feature of each neighbor point to get a feature vector pointing from the neighbor point to the center point.
[0020] Each feature vector pointing from the neighbor point to the center point is concatenated with the feature of the center point to obtain a 128-dimensional combined feature vector. Each dimension of the 128-dimensional combined features of the 32 points in each neighborhood is pooled to obtain a 128-dimensional vector representing the neighborhood features. The first 64 dimensions of the 128-dimensional vector represent the maximum eigenvalue of the neighboring points in each neighborhood in a certain dimension, and the last 64 dimensions of the 128-dimensional vector represent the features of the center point.
[0021] The features of 512 sampling points are changed from 64 dimensions to 128 dimensions. The 512 points are sampled again using the farthest point sampling to obtain 256 points. The 256 neighborhoods are divided again, and a 256-dimensional combined feature vector representing the features of each neighborhood is obtained. The first 128 dimensions represent the maximum eigenvalue of the neighborhood points in each neighborhood in a certain dimension, and the last 128 dimensions represent the features of the center point.
[0022] The 1024-dimensional features are obtained through the third and fourth multi-layer perceptrons, and the 1024-dimensional features are concatenated with the 256-dimensional features obtained after the second sampling and neighborhood division to obtain 1280-dimensional features; and then the 1024-dimensional feature vector is obtained through the fifth multi-layer perceptron;
[0023] Finally, a maximum pooling is performed to obtain a 1024-dimensional global feature.
[0024] Furthermore, the input dimension of the first layer of multilayer perceptron is 3, and the output dimension is 64; the input dimension of the second layer of multilayer perceptron is 64, and the input dimension is 64; the input dimension of the third layer of multilayer perceptron is 256, and the output dimension is 512; the input dimension of the fourth layer of multilayer perceptron is 512, and the input dimension is 1024; the input dimension of the fifth layer of multilayer perceptron is 1280, and the output dimension is 1024.
[0025] Specifically, step S4 is as follows:
[0026] The source point cloud and the template point cloud are input into the point cloud registration network. After passing through the local feature extraction network, two 1×1024-dimensional global features are obtained. The two global features are concatenated to obtain a 1×2048-dimensional feature vector. Then, a series of fully connected layers are passed to obtain a 1×7-dimensional vector. The rotation quaternion and translation vector are obtained based on the 1×7-dimensional vector.
[0027] Furthermore, in the 7-dimensional vector, the first 4 dimensions represent the rotation quaternion, and the last 3 dimensions represent the translation vector. The parameters of the fully connected layer are: fully connected layer 1: input 2048 dimensions, output 1024 dimensions; fully connected layer 2: input 1024 dimensions, output 1024 dimensions; fully connected layer 3: input 1024 dimensions, output 512 dimensions; fully connected layer 4: input 512 dimensions, output 512 dimensions; fully connected layer 5: input 512 dimensions, output 256 dimensions; fully connected layer 6: input 256 dimensions, output 7 dimensions.
[0028] Specifically, in step S5, the training set is input into the point cloud registration network, the source point cloud and the template point cloud are feature extracted and spliced together to obtain transformation parameters through a fully connected layer, and a transformation is applied to the source point cloud according to the transformation parameters to obtain the CDloss values of the transformed point cloud and the template point cloud. The CDLoss value is continuously reduced through the back propagation algorithm to optimize the network parameters of the point cloud registration network to obtain a trained point cloud registration network.
[0029] Furthermore, CDLoss is used as the loss function for point cloud registration network training.
[0030] Another technical solution of the present invention is a point cloud registration system based on local feature learning, comprising:
[0031] A preprocessing module reads point cloud data and preprocesses the point cloud data;
[0032] Select the module, apply transformation to the preprocessed point cloud data, and create training and test sets;
[0033] Feature module, constructs a local feature extraction network, obtains the local features of the point cloud to be registered through the local feature extraction network, and obtains the global features of the point cloud to be registered through the maximum pooling operation;
[0034] A construction module, which constructs a point cloud registration network based on the local features of the point cloud to be registered obtained by the feature module and the global features of the point cloud to be registered;
[0035] The training module uses the training set obtained by the selection module to train the point cloud registration network constructed by the construction module;
[0036] The registration module uses the point cloud registration network trained by the training module to register the test set selected by the selection module, obtain the transformation parameters and evaluate the registration results.
[0037] Compared with the prior art, the present invention has at least the following beneficial effects:
[0038] The present invention discloses a point cloud registration method based on local feature learning, which makes full use of the local features of point clouds, has good robustness to noise data and is not easy to fall into the global optimum. Experiments have proved that the accuracy and speed of point cloud registration can be effectively improved. By learning local features for registration, compared with Pcrnet using Pointnet to extract global features, the local features extracted by the present invention have better robustness to noise, are independent of the initial state, and are not easy to fall into the local optimum. An end-to-end point cloud registration network is adopted, and compared with the traditional two-stage structure, that is, first finding the correspondence between points and then calculating the transformation matrix, the network is more concise and clear.
[0039] Furthermore, in step S1, the farthest point sampling is performed on the input point cloud data to remove redundant data, and 1024 points are sampled, which can reduce the computational overhead and save computing power while ensuring the registration accuracy. All point cloud data are preprocessed, and the statistical average value of the point cloud coordinates is subtracted from each point in each point cloud, which is conducive to highlighting individual differences and improving the registration results.
[0040] Furthermore, in step S2, a training set and a test set are prepared by applying a rotation transformation and a translation transformation to the original point cloud, which are used for training and testing the point cloud registration network.
[0041] Furthermore, in step S3, a local feature extraction network is constructed. First, the point cloud is input into a two-layer multi-layer perceptron to increase the feature dimension to 64 dimensions and enrich the feature information of the point cloud. The point cloud is sampled to reduce the number of points. A neighborhood containing 32 neighborhood points is divided around the sampled point using distance as the scale, and each feature vector pointing from the neighborhood point to the center point is calculated. The feature vector is concatenated with the feature value of the center point to obtain a combined feature vector. All combined feature vectors in a neighborhood are subjected to maximum pooling operations in different dimensions, thereby obtaining a feature vector that can represent a neighborhood. Each dimensional feature value of the feature vector represents the maximum value of all combined feature vectors in the neighborhood in the dimensional vector, which fully reflects the local features of the neighborhood. Similarly, the above-mentioned sampling, neighborhood division, and maximum pooling operations are repeated again to further reduce the number of points in the point cloud and increase the dimension of the local features of the neighborhood. At this point, there are 256 points left in each point cloud, and the feature dimension of each point is 256. The feature matrix (256×256) is input into two layers of multi-layer perceptrons to increase the feature dimension, and the feature dimension of each point becomes 1024 (256×1024). The 1024-dimensional feature is concatenated with the 256-dimensional feature obtained after the second sampling and neighborhood division to obtain a 1280-dimensional feature (256×1280). This operation combines the point cloud features at two different sampling levels and provides deeper local feature information. Then, a 1024-dimensional feature vector (256×1024) is obtained through the fifth layer of multi-layer perceptron. Finally, a maximum pooling operation is performed to obtain a 1024-dimensional (1×1024) global feature, which can represent the global information of a point cloud and can be used as feature information for point cloud registration.
[0042] Further, in step S3, the input dimension of the first layer of multilayer perceptron is 3, and the output dimension is 64; the input dimension of the second layer of multilayer perceptron is 64, and the output dimension is 64; the input dimension of the third layer of multilayer perceptron is 256, and the output dimension is 512; the input dimension of the fourth layer of multilayer perceptron is 512, and the output dimension is 1024; the input dimension of the fifth layer of multilayer perceptron is 1280, and the output dimension is 1024. The first two layers of multilayer perceptrons increase the features of the point cloud from 3-dimensional coordinate features to 64-dimensional features, improve the feature dimension, and enrich the feature information of the point cloud; after two local feature extractions, the point cloud features are increased from 256 dimensions to 1024 dimensions by passing through two layers of multilayer perceptrons again, and while increasing the feature dimension, the interaction between the feature information of the points in the point cloud is realized; the last layer of perceptron reduces the point cloud features from 1280 dimensions to 1024 dimensions, realizes the interaction between the local features of the point cloud at different sampling levels, and enhances the robustness of the features.
[0043] Furthermore, in step S4, the local feature extraction network is used as a part of the point cloud registration network. The source point cloud and the template point cloud obtain two 1×1024-dimensional global features after passing through the local feature extraction network respectively. These two global features are concatenated into a 2048-dimensional feature vector and input into a series of fully connected layers to obtain a 1×7-dimensional vector, and the rotation parameters and translation parameters are separated from the 7-dimensional vector.
[0044] Furthermore, in this 7-dimensional vector, the values of the first four dimensions represent the rotation quaternion, and the last three dimensions represent the translation vector. There are five fully connected layers, and the parameters of the fully connected layers are: fully connected layer 1: input 2048 dimensions, output 1024 dimensions; fully connected layer 2: input 1024 dimensions, output 1024 dimensions; fully connected layer 3: input 1024 dimensions, output 512 dimensions; fully connected layer 4: input 512 dimensions, output 512 dimensions; fully connected layer 5: input 512 dimensions, output 256 dimensions; fully connected layer 6: input 256 dimensions, output 7 dimensions. After experiments, if the number of fully connected layers is too small, the performance of the point cloud registration network is poor, and 5 layers is a suitable number of layers. Quaternions can perform arbitrary rotations of vectors around the origin, which is convenient and fast. In some implementations, it is more efficient than rotation matrices, so quaternions are used to represent transformation parameters.
[0045] Furthermore, in step S5, the constructed point cloud registration network is trained using the training set prepared in step S2. CDLoss is used as a loss function to measure the distance between two point clouds. The smaller the CDLoss value, the closer the two point clouds are and the better the registration effect.
[0046] Furthermore, the training data is input into the point cloud registration network to obtain the estimated transformation parameters, and the source point cloud is transformed according to the transformation parameters to obtain the CDloss values of the transformed point cloud and the template point cloud. The CDLoss value is continuously reduced through the back propagation algorithm. When the value is reduced to a sufficiently small value, it can be considered that the source point cloud and the template point cloud have been registered.
[0047] In summary, the present invention adopts a unique local feature extraction structure, and the extracted features have significantly improved the registration accuracy, achieving very good registration results.
[0048] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] Figure 1 Schematic diagram of the point cloud registration network;
[0050] Figure 2 Schematic diagram of local feature extraction;
[0051] Figure 3The following are the registration effect diagrams, where (a) is the registration effect diagram of the bottle, and (b) is the registration effect diagram of the cap. DETAILED DESCRIPTION
[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0053] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.
[0054] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0055] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.
[0056] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.
[0057] The present invention provides a point cloud registration method based on local feature learning, which reads point cloud data from a data set and performs preprocessing; constructs local features using the three-dimensional coordinates of the points; generates global features through maximum pooling operations; constructs a registration network including a feature extraction module; selects a training set and a test set; trains the registration network using the training set; registers the test set using the trained registration network, obtains transformation parameters, and evaluates the registration results. The present invention focuses on extracting the local neighborhood features of the points, constructing a feature vector pointing from the neighborhood points to the center point. The feature is robust to noise and has a good registration effect; in addition, the present invention constructs an end-to-end point cloud registration network, which has a simpler structure than the traditional two-stage method, and only needs to input the source point cloud and the template point cloud to obtain the registration result.
[0058] See also Figure 1 The present invention provides a point cloud registration method based on local feature learning, comprising the following steps:
[0059] S1, read point cloud data from the data set and preprocess it;
[0060] Point cloud data is read from the ModelNet40 dataset. Each point in the point cloud data includes three coordinate values: x, y, and z. 1024 points are collected from each point cloud through FPS (furthest point sampling). In each point cloud data, the coordinate value of each point is subtracted from the mean value of the point cloud coordinate.
[0061] S2. Create training and test sets
[0062] Apply random transformation to the point cloud data preprocessed in step S1 (random transformation includes rotation transformation and translation transformation, the range of rotation transformation is -45 to +45°, and the range of translation transformation is -1 to +1). The point cloud before transformation is called template point cloud, and the point cloud after transformation is called source point cloud. The ModelNet40 dataset contains 40 object categories, the first 20 object categories contain 5112 point cloud data, and these 5112 template point clouds and their corresponding source point clouds are used as training sets; the last 20 object categories contain 1266 point cloud data, and these 1266 template point clouds and their corresponding source point clouds are used as test sets.
[0063] S3, learning local features;
[0064] S301, pass the source point cloud (the dimension of each point cloud is 1024×3) through two layers of multilayer perceptrons, the input dimension of the first layer of multilayer perceptron is 3, and the output dimension is 64; the input dimension of the second layer is 64, and the output dimension is also 64; so far, the 3D coordinate features are mapped to a 64-dimensional high-dimensional space;
[0065] S302, sampling, grouping, and aggregation;
[0066] See also Figure 2 , use FPS (farthest point sampling) to sample the source point cloud and get 512 points (512×64);
[0067] Taking each point as the center and the distance as the scale, find the 32 neighboring points (512×32×64) closest to each point in the original 1024 points. Subtract the features of each neighboring point from the features of its center point to obtain a feature vector (512×32×64) pointing from the neighborhood point to the center point.
[0068] Each eigenvector is concatenated with the eigenvalue of its center point to obtain a combined eigenvector (512×32×128). All combined eigenvectors in a neighborhood are subjected to maximum pooling operations in different dimensions to obtain a eigenvector (512×128) that can represent a neighborhood.
[0069] S303: Obtain global features.
[0070] Similar to the step of S302, the 512 points are sampled, grouped, and aggregated again to obtain 256 sampled points and a 256-dimensional feature vector (256×256) for each point;
[0071] Through two layers of multilayer perceptrons, the input dimension of the first layer of multilayer perceptron is 256 and the output dimension is 512; the input dimension of the second layer is 512 and the output dimension is 1024 (256×1024);
[0072] The obtained 1024-dimensional features are concatenated with the 256-dimensional features obtained after the second step S302 operation to obtain 1280-dimensional features; after a layer of multi-layer perceptron, the input dimension is 1280, the output dimension is 1024, and a 1024-dimensional feature vector (256×1024) is obtained;
[0073] Finally, a maximum pooling is performed to obtain a 1024-dimensional global feature (1×1024).
[0074] S4, constructing a point cloud registration network based on local features, concatenating the global features of the source point cloud and the template point cloud obtained in step S303 (1×2048), and then passing through a fully connected layer to obtain a 7-dimensional vector, where the first 4 dimensions represent the rotation quaternion and the last 3 dimensions represent the translation vector, thereby obtaining the estimated rotation parameters and translation parameters;
[0075] The parameters of the full connection are as follows:
[0076] Fully connected layer 1: input 2048 dimensions, output 1024 dimensions;
[0077] Fully connected layer 2: input 1024 dimensions, output 1024 dimensions;
[0078] Fully connected layer 3: input 1024 dimensions, output 512 dimensions;
[0079] Fully connected layer 4: input 512 dimensions, output 512 dimensions;
[0080] Fully connected layer 5: input 512 dimensions, output 256 dimensions;
[0081] Fully connected layer 6: input 256 dimensions, output 7 dimensions.
[0082] S5, training the constructed point cloud registration network using the training set;
[0083] S501, the training batch size is 16, the initial learning rate is 0.01, the number of training epochs is 200, and CDLoss is used as the loss function in training;
[0084] S502, CDLoss algorithm is as follows:
[0085] Given two point sets S1 and S2, the Chamfer Distance between them is defined as
[0086]
[0087] Where S1 and S2 represent two point sets, x is a point in S1, and y is a point in S2
[0088] S503, input the training set into the point cloud registration network, extract the source point cloud and the template point cloud to obtain the global features, splice the two global features together and pass through the fully connected layer to obtain the transformation parameters, apply the transformation to the source point cloud according to the transformation parameters, obtain the CDloss value of the transformed point cloud and the template point cloud, continuously reduce the CDLoss value through the back propagation algorithm, optimize the network parameters of the point cloud registration network, and obtain the trained point cloud registration network.
[0089] S6, using the point cloud registration network trained in step S5 to register the test set;
[0090] The sample blocks of the test set are used as the input of the trained point cloud registration network, and the output of the trained point cloud registration network is the transformation parameters obtained for the point cloud in the test set.
[0091] In another embodiment of the present invention, a point cloud registration system based on local feature learning is provided, which can be used to implement the above-mentioned point cloud registration method based on local feature learning. Specifically, the point cloud registration system based on local feature learning includes a preprocessing module, a selection module, a feature module, a construction module, a training module and a registration module.
[0092] Among them, the preprocessing module reads the point cloud data and preprocesses the point cloud data;
[0093] Select the module, apply transformation to the preprocessed point cloud data, and create training and test sets;
[0094] Feature module, constructs a local feature extraction network, obtains the local features of the point cloud to be registered through the local feature extraction network, and obtains the global features of the point cloud to be registered through the maximum pooling operation;
[0095] A construction module is used to construct a point cloud registration network based on the local feature extraction network of the feature module. The point cloud registration network includes the local feature extraction network.
[0096] The training module uses the training set obtained by the selection module to train the point cloud registration network constructed by the construction module;
[0097] The registration module uses the point cloud registration network trained by the training module to register the test set selected by the selection module, obtain the transformation parameters and evaluate the registration results.
[0098] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0099] Simulation conditions
[0100] The hardware platform is: HP-Z840 workstation, TITAN-X-12GB-GPU, 64GB RAM.
[0101] The software platform is: Python, Pytorch.
[0102] Simulation content and results
[0103] The simulation experiment of the present invention uses the ModelNet40 dataset, and the registration effect diagram is as follows: Figure 3 As shown, for the bottle point cloud in (a), the registered point cloud and the template point cloud are almost overlapped, with only a few points at the bottle mouth and the bottle bottom not being registered. The registered point cloud and the template point cloud in (b) are completely overlapped, which shows that the point cloud registration method proposed in the present invention has a very high registration accuracy.
[0104] Compared with the existing methods, the local features extracted by the present invention are more robust to noise, do not depend on the initial state, and are not prone to fall into the local optimum. On the ModelNet40 dataset, the present invention obtains a mean absolute rotation error of 10.4545 and a mean absolute translation error of 0.0035.
[0105] To sum up, the present invention provides a point cloud registration method and system based on local feature learning. Please summarize and explain the effects of the present invention.
[0106] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0107] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0108] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0109] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.
[0110] The above contents are only for explaining the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.
Claims
1. A point cloud registration method based on local feature learning, characterized in that: The following steps are involved: S1, read point cloud data and pre-process the point cloud data; S2, applying transformation to the point cloud data preprocessed in step S1 to create a training set and a test set; S3. Construct a local feature extraction network, obtain the local features of the point cloud to be registered through the local feature extraction network, and obtain the global features of the point cloud to be registered through the maximum pooling operation, as follows: The point cloud is passed through the first layer of multi-layer perceptron and the second layer of multi-layer perceptron to map the 3D coordinate features to a 64-dimensional high-dimensional space; the source point cloud is sampled using the farthest point sampling to obtain 512 points; Taking each point as the center and the distance as the scale, find the 32 neighboring points closest to each point among the original 1024 points. Subtract the center point feature from the feature of each neighbor point to get a feature vector pointing from the neighbor point to the center point. Each feature vector pointing from the neighbor point to the center point is concatenated with the feature of the center point to obtain a 128-dimensional combined feature vector. Each dimension of the 128-dimensional combined features of the 32 points in each neighborhood is pooled to obtain a 128-dimensional vector representing the neighborhood features. The first 64 dimensions of the 128-dimensional vector represent the maximum eigenvalue of the neighboring points in each neighborhood in a certain dimension, and the last 64 dimensions of the 128-dimensional vector represent the features of the center point. The features of 512 sampling points are converted from 64 dimensions to 128 dimensions. The 512 points are sampled again using the farthest point sampling to obtain 256 points. The 256 neighborhoods are divided again, and a 256-dimensional combined feature vector representing the features of each neighborhood is obtained. The first 128 dimensions represent the maximum eigenvalue of the neighborhood points in each neighborhood in a certain dimension, and the last 128 dimensions represent the features of the center point. The 1024-dimensional features are obtained through the third and fourth multi-layer perceptrons, and the 1024-dimensional features are concatenated with the 256-dimensional features obtained after the second sampling and neighborhood division to obtain 1280-dimensional features; and then the 1024-dimensional feature vector is obtained through the fifth multi-layer perceptron; Finally, a maximum pooling is performed to obtain a 1024-dimensional global feature; S4: constructing a point cloud registration network based on the local features of the point cloud to be registered obtained in step S3 and the global features of the point cloud to be registered, specifically: The source point cloud and the template point cloud are input into the point cloud registration network. After passing through the local feature extraction network, two 1×1024-dimensional global features are obtained. The two global features are concatenated to obtain a 1×2048-dimensional feature vector. Then, a series of fully connected layers are used to obtain a 1×7-dimensional vector. The rotation quaternion and translation vector are obtained based on the 1×7-dimensional vector. S5, using the training set obtained in step S2 to train the point cloud registration network constructed in step S4; S6. Use the point cloud registration network trained in step S5 to register the test set selected in step S2, obtain transformation parameters and evaluate the registration results.
2. The point cloud registration method based on local feature learning according to claim 1, characterized in that: In step S1, the point cloud data is preprocessed as follows: Point cloud data is read from the ModelNet40 dataset. Each point in the point cloud data includes three coordinate values: x, y, and z. 1024 points are collected from each point cloud through farthest point sampling. In each point cloud data, the coordinate value of each point is subtracted from the mean value of all coordinates in the point cloud.
3. The point cloud registration method based on local feature learning according to claim 1, characterized in that: Step S2 is specifically as follows: A random transformation is applied to the point cloud data preprocessed in step S1. The random transformation includes rotation transformation and translation transformation. The range of rotation transformation is -45~+45°, and the range of translation transformation is -1~+1. The point cloud before transformation is used as the template point cloud, and the point cloud after transformation is used as the source point cloud. The 5112 point cloud data contained in the first 20 object categories in the ModelNet40 dataset and the corresponding source point clouds are used as training sets, and the 1266 point cloud data contained in the last 20 object categories in the ModelNet40 dataset and the corresponding source point clouds are used as test sets.
4. The point cloud registration method based on local feature learning according to claim 1, characterized in that: In step S3, the input dimension of the first layer of multilayer perceptron is 3, and the output dimension is 64; the input dimension of the second layer of multilayer perceptron is 64, and the input dimension is 64; the input dimension of the third layer of multilayer perceptron is 256, and the output dimension is 512; the input dimension of the fourth layer of multilayer perceptron is 512, and the input dimension is 1024; the input dimension of the fifth layer of multilayer perceptron is 1280, and the output dimension is 1024.
5. The point cloud registration method based on local feature learning according to claim 1, characterized in that: In step S4, in the 7-dimensional vector, the first 4 dimensions represent the rotation quaternion, and the last 3 dimensions represent the translation vector. The parameters of the fully connected layer are: fully connected layer 1: input 2048 dimensions, output 1024 dimensions; fully connected layer 2: input 1024 dimensions, output 1024 dimensions; fully connected layer 3: input 1024 dimensions, output 512 dimensions; fully connected layer 4: input 512 dimensions, output 512 dimensions; fully connected layer 5: input 512 dimensions, output 256 dimensions; fully connected layer 6: input 256 dimensions, output 7 dimensions.
6. The point cloud registration method based on local feature learning according to claim 1, characterized in that: In step S5, the training set is input into the point cloud registration network, the source point cloud and the template point cloud are feature extracted and spliced together to obtain the transformation parameters through the fully connected layer, and the source point cloud is transformed according to the transformation parameters to obtain the CDloss values of the transformed point cloud and the template point cloud. The CDLoss value is continuously reduced through the back propagation algorithm to optimize the network parameters of the point cloud registration network and obtain a trained point cloud registration network.
7. The point cloud registration method based on local feature learning according to claim 6, characterized in that: CDLoss is used as the loss function for point cloud registration network training.
8. A point cloud registration system based on local feature learning, characterized in that: include: A preprocessing module reads point cloud data and preprocesses the point cloud data; Select the module, apply transformation to the preprocessed point cloud data, and create training and test sets; Feature module, builds a local feature extraction network, obtains the local features of the point cloud to be registered through the local feature extraction network, and obtains the global features of the point cloud to be registered through the maximum pooling operation, as follows: The point cloud is passed through the first layer of multi-layer perceptron and the second layer of multi-layer perceptron to map the 3D coordinate features to a 64-dimensional high-dimensional space; the source point cloud is sampled using the farthest point sampling to obtain 512 points; Taking each point as the center and the distance as the scale, find the 32 neighboring points closest to each point among the original 1024 points. Subtract the center point feature from the feature of each neighbor point to get a feature vector pointing from the neighbor point to the center point. Each feature vector pointing from the neighbor point to the center point is concatenated with the feature of the center point to obtain a 128-dimensional combined feature vector. Each dimension of the 128-dimensional combined features of the 32 points in each neighborhood is pooled to obtain a 128-dimensional vector representing the neighborhood features. The first 64 dimensions of the 128-dimensional vector represent the maximum eigenvalue of the neighboring points in each neighborhood in a certain dimension, and the last 64 dimensions of the 128-dimensional vector represent the features of the center point. The features of 512 sampling points are converted from 64 dimensions to 128 dimensions. The 512 points are sampled again using the farthest point sampling to obtain 256 points. The 256 neighborhoods are divided again, and a 256-dimensional combined feature vector representing the features of each neighborhood is obtained. The first 128 dimensions represent the maximum eigenvalue of the neighborhood points in each neighborhood in a certain dimension, and the last 128 dimensions represent the features of the center point. The 1024-dimensional features are obtained through the third and fourth multi-layer perceptrons, and the 1024-dimensional features are concatenated with the 256-dimensional features obtained after the second sampling and neighborhood division to obtain 1280-dimensional features; and then the 1024-dimensional feature vector is obtained through the fifth multi-layer perceptron; Finally, a maximum pooling is performed to obtain a 1024-dimensional global feature; The construction module builds a point cloud registration network based on the local features of the point cloud to be registered obtained by the feature module and the global features of the point cloud to be registered. Specifically: The source point cloud and the template point cloud are input into the point cloud registration network. After passing through the local feature extraction network, two 1×1024-dimensional global features are obtained. The two global features are concatenated to obtain a 1×2048-dimensional feature vector. Then, a series of fully connected layers are used to obtain a 1×7-dimensional vector. The rotation quaternion and translation vector are obtained based on the 1×7-dimensional vector. The training module uses the training set obtained by the selection module to train the point cloud registration network constructed by the construction module; The registration module uses the point cloud registration network trained by the training module to register the test set selected by the selection module, obtain the transformation parameters and evaluate the registration results.
