A 3D point cloud registration method based on multiple convolutional networks

Abstract

The present invention provides a three-dimensional point cloud registration method based on multiple convolutional networks. First, the point cloud data is subjected to bilateral filtering and voxel grid downsampling processing to remove its discrete points and filter out noise. This can screen redundant point cloud data and compensate for quality defects while ensuring its local details and edge features. Then, a multiple convolutional registration network model for point cloud edge position features is constructed, and the model is trained using the processed point cloud data. Finally, the point cloud registration is completed using the rotation transformation matrix output by the network model. Because the local information features of the point cloud data can be enhanced through multiple convolutional networks, the network model's ability to learn and understand the scene of the point cloud data can be improved, thereby improving the accuracy of the point cloud registration task.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional point cloud registration, and in particular relates to a three-dimensional point cloud registration method based on multiple convolutional networks. Background Art

[0002] Point cloud registration is a fundamental field in 3D computer vision and graphics, and is of special and significant importance for applications such as 3D reconstruction, 3D data fusion, and map construction. This process aims to estimate the rigid motion transformation parameters between the source and target point clouds. By applying these parameters, the two point clouds are aligned, thereby stitching point cloud data collected from different viewpoints into a complete 3D point cloud model. Generally speaking, point clouds from different viewpoints will have a certain amount of overlap, so finding "matching point pairs" for registration within this overlapping area is the key to the matching algorithm.

[0003] Despite significant research progress in the field of point cloud registration, some challenges still exist. One of them is the efficient processing of large-scale, high-density point clouds. Solutions to this problem may include distributed computing and more efficient point cloud representations. Another challenge is the modeling of non-rigid transformations. Current point cloud registration methods still have certain limitations when dealing with non-rigid transformations. Future research can explore more flexible network structures or introduce more prior knowledge to solve this problem. In addition, for large-scale point cloud data that lacks annotations, unsupervised or weakly supervised learning methods may become an interesting research direction to reduce the dependence on a large amount of labeled data.

[0004] Point cloud registration, as a key technology in the field of 3D computer vision and graphics, has made significant research progress in recent years. Traditional point cloud registration methods are mainly based on geometric features and optimization techniques. In terms of geometric features, commonly used ones include surface normals, feature points, and descriptors of point clouds. By calculating the similarity of these features, traditional methods can find matching point pairs and thus estimate the rigid transformation parameters between point clouds. Optimization techniques improve the registration results by minimizing the error between matching point pairs. However, traditional methods have the disadvantages of high computational complexity and strict requirements on initial alignment accuracy when processing large-scale, high-density point clouds. Therefore, researchers have gradually introduced deep learning into the field of point cloud registration in order to improve the robustness and efficiency of the algorithm. Summary of the Invention

[0005] In order to overcome the shortcomings of the existing technology, the present invention provides a three-dimensional point cloud registration method based on multiple convolutional networks. First, the point cloud data is subjected to bilateral filtering and voxel grid downsampling processing to remove its discrete points and filter out noise. This can screen the redundant point cloud data and make up for quality defects while ensuring its local details and edge features. Then, a multiple convolutional registration network model of point cloud edge position features is constructed, and the model is trained using the processed point cloud data. Finally, the point cloud registration is completed using the rotation transformation matrix output by the network model. Since the local information features of the point cloud data can be enhanced through multiple convolutional networks, the network model's ability to learn and understand the scene of the point cloud data can be improved, thereby improving the accuracy of the point cloud registration task.

[0006] A three-dimensional point cloud registration method based on multiple convolutional networks is characterized by the following steps:

[0007] Step 1: Perform bilateral filtering on the input source point cloud data and target point cloud data;

[0008] Step 2: Perform voxel grid downsampling on the filtered point cloud data;

[0009] Step 3: Construct a point cloud edge position feature multi-convolution registration network model, which includes a feature extraction module, a feature fusion module and a pose regression module. The feature extraction module includes an edge convolution module and a position adaptive convolution module. The edge convolution module extracts the edge features of the input point cloud data, and then passes through the position adaptive convolution module to perform weighted calculation on the position distance between the point clouds, and outputs high-dimensional features containing edge features and point cloud position features; the feature fusion module uses the spatial attention mechanism module and the channel attention mechanism module to perform reinforcement learning on the region of interest of the high-dimensional feature data output by the feature extraction module, and outputs weighted position features and channel features; the pose regression module performs regression prediction on the feature data output by the feature fusion module, and outputs a rotation transformation matrix;

[0010] The processing process of the position adaptive convolution module is as follows: for the edge feature values ​​between the vertex and the neighboring points extracted by the edge convolution module, the relative position relationship between the vertex and the neighboring points is calculated by learning the weight coefficients of different positions, and then the relative position relationship is combined by matrix splicing to obtain a weight matrix and the weight coefficient between the position of the points in the two point cloud data, and the position distance weight of the relative position relationship between the two point cloud data and the weight of the edge convolution are weightedly calculated to output a high-dimensional feature containing edge features and point cloud position features;

[0011] The specific processing process of the feature fusion module is as follows: the spatial attention mechanism is used to comprehensively learn the high-dimensional features of the input source point cloud data and the target point cloud data, including edge features and point cloud position features, and output the weighted features of each position; then, the channel attention mechanism is used to comprehensively learn the high-dimensional features of the input source point cloud data and the target point cloud data, including edge features and point cloud position features, and output the weighted features of each channel;

[0012] Step 4: Using the source point cloud data and target point cloud data processed in step 2 as input, train the point cloud edge position feature multi-convolution registration network model constructed in step 3;

[0013] Step 5: Use the rotation transformation matrix output by the trained network to transform the source point cloud data to complete the point cloud registration.

[0014] Furthermore, the specific process of the voxel grid downsampling processing described in step 2 is as follows: first, the point cloud data is placed in a three-dimensional coordinate system, the maximum value of the data in each direction in the coordinate system is found, and the volume of the point cloud data is calculated accordingly; then, the volume of the point cloud data is divided into a number of cubic grids of the same size, and the presence of data points in the grid cubes is detected. If no data points exist, the grid cubes are deleted; otherwise, the center point of the grid cube is calculated, and points whose distances to the center point are within a threshold range are retained, and points whose distances to the center point are greater than the threshold are deleted; all grid cubes are processed as before to complete the data downsampling processing.

[0015] Furthermore, the processing process of the edge convolution module is as follows: input point cloud data, pass through the spatial transformation module, and then pass through two edge convolution layers to extract the feature vector containing edge features; wherein, the spatial transformation module estimates the coordinate difference 3*3 estimation matrix of each tensor connection point and its k neighboring points through the k-nn graph, and uses the obtained estimation matrix to align the input point cloud data in the standard space; the edge convolution layer first constructs a K-nearest neighbor directed graph representing the local point cloud structure, and then aggregates the edge features between the vertices and neighboring points in the directed graph through the feature aggregation function, and finally obtains the edge features.

[0016] Furthermore, the pose regression module uses a fully connected layer to perform a splicing operation on the feature data output by the feature fusion module to obtain a rotation transformation matrix between the source point cloud data and the target point cloud data.

[0017] The beneficial effects of the present invention are as follows: since the redundant point cloud data is first subjected to bilateral filtering and voxel grid downsampling processing, its discrete points can be removed and noise can be filtered out, while ensuring its local details and edge features, the redundant point cloud data can be screened and quality defects can be compensated, which is beneficial to improving the registration accuracy and computational efficiency; since a multiple convolution registration network model is constructed that can enhance local information features such as the edge position of point cloud data, the network model's ability to learn and understand the scene of point cloud data can be improved, thereby improving the accuracy of the point cloud registration task. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of a three-dimensional point cloud registration method based on multiple convolutional networks of the present invention;

[0019] Figure 2 Schematic diagram of the edge convolution module processing flow of the present invention;

[0020] Figure 3 Schematic diagram of the processing flow of the position adaptive convolution module of the present invention;

[0021] Figure 4 Schematic diagram of the effect of point cloud registration using the method of the present invention;

[0022] In the figure, (a) is a schematic diagram of the initial positions of the source point cloud data and the target point cloud data, and (b) is a schematic diagram of the point cloud data after registration. DETAILED DESCRIPTION

[0023] The present invention is further described below with reference to the accompanying drawings and examples. The present invention includes but is not limited to the following examples.

[0024] The present invention provides a three-dimensional point cloud registration method based on multiple convolutional networks, such as Figure 1 The specific implementation process is as follows:

[0025] 1. Perform bilateral filtering on the input source point cloud data and target point cloud data to filter out the noise and outliers, and obtain point cloud data with more obvious features after filtering.

[0026] The basic principle of the bilateral filtering algorithm is: within the neighborhood of the template center target point, the weight distribution not only considers the distance between the pixel point and the template center target point, but also comprehensively considers the grayscale value relationship between the pixel point and the template center target point. If the grayscale value is closer to the center target point, the greater the distribution weight will be. The specific weight of each pixel point is calculated according to the range Gaussian function. The final convolution template of the bilateral filtering algorithm is obtained by comprehensive calculation of the weight of the distance size and the weight of the grayscale value. Its mathematical process expression is shown as follows:

[0027]

[0028] Among them, BF represents the grayscale value relationship between the pixel point of the point cloud data and the target point in the center of the template, I represents the grayscale value, and W p is the standard quantity, p represents p point cloud data, q represents the lower q point cloud data among p point cloud data, I p Indicates the grayscale value of pixel p at point p, I q Represents the grayscale value of the target pixel q at the center of the template, σ r and σ d represents the filtered value in the bilateral filtering process, Represents the range function, through It can reduce the interference caused by the pixel p that has a large difference in gray value from the target point. Represents a spatial function, through It can reduce the influence of pixels that are far away from the target point.

[0029] During the denoising process, the bilateral filtering algorithm comprehensively considers the relationship between the grayscale value and distance of the target point pixel at the center of the template. By comprehensively considering the influence of these two factors and making a compromise, it can effectively reduce the impact of various types of noise in the target data while also strengthening the protection of edge information in the target data, achieving the purpose of removing outliers in the target data. Therefore, after processing the point cloud to be registered dataset with the bilateral filtering algorithm, the accuracy of point cloud registration can be further improved.

[0030] 2. Perform voxel grid downsampling on the filtered point cloud data to reduce redundant point cloud data and obtain more streamlined point cloud data while retaining the original features.

[0031] In order to solve the problem of excessive amount of point cloud data and large amount of calculation, the present invention uses the voxel grid downsampling method to simplify the point cloud data. First, the point cloud data is placed in a three-dimensional coordinate system, and the maximum value of each direction in the coordinate system is calculated to calculate the volume of the point cloud data; then, the volume of the point cloud data is divided into a certain number of cubic grids of the same size, and then the presence of data points in the small cube is detected. If not, the cube is deleted; if the data point exists, the center of the small cube is calculated; finally, the points whose distance from the center of the small cube is within the threshold are retained, and the points that are not within the threshold are discarded. This method can simplify the number of point clouds while retaining the surface features of the object, and the method is easy to operate. A specific step of the algorithm is given below:

[0032] (1) Calculate the volume V of the rectangular bounding box surrounding the point cloud data according to the following formula:

[0033] V=R x R yR z (3)

[0034] Among them, R x Indicates the maximum length of the point cloud data in the X-axis direction, R y Indicates the maximum length of the point cloud data in the Y-axis direction, R z Indicates the maximum length of the point cloud data in the Z-axis direction. To prevent the rectangular bounding box from not enclosing all the point cloud data and to ensure the accuracy of the data point calculation results, the rectangular bounding box can be slightly expanded by a length of β. That is, the lengths of the corrected point cloud data bounding box in the three directions are calculated according to the following formula:

[0035]

[0036] Among them, x max Indicates the maximum value in the X-axis direction, x min Indicates the minimum value in the X-axis direction, y max Indicates the maximum value in the Y-axis direction, y min Indicates the minimum value in the Y-axis direction, z max Indicates the maximum value in the Z-axis direction, z min Indicates the minimum value along the Z axis.

[0037] According to the calculated point cloud bounding box volume, it is assumed that the side length R of the grid cube is:

[0038]

[0039] Among them, α is the adjustment coefficient factor, which is used to adjust the size of the segmented grid cube according to the size of the point cloud volume, k is a proportional coefficient, and n represents the number of data points contained in the grid cube, which is calculated as follows:

[0040]

[0041] Where N is the total number of data points contained in the point cloud.

[0042] Substituting equations (3) and (4) into equation (5), we obtain:

[0043]

[0044] (2) Divide the point cloud data into m×n×r grid cubes, where ceil(x) is a rounding function, representing the smallest integer not less than x.

[0045] For any point pi, let the grid cube where pi is located be (m pi ,npi ,r pi ),in:

[0046]

[0047] Among them, m pi Indicates the m grid in the X-axis direction where pi is located, n pi Indicates the n grid in the Y-axis direction where pi is located, r pi Represents the r grid in the Z-axis direction where pi is located.

[0048] In the entire grid coding, if a one-dimensional code is used to represent a point, the grid code of point pi can be expressed as:

[0049] V pi =m pi ×n×r+n pi ×n×r pi (9)

[0050] (3) Substitute the grid coding information of each point in the point cloud data into a hash list, build a spatial topological structure for these point cloud data, and detect whether there is a data point in the grid cube. If not, delete the cube; if there is a data point, determine the position of each data point and calculate the Euclidean distance between these points and the center point (i.e., the center of gravity) of the grid cube where they are located. Keep the points whose distance to the center of the cube is within the threshold, and discard the points that are not within the threshold. In this way, the point cloud data can be simplified.

[0051] Among them, the center point of the small cube (X ct ,Y cti ,Z ct ) is calculated as follows:

[0052]

[0053] Among them, X ct Indicates the center point of the small cube in the X-axis direction, Y ct Indicates the center point of the small cube in the Y-axis direction, Z ct Indicates the center point of the small cube in the Z-axis direction, x i Indicates the i-th point in the X-axis direction of the small cube, y i Indicates the i-th point on the Y axis of the small cube, z i Represents the i-th point in the Z-axis direction of the small cube.

[0054] 3. Construct a multi-convolution registration network model based on point cloud edge position features.

[0055] The input of the network model is two sets of point cloud data, the source point cloud and the target point cloud. The deep learning network calculates and outputs the optimal rotation transformation matrix between the source point cloud and the target point cloud. T The registered target point cloud data Q can be generated by the following relative pose transformation: T :

[0056] Q T =R·P T +t (11)

[0057] Where R represents the rotation matrix and t represents the translation vector.

[0058] The point cloud edge position feature multi-convolution registration network model constructed by the present invention mainly includes a feature extraction module, a feature fusion module and a pose regression module.

[0059] In order to establish the correspondence between the source point cloud and the target point cloud and improve the registration accuracy, it is necessary to extract features from the point cloud data. The feature extraction module extracts high-dimensional features including edge features and position features of the point cloud data. It mainly includes the edge convolution module and the position adaptive convolution module. The edge convolution module is consistent with the graph convolution network. It extracts edge features of the local set structure by constructing a local neighborhood graph, such as Figure 2 As shown in the figure, the processing process of the edge convolution module is as follows: input point cloud data, pass through the spatial transformation module to obtain n*3 vector features, and then pass through two edge convolution layers to extract n*a n The spatial transformation module estimates the coordinate difference between each tensor connection point and its k neighbors using a k-nn graph. The estimated matrix is ​​then used to align the input point cloud data in the canonical space. The edge convolution layer first constructs a K-nearest neighbor directed graph representing the local point cloud structure. It then aggregates the edge features between the vertices and neighboring points in the directed graph using a feature aggregation function, ultimately obtaining a feature vector containing edge features. Figure 2 In the figure, n is the number of input point cloud data points, a n is the number of edge convolution layers, MLP represents the fully connected layer of edge convolution, k represents the k-neighborhood point cloud data around the n-th point cloud, and f represents the vector dimension of the point cloud data.

[0060] The feature vector output by the edge convolution module is input into the position adaptive convolution module, and the position distance between point clouds is weighted to calculate the output of high-dimensional features containing edge features and point cloud position features. Figure 3 As shown in the figure, the position adaptive convolution module extracts the edge convolution value between the vertex and the adjacent point obtained by the edge convolution module, and learns the weight coefficient E at different positions. ij ={Ekij |k=1,2,...,K} to calculate the point p i and its neighboring point p j The relative position relationship between them, where E ij Represents the weight coefficient, E kij Represents the weight coefficient of the kth point cloud data feature, K represents K point cloud data; then, the relative position relationship is combined by matrix splicing to obtain the weight matrix W m And the weight coefficient E is obtained by learning the corresponding position of the point in the point cloud data kij , by weighted calculation of the position distance weight of the relative position relationship between point clouds in point cloud data and the weight of edge convolution, high-dimensional features containing edge features and point cloud position features are output. Figure 3 In, p i Represents the i-th point cloud in the point cloud data, p j represents the jth neighborhood point of the i-th point cloud, E represents the weight coefficient, W represents the weight matrix obtained by concatenating the weight coefficients, and K represents the high-dimensional features obtained by weighted calculation including edge features and point cloud position features.

[0061] In the feature fusion module, the extracted high-dimensional feature data is subjected to reinforcement learning on the regions of interest of the feature data through the spatial attention mechanism module and the channel attention mechanism module. First, through the spatial attention mechanism, the global information of the input source point cloud data and the target point cloud data and the abstract features of the local feature information are comprehensively learned to achieve feature focusing in the spatial dimension, and the weighted features of each position are output. Secondly, the channel attention mechanism is used to learn the abstract features of the integrated local features and global information, capture the mapping relationship between any two channel features, and output the weighted features of each channel.

[0062] The features obtained by the feature fusion module are input into the pose regression layer module. In the pose regression layer module, the rotation matrix and translation vector are regressed and predicted respectively through two fully connected layers with different numbers of channels. That is, the feature data output by the feature fusion module are spliced ​​through two fully connected layers to obtain the rotation transformation matrix between the source point cloud data and the target point cloud data, and the rotation transformation matrix is ​​output. The optimal alignment between the source point cloud data and the target point cloud data can be achieved through the rotation matrix.

[0063] 4. Using the source point cloud data and target point cloud data processed in step 2 as input, the point cloud edge position feature multi-convolution registration network model constructed in step 3 is trained. The trained network outputs the optimal rotation transformation matrix between the source point cloud data and the target point cloud data.

[0064] 5. Use the optimal rotation transformation matrix to rotate the source point cloud data, rotate and move the source point cloud to the target point cloud position, and complete the point cloud registration.

[0065] Figure 4 The initial positions of the source point cloud data and the target point cloud data in a specific example of the present invention are shown, as well as a schematic diagram of the result after point cloud registration using the method of the present invention. It can be seen that after registration using the point cloud registration method of the present invention, the source point cloud data and the target point cloud data are well registered.

[0066] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Those skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be based on the claims.

Claims

1. A three-dimensional point cloud registration method based on multiple convolutional networks, characterized by Here are the steps: Step 1: Perform bilateral filtering on the input source point cloud data and target point cloud data; Step 2: Perform voxel grid downsampling on the filtered point cloud data; Step 3: Construct a point cloud edge position feature multi-convolution registration network model, which includes a feature extraction module, a feature fusion module and a pose regression module. The feature extraction module includes an edge convolution module and a position adaptive convolution module. The edge convolution module extracts the edge features of the input point cloud data, and then passes through the position adaptive convolution module to perform weighted calculation on the position distance between the point clouds, and outputs high-dimensional features containing edge features and point cloud position features; the feature fusion module uses the spatial attention mechanism module and the channel attention mechanism module to perform reinforcement learning on the region of interest of the high-dimensional feature data output by the feature extraction module, and outputs weighted position features and channel features; the pose regression module performs regression prediction on the feature data output by the feature fusion module, and outputs a rotation transformation matrix; The processing process of the position adaptive convolution module is as follows: for the edge feature values ​​between the vertex and the neighboring points extracted by the edge convolution module, the relative position relationship between the vertex and the neighboring points is calculated by learning the weight coefficients of different positions, and then the relative position relationship is combined by matrix splicing to obtain a weight matrix and the weight coefficient between the position of the points in the two point cloud data, and the position distance weight of the relative position relationship between the two point cloud data and the weight of the edge convolution are weightedly calculated to output a high-dimensional feature containing edge features and point cloud position features; The specific processing process of the feature fusion module is as follows: the spatial attention mechanism is used to comprehensively learn the high-dimensional features of the input source point cloud data and the target point cloud data, including edge features and point cloud position features, and output the weighted features of each position; then, the channel attention mechanism is used to comprehensively learn the high-dimensional features of the input source point cloud data and the target point cloud data, including edge features and point cloud position features, and output the weighted features of each channel; Step 4: Using the source point cloud data and target point cloud data processed in step 2 as input, train the point cloud edge position feature multi-convolution registration network model constructed in step 3; Step 5: Use the rotation transformation matrix output by the trained network to transform the source point cloud data to complete the point cloud registration.

2. The three-dimensional point cloud registration method based on a multi-convolutional network according to claim 1, characterized in that: The specific process of the voxel grid downsampling processing described in step 2 is as follows: first, the point cloud data is placed in a three-dimensional coordinate system, the maximum value of the data in each direction in the coordinate system is found, and the volume of the point cloud data is calculated based on this; then, the volume of the point cloud data is divided into a number of cubic grids of the same size, and the presence of data points in the grid cubes is detected. If no data points exist, the grid cubes are deleted; otherwise, the center point of the grid cube is calculated, and points whose distance to the center point is within a threshold range are retained, and points whose distance to the center point is greater than the threshold are deleted; all grid cubes are processed as before to complete the data downsampling processing.

3. The three-dimensional point cloud registration method based on a multi-convolutional network according to claim 1, characterized in that: The processing process of the edge convolution module is as follows: input point cloud data, pass through the spatial transformation module, and then pass through two edge convolution layers to extract the feature vector containing edge features; wherein, the spatial transformation module estimates the coordinate difference 3*3 estimation matrix of each tensor connection point and its k neighboring points through the k-nn graph, and uses the obtained estimation matrix to align the input point cloud data in the standard space; the edge convolution layer first constructs a K-nearest neighbor directed graph representing the local point cloud structure, and then aggregates the edge features between the vertices and neighboring points in the directed graph through the feature aggregation function to finally obtain the edge features.

4. The three-dimensional point cloud registration method based on a multi-convolutional network according to claim 1, characterized in that: The pose regression module uses a fully connected layer to perform a splicing operation on the feature data output by the feature fusion module to obtain a rotation transformation matrix between the source point cloud data and the target point cloud data.

Images