Three-dimensional point cloud registration method based on multiple convolutional networks

Abstract

The invention provides a three-dimensional point cloud registration method based on multiple convolutional networks. Firstly, bilateral filtering and voxel grid downsampling processing are carried out on point cloud data, discrete points of the point cloud data are removed, noise is filtered, point cloud redundant data can be screened under the condition that local details and edge features of the point cloud data are guaranteed, and quality defects are made up; then, constructing a point cloud edge position feature multi-convolution registration network model, and training the model by using the processed point cloud data; and finally, finishing point cloud registration by using a rotation transformation matrix output by the network model. Due to the fact that local information features of the point cloud data can be enhanced through the multiple convolutional networks, the scene learning and understanding ability of the network model to the point cloud data can be improved, and the precision of a point cloud registration task is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional point cloud registration, and in particular relates to a three-dimensional point cloud registration method based on multiple convolutional networks. Background Art

[0002] Point cloud registration is a fundamental field in 3D computer vision and graphics, and is of special and important significance for applications such as 3D reconstruction, 3D data fusion, and map construction. The process aims to estimate the rigid motion transformation parameters between the source point cloud and the target point cloud. By applying these parameters, the two point clouds are aligned, thereby splicing the point cloud data collected from different perspectives into a complete 3D point cloud model. Generally speaking, there will be a certain overlap between point clouds from different perspectives. Therefore, finding the "matching point pairs" for registration from the overlapping area becomes the key to the matching algorithm.

[0003] Despite significant research progress in the field of point cloud registration, some challenges still exist. One of them is the efficient processing of large-scale, high-density point clouds. Solutions to this problem may include distributed computing and more efficient point cloud representation. Another challenge is the modeling of non-rigid transformations. Current point cloud registration methods still have certain limitations when dealing with non-rigid transformations. Future research can explore more flexible network structures or introduce more prior knowledge to solve this problem. In addition, for large-scale point cloud data that lacks annotations, unsupervised or weakly supervised learning methods may become an interesting research direction to reduce the dependence on a large amount of labeled data.

[0004] Point cloud registration, as a key technology in the field of 3D computer vision and graphics, has made significant research progress in recent years. Traditional point cloud registration methods are mainly based on geometric features and optimization techniques. In terms of geometric features, commonly used ones include surface normals, feature points, and descriptors of point clouds. By calculating the similarity of these features, traditional methods can find matching point pairs and estimate the rigid transformation parameters between point clouds. Optimization techniques improve the registration results by minimizing the error between matching point pairs. However, traditional methods have the disadvantages of high computational complexity and strict requirements on initial alignment accuracy when dealing with large-scale, high-density point clouds. Therefore, researchers have gradually introduced deep learning into the field of point cloud registration in order to improve the robustness and efficiency of the algorithm. Summary of the invention

[0005] In order to overcome the shortcomings of the prior art, the present invention provides a three-dimensional point cloud registration method based on multiple convolutional networks. First, the point cloud data is subjected to bilateral filtering and voxel grid downsampling processing to remove its discrete points and filter noise. The redundant point cloud data can be screened and quality defects can be compensated while ensuring its local details and edge features. Then, a multiple convolutional registration network model of point cloud edge position features is constructed, and the model is trained using the processed point cloud data. Finally, the point cloud registration is completed using the rotation transformation matrix output by the network model. Since the local information features of the point cloud data can be enhanced through multiple convolutional networks, the network model's ability to learn and understand the scene of the point cloud data can be improved, thereby improving the accuracy of the point cloud registration task.

[0006] A three-dimensional point cloud registration method based on multiple convolutional networks is characterized by the following steps:

[0007] Step 1: Perform bilateral filtering on the input source point cloud data and target point cloud data;

[0008] Step 2: Perform voxel grid downsampling on the filtered point cloud data;

[0009] Step 3: Construct a point cloud edge position feature multi-convolution registration network model, including a feature extraction module, a feature fusion module and a pose regression module, wherein the feature extraction module includes an edge convolution module and a position adaptive convolution module, the edge convolution module extracts the edge features of the input point cloud data, and then passes through the position adaptive convolution module to perform weighted calculation on the position distance between the point clouds, and outputs high-dimensional features including edge features and point cloud position features; the feature fusion module performs reinforcement learning on the region of interest of the high-dimensional feature data output by the feature extraction module through the spatial attention mechanism module and the channel attention mechanism module, and outputs the weighted position features and channel features; the pose regression module performs regression prediction on the feature data output by the feature fusion module, and outputs the rotation transformation matrix;

[0010] Step 4: Using the source point cloud data and target point cloud data processed in step 2 as input, train the point cloud edge position feature multi-convolution registration network model constructed in step 3;

[0011] Step 5: Use the rotation transformation matrix output by the trained network to transform the source point cloud data to complete the point cloud registration.

[0012] Furthermore, the specific process of the voxel grid downsampling processing described in step 2 is: first, the point cloud data is placed in a three-dimensional coordinate system, the maximum value of the data in each direction in the coordinate system is found, and the volume of the point cloud data is calculated accordingly; then, the volume of the point cloud data is divided into a number of cubic grids of the same size, and it is detected whether there are data points in the grid cubes. If there are no data points, the grid cubes are deleted; otherwise, the center point of the grid cube is calculated, and the points whose distances to the center point are within the threshold range are retained, and the points whose distances to the center point are greater than the threshold are deleted; all grid cubes are processed as before to complete the data downsampling processing.

[0013] Furthermore, the processing process of the edge convolution module is as follows: input point cloud data, pass through the spatial transformation module, and then pass through two edge convolution layers to extract a feature vector containing edge features; wherein the spatial transformation module estimates the coordinate difference 3*3 estimation matrix of each tensor connection point and its k neighboring points through a k-nn graph, and uses the obtained estimation matrix to align the input point cloud data in the standard space; the edge convolution layer first constructs a K-nearest neighbor directed graph representing the local point cloud structure, and then aggregates the edge features between the vertices and neighboring points in the directed graph through a feature aggregation function, and finally obtains the edge features.

[0014] Furthermore, the processing process of the position adaptive convolution module is as follows: for the edge feature values ​​between the vertices and the neighboring points extracted by the edge convolution module, the relative position relationship between the vertices and the neighboring points is calculated by learning the weight coefficients of different positions, and then the relative position relationship is combined by matrix splicing to obtain the weight matrix and the weight coefficients between the positions of the points in the two point cloud data, and the position distance weight of the relative position relationship between the two point cloud data and the weight of the edge convolution are weightedly calculated to output high-dimensional features including edge features and point cloud position features.

[0015] Furthermore, the specific processing process of the feature fusion module is: through the spatial attention mechanism, the high-dimensional features of the input source point cloud data and the target point cloud data including edge features and point cloud position features are comprehensively learned, and the weighted features of each position are output; then, the channel attention mechanism is used to comprehensively learn the high-dimensional features of the input source point cloud data and the target point cloud data including edge features and point cloud position features, and the weighted features of each channel are output.

[0016] Furthermore, the pose regression module uses a fully connected layer to perform a splicing operation on the feature data output by the feature fusion module to obtain a rotation transformation matrix between the source point cloud data and the target point cloud data.

[0017] The beneficial effects of the present invention are as follows: by first performing bilateral filtering and voxel grid downsampling processing on redundant point cloud data, discrete points can be removed and noise can be filtered, while ensuring local details and edge features, redundant point cloud data can be screened and quality defects can be compensated, which is beneficial to improving registration accuracy and computational efficiency; by constructing a multiple convolution registration network model that can enhance local information features such as the edge position of point cloud data, the network model's ability to learn and understand the scene of point cloud data can be improved, thereby improving the accuracy of the point cloud registration task. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a flow chart of a three-dimensional point cloud registration method based on a multiple convolutional network of the present invention;

[0019] Figure 2 It is a schematic diagram of the edge convolution module processing flow of the present invention;

[0020] Figure 3 Schematic diagram of the processing flow of the position adaptive convolution module of the present invention;

[0021] Figure 4 It is a schematic diagram of the effect of using the method of the present invention to perform point cloud registration;

[0022] In the figure, (a) is a schematic diagram of the initial positions of the source point cloud data and the target point cloud data, and (b) is a schematic diagram of the point cloud data after registration. DETAILED DESCRIPTION

[0023] The present invention is further described below in conjunction with the accompanying drawings and embodiments. The present invention includes but is not limited to the following embodiments.

[0024] The present invention provides a three-dimensional point cloud registration method based on multiple convolutional networks, such as Figure 1 As shown, the specific implementation process is as follows:

[0025] 1. Perform bilateral filtering on the input source point cloud data and target point cloud data to filter out the noise and outliers, and obtain point cloud data with more obvious features after filtering.

[0026] The basic principle of the bilateral filtering algorithm is: within the neighborhood of the central target point of the template, the weight distribution not only considers the distance between the pixel point and the central target point of the template, but also comprehensively considers the grayscale value relationship between the pixel point and the central target point of the template. If the grayscale value is closer to the central target point, the greater the distribution weight will be. The specific weight of each pixel point is calculated according to the range Gaussian function, and the final convolution template of the bilateral filtering algorithm is obtained by comprehensive calculation of the weight of the distance size and the weight of the grayscale value. The mathematical process expression is shown in the following formula:

[0027]

[0028]

[0029] Among them, BF represents the gray value relationship between the pixel point of the point cloud data and the target point in the center of the template, I represents the gray value, and W p is the standard quantity, p represents p point cloud data, q represents the lower q point cloud data among p point cloud data, I p Represents the gray value of pixel p at point p, I q Represents the gray value of the target pixel q at the center of the template, σ r and σ d represents the filtered value in the bilateral filtering process, Represents the range function, through It can reduce the interference caused by the pixel p whose gray value is greatly different from the target point. Represents a spatial function, through The influence of pixels that are far away from the target point can be reduced.

[0030] In the denoising process, the bilateral filtering algorithm comprehensively considers the relationship between the grayscale value and distance of the target point pixel in the template center. By comprehensively considering the influence of the two factors and making a compromise, it can not only effectively reduce the influence of various types of noise in the target data, but also strengthen the protection of edge information in the target data, so as to achieve the purpose of removing outliers in the target data. Therefore, after processing the point cloud to be registered data set through the bilateral filtering algorithm, the accuracy of point cloud registration can be further improved.

[0031] 2. Perform voxel grid downsampling on the filtered point cloud data to reduce redundant point cloud data and obtain more streamlined point cloud data while retaining the original features.

[0032] In order to solve the problem of excessive amount of point cloud data and large amount of calculation, the present invention uses a voxel grid downsampling method to simplify the point cloud data. First, the point cloud data is placed in a three-dimensional coordinate system, and the maximum value of each direction in the coordinate system is calculated to calculate the volume of the point cloud data; then, the volume of the point cloud data is divided into a certain number of cubic grids of the same size, and then the data points are detected in the small cubes. If not, the cubes are deleted; if data points exist, the center of the small cube is calculated; finally, the points whose distance from the center of the small cube is within the threshold are retained, and the points that are not within the threshold are discarded. This method can simplify the number of point clouds while retaining the surface features of the object, and the method is easy to operate. A specific step of the algorithm is given below:

[0033] (1) Calculate the volume V of the rectangular bounding box surrounding the point cloud data according to the following formula:

[0034] V=R x R y R z (3)

[0035] Among them, R x Indicates the maximum length of the point cloud data in the X-axis direction, R y Indicates the maximum length of the point cloud data in the Y-axis direction, R z Indicates the maximum length of the point cloud data in the Z-axis direction. In order to prevent the rectangular bounding box from being unable to enclose all the point cloud data and to ensure the accuracy of the data point calculation results, the rectangular bounding box can be slightly expanded with an expansion length of β, that is, the lengths of the corrected point cloud data bounding box in three directions are calculated according to the following formula:

[0036]

[0037] Among them, x max Indicates the maximum value in the X-axis direction, x min Indicates the minimum value in the X-axis direction, y max Indicates the maximum value on the Y axis, y min Indicates the minimum value in the Y-axis direction, z max Indicates the maximum value in the Z-axis direction, z min Indicates the minimum value along the Z axis.

[0038] According to the calculated point cloud bounding box volume, it is assumed that the side length R of the grid cube is:

[0039]

[0040] Among them, α is the adjustment coefficient factor, which is used to adjust the size of the segmented grid cube according to the size of the point cloud volume, k is a proportional coefficient, and n represents the number of data points contained in the grid cube, which is calculated as follows:

[0041]

[0042] Among them, N is the total number of data points contained in the point cloud.

[0043] Substituting equation (3) and equation (4) into equation (5), we obtain:

[0044]

[0045] (2) Divide the point cloud data into m×n×r grid cubes, where ceil(x) is a rounding function, which represents the smallest integer not less than x.

[0046] For any point pi, let the grid cube where pi is located be (m pi , n pi , r pi ),in:

[0047]

[0048] Among them, m pi Indicates the m grid in the X-axis direction where pi is located, n pi represents the n grid in the Y-axis direction where pi is located, r pi Represents the r grid in the Z-axis direction where pi is located.

[0049] In the entire grid coding, if one-dimensional coding is used to represent points, the grid coding of point pi can be expressed as:

[0050] V pi =m pi ×n×r+n pi ×n×r pi (9)

[0051] (3) Substitute the grid coding information of each point in the point cloud data into the hash linked list, build a spatial topological structure for these point cloud data, detect whether there is a data point in the grid cube, if not, delete the cube; if there is a data point, determine the position of each data point, and calculate the Euclidean distance between these points and the center point (i.e., the center of gravity) of the grid cube where they are located, retain the points whose distance to the center of the cube is within the threshold, and discard the points that are not within the threshold. In this way, the point cloud data can be simplified.

[0052] Among them, the center point of the small cube (X ct , Y ct i , Z ct ) is calculated as follows:

[0053]

[0054] Among them, X ct Indicates the center point of the small cube in the X-axis direction, Y ct Indicates the center point of the small cube in the Y-axis direction, Z ct Indicates the center point of the small cube in the Z-axis direction, x i represents the i-th point in the x-axis direction of the small cube, y i represents the i-th point on the Y axis of the small cube, z i Represents the i-th point in the Z-axis direction of the small cube.

[0055] 3. Construct a multi-convolutional registration network model for point cloud edge position features.

[0056] The input of the network model is two sets of point cloud data, the source point cloud and the target point cloud. The deep learning network calculates and outputs the optimal rotation transformation matrix between the source point cloud and the target point cloud. T Through the following relative pose transformation, the registered target point cloud data Q can be generated T :

[0057] Q T =R·P T +t (11)

[0058] Among them, R represents the rotation matrix and t represents the translation vector.

[0059] The point cloud edge position feature multiple convolution registration network model constructed by the present invention mainly includes a feature extraction module, a feature fusion module and a posture regression module.

[0060] In order to establish the correspondence between the source point cloud and the target point cloud and improve the registration accuracy, it is necessary to extract features from the point cloud data. The feature extraction module extracts high-dimensional features including edge features and position features of the point cloud data, mainly including the edge convolution module and the position adaptive convolution module. The edge convolution module is consistent with the idea of ​​the graph convolution network. It extracts the edge features of the local set structure by constructing a local neighborhood graph, such as Figure 2 As shown in the figure, the processing process of the edge convolution module is as follows: input point cloud data, pass through the spatial transformation module, obtain n*3 vector features, and then pass through two edge convolution layers to extract n*a n The feature vector containing edge features. The spatial transformation module estimates the coordinate difference 3*3 estimation matrix of each tensor connection point and its k neighboring points through the k-nn graph, and uses the obtained estimation matrix to align the input point cloud data in the standard space; the edge convolution layer first constructs a K-nearest neighbor directed graph representing the local point cloud structure, and then aggregates the edge features between the vertices and neighboring points in the directed graph through the feature aggregation function, and finally obtains the feature vector containing edge features. Figure 2 In the figure, n is the number of point clouds in the input point cloud data, a n is the number of edge convolution layers, MLP represents the fully connected layer of edge convolution, k represents the k-neighborhood point cloud data around the nth point cloud, and f represents the vector dimension of the point cloud data.

[0061] The feature vector output by the edge convolution module is input into the position adaptive convolution module, and the high-dimensional features including edge features and point cloud position features are output by weighted calculation of the position distance between point clouds. Figure 3 As shown in the figure, the position adaptive convolution module extracts the edge convolution value between the vertex and the neighboring point extracted by the edge convolution module, and learns the weight coefficient E at different positions ij ={Ekij |k=1,2,...,K} to calculate the point p i and its neighboring point p j The relative position relationship between them, where E ij Represents the weight coefficient, E kij represents the weight coefficient of the kth point cloud data feature, K represents K point cloud data; then, the relative position relationship is combined by matrix splicing to obtain the weight matrix W m And the weight coefficient E is obtained by learning the corresponding position of the midpoint of the point cloud data kij , by weighted calculation of the position distance weight of the relative position relationship between point clouds in point cloud data and the weight of edge convolution, high-dimensional features including edge features and point cloud position features are output. Figure 3 In, p i represents the i-th point cloud in the point cloud data, p j represents the jth neighborhood point of the i-th point cloud, E represents the weight coefficient, W represents the weight matrix obtained by concatenating the weight coefficients, and K represents the high-dimensional features including edge features and point cloud position features obtained by weighted calculation.

[0062] In the feature fusion module, the spatial attention mechanism module and the channel attention mechanism module are used to perform reinforcement learning on the regions of interest of the extracted high-dimensional feature data. First, the spatial attention mechanism is used to comprehensively learn the abstract features of the global information of the input source point cloud data and the target point cloud data as well as the local feature information to achieve feature focusing in the spatial dimension and output the weighted features of each position. Secondly, the channel attention mechanism is used to learn the abstract features that integrate the local features and the global information, capture the mapping relationship between any two channel features, and output the weighted features of each channel.

[0063] The features obtained by the feature fusion module are input into the pose regression layer module. In the pose regression layer module, the rotation matrix and translation vector are regressed and predicted respectively through two fully connected layers with different numbers of channels. That is, the feature data output by the feature fusion module are spliced ​​through two fully connected layers to obtain the rotation transformation matrix between the source point cloud data and the target point cloud data, and the rotation transformation matrix is ​​output. The optimal alignment between the source point cloud data and the target point cloud data can be achieved through the rotation matrix.

[0064] 4. Using the source point cloud data and target point cloud data processed in step 2 as input, the point cloud edge position feature multi-convolution registration network model constructed in step 3 is trained, and the trained network outputs the optimal rotation transformation matrix between the source point cloud data and the target point cloud data.

[0065] 5. Use the optimal rotation transformation matrix to rotate the source point cloud data, rotate and move the source point cloud to the target point cloud position, and complete the point cloud registration.

[0066] Figure 4 The initial positions of the source point cloud data and the target point cloud data of a specific example of the present invention are given, as well as a schematic diagram of the result after point cloud registration using the method of the present invention. It can be seen that after the point cloud registration method of the present invention is registered, the registration effect of the source point cloud data and the target point cloud data is good.

[0067] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. A person skilled in the art may modify or make equivalent substitutions for the technical solutions of the present invention without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to the claims.

Claims

1. A three-dimensional point cloud registration method based on multiple convolutional networks, characterized in that Here are the steps: Step 1: Perform bilateral filtering on the input source point cloud data and target point cloud data; Step 2: Perform voxel grid downsampling on the filtered point cloud data; Step 3: Construct a point cloud edge position feature multi-convolution registration network model, including a feature extraction module, a feature fusion module and a pose regression module, wherein the feature extraction module includes an edge convolution module and a position adaptive convolution module, the edge convolution module extracts the edge features of the input point cloud data, and then passes through the position adaptive convolution module to perform weighted calculation on the position distance between the point clouds, and outputs high-dimensional features including edge features and point cloud position features; the feature fusion module performs reinforcement learning on the region of interest of the high-dimensional feature data output by the feature extraction module through the spatial attention mechanism module and the channel attention mechanism module, and outputs the weighted position features and channel features; the pose regression module performs regression prediction on the feature data output by the feature fusion module, and outputs the rotation transformation matrix; Step 4: Using the source point cloud data and target point cloud data processed in step 2 as input, train the point cloud edge position feature multi-convolution registration network model constructed in step 3; Step 5: Use the rotation transformation matrix output by the trained network to transform the source point cloud data to complete the point cloud registration.

2. A three-dimensional point cloud registration method based on a multi-convolutional network as claimed in claim 1, characterized in that: The specific process of the voxel grid downsampling processing described in step 2 is: first, put the point cloud data into a three-dimensional coordinate system, find the maximum value of the data in each direction in the coordinate system, and calculate the volume of the point cloud data accordingly; then, divide the volume of the point cloud data into a number of cubic grids of the same size, and detect whether there are data points in the grid cubes. If there are no data points, delete the grid cubes; otherwise, calculate the center point of the grid cube, retain the points whose distances to the center point are within the threshold range, and delete the points whose distances to the center point are greater than the threshold; all grid cubes are processed as before to complete the data downsampling processing.

3. The three-dimensional point cloud registration method based on multiple convolutional networks according to claim 1, characterized in that: The processing process of the edge convolution module is as follows: input point cloud data, pass through the spatial transformation module, and then pass through two edge convolution layers to extract a feature vector containing edge features; wherein the spatial transformation module estimates the coordinate difference 3*3 estimation matrix of each tensor connection point and its k neighboring points through a k-nn graph, and uses the obtained estimation matrix to align the input point cloud data in the standard space; the edge convolution layer first constructs a K-nearest neighbor directed graph representing the local point cloud structure, and then aggregates the edge features between the vertices and neighboring points in the directed graph through a feature aggregation function, and finally obtains the edge features.

4. The three-dimensional point cloud registration method based on multiple convolutional networks according to claim 1, characterized in that: The processing process of the position adaptive convolution module is as follows: for the edge feature values ​​between the vertices and the neighboring points extracted by the edge convolution module, the relative position relationship between the vertices and the neighboring points is calculated by learning the weight coefficients of different positions, and then the relative position relationship is combined by matrix splicing to obtain the weight matrix and the weight coefficients between the positions of the points in the two point cloud data, and the position distance weight of the relative position relationship between the two point cloud data and the weight of the edge convolution are weightedly calculated to output high-dimensional features including edge features and point cloud position features.

5. The three-dimensional point cloud registration method based on multiple convolutional networks according to claim 1, characterized in that: The specific processing process of the feature fusion module is: the high-dimensional features including edge features and point cloud position features of the input source point cloud data and the target point cloud data are comprehensively learned through the spatial attention mechanism, and the weighted features of each position are output; then, the high-dimensional features including edge features and point cloud position features of the input source point cloud data and the target point cloud data are comprehensively learned using the channel attention mechanism, and the weighted features of each channel are output.

6. The three-dimensional point cloud registration method based on multiple convolutional networks according to claim 1, characterized in that: The posture regression module uses a fully connected layer to perform a splicing operation on the feature data output by the feature fusion module to obtain a rotation transformation matrix between the source point cloud data and the target point cloud data.

Images