A point cloud registration method combining graph feature context matching

Abstract

The present invention relates to a point cloud registration method combined with graph feature context matching, comprising the following steps: extracting graph features in a geometric domain to obtain graph feature A; extracting graph features in a feature domain to obtain graph feature B according to graph feature A of a source point cloud and a target point cloud; obtaining accurate features to obtain complete graph feature C according to graph feature A and graph feature B, performing feature context matching according to graph feature C to obtain transformation features, and obtaining accurate features according to transformation features and graph feature C; constructing a feature difference matching matrix according to a source point cloud and a target point cloud containing accurate features; estimating a rigid transformation from a source point cloud to a target point cloud according to the feature difference matching matrix; applying the rigid transformation to a source point cloud, setting the number of iterations N, performing iterations, and obtaining a refined rigid transformation; and performing point cloud registration according to the rigid transformation. The feature domain registration based on deep learning of the present invention is less affected by noise and outliers, and has good robustness for local point cloud registration.

Description

Technical Field

[0001] The invention relates to a point cloud registration method combined with image feature context matching, and belongs to the technical field of point cloud registration. Background Art

[0002] With the rapid development of point cloud acquisition devices such as Lidar and Kinect, point cloud has become the main data format for representing the three-dimensional world. Point cloud registration is a processing technology that transforms the point cloud acquired by the acquisition device at different viewing angles into the same reference system. When using a point cloud scanning device to scan the surrounding environment in the same coordinate system, the complete point cloud data cannot be obtained because the size of the environment exceeds the measurement range of the scanning device and there are occlusion problems between objects in the environment. Therefore, it is necessary to scan at multiple angles and use point cloud registration technology to unify the point cloud data from multiple angles into the same coordinate system to obtain complete environmental point cloud data.

[0003] The point cloud registration process can be divided into coarse registration and fine registration according to the initial conditions and accuracy. Coarse registration is to register the point cloud when the point cloud transformation is unknown and estimate the initial transformation of the registration; fine registration is to further refine the registration when the initial transformation is known. Traditional methods perform registration based on the geometric information of the collected point cloud, which is easily affected by noise and outliers. Some fine registration methods also require appropriate initial values.

[0004] Feature domain registration based on deep learning is less affected by noise and outliers, has good robustness, and does not need to consider the initial value of registration. Deep learning point cloud registration is affected by feature accuracy. By extracting accurate graph features through graph convolution, combining the matching ideas of traditional methods with rigid transformation estimation methods, it can not only improve the accuracy of point cloud registration, but also enhance robustness. Summary of the invention

[0005] In order to overcome the above problems, the present invention provides a point cloud registration method combined with graph feature context matching. The point cloud registration method is based on deep learning feature domain registration, which is less affected by noise and outliers, has good robustness, and does not need to consider the initial value of registration. Deep learning point cloud registration is affected by feature accuracy. By extracting accurate graph features through graph convolution, combining the matching ideas of traditional methods with rigid transformation estimation methods, it can not only improve the accuracy of point cloud registration, but also enhance robustness.

[0006] The technical solution of the present invention is as follows:

[0007] A point cloud registration method combined with graph feature context matching comprises the following steps:

[0008] Extracting geometric domain graph features: extracting geometric domain graph features from the input source point cloud and the target point cloud respectively to obtain graph features A of the source point cloud and the target point cloud;

[0009] Feature domain graph feature extraction: according to the graph feature A of the source point cloud and the target point cloud, feature domain graph feature extraction is performed on the source point cloud and the target point cloud respectively to obtain the graph feature B of the source point cloud and the target point cloud;

[0010] Accurate feature acquisition: according to the graph feature A and the graph feature B, a complete graph feature C of the source point cloud and the target point cloud is obtained; feature context matching is performed according to the graph feature C to obtain a transformation feature; according to the transformation feature and the graph feature C, accurate features of the source point cloud and the target point cloud are obtained;

[0011] Constructing a feature difference matching matrix based on the source point cloud and the target point cloud containing accurate features;

[0012] estimating a rigid transformation from the source point cloud to the target point cloud according to the feature difference matching matrix;

[0013] Apply the rigid transformation to the source point cloud, set the number of iterations N, repeat the above steps N times, and obtain a refined rigid transformation;

[0014] Point cloud registration is performed according to the rigid transformation.

[0015] Furthermore, the geometric domain graph feature extraction is specifically as follows:

[0016] Set the ball search radius and the number of search points K, and use the ball query method to find K neighboring points for each point cloud of the input source point cloud and the target point cloud;

[0017] According to each of the neighboring points, K graph edges between each local center point and the neighboring points are calculated, and local structure information PPF is calculated, wherein the local structure information PPF includes the angle between the center point normal vector and the graph edge, the angle between the neighboring point normal vector and the graph edge, the angle between the center point normal vector and the neighboring point normal vector, and the edge length;

[0018] For each local area, a local geometric domain graph structure is established based on the center point coordinates, the center point normal vector, the neighboring point coordinates, the graph edge (the difference between the three-dimensional coordinates of the center point and the neighboring points of the graph) and the local structure information PPF. The graph feature A of each local geometric domain graph structure is extracted through convolution, and the graph feature A is aggregated to the center point and mapped to the high-dimensional space through MLP (multi-layer perceptron).

[0019] Furthermore, the feature extraction of the feature domain graph is specifically as follows:

[0020] For a point in the source point cloud and the target point cloud, according to the graph feature A of the target point cloud, the K nearest neighbor method is used to find K feature neighboring points in the target point cloud, and the graph edges between the center point and the neighboring points are calculated to establish a local feature domain graph structure; the graph feature B of each local feature domain graph structure is extracted through convolution, the graph feature B is aggregated to the center point, and mapped to a high-dimensional space through MLP.

[0021] Furthermore, the accurate feature acquisition is specifically as follows:

[0022] The graph features A and B of the source point cloud are spliced ​​to obtain the complete graph feature C of the source point cloud; the graph features A and B of the target point cloud are spliced ​​to obtain the complete graph feature C of the target point cloud;

[0023] The graph feature C of the source point cloud is used as the encoding input, and the graph feature C of the target point cloud is used as the decoding input. The point cloud graph features of the source point cloud and the target point cloud are context-matched through the Transformer network, and the transformation information from the source point cloud to the target point cloud is learned and embedded in the source point cloud to obtain the accurate features of the source point cloud;

[0024] The graph feature C of the source point cloud is used as the decoding input, and the graph feature C of the target point cloud is used as the encoding input. The point cloud graph features of the target point cloud and the source point cloud are context-matched through the Transformer network, the transformation information from the target point cloud to the source point cloud is learned, and embedded in the source point cloud to obtain the accurate features of the target point cloud.

[0025] Furthermore, the constructing of a feature difference matching matrix based on the source point cloud and the target point cloud containing accurate features is specifically as follows:

[0026] Calculate the Euclidean distance between the source point cloud and the target point cloud containing accurate features, and output it as a feature difference matrix;

[0027] Adding outliers and local optimal restriction parameters to the feature difference matrix, and using the Sinkhorn algorithm (an algorithm for changing the probability distribution of a matrix) to numerically optimize each element of the feature difference matrix;

[0028] Each element of the feature difference matrix is ​​normalized using an exponential function, the corresponding relationship between points is represented by probability, and a feature difference matching matrix is ​​output.

[0029] Furthermore, the estimating the rigid transformation from the source point cloud to the target point cloud according to the feature difference matching matrix is ​​specifically:

[0030] The feature difference matching matrix is ​​applied to the target point cloud to obtain a virtual point cloud, the sum of each row of the feature difference matching matrix is ​​used as a weight, and the weighted SVD (singular value decomposition) method is applied to the source point cloud and the virtual point cloud to estimate the rigid transformation from the source point cloud to the target point cloud.

[0031] The present invention has the following beneficial effects:

[0032] In order to improve the accuracy of point cloud feature extraction, the present invention first adopts steps such as geometric domain graph convolution, feature domain graph convolution, and single codec Transformer matching feature context. In order to achieve the accuracy and robustness of the point cloud registration method, the feature difference matching matrix is ​​used to establish the correspondence relationship, and the weighted SVD is used to estimate the accurate rigid transformation. Therefore, the dual advantages of high accuracy and strong robustness of registration can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is an overall block diagram of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] Embodiment 1

[0036] refer to Figure 1 , a point cloud registration method combined with graph feature context matching, comprising the following steps:

[0037] Extracting geometric domain graph features: extracting geometric domain graph features from the input source point cloud and the target point cloud respectively to obtain graph features A of the source point cloud and the target point cloud;

[0038] Feature domain graph feature extraction: according to the graph feature A of the source point cloud and the target point cloud, feature domain graph feature extraction is performed on the source point cloud and the target point cloud respectively to obtain the graph feature B of the source point cloud and the target point cloud;

[0039] Accurate feature acquisition: according to the graph feature A and the graph feature B, a complete graph feature C of the source point cloud and the target point cloud is obtained; feature context matching is performed according to the graph feature C to obtain a transformation feature; according to the transformation feature and the graph feature C, accurate features of the source point cloud and the target point cloud are obtained;

[0040] Constructing a feature difference matching matrix based on the source point cloud and the target point cloud containing accurate features;

[0041] estimating a rigid transformation from the source point cloud to the target point cloud according to the feature difference matching matrix;

[0042] Apply the rigid transformation to the source point cloud, set the number of iterations N, repeat the above steps N times, and obtain a refined rigid transformation;

[0043] Point cloud registration is performed according to the rigid transformation.

[0044] Embodiment 2

[0045] A point cloud registration method combined with graph feature context matching, based on the first embodiment, the geometric domain graph feature extraction is specifically as follows:

[0046] Set the ball search radius and the number of search points K, and use the ball radius search method to find K neighboring points for each point cloud of the input source point cloud and the target point cloud;

[0047] According to each of the neighboring points, K graph edges between each local center point and the neighboring points are calculated, and local structure information PPF is calculated, wherein the local structure information PPF includes the angle between the center point normal vector and the graph edge, the angle between the neighboring point normal vector and the graph edge, the angle between the center point normal vector and the neighboring point normal vector, and the edge length;

[0048] For each local part, a local geometric domain graph structure is established according to the center point coordinates, center point normal vector, neighboring point coordinates, graph edges and local structure information PPF, and the graph features A of each local geometric domain graph structure are extracted through convolution. The graph features A are aggregated to the center point and mapped to the high-dimensional space through MLP.

[0049] Embodiment 3

[0050] A point cloud registration method combined with graph feature context matching, based on the second embodiment, the feature domain graph feature extraction is specifically as follows:

[0051] For a point in the source point cloud and the target point cloud, according to the graph feature A of the target point cloud, the K nearest neighbor method is used to find K feature neighboring points in the target point cloud, and the graph edges between the center point and the neighboring points are calculated to establish a local feature domain graph structure; the graph feature B of each local feature domain graph structure is extracted through convolution, the graph feature B is aggregated to the center point, and mapped to a high-dimensional space through MLP.

[0052] Embodiment 4

[0053] A point cloud registration method combined with image feature context matching, based on the third embodiment, the accurate feature acquisition is specifically as follows:

[0054] The graph features A and B of the source point cloud are spliced ​​to obtain the complete graph feature C of the source point cloud; the graph features A and B of the target point cloud are spliced ​​to obtain the complete graph feature C of the target point cloud;

[0055] The graph feature C of the source point cloud is used as the encoding input, and the graph feature C of the target point cloud is used as the decoding input. The point cloud graph features of the source point cloud and the target point cloud are context-matched through the Transformer network, and the transformation information from the source point cloud to the target point cloud is learned and embedded in the source point cloud to obtain the accurate features of the source point cloud;

[0056] The graph feature C of the source point cloud is used as the decoding input, and the graph feature C of the target point cloud is used as the encoding input. The point cloud graph features of the target point cloud and the source point cloud are context-matched through the Transformer network, the transformation information from the target point cloud to the source point cloud is learned, and embedded in the source point cloud to obtain the accurate features of the target point cloud.

[0057] Embodiment 5

[0058] A point cloud registration method combined with image feature context matching, based on the fourth embodiment, constructing a feature difference matching matrix according to the source point cloud and the target point cloud containing accurate features, specifically:

[0059] Calculate the Euclidean distance between the source point cloud and the target point cloud containing accurate features, and output it as a feature difference matrix;

[0060] Adding outliers and local optimal restriction parameters to the feature difference matrix, and using the Sinkhorn algorithm to numerically optimize each element of the feature difference matrix;

[0061] Each element of the feature difference matrix is ​​normalized using an exponential function, the corresponding relationship between points is represented by probability, and a feature difference matching matrix is ​​output.

[0062] Embodiment 6

[0063] A point cloud registration method combined with image feature context matching, based on the fifth embodiment, the rigid transformation from the source point cloud to the target point cloud is estimated according to the feature difference matching matrix, specifically:

[0064] The feature difference matching matrix is ​​applied to the target point cloud to obtain a virtual point cloud, the sum of each row of the feature difference matching matrix is ​​used as a weight, and the weighted SVD method is applied to the source point cloud and the virtual point cloud to estimate the rigid transformation from the source point cloud to the target point cloud.

[0065] The accuracy of the registration is evaluated by calculating the isotropic error (IE) of the rotation matrix and the translation respectively. The anisotropic error (AE) is also evaluated for the rotation matrix and the translation, that is, the average absolute error between the predicted value and the true value of the Euclidean angle rotation matrix and the translation vector. The similarity of the two point clouds is evaluated by calculating the chamfer distance (CD) from the source point cloud P to the target point cloud Q. In the environment closest to the real world (part of the point cloud is missing and noise interference is added), the comparison results of each performance index are shown in Table 1, with GCN-Net as the method of the present invention.

[0066] Table 1

[0067]

[0068]

[0069] It can be seen from Table 1 that the method of the present invention has excellent performance in all indicators of point cloud registration.

[0070] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure made using the contents of the present invention's specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A point cloud registration method combining graph feature context matching, characterized in that: The following steps are involved: Extracting geometric domain graph features: extracting geometric domain graph features from the input source point cloud and the target point cloud respectively to obtain graph features A of the source point cloud and the target point cloud; Feature domain graph feature extraction: according to the graph feature A of the source point cloud and the target point cloud, feature domain graph feature extraction is performed on the source point cloud and the target point cloud respectively to obtain the graph feature B of the source point cloud and the target point cloud; Accurate feature acquisition: according to the graph feature A and the graph feature B, a complete graph feature C of the source point cloud and the target point cloud is obtained; feature context matching is performed according to the graph feature C to obtain a transformation feature; according to the transformation feature and the graph feature C, accurate features of the source point cloud and the target point cloud are obtained; Constructing a feature difference matching matrix based on the source point cloud and the target point cloud containing accurate features; estimating a rigid transformation from the source point cloud to the target point cloud according to the feature difference matching matrix; Apply the rigid transformation to the source point cloud, set the number of iterations N, repeat the above steps N times, and obtain a refined rigid transformation; Performing point cloud registration according to the rigid transformation; The geometric domain graph feature extraction is specifically as follows: Set the ball search radius and the number of search points K, and use the ball radius search method to find K neighboring points for each point cloud of the input source point cloud and the target point cloud; According to each of the neighboring points, K graph edges between each local center point and the neighboring points are calculated, and local structure information PPF is calculated, wherein the local structure information PPF includes the angle between the center point normal vector and the graph edge, the angle between the neighboring point normal vector and the graph edge, the angle between the center point normal vector and the neighboring point normal vector, and the edge length; For each local part, a local geometric domain graph structure is established according to the center point coordinates, center point normal vector, neighboring point coordinates, graph edges and local structure information PPF, and the graph features A of each local geometric domain graph structure are extracted through convolution, and the graph features A are aggregated to the center point, and mapped to a high-dimensional space through MLP; The feature extraction of the feature domain graph is specifically as follows: For a point in the source point cloud and the target point cloud, according to the graph feature A of the target point cloud, the K nearest neighbor method is used to find K feature neighboring points in the target point cloud, and the graph edges between the center point and the neighboring points are calculated to establish a local feature domain graph structure; the graph feature B of each local feature domain graph structure is extracted by convolution, the graph feature B is aggregated to the center point, and mapped to a high-dimensional space by MLP; The accurate feature acquisition is specifically as follows: The graph features A and B of the source point cloud are spliced ​​to obtain the complete graph feature C of the source point cloud; the graph features A and B of the target point cloud are spliced ​​to obtain the complete graph feature C of the target point cloud; The graph feature C of the source point cloud is used as the encoding input, and the graph feature C of the target point cloud is used as the decoding input. The point cloud graph features of the source point cloud and the target point cloud are context-matched through the Transformer network, and the transformation information from the source point cloud to the target point cloud is learned and embedded in the source point cloud to obtain the accurate features of the source point cloud; The graph feature C of the source point cloud is used as a decoding input, and the graph feature C of the target point cloud is used as an encoding input. The point cloud graph features of the target point cloud and the source point cloud are context-matched through a Transformer network, and the transformation information from the target point cloud to the source point cloud is learned and embedded into the source point cloud to obtain accurate features of the target point cloud; The constructing of a feature difference matching matrix based on the source point cloud and the target point cloud containing accurate features is specifically as follows: Calculate the Euclidean distance between the source point cloud and the target point cloud containing accurate features, and output it as a feature difference matrix; Adding outliers and local optimal restriction parameters to the feature difference matrix, and using the Sinkhorn algorithm to numerically optimize each element of the feature difference matrix; Using an exponential function to normalize each element of the feature difference matrix, using probability to represent the correspondence between points, and outputting a feature difference matching matrix; The step of estimating the rigid transformation from the source point cloud to the target point cloud according to the feature difference matching matrix is ​​specifically as follows: The feature difference matching matrix is ​​applied to the target point cloud to obtain a virtual point cloud, the sum of each row of the feature difference matching matrix is ​​used as a weight, and the weighted SVD method is applied to the source point cloud and the virtual point cloud to estimate the rigid transformation from the source point cloud to the target point cloud.

Images