Low-overlap point cloud registration method and system

Abstract

The embodiment of the invention discloses a low-overlap point cloud registration method and system. The method comprises the following steps: acquiring an initial reference point cloud and an initial source point cloud; performing feature extraction on the initial reference point cloud and the initial source point cloud to obtain a first feature vector and a second feature vector; calculating a coarse alignment rotation matrix according to the first feature vector and the second feature vector; calculating a first completion point cloud and a second completion point cloud according to the coarse alignment matrix, the first feature vector and the second feature vector; pre-registering the initial reference point cloud and the initial source point cloud according to the initial transformation matrix to obtain a first complete point cloud and a second complete point cloud; optimizing the first completion point cloud and the second completion point cloud according to the first completion point cloud, the second completion point cloud, the first complete point cloud and the second complete point cloud; and according to the optimized first completion point cloud and the second completion point cloud, calculating a final transformation matrix, and according to the final transformation matrix, registering the first completion point cloud and the second completion point cloud and then splicing the first completion point cloud and the second completion point cloud. And the registration accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a low-overlap point cloud registration method and system. Background Art

[0002] 3D point cloud data can be obtained through depth cameras, LiDAR, structured light and other methods. In the process of collecting 3D point cloud data in natural scenes, due to the occlusion of the scanned object at certain angles or the limited field of view of the laser scanning device, a single scan can only obtain point cloud data within a limited field of view. If you want to get a complete point cloud of the target, you need to collect point cloud data from multiple perspectives in the scene, and then use 3D point cloud registration technology to stitch together multiple point clouds. Point cloud registration aims to predict a rigid transformation to align input point clouds of different poses to the same coordinate system. The rigid transformation consists of rotation angle and translation distance transformation.

[0003] Most of the traditional point cloud registration algorithms estimate the rigid transformation parameters based on the correspondence of points, which mainly includes two stages: coarse registration and fine registration. Coarse registration is mainly used to determine the approximate position and posture relationship between point clouds, providing an initial estimate for the subsequent fine registration. The fine registration criterion is further optimized to obtain more accurate transformation parameters, making the point clouds more naturally connected. Although the traditional two-stage point cloud registration algorithm has certain robustness, its registration results will be greatly affected when facing problems such as noise and data missing, and the traditional point cloud registration algorithm cannot handle low-overlap point clouds well. Compared with the traditional two-stage registration method, the deep learning-based point cloud registration method can better capture the details in the point cloud data, and can extract useful features even when affected by noise and outliers. When using deep learning to process point clouds with a small overlapping area, it can be input into the completion model, and the complete point cloud structure can be reconstructed before registration. This method can effectively increase the size of the overlapping area and improve the registration accuracy. However, most of the current point cloud completion models infer the complete point cloud structure based on partially visible point clouds. If the visible part lacks identifiable features, the inferred complete structure is often unclear, which will seriously affect the subsequent registration work.

[0004] There is currently no effective solution to the problem of poor registration accuracy in low-overlap point cloud scenarios in the existing technology. Summary of the invention

[0005] To solve the above problems, the present invention provides a low-overlap point cloud registration method and system, which completes the point cloud registration and splicing by completing the local point clouds with low overlap according to their respective features, and then determines the transformation matrix according to the completed point clouds, so as to solve the problem of poor registration accuracy in the prior art.

[0006] To achieve the above-mentioned purpose, the present invention provides a low-overlap point cloud registration method, including: S1, obtaining an initial reference point cloud and an initial source point cloud; S2, performing feature extraction on the initial reference point cloud according to core point convolution to obtain a first eigenvector; performing feature extraction on the initial source point cloud according to core point convolution to obtain a second eigenvector; calculating a coarse alignment rotation matrix according to the first eigenvector and the second eigenvector; S3, rotating the initial reference point cloud according to the coarse alignment rotation matrix to obtain a rotated reference point cloud, extracting a third eigenvector of the rotated reference point cloud and a fourth eigenvector of the initial source point cloud, and calculating a first completed point cloud according to the third eigenvector and the fourth eigenvector; S4, rotating the initial source point cloud according to the coarse alignment rotation matrix to obtain a rotated reference point cloud, extracting a third eigenvector of the rotated reference point cloud and a fourth eigenvector of the initial source point cloud, and calculating a first completed point cloud according to the third eigenvector and the fourth eigenvector; The rotation matrix is ​​rotated to obtain a rotated source point cloud, the fifth eigenvector of the rotated source point cloud and the sixth eigenvector of the initial reference point cloud are extracted, and the second completed point cloud is calculated according to the fifth eigenvector and the sixth eigenvector; S5, the initial reference point cloud and the initial source point cloud are pre-aligned according to the initial transformation matrix to obtain a first complete point cloud and a second complete point cloud; the first completed point cloud and the second completed point cloud are optimized according to the first completed point cloud, the second completed point cloud, the first complete point cloud and the second complete point cloud; S6, the final transformation matrix is ​​calculated according to the optimized first completed point cloud and the second completed point cloud using a random sampling consistency algorithm, and the first completed point cloud and the second completed point cloud are aligned and spliced ​​according to the final transformation matrix.

[0007] Further optionally, calculating a coarse alignment rotation matrix based on the first eigenvector and the second eigenvector includes: S201, calculating a first fused eigenvector by cascading and element-by-element subtraction of the first eigenvector and the second eigenvector; S202, calculating a coarse alignment rotation matrix based on the first fused eigenvector.

[0008] Further optionally, the calculating the first completed point cloud according to the third eigenvector and the fourth eigenvector includes: S301, performing feature fusion according to the third eigenvector and the fourth eigenvector to obtain a second fused feature vector; S302, calculating the first completed point cloud through the second fused feature vector.

[0009] Further optionally, the optimizing the first completed point cloud and the second completed point cloud according to the first completed point cloud, the second completed point cloud, the first complete point cloud and the second complete point cloud includes: S501, inputting the first completed point cloud, the second completed point cloud, the first complete point cloud and the second complete point cloud into the neural network, calculating the first bulldozer distance loss between the first completed point cloud and the second completed point cloud, the second bulldozer distance loss between the first completed point cloud and the first complete point cloud, and the third bulldozer distance loss between the second completed point cloud and the second complete point cloud; S502, establishing a distance loss function according to the first bulldozer distance loss, the second bulldozer distance loss and the third bulldozer distance loss, and optimizing the first completed point cloud and the second completed point cloud according to the distance loss function until the distance loss function converges or the number of iterations reaches a predetermined value.

[0010] Further optionally, the final transformation matrix is ​​calculated based on the optimized first completed point cloud and the second completed point cloud using a random sampling consistency algorithm, including: S601, randomly selecting at least three non-collinear target points from the first completed point cloud, and determining approximate points in the second completed point cloud based on the target points; S602, calculating the covariance matrix based on the target points and the approximate points, performing singular value decomposition on the covariance matrix, and obtaining the latest transformation matrix; S603, using the latest transformation matrix to transform the first completed point cloud to obtain a transformed completed point cloud, calculating the distance between each point in the transformed completed point cloud and the corresponding point in the second completed point cloud, and taking the points with a distance less than a preset distance threshold as inliers, and when the number of inliers is greater than the number of inliers of the optimal transformation matrix, taking the latest transformation matrix as the optimal transformation matrix; repeating steps S601-S603 until the number of iterations reaches a predetermined number, and taking the current optimal transformation matrix as the final transformation matrix.

[0011] On the other hand, the present invention also provides a low-overlap point cloud registration system, characterized in that it includes: a point cloud acquisition module, used to acquire an initial reference point cloud and an initial source point cloud; a coarse alignment rotation matrix calculation module, used to perform feature extraction on the initial reference point cloud according to core point convolution to obtain a first eigenvector; perform feature extraction on the initial source point cloud according to core point convolution to obtain a second eigenvector; calculate a coarse alignment rotation matrix based on the first eigenvector and the second eigenvector; a first point cloud completion module, used to rotate the initial reference point cloud according to the coarse alignment rotation matrix to obtain a rotated reference point cloud, extract a third eigenvector of the rotated reference point cloud and a fourth eigenvector of the initial source point cloud, and calculate a first completed point cloud based on the third eigenvector and the fourth eigenvector; a second point cloud completion module, Used to rotate the initial source point cloud according to the coarse alignment rotation matrix to obtain the rotated source point cloud, extract the fifth eigenvector of the rotated source point cloud and the sixth eigenvector of the initial reference point cloud, and calculate the second completed point cloud according to the fifth eigenvector and the sixth eigenvector; an optimization module, used to pre-align the initial reference point cloud and the initial source point cloud according to the initial transformation matrix to obtain the first complete point cloud and the second complete point cloud; optimize the first completed point cloud and the second completed point cloud according to the first completed point cloud, the second completed point cloud, the first complete point cloud and the second complete point cloud; a registration module, used to calculate the final transformation matrix according to the optimized first completed point cloud and the second completed point cloud using a random sampling consistency algorithm, and align the first completed point cloud and the second completed point cloud according to the final transformation matrix and then splice them.

[0012] Further optionally, the coarse alignment rotation matrix calculation module includes: a first feature fusion submodule, used to obtain a first fused feature vector by cascading and element-by-element subtraction of the first eigenvector and the second eigenvector; and a matrix conversion submodule, used to calculate the coarse alignment rotation matrix based on the first fused feature vector.

[0013] Further optionally, the first point cloud completion module includes: a second feature fusion submodule, used to perform feature fusion according to the third feature vector and the fourth feature vector to obtain a second fused feature vector; and a completed point cloud determination submodule, used to calculate the first completed point cloud through the second fused feature vector.

[0014] Further optionally, the optimization module includes: a distance calculation submodule, used to input the first completed point cloud, the second completed point cloud, the first complete point cloud and the second complete point cloud into the neural network, calculate the first bulldozer distance loss between the first completed point cloud and the second completed point cloud, the second bulldozer distance loss between the first completed point cloud and the first complete point cloud, and the third bulldozer distance loss between the second completed point cloud and the second complete point cloud; an optimization submodule, used to establish a distance loss function according to the first bulldozer distance loss, the second bulldozer distance loss and the third bulldozer distance loss, and optimize the first completed point cloud and the second completed point cloud according to the distance loss function until the distance loss function converges or the number of iterations reaches a predetermined value.

[0015] Further optionally, the registration module includes: a point selection submodule, which is used to randomly select at least three non-collinear target points from the first completed point cloud, and determine approximate points in the second completed point cloud based on the target points; a solution submodule, which is used to calculate the covariance matrix based on the target points and the approximate points, and perform singular value decomposition on the covariance matrix to obtain the latest transformation matrix; a final transformation matrix determination submodule, which is used to transform the first completed point cloud using the latest transformation matrix to obtain a transformed completed point cloud, calculate the distance between each point in the transformed completed point cloud and the corresponding point in the second completed point cloud, and take the points with distances less than a preset distance threshold as inliers, and when the number of inliers is greater than the number of inliers of the optimal transformation matrix, take the latest transformation matrix as the optimal transformation matrix; repeat the steps of the point selection submodule, the solution submodule and the final transformation matrix determination submodule until the number of iterations reaches a predetermined number, and take the current optimal transformation matrix as the final transformation matrix.

[0016] The above technical solution has the following beneficial effects: using one point cloud as the completion reference target of another point cloud, the two local point clouds use each other as reference, and the two parts of the point cloud targets are visible, avoiding the ambiguity introduced when predicting the invisible part, and improving the accuracy of the registration; constructing a loss function through the difference between the completed point cloud and the complete point cloud, accurately measuring the matching between the point clouds, and improving the accuracy of the completed point cloud after adjustment, thereby improving the accuracy of the registration; calculating an accurate transformation matrix based on the two completed point clouds to improve the accuracy of the registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1is a flow chart of a low overlap point cloud registration method provided by an embodiment of the present invention; Figure 2 is a flow chart of a method for calculating a coarse alignment rotation matrix provided by an embodiment of the present invention; Figure 3 is a flow chart of a first method for completing point cloud computing provided by an embodiment of the present invention; Figure 4 is a flow chart of a method for optimizing a completed point cloud provided by an embodiment of the present invention; Figure 5 is a flow chart of a method for calculating a final transformation matrix provided by an embodiment of the present invention; Figure 6 is a structural schematic diagram of a low-overlap point cloud registration system provided by an embodiment of the present invention; Figure 7 is a schematic diagram of the structure of a coarse alignment rotation matrix calculation module provided in an embodiment of the present invention; Figure 8 is a structural schematic diagram of a first point cloud completion module provided by an embodiment of the present invention; Fig. 9 is a schematic diagram of the structure of an optimization module provided in an embodiment of the present invention; Fig.10 It is a structural diagram of a registration module provided in an embodiment of the present invention.

[0019] Figure markings: 100-point cloud acquisition module; 200-coarse alignment rotation matrix calculation module; 2001-first feature fusion submodule; 2002-matrix conversion submodule; 300-first point cloud completion module; 3001-second feature fusion submodule; 3002-completed point cloud determination submodule; 400-second point cloud completion module; 500-optimization module; 5001-distance calculation submodule; 5002-optimization submodule; 600-registration module; 6001-point selection submodule; 6002-solution submodule; 6003-final transformation matrix determination submodule. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In order to solve the problem of inaccurate point cloud registration in the prior art, an embodiment of the present invention provides a low overlap point cloud registration method. Figure 1 is a flow chart of a low overlap point cloud registration method provided by an embodiment of the present invention. Figure 1As shown, the method includes: S1, obtaining the initial reference point cloud and the initial source point cloud; A point cloud is obtained by scanning with lidar, depth camera or structured light. A point cloud is a collection of points in three-dimensional space. Each point in the point cloud usually contains spatial coordinates and possible additional attributes such as color, normal vector, intensity, etc.

[0022] Initial reference point cloud As the reference point cloud for registration, the initial source point cloud The point clouds that need to be registered to the reference point cloud coordinate system may have different poses due to different acquisition device locations. Multiple point clouds to be registered belong to the same target, which is valid in most point cloud registration scenarios.

[0023] S2, extracting features from the initial reference point cloud according to the core point convolution to obtain a first eigenvector; extracting features from the initial source point cloud according to the core point convolution to obtain a second eigenvector; calculating a coarse alignment rotation matrix according to the first eigenvector and the second eigenvector; The initial reference point cloud and the initial source point cloud are respectively extracted by core point convolution to obtain the first feature vector and the second feature vector. The working principle of core point convolution (KPConv) is as follows: For any point cloud , let the characteristics of all its points be , based on point cloud A key point in is the center of the ball, As the radius, determine a sphere, and the point falling inside the sphere will be regarded as the point Neighborhood points, participate The feature calculation of The neighbor points of are defined as: ; Define the sphere as: ; In the sphere, find Points are called kernel points. Kernel points are not points in the point cloud, but some special positions calculated by specific rules. The core points are defined as follows: ; For each core point , each with a weight matrix Correspondingly, in the sphere range defined above , for points that fall within this range (point in the point cloud), its relative Distance For any , define the kernel function as: ; It can be seen intuitively that Equivalent to Weighted sum of weight matrices. Weight coefficient Defined as: ; It can be seen that each weight matrix The coefficient of Corresponding core points arrive The smaller the relative distance, the greater the weight, and the maximum value is 1; the larger the relative distance, the smaller the weight, and the minimum value is 0. Intuitive explanation: the closer the distance, the greater the correlation and the greater the result; vice versa. Can be seen as: specifically for A weight matrix is ​​calculated. Based on the above, point The core point convolution at is defined as, for each neighbor point Features , respectively, using matrices After transformation, add up: ; Use the above method to calculate the initial source point cloud The corresponding first eigenvector , and the initial reference point cloud The corresponding second eigenvector .

[0024] The first eigenvector With the second eigenvector After fusion processing, a rough alignment matrix is ​​obtained , which is used to perform a rough spatial alignment on the point clouds to be registered in order to enhance the mutual information between point clouds.

[0025] S3, rotating the initial reference point cloud according to the coarse alignment rotation matrix to obtain a rotated reference point cloud, extracting the third eigenvector of the rotated reference point cloud and the fourth eigenvector of the initial source point cloud, and calculating the first complement point cloud according to the third eigenvector and the fourth eigenvector; Since low-overlap point clouds do not have sufficient point-to-point correspondences, the resulting coarse alignment rotation matrix It can only help the point cloud to perform rough spatial alignment, but cannot provide reliable mutual reference information for subsequent completion tasks. Therefore, it is necessary to further improve the mutual information extraction capability by aligning the feature representations between some point clouds in the feature space, thereby optimizing the point cloud completion effect.

[0026] The initial reference point cloud Rotate through the coarse alignment matrix to obtain the rotated reference point cloud , and obtain its third eigenvector through feature extraction , and extract the initial source point cloud The fourth eigenvector of The feature extraction method here can also use the core point convolution feature extraction method. At this time, the initial source point cloud The fourth eigenvector of The first eigenvector above can be used directly , in order to increase the calculation speed.

[0027] Through the third eigenvector With the fourth eigenvector The residual is used to calculate the completion features, and then converted into the first completion point cloud according to the completion features .

[0028] S4, rotating the initial source point cloud according to the coarse alignment rotation matrix to obtain a rotated source point cloud, extracting the fifth eigenvector of the rotated source point cloud and the sixth eigenvector of the initial reference point cloud, and calculating the second complement point cloud according to the fifth eigenvector and the sixth eigenvector; The initial source point cloud Rotate through the coarse alignment matrix to get the rotated source point cloud , and obtain its fifth eigenvector through feature extraction , and extract the initial reference point cloud The sixth eigenvector of The feature extraction method here can also use the core point convolution feature extraction method. At this time, the initial reference point cloud The sixth eigenvector of The second eigenvector above can be used directly , in order to increase the calculation speed.

[0029] Through the fifth eigenvector With the sixth eigenvector The residual is used to calculate the completion features, and then converted into the second completion point cloud according to the completion features .

[0030] S5. Pre-register the initial reference point cloud and the initial source point cloud respectively according to the initial transformation matrix to obtain a first complete point cloud and a second complete point cloud; optimize the first supplementary point cloud and the second supplementary point cloud according to the first supplementary point cloud, the second supplementary point cloud, the first complete point cloud and the second complete point cloud; Initial reference point cloud With the initial source point cloud The ground truth pose between is defined as ,in is the rotation matrix, The rotation matrix and the translation vector together constitute an initial transformation matrix, which can be predetermined according to the parameters of the scanning device.

[0031] The initial reference point cloud is transformed by the initial transformation matrix Relative to the initial source point cloud Perform registration to obtain the first complete point cloud , the registration process is as follows: ; Similarly, the initial source point cloud The second complete point cloud is obtained by merging the matrix transformation and the initial reference point cloud. .

[0032] First complete the point cloud by measuring With the second completed point cloud The difference between the first complete point cloud With the first complete point cloud The difference between the two, and the second completed point cloud With the second complete point cloud The difference between them is used to optimize the first completed point cloud according to the difference. With the second completed point cloud , so that the completion result is as accurate as possible.

[0033] S6. Calculate a final transformation matrix using a random sampling consistency algorithm based on the optimized first complement point cloud and the second complement point cloud, and align and splice the first complement point cloud and the second complement point cloud based on the final transformation matrix.

[0034] Use the optimized first completed point cloud With the second completed point cloud The final transformation matrix, i.e. the rotation matrix, is solved by the random sampling consensus algorithm and translation vectors .

[0035] The second point cloud is completed by the final transformation matrix Perform coordinate system conversion and complete the point cloud with the first Perform stitching to obtain the registration stitching result.

[0036] As an optional implementation, Figure 2 is a flow chart of a method for calculating a coarse alignment rotation matrix provided by an embodiment of the present invention. Figure 2 As shown, the rough alignment rotation matrix is ​​calculated according to the first eigenvector and the second eigenvector, including: S201, obtaining a first fused feature vector by concatenating and element-by-element subtraction of the first feature vector and the second feature vector; The first eigenvector With the second eigenvector Perform concatenation and element-by-element subtraction to merge into the first fused feature vector , which is achieved by: ; in, represents cascade operation, Represents element-wise subtraction operation.

[0037] S202: Calculate a coarse alignment rotation matrix according to the first fused feature vector.

[0038] The first fused feature vector Send it to the multi-layer perceptron (MLP) to generate a rough rotation matrix MLP is a general function approximator with strong fitting ability and rotation matrix representation. The point cloud fusion features passed into MLP contain the global features of the two input point clouds and the difference information between them. This information provides enough context information for MLP. Through a large amount of training data, MLP can learn the mapping relationship from fusion features to rotation matrices, thereby outputting a coarse alignment rotation matrix. .

[0039] As an optional implementation, Figure 3 is a flow chart of a first method for completing point cloud computing provided by an embodiment of the present invention, such as Figure 3 As shown, the first completed point cloud is calculated according to the third eigenvector and the fourth eigenvector, including: S301, performing feature fusion according to the third eigenvector and the fourth eigenvector to obtain a second fused feature vector; By calculating the third eigenvector With the fourth eigenvector The residual is used to fuse a new feature containing complementary information, namely the second fusion feature : ; Similarly, the third fusion feature is obtained by performing the same operation on the fifth and sixth eigenvectors. .

[0040] S302: Calculate a first completed point cloud using the second fused feature vector.

[0041] The second fused feature vector Input to the decoder to generate the first completed point cloud after completion .

[0042] The third fused feature vector Input to the decoder to generate the completed second point cloud .

[0043] As an optional implementation, the decoding process of the decoder may be the inverse process of feature extraction.

[0044] As an optional implementation, Figure 4 is a flow chart of a method for completing point cloud optimization provided by an embodiment of the present invention. Figure 4 As shown, the first complement point cloud and the second complement point cloud are optimized according to the first complement point cloud, the second complement point cloud, the first complete point cloud and the second complete point cloud, including: S501, inputting the first complement point cloud, the second complement point cloud, the first complete point cloud and the second complete point cloud into the neural network, calculating the first bulldozer distance loss between the first complement point cloud and the second complement point cloud, the second bulldozer distance loss between the first complement point cloud and the first complete point cloud, and the third bulldozer distance loss between the second complement point cloud and the second complete point cloud; S502. Establish a distance loss function according to the first bulldozer distance loss, the second bulldozer distance loss and the third bulldozer distance loss, and optimize the first completed point cloud and the second completed point cloud according to the distance loss function until the distance loss function converges or the number of iterations reaches a predetermined value.

[0045] In order to improve the completion quality of the model, it is necessary to optimize the corresponding distance between low-overlap point clouds. In this embodiment, the Earth Mover's Distance (EMD) is used to measure the difference between point clouds. The Earth Mover's Distance (EMD) is a metric used to measure the difference between two point sets (such as point clouds), which is particularly suitable for situations where the distribution is uneven and point-to-point matching cannot be accurate.

[0046] The first and second completed point clouds are optimized through deep learning algorithms and supervised by loss functions.

[0047] The Earthmoving Distance (EMD) loss is calculated by finding a bijective function , maps points in point cloud p to points in point cloud g, and minimizes the average distance between points to measure the difference between point clouds, which is defined as follows: ; Among them, the bijective function The solution can be approximated by the auction algorithm.

[0048] The loss function corresponding to the deep learning neural network consists of two parts: (1) completion loss: completion result and Should be as close as possible to the complete point cloud after registration and Matching; (2) Mutual reference loss: completion result and The points should be matched as much as possible to avoid inconsistencies between multiple results of the same object. The purpose of these two criteria is to improve the mutual information between point clouds and thus improve the completion effect. The distance loss function is defined as: ; In the above formula, is the EMD loss, is the distance loss of the first bulldozer, is the distance loss of the second bulldozer, Distance loss for the third bulldozer.

[0049] The neural network continues to complete the first result With the second completion result The distribution is adjusted until the loss function converges (the function value gradually approaches a fixed value) or the preset number of iterations is reached, and the first completion result and the second completion result at this time are used as the optimized results.

[0050] As an optional implementation, Figure 5 is a flow chart of a method for calculating a final transformation matrix provided by an embodiment of the present invention. Figure 5 As shown, the final transformation matrix is ​​calculated using a random sampling consistency algorithm based on the optimized first complement point cloud and the second complement point cloud, including: S601, randomly selecting at least three non-collinear target points from the first completed point cloud, and determining approximate points in the second completed point cloud according to the target points; S602, calculating a covariance matrix according to the target point and the approximate point, performing singular value decomposition on the covariance matrix, and obtaining a latest transformation matrix; S603. Use the latest transformation matrix to transform the first completed point cloud to obtain a transformed completed point cloud, calculate the distance between each point in the transformed completed point cloud and the corresponding point in the second completed point cloud, and take the points whose distance is less than a preset distance threshold as inliers. When the number of inliers is greater than the number of inliers of the optimal transformation matrix, take the latest transformation matrix as the optimal transformation matrix; repeat steps S601-S603 until the number of iterations reaches a predetermined number, and take the current optimal transformation matrix as the final transformation matrix.

[0051] First, the first completed point cloud after completion Randomly select N (at least 3) non-collinear target points from , and then the second point cloud is completed Find the N approximate points that are most similar to its features , and calculate the centroid of the point cloud based on their coordinates: ; Subtract the centroids of the points in the two point clouds to obtain a decentralized point set, and calculate the covariance matrix H of the centralized point set: ; Next, perform singular value decomposition on the covariance matrix H: ; In the above formula, U and V are orthogonal matrices, is a diagonal matrix. The rotation matrix in the latest transformation matrix and translation vectors It can be calculated by the following formula: ; Then the estimated latest transformation matrix is ​​applied to the first completed point cloud , and calculate the transformed points and the second completed point cloud The distance between corresponding points in the matrix is ​​calculated. If the distance between a point and a corresponding point is less than a preset distance threshold, the point is marked as an inlier. If the number of inliers exceeds the number of inliers of the optimal transformation matrix, the best model is updated, that is, the latest transformation matrix is ​​used as the optimal transformation matrix. The above steps are repeated until the predetermined number of iterations is reached or a sufficiently good model is found, and the final optimal transformation matrix is ​​used as the final transformation matrix, that is, the transformation matrix with the largest number of inliers is used as the final transformation matrix.

[0052] The embodiment of the present invention also provides a low-overlap point cloud registration system. Figure 6 is a schematic diagram of the structure of a low overlap point cloud registration system provided by an embodiment of the present invention. Figure 6 As shown, the system includes: The point cloud acquisition module 100 is used to acquire an initial reference point cloud and an initial source point cloud; A point cloud is obtained by scanning with lidar, depth camera or structured light. A point cloud is a collection of points in three-dimensional space. Each point in the point cloud usually contains spatial coordinates and possible additional attributes such as color, normal vector, intensity, etc.

[0053] Initial reference point cloud As the reference point cloud for registration, the initial source point cloud The point clouds that need to be registered to the reference point cloud coordinate system may have different poses due to different acquisition device locations. Multiple point clouds to be registered belong to the same target, which is valid in most point cloud registration scenarios.

[0054] The coarse alignment rotation matrix calculation module 200 is used to extract features from the initial reference point cloud according to the core point convolution to obtain a first eigenvector; extract features from the initial source point cloud according to the core point convolution to obtain a second eigenvector; and calculate the coarse alignment rotation matrix according to the first eigenvector and the second eigenvector; The initial reference point cloud and the initial source point cloud are respectively extracted by core point convolution to obtain the first feature vector and the second feature vector. The working principle of core point convolution (KPConv) is as follows: For any point cloud , let the characteristics of all its points be , based on point cloud A key point in is the center of the ball, As the radius, determine a sphere, and the point falling inside the sphere will be regarded as the point Neighborhood points, participate The feature calculation of The neighbor points of are defined as: ; Define the sphere as: ; In the sphere, find Points are called kernel points. Kernel points are not points in the point cloud, but some special positions calculated by specific rules. The core points are defined as follows: ; For each core point , each with a weight matrix Correspondingly, in the sphere range defined above , for points that fall within this range (point in the point cloud), its relative Distance For any , define the kernel function as: ; It can be seen intuitively that Equivalent to Weighted sum of weight matrices. Weight coefficient Defined as: ; It can be seen that each weight matrix The coefficient of Corresponding core points arrive The smaller the relative distance, the greater the weight, and the maximum value is 1; the larger the relative distance, the smaller the weight, and the minimum value is 0. Intuitive explanation: the closer the distance, the greater the correlation and the greater the result; vice versa. Can be seen as: specifically for A weight matrix is ​​calculated. Based on the above, point The core point convolution at is defined as, for each neighbor point Features , respectively, using matrices After transformation, add up: ; Use the above method to calculate the initial source point cloud The corresponding first eigenvector , and the initial reference point cloud The corresponding second eigenvector .

[0055] The first eigenvector With the second eigenvector After fusion processing, a rough alignment matrix is ​​obtained , which is used to perform a rough spatial alignment on the point clouds to be registered in order to enhance the mutual information between point clouds.

[0056] The first point cloud completion module 300 is used to rotate the initial reference point cloud according to the coarse alignment rotation matrix to obtain a rotated reference point cloud, extract the third eigenvector of the rotated reference point cloud and the fourth eigenvector of the initial source point cloud, and calculate the first completed point cloud according to the third eigenvector and the fourth eigenvector; Since low-overlap point clouds do not have sufficient point-to-point correspondences, the resulting coarse alignment rotation matrix It can only help the point cloud to perform rough spatial alignment, but cannot provide reliable mutual reference information for subsequent completion tasks. Therefore, it is necessary to further improve the mutual information extraction capability by aligning the feature representations between some point clouds in the feature space, thereby optimizing the point cloud completion effect.

[0057] The initial reference point cloud Rotate through the coarse alignment matrix to obtain the rotated reference point cloud , and obtain its third eigenvector through feature extraction , and extract the initial source point cloud The fourth eigenvector of The feature extraction method here can also use the core point convolution feature extraction method. At this time, the initial source point cloud The fourth eigenvector of The first eigenvector above can be used directly , in order to increase the calculation speed.

[0058] Through the third eigenvector With the fourth eigenvector The residual is used to calculate the completion features, and then converted into the first completion point cloud according to the completion features .

[0059] The second point cloud completion module 400 is used to rotate the initial source point cloud according to the coarse alignment rotation matrix to obtain a rotated source point cloud, extract the fifth eigenvector of the rotated source point cloud and the sixth eigenvector of the initial reference point cloud, and calculate the second completed point cloud according to the fifth eigenvector and the sixth eigenvector; The initial source point cloud Rotate through the coarse alignment matrix to get the rotated source point cloud , and obtain its fifth eigenvector through feature extraction , and extract the initial reference point cloud The sixth eigenvector of The feature extraction method here can also use the core point convolution feature extraction method. At this time, the initial reference point cloud The sixth eigenvector of The second eigenvector above can be used directly , in order to increase the calculation speed.

[0060] Through the fifth eigenvector With the sixth eigenvector The residual is used to calculate the completion features, and then converted into the second completion point cloud according to the completion features .

[0061] The optimization module 500 is used to pre-register the initial reference point cloud and the initial source point cloud according to the initial transformation matrix to obtain the first complete point cloud and the second complete point cloud; optimize the first supplementary point cloud and the second supplementary point cloud according to the first supplementary point cloud, the second supplementary point cloud, the first complete point cloud and the second complete point cloud; Initial reference point cloud With the initial source point cloud The ground truth pose between is defined as ,in is the rotation matrix, The rotation matrix and the translation vector together constitute an initial transformation matrix, which can be predetermined according to the parameters of the scanning device.

[0062] The initial reference point cloud is transformed by the initial transformation matrix Relative to the initial source point cloud Perform registration to obtain the first complete point cloud , the registration process is as follows: ; Similarly, the initial source point cloud The second complete point cloud is obtained by merging the matrix transformation and the initial reference point cloud. .

[0063] First complete the point cloud by measuring With the second completed point cloud The difference between the first complete point cloud With the first complete point cloud The difference between the two, and the second completed point cloud With the second complete point cloud The difference between them is used to optimize the first completed point cloud according to the difference. With the second completed point cloud , so that the completion result is as accurate as possible.

[0064] The registration module 600 is used to calculate the final transformation matrix according to the optimized first complement point cloud and the second complement point cloud using a random sampling consistency algorithm, and to align and splice the first complement point cloud and the second complement point cloud according to the final transformation matrix.

[0065] Use the optimized first completed point cloud With the second completed point cloud The final transformation matrix, i.e. the rotation matrix, is solved by the random sampling consensus algorithm and translation vectors .

[0066] The second point cloud is completed by the final transformation matrix Perform coordinate system conversion and complete the point cloud with the first Perform stitching to obtain the registration stitching result.

[0067] As an optional implementation, Figure 7 is a schematic diagram of the structure of a coarse alignment rotation matrix calculation module provided in an embodiment of the present invention. Figure 7 As shown, the coarse alignment rotation matrix calculation module 200 includes: The first feature fusion submodule 2001 is used to calculate the first fused feature vector by cascading and element-by-element subtraction of the first feature vector and the second feature vector; The first fusion feature vector is obtained by concatenating the first feature vector and the second feature vector and subtracting them element by element; The first eigenvector With the second eigenvector Perform concatenation and element-by-element subtraction to merge into the first fused feature vector , which is achieved by: ; in, represents cascade operation, Represents element-wise subtraction operation.

[0068] The matrix conversion submodule 2002 is used to calculate a coarse alignment rotation matrix according to the first fused feature vector.

[0069] The first fused feature vector Send it to the multi-layer perceptron (MLP) to generate a rough rotation matrix MLP is a general function approximator with strong fitting ability and rotation matrix representation. The point cloud fusion features passed into MLP contain the global features of the two input point clouds and the difference information between them. This information provides enough context information for MLP. Through a large amount of training data, MLP can learn the mapping relationship from fusion features to rotation matrices, thereby outputting a coarse alignment rotation matrix. .

[0070] As an optional implementation, Figure 8 is a schematic diagram of the structure of a first point cloud completion module provided by an embodiment of the present invention, such as Figure 8 As shown, the first point cloud completion module 300 includes: The second feature fusion submodule 3001 is used to perform feature fusion according to the third feature vector and the fourth feature vector to obtain a second fused feature vector; By calculating the third eigenvector With the fourth eigenvector The residual is used to fuse a new feature containing complementary information, namely the second fusion feature : ; Similarly, the third fusion feature is obtained by performing the same operation on the fifth and sixth eigenvectors. .

[0071] The completed point cloud determination submodule 3002 is used to calculate the first completed point cloud through the second fused feature vector.

[0072] The second fused feature vector Input to the decoder to generate the first completed point cloud after completion .

[0073] The third fused feature vector Input to the decoder to generate the completed second point cloud .

[0074] As an optional implementation, the decoding process of the decoder may be the inverse process of feature extraction.

[0075] As an optional implementation, Fig. 9 is a schematic diagram of the structure of the optimization module provided by the embodiment of the present invention, such as Fig. 9 As shown, the optimization module 500 includes: The distance calculation submodule 5001 is used to input the first supplementary point cloud, the second supplementary point cloud, the first complete point cloud and the second complete point cloud into the neural network, and calculate the first bulldozer distance loss between the first supplementary point cloud and the second supplementary point cloud, the second bulldozer distance loss between the first supplementary point cloud and the first complete point cloud, and the third bulldozer distance loss between the second supplementary point cloud and the second complete point cloud; The optimization submodule 5002 is used to establish a distance loss function according to the first bulldozer distance loss, the second bulldozer distance loss and the third bulldozer distance loss, and optimize the first completed point cloud and the second completed point cloud according to the distance loss function until the distance loss function converges or the number of iterations reaches a predetermined value.

[0076] In order to improve the completion quality of the model, it is necessary to optimize the corresponding distance between low-overlap point clouds. In this embodiment, the Earth Mover's Distance (EMD) is used to measure the difference between point clouds. The Earth Mover's Distance (EMD) is a metric used to measure the difference between two point sets (such as point clouds), which is particularly suitable for situations where the distribution is uneven and point-to-point matching cannot be accurate.

[0077] The first and second completed point clouds are optimized through deep learning algorithms and supervised by loss functions.

[0078] The Earthmoving Distance (EMD) loss is calculated by finding a bijective function , maps points in point cloud p to points in point cloud g, and minimizes the average distance between points to measure the difference between point clouds, which is defined as follows: ; Among them, the bijective function The solution can be approximated by the auction algorithm.

[0079] The loss function corresponding to the deep learning neural network consists of two parts: (1) completion loss: completion result and Should be as close as possible to the complete point cloud after registration and Matching; (2) Mutual reference loss: completion result and The points should be matched as much as possible to avoid inconsistencies between multiple results of the same object. The purpose of these two criteria is to improve the mutual information between point clouds and thus improve the completion effect. The distance loss function is defined as: ; In the above formula, is the EMD loss, is the distance loss of the first bulldozer, is the distance loss of the second bulldozer, Distance loss for the third bulldozer.

[0080] The neural network continues to complete the first result With the second completion result The distribution is adjusted until the loss function converges (the function value gradually approaches a fixed value) or the preset number of iterations is reached, and the first completion result and the second completion result at this time are used as the optimized results.

[0081] As an optional implementation, Fig.10 is a schematic diagram of the structure of the registration module provided by an embodiment of the present invention, such as Fig.10 As shown, the registration module 600 includes: The point selection submodule 6001 is used to randomly select at least three non-collinear target points from the first completed point cloud, and determine approximate points in the second completed point cloud according to the target points; A solution submodule 6002 is used to calculate a covariance matrix according to the target point and the approximate point, perform singular value decomposition on the covariance matrix, and obtain a latest transformation matrix; The final transformation matrix determination submodule 6003 is used to use the latest transformation matrix to transform the first completed point cloud to obtain a transformed completed point cloud, calculate the distance between each point in the transformed completed point cloud and the corresponding point in the second completed point cloud, and take the points whose distance is less than a preset distance threshold as internal points. When the number of internal points is greater than the number of internal points of the optimal transformation matrix, the latest transformation matrix is ​​used as the optimal transformation matrix; repeat the steps of the point selection submodule, the solution submodule and the final transformation matrix determination submodule until the number of iterations reaches the predetermined number, and take the current optimal transformation matrix as the final transformation matrix.

[0082] First, the first completed point cloud after completion Randomly select N (at least 3) non-collinear target points from , and then the second point cloud is completed Find the N approximate points that are most similar to its features , and calculate the centroid of the point cloud based on their coordinates: ; Subtract the centroids of the points in the two point clouds to obtain a decentralized point set, and calculate the covariance matrix H of the centralized point set: ; Next, perform singular value decomposition on the covariance matrix H: ; In the above formula, U and V are orthogonal matrices, is a diagonal matrix. The rotation matrix in the latest transformation matrix and translation vectors It can be calculated by the following formula: ; Then the estimated latest transformation matrix is ​​applied to the first completed point cloud , and calculate the transformed points and the second completed point cloud The distance between corresponding points in the matrix is ​​calculated. If the distance between a point and a corresponding point is less than a preset distance threshold, the point is marked as an inlier. If the number of inliers exceeds the number of inliers of the optimal transformation matrix, the best model is updated, that is, the latest transformation matrix is ​​used as the optimal transformation matrix. The above steps are repeated until the predetermined number of iterations is reached or a sufficiently good model is found, and the final optimal transformation matrix is ​​used as the final transformation matrix, that is, the transformation matrix with the largest number of inliers is used as the final transformation matrix.

[0083] The above technical solution has the following beneficial effects: using one point cloud as the completion reference target of another point cloud, the two local point clouds use each other as reference, and the two parts of the point cloud targets are visible, avoiding the ambiguity introduced when predicting the invisible part, and improving the accuracy of the registration; constructing a loss function through the difference between the completed point cloud and the complete point cloud, accurately measuring the matching between the point clouds, and improving the accuracy of the completed point cloud after adjustment, thereby improving the accuracy of the registration; calculating an accurate transformation matrix based on the two completed point clouds to improve the accuracy of the registration.

[0084] The specific implementation methods of the above invention further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above content is only the specific implementation methods of the present invention and is not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A low-overlap point cloud registration method, characterized in that: include: S1, obtaining the initial reference point cloud and the initial source point cloud; S2, extracting features from the initial reference point cloud according to core point convolution to obtain a first feature vector; extracting features from the initial source point cloud according to core point convolution to obtain a second feature vector; and calculating a coarse alignment rotation matrix according to the first feature vector and the second feature vector; S3, rotating the initial reference point cloud according to the coarse alignment rotation matrix to obtain a rotated reference point cloud, extracting a third eigenvector of the rotated reference point cloud and a fourth eigenvector of the initial source point cloud, and calculating a first complement point cloud according to the third eigenvector and the fourth eigenvector; S4, rotating the initial source point cloud according to the coarse alignment rotation matrix to obtain a rotated source point cloud, extracting a fifth eigenvector of the rotated source point cloud and a sixth eigenvector of the initial reference point cloud, and calculating a second complement point cloud according to the fifth eigenvector and the sixth eigenvector; S5, pre-registering the initial reference point cloud and the initial source point cloud respectively according to the initial transformation matrix to obtain a first complete point cloud and a second complete point cloud; optimizing the first completed point cloud and the second completed point cloud according to the first completed point cloud, the second completed point cloud, the first complete point cloud and the second complete point cloud; S6. Calculate a final transformation matrix using a random sampling consistency algorithm based on the optimized first complement point cloud and the second complement point cloud, and align and splice the first complement point cloud and the second complement point cloud based on the final transformation matrix.

2. The low overlap point cloud registration method according to claim 1, characterized in that: The calculating of a coarse alignment rotation matrix according to the first eigenvector and the second eigenvector includes: S201, calculating a first fused feature vector by concatenating and element-by-element subtraction between the first feature vector and the second feature vector; S202: Calculate a coarse alignment rotation matrix according to the first fused feature vector.

3. The low overlap point cloud registration method according to claim 1, characterized in that: The calculating the first completed point cloud according to the third eigenvector and the fourth eigenvector includes: S301, performing feature fusion according to the third feature vector and the fourth feature vector to obtain a second fused feature vector; S302: Calculate a first completed point cloud using the second fused feature vector.

4. The low-overlap point cloud registration method according to claim 1, characterized in that: The optimizing the first completed point cloud and the second completed point cloud according to the first completed point cloud, the second completed point cloud, the first complete point cloud and the second complete point cloud comprises: S501, inputting the first complement point cloud, the second complement point cloud, the first complete point cloud and the second complete point cloud into the neural network, calculating the first bulldozer distance loss between the first complement point cloud and the second complement point cloud, the second bulldozer distance loss between the first complement point cloud and the first complete point cloud, and the third bulldozer distance loss between the second complement point cloud and the second complete point cloud; S502. Establish a distance loss function according to the first bulldozer distance loss, the second bulldozer distance loss and the third bulldozer distance loss, and optimize the first completed point cloud and the second completed point cloud according to the distance loss function until the distance loss function converges or the number of iterations reaches a predetermined value.

5. The low-overlap point cloud registration method according to claim 1, characterized in that: The method of calculating the final transformation matrix using a random sampling consistency algorithm according to the optimized first complement point cloud and the second complement point cloud includes: S601, randomly selecting at least three non-collinear target points from the first completed point cloud, and determining approximate points in the second completed point cloud according to the target points; S602, calculating a covariance matrix according to the target point and the approximate point, performing singular value decomposition on the covariance matrix, and obtaining a latest transformation matrix; S603. Use the latest transformation matrix to transform the first completed point cloud to obtain a transformed completed point cloud, calculate the distance between each point in the transformed completed point cloud and the corresponding point in the second completed point cloud, and use the points whose distance is less than a preset distance threshold as inliers. When the number of inliers is greater than the number of inliers of the optimal transformation matrix, use the latest transformation matrix as the optimal transformation matrix; repeat steps S601-S603 until the number of iterations reaches a predetermined number, and use the current optimal transformation matrix as the final transformation matrix.

6. A low-overlap point cloud registration system, characterized in that: include: Point cloud acquisition module, used to obtain initial reference point cloud and initial source point cloud; A coarse alignment rotation matrix calculation module is used to perform feature extraction on the initial reference point cloud according to core point convolution to obtain a first eigenvector; perform feature extraction on the initial source point cloud according to core point convolution to obtain a second eigenvector; and calculate a coarse alignment rotation matrix according to the first eigenvector and the second eigenvector; A first point cloud completion module is used to rotate the initial reference point cloud according to the coarse alignment rotation matrix to obtain a rotated reference point cloud, extract a third eigenvector of the rotated reference point cloud and a fourth eigenvector of the initial source point cloud, and calculate a first completed point cloud according to the third eigenvector and the fourth eigenvector; A second point cloud completion module is used to rotate the initial source point cloud according to the coarse alignment rotation matrix to obtain a rotated source point cloud, extract the fifth eigenvector of the rotated source point cloud and the sixth eigenvector of the initial reference point cloud, and calculate a second completed point cloud according to the fifth eigenvector and the sixth eigenvector; An optimization module is used to pre-register the initial reference point cloud and the initial source point cloud according to the initial transformation matrix to obtain a first complete point cloud and a second complete point cloud; and optimize the first completed point cloud and the second completed point cloud according to the first completed point cloud, the second completed point cloud, the first complete point cloud and the second complete point cloud; The registration module is used to calculate a final transformation matrix according to the optimized first completed point cloud and the second completed point cloud using a random sampling consistency algorithm, and to align and splice the first completed point cloud and the second completed point cloud according to the final transformation matrix.

7. The low overlap point cloud registration system according to claim 6, characterized in that: The coarse alignment rotation matrix calculation module includes: A first feature fusion submodule, configured to calculate a first fused feature vector by concatenating and element-by-element subtraction between the first feature vector and the second feature vector; The matrix conversion submodule is used to calculate the coarse alignment rotation matrix according to the first fused feature vector.

8. The low overlap point cloud registration system according to claim 6, characterized in that: The first point cloud completion module includes: A second feature fusion submodule is used to perform feature fusion according to the third feature vector and the fourth feature vector to obtain a second fused feature vector; The completed point cloud determination submodule is used to calculate the first completed point cloud through the second fused feature vector.

9. The low overlap point cloud registration system according to claim 6, characterized in that: The optimization module includes: a distance calculation submodule, for inputting the first complement point cloud, the second complement point cloud, the first complete point cloud and the second complete point cloud into the neural network, and calculating a first bulldozer distance loss between the first complement point cloud and the second complement point cloud, a second bulldozer distance loss between the first complement point cloud and the first complete point cloud, and a third bulldozer distance loss between the second complement point cloud and the second complete point cloud; The optimization submodule is used to establish a distance loss function according to the first bulldozer distance loss, the second bulldozer distance loss and the third bulldozer distance loss, and optimize the first completed point cloud and the second completed point cloud according to the distance loss function until the distance loss function converges or the number of iterations reaches a predetermined value.

10. The low overlap point cloud registration system according to claim 6, characterized in that: The registration module comprises: A point selection submodule, configured to randomly select at least three non-collinear target points from the first completed point cloud, and determine approximate points in the second completed point cloud according to the target points; A solution submodule, used for calculating a covariance matrix according to the target point and the approximate point, performing singular value decomposition on the covariance matrix, and obtaining a latest transformation matrix; The final transformation matrix determination submodule is used to use the latest transformation matrix to transform the first completed point cloud to obtain a transformed completed point cloud, calculate the distance between each point in the transformed completed point cloud and the corresponding point in the second completed point cloud, and take the points whose distance is less than a preset distance threshold as internal points. When the number of internal points is greater than the number of internal points of the optimal transformation matrix, the latest transformation matrix is ​​used as the optimal transformation matrix; repeat the steps of the point selection submodule, the solution submodule and the final transformation matrix determination submodule until the number of iterations reaches a predetermined number, and take the current optimal transformation matrix as the final transformation matrix.

Images