Registration method for point clouds of different scales based on matching of line vector pair constraints

Abstract

The application discloses a method for registering heterogeneous point clouds based on line vector pair constraint matching, and relates to the technical field of heterogeneous point cloud registration. The application firstly estimates the scale of the heterogeneous point clouds by using line vectors proposed by TGRS, and converts the registration problem of the heterogeneous point clouds into the registration problem of homogeneous point clouds; and according to the construction characteristics of the line vectors, the line vectors are grouped, so that the time complexity is reduced; secondly, a new undirected graph structure is proposed by taking the corresponding relationship of the line vectors as graph vertices, and the connection between the undirected graph vertices is established by using the proposed triangular constraint; and a statistical graph vertex degree approximation maximum group algorithm is further proposed to find the correct corresponding relationship, so that the case that the iteration time of the maximum group algorithm is too long when the undirected graph is a dense graph is avoided; and for the estimation of rotation and translation, the application proposes to estimate the rotation and translation of each group by using IRLS, then the number of inliers of each group is counted in the same way as RANSAC, and the best model parameter is taken as the result.

Description

Technical Field

[0001] This invention relates to the field of heterogeneous point cloud registration technology, specifically a heterogeneous point cloud registration method based on line vector pair constraint matching. Background Technology

[0002] In recent years, 3D point cloud registration has become a fundamental and crucial step in point cloud processing. Point cloud registration is further divided into same-scale point cloud registration and different-scale point cloud registration. Same-scale point cloud registration mainly involves registering point clouds from different frames acquired using the same device, while different-scale point cloud registration involves registering point clouds from different frames acquired using different devices. However, different-scale point cloud registration requires consideration of the scale differences between different frames, and due to its wide and flexible adaptability, it has received increasing attention in fields such as 3D reconstruction, Simultaneous Localization and Mapping (SLAM), indoor surveying, and ancient building preservation.

[0003] 3D point cloud registration involves transforming point clouds from different viewpoints to the same reference coordinate system to synthesize a 3D panoramic image. However, in practical applications, due to factors such as occlusion, lighting, and distance, the transformation relationship between point clouds from different viewpoints is unknown. Therefore, accurate and rapid 3D point cloud registration is a very difficult problem.

[0004] Iterative Closest Point (ICP) is a popular point cloud registration method. However, because ICP establishes correspondences based on the nearest distance in each transformation, its performance often relies too heavily on the initial poses of the two point cloud frames. Sahillioglu et al. proposed the Scale-Adaptive ICP algorithm, which completes cross-scale point cloud registration by adaptively selecting point-to-point correspondences, exhibiting a certain degree of robustness. Chen et al. used Mahalanobis distance to establish correspondences, which potentially significantly improves registration and calibration, reducing errors to some extent. However, ICP variants often cannot handle point clouds with poor initial poses, such as point clouds reconstructed by multi-view stereo vision (MVS) and point clouds obtained by radar scanning. MVS can provide rich spectral information, while LiDAR can provide accurate geometric information. Combining MVS and LiDAR can reconstruct more realistic and accurate 3D scenes, improving the accuracy of methods such as object detection, classification, and semantic segmentation.

[0005] With the development of 3D descriptors, point cloud registration methods based on correspondence have attracted increasing attention due to their advantages such as not requiring good initial poses and higher computational efficiency. Correspondence-based methods first obtain keypoints through keypoint detection, then perform feature descriptions on these keypoints, next establish initial correspondences based on feature similarity, and finally estimate pose relationships using robust estimation algorithms. Keypoint detection methods can be divided into two categories: manual keypoint detection and learning-based keypoint detection. Manual keypoint detection is obtained by analyzing the structural features of points, such as Internal Shape Description (ISS) and MeshDoG; learning-based keypoint detection is obtained through data training, such as KeypointNet and Unsupervised Stable Interest Points (USIP).

[0006] Keypoint feature descriptions can be categorized into hand-crafted feature descriptions and learning-based feature descriptions. Common hand-crafted feature descriptors include FPFH and SHOT, while common learning-based feature descriptors include FCGF and PointDsc. Feature matching primarily involves obtaining matching scores through feature descriptions and then establishing a one-to-one correspondence based on these scores, such as the nearest neighbor distance ratio strategy.

[0007] After feature matching, an initial correspondence can be obtained. However, due to the noise of the LiDAR itself and the quality of 3D keypoint detection, the outlier rate of the initial correspondence is very high, often reaching 95%. Under such a high outlier rate, commonly used robust estimation algorithms, such as RANSAC and its variants, often require a large amount of random sampling to obtain satisfactory results. However, this greatly reduces the efficiency of model fitting. Therefore, improving the efficiency of model fitting becomes extremely important.

[0008] In same-scale point cloud registration, to improve model fitting efficiency, Parra et al. proposed constructing an undirected graph of the correspondences of matching points and obtaining pairwise consistent correspondences through an efficient maximum clique algorithm, which is more efficient than RASNSAC. In heterogeneous point cloud registration, Yang et al. proposed the TEASER++ algorithm, which decouples the heterogeneous point cloud registration problem into subproblems and uses truncated least squares (TLS) for the estimation of each subproblem.

[0009] LI et al. proposed the TGRS algorithm, which estimates scale information by constructing line vectors and combining them with a single-point RANSAC algorithm. This algorithm estimates scale information without requiring other prior information and is more efficient than other RANSAC variants. However, because it uses the maximum consensus set (MC) method to estimate scale information, the scale accuracy is slightly lower. Fan Yiping et al. treated the scale factor, rotation, and translation parameters as common variables to be solved using the artificial bee colony algorithm. However, when the point cloud is large, the computational cost is high and the running speed is slow. Gentner et al. proposed constructing scale, rotation, and translation as an undirected graph and solving the maximum consensus problem using the maximum clique algorithm of the graph. Peng Shu et al. extracted planes from two heterogeneous point clouds and proposed using the shortest distance of the intersection of the two planes to estimate the scale. However, when there are few planes between the heterogeneous point clouds, the algorithm struggles to estimate the scale. Therefore, a new solution is needed to address these problems. Summary of the Invention

[0010] The purpose of this invention is to provide a method for registration of point clouds at different scales based on pairwise constraint matching of line vectors, so as to solve the technical problems mentioned in the background art.

[0011] To achieve the above objectives, the present invention provides the following technical solution: a method for cross-scale point cloud registration based on line vector pair constraint matching, comprising at least the following steps:

[0012] S1: Input two point clouds, extract feature points and construct feature descriptors for the feature points;

[0013] S2: Construct initial correspondences using feature similarity and kd-tree;

[0014] S3: Construct each feature point and the corresponding line vector of the feature point;

[0015] S4: Calculate scale information using line vectors;

[0016] S5: Group the line vectors according to their construction characteristics;

[0017] S6: Combine the line vector correspondence of each group with triangular constraints to construct an undirected graph. Calculate the degree of each vertex in the undirected graph and filter out the line vectors that meet the requirements.

[0018] S7: The correspondence between each group is estimated by IRLS to determine the rotation and translation, thus obtaining the transformation relationship between the two point clouds;

[0019] S8: Apply the obtained transformation relationship to the original point cloud to complete the registration.

[0020] Preferably, step S4 includes at least the following steps:

[0021] To estimate scale information, construct a line vector from two corresponding points. Then, given a pair of point correspondences (xi, yi) and (xj, yj), construct a set of line vector correspondences (xij = xi - xj, yij = yi - yj).

[0022] Line vectors are unaffected by translation, but are affected by scale s and rotation R, therefore:

[0023] yij=sR(xi-xj)+(ni-nj)=sRxij+nij

[0024] Where nij = ni - nj is vector noise, TGRS analyzes yij = sR(xi - xj) + (ni - nj) = sRxij + nij into a consensus set Is problem, and estimates the scaling factor by maximizing the consensus set. The consensus set Is problem is transformed into a truncated least squares estimation problem:

[0025]

[0026] Where represents the upper limit of noise at a given interior point, β is the noise threshold, and SK = ||Yk|| / ||Xk||.

[0027] Preferably, step S5 includes at least the following steps:

[0028] Each line vector is divided into M groups according to its common endpoint, where M is the preset number of seed points. The selection of seed points is determined by the feature similarity obtained by FPFH matching.

[0029] Preferably, the step S6, which involves constructing an undirected graph by combining the line vector correspondence of each group with triangular constraints, includes at least the following steps:

[0030] By constructing each group as an undirected graph, the correspondences are linked by constraints, where the construction of the graph is defined as follows: Let G = (V, E) represent vertices V = {v...} i} and edge E={(v i ,v j An undirected graph of type} is constructed through multiple definitions;

[0031] After scale estimation, the problem of registering point clouds at different scales is transformed into registering point clouds at the same scale.

[0032] In point cloud registration at the same scale, if there are two correct corresponding points, the Euclidean distance between the two feature points is equal.

[0033] The consistency constraint of corresponding points at the same scale is improved and used in undirected graphs, and corresponding undirected graphs are constructed according to different constraint relationships.

[0034] Preferably, the step S6, which involves calculating the degree of each vertex in the undirected graph and selecting the line vectors that meet the requirements, includes at least the following steps: after constraining the line vectors by grouping them, each group obtains a completely new undirected graph structure, and then the interior points are matched by statistically analyzing the degree of each vertex in the undirected graph.

[0035] Preferably, step S7 includes at least the following steps:

[0036] The filtered line vectors are mapped to feature points, and then iterative re-IRLS is used for rotation and translation estimation.

[0037] Will Redefining:

[0038]

[0039] The weighting function is:

[0040]

[0041] Where u is a scale of the weight function, and r represents the residual value of the corresponding point relative to the estimated model, which is obtained from the weight function. As the residual value is larger, the weight value is smaller.

[0042] In this iterative process, a stepwise optimization strategy is adopted. At the beginning, a large initial value is assigned to the scale u. Then, in each iteration, u = u / α is updated iteratively by step size α to control the weight of each iteration. Finally, the paper assigns a new weight to each corresponding point and uses SVD to solve for R and t until the loop ends.

[0043] The IRLS algorithm is used to obtain the estimate of a single group. In the step of constructing an undirected graph using line vector grouping constraints, the line vectors are divided into M groups. For multiple estimates, the model quality is evaluated using the same interior point counting criterion as RANSAC.

[0044]

[0045] Where [] represents Iverson brackets, and is the interior point threshold. The model with the highest score is selected as the final rigid body transformation result.

[0046] Compared with the prior art, the beneficial effects of the present invention are:

[0047] This invention addresses issues related to algorithm accuracy, high outlier rate, and initialization by designing a heteroscale point cloud registration method with the following characteristics: high efficiency, high robustness, and no initialization. First, the scale of heteroscale point clouds is estimated using line vectors proposed by TGRS, thereby transforming the heteroscale point cloud registration problem into homoscale point cloud registration.

[0048] Furthermore, the line vectors are grouped according to their construction characteristics, which greatly reduces the time complexity. In addition, a new undirected graph structure is proposed based on the correspondence between line vectors as graph vertices, and the connection between undirected graph vertices is established through the proposed triangular constraints.

[0049] It also proposes a method to approximate the maximum clique algorithm by statistical graph vertex degree in order to find the correct correspondence. This avoids the problem that the maximum clique algorithm has too long an iteration time when the undirected graph is dense, and improves the estimation of rotation and translation.

[0050] We propose to estimate the rotation and translation of each group using IRLS, then count the number of interior points in each group in the same way as RANSAC, and use the optimal model parameters as the result.

[0051] The proposed method of pairwise constraints on line vectors only requires grouped line vectors as input, and each group searches for its optimal parameters. This greatly improves the robustness and efficiency of the algorithm. Furthermore, the method does not require a good initial pose like ICP and its variants. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of the method of the present invention;

[0054] Figure 2 This is a schematic diagram of the line vector combination of the present invention;

[0055] Figure 3 This is a schematic diagram illustrating the consistency constraints of registration of corresponding points at the same scale in this invention.

[0056] Figure 4 This is a schematic diagram of vertex constraints in the present invention;

[0057] Figure 5 This is a schematic diagram of vertex degree statistics for the present invention;

[0058] Figure 6 This is a schematic diagram illustrating the visualization of the correspondence between the KITTI dataset in this invention.

[0059] Figure 7 This is a schematic diagram illustrating the hole problem in the MVS point cloud of the present invention;

[0060] Figure 8 This is a schematic diagram for visualizing the MVS results of this invention;

[0061] Figure 9 This is a statistical chart showing the heterogeneous point cloud registration results of the present invention. Detailed Implementation

[0062] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0063] Example 1:

[0064] Please see Figures 1-9 A method for cross-scale point cloud registration based on pairwise constraint matching of line vectors includes at least the following steps:

[0065] S1: Input two point clouds, extract feature points and construct feature descriptors for the feature points;

[0066] S2: Construct initial correspondences using feature similarity and kd-tree;

[0067] S3: Construct each feature point and the corresponding line vector of the feature point;

[0068] S4: Calculate scale information using line vectors;

[0069] S5: Group the line vectors according to their construction characteristics;

[0070] S6: Combine the line vector correspondence of each group with triangular constraints to construct an undirected graph. Calculate the degree of each vertex in the undirected graph and filter out the line vectors that meet the requirements.

[0071] S7: The correspondence between each group is estimated by IRLS to determine the rotation and translation, thus obtaining the transformation relationship between the two point clouds;

[0072] S8: Apply the obtained transformation relationship to the original point cloud to complete the registration.

[0073] Point clouds obtained from different angles by lidar scanning are called source point cloud P and target point cloud Q;

[0074] Point cloud registration is to unify the source point cloud P = {xi|i = 1, 2, ..., N} with the target point cloud Q = {yi|i = 1, 2, ..., M} into the same coordinate system through transformation relations, where xi, yi ∈ R3, and N and M are both represented as the number of points;

[0075] Same-scale point cloud registration only includes two forms of transformation: rotation and translation. However, in the case of different-scale point cloud registration, in addition to rotation and translation transformations, the scale transformation also needs to be determined.

[0076] Therefore, given two noise-free three-dimensional correspondences The heteroscale point cloud transformation is represented as:

[0077] y i =sRx i +t

[0078] In the formula, s∈R 1 , where is the scalar factor for scaling, R∈R 3×3 Given a 3×3 rotation matrix, t∈R 3×1 It is a 3×1 translation vector;

[0079] In the absence of measurement errors and noise, the heterogeneous point cloud registration problem is transformed into the following problem:

[0080]

[0081] Where ρ(.) is a robust estimation function, x i yi and yi are corresponding points;

[0082] Determined by the feature descriptor FPFH;

[0083] The S4 includes at least the following steps:

[0084] To estimate scale information, construct a line vector from two corresponding points. Then, given a pair of point correspondences (xi, yi) and (xj, yj), construct a set of line vector correspondences (xij = xi - xj, yij = yi - yj).

[0085] Line vectors are unaffected by translation, but are affected by scale s and rotation R, therefore:

[0086] yij=sR(xi-xj)+(ni-nj)=sRxij+nij

[0087] Where nij = ni - nj is vector noise, TGRS analyzes yij = sR(xi - xj) + (ni - nj) = sRxij + nij into a consensus set Is problem, and estimates the scaling factor by maximizing the consensus set. The consensus set Is problem is transformed into a truncated least squares estimation problem:

[0088]

[0089] Where represents the upper limit of noise at a given inlier, β is the noise threshold, and SK = ||Yk|| / ||Xk||. This paper estimates the scale s using the Adaptive Voting algorithm. Unlike the TGRS algorithm, the Adaptive Voting algorithm estimates the scale s through... The optimal scale is found by identifying the scale with the minimum cost, while the TGRS algorithm finds the optimal scale by maximizing the consensus set. Experimental results show that simply maximizing the consensus set cannot accurately estimate scale information.

[0090] S5 includes at least the following steps:

[0091] After scaling, the initial line vector correspondence can be obtained, but this line vector correspondence still contains a large number of outliers.

[0092] like Figure 2 As shown, this paper visualizes the line vectors selected in the initial screening, where red represents outlier feature points, green represents correct feature points, and blue is indistinguishable. The line connecting two feature points represents the line vectors obtained through scaling estimation. If there are N×M line vectors, screening such a large number of line vectors is difficult. This paper assumes a known green feature point, then there are a total of N line vectors with it as a common endpoint. Furthermore, these N line vectors can determine whether other endpoints belong to interior points through constraint relationships. Therefore, it is not necessary to retain all line vectors to avoid increasing computational overhead.

[0093] Each line vector is divided into M groups according to its common endpoint, where M is the preset number of seed points. The selection of seed points is determined by the feature similarity obtained by FPFH matching. For example, the top 100 feature points with the highest feature similarity are selected as seed points.

[0094] S6 constructs an undirected graph by combining the line vector correspondence of each group with triangular constraints, which includes at least the following steps:

[0095] By constructing each group as an undirected graph, the correspondences are linked by constraints, where the construction of the graph is defined as follows: Let G = (V, E) represent vertices V = {v...} i} and edge E={(v i ,v j An undirected graph of type} is constructed through multiple definitions;

[0096] Definition 1: If (vi, vj) ∈ E, then a pair of vertices v in G is considered equal to a given pair of vertices v in E. i and v j Adjacent, for each v i ∈V, v i The adjacency representation is:

[0097]

[0098] Definition 2: Given a set of correspondences C, if they satisfy the consistency constraint, then the vertices associated with the correspondences are adjacent, and the edges are represented as follows:

[0099] E = {(ci c j )∈C×C|r(c i c j )≤τ,i≠j}

[0100] Definition 3: For two point clouds X and Y to be registered, given the feature point correspondence M i =(x i ,y i The line vectors formed by the combination of two feature points constitute a correspondence C, and each correspondence C is represented as a vertex V of the graph. The correspondence C is constructed as follows:

[0101] C = {c k =(x i -x j y i -y j ), i≠j, i <N}

[0102] Definition 4: Given two correspondences c k and c k+1 The triangular constraints are given below:

[0103]

[0104] After scale estimation, the problem of registering point clouds at different scales is transformed into registering point clouds at the same scale.

[0105] like Figure 3 This section defines the consistency constraints for corresponding points in point cloud registration at the same scale. The green line represents the correspondence of correct feature points, while the red line represents the correspondence of incorrect feature points. In point cloud registration at the same scale, if two correct corresponding points exist, then the Euclidean distance between the two feature points is equal; that is, the d12 values ​​constructed by the two corresponding points c1 and c2 are approximately equal.

[0106] The consistency constraint of corresponding points at the same scale is improved and used in undirected graphs, and corresponding undirected graphs are constructed according to different constraint relationships.

[0107] For triangular constraints Since each group is a line vector composed of a common endpoint and other feature points.

[0108] like Figure 4 As shown, (x2, y2) are the common endpoints of the line vectors, and (x1, y1) and (x3, y3) are the other feature points of the line vectors. The red and blue solid lines represent the line vectors formed by the combination of the common endpoints and feature points. In rigid registration, the L2 norm of the corresponding line vectors remains unchanged, i.e.:

[0109] ||x1x2||-||y1y2||+||x2x3||-||y2y3||<τ

[0110] Where τ is the vector threshold, since the line vectors of each group are constructed with common endpoints, the constraint ||x1x2||-||y1y2||+||x2x3||-||y2y3||<τ is only to find the corresponding point that satisfies the same scale constraint as the common endpoint, which is a local constraint.

[0111] like Figure 3 As shown, assuming feature point 1 is a common endpoint, and constructing a line vector with feature points 3 and 5, where ||X 15 ||-||Y 15 ||+||X 13 ||-||Y 13 ||< τ ||x 15 ||-||y 15 ||+||x 13 ||-||y 13 ||<τ satisfies the constraint ||x1x2||-||y1y2||+||x2x3||-||y2y3||<τ, but the correspondence of feature point 3 is an incorrect correspondence.

[0112] To address the aforementioned issues, this paper introduces new line vector constraints, specifically by constructing new line vectors using non-common feature points. Figure 4 The green dashed line in the diagram, and the improved constraint is E = {(c i c j )∈C×C|r(c i c j If the common endpoint and the feature point satisfy this triangular constraint, it means that the vertices of the graph formed by the correspondence of the two line vectors are connected (connected).

[0113] The step S6, which involves calculating the degree of each vertex in the undirected graph and selecting the line vectors that meet the requirements, includes at least the following steps: after constraining the line vectors by grouping them, each group obtains a completely new undirected graph structure, and then the interior points are matched by statistically analyzing the degree of each vertex in the undirected graph.

[0114] For undirected graphs, Parra proposed matching correspondences using the maximum clique algorithm. The maximum clique algorithm finds the largest complete subgraph in an undirected graph, and the vertices of this largest complete subgraph satisfy the characteristics of correct correspondences. However, the drawback of the maximum clique algorithm is that when there are closely connected vertices in the graph, the undirected graph formed by the constraints becomes dense, making the maximum clique algorithm difficult to perform.

[0115] like Figure 5As shown, the left side is the constructed undirected graph, and the right side is a statistical graph of the vertex degrees of the undirected graph. In the subgraph on the left, vertices v3, v4, v5, and v6 form a maximum clique, constituting the maximum clique of the undirected graph. The vertex degree statistics on the right show that the vertices in the maximum clique have relatively high degrees. The method proposed in this paper is more stable than the maximum clique algorithm, and its time complexity is O(N) for both sparse and dense graphs. 2 ), where N is the number of vertices in the graph. However, using the maximum clique algorithm requires not only constructing the graph structure but also finding the maximum clique in the undirected graph, which is often difficult to estimate. The pseudocode for constructing the graph structure and filtering outlier line vectors using line vectors is shown in Algorithm 1;

[0116]

[0117] The S7 includes at least the following steps:

[0118] While filtering outlier line vectors by matching graph vertices can filter out a large number of outliers, it still cannot completely eliminate a very small number of them. Traditional Singular Value Decomposition (SVD) for solving rotation and translation is affected by outliers. By mapping the filtered line vectors to feature points and then using iterative IRLS for rotation and translation estimation, the IRLS algorithm assigns smaller weights to points with larger residual values ​​with each iteration, thus reducing the impact of outliers on rotation and translation estimation.

[0119] Will Redefining:

[0120]

[0121] The weighting function is:

[0122]

[0123] Where u is a scale of the weighting function, and r represents the residual value of the corresponding point relative to the estimated model, which is obtained from the weighting function. As the residual value is larger, the weight value is smaller, which can avoid the inaccuracy of parameter estimation due to the influence of outliers.

[0124] In this iterative process, a stepwise optimization strategy is adopted. At the beginning, a large initial value is assigned to the scale u, such as u = 1000. Then, in each iteration, u = u / α is updated iteratively by step size α to control the weight of each iteration. Finally, new weights are assigned to each corresponding point and SVD is used to solve R and t until the loop ends. Its pseudocode is shown in Algorithm 2.

[0125]

[0126] The IRLS algorithm is used to obtain the estimate of a single group. In the step of constructing an undirected graph using line vector grouping constraints, the line vectors are divided into M groups. For multiple estimates, the model quality is evaluated using the same interior point counting criterion as RANSAC.

[0127]

[0128] Where [] represents Iverson brackets, τ is the interior point threshold, and the model with the highest score is selected as the final rigid body transformation result;

[0129] Example 2:

[0130] This embodiment is used to further disclose the corresponding experimental methods based on the above embodiments;

[0131] This section provides a comprehensive evaluation of the proposed method through experiments. First, the method is evaluated on homologous datasets, all with a scale of 1 and good initial poses. Second, the method is evaluated on heterologous point clouds, namely MVS point clouds and structured light point clouds, where the scale is unknown and the initial poses are poor. Finally, the proposed method is compared with state-of-the-art methods, including TGRS and TEASER++ algorithms.

[0132] Evaluation metrics and experimental environment:

[0133] This paper uses a commonly used indicator for evaluation, namely the scaling error E. S Rotational error E R Translation error E t .

[0134] E S =|S t -S e |

[0135]

[0136] E t =||t t -t e

[0137] Where the superscripts t and e represent the true value and the estimated value, respectively, and tr(.) is the trace of the matrix. To intuitively demonstrate the success rate of the method, this paper statistically analyzes the success rate of point clouds under certain criteria:

[0138]

[0139] Where R success The number of point cloud groups successfully registered, R alllThe total number of groups in the dataset. The algorithm proposed in this paper is developed and implemented based on MATLAB, and runs on Ubuntu 20.04, with an Intel i5-10200h CPU and 16GB of memory.

[0140] Homogeneous point cloud registration

[0141] The experimental dataset for source point clouds consists of keypoints and FPFP features from the KITTI scene provided by FCGF. This paper selects the test set for experiments, which comes from scenes 8 to 10. Each data set is at least 10 meters apart and represents the first scan. There are a total of 554 data sets. Each scan is assigned an approximate ground truth (Rt, tt) rigid transformation with a scale of 1. The ground truth values ​​for each data set are first obtained from the initial pose using GPS positioning and then refined and aligned using the ICP algorithm.

[0142] The experimental dataset in this paper was obtained by randomly selecting one set every 10 sets from the test set, for a total of 55 sets. The scaling, rotation, and translation of each set were evaluated. In the homologous point cloud registration experiment, the feature correspondence was used as input, and the average of the 55 sets was taken as the result. The runtime is the time from inputting the feature correspondence to outputting the result.

[0143] like Figure 6 As shown, this paper maps the line vectors in each step to feature points and visualizes the correspondence of these feature points. It clearly shows the filtering of outlier correspondences in the KITTI dataset at each stage. For the KITTI dataset in the figure, it can be seen that there are outliers in over 90% of the initial correspondences, but the proposed method is still effective. Scale estimation filters out most outliers, but approximately 62.25% still remain. Pairwise constraint matching yields sufficiently accurate correspondences. The yellow box clearly shows that the correspondences are mostly correct, but inevitably, a very small number of outlier correspondences still exist. Therefore, this paper uses IRLS, which is insensitive to outliers, for translation and rotation estimation. This paper summarizes the mean and variance of the experimental results for all datasets, and the detailed experimental results are shown in Table 1.

[0144] Table 1. Statistical Table of KITTI Experiment Results

[0145]

[0146] As shown in Table 1, the TGRS algorithm achieved good results in rotation estimation, but the translation error was as high as 1.099m, and the average running time was 57.61 seconds. The TEASER++ algorithm had poor rotation accuracy, but better translation accuracy than TGRS. This may be because truncated least squares can achieve better results in registration with good initial pose and low outlier points.

[0147] In summary, regarding algorithm efficiency, the proposed algorithm improves rotation accuracy by 10.9%, translation accuracy by 92.5%, and scale accuracy by 89% compared to the TGRS algorithm, while also achieving a 14-fold increase in running speed. Compared to the TEASER++ algorithm, it improves rotation accuracy by 62.64% and translation accuracy by 33.14%. Although its running speed is lower than the TEASER++ algorithm, it achieves higher accuracy. Overall, the algorithm improves scale accuracy by an average of 31.32%, rotation accuracy by an average of 36.77%, and translation accuracy by an average of 62.82%, with a smaller variance.

[0148] Heterogeneous point cloud registration

[0149] To verify the effectiveness of the proposed algorithm in solving the heterogeneous point cloud registration problem, this paper selects 10 typical scenes from the MVS dataset as the evaluation dataset. Each scene in the MVS dataset consists of a structured light point cloud, an image sequence, and a calibration file. This paper uses visualSFM software to generate MVS point clouds for each image sequence. For each scene, we manually select keypoints for initial alignment using CloudCompare software, and then use its Fine registration algorithm with an overlap rate of 40% to perform refined registration and obtain accurate pose transformation.

[0150] In the heterogeneous point cloud dataset, the average number of points in the structured light point cloud is 3,807,481, and the average number of points in the MVS point cloud is 164,452. To reduce the size of the point cloud, this paper first uses a voxel filtering algorithm with voxel side lengths of 0.01m (MVS point cloud) and 0.2m (structured light point cloud) for downsampling. Then, the ISS method is used to determine key points, and the number of key points is limited to 8,000 to 12,000. Next, FPFH is used to describe the features of the key points. Finally, kd-tree is used for feature matching. The average number of matching pairs for all scenes is 1,553.

[0151] This paper selects different scenarios for comparison. Although the overlap of these point clouds is large, the initial poses are relatively poor, and the correspondence obtained through feature matching has an extremely high outlier rate. For example... Figure 8As shown, the point cloud generated by MVS has many holes, which leads to a very high number of outlier correspondences. This is mainly because many key points in the structured light point cloud do not have corresponding points in the MVS point cloud. Secondly, there are local geometric deformations in the MVS point cloud. Therefore, the experiment of the algorithm on this heterogeneous point cloud dataset is extremely challenging.

[0152] This paper selects three different scenarios for visualization evaluation, each with a different structure. Figure 9 The initial pose, ground truth (GT), and comparative effects of different algorithms were visualized, with the cyan point cloud representing the source point cloud and the red point cloud representing the target point cloud. It is evident that the initial poses of the two point clouds in each scene are poor. While both TEASER++ and TGRS algorithms achieve some success with heterogeneous point clouds, the proposed method achieves better results in detail compared to other algorithms. However, due to the hole problem in the MVS point cloud and the fact that the estimated scale is only an approximation, slight ghosting exists after registration of the two point clouds.

[0153] Detailed registration results of heterogeneous point cloud datasets are as follows: Figure 9 As shown in the figure, our algorithm outperforms other algorithms in most scenarios. For example, in scenario 8, the TGRS algorithm suffers from a rotation error of 154° and a translation error of 18.01m due to inaccurate scale estimation. The TEASER++ algorithm has rotation and translation errors of 0.737° and 0.11m, respectively. In contrast, our method has rotation and translation errors of 0.356° and 0.0604m, respectively. Detailed results for heterogeneous point cloud registration are shown in Table 2. Our method improves the scale accuracy by an average of 31.4%, the rotation accuracy by an average of 87%, and the translation accuracy by an average of 81.54%, and the variance of our method is lower than that of other algorithms. The success rate of heterogeneous point cloud registration is shown in Table 3. Under strict standards (E... S <10 -2 E R <2°,E t The success rate for the <0.5m) algorithm is 100%, while the success rates for the TGRS and TEASER++ algorithms are 70%.

[0154] Table 2. Detailed information on heterogeneous point cloud registration.

[0155]

[0156]

[0157] Table 3. Success Rate of Heterogeneous Point Cloud Registration

[0158]

[0159] This paper proposes a heterogeneous point cloud registration method based on line vectors. It employs keypoint detection and feature descriptor matching, then uses line vectors to estimate the scale and filter out outlier line vectors. Further refinement of correct correspondences is achieved through a proposed grouping strategy and the construction of an undirected graph. Compared to directly using the maximum clique algorithm to estimate correct points, this method offers advantages in both time efficiency and robustness. To further enhance robustness, IRLS, which is insensitive to outliers, is proposed for estimating rotation and translation. Registration experiments were conducted on both homogeneous and heterogeneous point clouds. In homogeneous point cloud registration, the proposed method achieved satisfactory results, improving rotation accuracy by 36.77% and translation accuracy by 62.82% compared to other methods. In heterogeneous point cloud registration, the hole problem in MVS point clouds resulted in lower accuracy compared to homogeneous point cloud registration; however, compared to other methods, this method still achieved an 87% improvement in rotation accuracy and an 81.54% improvement in translation accuracy.

[0160] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A method for cross-scale point cloud registration based on pairwise constraint matching of line vectors, characterized in that: At least the following steps are included: S1: Input two point clouds, extract feature points and construct feature descriptors for the feature points; S2: Construct initial correspondences using feature similarity and kd-tree; S3: Construct each feature point and the corresponding line vector of the feature point; S4: Calculate scale information using line vectors; S5: Group the line vectors according to their construction characteristics; S6: Combine the line vector correspondence of each group with triangular constraints to construct an undirected graph. Calculate the degree of each vertex in the undirected graph and filter out the line vectors that meet the requirements. The step S6, which combines the line vector correspondence of each group with triangular constraints to construct an undirected graph, includes at least the following steps: By constructing each group as an undirected graph, the correspondences are linked by constraints, where the construction of the graph is defined as follows: Let G=(V,E) represent vertices V={v...} i And edge E = {(v i ,v j An undirected graph of type} is constructed through multiple definitions; After scale estimation, the problem of registering point clouds at different scales is transformed into registering point clouds at the same scale. In point cloud registration at the same scale, if there are two correct corresponding points, the Euclidean distance between the two feature points is equal. The consistency constraint of corresponding points at the same scale is improved and used in undirected graphs, and corresponding undirected graphs are constructed according to different constraint relationships. The step S6, which involves calculating the degree of each vertex in an undirected graph and selecting the line vectors that meet the requirements, includes at least the following steps: after constraining the line vectors by grouping them, each group obtains a completely new undirected graph structure, and then the interior points are matched by counting the degree of each vertex in the undirected graph. S7: The correspondence between each group is estimated by IRLS to determine the rotation and translation, thus obtaining the transformation relationship between the two point clouds; S8: Apply the obtained transformation relationship to the original point cloud to complete the registration.

2. The method for cross-scale point cloud registration based on pairwise constraint matching of line vectors according to claim 1, characterized in that: The S4 includes at least the following steps: The scale information is estimated by constructing a line vector from two corresponding points, and then given a pair of point correspondences. and Construct a set of corresponding line vectors ; Line vectors are unaffected by translation, but are affected by scale s and rotation R, therefore: in For vector noise, TGRS will The problem can be parsed into a consensus set Is problem, and the scaling factor can be estimated by maximizing the consensus set. The consensus set Is problem is then transformed into a truncation least squares estimation problem. in, This represents the upper limit of the noise level at a given interior point. Noise threshold .

3. The method for cross-scale point cloud registration based on line vector pair constraint matching according to claim 1, characterized in that: The S5 includes at least the following steps: Each line vector is divided into M groups according to its common endpoint, where M is the preset number of seed points. The selection of seed points is determined by the feature similarity obtained by FPFH matching.

Images