Alignment method for partially overlapping point clouds based on double coincidence

Abstract

This provides a method for aligning partially overlapping point clouds based on double coincidence. [Solution] The method involves acquiring the source and target point clouds of an entity object, generating a matching point set using a low-level matching method, then filtering and sampling the matching tetrahedron set using a high-level matching method, and finally obtaining the optimal transformation matrix by progressing from coarse estimation to fine estimation. During alignment, the optimal transformation matrix is ​​used to convert the source point cloud of the target entity object into a target point cloud, completing the alignment of partially overlapping point clouds.

Description

[Technical Field]

[0001] The present invention relates to a method for aligning point clouds, and more specifically to the technical field of point cloud alignment, and more specifically to a method for aligning partially overlapping point clouds based on double coincidence. [Background technology]

[0002] Point cloud alignment is a crucial direction in the field of point cloud research, aiming to integrate point clouds obtained from sensors (e.g., LiDAR, depth cameras, etc.) from different viewpoints and locations into a single coordinate system. This process involves two point clouds: a source point cloud and a target point cloud. The target point cloud remains unchanged, while the source point cloud is aligned to the target point cloud via attitude transformation. Specifically, the core of point cloud alignment involves determining a transformation matrix from the source point cloud to the target point cloud and applying this matrix to achieve precise alignment of the two. Aligning partially overlapping point clouds presents a challenge in the alignment process due to the fact that only a partial region overlaps between the source and target point clouds.

[0003] Conventional point cloud alignment methods include iterative nearest neighbor (ICP), random sample consensus (RANSAC), and normal distribution transformation (NDT). These methods establish correspondences between point clouds and estimate transformation matrices based on the similarity of the geometric structure or other features between the source and target point clouds. However, when dealing with partially overlapping point clouds, these methods tend to fall into local optima, often resulting in inaccurate alignment results. This is mainly because conventional methods have limited ability to identify correspondence matching relationships when dealing with the alignment of complex partially overlapping point clouds.

[0004] Deep learning-based point cloud alignment methods have been introduced for the alignment of partially overlapping point clouds with the rise of deep learning. These methods achieve end-to-end alignment by automatically searching for correspondences between source and target point clouds using deep learning models, or by directly predicting transformation matrices. However, deep learning-based methods also face significant challenges. First, they rely on large-scale training data, requiring considerable time and resources to build the dataset. Second, the training process is lengthy, making them unsuitable for applications requiring real-time performance. Finally, the generalization ability of deep learning models is limited, and retraining of the model may be necessary for point cloud data from different sources (e.g., LiDAR, depth cameras), increasing application costs. [Overview of the project]

[0005] To address the challenges in the background technology, the present invention provides a method for aligning partially overlapping point clouds based on double coincidence.

[0006] The technical solutions employed by this invention are as follows: The present invention provides a method for aligning partially overlapping point clouds based on double coincidence, performed by a computer under software control, and includes the following steps: Step 1): The sensor acquires the source and target point clouds of the current object, with each source point in the source point cloud corresponding to multiple target points in the target point cloud. A matching point set is generated between the source and target point clouds using a low-level matching method. Step 2): A high-level matching method is used to filter and sample the matching tetrahedron set between the source and target point clouds within the matching point set. Step 3): Based on the matching tetrahedron set, obtain the optimal transformation matrix between the source and target point clouds of the current entity object using coarse-to-fine estimation. Step 4): When aligning the target entity object, the source point cloud of the target entity object is obtained, the source point cloud is converted to the target point cloud via the optimal transformation matrix, and the alignment of the partially overlapping point cloud is completed by displaying it on the screen.

[0007] In Step 1), the specific procedure for generating a matching point set between the source point cloud and the target point cloud using a low-level matching method is as follows: Step 1.1): Obtain the feature descriptors for each point in the source and target point clouds of the current entity object, and normalize each feature descriptor using the L2-norm normalization method. Step 1.2): For the normalized feature descriptors of each point pair in the source and target point clouds, obtain the feature consistency cost matrix C for the current pair of feature descriptors, and use the Sink-horn algorithm to obtain the assignment matrix A from the cost matrix. Step 1.3): For each row of the assignment matrix A, select the index of the k largest element using the index selection top-K method, extract multiple source points and multiple target points corresponding to each source point, and construct the initial matching point set. Step 1.4): Filter the initial set of matching points using geometric agreement based on curvature and distance, and construct the set of matching points. In step 1.4), for each source point and its target point in the initial matching point set, the curvature error between the source point and the target point is the curvature threshold C. th If less than the distance threshold D, the current source and target points are retained. For each retained source point and its respective target point, the distance error between them is considered to be the distance threshold D. th Two or more target points less than a certain value are clustered into a single target point, the source point and one or more target points after clustering are obtained to form a pair of matching points, and each pair of matching points is constructed as a matching point set.

[0008] The specific steps for Step 2) are as follows: Step 2.1): Based on each source point and target point in the matching point set, obtain a plurality of source point group lines and target point group lines, and construct a matching line set after re-matching. Step 2.2): Based on each source point group line and target point group line in the matching line set, obtain a plurality of source point group tetrahedrons and target point group tetrahedrons, and construct a matching tetrahedron set after filtering and sampling.

[0009] JPEG0007876920000002.jpg100165

[0010] JPEG0007876920000003.jpg177164

[0011] The specific content of the foregoing step 3) is as follows. Step 3.1): In the rough estimation stage, for each pair of matched source point group tetrahedrons and target point group tetrahedrons in the matching tetrahedron set, use the singular value decomposition (SVD: Singular Value Decomposition) algorithm to obtain an initial transformation matrix T between the source point group tetrahedron and the target point group tetrahedron. Specifically, it is as follows.

[0012]

Number

[0013] Here, R and t respectively represent the rotation matrix and the translation vector of the initial transformation matrix T. Step 3.2): For each initial transformation matrix T, convert the rotation matrix R to Euler angles Ea, and combine the translation vector t and the Euler angles Ea into a 6-dimensional pose vector. Map all 6-dimensional pose vectors to a 6-dimensional space, sparsify each 6-dimensional pose vector through each 6-dimensional block in the 6-dimensional space, and obtain a set of re-mapping transformation matrices. Step 3.3): Based on the set of matching points in Step 1), the remapping transformation matrix with the maximum number of matching point pairs within the set of remapping transformation matrices is taken as the optimal rough transformation matrix T c and used. Step 3.4): In the fine estimation stage, based on the set of matching points in Step 1), the optimal rough transformation matrix T c is used to resample the set of matching points to obtain the fine transformation matrix T f . Finally, the optimal transformation matrix T o is obtained, and T o = T f ×T c .

[0014] In Step 3.2), the 6D space is divided into multiple 6D blocks according to a preset leaf node size, and each 6D block contains multiple 6D pose vectors. During sparsification, for each 6D block, the center of the 6D block is used as the feature representation of all the 6D pose vectors within that block and remapped as the remapping transformation matrix. Each remapping transformation matrix is constructed as a set of remapping transformation matrices. In Step 3.3), for each source point in the set of matching points in Step 1) and its remapping transformation matrix, the source point is transformed using the remapping transformation matrix to obtain multiple transformed source points. The distance between each transformed source point and each target point is calculated, and the number of transformed source points whose distance from the target point is less than the fifth distance threshold τ is taken as the number of matches. The remapping transformation matrix with the maximum number of matches is taken as the optimal rough transformation matrix T c and used. In Step 3.4) described above, for each source point in the set of matching points in Step 1), the source point is transformed using the optimal rough transformation matrix T c to obtain multiple roughly transformed source points. For each roughly transformed source point, the distance between the roughly transformed source point and the target point is calculated, and the distance from the target point is a preset sixth distance threshold d 6thRetain the rough conversion source points that are less than. Construct the retained rough conversion source points and their corresponding target points as the final matching point set, and apply the singular value decomposition (SVD) algorithm to the final matching point set to obtain the precise transformation matrix T f to obtain.

[0015] The electronic device of the present invention includes a memory and a processor that are coupled to each other. The memory stores program data, and the processor calls the program data to execute the method described above. The readable storage medium of the present invention stores program data, and when the program data is executed by a processor, the method described above is realized.

[0016] The double consistency of the present invention is divided into low-level consistency and high-level consistency. Low-level consistency refers to the feature consistency of points within a point cloud and the consistency of curvature and distance. By using low-level consistency, a robust matching point set can be generated. High-level consistency refers to the geometric structure consistency of lines and tetrahedrons within a point cloud. With high-level consistency, filtering and sampling of the matching line set and the matching tetrahedron set are possible. For the matching tetrahedron set, the present invention proceeds from a rough estimation to a fine estimation to obtain an optimal transformation matrix, and realizes high-precision alignment of partially overlapping point clouds.

[0017] The beneficial effects of the present invention are as follows. The present invention addresses the drawback that non-overlapping regions interfere with the selection of corresponding matching relationships in the alignment process in partially overlapping point clouds by employing a method of generating matching points based on low-level agreement, sequentially selecting and sampling matching lines and matching tetrahedrons based on high-level agreement, and obtaining an optimal transformation matrix through rough-to-refined estimation. Low-level matching agreement is designed based on feature agreement and point curvature / distance agreement, while high-level matching agreement is designed based on geometric structure analysis of matching lines and matching tetrahedrons. Through agreement determination from low to high levels, the correct matching relationships located in the overlapping regions of the source and target point clouds can be filtered and sampled to the maximum extent, and interference from non-overlapping regions can be avoided. By employing double agreement, the problem of falling into local optima in the alignment process of partially overlapping point clouds, which reduces the alignment effect, is overcome. [Brief explanation of the drawing]

[0018] [Figure 1] Figure 1 is a schematic diagram showing the flow of the method of the present invention. [Figure 2] Figure 2 is a schematic diagram showing a quaternary set of source and target point cloud lines for matching lines in the present invention. [Figure 3] Figure 3 is a schematic diagram showing a quintuple set of source point cloud tetrahedrons and target point cloud tetrahedrons in the matching tetrahedron of the present invention. [Figure 4] Figure 4 is a schematic diagram showing the source point cloud and target point cloud in the initial orientation state in an embodiment of the present invention. [Figure 5] Figure 5 is a schematic diagram showing the corresponding points of the matching point set in an embodiment of the present invention. [Figure 6] Figure 6 is a schematic diagram showing the corresponding points of the matching line set in an embodiment of the present invention. [Figure 7] Figure 7 is a schematic diagram showing the corresponding points of the matching tetrahedron set in an embodiment of the present invention. [Figure 8]Figure 8 is a schematic diagram showing the source point cloud and target point cloud that have been aligned in an embodiment of the present invention. [Modes for carrying out the invention]

[0019] To better understand the technical solutions of the present invention, the present invention will be described in more detail below with reference to the drawings and specific embodiments.

[0020] JPEG0007876920000005.jpg148164

[0021]

number

[0022] JPEG0007876920000007.jpg87164

[0023] JPEG0007876920000008.jpg185165

[0024]

number

[0025] JPEG0007876920000010.jpg23164

[0026]

number

[0027] JPEG0007876920000012.jpg228163

[0028] Step 3) obtains the optimal transformation matrix between the source and target point clouds of the current entity object using coarse-to-fine estimation based on the matching tetrahedron set. Details are as follows: Step 3.1) is performed in the rough estimation stage, where the initial transformation matrix T between the source and target point cloud tetrahedra is obtained for each pair of matched source and target point cloud tetrahedra in the matching tetrahedron set using the Singular Value Decomposition (SVD) algorithm. Specifically, this is done as follows:

[0029]

number

[0030] Here, R and t represent the rotation matrix and translation vector of the initial transformation matrix T, respectively. Step 3.2) For each initial transformation matrix T, the rotation matrix R is transformed into Euler angles Ea. Euler angles Ea consist of three angles {α,β,γ}, representing the roll angle, pitch angle, and yaw angle, respectively. Next, the translation vector t and Euler angles Ea are combined to construct a 6-dimensional attitude vector {x,y,z,α,β,γ}, where {x,y,z} represent the translation in the x, y, and z directions, respectively. All 6-dimensional attitude vectors are mapped to a 6-dimensional space, and each 6-dimensional attitude vector is sparsified through each 6-dimensional block in the 6-dimensional space to obtain a set of remapping transformation matrices. This 6-dimensional space is divided into 6-dimensional blocks with a leaf node size of 0.01m, and each 6-dimensional block contains multiple 6-dimensional attitude vectors. During sparsification, for each 6-dimensional block, the center of the 6-dimensional block is used as the feature representation of all 6-dimensional attitude vectors within that block and is remapped as a remapping transformation matrix. Each remapping transformation matrix is ​​constructed as a set of remapping transformation matrices. Step 3.3) is the optimal coarse transformation matrix T, which is the remapping transformation matrix with the largest matching point pair in the set of remapping transformation matrices based on the matching point set in Step 1). cSpecifically, for each source point in the matching point set from step 1) and its remapping transformation matrix, the source point is transformed using the remapping transformation matrix to obtain multiple transformed source points. The distance between each transformed source point and each target point is calculated, and the number of transformed source points whose distance to the target point is less than the fifth distance threshold τ (τ=0.05m) is defined as the number of matches. The remapping transformation matrix that maximizes the number of matches is defined as the optimal coarse transformation matrix T. c Let's assume that the coarse transformation matrix T has the most internal points. c We will adopt this as the result of our rough estimation. Step 3) is performed, and in the precise estimation stage, the optimal coarse transformation matrix T is determined based on the matching point set in Step 1). c The matching point set is resampled using the precise transformation matrix T f Obtain the optimal transformation matrix T. o Obtain T o =T f ×T c Specifically, for each source point in the matching point set of step 1), the optimal coarse transformation matrix T c The source points are transformed using [a specific method] to obtain multiple coarse-transformed source points. For each coarse-transformed source point, the distance between the coarse-transformed source point and the target point is calculated, and the distance to the target point is set to a predetermined sixth distance threshold d. 6th (d 6th The coarse transformation source points (less than 0.05m) are retained. Each retained coarse transformation source point and its corresponding target point are constructed as the final matching point set, and the singular value decomposition (SVD) algorithm is applied to the final matching point set to obtain the precise transformation matrix T f Obtain it. When aligning a target entity object, the source point cloud of the target entity object is obtained, and the optimal transformation matrix T is calculated. o The source point cloud is converted to a target point cloud and displayed on the screen. The aligned source and target point clouds are obtained as shown in Figure 8, completing the alignment of the partially overlapping point clouds.

[0031] Those skilled in the art should understand that embodiments of the present application may be provided as methods, systems, or computer program products. Accordingly, the application may take the form of complete hardware embodiments, complete software embodiments, or embodiments combining both software and hardware aspects. Furthermore, the application may take the form of computer program products implemented on a computer-available storage medium (including, but not limited to, disk storage devices, optical storage devices, etc.) containing one or more computer-available program codes. Solutions in embodiments of the present application may be implemented in various computer languages. The application will be described with reference to flowcharts of methods, systems, and computer program products according to embodiments of the present application.

[0032] Although preferred embodiments of the present application have already been described, those skilled in the art may make additional changes and modifications to these embodiments after understanding the basic creative concepts. Therefore, the present invention is intended to be construed as encompassing the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0033] Clearly, a person skilled in the art can make various modifications and alterations to the present application without departing from the spirit and scope of the present application. Thus, if such modifications and alterations of the present application fall within the scope of the equivalent art of the present invention, the present application is intended to include such modifications and alterations as well.

Claims

1. A method for aligning partially overlapping point clouds based on double coincidence, performed by a computer under software control, Step 1) The sensor acquires the source point cloud and target point cloud of the current physical object, and a low-level matching method is used to generate a matching point set between the source point cloud and the target point cloud. Step 2) Using a high-level matching method, the matching tetrahedron set is filtered and sampled between the source point cloud and the target point cloud within the matching point set. Step 3) Using a coarse-to-fine estimation based on a matching tetrahedron set, obtain the optimal transformation matrix between the source and target point clouds of the current entity object. Step 4) A method for aligning partially overlapping point clouds based on double coincidence, performed by a computer under software control, characterized in that when aligning a target entity object, the source point cloud of the target entity object is obtained, the source point cloud is converted to a target point cloud via an optimal transformation matrix, and the alignment of partially overlapping point clouds is completed by displaying it on a screen.

2. In step 1, the specific procedure for generating a matching point set between the source point cloud and the target point cloud using a low-level matching method is as follows: Step 1.1) Obtain the feature descriptors of each point in the source point cloud and target point cloud of the current entity object, and normalize each feature descriptor using the L2-normalization method. Step 1.2) For each pair of points in the source point cloud and the target point cloud, after normalization of the feature descriptors, obtain the feature consistency cost matrix C for the current pair of feature descriptors, and obtain the assignment matrix A from the cost matrix using the Sink-horn algorithm. Step 1.3) In each row of the assignment matrix A, the index of the k largest element is selected using the index selection top-K method, multiple source points and multiple target points corresponding to each source point are extracted, and an initial matching point set is constructed. Step 1.4) A method for aligning a partially overlapping point cloud based on double coincidence according to claim 1, characterized in that an initial matching point set is filtered using geometric coincidence based on curvature and distance, and a matching point set is constructed.

3. In step 1.4) above, for each source point and its target point in the initial matching point set, the curvature error between the source point and the target point is set to the curvature threshold C. ｔｈ If less than, the current source and target points are retained. For each retained source point and its respective target point, the distance error between them is the distance threshold D. ｔｈ The method for aligning a partially overlapping point group based on double coincidence according to claim 2, characterized in that two or more target points less than a certain value are clustered into a single target point, the source point and one or more target points after clustering are obtained to form a pair of matching points, and each pair of matching points is constructed as a matching point set.

4. The specific steps for step 2) are as follows: Step 2.1) Based on each source point and target point in the matching point set, multiple source point cloud lines and target point cloud lines are obtained, and a matching line set is constructed after rematching. Step 2.2) The method for aligning partially overlapping point clouds based on double coincidence according to claim 1, characterized in that a plurality of source point cloud tetrahedra and target point cloud tetrahedra are obtained based on each source point cloud line and target point cloud line in the matching line set, and a matching tetrahedron set is constructed after filtering and sampling.

5.

6.

7.

8. In step 3.2) above, the 6-dimensional space is divided into a plurality of 6-dimensional blocks according to a predetermined leaf node size, each 6-dimensional block contains a plurality of 6-dimensional pose vectors, and during sparsification, for each 6-dimensional block, the center of the 6-dimensional block is used as a feature representation of all 6-dimensional pose vectors within that block and is remapped as a remapping transformation matrix, and each remapping transformation matrix is ​​constructed as a set of remapping transformation matrices, characterized in that, the method for aligning partially overlapping point clouds based on double coincidence according to claim 7.

9. In step 3.3) above, for each source point in the matching point set of step 1) and its remapping transformation matrix, the source points are transformed using the remapping transformation matrix to obtain multiple transformed source points, the distance between the transformed source points and each target point is calculated, the number of transformed source points whose distance to the target point is less than the fifth distance threshold τ is defined as the number of matches, and the remapping transformation matrix that maximizes the number of matches is defined as the optimal coarse transformation matrix T. ｃ The method for aligning partially overlapping point groups based on double coincidence as described in claim 7.

10. In step 3.4) above, for each source point in the matching point set of step 1), the optimal coarse transformation matrix T ｃ After transforming the source point using [a specific method], multiple coarsely transformed source points are obtained, and for each coarsely transformed source point, the distance between the coarsely transformed source point and the target point is calculated, and the distance to the target point is set to a predetermined sixth distance threshold d. ６ｔｈ The coarse transformation source points that are less than are retained, and each retained coarse transformation source point and its corresponding target point are constructed as the final matching point set. The singular value decomposition SVD algorithm is then applied to the final matching point set to obtain the precise transformation matrix T ｆ A method for aligning partially overlapping point clouds based on double coincidence according to claim 7, characterized by obtaining the required information.

Images