Point cloud registration method adaptive to point density change

Abstract

The invention discloses a point cloud registration method adaptive to point density change. The method comprises the following steps: step 1, collecting point cloud data; step 2, point feature extraction; 3, removing point pairs based on the relative height of the points; 4, carrying out point filtering and fine registration based on probability distribution; according to the invention, large-scale point cloud registration adapting to point density change can be realized in a large-range outdoor environment with communication bandwidth change.

Description

Technical Field

[0001] This invention relates to the field of point cloud registration technology, and in particular to a point cloud registration method that adapts to changes in point density. Background Technology

[0002] Point cloud registration aims to align two point clouds by predicting a rigid 3D rotation transformation, forming the basis for multi-robot collaborative online mapping tasks. However, in outdoor environments, the communication capabilities between multiple robots are affected by cluttered obstacles and long-distance communication, leading to dynamic changes in communication bandwidth. This causes the density of the large-scale point cloud acquired by the robots to vary with the communication bandwidth during real-time transmission to the server, significantly reducing registration accuracy. Therefore, in outdoor environments with varying communication bandwidth, implementing a large-scale point cloud registration method that adapts to changes in point density is crucial for multi-robot collaborative mapping tasks. Summary of the Invention

[0003] The purpose of this invention is to overcome the above-mentioned shortcomings and provide a point cloud registration method that adapts to changes in point density, enabling large-scale point cloud registration that adapts to changes in point density in a wide range of outdoor environments with varying communication bandwidth.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a point cloud registration method adapting to changes in point density, comprising the following steps: Step 1: Point cloud data acquisition: The acquisition system is used to acquire sparse point cloud data and accumulate it into dense local point cloud. The dense local point cloud is preprocessed by a combination of voxel filtering and random sampling to obtain point clouds with different point densities. Step 2: Point feature extraction: Voxelization preprocessing is performed on the preprocessed dense local point cloud, and robust point features are extracted using a sparse convolutional neural network based on a pyramid structure to establish the correspondence between the point clouds to be registered. Step 3: Point pair elimination based on relative height: Find potential corresponding point pairs between the two point clouds to be registered in cosine space, and use a point pair elimination algorithm based on the relative height of points between the point clouds to be registered to eliminate erroneous matching point pairs in the coarse registration process. Step 4: Point filtering and fine registration based on probability distribution: Use the random sampling consensus method to perform coarse registration on the selected point pairs. After completing the coordinate transformation, divide the point cloud to be registered into voxel partitions, determine the overlapping area of ​​the point cloud, calculate the mean vector and covariance matrix of the voxel blocks in the overlapping area, define the similarity value within the voxel block and set a threshold, retain voxels with similarity values ​​higher than the threshold, and use the point cloud within the voxel to complete fine registration to obtain the rotation and translation matrix.

[0005] Preferably, in step 1, voxel filtering simplifies the point cloud data in each voxel into a representative point by dividing the three-dimensional space into a regular voxel grid.

[0006] Preferably, in step 1, the proportion of random sampling is not less than 25% to avoid the geometric features in the point cloud being severely damaged due to oversampling.

[0007] Preferably, in step 2, the sparse convolutional neural network based on the pyramid structure consists of multiple sparse convolutional blocks Convn and transposed convolutional blocks TConvn, n=0,1,2,3; each convolutional block is arranged in a pyramid structure, and the receptive field gradually increases to capture feature information from local to global.

[0008] Preferably, in step 2, the voxelization preprocessing quantizes each point in the point cloud into a sparse tensor, transforming the irregular point cloud data into a structured form suitable for convolution operations.

[0009] Preferably, in step 3, potential corresponding point pairs are determined by calculating the cosine distance between the feature vectors of two points in the point cloud. The point pair with the smallest cosine distance, i.e. the largest cosine similarity, is determined to be a potential corresponding point pair.

[0010] Preferably, in step 3, the point pair elimination method based on the relative height of the points is as follows: ; in, This is a highly matched indicator used to determine whether a point pair should be retained; the sign of this indicator determines whether a point pair should be kept or discarded. It is the relative height difference between each pair of points; p i It is one of the points in the point cloud to be registered; q j It is another point in the cloud to be registered with p i The corresponding potential matching point; the superscript z refers to the coordinate of the point on the Z-axis of the height dimension; ε is a small value that makes the denominator non-zero; Δz is the average relative height difference of all point pairs, defined as:

[0011] Where M represents the set of all pairs of points; for each pair of points in M, if If it is positive, then save it; if... If it is negative, then discard it. This represents the number of potential corresponding point pairs in set M.

[0012] Preferably, in step 4, voxel partitioning is achieved by dividing the space where the two aligned point clouds are located into regular voxel grids, and by determining which voxel grids simultaneously contain points from the two point clouds, the overlapping area between the point clouds is determined.

[0013] Preferably, in step 4, for each voxel block in the overlapping region, the mean vector and covariance matrix of its midpoint are calculated, wherein the covariance matrix is ​​set as a diagonal matrix and defined as follows: ,in , , Represents the variance of three-dimensional points.

[0014] Preferably, in step 4, the similarity value within the voxel block is: ; in for: ; Where d represents the three dimensions of the point cloud. and This represents the uncertainty of a point within a voxel. and This represents the mean vector of points within a voxel.

[0015] Preferably, in step 4, the threshold is the average of the similarity values ​​of all voxels within the overlapping region, and the threshold is defined as follows: ; In the formula, S represents the set of all voxel pairs in the overlapping region for which similarity values ​​are to be calculated, and each element in the set is a pair of corresponding voxel blocks of the point cloud to be registered. , This represents the number of elements in set S.

[0016] Preferably, in step 4, detailed registration is performed using point clouds within voxels with similar normal distributions to obtain rotation and translation matrices, which are respectively and ,in This represents a rotation matrix, belonging to a special orthogonal group in three dimensions. , It is a translation vector. That is, three-dimensional real space.

[0017] Beneficial effects of this invention: 1. Adapting to changes in point density: This invention uses "voxel filtering + random sampling" preprocessing and pyramid structure sparse convolution to stably extract robust features of point clouds with different densities, avoiding the loss of geometric features.

[0018] 2. Improved registration accuracy: This invention is optimized by "relative height point pair elimination + probability distribution screening" to reduce interference from incorrect matching and greatly improve the accuracy of coarse and fine registration.

[0019] 3. Optimize computational efficiency: This invention significantly reduces the amount of data processing by simplifying voxels, sparsifying computation, and focusing on overlapping regions, making it suitable for rapid processing of large-scale point clouds.

[0020] 4. Enhanced environmental robustness: Through multi-scale feature capture and distribution screening design, this invention can resist outdoor environmental interference such as light and noise, and adapt to complex scenarios.

[0021] 5. Practical Application Fit: This invention adapts to multi-robot collaborative mapping, supports real-time registration, and can cope with point density fluctuations caused by dynamic changes in communication bandwidth. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of the original point cloud. Figure 2 A schematic diagram of a point cloud with a 70% random sampling ratio; Figure 3 A schematic diagram of a point cloud with a 50% random sampling ratio; Figure 4 A schematic diagram of a point cloud with a 35% random sampling ratio; Figure 5 A schematic diagram of a point cloud with a 25% random sampling ratio; Figure 6 This diagram illustrates the correct (green) and incorrect (red) point pairs between point clouds to be registered when no point pair removal method is used. Figure 7 This diagram illustrates the correct (green) and incorrect (red) point pairs between the point clouds to be registered when the point pair rejection method is used. Figure 8 Here are schematic diagrams of the two point clouds to be registered; Figure 9 A visualization of the point cloud registration results. Detailed Implementation

[0023] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0024] Example 1: A point cloud registration method adapting to changes in point density, comprising the following steps: The first step is the collection of point cloud data. In the process of 3D environmental reconstruction, a lidar-inertial odometry system is typically used for data acquisition to obtain a high-precision environmental model. In practice, at regular intervals (set to 5 meters in this invention), the system accumulates the sparse point cloud data obtained from lidar scanning into a dense local point cloud. These local point clouds can describe the surfaces and spatial structures of objects in the environment in detail, providing fundamental data for subsequent 3D reconstruction.

[0025] After acquiring dense local point clouds, preprocessing is typically performed to improve data processing efficiency and reduce computational resource consumption. This preprocessing includes voxel filtering and random sampling. Voxel filtering is a spatial partitioning-based downsampling method that divides the 3D space into a regular voxel grid. The point cloud data within each voxel is simplified to a representative point, typically the average of all points within that voxel. This method can smooth the point cloud data to some extent while removing noise points and redundant information. To further reduce the amount of point cloud data while minimizing damage to geometric features, random sampling is also performed. During random downsampling, to ensure that the geometric features in the point cloud are not severely damaged by oversampling, the random sampling ratio is specified to be no less than 25%. This means that after random sampling, at least 25% of the original point cloud data is retained.

[0026] The second step is the extraction of point features. After obtaining a dense local point cloud, voxelization preprocessing is performed first. This process quantizes each point in the point cloud into a sparse tensor, thereby transforming the originally irregular point cloud data into a structured form suitable for convolution operations. This transformation is necessary because traditional convolution operations are typically suitable for regular grid data (such as images), while point cloud data is irregularly distributed. Through voxelization, the three-dimensional space can be divided into regular voxel grids.

[0027] Next, a pyramid-structured network architecture is used to progressively extract point features. This architecture consists of multiple sparse convolutional blocks (Conv... n ,in n = 0, 1, 2, 3) and transposed convolutional blocks (TConv) n ,in n The network consists of blocks of 0, 1, 2, and 3. Sparse convolutional blocks are used to extract local features, while transposed convolutional blocks are used for upsampling to restore the spatial resolution of feature points. These convolutional blocks are arranged in a pyramid structure with gradually increasing receptive fields, enabling the network to capture feature information from local to global perspectives. This structural design allows the network to focus on both detailed features and global structure simultaneously.

[0028] Where Conv n and TConv nThe input and output are defined as follows:

[0029]

[0030] Where ST and ST′ represent the sparse tensors in the sparse convolution and transposed sparse convolution processes, respectively. The transposed sparse convolution process of TConv3 is as follows: ,in equal . This represents the point features of the final output point cloud.

[0031] The third step is point pair removal based on the relative height of the points. After obtaining the point features of the two point clouds to be registered, the next step is to calculate the point pairs. Specifically, this process is performed in cosine space to find the optimal matching relationship between points in the two point clouds. For each pair of matched points, their feature vectors have the minimum cosine distance, meaning their cosine similarity is the maximum. This maximum similarity indicates that these points are closest in the feature space and are therefore considered potential corresponding point pairs. These corresponding point pairs form the basis of subsequent point cloud registration algorithms, providing crucial information for calculating the transformation relationship between the point clouds.

[0032] However, the stability of point features can be affected by environmental changes. For example, variations in lighting conditions, different scanning angles, or the presence of noise in point cloud data can all lead to instability in point features. This instability can introduce erroneous point pairs. Specifically, some point pairs whose feature vectors appear close in cosine distance may not actually correspond to each other in 3D space. These erroneous point pairs can mislead the subsequent registration process, leading to decreased registration accuracy. The point pair removal method based on the relative height of points is defined as follows:

[0033] in ε is the relative height difference between each pair of points, where ε is a small value that makes the denominator non-zero. Δz is the average relative height difference across all pairs of points, defined as:

[0034] Where M represents the set of all pairs of points. For each pair of points in M, if If it is positive, then save it; if... If it is negative, then remove it.

[0035] The fourth step is to use a point filter based on probability distribution. Get After finding positive point matches, the Random Sample Consensus (RANSAC) method is used for coarse registration of the two point clouds to be registered. RANSAC randomly selects a subset of matching point pairs to estimate the rotation and translation matrices and verifies the applicability of the model to all matching point pairs, thereby progressively optimizing the registration results. After coordinate transformation, the two point clouds are transformed into the same coordinate system, which provides a foundation for further fine registration.

[0036] Next, the point clouds to be registered are voxel-partitioned. Since the two point clouds have already been coarsely registered and aligned, the space can be divided into regular voxel grids using voxelization. By checking which voxel grids contain points from both point clouds, the overlapping regions between the point clouds can be determined. These overlapping regions are crucial for subsequent fine registration and fusion, as they provide a direct spatial correspondence between the two point clouds.

[0037] For each voxel block in an overlapping region, the mean vector and covariance matrix of the points within it are first calculated. The mean vector represents the average position of all points within the voxel block, while the covariance matrix describes the spatial distribution of the points. To simplify the calculation, let Σ be a diagonal matrix, defined as... ,in , , Represents the variance of three-dimensional points.

[0038] Define the similarity value within voxel blocks between point clouds to be registered as:

[0039] in for:

[0040] Where d represents the three dimensions of the point cloud. and This represents the uncertainty of a point within a voxel. and This represents the mean vector of points within a voxel. The average of the similarity values ​​of all voxels within the overlapping region is used as a threshold, and voxels with similarity values ​​higher than the threshold are retained. The threshold is defined as:

[0041] Finally, detailed registration was performed using point clouds within voxels with similar normal distributions to obtain the rotation and translation matrices, which are respectively... and .

[0042] Example 2: A point cloud registration method adapting to changes in point density, comprising the following steps: The first step is the collection of point cloud data. In the process of 3D environmental reconstruction, a lidar-inertial odometry system is typically used for data acquisition to obtain a high-precision environmental model. In practice, at regular intervals (set to 5 meters in this invention), the system accumulates the sparse point cloud data obtained from lidar scanning into a dense local point cloud. The size of the local point cloud is 50 meters * 50 meters * 25 meters.

[0043] After acquiring dense local point clouds, preprocessing is typically performed to improve data processing efficiency and reduce computational resource consumption. This preprocessing includes voxel filtering and random sampling. Voxel filtering is a spatial partitioning-based downsampling method that divides the 3D space into a regular voxel grid. The point cloud data within each voxel is simplified to a representative point, usually the average value of all points within that voxel. This method can smooth the point cloud data to some extent while removing noise points and redundant information. To ensure registration accuracy, the voxelization size is set to 0.3 meters. To further reduce the amount of point cloud data while minimizing damage to geometric features, random sampling is also performed. Figures 1 to 5 The original point cloud and point clouds with sampling ratios of 70%, 50%, 35%, and 25% are shown respectively.

[0044] The second step is the extraction of point features. After obtaining the dense local point cloud, voxelization preprocessing is performed first. This process quantizes each point in the point cloud into a sparse tensor, thereby transforming the originally irregular point cloud data into a structured form suitable for convolution operations. To ensure registration accuracy, the voxelization size is set to 0.3 meters.

[0045] Next, a pyramid-structured network architecture is used to progressively extract point features. This architecture consists of multiple sparse convolutional blocks (Conv... n ,in n = 0, 1, 2, 3) and transposed convolutional blocks (TConv) n ,in n The network consists of blocks of 0, 1, 2, and 3. Sparse convolutional blocks are used to extract local features, while transposed convolutional blocks are used for upsampling to restore the spatial resolution of feature points. These convolutional blocks are arranged in a pyramid structure with gradually increasing receptive fields, enabling the network to capture feature information from local to global perspectives. This structural design allows the network to focus on both detailed features and global structure simultaneously.

[0046] Where Conv n and TConv n The input and output are defined as follows:

[0047]

[0048] Where ST and ST′ represent the sparse tensors in the sparse convolution and transposed sparse convolution processes, respectively. The transposed sparse convolution process of TConv3 is as follows: ,in equal . This represents the point features of the final output point cloud.

[0049] The third step is point pair removal based on the relative height of the points. After obtaining the point features of the two point clouds to be registered, the next step is to calculate the point pairs. Specifically, this process is performed in cosine space to find the optimal matching relationship between points in the two point clouds. For each pair of matched points, their feature vectors have the minimum cosine distance, meaning their cosine similarity is the maximum. This maximum similarity indicates that these points are closest in the feature space and are therefore considered potential corresponding point pairs.

[0050] However, the stability of point features can be affected by environmental changes. For example, changes in lighting conditions, different scanning angles, or the presence of noise in the point cloud data can all lead to instability in point features. This instability can introduce erroneous point pairs. Specifically, some point pairs whose feature vectors appear close in cosine distance are not actually corresponding points in 3D space. These erroneous point pairs can mislead the subsequent registration process, leading to a decrease in registration accuracy. The point pair removal method based on the relative height of points is defined as follows:

[0051] in ε is the relative height difference between each pair of points, where ε is a small value that makes the denominator non-zero. Δz is the average relative height difference across all pairs of points, defined as:

[0052] Where M represents the set of all pairs of points. For each pair of points in M, if If it is positive, then save it; if... If it is negative, then remove it.

[0053] Figure 6 and Figure 7 This demonstrates the effectiveness of a point pair removal method based on relative point height. In these two graphs, blue and yellow points represent two point clouds to be registered, green indicates correct point pairs, and red indicates incorrect point pairs. Figure 6 This indicates that no point-to-point elimination method was used, and the proportion of correct point-to-point pairs was 62.15%. Figure 7This indicates that a point-to-point elimination method was used, and the proportion of correct point-to-point pairs was 65.53%.

[0054] The fourth step is to use a point filter based on probability distribution. Get After finding positive point matches, the Random Sample Consensus (RANSAC) method is used for coarse registration of the two point clouds to be registered. RANSAC randomly selects a subset of matching point pairs to estimate the rotation and translation matrices and verifies the applicability of the model to all matching point pairs, thereby progressively optimizing the registration results. After coordinate transformation, the two point clouds are transformed into the same coordinate system, which provides a foundation for further fine registration.

[0055] Next, the point clouds to be registered are voxel-partitioned. Since the two point clouds have already been coarsely registered and aligned, the space can be divided into regular voxel grids using voxelization. By checking which voxel grids contain points from both point clouds, the overlapping regions between the point clouds can be determined. These overlapping regions are crucial for subsequent fine registration and fusion, as they provide a direct spatial correspondence between the two point clouds.

[0056] For each voxel block in an overlapping region, the mean vector and covariance matrix of the points within it are first calculated. The mean vector represents the average position of all points within the voxel block, while the covariance matrix describes the spatial distribution of the points. To simplify the calculation, let Σ be a diagonal matrix, defined as... ,in , , Represents the variance of three-dimensional points.

[0057] Define the similarity value within voxel blocks between point clouds to be registered as:

[0058] in for:

[0059] Where d represents the three dimensions of the point cloud. and This represents the uncertainty of a point within a voxel. and This represents the mean vector of points within a voxel. The average of the similarity values ​​of all voxels within the overlapping region is used as a threshold, and voxels with similarity values ​​higher than the threshold are retained. The threshold is defined as:

[0060] Finally, detailed registration was performed using point clouds within voxels with similar normal distributions to obtain the rotation and translation matrices, which are respectively... and .

[0061] Figure 8 and Figure 9 The results of registering two point clouds of the same region using this method are shown. Figure 8 The two point clouds on the left and right sides were obtained by random sampling at 70% and 35% respectively; Figure 9 This provides a visualization of the completed point cloud registration.

[0062] We are Figure 7 and Figure 8 The image shows the registration results of two point clouds. Figure 7 The two point clouds in the image represent the same region, but they have different random sampling ratios: the sampling ratio of the left point cloud is 70%, and that of the right point cloud is 35%. Figure 8 This represents the visualization effect after the two point clouds are registered.

[0063] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A point cloud registration method adaptable to changes in point density, characterized in that: Includes the following steps: Step 1: Point cloud data acquisition: The acquisition system is used to acquire sparse point cloud data and accumulate it into dense local point cloud. The dense local point cloud is preprocessed by a combination of voxel filtering and random sampling to obtain point clouds with different point densities. Step 2: Point feature extraction: Voxelization preprocessing is performed on the preprocessed dense local point cloud, and robust point features are extracted using a sparse convolutional neural network based on a pyramid structure to establish the correspondence between the point clouds to be registered. Step 3: Point pair elimination based on relative height: Find potential corresponding point pairs between the two point clouds to be registered in cosine space, and use a point pair elimination algorithm based on the relative height between the points in the point clouds to be registered to eliminate erroneous matching point pairs in the coarse registration process. Step 4: Point filtering and fine registration based on probability distribution: Use the random sampling consensus method to perform coarse registration on the selected point pairs. After completing the coordinate transformation, divide the point cloud to be registered into voxel partitions, determine the overlapping area of ​​the point cloud, calculate the mean vector and covariance matrix of the voxel blocks in the overlapping area, define the similarity value within the voxel block and set a threshold, retain voxels with similarity values ​​higher than the threshold, and use the point cloud within the voxel to complete fine registration to obtain the rotation and translation matrix.

2. The point cloud registration method adapting to changes in point density according to claim 1, characterized in that: In step 1, voxel filtering simplifies the point cloud data in each voxel into a representative point by dividing the three-dimensional space into a regular voxel grid.

3. The point cloud registration method adapting to changes in point density according to claim 1, characterized in that: In step 1, the proportion of random sampling is not less than 25% to avoid the geometric features in the point cloud being severely damaged by oversampling.

4. The point cloud registration method adapting to changes in point density according to claim 1, characterized in that: In step 2, the sparse convolutional neural network based on the pyramid structure consists of multiple sparse convolutional blocks Convn and transposed convolutional blocks TConvn, n=0,1,2,3; each convolutional block is arranged in a pyramid structure, and the receptive field gradually increases to capture feature information from local to global.

5. The point cloud registration method adapting to changes in point density according to claim 1, characterized in that: In step 2, the voxelization preprocessing quantizes each point in the point cloud into a sparse tensor, transforming the irregular point cloud data into a structured form suitable for convolution operations.

6. The point cloud registration method adapting to changes in point density according to claim 1, characterized in that: In step 3, potential corresponding point pairs are determined by calculating the cosine distance between the feature vectors of two points in the point cloud. The point pair with the smallest cosine distance, i.e. the largest cosine similarity, is determined to be a potential corresponding point pair.

7. The point cloud registration method adapting to changes in point density according to claim 1, characterized in that: In step 3, the point pair elimination method based on the relative height of points is as follows: ； in, This is a highly matched indicator used to determine whether a point pair should be retained; the sign of this indicator determines whether a point pair should be kept or discarded. It is the relative height difference between each pair of points; p i It is one of the points in the point cloud to be registered; q j It is another point in the cloud to be registered with p i The corresponding potential matching point; the superscript z refers to the coordinate of the point on the Z-axis of the height dimension; ε is a small value that makes the denominator non-zero; Δz is the average relative height difference of all point pairs, defined as: Where M represents the set of all pairs of points; for each pair of points in M, if If it is positive, then save it; if... If it is negative, then discard it. This represents the number of potential corresponding point pairs in set M.

8. The point cloud registration method adapting to changes in point density according to claim 1, characterized in that: In step 4, voxel partitioning is achieved by dividing the space of the two aligned point clouds into regular voxel grids. The overlapping area between the point clouds is determined by judging which voxel grids contain points from both point clouds.

9. The point cloud registration method adapting to changes in point density according to claim 1, characterized in that: In step 4, for each voxel block in the overlapping region, the mean vector and covariance matrix of its midpoints are calculated. The covariance matrix is ​​set as a diagonal matrix and defined as follows: ,in , , Represents the variance of three-dimensional points.

10. The point cloud registration method adapting to changes in point density according to claim 9, characterized in that: In step 4, the similarity value within the voxel block is: ； in for: ； Where d represents the three dimensions of the point cloud. and This represents the uncertainty of a point within a voxel. and This represents the mean vector of points within a voxel.

11. The point cloud registration method adapting to changes in point density according to claim 1, characterized in that: In step 4, the threshold is the average of the similarity values ​​of all voxels within the overlapping region, and the threshold is defined as follows: ； In the formula, S represents the set of all voxel pairs in the overlapping region for which similarity values ​​are to be calculated, and each element in the set is a pair of corresponding voxel blocks of the point cloud to be registered. , This represents the number of elements in set S.

12. The point cloud registration method adapting to changes in point density according to claim 11, characterized in that: In step 4, detailed registration is performed using point clouds within voxels that have similar normal distributions to obtain rotation and translation matrices, which are respectively... and ,in This represents a rotation matrix, belonging to a special orthogonal group in three dimensions. , It is a translation vector. That is, three-dimensional real space.

Images