An ultra-low overlap point cloud registration method and device

Abstract

The application provides an ultra-low overlap point cloud registration method and device. The method comprises: obtaining point cloud data to be registered; performing registration processing on the point cloud data to be registered through a pre-trained point cloud registration network to obtain a registration result; the pre-trained point cloud registration network is provided with an outlier removal module; the outlier removal module is used for performing outlier division on the initial overlap point cloud after obtaining the estimated initial overlap point cloud, and performing point cloud removal of the far point outlier and the near point outlier based on the result of the outlier division to obtain the final overlap point cloud with the outliers removed; the outlier division comprises far point outlier division and near point outlier division. By setting the outlier removal module, the far point outlier division and the near point outlier division are performed by using the module, and on this basis, the point cloud removal of the far point outlier and the near point outlier is performed, so that the interference problem of non-overlapping redundant information on registration is solved, and the registration accuracy is improved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically relating to a method and apparatus for ultra-low overlap point cloud registration. Background Technology

[0002] 3D point cloud registration is a crucial technique in computer vision and robotics. It aligns two point clouds using rigid transformations, involving finding the optimal alignment from 3D point cloud data acquired from different viewpoints or time points. 3D point cloud registration has a wide range of applications, including 3D reconstruction, robot navigation, and virtual / augmented reality. The advent of 3D scanner technology has made acquiring large-scale, high-resolution point clouds easier; however, ensuring complete overlap of scanned point clouds is challenging when acquiring high-resolution, large-scene point clouds. These non-overlapping point clouds—points without corresponding relationships in another point cloud—are called outliers, posing a significant challenge to point cloud registration.

[0003] Currently, numerous methods have been proposed to solve the point cloud registration problem, which can be broadly categorized into two types: traditional methods and deep learning methods. The Iterative Closest Point (ICP) algorithm, a traditional method, is one of the most well-known traditional point cloud registration algorithms and was widely used in the past to solve the registration problem. However, the ICP algorithm is prone to getting trapped in local optima and is highly sensitive to initial pose and outliers. In recent years, due to the powerful learning capabilities of deep learning, many methods have integrated deep neural networks to establish correspondences and then used singular value decomposition (SVD) algorithms to calculate the transformation matrix. Inspired by ICP, the Deep Closest Point (DCP) algorithm proposes a point cloud feature extraction network to extract depth features instead of the coordinate features in ICP. To address the issue of partial overlap, Predator (S. Huang, Z. Gojcic, M. Usvyatsov, A. Wieser, and K. Schindler, Predator: Registration of 3d point clouds with low overlap [C], IEEE Conference on Computer Vision and Pattern Recognition, 2021: 4267–4276) proposed an overlap attention module, which is used to estimate overlapping regions and then directly used for point cloud registration. MaskNet (V. Sarode, A. Dhagat, RA Srivatsan, N. Zevallos, S. Lucey, and H. Choset, MaskNet: A fully-convolutional network to estimate inlierpoints [C], IEEE International Conference on 3D Vision, 2020: 1029-1038) proposed a fully convolutional network for estimating overlapping regions and used a fully overlap estimation method for registration.

[0004] Existing point cloud registration methods rely on either perfectly accurate estimated overlapping regions or completely overlapping point clouds for registration. Therefore, these methods suffer from inaccurate registration results. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, this invention provides an ultra-low overlap point cloud registration method and apparatus.

[0006] The technical problem to be solved by this invention is achieved through the following technical solution: In a first aspect, the present invention provides an ultra-low overlap point cloud registration method, comprising: Acquire the point cloud data to be registered; the point cloud data to be registered includes: first point cloud data and second point cloud data; A pre-trained point cloud registration network is used to register the point cloud data to be registered, and the registration result is obtained. The pre-trained point cloud registration network includes an outlier removal module. After obtaining the estimated initial overlapping point cloud, the outlier removal module is used to divide the initial overlapping point cloud into outliers, and based on the outlier division result, to remove far-point outliers and near-point outliers, to obtain the final overlapping point cloud with outliers removed. Outlier division includes far-point outlier division and near-point outlier division.

[0007] Optionally, the pre-trained point cloud registration network may also include: a feature extraction module, an overlapping region estimation module, and a registration module; The feature extraction module, overlapping region estimation module, outlier removal module, and registration module are connected in sequence. The feature extraction module is used to acquire the point cloud data to be registered, and to acquire the depth point cloud features according to the correlation between the current point cloud and the neighboring point clouds in the point cloud data to be registered; The overlapping region estimation module is used to calculate the feature distance map of the depth point cloud features and obtain the initial overlapping point cloud based on the result of the feature distance map calculation. The registration module is used to perform soft matching and rigid transformation on the final overlapping point cloud to obtain the registration result.

[0008] Optionally, the feature extraction module obtains deep point cloud features based on the correlation between the current point cloud and neighboring point clouds in the point cloud data to be registered, including: The K-nearest neighbor algorithm is used to obtain the relevant point cloud corresponding to the current point cloud in the point cloud data to be registered. The correlation coefficient between the current point cloud and related point clouds is calculated using a preset algorithm, and neighborhood features are generated using the correlation coefficient and related point clouds. The neighborhood features are aggregated to obtain aggregated features; the aggregated features include: a first aggregated feature and a second aggregated feature; The aggregated features are combined globally and embedded based on a multilayer perceptron to obtain deep point cloud features.

[0009] Optionally, the correlation coefficient between the current point cloud and related point clouds is calculated using a preset algorithm, and neighborhood features are generated using the correlation coefficient and related point clouds, including: The distance between the current point cloud and related point clouds is calculated using Euclidean distance, and the distance value is used as the correlation coefficient. The neighborhood features are obtained by multiplying the correlation coefficient by the correlation point cloud.

[0010] Optionally, feature aggregation is performed on neighborhood features to obtain aggregated features, including: Aggregated features are obtained by performing feature aggregation processing on neighborhood features using aggregation functions.

[0011] Optionally, the aggregated features are combined globally and embedded using a multilayer perceptron to obtain deep point cloud features, including: The feature distance between the first aggregated feature and the second aggregated feature is calculated using Euclidean distance, and the negative value of the feature distance is used as the feature distance map. Multiply the first aggregated feature by the feature distance map to obtain the first global feature; The first global feature and the second aggregated feature are concatenated to obtain the first concatenated global feature; The first concatenated global feature is input into the feature extractor to obtain the first multidimensional feature, and the first multidimensional feature is used as the first depth point cloud feature. Multiply the second aggregated feature by the feature distance map to obtain the second global feature; The second global feature is concatenated with the first aggregated feature to obtain the second concatenated global feature; The second concatenated global feature is input into the feature extractor to obtain the second multidimensional feature, and the second multidimensional feature is used as the second depth point cloud feature; the first depth point cloud feature and the second depth point cloud feature together constitute the depth point cloud feature.

[0012] Optionally, the outlier removal module performs outlier segmentation and removal based on the outlier removal function; The outlier removal function is expressed as: ; ; This represents the initial overlapping point cloud after removing distant outliers. This represents the final overlapping point cloud after removing outliers. This represents the first overlapping point cloud in the initial overlapping point cloud. The output of the feature extraction module This represents the second overlapping point cloud in the initial overlapping point cloud. The output of the feature extraction module express and Feature distance map between them (.) indicates taking the maximum value of each row of the matrix. This means taking the first k largest elements. express The output of the feature extraction module express and Feature distance map between them This indicates that the sum of each row of a matrix is ​​to be calculated.

[0013] Optionally, the registration module performs soft matching and rigid transformation processing on the final overlapping point cloud to obtain the registration result, including: The final overlapping point cloud is soft-matched using the soft correspondence method to obtain pseudo-point cloud data. A rigid transformation matrix is ​​obtained by performing positional transformations on the pseudo-point cloud data and the final overlapping point cloud using the singular value decomposition algorithm. The rigid transformation matrix includes: a rotation matrix. Translation vector

[0014] Rotation matrix Multiply the data by the first or second point cloud data to obtain the initial registration result, and then multiply the initial registration result by the translation vector. Summing yields the registration result.

[0015] Optionally, the loss function of the pre-trained point cloud registration network is the sum of the overlap estimation loss and the registration loss.

[0016] In a second aspect, the present invention provides an ultra-low overlap point cloud registration device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the ultra-low overlap point cloud registration device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the method described in the first aspect above.

[0017] This invention provides a method and apparatus for ultra-low overlap point cloud registration. The method includes: acquiring point cloud data to be registered; the point cloud data to be registered includes: first point cloud data and second point cloud data; performing registration processing on the point cloud data to be registered using a pre-trained point cloud registration network to obtain a registration result; the pre-trained point cloud registration network includes an outlier removal module; the outlier removal module is used to, after acquiring an estimated initial overlapping point cloud, divide the initial overlapping point cloud into outliers, and based on the outlier division result, remove distant outliers and nearby outliers to obtain a final overlapping point cloud with outliers removed; the outlier division includes: distant outlier division and near outlier division. In this invention, an outlier removal module is set up to perform far-point outlier segmentation and near-point outlier segmentation. Based on this, point cloud removal of far-point outliers and near-point outliers is performed, which solves the problem of interference of non-overlapping redundant information on registration and improves the accuracy of the final point cloud registration result.

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0019] Figure 1 A flowchart illustrating an ultra-low overlap point cloud registration method provided in an embodiment of the present invention; Figure 2 Visualization results of partial samples on the Modelnet40 dataset using the method and comparison method of the present invention, provided for embodiments of the present invention; Figure 3 A quantization result diagram of key steps on Modelnet40 based on the method of the present invention, provided for an embodiment of the present invention; Figure 4 A graph showing the quantization results of key steps in KITTI based on the method of the present invention, provided for an embodiment of the present invention; Figure 5 This is a schematic diagram of an ultra-low overlap point cloud registration device provided in an embodiment of the present invention. Detailed Implementation

[0020] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0021] To improve the accuracy of point cloud registration results, this invention provides an ultra-low overlap point cloud registration method. Figure 1 This is a flowchart illustrating an ultra-low overlap point cloud registration method provided in an embodiment of the present invention. Figure 1 As shown, it includes: S101. Obtain the point cloud data to be registered.

[0022] The point cloud data to be registered includes: the first point cloud data and the second point cloud data.

[0023] S102. The pre-trained point cloud registration network is used to perform registration processing on the point cloud data to be registered, and the registration result is obtained.

[0024] The pre-trained point cloud registration network includes an outlier removal module. After obtaining the estimated initial overlapping point cloud, the outlier removal module performs outlier segmentation on the initial overlapping point cloud and removes both distant and near outliers based on the segmentation results, resulting in the final overlapping point cloud with outliers removed. Outlier segmentation includes distant outlier segmentation and near outlier segmentation.

[0025] Optionally, the outlier removal module performs outlier segmentation and removal based on the outlier removal function; The outlier removal function is expressed as: ; ; This represents the initial overlapping point cloud after removing distant outliers. This represents the final overlapping point cloud after removing outliers. This represents the first overlapping point cloud in the initial overlapping point cloud. The output of the feature extraction module This represents the second overlapping point cloud in the initial overlapping point cloud. The output of the feature extraction module express and Feature distance map between them (.) indicates taking the maximum value of each row of the matrix. This means taking the first k largest elements. express The output of the feature extraction module express and Feature distance map between them This indicates that the sum of each row of a matrix is ​​to be calculated.

[0026] It should be noted that different outliers have different characteristics. If a single method is used for outlier removal, a large number of outliers may still exist. Therefore, this invention proposes a method for outlier segmentation and removal to achieve registration of partially overlapping point clouds. This method addresses the problem of outliers remaining after overlap estimation. Based on the different characteristics of outliers far from and near the overlap region, it designs outlier removal methods for both regions, solving the problem of interference from non-overlapping redundant information on registration and improving the accuracy of point cloud registration to a certain extent.

[0027] Optionally, the loss function of the pre-trained point cloud registration network is the sum of the overlap estimation loss and the registration loss.

[0028] Optionally, the pre-trained point cloud registration network may also include a feature extraction module, an overlapping region estimation module, and a registration module.

[0029] The feature extraction module, overlapping region estimation module, outlier removal module, and registration module are connected in sequence.

[0030] The feature extraction module is used to acquire the point cloud data to be registered, and to obtain the depth point cloud features according to the correlation between the current point cloud and the neighboring point clouds in the point cloud data to be registered.

[0031] Optionally, the feature extraction module obtains deep point cloud features based on the correlation between the current point cloud and neighboring point clouds in the point cloud data to be registered, including: The K-nearest neighbor algorithm is used to obtain the relevant point cloud corresponding to the current point cloud in the point cloud data to be registered. The correlation coefficient between the current point cloud and related point clouds is calculated using a preset algorithm, and neighborhood features are generated using the correlation coefficient and related point clouds. The neighborhood features are aggregated to obtain aggregated features; the aggregated features include: a first aggregated feature and a second aggregated feature; The aggregated features are combined globally and embedded based on a multilayer perceptron to obtain deep point cloud features.

[0032] Optionally, the correlation coefficient between the current point cloud and related point clouds is calculated using a preset algorithm, and neighborhood features are generated using the correlation coefficient and related point clouds, including: The distance between the current point cloud and related point clouds is calculated using Euclidean distance, and the distance value is used as the correlation coefficient. The neighborhood features are obtained by multiplying the correlation coefficient by the correlation point cloud.

[0033] Optionally, feature aggregation is performed on neighborhood features to obtain aggregated features, including: Aggregated features are obtained by performing feature aggregation processing on neighborhood features using aggregation functions.

[0034] Optionally, the aggregated features are combined globally and embedded using a multilayer perceptron to obtain deep point cloud features, including: The feature distance between the first aggregated feature and the second aggregated feature is calculated using Euclidean distance, and the negative value of the feature distance is used as the feature distance map. Multiply the first aggregated feature by the feature distance map to obtain the first global feature; The first global feature and the second aggregated feature are concatenated to obtain the first concatenated global feature; The first concatenated global feature is input into the feature extractor to obtain the first multidimensional feature, and the first multidimensional feature is used as the first depth point cloud feature. Multiply the second aggregated feature by the feature distance map to obtain the second global feature; The second global feature is concatenated with the first aggregated feature to obtain the second concatenated global feature; The second concatenated global feature is input into the feature extractor to obtain the second multidimensional feature, and the second multidimensional feature is used as the second depth point cloud feature; the first depth point cloud feature and the second depth point cloud feature together constitute the depth point cloud feature.

[0035] Specifically, in this embodiment, the processing procedure of the feature extraction module is described in detail.

[0036] Given the first point cloud data Second point cloud data . express The point in the middle, express There is One point, express The point in the middle, express There is One point, Represents the space of real numbers. Because... and The processing procedure is the same in this module, therefore... The process is illustrated using the following example.

[0037] First, the K-nearest neighbor algorithm is used to calculate Spatial neighborhood set of each point cloud Assuming the current point cloud One of the characteristics of a network layer is dot clouds The characteristics in the network layer are . Indicates searching and The closest One point, Indicates and The closest A neighbor pointed to the cloud.

[0038] Since the distance and correlation between point pairs are negatively correlated, The weights of neighbors can be calculated using the negative value of the Euclidean distance, from which we obtain... and Correlation coefficient between .

[0039] Since the present invention focuses on It calculates the entire neighborhood of the center point, not the neighborhood itself, so each point cloud is computed relative to the center point. neighborhood coordinates The formula is as follows: ; Then, the following formula can be used to strengthen effective nearest neighbors and weaken ineffective nearest neighbors: ; in represent and The correlation coefficient between them After representing weighted neighborhood information The neighborhood characteristics.

[0040] To better integrate neighborhood information, the following aggregation function can be used to aggregate the entire neighborhood information to obtain initial aggregated features. : ; in Represents the first in the network layer, Indicates copying sequence Initial aggregation features of the layer Keep the number of neighboring points consistent with the selected number. This indicates a splicing operation. Indicates that there are learnable parameters Multilayer perceptron (MLP). This represents an aggregate function.

[0041] By iteratively executing the above method, the first aggregated feature can be obtained. Similarly, the second aggregation feature can be obtained. ,in It is the dimension of the feature.

[0042] At this time, the first aggregation feature Second aggregation features They remain independent of each other. Because partial overlap leads to some information redundancy, the second point cloud data cannot be directly obtained. Information, because some information is related to the first point cloud data. It is redundant. To solve this problem, we can use the feature distance as... Each point in the text is customized with a specific information about... The global features are then combined with the customized global features and point features. Combined. First, the first aggregation feature is calculated using Euclidean distance. Second aggregation features Feature distance map between .

[0043] Then, calculate First global feature ,as well as Combination The subsequent features, namely the first concatenated global features : ; ; This indicates a feature splicing operation.

[0044] at last, The feature cloud features are then further embedded using a feature extractor (multi-layer self-attention mechanism) to obtain the first depth point cloud features. : ; in Indicates that it has learnable parameters The multi-layered self-attention mechanism (MLP).

[0045] Second-depth point cloud features The calculation process, and The calculation process is similar, and will not be described in detail in this embodiment.

[0046] It should be noted that, in this embodiment, by adopting a neighborhood feature aggregation method based on an attention mechanism, the acquired deep point cloud features can be made more accurate. Subsequent point cloud registration processing based on these deep point cloud features can make the registration results more accurate.

[0047] The overlapping region estimation module is used to calculate the feature distance map of the depth point cloud features and obtain the initial overlapping point cloud based on the result of the feature distance map calculation.

[0048] In this embodiment, the execution process of the overlapping region estimation module is described in detail: The above steps yield the first depth point cloud features. Second depth point cloud features First calculate from arrive The feature distance map is then mapped using MLP to... binary overlap mask The formula is as follows: ; in, Indicates that it has learnable parameters MLP, express and Feature distance map between them.

[0049] Then, the first initial overlapping point cloud can be calculated. The calculation formula is: ; in This represents the Hadamard product.

[0050] Furthermore, the second initial overlapping point cloud The calculation process and The calculation process is similar and will not be repeated in this embodiment. First initial overlapping point cloud Second initial overlapping point cloud Together they form the initial overlapping point cloud.

[0051] The registration module is used to perform soft matching and rigid transformation on the final overlapping point cloud to obtain the registration result.

[0052] Optionally, the registration module performs soft matching and rigid transformation processing on the final overlapping point cloud to obtain the registration result, including: The final overlapping point cloud is soft-matched using the soft correspondence method to obtain pseudo-point cloud data. A rigid transformation matrix is ​​obtained by performing positional transformations on the pseudo-point cloud data and the final overlapping point cloud using the singular value decomposition algorithm. The rigid transformation matrix includes: a rotation matrix. Translation vector

[0053] Rotation matrix Multiply the data by the first or second point cloud data to obtain the initial registration result, and then multiply the initial registration result by the translation vector. Summing yields the registration result.

[0054] Specifically, taking the registration of the second point cloud data with the assistance of the first point cloud data as an example, the implementation process of the registration module is as follows: For the final overlapping point cloud Perform soft matching to obtain pseudo-point cloud data. : ; This indicates soft matching. This represents the second initial overlapping point cloud after outlier removal. express The output of the feature extraction module express The output of the feature extraction module express and Feature distance map between them.

[0055] Registration results as follows: ； Registration is now complete.

[0056] To optimize the overall network, the overlap estimation loss and registration loss can be combined as a loss function. Loss Function . Indicates the overlap estimation loss, This indicates the registration loss.

[0057] In this embodiment, the overlap estimation loss is treated as a binary classification task, and the overlap estimation loss is supervised by the binary cross-entropy function, as shown in the following formula: ; ; ; in, express Overlap estimation loss, express The overlap estimation loss. Superscript Indicates the truth value. represent Binary overlap mask. represent Binary overlap mask. Define overlap mask labels. to indicate Is it an overlapping point, that is, at Does it exist in the middle? Corresponding points , Point cloud Is it an overlapping point, that is, at Does it exist in the middle? Corresponding points .

[0058] For the registration loss, a real rigid transformation is used as supervision, and a standard loss function is employed to directly constrain the transformation prediction. Registration Loss The calculation formula is as follows: ; in Represents the identity matrix. represent The transpose of . Representing the truth , Representing the truth , Represents the actual value.

[0059] This invention provides an ultra-low overlap point cloud registration method, comprising: acquiring point cloud data to be registered; the point cloud data to be registered includes: first point cloud data and second point cloud data; performing registration processing on the point cloud data to be registered using a pre-trained point cloud registration network to obtain a registration result; the pre-trained point cloud registration network includes an outlier removal module; the outlier removal module is used to divide the estimated initial overlapping point cloud into outliers after acquiring it, and to remove distant outliers and nearby outliers based on the outlier division result, to obtain a final overlapping point cloud with outliers removed; the outlier division includes: distant outlier division and near outlier division. In this invention, by setting an outlier removal module, using this module to divide distant outliers and near outliers, and then removing distant outliers and near outliers based on this, the problem of interference from non-overlapping redundant information on registration is solved, and the accuracy of the final point cloud registration result is improved.

[0060] To verify the accuracy of the ultra-low overlap point cloud registration method of this invention, simulation experiments were also conducted.

[0061] Experiment 1 First, validation was performed on the ModelNet40 dataset, which includes 12,311 meshed CAD samples across 40 categories. The experiment used 9,843 samples for training and 2,468 samples for testing. Two point clouds were manually sampled, with two anchor points symmetrically distributed at the center of the point clouds to ensure minimal overlap (20%) when using the farthest point sampling (FPS) algorithm. A randomly generated transformation matrix was then used in […]. 45 45 The rotation and translation along each axis are uniformly sampled in [ ]. [0.5, 0.5], transform each source point cloud S in the training set into a target point cloud T.

[0062] Table 1 shows the registration results of the method of the present invention on Modelnet40 in different scenarios provided by the embodiments of the present invention: Table 1. Registration results of the method of the present invention on Modelnet40 in different scenarios.

[0063] Where RMSE(R) represents the root mean square error of rotation, RMSE(t) represents the root mean square error of translation, MAE(R) represents the mean absolute error of rotation, and MAE(t) represents the mean absolute error of translation. This embodiment uses data from the first 20 categories to train the model, and uses data from the first 20 categories to test the visible categories, and uses data from the last 20 categories to test the invisible categories. Furthermore, additional... , Experiments were conducted using Gaussian noise, and the results are shown in Table 1. Table 1 shows that regardless of whether the class is visible, invisible, or Gaussian noise is added, the rotation and translation errors of the method described in this invention are within acceptable limits, and satisfactory registration results are achieved. That is, this invention has good generalization ability.

[0064] Figure 2 This document presents visualization results of partial samples on the Modelnet40 dataset using the method and comparison method of this invention, as provided in embodiments of the present invention. Figure 2 As shown, the first column is the input point cloud, and the following columns are registration results from different algorithms (including ICP, OMNet, FINet, and this invention). Green represents the source point cloud, blue represents the template point cloud, and red represents the source point cloud after rigid transformation. Figure 2 It can be seen that, compared with the algorithm of the present invention, the point cloud is still not closed after registration (ICP, MNet). Even FINet, which has a better registration effect, has a small range of point pair mismatch. However, the algorithm of the present invention can almost achieve complete closure of the point cloud after registration, which shows that the algorithm of the present invention can achieve point cloud registration under low overlap conditions.

[0065] Furthermore, to more intuitively understand the algorithm proposed in this patent, Figure 3 The quantization results of key steps on Modelnet40 based on the method of the present invention are shown in the embodiments of the present invention. Figure 3 As shown in the figure. The first column is the input point cloud, the second column is the correspondence after removing outliers, and the third column is the registration result. Figure 3 As can be seen, the method of the present invention can find the correct outlier removal points, identify overlapping regions, and establish a correspondence between overlapping regions when inputting low-overlap point clouds, thereby obtaining a more accurate registration result.

[0066] Experiment 2 The KITTI dataset is a large-scale outdoor LiDAR point cloud dataset. It consists of 11 outdoor scan sequences, primarily depicting urban scenes such as streets, cars, and buildings. Sequences 0-5 were used for training, 6-7 for validation, and 8-10 for testing. Further partial-to-partial registration experiments were conducted on the KITTI dataset, and the results are shown in Table 2 below. Table 2. Registration results on the KITTI dataset using different methods.

[0067] Table 2 shows that the ICP and OMNet algorithms are almost ineffective. Although FINet is superior to ICP and OMNet, it still falls short of the method of this invention. The method of this invention outperforms other methods in both rotation and translation errors. This indicates that the method of this invention can still achieve good registration results in large, low-overlap scenes.

[0068] Figure 4 The graph shows the quantization results of key steps in KITTI based on the method of the present invention, as provided in the embodiments of the present invention. Figure 4 As shown, the red point cloud and the blue point cloud correspond to respectively Dot clouds and In a point cloud, the green lines represent the calculated correspondences. For example... Figure 4 It can be seen that the method of the present invention can still correctly remove outliers, identify overlapping areas, and establish correspondences in large, low-overlap scenes, thereby achieving accurate registration.

[0069] The method provided in this invention can be applied to electronic devices. Specifically, the electronic device can be a desktop computer, a portable computer, a smart mobile terminal, a server, etc. No limitation is made herein; any electronic device that can implement this invention falls within the protection scope of this invention.

[0070] Based on the same inventive concept, embodiments of the present invention also provide an ultra-low overlap point cloud registration device. Figure 5 This is a schematic diagram of an ultra-low overlap point cloud registration device provided in an embodiment of the present invention. It includes a processor 710, a storage medium 720, and a bus 730. The storage medium 720 stores machine-readable instructions executable by the processor 710. When the ultra-low overlap point cloud registration device is running, the processor 710 communicates with the storage medium 720 via the bus 730. The processor 710 executes the machine-readable instructions to perform the steps of the above-described method embodiment. Specific implementations and technical effects are similar and will not be repeated here.

[0071] The storage medium may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the storage medium may also be at least one storage device located remotely from the aforementioned processor.

[0072] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0073] The present invention also provides a computer-readable storage medium. A computer program is stored in the computer-readable storage medium, and when executed by a processor, the computer program implements the steps of any of the above-described ultra-low overlap point cloud registration methods.

[0074] It should be noted that the terms "first," "second," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure.

[0075] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0076] Although the invention has been described herein in conjunction with various embodiments, those skilled in the art will understand and implement other variations of the disclosed embodiments by reviewing the accompanying drawings and the disclosure in carrying out the claimed invention. In the description of the invention, the word "comprising" does not exclude other components or steps, "a" or "an" does not exclude a plurality, and "a plurality" means two or more, unless otherwise explicitly specified. Furthermore, while different embodiments may describe certain measures, this does not mean that these measures cannot be combined to produce good results.

[0077] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for ultra-low overlap point cloud registration, characterized in that, include: Obtain the point cloud data to be registered; The point cloud data to be registered includes: first point cloud data and second point cloud data; The point cloud data to be registered is registered using a pre-trained point cloud registration network to obtain a registration result. The pre-trained point cloud registration network includes an outlier removal module. After obtaining the estimated initial overlapping point cloud, the outlier removal module performs outlier segmentation on the initial overlapping point cloud and removes distant and near outliers based on the outlier segmentation results to obtain the final overlapping point cloud with outliers removed. The outlier segmentation includes distant outlier segmentation and near outlier segmentation. The pre-trained point cloud registration network also includes: a feature extraction module, an overlapping region estimation module, and a registration module; The feature extraction module, the overlapping region estimation module, the outlier removal module, and the registration module are connected in sequence. The feature extraction module is used to acquire the point cloud data to be registered, and to acquire deep point cloud features according to the correlation between the current point cloud and the neighboring point clouds in the point cloud data to be registered; The overlapping region estimation module is used to calculate the feature distance map of the depth point cloud features and obtain the initial overlapping point cloud based on the result of the feature distance map calculation. The registration module is used to perform soft matching and rigid transformation processing on the final overlapping point cloud to obtain the registration result; The outlier removal module performs outlier segmentation and removal based on the outlier removal function; The outlier removal function is expressed as follows: ； ； This represents the initial overlapping point cloud after removing distant outliers. This represents the final overlapping point cloud after removing outliers. This represents the first overlapping point cloud in the initial overlapping point cloud. The output of the feature extraction module This represents the second overlapping point cloud in the initial overlapping point cloud. The output of the feature extraction module express and Feature distance map between them (.) indicates taking the maximum value of each row of the matrix. This means taking the first k largest elements. express The output of the feature extraction module express and Feature distance map between them This indicates that the sum of each row of a matrix is ​​to be calculated.

2. The ultra-low overlap point cloud registration method according to claim 1, characterized in that, The feature extraction module obtains deep point cloud features based on the correlation between the current point cloud and neighboring point clouds in the point cloud data to be registered, including: The relevant point cloud corresponding to the current point cloud in the point cloud data to be registered is obtained by using the K-nearest neighbor algorithm; The correlation coefficient between the current point cloud and the related point clouds is calculated using a preset algorithm, and neighborhood features are generated using the correlation coefficient and the related point clouds. The neighborhood features are aggregated to obtain aggregated features; the aggregated features include: a first aggregated feature and a second aggregated feature; The aggregated features are combined globally and embedded using a multilayer perceptron to obtain the deep point cloud features.

3. The ultra-low overlap point cloud registration method according to claim 2, characterized in that, The step of calculating the correlation coefficient between the current point cloud and the relevant point clouds using a preset algorithm, and generating neighborhood features using the correlation coefficient and the relevant point clouds, includes: The distance between the current point cloud and the relevant point cloud is calculated using Euclidean distance, and the distance value is used as the correlation coefficient. The neighborhood features are obtained by multiplying the correlation coefficient by the correlation point cloud.

4. The ultra-low overlap point cloud registration method according to claim 2, characterized in that, The step of aggregating the neighborhood features to obtain aggregated features includes: The aggregated features are obtained by performing feature aggregation processing on the neighborhood features using an aggregation function.

5. The ultra-low overlap point cloud registration method according to claim 2, characterized in that, The process of combining global features and embedding features based on a multilayer perceptron to obtain the deep point cloud features includes: The feature distance value between the first aggregated feature and the second aggregated feature is calculated using Euclidean distance, and the negative value of the feature distance value is used as the feature distance map. Multiply the first aggregated feature by the feature distance map to obtain the first global feature; The first global feature and the second aggregated feature are concatenated to obtain the first concatenated global feature. The first stitched global feature is input into the feature extractor to obtain the first multidimensional feature, and the first multidimensional feature is used as the first depth point cloud feature. The second aggregated feature is multiplied by the feature distance map to obtain the second global feature; The second global feature is concatenated with the first aggregated feature to obtain the second concatenated global feature; The second concatenated global feature is input into the feature extractor to obtain the second multidimensional feature, and the second multidimensional feature is used as the second depth point cloud feature; the first depth point cloud feature and the second depth point cloud feature together constitute the depth point cloud feature.

6. The ultra-low overlap point cloud registration method according to claim 1, characterized in that, The registration module performs soft matching and rigid transformation processing on the final overlapping point cloud to obtain the registration result, including: The final overlapping point cloud is soft-matched using a soft correspondence method to obtain pseudo-point cloud data. A rigid transformation matrix is ​​obtained by performing positional transformation on the pseudo-point cloud data and the final overlapping point cloud using a singular value decomposition algorithm; the rigid transformation matrix includes: a rotation matrix. Translation vector The rotation matrix Multiply the data by the first point cloud data or the second point cloud data to obtain the initial registration result, and then multiply the initial registration result by the translation vector. Summing yields the registration result.

7. The ultra-low overlap point cloud registration method according to claim 1, characterized in that, The loss function of the pre-trained point cloud registration network is the sum of the overlap estimation loss and the registration loss.

8. A point cloud registration device with ultra-low overlap, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the ultra-low overlap point cloud registration device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the method as described in any one of claims 1-7.

Images