A point cloud registration method, system, device, and medium based on deep learning

Abstract

This invention discloses a point cloud registration method, system, device, and medium based on deep learning, relating to the fields of computer vision and lidar measurement and calibration. The method includes: acquiring lidar data and generating a point cloud dataset based on the lidar data; determining a first point cloud and a second point cloud based on the point cloud dataset; using a trained Siamese neural network to determine the geometric descriptor and key score of each point in the first and second point clouds; determining several matching pairs from the first and second point clouds based on the geometric descriptor and key score; using a trained matching pair consistency learning network to determine the confidence level of each matching pair; calculating the transformation matrix between the first and second point clouds based on each confidence level; and registering the first and second point clouds based on the transformation matrix. This invention can improve the point cloud registration accuracy, thereby improving the accuracy of lidar measurement and calibration.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and lidar measurement and calibration, and in particular to a point cloud registration method, system, device and medium based on deep learning. Background Art

[0002] 3D point cloud registration refers to the process of aligning two given point cloud frames to the same coordinate system by calculating the transformation matrix between them and the relative poses of the two frames.

[0003] In recent years, autonomous driving and digital humans have become hot research topics in both industry and academia. Point cloud registration technology plays a crucial role in these fields. Although traditional Simultaneous Localization and Mapping (SLAM) has made significant progress in recent years, it is inevitably limited by lighting conditions and texture quality. Algorithms based on Iterative Closest Point (ICP) require good initial values ​​in point cloud registration; otherwise, they are prone to getting trapped in local optima. Therefore, deep learning-based point cloud registration methods have gradually become a research hotspot.

[0004] Deep learning-based point cloud registration methods can be broadly categorized into two main types. The first type uses local feature descriptors to obtain matching relationships and then employs the Random Sample Consensus (RANSAC) algorithm to calculate the transformation matrix from these relationships. This method requires the local feature descriptors to be robust to radar pose information and to differences caused by different sampling angles (such as self-occlusion). Even if these two conditions are met, it is still difficult to obtain accurate matching relationships based solely on local surface information. The second type is a regression-based method, where the neural network directly outputs the transformation matrix between two input point cloud frames. This method has different solutions, but ultimately, its accuracy is difficult to surpass that of the first method.

[0005] To address the accuracy issues existing in current technologies, a novel point cloud registration method is needed. Summary of the Invention

[0006] The purpose of this invention is to provide a point cloud registration method, system, device, and medium based on deep learning to improve point cloud registration accuracy, thereby improving the accuracy of lidar measurement and calibration.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] A point cloud registration method based on deep learning, the point cloud registration method comprising:

[0009] Acquire lidar data and generate a point cloud dataset based on the lidar data;

[0010] The first point cloud and the second point cloud are determined based on the point cloud dataset; the first point cloud and the second point cloud respectively represent lidar data from different frames;

[0011] Using a trained Siamese neural network, the geometric descriptor and key score of each point in the first point cloud and the second point cloud are determined;

[0012] Based on the geometric descriptor and the key score, several matching pairs are determined from the first point cloud and the second point cloud; the matching pairs represent a one-to-one correspondence between points in the first point cloud and points in the second point cloud;

[0013] The confidence level of each pair of matches is determined by using a pre-trained matching pair consistency learning network.

[0014] Calculate the transformation matrix between the first point cloud and the second point cloud based on the confidence levels described above;

[0015] The first point cloud and the second point cloud are registered according to the transformation matrix to obtain a registered point cloud; the registered point cloud is used to construct a point cloud map of the lidar scanning area.

[0016] Optionally, the trained Siamese neural network includes: an encoder network, a mutual attention network, and a decoder network; using the trained Siamese neural network, the geometric descriptor and key score of each point in the first point cloud and the second point cloud are determined, specifically including:

[0017] An encoder network is used to downsample the first point cloud and the second point cloud respectively to obtain the first local shape feature and the second local shape feature;

[0018] A mutual attention network is used to exchange information between the first local shape feature and the second local shape feature to obtain the first fused feature and the second fused feature;

[0019] A decoder network is used to upsample the first fused feature and the second fused feature to obtain the geometric descriptor and key score of each point in the first point cloud and the second point cloud.

[0020] Optionally, the trained pair consistency learning network includes: a low-order consistency determination network, a high-order consistency determination network based on a self-attention mechanism, and a fully connected network; the trained pair consistency learning network is used to determine the confidence of each pair, specifically including:

[0021] A low-order consistency determination network is used to determine the relative equidistance between each pair of matching pairs based on the geometric descriptors of the corresponding points in each matching pair, thereby obtaining a low-order consistency matrix.

[0022] A high-order consistency determination network based on a self-attention mechanism is used to determine the high-order consistency features of each matching pair based on the low-order consistency matrix.

[0023] A fully connected network is used to perform binary classification on each matching pair based on the higher-order consistency features, thereby obtaining the confidence level of each matching pair.

[0024] Optionally, the transformation matrix between the first point cloud and the second point cloud is calculated based on each of the confidence levels, specifically including:

[0025] Based on the confidence levels described, the first point cloud and the second point cloud are decentralized respectively to obtain the decentralized result;

[0026] Calculate the covariance matrix based on the confidence level and the decentralization result;

[0027] Singular value decomposition is performed on the covariance matrix, and the transformation matrix between the first point cloud and the second point cloud is determined based on the singular value decomposition result.

[0028] Optionally, based on the geometric descriptor and the key score, several matching pairs are determined from the first point cloud and the second point cloud, specifically including:

[0029] Select several points with the highest key scores from the first point cloud and the second point cloud respectively, and use them as the first set of points to be matched and the second set of points to be matched;

[0030] The points in the first set of points to be matched and the second set of points to be matched are matched according to the geometric descriptor to obtain several matching pairs.

[0031] Optionally, determining the first point cloud and the second point cloud based on the point cloud dataset specifically includes:

[0032] Select any two point clouds with an overlap of 30% to 60% from the point cloud dataset, and perform data augmentation on the two point clouds respectively to obtain the first point cloud and the second point cloud.

[0033] Optionally, both the encoder network and the decoder network adopt the KPConv convolution method, and the encoder network and the decoder network are connected across layers using the U-Net method.

[0034] A point cloud registration system based on deep learning, the point cloud registration system comprising:

[0035] A dataset generation module is used to acquire lidar data and generate a point cloud dataset based on the lidar data.

[0036] A point cloud determination module is used to determine a first point cloud and a second point cloud based on the point cloud dataset; the first point cloud and the second point cloud respectively represent lidar data from different frames;

[0037] A geometric descriptor and key score determination module is used to determine the geometric descriptor and key score of each point in the first point cloud and the second point cloud using a trained Siamese neural network.

[0038] The matching pair determination module is used to determine a number of matching pairs from the first point cloud and the second point cloud based on the geometric descriptor and the key score; the matching pair represents a one-to-one correspondence between points in the first point cloud and points in the second point cloud;

[0039] The confidence determination module is used to determine the confidence of each pair of matches using a trained pair consistency learning network.

[0040] The transformation matrix calculation module is used to calculate the transformation matrix between the first point cloud and the second point cloud based on the confidence levels.

[0041] The point cloud registration module is used to register the first point cloud and the second point cloud according to the transformation matrix to obtain a registered point cloud; the registered point cloud is used to construct a point cloud map of the lidar scanning area.

[0042] An electronic device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the point cloud registration method described above.

[0043] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the point cloud registration method described above.

[0044] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0045] The point cloud registration method based on deep learning provided by this invention employs a trained Siamese neural network to determine the geometric descriptor and key score of each point in the point cloud. Based on the geometric descriptor and key score, several matching pairs are determined from a first point cloud and a second point cloud, enabling the extraction of the most characteristic key points from both points for matching. A trained matching pair consistency learning network is then used to determine the confidence level of each matching pair. Based on the confidence level, more accurate matching pairs are further determined from these pairs, thereby accurately calculating the transformation matrix and achieving point cloud registration. Therefore, this invention can improve the accuracy of point cloud registration, thereby improving the accuracy of LiDAR measurement and calibration, and providing more precise positioning services and more realistic scene simulations in applications such as virtual reality, autonomous driving, and 3D reconstruction. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart of the point cloud registration method based on deep learning provided by this invention;

[0048] Figure 2 The network structure diagram of the point cloud registration method based on deep learning provided by this invention is shown below.

[0049] Figure 3 A block diagram of the point cloud registration system based on deep learning provided by this invention.

[0050] Symbol explanation:

[0051] Dataset generation module-1, point cloud determination module-2, geometric descriptor and key score determination module-3, matching pair determination module-4, confidence determination module-5, transformation matrix calculation module-6, point cloud registration module-7. Detailed Implementation

[0052] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0053] The purpose of this invention is to provide a point cloud registration method, system, device, and medium based on deep learning to improve point cloud registration accuracy.

[0054] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Example 1

[0056] This invention provides a point cloud registration method based on deep learning. For example... Figure 1 and Figure 2 As shown, the point cloud registration method includes:

[0057] Step S1: Acquire lidar data and generate a point cloud dataset based on the lidar data. Specifically, the point cloud dataset is generated using historical data collected by the lidar system.

[0058] Step S2: Determine the first point cloud and the second point cloud based on the point cloud dataset; the first point cloud and the second point cloud respectively represent LiDAR data from different frames. Preferably, select any two frames of point clouds with an overlap of 30% to 60% from the point cloud dataset, and perform data augmentation on the two frames of point clouds respectively to obtain the first point cloud (denoted as point cloud S) and the second point cloud (denoted as point cloud T).

[0059] Step S3: Using a trained Siamese neural network, determine the geometric descriptor and key score for each point in the first and second point clouds. The Siamese neural network is trained using supervised training. The trained Siamese neural network includes an encoder network, a mutual attention network, and a decoder network.

[0060] Preferably, both the encoder network and the decoder network adopt the kernel point convolution (KPConv) method of convolution, and the encoder network and the decoder network are connected across layers using the U-Net method, which can avoid information loss.

[0061] Step S3 specifically includes:

[0062] Step S3.1: Using an encoder network, downsample the first point cloud and the second point cloud respectively to obtain the first local shape feature and the second local shape feature.

[0063] Specifically, the encoder network takes point cloud data as input, downsamples the point cloud to generate low-resolution local shape features, and outputs corresponding to point cloud S and point cloud T are respectively... and

[0064] Step S3.2: Using a mutual attention network, information is exchanged between the first local shape feature and the second local shape feature to obtain the first fused feature and the second fused feature.

[0065] Specifically, the mutual attention network transformer uses and Using the two point cloud frames as input, information is exchanged. The specific steps for information exchange are: 1. In the transformer, using... As a query, with As key and value, from Extract information and integrate it into In, and output 2. In the transformer, with As a query, with As key and value, from Extract information and integrate it into In, and output At this point, and Each contains information about the other, thus completing the information exchange.

[0066] Step S3.3: Using a decoder network, the first fused feature and the second fused feature are upsampled respectively to obtain the geometric descriptor and key score of each point in the first point cloud and the second point cloud.

[0067] Specifically, the inputs to the decoder network are respectively and Low-resolution point cloud features and Resampling yields the geometric descriptor F for each point. S and F T And the key score.

[0068] Step S4: Based on the geometric descriptor and the key score, determine several matching pairs from the first point cloud and the second point cloud; the matching pairs represent the one-to-one correspondence between points in the first point cloud and points in the second point cloud.

[0069] Step S4 specifically includes:

[0070] Step S4.1: Select several points with the highest criticality scores from the first point cloud and the second point cloud respectively, as the first set of points to be matched and the second set of points to be matched.

[0071] Step S4.2: Match the points in the first set of points to be matched and the second set of points to be matched according to the geometric descriptor to obtain several matching pairs.

[0072] Specifically, the points in the first and second point clouds are sorted from high to low according to the obtained key scores. Based on the sorting results, K' points with the highest scores are selected from the two point clouds as key points, and coarse matching pairs are generated according to their corresponding geometric descriptors (i.e., an initial matching relationship is formed based on the geometric descriptors corresponding to the K' key points of point cloud S and point cloud T).

[0073] In this process, K' key points are selected from two point clouds. There are two types of overlapping points in the two point clouds: ordinary points (such as points located on a plane) and feature points that actually carry information (such as corner points). Only the latter are repeatable key points. Repeatability means that they can be determined as key points in point clouds sampled in multiple consecutive frames.

[0074] In this embodiment, the Siamese neural network uses a mutual attention network to enhance the information exchange between two frames of point clouds, providing a way and basis for the decoder network to mine repeatable feature points in the two frames of point clouds and assign them a high key score.

[0075] The decoder network extracts feature points that appear in both frames of the point cloud. To extract the correct feature points, a mutual attention network is introduced into the forward propagation of the network to provide a pathway. Furthermore, an incentive mechanism for the key scores of correct feature points and a penalty mechanism for the geometric descriptors of ordinary points are introduced into the loss function during network training. This is manifested as follows:

[0076]

[0077] in S represents the i-th point S in the point cloud S. i The key score, it's important to note, is T here. i This represents point S. i The correctly matched points obtained from the labels in point cloud T. Then it means T i Key score; m i The expression representing the distinguishability of this match against the descriptor is m. i =D i- -D i+ D i+ For S i and T i The Euclidean distance between their respective descriptors, D i- For distance S i The Euclidean distance between the descriptor of the nearest point outside a certain range r and S (i.e., for S)i Take the point cloud S and S i The nearest point S j and satisfy S i and S j If the Euclidean distance between them is greater than r, then D i- For S i and S j (Euclidean distance between their respective descriptors).

[0078] According to D i+ and D i- As can be seen from the definition, D i+ S was described i The difficulty of finding the correct matching pair from point cloud T, D i+ The smaller the value, the easier it is to find a match with S. i Correctly matched T i Conversely, the more difficult it is, the more difficult it is; and D i- Describes point S i Local saliency within point cloud S, D i- The smaller the value, the more likely it is to represent point S. i The less significant the value (possibly located in a smooth region), the more significant the value of point S. i The more significant (possibly located at a corner). From the expression, we know that when m... i When the value is greater than 0, it indicates that the matching pair has high discriminative power, and the loss function will tend to let the pair be less discriminative. This score increases, while also making m i To grow larger; conversely, to let... Get smaller, m i This can be further reduced. Based on this mechanism, it is possible to incentivize key scores for correct feature points and penalize ordinary point geometric descriptors.

[0079] Step S5: Use the trained pair consistency learning network to determine the confidence level of each pair. The trained pair consistency learning network includes: a low-order consistency determination network, a high-order consistency determination network based on a self-attention mechanism, and a fully connected network.

[0080] Specifically, the matching pair consistency learning network uses the transformer's self-attention mechanism to compare and analyze the coarse matching pairs of the input point cloud, extract the high-dimensional consistency between the matching pairs, and output the confidence score of whether each matching pair is correct.

[0081] Step S5 specifically includes:

[0082] Step S5.1: Using a low-order consistency determination network, the relative equidistance between each pair of matching pairs is determined based on the geometric descriptors of the corresponding points in each matching pair, thus obtaining a low-order consistency matrix.

[0083] Specifically, in K' pairs of input matching pairs, for any two matching pairs (x i ,y i ) and (x j ,y j The corresponding geometric descriptors output by the decoder are respectively and The low-order consistency of these two pairs can be described as follows: in ||·||2 represents the L2 norm operation. It can be seen that C ij This represents the relative isometry of the two matching pairs in the geometric descriptor. Therefore, it is an imperfect variable that cannot accurately describe whether the two matching pairs are completely consistent, hence it is called low-order. The final result is a K'×K' low-order consistency matrix C.

[0084] Step S5.2: Use a high-order consistency determination network based on a self-attention mechanism to determine the high-order consistency features of each matching pair according to the low-order consistency matrix.

[0085] Specifically, the high-order consistency determination network based on the self-attention mechanism is a multi-layer transformer network structure, and its input is... For K'×6 point cloud data, the dot product operation of Q and K in the original transformer is as follows: Here, the dot product operation of Q and K in each transformer layer is modified as follows: That is, multiplying the original value by a low-order consistency matrix C, so that C can guide the result of the attention mechanism. Specifically, C... ij If the value is large, it indicates that the i-th and j-th matching pairs are low-order consistent in terms of geometric descriptors, and their corresponding attention A can be appropriately increased. ij The strength of the transformer increases, and vice versa. Except for the first transformer layer, each subsequent transformer layer takes as input the output of the previous layer a feature of size K'×D (D is the number of channels), for a total of 12 layers. The final layer outputs features F and a higher-order consistency matrix M = F. ' ·F 'T ,in

[0086] Step S5.3: Using a fully connected network, perform binary classification judgment on each matching pair according to each of the higher-order consistency features to obtain the confidence level of each matching pair.

[0087] Specifically, the final fully connected network performs binary classification on the correctness of the input point cloud matching pairs based on high-order consistency features, obtaining the confidence score (σ) of each matching pair. i (i = 1, 2, ..., K'), the specific formula is: σ i =FC(F i ), where FC(·) represents a fully connected network, and the input is the feature F output by the matching pair consistency learning network.

[0088] In this embodiment, the input to the matching pair consistency learning network is a set of point cloud matching pairs containing both correct and incorrect pairs. The matching pair consistency learning network constructs a low-order consistency relation by analyzing the relationships between descriptors of input matching pairs. This low-order relation is then continuously optimized using the transformer's self-attention mechanism to form a higher-order, more accurate consistency relation. Finally, a fully connected network uses this relation to perform binary classification on the correctness of the input point cloud matching pairs, obtaining the confidence score (σ) for each matching pair. i (i = 1, 2, ..., K'). The training loss function for the matching pair consistency learning network includes the following two items.

[0089] First, the cross-entropy function of the higher-order consistency matrix M:

[0090]

[0091] in, This indicates a matching pair of tags.

[0092] Secondly, the cross-entropy function is matched against the confidence level σ:

[0093]

[0094] in, The confidence level label is represented by CrossEntropy(), which stands for the cross-entropy function.

[0095] Therefore, the overall loss function is loss = loss M +loss σ .

[0096] Step S6: Calculate the transformation matrix between the first point cloud and the second point cloud based on the confidence scores. Specifically, after obtaining the confidence score of each point cloud matching pair, this confidence score is used as the weight of the matching pair, and the transformation matrix between the two frame point clouds can be calculated by weighted singular value decomposition (SVD).

[0097] Step S6 specifically includes:

[0098] Step S6.1: Decentralize the first point cloud and the second point cloud according to the confidence levels to obtain the decentralization results.

[0099] Specifically, the two point clouds are decentered separately, using the following formula: x ' i y ' i These are the decentralized results, where... and It is the center of mass.

[0100] Step S6.2: Calculate the covariance matrix based on the confidence levels and the decentralization results.

[0101] Specifically, using the formula Calculate the covariance matrix. Where, x represents ' i The transpose of .

[0102] Step S6.3: Perform singular value decomposition on the covariance matrix, and determine the transformation matrix between the first point cloud and the second point cloud based on the singular value decomposition result.

[0103] Specifically, for the singular value decomposition of the covariance matrix: H = UΣV T The singular value decomposition results are obtained as a left unitary matrix U, a diagonal matrix Σ, and a right unitary matrix V. The rotation matrix is ​​then R = VU. T The translation vector is Therefore, the transformation matrix is ​​determined to be T = [R|t] T ], where U T R represents the transpose of the left unitary matrix U. T Let t denote the transpose of the rotation matrix R. T This represents the transpose of the translation vector t.

[0104] Step S7: Register the first point cloud and the second point cloud according to the transformation matrix to obtain a registered point cloud; the registered point cloud is used to construct a point cloud map of the lidar scanning area.

[0105] Specifically, the obtained coarse matching pairs are input into a matching pair consistency learning network to extract more accurate matching pairs and calculate the transformation matrix between the two frame point clouds, thereby completing point cloud registration. For two frame point clouds with overlapping areas acquired by lidar, this invention can calculate the transformation matrix between them to align them to the same coordinate system.

[0106] Example 2

[0107] To implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a point cloud registration system based on deep learning is provided below. For example... Figure 3 As shown, the point cloud registration system includes:

[0108] Dataset generation module 1 is used to acquire lidar data and generate a point cloud dataset based on the lidar data.

[0109] Point cloud determination module 2 is used to determine a first point cloud and a second point cloud based on the point cloud dataset; the first point cloud and the second point cloud respectively represent lidar data from different frames.

[0110] The geometric descriptor and key score determination module 3 is used to determine the geometric descriptor and key score of each point in the first point cloud and the second point cloud using a trained Siamese neural network.

[0111] The matching pair determination module 4 is used to determine a number of matching pairs from the first point cloud and the second point cloud based on the geometric descriptor and the key score; the matching pair represents the one-to-one correspondence between points in the first point cloud and points in the second point cloud.

[0112] The confidence determination module 5 is used to determine the confidence of each pair of matches using a trained matching pair consistency learning network.

[0113] Transformation matrix calculation module 6 is used to calculate the transformation matrix between the first point cloud and the second point cloud based on the confidence levels.

[0114] The point cloud registration module 7 is used to register the first point cloud and the second point cloud according to the transformation matrix to obtain a registered point cloud; the registered point cloud is used to construct a point cloud map of the lidar scanning area.

[0115] Example 3

[0116] This invention also provides an electronic device, including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the point cloud registration method of Embodiment 1. The electronic device may be a server.

[0117] In addition, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the point cloud registration method in Embodiment 1.

[0118] In summary, the point cloud registration method, system, device, and medium based on deep learning provided by this invention adopt a deep learning network structure consisting of an encoder, a mutual attention network, a decoder, and a matching pair consistency learning network. First, the encoder downsamples the point cloud to learn low-resolution local shape features. Then, the mutual attention network further enhances the fusion of global information to obtain high-level features. Next, the geometric feature decoder upsamples to obtain the geometric descriptor and key score of each point. Based on the key scores, the most characteristic key points are extracted, and coarse matching pairs are calculated based on their corresponding geometric descriptors. Finally, the matching pair consistency learning network extracts more accurate matching pairs, thereby calculating the transformation matrix and achieving point cloud registration. This improves the accuracy of point cloud registration, thus enhancing the accuracy of lidar measurement and calibration.

[0119] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0120] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A point cloud registration method based on deep learning, characterized in that, The point cloud registration method includes: Acquire lidar data and generate a point cloud dataset based on the lidar data; The first point cloud and the second point cloud are determined based on the point cloud dataset; the first point cloud and the second point cloud respectively represent lidar data from different frames; Using a trained Siamese neural network, the geometric descriptor and key score of each point in the first point cloud and the second point cloud are determined. Specifically, the trained Siamese neural network includes an encoder network, a mutual attention network, and a decoder network. The encoder network is used to downsample the first point cloud and the second point cloud respectively to obtain a first local shape feature and a second local shape feature. The mutual attention network is used to exchange information between the first local shape feature and the second local shape feature to obtain a first fusion feature and a second fusion feature. The decoder network is used to upsample the first fusion feature and the second fusion feature respectively to obtain the geometric descriptor and key score of each point in the first point cloud and the second point cloud. Based on the geometric descriptor and the key score, several matching pairs are determined from the first point cloud and the second point cloud; the matching pairs represent a one-to-one correspondence between points in the first point cloud and points in the second point cloud; The confidence level of each pair of matches is determined by using a pre-trained matching pair consistency learning network. Calculate the transformation matrix between the first point cloud and the second point cloud based on the confidence levels described above; The first point cloud and the second point cloud are registered according to the transformation matrix to obtain a registered point cloud; the registered point cloud is used to construct a point cloud map of the lidar scanning area.

2. The point cloud registration method based on deep learning according to claim 1, characterized in that, The trained pair consistency learning network includes: a low-order consistency determination network, a high-order consistency determination network based on a self-attention mechanism, and a fully connected network; the trained pair consistency learning network is used to determine the confidence of each pair, specifically including: A low-order consistency determination network is used to determine the relative equidistance between each pair of matching pairs based on the geometric descriptors of the corresponding points in each matching pair, thereby obtaining a low-order consistency matrix. A high-order consistency determination network based on a self-attention mechanism is used to determine the high-order consistency features of each matching pair based on the low-order consistency matrix. A fully connected network is used to perform binary classification on each matching pair based on the higher-order consistency features, thereby obtaining the confidence level of each matching pair.

3. The point cloud registration method based on deep learning according to claim 1, characterized in that, Calculate the transformation matrix between the first point cloud and the second point cloud based on the stated confidence levels, specifically including: Based on the confidence levels described, the first point cloud and the second point cloud are decentralized respectively to obtain the decentralized result; Calculate the covariance matrix based on the confidence level and the decentralization result; Singular value decomposition is performed on the covariance matrix, and the transformation matrix between the first point cloud and the second point cloud is determined based on the singular value decomposition result.

4. The point cloud registration method based on deep learning according to claim 1, characterized in that, Based on the geometric descriptor and the key score, several matching pairs are determined from the first point cloud and the second point cloud, specifically including: Select several points with the highest key scores from the first point cloud and the second point cloud respectively, and use them as the first set of points to be matched and the second set of points to be matched; The points in the first set of points to be matched and the points in the second set of points to be matched are matched according to the geometric descriptor to obtain several matching pairs.

5. The point cloud registration method based on deep learning according to claim 1, characterized in that, Determining the first point cloud and the second point cloud based on the point cloud dataset specifically includes: Select any two point clouds with an overlap of 30% to 60% from the point cloud dataset, and perform data augmentation on the two point clouds respectively to obtain the first point cloud and the second point cloud.

6. The point cloud registration method based on deep learning according to claim 1, characterized in that, Both the encoder network and the decoder network use KPConv convolution, and the encoder network and the decoder network are connected across layers using U-Net.

7. A point cloud registration system based on deep learning, characterized in that, The point cloud registration system includes: A dataset generation module is used to acquire lidar data and generate a point cloud dataset based on the lidar data. The point cloud determination module is used to determine a first point cloud and a second point cloud based on the point cloud dataset; the first point cloud and the second point cloud respectively represent lidar data from different frames; The geometric descriptor and key score determination module is used to determine the geometric descriptor and key score of each point in the first point cloud and the second point cloud using a trained Siamese neural network. Specifically, the trained Siamese neural network includes an encoder network, a mutual attention network, and a decoder network. The encoder network is used to downsample the first point cloud and the second point cloud respectively to obtain a first local shape feature and a second local shape feature. The mutual attention network is used to exchange information between the first local shape feature and the second local shape feature to obtain a first fusion feature and a second fusion feature. The decoder network is used to upsample the first fusion feature and the second fusion feature respectively to obtain the geometric descriptor and key score of each point in the first point cloud and the second point cloud. The matching pair determination module is used to determine a number of matching pairs from the first point cloud and the second point cloud based on the geometric descriptor and the key score; the matching pair represents a one-to-one correspondence between points in the first point cloud and points in the second point cloud; The confidence determination module is used to determine the confidence of each pair of matches using a trained pair consistency learning network. The transformation matrix calculation module is used to calculate the transformation matrix between the first point cloud and the second point cloud based on the confidence levels. The point cloud registration module is used to register the first point cloud and the second point cloud according to the transformation matrix to obtain a registered point cloud; the registered point cloud is used to construct a point cloud map of the lidar scanning area.

8. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the point cloud registration method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the point cloud registration method as described in any one of claims 1 to 6.

Images