Multi-view radar point cloud registration system

Abstract

This invention relates to the field of deep learning technology, and more particularly to a multi-view radar point cloud registration system. The system includes: an encoder, which extracts multi-view point cloud features from an input incomplete point cloud; a point registration module, which uses a gated loop unit configured with confidence gating to dynamically filter the multi-view point cloud features to complete point cloud registration, obtaining point cloud registration features; a point bias module, which uses an attention mechanism to perform bias optimization on the point cloud registration features, determines the point positions in the point cloud registration features, and outputs bias-optimized features; and a decoder, which decodes the bias-optimized features to obtain the registered output point cloud. This invention utilizes inter-point interactions and adds multiple confidence gating mechanisms to the point registration module to further control the accuracy of the reset, update, and output parts, improving the precision of point position movement. The point bias module provided by this invention further enhances the probability of point positions, increasing the registration effect.

Description

Technical Field

[0001] This invention belongs to the field of deep learning technology, and in particular relates to a multi-view radar point cloud registration system. Background Technology

[0002] Point cloud data is typically created through radar scans. However, due to adverse environmental factors, such as hardware limitations or occlusion, a single radar scan cannot create a complete point cloud of the target scene. Therefore, point cloud target data can be acquired from different perspectives, and then a complete point cloud can be obtained through registration. The task of point cloud registration is to find the spatial transformation relationship that can align point clouds from different views. Since point clouds are often incomplete in many real-world scenarios, the registration process is quite challenging.

[0003] Currently, most mainstream deep learning-based point cloud registration methods are ensemble end-to-end approaches. They feed two point clouds into a network and directly obtain the matching result through learning, without outputting intermediate features. These methods combine a deep neural network representing the correspondence between two sets of misaligned point clouds with linear or nonlinear algorithms that compute the rigid body motion transformations of the two sets of misaligned point clouds, establishing an end-to-end network for point cloud registration. While exhibiting good performance, they suffer from poor recognition of all key features extracted from point clouds and poor robustness in handling complex scenes. Outliers are easily generated in the matching correspondence, reducing the accuracy of registration.

[0004] Existing methods focus solely on matching two or more point clouds, concentrating on the registration problem. However, the real key to increased registration difficulty lies in the degradation of point cloud quality. Especially in practical applications, scenes are often complex and unpredictable, with point clouds exhibiting incomplete, distorted, and uncontrollable variations from different angles. Ideally, two point clouds should be perfectly matched point-to-point. However, due to the degraded point quality, registration between point clouds becomes problematic. Summary of the Invention

[0005] In view of this, the present invention aims to provide a multi-view radar point cloud registration system, which further enhances the low-dimensional point representation learned in the encoder-decoder architecture, and proposes two easily integrated modules to support the registration task: a point registration module and a point offset module, which greatly improves the registration accuracy.

[0006] To achieve the above objectives, the technical solution created by this invention is implemented as follows:

[0007] A multi-view radar point cloud registration system includes: an encoder, which extracts multi-view point cloud features from multiple single-view defective input points; a point registration module, which uses a gated loop unit configured with confidence gating to dynamically filter the multi-view point cloud features to complete point cloud registration and obtain point cloud registration features; a point bias module, which uses an attention mechanism to perform bias optimization on the point cloud registration features, determines the point positions in the point cloud registration features, and outputs bias-optimized features; and a decoder, which decodes the bias-optimized features to obtain the registered output point cloud.

[0008] Furthermore, the encoder and decoder adopt the KPConv encoder-decoder architecture.

[0009] Furthermore, in the point registration module: the input multi-view point cloud features and their corresponding hidden states are input into the first confidence gate, and the features processed by the first confidence gate are input into the reset gate to obtain the first feature; based on the multi-view point cloud features and their corresponding hidden states, candidate hidden states are determined, and the candidate hidden states are input into the second confidence gate to obtain the second feature; the input multi-view point cloud features and their corresponding hidden states are input into the second confidence gate, and the features processed by the first confidence gate are input into the update gate to obtain the third feature; the first feature is combined with the multi-view point cloud features and their corresponding hidden states to obtain the fused feature; based on the third feature, the second feature and the fused feature are combined to obtain the output point cloud registration feature.

[0010] Furthermore, in the point bias module: the input point cloud registration features are divided into a key matrix, a value matrix, and a query matrix; a random bias matrix is ​​generated based on the point cloud registration features; the random bias matrix is ​​fused with the key matrix, the value matrix, and the query matrix to obtain a confidence mask; the point cloud registration features are masked based on the confidence mask to obtain bias-optimized features.

[0011] Furthermore, the first confidence gated output The calculation process is as follows:

[0012]

[0013] in, This represents the weight matrix for the first confidence gate. This indicates the hidden state of the input. This represents the multi-view point cloud features of the input. This represents the sigmoid activation function.

[0014] Furthermore, the second confidence gated output The calculation process is as follows:

[0015]

[0016] in, This represents the weight matrix for the second confidence gate. Indicates the candidate hidden state. This represents the sigmoid activation function.

[0017] Furthermore, the third confidence gated output The calculation process is as follows:

[0018]

[0019] in, This represents the weight matrix for the third confidence gate. This represents the multi-view point cloud features of the input. This represents the sigmoid activation function.

[0020] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0021] This invention presents a multi-view radar point cloud registration system that addresses the fundamental challenges of point cloud registration from a novel perspective. It leverages inter-point interactions to improve point cloud quality, bringing it closer to an ideal state and thus enhancing registration accuracy. Specifically, this invention proposes a point registration module to correctly move offset points during point cloud registration and correct small-scale errors. It also proposes a point offset module to increase the likelihood of correct point placement and correct large-scale errors. This allows the point cloud registration task to be combined with other point cloud optimization processing tasks, which is more conducive to the future development of this field. Attached Figure Description

[0022] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0023] Figure 1 A schematic diagram of the multi-view radar point cloud registration system described in an embodiment of the present invention;

[0024] Figure 2 A flowchart illustrating the PMPNet point movement method described in an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of the point registration module described in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0027] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0028] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0029] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0030] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0031] like Figure 1As shown in the embodiment of the present invention, the multi-view radar point cloud registration system includes an encoder, a point registration module, a point offset module, and a decoder. The encoder extracts multi-view point cloud features from multiple input single-view residual point clouds; the point registration module uses a gated loop unit configured with confidence gating to dynamically filter the multi-view point cloud features to complete point cloud registration, obtaining point cloud registration features; the point offset module uses an attention mechanism to perform offset optimization on the point cloud registration features, determining the point positions in the point cloud registration features and outputting offset optimized features; the decoder decodes the offset optimized features to obtain the registered output point cloud.

[0032] In some embodiments, the encoder and decoder employ a KPConv encoder-decoder architecture. Point clouds are sparse structures with unordered characteristics, thus point convolution is limited by spatial localization. Thomas et al. proposed KPConv in 2019, which consists of a set of local 3D filters, using a set of kernel points instead of kernel pixels to define the region where each kernel weight is applied. Since the kernel weights are carried by points, their range of influence is defined by a correlation function. There is no limit to the number of kernel points, which overcomes the limitations of point convolution.

[0033] In the point movement module, PMPNet, used for point cloud completion tasks, mentions that the point movement module can be used to refine the position of points, utilizing GRU (Recurrent Recurrent Neural Network) structural units in recurrent neural networks (such as...). Figure 2 (As shown) The search point movement path. However, the improvement to GRU is limited to making it more suitable for point cloud processing, and the accuracy of point optimization is somewhat lacking. Point cloud registration requires more accurate point movement accuracy. Therefore, this method improves the GRU structural unit by adding three confidence levels to control the reset gate, update gate, and candidate hidden state respectively, further constraining the confidence level of the gating unit. Specifically, in some embodiments, the point registration module is as follows: Figure 3 As shown, the input multi-view point cloud features and their corresponding hidden states are input into a first confidence gate. The features processed by the first confidence gate are then input into a reset gate to obtain the first feature. Based on the multi-view point cloud features and their corresponding hidden states, candidate hidden states are determined and input into a second confidence gate to obtain the second feature. The input multi-view point cloud features and their corresponding hidden states are then input into the second confidence gate. The features processed by the first confidence gate are then input into an update gate to obtain the third feature. The first feature is combined with the multi-view point cloud features and their corresponding hidden states to obtain a fused feature. Finally, the second feature and the fused feature are combined based on the third feature to obtain the output point cloud registration feature. This invention adds additional confidence gates before both gates and the output step to further control the accuracy of the reset, update, and output parts, improving the precision of point position movement.

[0034] In some embodiments, the first confidence-gated output The calculation process is as follows:

[0035]

[0036] in, This represents the weight matrix for the first confidence gate. This represents the hidden state of the input, that is, the hidden state of the output of the gated loop unit at the previous time step. This represents the input multi-view point cloud features, which are the output features of the gated recurrent unit at the previous time step.

[0037] Second confidence gated output The calculation process is as follows:

[0038]

[0039] in, This represents the weight matrix for the second confidence gate. This indicates the candidate is in a hidden state.

[0040] Third confidence gated output The calculation process is as follows:

[0041]

[0042] in, This represents the weight matrix of the third confidence gate.

[0043] In some embodiments, the point bias module divides the input point cloud registration features into a key matrix, a value matrix, and a query matrix; generates a random bias matrix based on the point cloud registration features; fuses the random bias matrix with the key matrix, value matrix, and query matrix to obtain a confidence mask; and performs masking processing on the point cloud registration features based on the confidence mask to obtain bias-optimized features. In this embodiment, the point bias module also performs a linear transformation between the key matrix, value matrix, and query matrix, that is, fuses the random bias matrix with the key matrix, value matrix, and query matrix obtained from the linear transformation between matrices to obtain a confidence mask. The confidence mask is used to control the accuracy of the input signal to obtain bias-optimized features.

[0044] In the point bias module, an attention mechanism is used to add random point positions. Although attention mechanisms are common in various deep learning tasks, they are usually used to enhance feature processing. This invention uses the attention mechanism to add random positions to 3D data, making it an auxiliary step in model building.

[0045] Finally, the decoder performs a decoding operation on the bias-optimized features to improve the resolution and obtain a complete multi-view point cloud.

[0046] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0047] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A multi-view radar point cloud registration system, comprising: The method comprises the following steps: an encoder extracts multi-view point cloud features from inputted multiple single-view incomplete point clouds; a point registration module uses a gated recurrent unit configured with a confidence gate to dynamically filter multi-view point cloud features to complete point cloud registration and obtain point cloud registration features; in the point registration module: input the multi-view point cloud features and their corresponding hidden states into a first confidence gate, the processed features of the first confidence gate are input into a reset gate to obtain first features; determine candidate hidden states according to the multi-view point cloud features and their corresponding hidden states, and input the candidate hidden states into a second confidence gate to obtain second features; input the multi-view point cloud features and their corresponding hidden states into the second confidence gate, and input the processed features of the first confidence gate into an update gate to obtain third features; combine the first features with the multi-view point cloud features and their corresponding hidden states to obtain fusion features; combine the second features and the fusion features according to the third features to obtain output point cloud registration features; a point biasing module uses an attention mechanism to bias and optimize the point cloud registration features to determine the positions of the points in the point cloud registration features and output biasing optimization features; in the point biasing module: divide the inputted point cloud registration features into key matrices, value matrices and query matrices; generate a random biasing matrix according to the point cloud registration features; fuse the random biasing matrix with the key matrices, the value matrices and the query matrices to obtain a confidence mask; perform mask processing on the point cloud registration features according to the confidence mask to obtain biasing optimization features; a decoder decodes the biasing optimization features to obtain registered output point clouds.

2. The multi-view radar point cloud registration system of claim 1, wherein, The encoder and the decoder use a KPConv encoder-decoder architecture.

3. The multi-view radar point cloud registration system of claim 1, wherein, first confidence-gated output The operation process is: wherein, denotes a weight matrix of the first confidence gating, denotes an input hidden state, denotes an input multi-view point cloud feature, denotes a sigmoid activation function.

4. The multi-view radar point cloud registration system of claim 1, wherein, Second confidence-gated output The operation process is: wherein, denotes a weight matrix of the second confidence gating, denotes a candidate hidden state, denotes a sigmoid activation function.

5. The multi-view radar point cloud registration system of claim 1, wherein, Third confidence-gated output The operation process is: wherein, denotes a weight matrix of the third confidence gating, denotes an input multi-view point cloud feature, denotes a sigmoid activation function.

Images