Multi-view radar point cloud registration system

Abstract

The invention relates to the technical field of deep learning, in particular to a multi-view radar point cloud registration system, which comprises an encoder for extracting multi-view point cloud features from input incomplete point cloud; the point registration module is used for performing dynamic information screening on the multi-view point cloud features by adopting a gating circulation unit configured with confidence gating to complete point cloud registration so as to obtain point cloud registration features; the point offset module is used for performing offset optimization on the point cloud registration features by adopting an attention mechanism, determining point positions in the point cloud registration features and outputting offset optimization features; and the decoder is used for decoding the offset optimization features to obtain the registered output point cloud. According to the invention, interaction among points is utilized, multiple confidence gates are added in the point registration module, the accuracy of reset, update and output parts is further controlled, and the precision of point position movement is improved; according to the point bias module provided by the invention, the possibility of the point position is further enhanced, and the registration effect is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of deep learning, and particularly relates to a multi-view radar point cloud registration system. BACKGROUND

[0002] Point cloud data is usually created by radar scanning, however, due to adverse factors of the environment, such as defects of hardware devices or occlusion, a single radar scan cannot create a complete point cloud of a target scene. Therefore, the same point cloud target data can be obtained from different perspectives, and a complete point cloud can be obtained through registration. The task of point cloud registration is to find a spatial transformation relationship that can align point clouds of different views. Since in many real scenes, point clouds are often relatively missing, the registration process is relatively difficult.

[0003] At present, the mainstream point cloud registration method based on deep learning is more of an integrated end-to-end method. Two point clouds are input into the network, and the matching result is directly obtained through learning, without outputting intermediate features. These methods combine a deep neural network representing the corresponding relationship between two sets of unaligned point clouds with a linear or nonlinear algorithm for calculating the rigid body motion transformation of two sets of unaligned point clouds, and establish an end-to-end network for point cloud registration. Although it has good performance, the recognition ability of all key features extracted from the point cloud and the robustness of processing complex scenes are poor, and the matching correspondence is prone to abnormal values, which reduces the accuracy of registration.

[0004] Existing methods only focus on the matching between two or more point clouds, and focus attention on the registration problem. However, in fact, the real key to the increase in registration difficulty lies in the decline in point cloud quality. Especially in practical applications, the scene is often complex and unpredictable, and the changes in point clouds of different angles are missing, deformed and uncontrollable. Under ideal conditions, two point clouds should be completely matched point by point. However, due to the decline in the quality of points, the registration between point clouds has problems. SUMMARY

[0005] Therefore, the present application aims to provide a multi-view radar point cloud registration system, which further enhances the low-dimensional point representation learned in the encoder-decoder architecture, and proposes two easily integrated modules to support the registration task: a point registration module and a point bias module, which greatly improves the registration accuracy.

[0006] To achieve the above purpose, the technical scheme of the present application is as follows: The application discloses a multi-view radar point cloud registration system, which comprises an encoder, a point registration module and a point biasing module.

[0007] Further, the encoder and the decoder adopt a KPConv encoder-decoder architecture.

[0008] Further, in the point registration module: the input multi-view point cloud features and the corresponding hidden states are input into a first confidence gate, the features processed by the first confidence gate are input into a reset gate to obtain first features; candidate hidden states are determined according to the multi-view point cloud features and the corresponding hidden states, and the candidate hidden states are input into a second confidence gate to obtain second features; the input multi-view point cloud features and the corresponding hidden states are input into the second confidence gate, and the features processed by the first confidence gate are input into an update gate to obtain third features; the first features are combined with the multi-view point cloud features and the corresponding hidden states to obtain fusion features; the second features and the fusion features are combined according to the third features to obtain output point cloud registration features.

[0009] Further, in the point biasing module: the input point cloud registration features are divided into key matrices, value matrices and query matrices; a random biasing matrix is generated according to the point cloud registration features; the random biasing matrix is fused with the key matrices, the value matrices and the query matrices to obtain a confidence mask; the point cloud registration features are processed by the confidence mask to obtain biasing optimization features.

[0010] Further, the operation process of the first confidence gate output is as follows:

[0011] wherein, represents a weight matrix of the first confidence gate, represents an input hidden state, represents an input multi-view point cloud feature, represents a sigmoid activation function.

[0012] Further, the operation process of the second confidence gate output is as follows:

[0013] wherein, a weight matrix representing the second confidence gating, representing a candidate hidden state, representing a sigmoid activation function.

[0014] Further, the operation process of the third confidence gating output is as follows:

[0015] wherein, a weight matrix representing the third confidence gating, representing an input multi-view point cloud feature, representing a sigmoid activation function.

[0016] Compared with the prior art, the present application can achieve the following beneficial effects: The multi-view radar point cloud registration system of the present application considers the essence of the difficulty of point cloud registration from a new angle, improves the quality of the point cloud by using the interaction between points, makes it closer to the ideal state, and further improves the registration accuracy. Specifically, the present application proposes a point registration module to correctly move the position of the offset point in point cloud registration and correct the position of small-range error points; on the other hand, it proposes a point biasing module to enhance the possibility of correct point position and correct large-range error point position. Thus, the point cloud registration task can be combined with other point cloud optimization processing tasks, which is more conducive to the future development of this field. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The embodiments of the present application illustrated in the drawings and their descriptions are used to explain the present application and are not intended to limit the present application. In the drawings: Figure 1 a schematic diagram of the multi-view radar point cloud registration system according to the present application; Figure 2 a flowchart of the PMPNet point moving method according to the present application; Figure 3 a schematic diagram of the point registration module according to the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0019] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other in the case of no conflict.

[0020] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application. In addition, the terms "first", "second" and the like are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined with "first", "second" and the like can explicitly or implicitly include one or more of the features. In the description of the present application, unless otherwise stated, the meaning of "a plurality of" is two or more.

[0021] In the description of the present application, it should be noted that, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood through specific circumstances.

[0022] The present application will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0023] As Figure 1 shown, the multi-view radar point cloud registration system described in the embodiments of the present application includes an encoder, a point registration module, a point biasing module and a decoder. Among them, the encoder extracts multi-view point cloud features from the input multiple single-view incomplete point clouds; the point registration module uses a gated recurrent unit configured with a confidence gate to dynamically filter the multi-view point cloud features to complete point cloud registration and obtain point cloud registration features; the point biasing module uses an attention mechanism to optimize the bias of the point cloud registration features to determine the position of the points in the point cloud registration features and output the bias optimization features; the decoder decodes the bias optimization features to obtain the output point cloud after registration.

[0024] In some embodiments, the encoder and the decoder employ a KPConv encoder-decoder architecture. Point cloud is a kind of sparse structure with unordered characteristics, so point convolution is limited by spatial localization. Thomas et al. proposed KPConv in 2019, which consists of a set of local 3D filters, uses a set of kernel points instead of kernel pixels, and defines the area where each kernel weight is applied. Since the kernel weight is carried by the point, its influence range is defined by the correlation function. There is no limit to the number of kernel points, which can overcome the limitations of point convolution.

[0025] In the point moving module, it is mentioned in PMPNet for point cloud completion task that the point moving module can refine the position of the point, and the GRU (Gated Recurrent Unit) structure unit in the recurrent neural network (as shown in Figure 2 ) searches for the moving path of the point. However, the improvement of GRU is only to make it more suitable for point cloud processing, and the accuracy of point optimization is lacking. Point cloud registration requires more accurate point moving accuracy. Therefore, on this basis, the present method improves the GRU structure unit and adds three confidence degrees to control the reset gate, the update gate and the candidate hidden state, further restricting the credibility of the gated unit. Specifically, in some embodiments, as shown in Figure 3 , the input multi-view point cloud feature and its corresponding hidden state are input into the first confidence gate, the feature processed by the first confidence gate is input into the reset gate, and the first feature is obtained; according to the multi-view point cloud feature and its corresponding hidden state, the candidate hidden state is determined, and the candidate hidden state is input into the second confidence gate, and the second feature is obtained; the input multi-view point cloud feature and its corresponding hidden state are input into the second confidence gate, and the feature processed by the first confidence gate is input into the update gate, and the third feature is obtained; the first feature is combined with the multi-view point cloud feature and its corresponding hidden state to obtain the fusion feature; according to the third feature, the second feature and the fusion feature are combined to obtain the output point cloud registration feature. The present application adds an additional confidence gate before the two gate and output steps to further control the accuracy of the reset, update and output parts, and improves the accuracy of point position moving.

[0026] In some embodiments, the operation process of the first confidence gate output is as follows:

[0027] wherein, represents the weight matrix of the first confidence gate, represents the input hidden state, i.e. the output hidden state of the gating recurrent unit at the last time, represents the input multi-view point cloud feature, i.e. the output feature of the gating recurrent unit at the last time.

[0028] The operation process of the second confidence gate output The operation process of the first confidence gating output is as follows:

[0029] wherein, denotes the weight matrix of the second confidence gating, denotes the candidate hidden state.

[0030] The operation process of the third confidence gating output is as follows:

[0031] wherein, denotes the weight matrix of the third confidence gating.

[0032] In some embodiments, the point biasing module divides the input point cloud registration feature into a key matrix, a value matrix and a query matrix; generates a random biasing matrix according to the point cloud registration feature; fuses the random biasing matrix with the key matrix, the value matrix and the query matrix to obtain a confidence mask; and performs mask processing on the point cloud registration feature according to the confidence mask to obtain a bias-optimized feature. In the embodiments of the present application, the point biasing module also performs linear transformation between the matrices on the key matrix, the value matrix and the query matrix, that is, the key matrix, the value matrix and the query matrix of the linear transformation between the matrices are fused with the random biasing matrix to obtain the confidence mask. The confidence mask is used to control the accuracy of the input signal to obtain the bias-optimized feature.

[0033] In the point biasing module, the attention mechanism is used to increase the random point position. Although the attention mechanism is commonly used in various tasks of deep learning, it is used to enhance the feature processing effect. However, the present application uses the attention mechanism to increase the random position for three-dimensional data, which is used to assist one link of model establishment.

[0034] Finally, the decoder performs decoding operation on the bias-optimized feature to improve the resolution and obtain a complete multi-view point cloud.

[0035] It should be understood that the various forms of flow shown above can be used to reorder, add or delete steps. For example, the steps described in the present disclosure can be executed in parallel, in sequence or in different order, as long as the desired results of the technical solutions of the present disclosure can be achieved, which is not limited herein.

[0036] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.​

Claims

1. A multi-view radar point cloud registration system, comprising: The application relates to a method for registering multi-view point clouds, comprising the following steps: an encoder extracts multi-view point cloud features from inputted multiple single-view incomplete point clouds; a point registration module adopts a gated recurrent unit configured with a confidence gate to dynamically filter information of the multi-view point cloud features to complete point cloud registration and obtain point cloud registration features; a point biasing module adopts an attention mechanism to bias and optimize the point cloud registration features, determines the positions of points in the point cloud registration features, and outputs bias-optimized features; a decoder decodes the bias-optimized features to obtain outputted registered point clouds.

2. The multi-view radar point cloud registration system of claim 1, wherein, The encoder and the decoder adopt a KPConv encoder-decoder architecture.

3. The multi-view radar point cloud registration system of claim 1, wherein, In the point registration module: inputted multi-view point cloud features and corresponding hidden states are inputted into a first confidence gate, features processed by the first confidence gate are inputted into a reset gate to obtain first features; candidate hidden states are determined according to the multi-view point cloud features and the corresponding hidden states, and the candidate hidden states are inputted into a second confidence gate to obtain second features; the inputted multi-view point cloud features and the corresponding hidden states are inputted into the second confidence gate, and features processed by the first confidence gate are inputted into an update gate to obtain third features; the first features are combined with the multi-view point cloud features and the corresponding hidden states to obtain fusion features; the second features and the fusion features are combined according to the third features to obtain outputted point cloud registration features.

4. The multi-view radar point cloud registration system of claim 1, wherein, In the point biasing module: inputted point cloud registration features are divided into key matrices, value matrices and query matrices; a random biasing matrix is generated according to the point cloud registration features; the random biasing matrix is fused with the key matrices, the value matrices and the query matrices to obtain a confidence mask; the point cloud registration features are processed by the confidence mask to obtain bias-optimized features.

5. The multi-view radar point cloud registration system of claim 3, wherein, first confidence-gated output The operation process is: wherein, denotes a weight matrix of the first confidence gating, denotes an input hidden state, denotes an input multi-view point cloud feature, denotes a sigmoid activation function.

6. The multi-view radar point cloud registration system of claim 3, wherein, Second confidence-gated output The operation process is: wherein, denotes a weight matrix of the second confidence gating, denotes a candidate hidden state, denotes a sigmoid activation function.

7. The multi-view radar point cloud registration system of claim 3, wherein, Third confidence-gated output The operation process is: wherein, denotes a weight matrix of the third confidence gating, denotes an input multi-view point cloud feature, denotes a sigmoid activation function.

Images