A method and device for spatial target point cloud registration based on neural network

Abstract

The present application discloses a method and device for spatial target point cloud registration based on a neural network, which obtains a target point cloud data set of a target object to be registered, and then uses a pre-trained point cloud registration neural network model to process the target point cloud data set to obtain the target motion truth value of the target object to be registered. Among them, the point cloud registration neural network model is trained using a point cloud data sample set of a training object annotated with a motion truth value label, and the target motion truth value is the relative motion of the target object to be registered under different viewing angles. Finally, the structure of the target object to be registered is determined according to the target motion truth value to complete the registration process of the spatial target point cloud. This solution can use the trained point cloud registration neural network model to obtain the accurate target rotation matrix and target translation vector of the target object to be registered, which can reduce the error of point cloud registration and improve the registration rate.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional reconstruction and computer vision, and specifically to a method and device for spatial target point cloud registration based on a neural network. Background Art

[0002] In recent years, many countries and regions have been conducting research in the field of three-dimensional reconstruction and computer vision, specifically involving the research of space-based space target detection and recognition systems. This system can combine the advantages of optical systems and radar systems, and use point cloud registration methods to perform high-precision orbit determination and high-resolution imaging of the target, thereby obtaining the target's architecture.

[0003] The existing point cloud registration method accumulates each frame of point cloud data according to the posture transformation parameters between each frame of point cloud data and every two adjacent frames of point cloud data. However, when registering every two adjacent frames of point cloud data, registration errors may occur due to the presence of noise data, resulting in inaccurate registration results. Summary of the invention

[0004] In view of this, the present application provides a spatial target point cloud registration method and device based on a neural network, which is used to solve the problem that the existing point cloud registration method has registration errors due to the presence of noisy data and other reasons, resulting in inaccurate registration results.

[0005] To achieve the above objectives, the proposed solution is as follows:

[0006] In a first aspect, a spatial target point cloud registration method based on a neural network comprises:

[0007] Acquire a target point cloud data set of the target object to be registered; use a pre-trained point cloud registration neural network model to process the target point cloud data set to obtain a target motion truth value of the target object to be registered, wherein the point cloud registration neural network model is trained using a point cloud data sample set of a training object annotated with a motion truth value label, and the target motion truth value is the relative motion of the target object to be registered under different viewing angles; determine the structure of the target object to be registered according to the target motion truth value, and complete the registration process of the spatial target point cloud.

[0008] Preferably, the pre-trained point cloud registration neural network model is used to process the target point cloud data set to obtain the target motion true value of the target object to be registered, including: using the point cloud feature extraction module of the point cloud registration neural network model to determine the feature vector of the target point cloud data set; using the point cloud matching point prediction module of the point cloud registration neural network model to perform weight prediction on the feature vector to obtain the weight vector of the target point cloud data; using the target motion true value determination module of the point cloud registration neural network model to multiply the weight vector with a preset objective function to obtain a first objective function, and construct a second objective function, and obtain the target motion true value of the target object to be registered according to the first objective function and the second objective function.

[0009] Preferably, the point cloud matching point prediction module includes an encoding module and a decoding module; the encoding module includes three convolutional layers connected in sequence, each of which includes a convolution operation unit, a batch normalization operation unit and a residual unit connected in sequence; the decoding module includes two deconvolution modules and a random dropout module connected in sequence, each of which includes a deconvolution operation unit, a batch normalization operation unit and a residual unit connected in sequence; the random dropout module includes a deconvolution operation unit and a random dropout unit connected to each other.

[0010] Preferably, the step of obtaining a target point cloud data set of a target object to be registered includes:

[0011] The target point cloud data of the target object to be registered under multiple different viewing angles are obtained, and the target point cloud data under every two viewing angles among the target point cloud data under the multiple different viewing angles are taken as a group of target point cloud data to obtain multiple groups of target point cloud data and form a target point cloud data set.

[0012] Preferably, the target motion true value includes a target rotation matrix and a target translation vector, and obtaining the target motion true value of the target object to be registered according to the first objective function and the second objective function includes: defining the point clouds of two perspectives in a set of target point cloud data as and ,and , the weight vector is defined as , the first objective function is defined as:

[0013] ,

[0014] The second objective function is defined as:

[0015] ,

[0016] in, represents the trace of the matrix, represents the target rotation matrix, represents the target translation vector, Indicates that there are a total of Group target point cloud data, , , , , , , , , represents a diagonal matrix, represents the matrix, represents the matrix, represents the filter function, , represents the loss function, represents a positive integer; according to the first objective function and the second objective function, the target rotation matrix is ​​obtained as follows: , the target translation vector is .

[0017] Preferably, the method for acquiring the point cloud data sample set includes:

[0018] Obtain point cloud data samples of the training object at multiple different perspectives; use point cloud data samples of every two perspectives in the point cloud data samples at multiple different perspectives as a group of point cloud data samples; calculate the point cloud overlap rate of the two perspectives in each group of point cloud data samples; delete the point cloud data sample group whose point cloud overlap rate is less than a first threshold, and obtain multiple groups of point cloud data samples whose point cloud overlap rate is greater than or equal to the first threshold, and form a point cloud data sample set.

[0019] Preferably, the training process of the point cloud registration neural network model includes:

[0020] The point cloud data samples in the point cloud data sample set are input into the point cloud registration neural network model; the sample feature vector of the point cloud data sample is determined by using the point cloud feature extraction module of the point cloud registration neural network model; the sample feature vector is weight predicted by using the point cloud matching point prediction module of the point cloud registration neural network model to obtain the sample weight vector of the point cloud data sample; the target motion true value determination module of the point cloud registration neural network model is used to multiply the weight vector with the preset sample objective function to obtain the first sample objective function, and construct the second sample objective function, and obtain the motion prediction value of the point cloud data sample according to the first sample objective function and the second sample objective function; the point cloud registration neural network model is trained with the goal that the predicted motion value is close to the motion true value label corresponding to the point cloud data sample.

[0021] In a second aspect, a spatial target point cloud registration device based on a neural network includes: a memory and a processor;

[0022] The memory is used to store programs;

[0023] The processor is used to execute the program to implement the various steps of the neural network-based spatial target point cloud registration method as described in the first aspect.

[0024] It can be seen from the above technical scheme that the present application obtains the target point cloud data set of the target to be registered, and processes the target point cloud data set using a pre-trained point cloud registration network model to obtain the target motion truth value of the target object to be registered, wherein the point cloud registration neural network model is pre-trained using point cloud data samples of training objects annotated with motion truth value labels, and the target motion truth value is the relative motion of the target object to be registered under different viewing angles. This scheme can obtain a network model that can accurately output the target rotation matrix and target translation vector by constructing a point cloud registration neural network model and training it, thereby inputting the target point cloud data set into the network model and outputting the target rotation matrix and target translation vector, which can reduce the error of point cloud registration and improve the registration rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 An optional flow chart of a spatial target point cloud registration method based on a neural network provided in an embodiment of the present application;

[0026] Figure 2 A schematic diagram of the structure of a point cloud registration neural network model provided in an embodiment of the present application;

[0027] Figure 3a-3c A schematic diagram of a point cloud of an aircraft model provided in an embodiment of the present application at different viewing angles;

[0028] Figure 4a-4c Physical pictures of satellite models, aircraft models and ship models provided in the embodiments of the present application;

[0029] Figure 5a-5c A schematic diagram of a point cloud of a satellite model, an aircraft model, and a ship model provided in an embodiment of the present application;

[0030] Figure 6a and Figure 6b They are respectively schematic diagrams of point clouds of the satellite model provided in the embodiments of the present application before and after registration at a rotation angle within the range of 0-10°;

[0031] Figure 7a and Figure 7bThey are respectively schematic diagrams of point clouds of the satellite model provided in the embodiments of the present application before and after registration at a rotation angle within the range of 0-30°;

[0032] Figure 8 A schematic diagram of a point cloud after registration of an aircraft model provided in an embodiment of the present application;

[0033] Fig. 9 A schematic diagram of a point cloud after registration of a ship model provided in an embodiment of the present application;

[0034] Fig.10 A schematic diagram of an outdoor scene provided in an embodiment of the present application;

[0035] Fig.11a and Fig.11b Schematic diagrams of point clouds before and after registration of an outdoor scene provided in an embodiment of the present application;

[0036] Fig.12a and Figure 12b Schematic diagrams of point clouds of a single object before and after registration provided in the embodiments of the present application;

[0037] Fig.13a and Fig.13b Schematic diagrams of point clouds of a single object before and after registration provided in the embodiments of the present application;

[0038] Fig.14 A schematic diagram of the structure of a neural network-based spatial target point cloud registration device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0040] In the research of space-based space target detection and recognition system, the laser radar system can combine the advantages of optical system and radar system to perform high-precision orbit determination and high-resolution imaging of important targets, which is of great significance. Due to the limitations of the volume, energy and weight of the target, the advantages of photoelectric detection technology are particularly prominent, and the photoelectric detection system based on Geiger mode avalanche photodiode (Gm-APD) has single-photon sensitivity, high precision, small size and long range, and is widely used in the field of aerospace situational awareness. At present, the laser point cloud obtained by the domestic laser radar system based on Gm-APD has limited resolution, is insensitive to small objects with similar size and resolution and small angle movement, and the obtained three-dimensional point cloud has high noise, low quality and poor accuracy. Most traditional point cloud registration algorithms are based on the iterative closest point algorithm (ICP, Iterative Closest Point). This method is simple and efficient, and is the most widely used registration algorithm in engineering projects, but it has the problem of being easily trapped in local optimal values ​​and requiring coarse registration as the initial value. In recent years, many scholars have introduced deep learning methods into the problem of point cloud registration, using superpoint sets instead of key points, applying autoencoder deep neural networks to local point clouds, making full use of the geometric structure relationship of point clouds, and applying them to point cloud registration of large scenes. Some researchers have proposed an end-to-end neural network point cloud registration method DCP. DCP uses the DGCNN network to learn local geometric features by constructing a k-NN graph to achieve better registration results, but the EdgeConv neural network module in the network considers the distance between the coordinates of the point and the neighborhood points, ignores the vector direction between adjacent points, loses some local geometric information, and can only be applied to small-scale data sets. In 2019, Christopher Choy et al. proposed an end-to-end point cloud registration algorithm DGR, and designed a 6-dimensional convolutional neural network based on ResUNet to predict matching point pairs. However, this method is applied to the 3DMatch large-scale indoor scene and KITTI autonomous driving scene datasets. The registration accuracy of multiple views of a single object is not high, especially for small targets with low resolution. In addition, DGR has a high misregistration rate for scenes with low overlap and similarity.

[0041] Based on the above defects, the present application embodiment provides a spatial target point cloud registration solution based on a neural network. Figure 1 The registration method of this application is described as follows: Figure 1 As shown, the method includes:

[0042] S1: Obtain a target point cloud dataset of the target object to be registered.

[0043] The laser radar system can be used to obtain point cloud data of the target object to be registered, and the laser radar system is placed at a certain distance from the target to be registered, so as to obtain some point cloud data of the target object to be registered. The laser radar system can use Gm-APD.

[0044] S2: Using the pre-trained point cloud registration neural network model, the target point cloud data set is processed to obtain the target motion true value of the target object to be registered.

[0045] The point cloud registration neural network model is obtained by training a point cloud data sample set of a training object annotated with a motion truth value label, and the target motion truth value is the relative motion of the target object to be registered under different viewing angles.

[0046] In the present application, we can first obtain some point cloud data sets of training objects of the same type as the target object to be registered in the open source data set according to the type of the target object to be registered, as the point cloud data sample set, these point cloud data samples are annotated with the motion truth labels of the corresponding training objects, and use the point cloud data sample set to train the point cloud registration neural network model. After the training is completed, the target point cloud data set of the target object to be registered is input into the trained point cloud registration neural network model, and the target motion truth value of the target object to be registered is output.

[0047] S3: Determine the structure of the target object to be registered according to the target motion true value, and complete the registration process of the spatial target point cloud.

[0048] The true motion value is the relative motion of the target object to be registered under different viewing angles. The specific model architecture of the target to be registered can be determined based on its true motion value.

[0049] It can be seen from the above technical scheme that the present application obtains the target point cloud data set of the target to be registered, and processes the target point cloud data set using a pre-trained point cloud registration network model to obtain the target motion truth value of the target object to be registered, wherein the point cloud registration neural network model is pre-trained using point cloud data samples of training objects annotated with motion truth value labels, and the target motion truth value is the relative motion of the target object to be registered under different viewing angles. This scheme can obtain a network model that can accurately output the target rotation matrix and target translation vector by constructing a point cloud registration neural network model and training it, thereby inputting the target point cloud data set into the network model and outputting the target rotation matrix and target translation vector, which can reduce the error of point cloud registration and improve the registration rate.

[0050] Optionally, for step S1, a target point cloud dataset of the target object to be registered is obtained, and an optional method is as follows:

[0051] The target point cloud data of the target object to be registered under multiple different viewing angles are obtained, and the target point cloud data under every two viewing angles in the target point cloud data under multiple different viewing angles are taken as a group of target point cloud data to obtain multiple groups of target point cloud data and form a target point cloud data set.

[0052] For example, the target object to be registered is a satellite model with a size of 20cm×50cm×5cm. Since the model generally has a turntable, the turntable can be cut off when processing the target point cloud data later, leaving only the satellite part, and the Gm-APD laser radar system is placed about 40m away from the satellite model. The turntable is used to control the target to rotate along the central axis, with an interval of 5°, and 13 point clouds of the satellite model are selected when rotating from -30° to 30°, with a point cloud scale of 1500-3200 points.

[0053] If a target to be registered is a combination of an airplane and a box, with a size of 140cm×40cm×10cm, the box can be cut off when processing the point cloud data later, and only the airplane part can be retained as much as possible. The Gm-APD laser radar system is placed about 45m away from the satellite model. Similarly, the turntable is used to control the target to rotate along the central axis, with an interval of 5°, and 13 point clouds of the satellite model are selected when rotating from -30° to 30°, and the point cloud scale is 10,000-14,000 points.

[0054] Specifically, for step S2, an end-to-end ResUNet-based point cloud registration neural network model can be constructed. The target point cloud dataset is processed using the network model to obtain the target motion true value of the target object to be registered. The specific structure is as follows: Figure 2 As shown, an optional method is as follows:

[0055] S21: Determine the feature vector of the target point cloud data set by utilizing the point cloud feature extraction module of the point cloud registration neural network model.

[0056] The point cloud feature extraction module can be constructed using a sparse fully convolutional network point cloud feature descriptor (FCGF), which is a U-Net-like network architecture including an encoding module and a decoding module, wherein both the encoding module and the decoding module include a residual structure, and the encoding module and the decoding module enhance the expression capability of point cloud features through jump connections.

[0057] Select a set of point clouds of the target point cloud data, which contains point cloud data from two perspectives. The point clouds from these two perspectives are called source point clouds. and destination point cloud , which are a pair of corresponding points, see Figure 2, the source point cloud and the destination point cloud are taken as point cloud 1 and point cloud 2 respectively, and then input into the point cloud feature extraction module for feature extraction, and two feature vectors can be obtained. and , whose dimensions are all 32, and then the nearest neighbor method k-nn is used to calculate the matching points (i.e. corresponding points), which can generate a set of hypothetical corresponding point pairs index set as , that is, multiple 3D+3D matching point pairs are obtained. Indicates that there is For corresponding points.

[0058] S22: Utilizing the point cloud matching point prediction module of the point cloud registration neural network model, weight prediction is performed on the feature vector to obtain a weight vector of the target point cloud data.

[0059] Specifically, the point cloud matching point prediction module can also be implemented using the ResUnet network architecture.

[0060] In this step, the feature vector obtained in the above step S21 is input into the point cloud matching point prediction module, the point cloud matching point prediction module includes an encoding module and a decoding module, the encoding module includes three 6D convolutional layers, and the point cloud matching point prediction module includes an encoding module and a decoding module;

[0061] Among them, the encoding module includes three convolution units connected in sequence, each convolution unit includes a convolution operation layer (Conv), a batch normalization operation layer (BN) and a residual layer (ResBlock) connected in sequence; the decoding module includes two deconvolution units and a random dropout unit connected in sequence, each deconvolution unit includes a deconvolution operation layer, a batch normalization operation layer and a residual layer connected in sequence; the random dropout unit includes a deconvolution operation layer and a random dropout layer (dropout layer) connected to each other. Specifically, each residual unit includes two convolution operation layers and two batch normalization operation layers. The use of the Maxout activation function can improve the nonlinear description ability and better fit the point cloud registration neural network model. Its definition is as follows:

[0062] ,

[0063] The final output channel number (dimension) is 64, and the decoding module of the point cloud matching point prediction module in this application includes six symmetrical deconvolution modules (TransConv), which can reduce the number of channels in the feature space to 16 and transmit the output weight vector (predict weight), also called tensor, to the next module.

[0064] Optionally, in existing neural networks, nonlinear functions such as tanh, sigmoid, and ReLU (rectified linear unit) are generally used to improve the expressive power of the model. Since the ReLU function is used as an activation function, the negative gradient will be set to zero, and the neuron may never be activated by any data again. In order to avoid the "extinction" of a large number of neurons, the Maxout activation function and the random inactivation layer (dropout layer) are introduced in the point cloud registration neural network model in the example provided in this application. The Maxout activation function is a learnable piecewise linear function. Compared with ReLU, sigmoid, etc., its special feature is that k neurons are added and then the value with the maximum activation value is output. Maxout has a very strong fitting ability and is linear and unsaturated, which can avoid the death of neurons, but the function will cause a surge in the number of overall parameters. The combined use of Maxout and Dropout layers can achieve better results and obtain better registration results. Therefore, by adding a Dropout layer to the point cloud registration neural network model, some neurons can be discarded, so that only part of the network structure is updated each time, and efficient point cloud registration neural network model averaging is achieved.

[0065] S23: Utilizing the target motion true value determination module of the point cloud registration neural network model, the weight vector is multiplied by a preset objective function to obtain a first objective function, and a second objective function is constructed to obtain the target motion true value of the target object to be registered based on the first objective function.

[0066] In the target motion true value determination module (WeightedSVD), an optimal mathematical model is established, and the singular value decomposition (SVD) method based on weighted practicum decomposition is used to establish the objective function, and the target motion value of the target object to be registered is obtained using the objective function. and .

[0067] Optionally, the target motion true value includes a target rotation matrix and a target translation vector.

[0068] Specifically, the process of target point cloud registration is to solve the source point cloud and destination point cloud The rigid transformation matrix between , so that the source point cloud Transform to the target point cloud In the same coordinate system, that is: find the source point cloud and destination point cloud The relative motion between them can be expressed as:

[0069]

[0070] in, represents the rotation matrix, Represents the translation vector. Source point cloud With the destination point cloud The following equation is satisfied:

[0071]

[0072] Among them, assuming Represents the index of the matching point pair. The objective function can be established as follows, that is, the preset objective function:

[0073] ,

[0074] The objective function is to calculate the sum of the squares of the distances between all corresponding points, and finally obtain the rotation matrix and translation vector corresponding to the minimum objective function, which can also be expressed as:

[0075] ,

[0076] Restoring relative motion using the three-dimensional coordinates of known corresponding point pairs is a least squares estimation problem. In this application, the weight vector can be used to solve the objective function, that is, the weight vector is multiplied by the preset objective function to obtain the following first objective function:

[0077]

[0078] in, represents the weight vector;

[0079] This formula can be solved by weighted Platts analysis method, that is, the first objective function can be expressed as:

[0080] ,

[0081] in, represents the trace of the matrix, represents the target rotation matrix, represents the target translation vector, Indicates that there are a total of Group target point cloud data, , , , , , , , , represents a diagonal matrix, represents the matrix, represents a matrix;

[0082] Specifically, a robust second objective function can be constructed to improve the registration accuracy. Because when training the point cloud registration neural network model, if the gradient is too large, the minimum value will be missed. Therefore, the Huber loss function is used here to reduce the gradient near the minimum value, which can be more robust to outliers. The second objective function is defined as:

[0083]

[0084] in, represents the filter function, , when the point cloud registration neural network model outputs a bounded logit score, weights less than the threshold τ can be filtered; Represents the Huber loss function.

[0085] Therefore, the target rotation matrix can be obtained from the above first objective function and second objective function: , the target translation vector is .

[0086] Optionally, before training the point cloud registration neural network model, a point cloud data sample set is first obtained, and the method for obtaining the sample set may include the following steps:

[0087] Get point cloud data samples of the training object from multiple different perspectives. For example, get 30 aircraft CAD models from the existing open source dataset (ModelNet40), generate a bounding sphere for each aircraft sample, and then get point cloud data samples of each aircraft sample from 12 perspectives.

[0088] The point cloud data samples at every two perspectives among the point cloud data samples at multiple different perspectives are taken as a group of point cloud data samples.

[0089] Because there is no overlap or the overlap area is very small between some perspectives, which affects the registration rate, it is necessary to calculate the point cloud overlap rate of two perspectives in each group of point cloud data samples, and then delete the point cloud data sample group with a point cloud overlap rate less than the first threshold, and obtain multiple groups of point cloud data samples with a point cloud overlap rate greater than or equal to the first threshold, and form a point cloud data sample set. The first threshold can be 30%.

[0090] In addition, the point cloud data samples of 25 aircraft samples in the CAD models of these 30 aircraft can be used as the point cloud data samples actually used for training, and the point cloud data samples of the remaining 5 aircraft samples can be used as the verification set. After the training of the point cloud registration neural network model is completed, the verification set is used to verify the training results.

[0091] Furthermore, after the point cloud data sample set is obtained, the point cloud data sample set can be used to train the point cloud registration neural network model. An optional method is as follows:

[0092] The point cloud data samples in the point cloud data sample set are input into the point cloud registration neural network model.

[0093] The point cloud feature extraction module of the point cloud registration neural network model is used to determine the sample feature vector of the point cloud data sample. The point cloud matching point prediction module of the point cloud registration neural network model is used to perform weight prediction on the sample feature vector to obtain the sample weight vector of the point cloud data sample. The target motion true value determination module of the point cloud registration neural network model is used to multiply the weight vector with the preset sample objective function to obtain the first sample objective function, and construct the second objective function, and obtain the motion prediction value of the point cloud data sample according to the first sample objective function and the second objective function.

[0094] The point cloud registration neural network model is trained with the goal of predicting motion values ​​close to the motion truth labels corresponding to the point cloud data samples. Specifically, the motion truth of the training object is divided into the rotation matrix and translation vector , using the rotation matrix , translation vector The total loss function is constructed by combining the predicted motion value, and the total loss function is divided into the first loss function, the second loss function and the third loss function, wherein the first loss function is defined as:

[0095] ,

[0096] The second loss function is defined as:

[0097] ,

[0098] Order Collection The translation error is The set of corresponding point pairs within the range, also called the inliers, and the set of other matching point pairs is called the outlier set. During the training process, the likelihood is estimated Corresponding to the true value The binary cross entropy loss between is used to optimize the network parameters, so the third loss function is defined as:

[0099] ,

[0100] in, Represents the true value of relative motion.

[0101] The third loss function can determine the possibility that the corresponding point pair is an internal point, and then perform nonlinear transformation and normalization operations on the obtained probability value, using a The vector represents the weight vector of the point cloud registration neural network model.

[0102] The first loss function, the second loss function and the third loss function are linearly weighted to obtain a total loss function.

[0103] During the training process, the total loss function is used to adjust the parameters. Specifically, the first-order optimization algorithm SGD is used to minimize the above-mentioned second objective function. However, due to the small number of parameters, a high-order optimizer can also be used to minimize the above-mentioned second objective function. And because the point cloud coordinate scale is large, it is necessary to adjust the key parameters of downsampling voxelization voxel_size, iteration number epoch, learning rate lr, translation error threshold trans_thresh and rotation angle error threshold rot_thresh (this threshold is used to determine whether the registration is successful) to obtain the optimal parameters. In one embodiment of the present application, when voxel_size=0.5, epoch=100, lr=0.01, trans_thresh=15, rot_thresh=1.5, the training effect of the point cloud registration neural network model is the best, the total loss function can reach the minimum value, or the loss tends to be stable, and the final model is the trained point cloud registration neural network model.

[0104] A specific example is given below to specifically illustrate a method for registering a spatial target point cloud based on a neural network provided in the embodiment.

[0105] The point cloud data samples of the CAD models of the above 30 aircraft are called the Plane30 dataset. Taking the Plane30 dataset as an example, FIG3 is the visualization result of the Plane30 dataset.

[0106] During training, the Plane30 dataset is divided into several groups, with point cloud data from two perspectives as one group. For each group of input point clouds, a set of rotation matrices R is generated. The rotation matrix can be represented by rotation angles (Euler angles), and the rotation angles are randomly distributed in the range of 0°-360°. Figure 3a , Figure 3a It shows the overlapping views of two perspectives of an aircraft model. Figure 3b and Figure 3c Schematic diagram of the two point clouds after rotation.

[0107] FIG. 4 is a picture of the acquired physical model, and FIG. 5 is a schematic diagram of a point cloud of the acquired physical model.

[0108] Next, taking the satellite model, the aircraft model and the ship model as examples, the ICP method and the registration method provided in the embodiment of the present application are used for registration, and the registration results are compared.

[0109] Let's take the satellite model as an example. The specific steps are as follows:

[0110] refer to Figure 4a and Figure 5a Considering that the satellite model rotates around the central axis, theoretically, only the Euler angle in the central axis direction is about 5 degrees, and the other directions should be close to 0. For the satellite model, using the ICP method (Iterative Closest Point Algorithm), the registration results at intervals of 5°, 10°, and 15° are as follows:

[0111] 1) The registration results with an interval of 5° are as follows:

[0112] -20 degrees => -25 degrees Euler angle: 0.83

[0113] -15 degrees => -20 degrees Euler angle: 0.39

[0114] -10 degrees => -15 degrees Euler angle is: 0.33

[0115] -5 degrees => -10 degrees Euler angle is: 1.10

[0116] 0 degrees => -5 degrees The Euler angle is: 0.84

[0117] 5 degrees => 0 degrees Euler angle is: 0.12

[0118] 10 degrees => 5 degrees Euler angle: 0.56

[0119] 15 degrees => 10 degrees Euler angle is: 0.61

[0120] 20 degrees => 15 degrees Euler angle: 0.95

[0121] 25 degrees => 20 degrees Euler angle: 0.57

[0122] 30 degrees => 25 degrees Euler angle: 0.83

[0123] 2) The registration results at intervals of 10° are as follows:

[0124] -15 degrees => -25 degrees Euler angle: 9.74

[0125] -5 degrees => -15 degrees The Euler angle is: 6.18

[0126] 5 degrees => -5 degrees Euler angle is: 3.27

[0127] 15 degrees => 5 degrees Euler angle is: 5.08

[0128] 25 degrees => 15 degrees Euler angle is: 9.51

[0129] 3) The registration results at intervals of 15° are as follows:

[0130] -10 degrees => -25 degrees Euler angle is: 14.70

[0131] 5 degrees => -10 degrees The Euler angle is: 5.52

[0132] 20 degrees => 5 degrees Euler angle is: 10.20

[0133] Since the current distance resolution of the lidar is only 1.7cm, and the width of the satellite model being measured is 50cm, when the satellite model rotates along the central axis, its maximum displacement is 25cm*sin(5°)=2.18cm. This displacement can easily be covered by the measurement error, resulting in an error in the estimation of the rotation angle.

[0134] From the above results, it can be seen that when the rotation of the satellite model is greater than 10 degrees, the probability of the registration error being within 1° is very low, that is, many Euler angles are less than 9°; and when the target rotation angle is 5 degrees, the Euler angle is less than 1°, which is far from 5°. Therefore, it can be considered that when the rotation angle of the satellite model is very small, the registration rate using the ICP method for registration is very low.

[0135] If the registration method provided in the embodiment of the present application is used for registration, the registration result within the range of 0-10° is shown in FIG6 . Figure 6a Before alignment, Figure 6b After registration; the registration result in the range of 0-30° is shown in Figure 7. Figure 7a Before registration, Figure 7b After registration, the specific registration results are as follows:

[0136] Taking the spatial rectangular coordinate system as an example, the point cloud coordinate range of the satellite model is xyz: 0~[0.3 0.3 40] (unit: m). The point cloud registration results at 0°~30° are shown in Table 1 below. The first row and the first column in the table are angle values. The lower left triangle is the registration time (s), and the upper right triangle is the rotation angle (degrees) obtained by registration and the corresponding voxel size (m, bold).

[0137] Table 1

[0138]

[0139] As shown in Table 1, since the number of point clouds in each group is different, the number of corresponding points calculated is also different. By continuously trying different voxel sizes, the result that minimizes the Euler angle error of the rotation is selected as the optimal result. As can be seen from Table 1, the registration result obtained by using the registration method provided in the embodiment of the present application can control the registration error within 1°, and the registration rate is relatively high.

[0140] Next, take the aircraft model as an example. Figure 4b and Figure 5b , respectively use the ICP method and the registration method provided in the embodiment of the present application to perform registration, and compare the registration results, as follows:

[0141] Please refer to Table 2 and Table 3. Table 2 is the error analysis table obtained by using the ICP method when the rotation angle is 5° intervals. Table 3 is the error analysis table obtained by using the ICP method when the rotation angle is 20° intervals:

[0142] Table 2

[0143]

[0144] Table 3

[0145]

[0146] Take the rotation angle of 30° as an example. Figure 8 is the result after the aircraft model is registered. The registration results of other angles are shown in Table 4 below, where GT_Rot ​​deg is the actual rotation angle of the aircraft model, Best result is the best result, Minavg rot err is the minimum rotation error, and voxel is the size of the volume element corresponding to the minimum error.

[0147] Table 4

[0148]

[0149] It can be seen from Table 4 that for the aircraft model, in the range of voxel values ​​[0.011, 0.045], the average rotation angle error can reach about 0.154°, and its registration accuracy is much higher than that of the ICP method.

[0150] For ship models, such as Figure 4c and Figure 5c As shown, Fig. 9 The ship model can be placed on a rotatable display stand, and the ship model can be scanned by a laser, rotating 30° each time, and the registration method provided in the embodiment of the present application can be used for registration. The schematic diagram of the point cloud after registration is as follows: Figure 5cAs shown, the specific registration results are as follows:

[0151] Relative motion obtained by registration for:

[0152]

[0153] That is, the rotation angle is 28.348734°, while the true value of the rotation angle is 30°, and the rotation error is 1.651265°. It can be concluded that the registration accuracy using the registration method provided in the embodiment of the present application is higher.

[0154] In another embodiment of the present application, the registration method provided in the present application is also applicable to multiple scenes, such as complex outdoor scenes, where scenes of road edges and trees are obtained, such as Fig.10 , Fig.11a and Fig.11b As shown; it also applies to single objects in the Stanford 3D scanning dataset, such as Fig.12a , Figure 12b , Fig.13a , Fig.13b shown.

[0155] Furthermore, the embodiment of the present application provides a spatial target point cloud registration device based on a neural network. Optionally, Fig.14 The hardware structure diagram of the spatial target point cloud registration device based on neural network is shown. Fig.14 The hardware structure of the device may include: at least one processor 01, at least one communication interface 02, at least one memory 03 and at least one communication bus 04.

[0156] In the embodiment of the present application, the number of the processor 01 , the communication interface 02 , the memory 03 , and the communication bus 04 is at least one, and the processor 01 , the communication interface 02 , and the memory 03 communicate with each other through the communication bus 04 .

[0157] The processor 01 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0158] The memory 03 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), etc., such as at least one disk memory.

[0159] Among them, the memory stores a program, and the processor can call the program stored in the memory, and the program is used to execute the neural network-based spatial target point cloud registration method described in the method embodiment.

[0160] Optionally, the refined and extended functions of the program may refer to the description of the spatial target point cloud registration method based on neural network in the method embodiment.

[0161] In addition, the functional modules in the various embodiments of the present disclosure can be integrated together to form an independent part, or each module can exist separately, or two or more modules can be integrated to form an independent part. If the function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a live broadcast device, or a network device, etc.) to perform all or part of the steps of the methods of the various embodiments of the present disclosure.

[0162] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0163] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0164] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A spatial target point cloud registration method based on neural network, characterized in that: include: Obtain a target point cloud dataset of the target object to be registered; Using a pre-trained point cloud registration neural network model, the target point cloud data set is processed to obtain a target motion truth value of the target object to be registered, wherein the point cloud registration neural network model is trained using a point cloud data sample set of a training object annotated with a motion truth value label, and the target motion truth value is the relative motion of the target object to be registered under different viewing angles; Determine the structure of the target object to be registered according to the target motion true value, and complete the registration process of the spatial target point cloud; The target motion true value includes the target rotation matrix and the target translation vector. The point clouds of two perspectives in a set of target point cloud data are defined as and ,and , the weight vector is defined as , the first objective function is defined as: ， The second objective function is defined as: ， in, represents the trace of the matrix, represents the target rotation matrix, represents the target translation vector, Indicates that there are a total of Group target point cloud data, , , , , , , , , represents a diagonal matrix, represents the matrix, represents the matrix, represents the filter function, , when the point cloud registration neural network model outputs a bounded logit score, filter the weights less than the threshold τ, represents the loss function, represents a positive integer; According to the first objective function and the second objective function, the target rotation matrix is ​​obtained as follows: , the target translation vector is .

2. The method according to claim 1, characterized in that The method of using the pre-trained point cloud registration neural network model to process the target point cloud data set to obtain the target motion true value of the target object to be registered includes: Determine the feature vector of the target point cloud data set by using the point cloud feature extraction module of the point cloud registration neural network model; Using the point cloud matching point prediction module of the point cloud registration neural network model, weight prediction is performed on the feature vector to obtain a weight vector of the target point cloud data; The target motion true value determination module of the point cloud registration neural network model is used to multiply the weight vector by a preset objective function to obtain a first objective function, and a second objective function is constructed. The target motion true value of the target object to be registered is obtained according to the first objective function and the second objective function.

3. The method according to claim 2, characterized in that The point cloud matching point prediction module includes an encoding module and a decoding module; The encoding module includes three convolution units connected in sequence, each convolution unit includes a convolution operation layer, a batch normalization operation layer and a residual layer connected in sequence; The decoding module includes two deconvolution units and a random dropout unit connected in sequence, each of the deconvolution units includes a deconvolution operation layer, a batch normalization operation layer and a residual layer connected in sequence; the random dropout unit includes a deconvolution operation layer and a random dropout layer connected to each other.

4. The method according to claim 2, characterized in that: The step of obtaining a target point cloud dataset of a target object to be registered comprises: The target point cloud data of the target object to be registered under multiple different viewing angles are obtained, and the target point cloud data under every two viewing angles among the target point cloud data under the multiple different viewing angles are taken as a group of target point cloud data to obtain multiple groups of target point cloud data and form a target point cloud data set.

5. The method according to claim 1, characterized in that The method for obtaining the point cloud data sample set includes: Acquire point cloud data samples of the training object at multiple different viewing angles; taking point cloud data samples at every two perspectives among the point cloud data samples at multiple different perspectives as a group of point cloud data samples; Calculate the point cloud overlap rate of two perspectives in each set of point cloud data samples; The point cloud data sample groups whose point cloud overlap rate is less than the first threshold are deleted to obtain multiple groups of point cloud data samples whose point cloud overlap rate is greater than or equal to the first threshold and form a point cloud data sample set.

6. The method according to claim 1, characterized in that The training process of the point cloud registration neural network model includes: Inputting the point cloud data samples in the point cloud data sample set into the point cloud registration neural network model; Determine a sample feature vector of the point cloud data sample by using a point cloud feature extraction module of the point cloud registration neural network model; Using the point cloud matching point prediction module of the point cloud registration neural network model, weight prediction is performed on the sample feature vector to obtain a sample weight vector of the point cloud data sample; Utilizing the target motion true value determination module of the point cloud registration neural network model, the weight vector is multiplied by a preset sample objective function to obtain a first sample objective function, and a second sample objective function is constructed, and a motion prediction value of the point cloud data sample is obtained according to the first sample objective function and the second sample objective function; With the goal of making the predicted motion value close to the motion true value label corresponding to the point cloud data sample, a point cloud registration neural network model is trained.

7. A spatial target point cloud registration device based on a neural network, characterized in that: include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement the various steps of the neural network-based spatial target point cloud registration method as described in any one of claims 1 to 6.

Images