Point cloud registration method, device and computer equipment based on high-order Markov

Abstract

The present application relates to a point cloud registration method, apparatus, and computer device based on a high-order Markov decision algorithm. The method comprises: obtaining training samples; constructing a cyclic point cloud registration model; the cyclic point cloud registration model comprises a cross-feature embedding module and an iterative optimization module; using the cross-feature embedding module, the initial source point cloud and the target point cloud in the training samples are subjected to feature embedding and then cross-stitching to obtain a cross-stitched feature vector; using the iterative optimization module, the current rotation vector and translation vector are updated based on the current cross-stitched feature vector and the rotation vector and translation vector outputted by the previous high-order Markov decision iteration; the rigid transformation between the current source point cloud and the target point cloud is outputted based on the current rotation vector and translation vector; the current source point cloud is iteratively updated based on the rigid transformation; and when the iteration stops, the rigid transformation between the initial source point cloud and the current source point cloud is outputted as the point cloud registration result. This method achieves high registration accuracy and enhanced performance.

Description

Technical Field

[0001] The present application relates to the technical field of point cloud registration, and in particular to a point cloud registration method, apparatus, and computer equipment based on high-order Markov. Background Art

[0002] Point cloud registration is a fundamental task in computer vision, focusing on finding the rigid alignment between two point clouds. With the continuous promotion of 3D acquisition equipment, point cloud registration has been applied in many fields such as autonomous driving, augmented reality (AR), and 3D reconstruction.

[0003] Learning-based methods have shown dominant progress in the field of point cloud registration. Compared to traditional methods, learning-based methods have achieved better performance in terms of accuracy, robustness, and generalization. Specifically, most learning-based methods obtain point-level correspondences through local feature extraction and matching and use them to recover the rigid transformation. However, estimating the transformation in this way can introduce unreliable geometric information because it heavily relies on the correctness of feature correspondences, and despite the use of a variety of pruning techniques, a large number of outliers still exist.

[0004] Therefore, another line of research explores point cloud registration via a non-corresponding pipeline. Generally speaking, this type of work first extracts global features using a weight-sharing network to abstract the point cloud's structural information. These embedded global features are then concatenated to encode implicit geometric motion. Finally, a network is employed to decode the high-level features into a relative 6D transformation. In contrast, non-corresponding methods focus more on global relative transformation information rather than local geometric information. Global features are more abstract and therefore suitable for network adaptive learning.

[0005] To further improve performance, some non-corresponding methods have introduced an iterative approach. The point cloud alignment process is typically considered a first-order Markov decision process, where the rotation and translation transformations are determined entirely by the current state of the two point clouds without considering the past. This mathematical modeling ignores the temporal nature of the transformation updates during iterative adjustments. Furthermore, traditional non-corresponding methods simply extract and concatenate global features from the two point clouds to embed the current state, resulting in weak representational capabilities and a lack of relative position information. Summary of the Invention

[0006] Based on this, it is necessary to provide a point cloud registration method, device and computer equipment based on high-order Markov to address the above technical problems.

[0007] A point cloud registration method based on high-order Markov, the method comprising:

[0008] Obtain preprocessed initial source point cloud and target point cloud as training samples;

[0009] Constructing a cyclic point cloud registration model for point cloud registration; the cyclic point cloud registration model includes a cross feature embedding module and an iterative optimization module;

[0010] The cross feature embedding module performs feature embedding on the initial source point cloud and the target point cloud respectively, and then cross-splicing them to obtain a cross-splicing feature vector;

[0011] The iterative optimization module updates the current rotation vector and translation vector according to the current cross-stitching feature vector and the rotation vector and translation vector outputted by the last high-order Markov decision iteration, outputs the rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector, iteratively updates the current source point cloud according to the rigid transformation, and when a preset iteration stop condition is met, outputs the rigid transformation between the initial source point cloud and the current source point cloud as the point cloud registration result;

[0012] Training the cyclic point cloud registration model according to the training samples and the prediction results obtained by inputting the training samples into the cyclic point cloud registration model to obtain a trained cyclic point cloud registration model;

[0013] The pre-processed initial source point cloud to be registered and the target point cloud are input into the trained cyclic point cloud registration model to obtain a point cloud registration result of the initial source point cloud to be registered.

[0014] In one embodiment, it also includes: the first branch cyclic iteration layer, including a first feature encoding layer, a first recurrent neural network layer and a rotation increment estimation layer connected in sequence, for outputting a rotation increment based on the current cross-spliced feature vector and the rotation vector of the previous iteration; the rotation increment is used to update the rotation vector; the second branch cyclic iteration layer, including a second feature encoding layer, a second recurrent neural network layer and a translation increment estimation layer, for outputting a translation increment based on the current cross-spliced feature vector and the translation vector of the previous iteration; the translation increment is used to update the translation vector.

[0015] In one embodiment, it also includes: encoding the rotation vector of the previous iteration through the first feature encoding module, splicing it with the current cross-splicing feature vector after encoding to obtain a first input feature, and outputting the rotation increment after the first input feature passes through the first recurrent neural network layer and the rotation increment estimation layer.

[0016] In one embodiment, it also includes: encoding the translation vector of the previous iteration through the second feature encoding module, splicing it with the current cross-splicing feature vector after encoding to obtain a second input feature, and outputting the translation increment after the second input feature passes through the second recurrent neural network layer and the translation increment estimation layer.

[0017] In one of the embodiments, it further includes: the transformation update layer is used to update the rotation vector and the translation vector according to the rotation increment and the translation increment respectively, and output the rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector.

[0018] In one embodiment, it also includes: the cross-feature embedding module includes a feature extraction layer, a cross-stitching layer and a sliding window: the feature extraction layer extracts geometric features from the initial source point cloud and the target point cloud to obtain a pair of global feature vectors with the same network mapping relationship under the same dimension; the cross-stitching layer cross-stitches the global feature vectors to obtain a cross-stitched feature vector; the sliding window extracts the spatial relative transformation information of the cross-stitched feature vector to obtain a cross-stitched feature vector.

[0019] In one embodiment, the method further includes: acquiring an initial source point cloud set and a target point cloud set in the observation scene; and obtaining an initial source point cloud and a target point cloud to be registered based on overlapping point clouds in the initial source point cloud set and the target point cloud set.

[0020] A point cloud registration device based on high-order Markov, comprising:

[0021] Point cloud acquisition module, used to obtain pre-processed initial source point cloud and target point cloud as training samples;

[0022] A model building module, used to build a cyclic point cloud registration model for point cloud registration; the cyclic point cloud registration model includes a cross feature embedding module and an iterative optimization module;

[0023] a cross-stitching module, configured to perform feature embedding on the initial source point cloud and the target point cloud respectively through the cross-feature embedding module and then cross-stitch them to obtain a cross-stitching feature vector;

[0024] an iterative optimization module, configured to update the current rotation vector and translation vector according to the current cross-stitching feature vector and the rotation vector and translation vector iteratively outputted by the high-order Markov decision algorithm in the previous iteration, output a rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector, iteratively update the current source point cloud according to the rigid transformation, and output the rigid transformation between the initial source point cloud and the current source point cloud as a point cloud registration result when a preset iteration stop condition is met;

[0025] A model training module is used to train the cyclic point cloud registration model according to training samples and prediction results obtained by inputting the training samples into the cyclic point cloud registration model to obtain a trained cyclic point cloud registration model;

[0026] The result output module is used to input the preprocessed initial source point cloud to be registered and the target point cloud into the trained cyclic point cloud registration model to obtain the point cloud registration result of the initial source point cloud to be registered.

[0027] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0028] Obtain preprocessed initial source point cloud and target point cloud as training samples;

[0029] Constructing a cyclic point cloud registration model for point cloud registration; the cyclic point cloud registration model includes a cross feature embedding module and an iterative optimization module;

[0030] The cross feature embedding module performs feature embedding on the initial source point cloud and the target point cloud respectively, and then cross-splicing them to obtain a cross-splicing feature vector;

[0031] The iterative optimization module updates the current rotation vector and translation vector according to the current cross-stitching feature vector and the rotation vector and translation vector output by the last high-order Markov decision iteration, outputs the rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector, iteratively updates the current source point cloud according to the rigid transformation, and when the preset iteration stop condition is met, outputs the rigid transformation between the initial source point cloud and the current source point cloud as the point cloud registration result.

[0032] Training the cyclic point cloud registration model according to the training samples and the prediction results obtained by inputting the training samples into the cyclic point cloud registration model to obtain a trained cyclic point cloud registration model;

[0033] The pre-processed initial source point cloud to be registered and the target point cloud are input into the trained cyclic point cloud registration model to obtain a point cloud registration result of the initial source point cloud to be registered.

[0034] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0035] Obtain preprocessed initial source point cloud and target point cloud as training samples;

[0036] Constructing a cyclic point cloud registration model for point cloud registration; the cyclic point cloud registration model includes a cross feature embedding module and an iterative optimization module;

[0037] The cross feature embedding module performs feature embedding on the initial source point cloud and the target point cloud respectively, and then cross-splicing them to obtain a cross-splicing feature vector;

[0038] The iterative optimization module updates the current rotation vector and translation vector according to the current cross-stitching feature vector and the rotation vector and translation vector output by the last high-order Markov decision iteration, outputs the rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector, iteratively updates the current source point cloud according to the rigid transformation, and when the preset iteration stop condition is met, outputs the rigid transformation between the initial source point cloud and the current source point cloud as the point cloud registration result.

[0039] Training the cyclic point cloud registration model according to the training samples and the prediction results obtained by inputting the training samples into the cyclic point cloud registration model to obtain a trained cyclic point cloud registration model;

[0040] The pre-processed initial source point cloud to be registered and the target point cloud are input into the trained cyclic point cloud registration model to obtain a point cloud registration result of the initial source point cloud to be registered.

[0041] The above-mentioned high-order Markov-based point cloud registration method, device and computer equipment, through a cyclic point cloud registration model, utilizes the sequential nature of transformations to implicitly learn the relative spatial position between two point clouds in the cyclic registration. Specifically, the cross-feature embedding module in the cyclic point cloud registration model explores the relative transformation between the source and target point clouds, enhances the representation of the embedded global feature vector, and uses a recurrent neural network to induce high-order Marcov decisions during the iterative optimization process. In the embodiment of the present invention, when performing state representation on point cloud pairs, the representation capability is stronger, relative position information can be obtained, the registration accuracy is high, and the registration performance is stronger. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 1 is a flow chart of a point cloud registration method based on a high-order Markov algorithm in one embodiment;

[0043] Figure 2 Schematic diagram of the structure of a cyclic point cloud registration model in one embodiment;

[0044] Figure 3 Schematic diagram of qualitative comparison on ModelNet40 and ScanObjectNN datasets in one embodiment;

[0045] Figure 4 Schematic diagram of a qualitative example of the iteration state of ModelNet40 in one embodiment;

[0046] Figure 5 A qualitative example schematic diagram of iterative inference updating in another embodiment;

[0047] Figure 6 is a schematic diagram of ablation results of an ablation experiment in one embodiment;

[0048] Figure 7 A schematic diagram of the numerical results of selecting the number of sliding windows and updating in one embodiment;

[0049] Figure 8 is a structural block diagram of a point cloud registration device based on a high-order Markov model in one embodiment;

[0050] Figure 9 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0052] Traditional methods usually treat the loop iteration as a first-order Markov decision process:

[0053] P(A t )=h(x t )

[0054] The current action A of the first-order Markov decision process t Depends only on the current state x t In the method of the present invention, high-order Markov decision is introduced into point cloud registration, and the current action may also be affected by the previous decision sequence: P(A t )=h(x t ,[A t-1 , A t-2 , ..., A1])

[0055] x t By combining the previously generated cross-stitched features with the transformed features, the transformation estimation will capture more effective information from the final step. The current decision of a first-order Markov is only affected by the current action and does not consider the results of previous actions. Point cloud registration is usually regarded as a first-order Markov decision process. The transformation results of this mathematical model during the iterative process only consider the state of the current point cloud pair and do not consider the previous state. However, the transformation results of a stable registration system are correlated in time series. Therefore, the present invention regards point cloud registration as a high-order Markov decision process.

[0056] In one embodiment, Figure 1 As shown, a point cloud registration method based on high-order Markov is provided, comprising the following steps:

[0057] Step 102: Obtain pre-processed initial source point cloud and target point cloud as training samples.

[0058] Get two sets of point clouds X = {x1, x2, ..., x n}, Y={y1,y2,...,y n}, the point clouds in X and Y are in one-to-one correspondence, representing a pair of overlapping 3D point clouds. The source X is transformed by the rigid estimation T = [R, t] during the iteration process, Perform the transformation, where R is the rotation vector and t is the translation vector. It is a special orthogonal group in Lie group theory.

[0059] Step 104: construct a cyclic point cloud registration model for point cloud registration; the cyclic point cloud registration model includes a cross-feature embedding module and an iterative optimization module.

[0060] like Figure 2 As shown in the figure, the cross feature embedding module includes a feature extraction layer, a cross concatenation layer, and a sliding window. The feature extraction layer extracts features from the initial source point cloud and the target point cloud respectively through a weight sharing network, and selects the PointNet architecture for geometric feature extraction. In this embodiment of the present invention, a 1D convolution layer with a size of [64, 128, 1024] is used to convert the original point cloud spatial position into a high-dimensional feature space. In order to ensure the disorder of the point cloud, the maximum pooling is used to obtain the global feature vector, R N×3 →R 1×M , where N is the point cloud sample and M is the output channel feature vector. A pair of global feature vectors with the same network mapping relationship in the same dimension is obtained. During the feature extraction process, the point cloud feature pairs are not fused to maintain the independence of each point cloud. The cross-stitched feature vectors still lack effective relative position analysis between the two point clouds. Therefore, after cross-stitching, a convolution layer is applied as a sliding window to fuse and refine these features and mine the spatial relative transformation information in the feature dimension. In a specific embodiment, a 16-size convolution kernel with a large receptive field is used to pay more attention to geometric shape and spatial position features. Experiments show that in the process of local feature extraction, large shallow convolution layers perform better than small deep convolution layers.

[0061] The iterative optimization process is represented as a series of iterative update steps. At each iteration, the rigid transformation between the current source point cloud and the target point cloud is output, and the position of the source point cloud is updated. After a set number of iterations, the rigid transformation between the initial source point cloud and the current source point cloud is output as the registration result. The iterative optimization module includes a transformation estimation network, which learns the mapping from features to transformations. It contains two branches, each predicting two vectors: a rotation vector and a translation vector. The transformation estimation network consists of a first-branch recurrent iterative layer, a second-branch recurrent iterative layer, and a transformation update layer (RT update). Each recurrent iterative layer includes a feature encoding layer (Encoder), a recurrent neural network layer, and a vector increment estimation layer. The recurrent neural network layer includes a recurrent GRU (Gate Recurrent Unit) unit. The high-order Markov process considers past states. During iterative optimization, the recurrent neural network is used to add sequential information of the transformation. GRU training is stable and efficient, making it the optimal choice for recurrent neural networks. The vector increment estimation layer on each branch is a rotation increment estimation layer (rotation estimation) and a translation increment estimation layer (translation estimation).

[0062] In step 106 , the cross feature embedding module is used to perform feature embedding on the initial source point cloud and the target point cloud respectively, and then cross-join them to obtain a cross-joined feature vector.

[0063] The cross-feature embedding module uses a weighted feature extraction network to extract global features between the initial source point cloud and the target point cloud. Based on the characteristics of the weighted shared network, the two sets of output global feature vectors contain a large amount of point cloud geometric feature information extracted by the same encoding function. Therefore, during feature fusion, the two sets of feature vectors are cross-joined. Specifically, the features at the same index position of the feature vectors are arranged together. The formula is as follows:

[0064]

[0065]

[0066]

[0067] Among them, ψ (st) is the global eigenvector of the initial source point cloud, ψ (s) is the global eigenvector ψ of the current source point cloud (t)It is the global feature vector of the target point cloud. Considering that the direct concatenation of these M-dimensional global features brings too many weak global feature representation capabilities and does not utilize the feature correspondence between the two global features, the cross-splicing layer fuses the global feature vector into a 2×M feature. This will learn more feature representation capabilities between the source and the target. The global features of the source and the target are concatenated into a feature vector according to the same network mapping relationship. After cross-splicing, ψ (st) Finally, a convolution layer with a convolution kernel size of 16 is used to further fuse the geometric information of the two sets of point clouds to improve the feature semantic representation ability.

[0068] Step 108: Update the current rotation vector and translation vector through the iterative optimization module according to the current cross-stitching feature vector and the rotation vector and translation vector output by the last high-order Markov decision iteration, output the rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector, iteratively update the current source point cloud according to the rigid transformation, and when the preset iteration stop condition is met, output the rigid transformation between the initial source point cloud and the current source point cloud as the point cloud registration result.

[0069] According to the current cross-joined feature vector, the rotation vector and the translation vector of the previous iteration, the input feature vector of the iterative optimization module is obtained. The input feature vector is obtained by the following steps: construct a skip connection between the last update operation of the iterative optimization module and the current update operation. Given the estimated transformation [R i-1 , T i-1 ], first convert the rotation matrix into Euler angle form, expressed as a 1×3 vector [α, β, γ]. Then, encode the rotation vector [α, β, γ] and the translation vector [a, b, c], respectively generate high-dimensional rotation features and translation features, and obtain transformation features. According to the current cross-stitching feature vector and transformation features, the input feature vector is obtained. The input feature vector includes the first input feature vector and the second input feature vector, which are used to update the rotation vector and the translation vector respectively. In the i-th iteration, the update process of the source point cloud and the camera pose is as follows:

[0070] Estimate a point cloud X in the source i The rigid transformation ΔT between the target point cloud Y i , then update the current pose T i Heyuan X i :

[0071] T i =ΔT i ×T i-1

[0072] X i =ΔT i ×X i-1

[0073] It takes n steps to calculate the transformation between the initial source X0 and the target Y. The final estimate after n steps is:

[0074] X n =ΔT n ×...ΔT1×X0

[0075] Estimated pose T i Decoupling, T i =[R i , T i ]Updated to:

[0076] R i =ΔR i R i-1

[0077] t i =Δt i t i-1

[0078] Current X i Updated to:

[0079] X i =R i (X-μx)+μx+t i

[0080] Among them, μx is the center of mass of the initial source point cloud. In this way, rotation will not induce translation, and the two are independent.

[0081] Step 110 : training the cyclic point cloud registration model based on the training samples and the prediction results obtained by inputting the training samples into the cyclic point cloud registration model to obtain a trained cyclic point cloud registration model.

[0082] In step 112 , the pre-processed initial source point cloud to be registered and the target point cloud are input into the trained cyclic point cloud registration model to obtain a point cloud registration result of the initial source point cloud to be registered.

[0083] In the above-mentioned point cloud registration method based on high-order Markov, the above-mentioned point cloud registration method, device and computer equipment based on high-order Markov, through a cyclic point cloud registration model, utilizes the sequential nature of the transformation to implicitly learn the relative spatial position between the two point clouds in the cyclic registration. Among them, through the cross-feature embedding module in the cyclic point cloud registration model, the relative transformation between the source and target point clouds is explored, the representation of the embedded global feature vector is enhanced, and a recurrent neural network is used to induce high-order Marcov (Markov) decision in the iterative optimization process. In the embodiment of the present invention, when performing state representation of point cloud pairs, the representation ability is stronger, relative position information can be obtained, the registration accuracy is high, and the registration performance is stronger.

[0084] In one embodiment, the step of obtaining the preprocessed source point cloud and target point cloud as training samples includes: obtaining an initial source point cloud set and a target point cloud set in the observation scene; and obtaining the initial source point cloud and target point cloud to be aligned based on the overlapping point clouds in the initial source point cloud set and the target point cloud set.

[0085] In one embodiment, the cross-feature embedding module includes a feature extraction layer, a cross-splicing layer and a sliding window: the feature extraction layer extracts geometric features from the initial source point cloud and the target point cloud to obtain a pair of global feature vectors with the same network mapping relationship under the same dimension; the cross-splicing layer cross-splices the global feature vectors to obtain a cross-splicing feature vector; the sliding window extracts the spatial relative transformation information of the cross-splicing feature vector to obtain a cross-splicing feature vector.

[0086] In one embodiment, the iterative optimization module includes a transformation estimation network; the transformation estimation network includes a first branch loop iteration layer and a second branch loop iteration layer; the first branch loop iteration layer includes a first feature encoding layer, a first recurrent neural network layer and a rotation increment estimation layer connected in sequence, and is used to output a rotation increment based on the current cross-spliced feature vector and the rotation vector of the previous iteration; the rotation increment is used to update the rotation vector; the second branch loop iteration layer includes a second feature encoding layer, a second recurrent neural network layer and a translation increment estimation layer, and is used to output a translation increment based on the current cross-spliced feature vector and the translation vector of the previous iteration; the translation increment is used to update the translation vector.

[0087] In this embodiment, the transformation estimation network uses two independent recurrent neural networks to estimate R and t, with initial values of R0=I 3×3 , t0=0, GRU has convolutional layers:

[0088] z i =σ(Convld([h i-1 , x i ],W z ))

[0089] r i =σ(Convld([h i-1 , x i ],W r ))

[0090]

[0091]

[0092] Among them, x i Input feature vector containing transformation (rotation / translation), hi-1 is the hidden state at the previous moment, z i is the update gate, σ(·) is the sigmoid function, Convld(·) is the convolution operation, W z 、W r and W h is the weight matrix, r i To reset the gate, is the candidate hidden state, tanh(·) is the tanh function, ⊙ is the multiplication of the corresponding elements of the matrix, h i is the hidden state passed to the next moment. The hidden state is used to represent the state of the network at each time step, which is essentially a lossy summary of the task-related aspects of the past input sequence.

[0093] In one embodiment, the step of outputting a rotation increment based on the current cross-splicing feature vector and the rotation vector of the previous iteration includes: encoding the rotation vector of the previous iteration through the first feature encoding module, splicing it with the current cross-splicing feature vector after encoding to obtain a first input feature, and outputting a rotation increment after the first input feature passes through a first recurrent neural network layer and a rotation increment estimation layer.

[0094] In one embodiment, the step of outputting the translation increment based on the current cross-splicing feature vector and the translation vector of the previous iteration includes: encoding the translation vector of the previous iteration through the second feature encoding module, splicing it with the current cross-splicing feature vector after encoding to obtain a second input feature, and outputting the translation increment after the second input feature passes through the second recurrent neural network layer and the translation increment estimation layer.

[0095] In one embodiment, the transformation estimation network also includes a transformation update layer; the transformation update layer is used to update the rotation vector and the translation vector according to the rotation increment and the translation increment respectively, and output the rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector.

[0096] In this embodiment, Euler angles and offset vectors are used to represent the pose information in six dimensions. One method is to design the output of the transformation network as six degrees of freedom, which is used to directly predict the Euler angles and offset vectors between two sets of point clouds. However, during the iteration process, once an abnormal estimated value appears using this single output, it will easily lead to the divergence of the final pose estimation result. The present invention uses a more robust loss function to supervise the network to obtain an accurate and stable transformation network, sets upper and lower limits for the step space of each degree of freedom of the Euler angle and offset vector, and selects 11 discrete steps in the range by exponential interpolation. The output layer of the transformation network is set to 3*11 degrees of freedom, and two transformation networks are used to predict the discrete step probabilities of the Euler angles and offset vectors respectively. The step size with the maximum probability is used as the estimation result, and the cross entropy loss is used as the supervision of the L learning transformation class=CE(l,l*), where l is the prediction of the logarithm of the multi-category distribution and l* is the true value. Eleven discrete intervals are used for all six output degrees of freedom. The two branches of the transformation estimation network each contain two fully connected layers of dimension [512, 256], followed by an additional fully connected layer of size 33, which are used to predict the step probabilities of the rotation and translation transformations, respectively. By using a discrete, finite step size to update the transformation step in each iteration, this will reduce the possibility of divergence of the results due to incorrect estimation.

[0097] In a specific embodiment, Figure 3 As shown in FIG, a schematic diagram of the comparison of the registration results of the cyclic point cloud registration model on the ModelNet40 and ScanObjectNN datasets is provided, wherein (a) is the initial state, (b) is the registration result using the ReAgent method, and (c) is the registration result using the method of the present invention. Figure 3 The first row in the figure shows an example of point cloud registration results on ModelNet40. The present invention sets 12 inference iterations. The transformation estimation network outputs a discrete estimated step size and updates the current position of the source in each iteration. During the iteration, the step size is dynamically adjusted according to the position between the source and the target. Figure 4 The figure shows a qualitative example of the iteration state when using ModelNet40 as a training sample. In iterations 1 and 2, the source adjusts itself in large steps and converges in small steps in the last few iterations. Figure 3 The second row in shows an example of point cloud registration results on ScanObjectNN. Figure 5 This is a qualitative example of the iterative state when using ScanObjectNN as a training sample, where the source and target are gradually aligned.

[0098] In a specific embodiment, the present invention evaluates the point cloud registration method on ModelNet40 and ScanObjectNN. The present invention uses a single 3090Ti GPU to train R-PCR and implements the model in Pytorch. R-PCR is the method of the present invention. During the training process, the AdamW optimizer is used and the weight decay is set to 1×10 -5 And set the gradient range to [-1,1]. The present invention sets an update operator of 12 iterations for all experiments. During the iteration process, the present invention backpropagates the gradient branch through ΔT and zeroes the gradient branch through T. For the ModelNet40 and ScanObjectNN datasets, the present invention trains a single model on the ModelNet40 training in two steps and evaluates it on the ModelNet40 test set and ObjectScanNN. The present invention first tests the method of the present invention on ModelNet40, which contains 40 categories and is constructed from 12,311 CAD models. Afterwards, the present invention uses four evaluation indicators, mean absolute error (MAE, Mean AbsoluteError), isotropic error (ISO), and chamfer distance (CD). Among these four indicators, the first two are explicit indicators and the last two are implicit indicators. The method of the present invention is compared with ReAgent and several baselines. The method of the present invention achieved the best ADI AUC and the lowest MAE, ISO and chamfer distance in both test splits. The generalization performance of this paper is tested on hold-out point clouds of 40 categories, and the model outperforms all baseline methods on both datasets.

[0099] In a specific embodiment, the present invention conducts a set of ablation experiments to show the relative importance of each component. All ablation experiments are trained on ModelNet40. The ablation results are as follows: Figure 6 As shown, the numerical results of the selected sliding window number and update are shown in Figure 7, where cross concatenation operation is cross concatenation operation, update refinement module is update optimization module, inference Updates are reasoning updates. The ablation experiment removes the following components in the point cloud registration task to analyze the relative importance of the components: First, the present invention experiments by replacing the GRU unit to make the model of the present invention a Markov model. Without the GRU unit, all metrics become worse and the convergence speed is greatly reduced. This may be because the gated activation makes the transformation sequence easier to converge; the present invention replaces the cross-splicing module with a single connection operation. Experiments show that the model using the cross-splicing module performs better because it mines more corresponding feature information between the source and the target; the present invention encodes the input feature vector of the GRU unit with the last step transformation of the jump connection. In this experiment, the present invention found that including skip connections will bring better performance; the sliding window size specifies the receptive field of the local feature extraction operation. The present invention tried a range of sliding window sizes of 1, 8, 16, and 32. Although R-PCR performs well when the sliding window size is higher than 16, after balancing the computational amount and performance, the present invention finally chose 16 as the sliding window size. The results show that a large receptive field can capture more correspondences between the source and target feature vectors from the weight sharing network; in addition, the present invention converts the convolutional layer into an MLP (Multilayer CNN). Perceptron, multi-layer perceptron), the results are slightly worse than the convolutional layer with a larger kernel size because parameter redundancy leads to overfitting; the present invention experiments on the number of iterative updates and gives 4 candidates with 4, 8, 12 and 16 iterations respectively. The results show that R-PCR performs well when it is updated with 12 iterations.

[0100] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0101] In one embodiment, Figure 8As shown, a point cloud registration device based on high-order Markov is provided, comprising: a sample acquisition module 802, a model construction module 804, a cross-stitching module 806, an iterative optimization module 808, a model training module 810 and a result output module 812, wherein:

[0102] The sample acquisition module 802 is used to acquire the pre-processed initial source point cloud and target point cloud as training samples;

[0103] A model building module 804 is used to build a cyclic point cloud registration model for point cloud registration; the cyclic point cloud registration model includes a cross feature embedding module and an iterative optimization module;

[0104] A cross-stitching module 806 is configured to perform feature embedding on the initial source point cloud and the target point cloud respectively through the cross-feature embedding module and then cross-stitch them to obtain a cross-stitching feature vector;

[0105] an iterative optimization module 808, configured to update the current rotation vector and translation vector according to the current cross-stitching feature vector and the rotation vector and translation vector iteratively outputted by the previous high-order Markov decision process through the iterative optimization module, output a rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector, iteratively update the current source point cloud according to the rigid transformation, and output the rigid transformation between the initial source point cloud and the current source point cloud as a point cloud registration result when a preset iteration stop condition is met;

[0106] A model training module 810 is configured to train the cyclic point cloud registration model based on training samples and prediction results obtained by inputting the training samples into the cyclic point cloud registration model to obtain a trained cyclic point cloud registration model;

[0107] The result output module 812 is used to input the pre-processed initial source point cloud to be registered and the target point cloud into the trained cyclic point cloud registration model to obtain the point cloud registration result of the initial source point cloud to be registered.

[0108] In one embodiment, the iterative optimization module 808 is also used for the first branch cyclic iteration layer, including a first feature encoding layer, a first recurrent neural network layer and a rotation increment estimation layer connected in sequence, for outputting a rotation increment based on the current cross-spliced feature vector and the rotation vector of the previous iteration; the rotation increment is used to update the rotation vector; the second branch cyclic iteration layer, including a second feature encoding layer, a second recurrent neural network layer and a translation increment estimation layer, for outputting a translation increment based on the current cross-spliced feature vector and the translation vector of the previous iteration; the translation increment is used to update the translation vector.

[0109] In one embodiment, the iterative optimization module 808 is also used to encode the rotation vector of the previous iteration through the first feature encoding module, and after encoding, splice it with the current cross-splicing feature vector to obtain a first input feature. After the first input feature passes through the first recurrent neural network layer and the rotation increment estimation layer, the rotation increment is output.

[0110] In one embodiment, the iterative optimization module 808 is also used to encode the translation vector of the previous iteration through the second feature encoding module, and after encoding, splice it with the current cross-splicing feature vector to obtain a second input feature. After the second input feature passes through the second recurrent neural network layer and the translation increment estimation layer, the translation increment is output.

[0111] In one embodiment, the iterative optimization module 808 is also used in the transformation update layer to update the rotation vector and the translation vector according to the rotation increment and the translation increment respectively, and output the rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector.

[0112] In one embodiment, the cross-stitching module 806 is also used in the cross-feature embedding module, which includes a feature extraction layer, a cross-stitching layer and a sliding window: the feature extraction layer extracts geometric features from the initial source point cloud and the target point cloud to obtain a pair of global feature vectors with the same network mapping relationship under the same dimension; the cross-stitching layer cross-stitches the global feature vector to obtain a cross-stitched feature vector; the sliding window extracts the spatial relative transformation information of the cross-stitched feature vector to obtain a cross-stitched feature vector.

[0113] In one embodiment, the sample acquisition module 802 is further used to obtain an initial source point cloud set and a target point cloud set in the observation scene; and obtain the initial source point cloud and target point cloud to be aligned based on the overlapping point clouds in the initial source point cloud set and the target point cloud set.

[0114] The specific limitations of the high-order Markov-based point cloud registration device can be found in the limitations of the high-order Markov-based point cloud registration method described above and will not be repeated here. Each module in the high-order Markov-based point cloud registration device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of the modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the form of software in the computer device so that the processor can call and execute the operations corresponding to each of the modules.

[0115] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 9As shown. The computer device includes a processor, a memory, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a point cloud registration method based on a high-order Markov model is implemented. The display screen of the computer device can be a liquid crystal display screen, and the input device of the computer device can be an external keyboard, mouse, etc.

[0116] Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0117] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0118] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0119] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0120] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0121] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A point cloud registration method based on high-order Markov, characterized in that: The method comprises: Obtain preprocessed initial source point cloud and target point cloud as training samples; Constructing a cyclic point cloud registration model for point cloud registration; the cyclic point cloud registration model includes a cross feature embedding module and an iterative optimization module; The cross feature embedding module performs feature embedding on the initial source point cloud and the target point cloud respectively, and then cross-splicing them to obtain a cross-splicing feature vector; The iterative optimization module updates the current rotation vector and translation vector according to the current cross-stitching feature vector and the rotation vector and translation vector outputted by the last high-order Markov decision iteration, outputs the rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector, iteratively updates the current source point cloud according to the rigid transformation, and when a preset iteration stop condition is met, outputs the rigid transformation between the initial source point cloud and the current source point cloud as the point cloud registration result; Training the cyclic point cloud registration model according to the training samples and the prediction results obtained by inputting the training samples into the cyclic point cloud registration model to obtain a trained cyclic point cloud registration model; The pre-processed initial source point cloud to be registered and the target point cloud are input into the trained cyclic point cloud registration model to obtain a point cloud registration result of the initial source point cloud to be registered.

2. The method according to claim 1, characterized in that The iterative optimization module includes a transformation estimation network; the transformation estimation network includes a first branch loop iteration layer and a second branch loop iteration layer; The first branch cyclic iteration layer includes a first feature encoding layer, a first cyclic neural network layer and a rotation increment estimation layer connected in sequence, and is used to output a rotation increment according to the current cross-spliced feature vector and the rotation vector of the previous iteration; The rotation increment is used to update the rotation vector; The second branch cyclic iteration layer includes a second feature encoding layer, a second cyclic neural network layer and a translation increment estimation layer, which is used to output a translation increment based on the current cross-spliced feature vector and the translation vector of the previous iteration; the translation increment is used to update the translation vector.

3. The method according to claim 2, characterized in that The step of outputting a rotation increment according to the current cross-joined feature vector and the rotation vector of the previous iteration comprises: The rotation vector of the previous iteration is encoded through the first feature encoding layer, and after encoding, it is spliced with the current cross-splicing feature vector to obtain a first input feature. After the first input feature passes through the first recurrent neural network layer and the rotation increment estimation layer, the rotation increment is output.

4. The method according to claim 2, characterized in that The step of outputting a translation increment according to the current cross-stitching feature vector and the translation vector of the previous iteration comprises: The translation vector of the previous iteration is encoded through the second feature encoding layer and then spliced with the current cross-joined feature vector to obtain a second input feature. After the second input feature passes through the second recurrent neural network layer and the translation increment estimation layer, the translation increment is output.

5. The method according to claim 2, characterized in that The transform estimation network also includes a transform update layer; The transformation update layer is used to update the rotation vector and the translation vector according to the rotation increment and the translation increment respectively, and output the rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector.

6. The method according to claim 1, characterized in that The cross feature embedding module includes a feature extraction layer, a cross splicing layer and a sliding window: The feature extraction layer extracts geometric features from the initial source point cloud and the target point cloud to obtain a pair of global feature vectors with the same network mapping relationship in the same dimension; Cross-joining the global feature vectors through a cross-joining layer to obtain a cross-joined feature vector; The spatial relative transformation information of the cross-spliced feature vector is extracted through a sliding window to obtain a cross-spliced feature vector.

7. The method according to claim 1, characterized in that The step of obtaining the pre-processed source point cloud and target point cloud as training samples comprises: Obtain the initial source point cloud set and target point cloud set in the observation scene; The initial source point cloud and target point cloud to be registered are obtained according to the overlapping point clouds in the initial source point cloud set and the target point cloud set.

8. A point cloud registration device based on high-order Markov, characterized in that: The device comprises: A sample acquisition module is used to obtain preprocessed initial source point cloud and target point cloud as training samples; A model building module, used to build a cyclic point cloud registration model for point cloud registration; the cyclic point cloud registration model includes a cross feature embedding module and an iterative optimization module; a cross-stitching module, configured to perform feature embedding on the initial source point cloud and the target point cloud respectively through the cross-feature embedding module and then cross-stitch them to obtain a cross-stitching feature vector; an iterative optimization module, configured to update the current rotation vector and translation vector according to the current cross-stitching feature vector and the rotation vector and translation vector iteratively outputted by the high-order Markov decision algorithm in the previous iteration, output a rigid transformation between the current source point cloud and the target point cloud according to the current rotation vector and translation vector, iteratively update the current source point cloud according to the rigid transformation, and output the rigid transformation between the initial source point cloud and the current source point cloud as a point cloud registration result when a preset iteration stop condition is met; A model training module is used to train the cyclic point cloud registration model according to training samples and prediction results obtained by inputting the training samples into the cyclic point cloud registration model to obtain a trained cyclic point cloud registration model; The result output module is used to input the preprocessed initial source point cloud to be registered and the target point cloud into the trained cyclic point cloud registration model to obtain the point cloud registration result of the initial source point cloud to be registered.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images