3D visual point cloud registration method and system, storage medium and equipment

Abstract

The invention provides a 3D visual point cloud registration method and system, a storage medium and equipment, and the method comprises the steps: building a deep learning network model, and inputting a source point cloud and a target point cloud; introducing a density attention algorithm into the deep learning network model, and processing the source point cloud and the target point cloud; introducing a rotation sensing algorithm into the deep learning network model, and extracting rotation parameters according to the point cloud features after the weights are added; according to the rotation parameter, rotating the source point cloud; inputting an output layer in the deep learning network model according to the rotated source point cloud, and outputting a calculation result; and performing point cloud registration according to the calculation result. In combination with a density attention mechanism, important features are rapidly extracted; key features are effectively identified, the robustness to noise and complex shapes is improved, and the accuracy and stability of registration are ensured; a rotation sensing module is used for extracting rotation parameters, global maximum pooling is combined, global information of the point cloud is effectively captured, and therefore the matching precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of point cloud registration, and in particular relates to a 3D visual point cloud registration method, system, storage medium and device. Background Art

[0002] In recent years, the progress of deep learning technology has brought new opportunities for point cloud registration. The method based on density attention mechanism can automatically identify and focus on important features to improve the accuracy of registration. At the same time, rotation perception technology can effectively process data from different perspectives and reduce the registration error caused by rotation. Combining these two technologies, a point cloud registration method is proposed, which is expected to improve the registration accuracy and efficiency while enhancing its application ability in complex environments.

[0003] Existing point cloud registration technology can realize the basic functions required for point cloud registration. The patent "Point cloud registration method and system based on ISS feature points", application number CN202411169993.5, determines the best matching point pair through Euclidean distance and estimates the optimal transformation matrix through the least squares method.

[0004] The patent "A method and system for point cloud registration based on rotation equivariant network", application number CN202310016404.9, proposes to build an efficient and universal feature learning framework that can use rotation invariant and SO(2) equivariant features to improve the inlier rate of point correspondence. The paper "Deep Closest Point: Learning Representations for Point Cloud Registration" model consists of three parts: a point cloud embedding network, an attention-based module combined with a pointer generation layer for approximate combinatorial matching, and a differentiable singular value decomposition (SVD) layer for extracting the final rigid transformation.

[0005] However, the existing technology still has the following shortcomings: (1) When the existing method uses the plane fitting method to process point clouds with complex or curved shapes, it may not be able to fully capture the characteristics of the data, thus affecting the registration effect; (2) The detection of manual feature points or the calculation of feature descriptors may be sensitive to the density and distribution of the point cloud, resulting in inconsistent performance in different scenarios; (3) The extraction method that relies on specific features (such as plane features or rotation-invariant features) may not be able to effectively capture the global and local information of the point cloud, thus affecting the matching effect; Summary of the invention

[0006] In view of the shortcomings of the prior art, the present invention provides a 3D visual point cloud registration method, the method comprising:

[0007] Establish a deep learning network model, and input the source point cloud and the target point cloud into the deep learning network model;

[0008] Introduce the density attention algorithm into the backbone network in the deep learning network model, and process the source point cloud and the target point cloud to obtain the point cloud features after adding weights;

[0009] Introduce a rotation perception algorithm into the deep learning network model to extract rotation parameters based on the weighted point cloud features;

[0010] According to the rotation parameters, the source point cloud is rotated to obtain a rotated source point cloud;

[0011] According to the rotated source point cloud, the output layer in the deep learning network model is input and the calculation result is output;

[0012] Point cloud registration is performed according to the calculation results.

[0013] Furthermore, the backbone network adopts a dynamic graph convolutional neural network model;

[0014] The output layer outputs the calculation results using the SVD decomposition algorithm.

[0015] Furthermore, the point cloud features after adding weights include:

[0016] Input the point cloud data into the dynamic graph convolutional neural network model, use the K-nearest neighbor algorithm to obtain the index, and adjust and expand the dimension of the index;

[0017] Use the adjusted index to extract the point cloud features of K nearest neighbors;

[0018] Calculate the variance of point cloud features to obtain local density and adjust the shape of density information;

[0019] Calculate the attention weight of each point cloud, and apply the attention weight to the input point cloud features to obtain the point cloud features after adding the weight.

[0020] Furthermore, the extracting of the rotation parameters includes: inputting the point cloud features after adding weights, and calculating the rotation parameters through a fully connected layer.

[0021] Furthermore, obtaining the rotated source point cloud includes:

[0022] Reshaping the rotation parameters into a rotation matrix;

[0023] According to the rotation matrix, the source point cloud is rotated to obtain the rotated source point cloud.

[0024] Furthermore, the output calculation result includes:

[0025] Inputting the rotated source point cloud into the output layer, and calculating the similarity between the features of the source point cloud and the target point cloud;

[0026] Using a density attention algorithm to centralize the similarity, obtaining a covariance matrix;

[0027] Perform SVD decomposition on the covariance matrix and output the calculation results;

[0028] The calculation results include: a calculated rotation matrix and a calculated translation matrix.

[0029] A 3D visual point cloud registration system, the system comprising: a density attention algorithm module, a fully connected layer module and a rotation perception module, an output module and a point cloud registration module.

[0030] Furthermore, the density attention algorithm module is used to extract source point cloud and target point cloud features, calculate density information, and assign weights to source point cloud features and target point cloud features;

[0031] The fully connected layer module is used to calculate the density attention weight and rotation parameters;

[0032] The rotation perception module is used to capture the global information of the point cloud and extract the rotation parameters;

[0033] The output layer is used to calculate the similarity between the features of the source point cloud and the target point cloud, and use centralization to obtain the covariance matrix, and perform SVD decomposition to output the calculation results of the rotation matrix and the translation matrix;

[0034] The point cloud registration module is used to perform point cloud registration.

[0035] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the method steps described above is implemented.

[0036] An electronic device, characterized in that it comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0037] Memory, used to store computer programs;

[0038] The processor is used to implement any of the above-mentioned method steps when executing the program stored in the memory.

[0039] Compared with the prior art, the present invention has the following advantages:

[0040] 1. This paper proposes a point cloud registration method based on density attention and rotation perception, which quickly extracts important features through the local density calculation and attention mechanism of the density attention algorithm module;

[0041] 2. Combined with the density attention mechanism, it can effectively identify key features, improve the robustness to noise and complex shapes, and ensure the accuracy and stability of registration;

[0042] 3. Use the RotationAware module to extract rotation parameters and combine it with global maximum pooling to effectively capture the global information of the point cloud, thereby improving matching accuracy;

[0043] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A schematic flow chart of a 3D visual point cloud registration method is shown.

[0046] Figure 2 A schematic diagram of the structure of a 3D visual point cloud registration system is shown.

[0047] Figure 3 A schematic flow chart of the density attention weight calculation method in the present invention is shown.

[0048] Figure 4 A schematic flow chart of the source point cloud rotation method in the present invention is shown.

[0049] Figure 5 A schematic diagram of a registration result according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] like Figure 1 As shown, a 3D visual point cloud registration method comprises the following steps:

[0052] S1. Establish a deep learning network model and input the source point cloud (Psource) and the target point cloud (Ptarget) into the deep learning network model.

[0053] Optionally, the deep learning network model selects a dynamic graph convolutional neural network (DGCNN) model as a backbone network and an SVD decomposition algorithm as an output layer (Head) to output the result.

[0054] S2. Add the density attention algorithm to the DGCNN model, input the source point cloud and the target point cloud for calculation, and obtain the point cloud features after adding the weights, such as Figure 3 shown.

[0055] Optionally, there are four stages in the DGCNN model, and the point cloud features output by each stage are processed by the density attention algorithm to obtain point cloud saliency enhanced features.

[0056] S2.1. Input the point cloud data into the DGCNN model, use the K-nearest neighbor (KNN) algorithm to obtain the index idx, and adjust and expand the dimension of the index.

[0057] S2.1.1. The shape of the received point cloud data is expressed as,

[0058]

[0059] Among them, B is the batch size, C is the number of feature channels, and N is the number of points in the point cloud feature.

[0060] S2.1.2. By calculating the K nearest neighbors of each point, we get the index idx, whose index shape is expressed as:

[0061]

[0062] S2.1.3. Adjust and expand the index dimension to obtain an adjusted index with the shape of

[0063]

[0064] S2.2. Extract the K nearest neighbor point cloud features F using the adjusted index knn , its formula is expressed as,

[0065] F knn =gather(X expanded , idx)

[0066] Optionally, the point cloud feature F of the K nearest neighbors knn The output shape is,

[0067]

[0068] S2.3. Calculation of F knn The variance of has obtained the local density D and adjusted the shape of the density information.

[0069] S2.3.1. Calculation of F knn The variance of has obtained the local density D, which is expressed as,

[0070] D=var(F knn ,dim=-1)

[0071] S2.3.2. Adjust the shape of density information. The formula is:

[0072] D reshaped =permute(D,(0,2,1)).reshape(-1,C)

[0073] Optionally, the shape of the adjusted density information is,

[0074]

[0075] S2.4. Calculate the attention weight of each point cloud, and apply the attention weight to the input point cloud features to obtain the point cloud features after adding the weight.

[0076] S2.4.1. Calculate the attention weight W of each point through the fully connected layer. The calculation formula is:

[0077] W=fc(D reshaped )

[0078] Optionally, the output shape of the attention weight W is,

[0079]

[0080] S2.4.2. Apply the attention weight to the input point cloud features, and its formula is expressed as:

[0081] X att =X·W

[0082] Optionally, the weighted point cloud feature X att The shape is,

[0083]

[0084] S3, such as Figure 4 As shown in Figure 2, rotation perception (RotationAwar) is added before the backbone network and the output layer (Head) of the deep learning network. e ) algorithm to extract the rotation parameters.

[0085] S3.1. Input the point cloud features after adding weights, and its shape is:

[0086]

[0087] Among them, B is the batch size, C is the number of feature channels, and N is the number of points in the point cloud feature.

[0088] S3.2, the rotation parameters are calculated through the fully connected layer, and the formula is expressed as:

[0089] R arams =fc rotation (x global )

[0090] Among them, x global = max(X, dim = -1); the output rotation parameter

[0091] S4. Rotate the source point cloud according to the rotation parameter to obtain a rotated source point cloud.

[0092] S4.1. Reshape the rotation parameters obtained in step S3 into a rotation matrix, which is expressed as follows:

[0093] R=view(R params , (B, 3, 3))

[0094] S4.2. According to the rotation matrix, the source point cloud is rotated to obtain the rotated source point cloud, which is expressed as:

[0095] X src_rotated =X src +α·(R·X src )

[0096] Here, α is a learnable parameter that controls the influence of the rotation.

[0097] S5. Input the rotated source point cloud into the output layer (Head), calculate the similarity between the features of the source point cloud and the target point cloud, and use the density attention algorithm for centralization to obtain the covariance matrix, perform SVD decomposition, and output the calculation results of the rotation matrix and translation matrix.

[0098] S5.1. Calculate the embedding features of the source point cloud and the target point cloud through the embedding network to obtain the source embedding and the target embedding.

[0099] S5.2. Calculate the density attention score.

[0100] S5.2.1. Calculate the score matrix between the source embedding and the target embedding.

[0101]

[0102] Among them, src_embedding is the input embedding matrix, T is the matrix transpose, tgt_embedding is the output embedding matrix, and dk is the dimension of the embedded features.

[0103] S5.2.2. Apply the softmax function to normalize the scores to obtain the density attention weight scores that match each source point cloud and target point cloud.

[0104] S5.3. Calculate the mean of the source point cloud and the target point cloud and perform centralization. The formula is:

[0105] src_centered=src-src.mean(dim=2, keepdim=True)

[0106] src_corr_centered=src_corr-src_corr.mean(dim=2, keepdim=True)

[0107] Among them, src is the source point cloud, mean() represents the mean calculation, and dim represents the mean processing along the second dimension. keepdim is set to True, which means that the dimensions of the input and output features are the same, but the size of the specified dimension is 1; Src_corr represents the target point cloud.

[0108] S5.4. Using the mean of the centralized source point cloud and target point cloud, calculate the covariance matrix. The formula is:

[0109] H=src_centered·src_corr_centered T

[0110] Among them, H is the covariance matrix; src_centered is the centralized mean of the source point cloud; src_corr_eentered T is the central mean of the target point cloud.

[0111] S5.5. Perform SVD decomposition on the covariance matrix of each batch to obtain U, S, and V matrices; calculate the rotation matrix and handle the reflection situation to ensure that the rows and columns of the rotation matrix are positive.

[0112] S5.6. Calculate the translation matrix to align the source point cloud with the target point cloud. The formula is:

[0113] t=-R·src.mean(dim=2, keepdim=True)+src_corr.mean(dim=2, keepdim=True)

[0114] S5.7. Output the rotation matrix and translation matrix as results.

[0115] S6. Perform point cloud registration based on the results of the rotation matrix and the translation matrix.

[0116] In another specific embodiment of the present invention, Figure 5 As shown,

[0117] Under different point cloud rotation and translation conditions, the DGCNN model combined with the density attention weight module and the rotation perception module can better capture the local and global features in the point cloud, dynamically adjust the weights of key points, and maintain high-precision registration for different objects.

[0118] like Figure 2 As shown, a 3D visual point cloud registration system includes: a dynamic graph convolutional neural network module (DGCNN), an output module (Head) and a point cloud registration module.

[0119] The output layer is used to calculate the similarity between the features of the source point cloud and the target point cloud, and use centralization processing to obtain the covariance matrix, and perform SVD decomposition to output the calculation results of the rotation matrix and the translation matrix.

[0120] The point cloud registration module is used to perform point cloud registration.

[0121] The dynamic graph convolutional neural network module includes: a density attention algorithm module (Demsity Attention n Modul e ), fully connected layer module (Concat) and rotation perception module (Rotatio n Awar e Modul e).

[0122] The density attention algorithm module is used to extract source point cloud and target point cloud features, calculate density information, and assign weights to source point cloud features and target point cloud features.

[0123] The fully connected layer module is used to calculate the density attention weight and rotation parameters.

[0124] The rotation perception module is used to capture the global information of the point cloud and extract rotation parameters.

[0125] Based on the above disclosed content, the present invention also provides an electronic device accordingly. The electronic device of an embodiment of the present invention includes at least one electrically connected processor and at least one storage medium, the storage medium is electrically connected to the processor, wherein the storage medium stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the method as described above.

[0126] Based on the same inventive concept, the present invention also provides a storage medium, which stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method as described above.

[0127] The above description and the accompanying drawings fully illustrate the embodiments of the present invention so that those skilled in the art can practice them. Other embodiments may include structural and other changes. The embodiments represent only possible variations. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The embodiments of the present invention are not limited to the structures described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A 3D visual point cloud registration method, characterized in that: The method comprises: Establish a deep learning network model, and input the source point cloud and the target point cloud into the deep learning network model; Introduce the density attention algorithm into the backbone network in the deep learning network model, and process the source point cloud and the target point cloud to obtain the point cloud features after adding weights; Introduce a rotation perception algorithm into the deep learning network model to extract rotation parameters based on the weighted point cloud features; According to the rotation parameters, the source point cloud is rotated to obtain a rotated source point cloud; According to the rotated source point cloud, the output layer in the deep learning network model is input and the calculation result is output; Point cloud registration is performed according to the calculation results.

2. The 3D visual point cloud registration method according to claim 1, characterized in that: The backbone network adopts a dynamic graph convolutional neural network model; The output layer outputs the calculation results using the SVD decomposition algorithm.

3. The 3D visual point cloud registration method according to claim 1, characterized in that: The point cloud features after adding weights include: Input the point cloud data into the dynamic graph convolutional neural network model, use the K-nearest neighbor algorithm to obtain the index, and adjust and expand the dimension of the index; Use the adjusted index to extract the point cloud features of K nearest neighbors; Calculate the variance of point cloud features to obtain local density and adjust the shape of density information; Calculate the attention weight of each point cloud, and apply the attention weight to the input point cloud features to obtain the point cloud features after adding the weight.

4. The 3D visual point cloud registration method according to claim 1, characterized in that: The extraction of rotation parameters includes: inputting the point cloud features after adding weights, and calculating the rotation parameters through a fully connected layer.

5. The 3D visual point cloud registration method according to claim 1, characterized in that: The step of obtaining the rotated source point cloud comprises: Reshaping the rotation parameters into a rotation matrix; According to the rotation matrix, the source point cloud is rotated to obtain the rotated source point cloud.

6. The 3D visual point cloud registration method according to claim 1, characterized in that: The output calculation result includes: Inputting the rotated source point cloud into the output layer, and calculating the similarity between the features of the source point cloud and the target point cloud; Using a density attention algorithm to centralize the similarity, obtaining a covariance matrix; Perform SVD decomposition on the covariance matrix and output the calculation results; The calculation results include: a calculated rotation matrix and a calculated translation matrix.

7. A 3D visual point cloud registration system, characterized in that: The system includes: a density attention algorithm module, a fully connected layer module and a rotation perception module, an output module and a point cloud registration module.

8. The 3D visual point cloud registration system according to claim 7, characterized in that: The density attention algorithm module is used to extract source point cloud and target point cloud features, calculate density information, and assign weights to source point cloud features and target point cloud features; The fully connected layer module is used to calculate the density attention weight and rotation parameters; The rotation perception module is used to capture the global information of the point cloud and extract the rotation parameters; The output layer is used to calculate the similarity between the features of the source point cloud and the target point cloud, and use centralization to obtain the covariance matrix, and perform SVD decomposition to output the calculation results of the rotation matrix and the translation matrix; The point cloud registration module is used to perform point cloud registration.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 6 are implemented.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 6 when executing a program stored in a memory.

Images