Transform-based coarse aggregate point cloud registration method
Through the TRANSFORMER -based rough set of materials, the problem of high -precision and robust collection of high -precision and robust collective cloud allocation effects are solved by the TRANSFORMER.
Patent Information
- Application Number
- CN202411517160.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-01-28
AI Technical Summary
In the prior art, complex aggregate point clouds are difficult to register with high precision, resulting in scarcity of data sets, improvement of three-dimensional data quality, insufficient registration accuracy, and poor robustness.
Adopt a rough set cloud allocation method based on Transformer, collect point cloud data through the SR7400 line structure optical camera to perform pre -processing and data segmentation, and establish a convolutional neural network with PCRNet as BackBone, including feature extraction modules, transformer feature fusion modules , Features soft matching modules and bit position return modules, and train point cloud distribution networks to achieve high precision standards for coarse materials.
Achieve the highest standard accuracy on the collection of materials, improve the robustness and generalization, and solve the problem of not high accuracy of collecting cloud allocation accuracy in the existing technology.
Smart Images
Figure CN119379753A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of three-dimensional point cloud registration, and relates to a coarse aggregate point cloud registration method based on Transformer. Background Art
[0002] Asphalt concrete pavement occupies an important position in my country's high-grade pavement due to its excellent characteristics such as high temperature stability, low temperature crack resistance, flatness, low dust, and low noise. Aggregate is the main material for paving asphalt pavement, playing the role of skeleton and filling. Compared with two-dimensional images, three-dimensional data can obtain the height information of the target to be measured, so it can more accurately characterize the shape, angle, surface texture and other characteristics of the aggregate.
[0003] With the rapid development of new sensor technology and point cloud processing technology, point cloud data collection has become more convenient. However, due to the limitations of scanning equipment and environment, multiple and multi-view scans are usually required to obtain a complete point cloud of the target object. Registering and stitching multiple point clouds into a whole is the basis and key step to achieve point cloud reconstruction. How to improve the efficiency and accuracy of point cloud registration is a hot topic and difficult problem in current research.
[0004] In recent years, point cloud registration has generally achieved end-to-end fast registration. At the same time, the application of deep learning in point cloud registration has also achieved certain results, but it is rarely used in aggregate. The aggregate point cloud registration model is used to find the positional relationship between two aggregate point clouds, which helps to optimize 3D feature extraction and aggregate 3D reconstruction. The current research on aggregate point cloud registration faces three main problems: scarce data sets, problems in improving the quality of aggregate 3D data, and insufficient registration accuracy and weak robustness. It is urgent to build a more stable and accurate aggregate point cloud registration model. Summary of the invention
[0005] The purpose of the present invention is to provide a Transformer-based aggregate point cloud registration method, which solves the problem that complex aggregate point clouds are difficult to register with high precision in the prior art.
[0006] The technical solution adopted by the present invention is a coarse aggregate point cloud registration method based on Transformer, which is characterized by the following specific steps:
[0007] Step 1: Collect multiple sets of aggregate point cloud data with different particle size distributions using the SR7400 line structured light camera, add Gaussian noise and crop noise to some of them, and downsample the point cloud to 2048 points through preprocessing;
[0008] Step S2: According to the "Highway Engineering Aggregate Test Regulations", square hole sieves with particle sizes of 4.75 mm, 9.5 mm, 13.2 mm, and 16 mm are used to mark the aggregates that cannot pass through according to the current hole diameter and divide them into training set and test set in a ratio of 7:3;
[0009] Step S3: Generate a target point cloud, by copying the source point cloud and transforming the coordinates to form a point cloud to be registered. Since this point cloud does not have noise, it is also called noise-free data;
[0010] Step S4: Establish a convolutional neural network with PCRNet as the backbone. The network structure includes a feature extraction module, a transformer feature fusion module, a feature soft matching module, and a pose regression module.
[0011] Step S5, using the data set produced in steps S1-S3 to train the point cloud registration network, and after testing with different noise data, the final model is used for coarse aggregate point cloud registration.
[0012] Specifically, the aggregate particle size distributions collected in step S1 are 4.75-9.5 mm, 9.5-13.2 mm, 13.2-16 mm, and 16-19 mm, respectively.
[0013] Specifically, the downsampling method in step S1 adopts the farthest point sampling method to ensure a uniform input scale of data and computational efficiency.
[0014] Specifically, the value ranges of the three Euler angles and translation vectors of the coordinate transformation in step S3 are respectively between [0°, 45°] and [-0.5, 0.5], and their scopes are copied to the source point cloud.
[0015] Specifically, the feature extraction module in step S4 is used to extract features from the input three-dimensional data, specifically including first extracting 64-dimensional features through the MLP layer, then processing through the adaptive convolution layer to obtain 128-dimensional high-dimensional features containing multiple scales, and then changing the feature dimension to 1024 layers through the MLP layer.
[0016] Specifically, the adaptive convolution in step S4 is used to adapt to point clouds of different shapes and densities. It mainly consists of a weight library, an MLP and a convolution kernel function. The learnable parameter of the weight library is K. In the present invention, K=16 is taken to balance the diversity of the convolution kernel and the model burden. The MLP is used to predict the normalized coefficient vector. The convolution kernel function is obtained by combining the weight matrix with the predicted corresponding coefficients, that is, weighted summing the weight matrix and the coefficient vector.
[0017] Specifically, the transformer feature fusion module in step S4 is used to effectively capture the correlation between two point clouds. The difference between it and the original transformer model is that the improved module uses the features obtained by the feature extraction module as the input of the encoder and the encoder, and adjusts the features by adding residual connections. In addition, in order to optimize its training efficiency, residual links and normalization operations are added after each sublayer.
[0018] Specifically, the feature soft matching module in step S4 is used to match the similarity between the feature vectors of the source point cloud and the target point cloud. The present invention uses a probabilistic method to locate corresponding points, that is, to generate a probability mapping matrix from one set of feature sequences to another set of feature sequences. First, the 2-norm distance between each input feature vector and the target feature vector is calculated. The similarity of the two features is inversely proportional to their values, that is, the closer to 0, the higher the similarity. Then, the softmax function is applied to the distance to convert the feature vector into a probability distribution. The final soft mapping matrix is determined by the position of the maximum probability in each row.
[0019] Specifically, the pose regression module in step S4 is used to align the input point cloud, including softmax normalization layer, decentralization and SVD decomposition. First, the registration point cloud is softmax normalized to convert the similarity into probability. After decentralization, the covariance matrix is calculated and decomposed into rotation matrix and translation vector according to SVD.
[0020] Specifically, the loss function of the network training in step S5 includes the addition of a rotation matrix loss function and a translation vector loss function. The rotation matrix loss function is described as the sum of the squares of the bi-norm of the distance between the rotation matrix prediction value and the rotation matrix true value, and the translation vector loss function is described as the sum of the squares of the bi-norm of the distance between the translation vector prediction value and the translation vector true value.
[0021] Compared with the prior art, the present invention has the following advantages and beneficial effects: the present invention adopts point cloud adaptive convolution and SVD to solve the problem that PCRNet is difficult to extract multi-scale features and the rigid body transformation calculation efficiency of the fully connected layer is low and easy to generate errors. In addition, after improvement based on Transformer, the extracted features are fused to achieve the highest alignment accuracy on various types of aggregate point cloud data, with the highest accuracy, strongest robustness and strongest generalization. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 It is a schematic diagram of point cloud preprocessing of the present invention;
[0023] Figure 2 is a point cloud registration network structure diagram of the present invention;
[0024] Figure 3It is the feature extraction network structure diagram;
[0025] Figure 4 It is an iteration diagram of the network training process;
[0026] Figure 5 This is the normal point cloud data registration result image;
[0027] Figure 6 This is the point cloud data registration result image after adding noise;
[0028] Figure 7 This is a comparison chart of the registration accuracy of different models. DETAILED DESCRIPTION
[0029] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments.
[0030] A coarse aggregate point cloud registration method based on Transformer, the specific steps are as follows:
[0031] Step S1: Collect multiple groups of aggregate point cloud data with different particle size distributions based on the SR7400 line structured light camera, add Gaussian noise and crop noise to some of them, and downsample the point cloud to 2048 points through preprocessing;
[0032] Specifically, the aggregate particle size distribution collected in step S1 is 4.75-9.5 mm, 9.5-13.2 mm, 13.2-16 mm, and 16-19 mm;
[0033] Specifically, the downsampling method in step 1 adopts the farthest point sampling method to ensure the uniform input scale and computational efficiency of the data;
[0034] Step S2: According to the "Highway Engineering Aggregate Test Procedure", square hole sieves with particle sizes of 4.75 mm, 9.5 mm, 13.2 mm, and 16 mm are used to mark the aggregates that cannot pass through according to the current aperture and divide them into training set and test set in a ratio of 7:3.
[0035] Step S3: Generate a target point cloud, by copying the source point cloud and transforming the coordinates to form a point cloud to be registered. Since this point cloud does not have noise, it is also called noise-free data;
[0036] Specifically, the value ranges of the three Euler angles and translation vectors of the coordinate transformation in step 3 are respectively between [0°, 45°] and [-0.5, 0.5], and their scopes are copied to the source point cloud.
[0037] Step 4: Establish a convolutional neural network with PCRNet as the backbone. The network structure includes feature extraction module, transformer feature fusion module, feature soft matching module and pose regression module.
[0038] Specifically, the feature extraction module in step S4 is used to extract features from the input three-dimensional data, specifically including first extracting 64-dimensional features through the MLP layer, then processing through the adaptive convolution layer to obtain 128-dimensional high-dimensional features containing multiple scales, and then changing the feature dimension to 1024 layers through the MLP layer;
[0039] Furthermore, the adaptive convolution in step 4 is used to adapt to point clouds of different shapes and densities. It mainly consists of a weight library, MLP and a convolution kernel function. The learnable parameter of the weight library is K. In the present invention, K=16 is taken to balance the diversity of the convolution kernel and the model burden. MLP is used to predict the normalized coefficient vector. The convolution kernel function is obtained by combining the weight matrix with the predicted corresponding coefficients, that is, weighted summing the weight matrix and the coefficient vector.
[0040] Specifically, the transformer feature fusion module in step S4 is used to effectively capture the correlation between two point clouds. The difference between it and the original transformer model is that the improved module uses the features obtained by the feature extraction module as the input of the encoder and the encoder, and adjusts the features by adding residual connections. In addition, in order to optimize its training efficiency, residual links and normalization operations are added after each sublayer.
[0041] Specifically, the feature soft matching module in step S4 is used to match the similarity between the source point cloud and the target point cloud feature vectors. The present invention uses a probabilistic method to locate the corresponding points, that is, to generate a probability mapping matrix from one set of feature sequences to another set of feature sequences. First, the 2-norm distance between each input feature vector and the target feature vector is calculated. The similarity of the two features is inversely proportional to their values, that is, the closer to 0, the higher the similarity. Then, the softmax function is applied to the distance to convert the feature vector into a probability distribution. The final soft mapping matrix is determined by the position of the maximum probability in each row.
[0042] Specifically, the pose regression module in step S4 is used to align the input point cloud, including softmax normalization layer, decentralization and SVD decomposition. First, the registration point cloud is softmax normalized to convert the similarity into probability. After decentralization, the covariance matrix is calculated and decomposed into rotation matrix and translation vector according to SVD.
[0043] Step 5: Use the data set produced in steps S1-S3 to train the point cloud registration network, and use the final model for coarse aggregate point cloud registration after testing it on different noise data.
[0044] Specifically, the loss function of the network training in step 5 includes the sum of two parts: the rotation matrix loss function and the translation vector loss function. The rotation matrix loss function is described as the sum of the squares of the two norms of the distance between the predicted value of the rotation matrix and the true value of the rotation matrix, and the translation vector loss function is described as the sum of the squares of the two norms of the distance between the predicted value of the translation vector and the true value of the translation vector.
[0045] Compared with the prior art, the present invention has the following technical features:
[0046] (1) Adaptive point cloud convolution is used to replace the PointNet module to solve the problem of poor ability to extract local structure and contextual information of point clouds. The replaced adaptive convolution can adaptively generate convolution kernels for feature extraction based on the position and feature information of the input points.
[0047] (2) SVD is used to replace the fully connected layer to calculate the rigid body transformation matrix, solving the problem that the parameter setting of the fully connected layer depends on expert experience and information loss. The replaced SVD algorithm can ignore the noise and local outliers in the point cloud and extract more robust features. It can also perform local rigid transformation calculations on each point based on the local features of the point cloud, thereby improving the registration accuracy.
[0048] (3) The Transformer module is added to realize feature fusion, which solves the problem that the traditional deep learning registration algorithm has a decreased accuracy due to ignoring the point cloud structure information. The improved model can establish the local structure relationship of the point cloud through the sub-attention mechanism and capture the global structure information in the point cloud.
[0049] Example 1
[0050] This embodiment discloses a method for establishing a coarse aggregate point cloud registration model based on deep learning. On the basis of the above embodiment, the following technical features are also disclosed:
[0051] In steps S1-S3, the sampling numbers of aggregates with particle sizes of 4.75-9.5 mm, 9.5-13.2 mm, 13.2-16 mm, and 16-19 mm are 1000, 1200, 1200, and 1000, respectively, and the training set and test set are divided into 7:3.
[0052] The data percentage of Gaussian and random clipping noise is 10%, the mean of Gaussian noise is 0, the standard deviation is 0.03, and the clipping ratio of random clipping noise is 20%. Figure 1 As shown, (a) (b) are schematic diagrams before and after downsampling, (c) (d) are schematic diagrams before and after matrix transformation, (e) (f) are schematic diagrams before and after Gaussian noise addition, and (g) (h) are schematic diagrams before and after cropping.
[0053] The convolutional neural network structure established in step S4 is as follows: Figure 2 As shown in Figure 2, the structure of the feature extraction module is as follows: Figure 3 shown.
[0054] In step S5, the batch size of the training set and the test set in the model training is set to 16, and the mean square error (MSE), root mean square error (RMSE) and mean absolute error (MAE) are used to evaluate the model effect. The training process is shown in the figure, where (a), (c) and (e) respectively show the MSE, RMSE and MAE of the rotation matrix, and (b), (d) and (f) show the MSE, RMSE and MAE of the translation vector. The calculation formulas of MSE, RMSE and MAE are as follows:
[0055]
[0056] Where n is the number of points in the point cloud, P i and Q i Represent the registered points and target points respectively. ||P i -Q i ||2 represents the Euclidean distance, that is:
[0057]
[0058] Among them, p ix 、p iy and p iz Respectively represent point P i The coordinates in the x, y and z directions, q ix ,q iy and q iz Respectively represent point Q i Coordinates in the x, y and z directions.
[0059] Figure 4 The training set and test set loss curves for the noise-free data training process. It can be seen that the overall error and the training error and test error on various evaluation indicators are on a downward trend. Both the training set and the test set drop sharply in the first 20 steps or so, and begin to converge before 100 training steps. After that, the curve flattens out and the model reaches the global optimum.
[0060] Figure 5 The registration effect of the model in this chapter on each gear is shown. Note that the point cloud size has nothing to do with the actual difference in the aggregate size of each gear. (a) to (d) correspond to the point cloud registration effects of aggregates at gears of 4.75, 9.5, 13.2, and 16, respectively.
[0061] Figure 6 The registration effect of different noise point clouds is demonstrated. (a), (b), and (c) are Gaussian noise point cloud data and the results after registration, respectively. (d), (e), and (f) are cropped noise point cloud data and the results after registration, respectively. It can be seen that the result point cloud after registration and the target point cloud are in the same coordinate system.
[0062] Figure 7 is the result of the accuracy test. As can be seen from the figure, based on the noise-free aggregate point cloud data, the model in this chapter has the smallest error value in each evaluation index. Compared with the FGR method, which performs best in non-learning methods, the various evaluation indicators of the model in this chapter are improved by 75.72%, 50.73%, 50.38%, 76.79%, 51.07%, and 59.78% respectively; compared with the DCP model, which performs best in the learning-based registration model, they are improved by 84.09%, 60.12%, 60.23%, 91.61%, 70.64%, and 76.89% respectively; compared with the original PCRNet model, they are improved by 96.78%, 82.06%, 82.65%, 99.08%, 90.27%, and 87.22% respectively. Therefore, the model in this chapter performs best and has the highest registration accuracy among all the comparison methods.
Claims
1. A coarse aggregate point cloud registration method based on Transformer, characterized in that The specific steps are: Step S1: Collect multiple groups of aggregate point cloud data with different particle size distributions based on the SR7400 line structured light camera, add Gaussian noise and crop noise to some of them, and downsample the point cloud to 2048 points through preprocessing; Step S2: According to the "Highway Engineering Aggregate Test Regulations", square hole sieves with particle sizes of 4.75 mm, 9.5 mm, 13.2 mm, and 16 mm are used to mark the aggregates that cannot pass through according to the current hole diameter and divide them into training set and test set in a ratio of 7:3; Step S3: Generate a target point cloud, by copying the source point cloud and transforming the coordinates to form a point cloud to be registered. Since this point cloud does not have noise, it is also called noise-free data; Step S4: Establish a convolutional neural network with PCRNet as the backbone. The network structure includes a feature extraction module, a transformer feature fusion module, a feature soft matching module, and a pose regression module. Step S5, using the data set produced in steps S1-S3 to train the point cloud registration network, and after testing with different noise data, the final model is used for coarse aggregate point cloud registration.
2. The coarse aggregate point cloud registration method based on Transformer according to claim 1, characterized in that: The aggregate particle size distributions collected in step S1 are 4.75-9.5 mm, 9.5-13.2 mm, 13.2-16 mm, and 16-19 mm, respectively.
3. The coarse aggregate point cloud registration method based on Transformer according to claim 1, characterized in that: The downsampling method in step S1 adopts the farthest point sampling method to ensure uniform data input scale and computational efficiency.
4. The coarse aggregate point cloud registration method based on Transformer according to claim 1, characterized in that: The value ranges of the three Euler angles and translation vectors of the coordinate transformation in step S3 are respectively between [0°, 45°] and [-0.5, 0.5], and their scopes are copied to the source point cloud.
5. The coarse aggregate point cloud registration method based on Transformer according to claim 1, characterized in that: The feature extraction module in step S4 is used to extract features from the input three-dimensional data, specifically including first extracting 64-dimensional features through the MLP layer, then processing through the adaptive convolution layer to obtain 128-dimensional high-dimensional features containing multiple scales, and then changing the feature dimension to 1024 layers through the MLP layer.
6. The coarse aggregate point cloud registration method based on Transformer according to claim 1, characterized in that: The adaptive convolution in step S4 is used to adapt to point clouds of different shapes and densities. It mainly consists of a weight library, MLP and a convolution kernel function. The learnable parameter of the weight library is K. In the present invention, K=16 is taken to balance the diversity of the convolution kernel and the model burden. MLP is used to predict the normalized coefficient vector. The convolution kernel function is obtained by combining the weight matrix with the predicted corresponding coefficients, that is, weighted summing the weight matrix and the coefficient vector.
7. The coarse aggregate point cloud registration method based on Transformer according to claim 1, characterized in that: The transformer feature fusion module in step S4 is used to effectively capture the correlation between two point clouds. The difference between it and the original transformer model is that the improved module uses the features obtained by the feature extraction module as the input of the encoder and encoder, and adjusts the features by adding residual connections. In addition, in order to optimize its training efficiency, residual links and normalization operations are added after each sublayer.
8. The coarse aggregate point cloud registration method based on Transformer according to claim 1, characterized in that: The feature soft matching module in step S4 is used to match the similarity between the feature vectors of the source point cloud and the target point cloud. The present invention uses a probabilistic method to locate corresponding points, that is, to generate a probability mapping matrix from one set of feature sequences to another set of feature sequences. First, the 2-norm distance between each input feature vector and the target feature vector is calculated. The similarity of the two features is inversely proportional to their values, that is, the closer to 0, the higher the similarity. Then, the softmax function is applied to the distance to convert the feature vector into a probability distribution. The final soft mapping matrix is determined by the position of the maximum probability in each row.
9. The coarse aggregate point cloud registration method based on Transformer according to claim 1, characterized in that: The pose regression module in step S4 is used to align the input point cloud, including softmax normalization layer, decentralization and SVD decomposition. First, the registered point cloud is softmax normalized to convert the similarity into probability. After decentralization, the covariance matrix is calculated and decomposed into rotation matrix and translation vector according to SVD.
10. The coarse aggregate point cloud registration method based on Transformer according to claim 1, characterized in that: The loss function of the network training in step S5 includes the addition of a rotation matrix loss function and a translation vector loss function. The rotation matrix loss function is described as the sum of the squares of the two norms of the distance between the rotation matrix prediction value and the rotation matrix true value, and the translation vector loss function is described as the sum of the squares of the two norms of the distance between the translation vector prediction value and the translation vector true value.