A point cloud registration method for arch dams based on deep learning and multi-objective feature input

Abstract

This invention discloses a method for arch dam point cloud registration based on deep learning and multi-target feature input, comprising: 1) arranging survey stations on the arch dam and scanning the arch dam surface using a ground-based 3D laser scanner to obtain point cloud data of the arch dam; 2) constructing an improved point cloud segmentation neural network to obtain segmentation results of the arch dam point cloud; 3) designing a feature extractor to extract the contour features of the segmented arch dam components; and 4) designing a rigid transformation estimation network to register the arch dam components and the rock mass contour as input data. The method first extracts dam components using a point cloud segmentation network based on PointNet++. Then, combining the arch dam's environmental characteristics, contour features are obtained using the Transformer framework and the clustering plane boundary extraction method, respectively, based on the differences between the dam body and the rock mass surface. The contour features are input into the registration network as guiding information, and the 12-dimensional feature vector is optimized using an attention mechanism and an improved MLP. Finally, a rigid registration network is constructed to calculate the transformation matrix.

Description

Technical Field

[0001] The present invention belongs to the technical field of arch dam point cloud registration, and specifically is an arch dam point cloud registration method based on deep learning and multi-target feature input. Background Art

[0002] An arch dam differs from traditional building structures in that it curves upstream in a plane, its ends clinging to the canyon walls, forming a curved spatial shell. Because an arch dam can transfer some of the horizontal load to the "V"-shaped rock masses on both sides, the arch dam area not only features tall arch dams but also towering rock masses with good lithology on both sides. During its service life, an arch dam is affected by water levels and environmental factors, causing varying degrees of deformation. Extreme deformation can lead to instability. Therefore, the construction of a digital platform for monitoring arch dam operations is particularly important. Traditional detection methods primarily rely on point detection and lack comprehensive control over the overall deformation of the arch dam. In recent years, the rapid development of terrestrial laser scanning technology (TLS) has prompted numerous researchers to actively explore its potential for application in the construction of digital platforms for arch dams, aiming to enhance the information management capabilities of arch dam structures.

[0003] 3D laser scanning technology can quickly generate high-precision 3D surface models and has been applied in tunnel engineering, bridge engineering, geotechnical engineering, and other fields. These projects are characterized by large size and long distances. In order to collect complete 3D data of the engineering surface, multi-station and multi-angle scanning combined with point cloud registration becomes an important step. Existing point cloud registration methods are mainly divided into two categories: traditional point cloud registration methods and deep learning-based point cloud methods. Traditional point cloud registration methods include classic methods such as ICP, RANSAC, and 4PCS. However, these traditional methods are sensitive to initial pose and parameters, resulting in low registration accuracy. Summary of the Invention

[0004] The purpose of the present invention is to address the shortcomings of the existing technology and propose a deep neural network point cloud registration method based on the structural features of the arch dam. The feature extraction network is used to extract the contours of the segmented arch dam components, and then the multi-dimensional feature vectors of these contours are used as the input of the registration network to perform the registration work. At the same time, before performing the registration, feature extraction is performed on the two point cloud data to be registered to highlight their sharp contour features, so that the registration network only focuses on these sharp contour features, thereby achieving more accurate alignment.

[0005] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions.

[0006] A method for arch dam point cloud registration based on deep learning and multi-objective feature input includes the following steps:

[0007] Step S1: Arrange measurement stations on the arch dam, scan the surface of the arch dam using a ground-based three-dimensional laser scanner, and obtain point cloud data of the arch dam;

[0008] Step S2: constructing an improved point cloud segmentation neural network to obtain the segmentation result of the arch dam point cloud;

[0009] Based on the traditional PointNet++, sampling processing and local area combination are added to form a set abstraction layer. High-dimensional features are extracted through multiple set abstraction layers, and the final segmentation result is obtained through skip connections and reverse interpolation.

[0010] Step S3: designing a feature extractor to extract the contour features of the segmented arch dam components;

[0011] Step S4: design a rigid transformation estimation network and register the arch dam components and rock mass contours as input data.

[0012] Specifically, the improved point cloud segmentation neural network described in step S2 makes improvements to the neural network data output based on the traditional PointNet++ to adapt the point cloud segmentation. The improved segmentation process of the arch dam point cloud is as follows:

[0013] Step S21: sampling and processing of point cloud data;

[0014] The input point cloud data is sampled using the farthest point sampling, the sampled data is connected and denormalized, and the tensor position information is saved;

[0015] Step S22: performing local area combination on the downsampled points;

[0016] Set the scale of the domain points and use the ball query method to query the domain point set S within the radius R. The formula is expressed as:

[0017] ;

[0018] In the above formula, is the number of points in the neighborhood point set S; The size of the local area combination is set; Indicates upsampling of the point set S;

[0019] Step S23: Using PointNet to extract features in each local area combination;

[0020] When combining local areas, multi-scale grouping (MSG) is used. PointNet is used to extract features from areas of different radii, and then these features are spliced ​​together. The process of extracting features using PointNet is expressed as follows:

[0021] ;

[0022] In the above formula, Represents a multilayer perceptron with one hidden layer; is the output of PointNet; is the input of PointNet; It is a transformation network, which represents the rotation and translation of the point cloud; represents the first conversion network; It is the maximum pooling, which is used to obtain the global features of the point cloud;

[0023] Step S24: obtaining the final segmentation result through skip connection and reverse interpolation;

[0024] When performing point cloud segmentation tasks, the reverse interpolation upsampling method is used to upsample the features of the point cloud after the set abstraction layer, and the features of the corresponding layer of the encoder are directly spliced ​​into the decoder through skip connections to obtain the final segmentation result;

[0025] Step S25: trace back to the original tensor position information in step S21 according to the final segmentation result of step S24, restore the original coordinates, and eliminate the spatial deviation between the segmentation result after reverse interpolation and the original data.

[0026] Specifically, the feature extractor in step S3 has a built-in rock mass contour feature extraction algorithm. The calculation steps of the rock mass contour feature extraction algorithm are as follows:

[0027] Step S31, down-sampling the original point cloud and calculating the normal vectors of the remaining points;

[0028] Step S32: Apply the region growing algorithm to perform preliminary clustering on the point cloud, grouping points with similar normal vectors into one category, and then further refine the clustering results through the clustering algorithm to separate and extract clear independent planes and plane features;

[0029] Step S33: After extracting the plane features, for each point in the point cloud Calculate the normal vector and construct the tangent plane , point Neighborhood point set Project onto the tangent plane to form a new point set ;

[0030] Step S34: In the point set , select a reference point , and point As the origin, construct a local coordinate system , the local coordinate system is Direction Axis, normal vector for axis, for axis;

[0031] Step S35: Calculate point set Other points arrive Vector With the coordinate axis The clockwise angle , take the difference between the two adjacent angles to get the angle set ,in ; Find the set Maximum angle ,when Greater than threshold When , the point is considered as an edge point, and the boundary points of the independent plane are extracted;

[0032] Step S36: Use the improved Transformer model to extract the shape features of the dam point cloud. The relationship function of the vector attention mechanism in the improved Transformer model y i The expression is as follows:

[0033] ;

[0034] ;

[0035] In the above formula, for A collection of fields of points; is the target point; for points in the neighborhood; 、 、 The query, key, and value in the traditional Transformer model; 、 is a multi-layer perceptron; represents the normalization function; Indicates positional encoding; and is the coordinate vector.

[0036] Specifically, the improved Transformer model network uses four attributes of point cloud coordinates, curvature, density and normal vector as the input of the model, where the normal vector is replaced by the normal vector feature. The calculation formula of the normal vector feature of the point cloud is as follows:

[0037] ;

[0038] In the above formula, is the number of points in the neighborhood; for point The normal vector characteristic degree of is a point in the neighborhood; for point Normal vector at ; is the normal vector of each point in the neighborhood.

[0039] Specifically, the structure of the rigid transformation estimation network in step S4 includes an attention mechanism layer, an improved MLP layer, a feature extraction layer, a rigid change estimation layer and a loss function;

[0040] The attention mechanism layer focuses the model’s computational resources on important information through attention distribution. The mathematical expression of the attention mechanism is as follows:

[0041] ;

[0042] ;

[0043] In the above formula, represents the channel attention map; is the activation function; MLP stands for multi-layer perceptron with one hidden layer; is the input feature; is the output of the attention mechanism; 、 、 Represent average pooling, maximum pooling and random pooling respectively;

[0044] The improved MLP layer, based on ResMLP, designs Point-ResMLP (P-RMLP) suitable for point cloud data. It adopts a sequence-to-sequence architecture. The improved MLP layer contains a Gelu activation function, two MLPs and two affine transformations, and adds residual connections to ensure that deeper network structures can be fully trained. The improved MLP layer formula is expressed as follows:

[0045] ;

[0046] ;

[0047] In the above formula, It is the input of the improved MLP layer; and is a learnable weight vector; and is the main learnable weight matrix of this layer; Express Perform affine transformation; To vector Convert to a diagonal matrix; Y is the output of the improved MLP layer; Aff( ) is the affine transformation; GeLU( ) is the activation function;

[0048] The feature extraction layer is used to construct a more comprehensive point cloud feature. The radius is established The local neighborhood of , , extracted features Expressed as:

[0049] ;

[0050] In the above formula, For local neighborhood points within minus coordinates of It is based on the point pair feature PPF and supplements the point cloud features in the local area, which can be expressed as:

[0051] ;

[0052] In the above formula, and They are points and The normal vector of Indicates the angle; for point and the distance between them; for norm; Indicates a point and The curvature difference, Indicates a point density.

[0053] Furthermore, suppose two 3D point sets and , and these two point sets can be transformed by an affine transformation {R, t}, where is the original point cloud, For the target point cloud, the rigid change estimation layer uses a matching matrix Record the matching of two point sets, where If you click Corresponding to point ,but ,otherwise , in order to Better mapping to Above, the following conditions should be met:

[0054] ;

[0055] In the above formula, I Point set the number of midpoints; J Point set The number of midpoints; R is the linear transformation matrix; t is the translation matrix; is an annealing parameter, the purpose of which is to make the distance greater than The points are considered as outliers;

[0056] Since the matching matrix M Subject to the constraints of the matrix rows and columns and To simplify the problem, the matching matrix Perform relaxation changes to obtain , , is an arbitrary symbol; then the deterministic annealing algorithm is used to solve the minimum value of formula (12), where Initialized as:

[0057] ;

[0058] In the above formula, is the annealing parameter, The smaller the initial value of , the higher the overall randomness, and the less likely it is to fall into a local minimum. It will gradually increase during the iterative calculation process. hour, will approach {0,1}; where the annealing parameter and Prediction is obtained through a secondary network;

[0059] Substituting the learned feature distance for the spatial distance in formula (13), we get:

[0060] ;

[0061] In the above formula, and are the features of the point cloud;

[0062] After completing feature extraction and and After the prediction of the annealing parameters, the Sinkhorn normalization process is performed, and the rows and columns are normalized alternately to obtain a double random matrix from all positive square matrices, and then the optimal distribution is obtained; then the rigid transformation is estimated using SVD, and for each original point cloud , calculate the target point cloud Corresponding coordinates :

[0063] ;

[0064] In the above formula, Point Cloud Corresponding point cloud coordinates of is the matching matrix Elements in Point Cloud The point in

[0065] The loss function is the actual transformation and prediction transform Considering the different overlaps of the registration data in the actual registration process, the Huber loss function is used, but there are many outliers and The influence of the norm will lead to fewer intrinsic values, so the increase in quadratic loss L is expressed as:

[0066] ;

[0067] In the above formula, is the actual rotation matrix; Point set The point in is the actual translation vector; is the predicted rotation matrix; is the predicted translation vector; It is a hyperparameter and needs to be set manually.

[0068] Compared with the prior art, the present invention has the following beneficial effects:

[0069] (1) An arch dam is a large shell structure that is tightly attached to the rock masses on both sides. To improve the efficiency of arch dam point cloud registration, the present invention uses a deep learning algorithm to remove the arch dam surface with single features and segment the arch dam components and surrounding rock masses with angular features.

[0070] (2) In view of the influence of different surface features of arch dam components and surrounding high rock masses on the registration network, the present invention designs a point cloud contour extraction network model based on Transformer structure for extracting the arch dam point cloud contour and a clustering-based plane boundary extraction method for extracting rock mass contour features, providing more accurate feature input for subsequent rigid registration work.

[0071] (3) Aiming at the complex weight sharing and inefficient MLP of the registration network, the present invention simplifies the cross-patch sublayer on the basis of ResMLP and designs an MLP that is efficient and suitable for point cloud data. In order to improve the utilization efficiency and computing resources of multi-dimensional point cloud features, the present invention constructs a channel attention mechanism based on average pooling, maximum pooling and random pooling. According to the positional relationship between points, a 12-dimensional feature extraction network model is constructed to extract deeper point cloud features of the arch dam point cloud. Based on the learned point cloud features, a rigid registration network is constructed to achieve more accurate registration. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a flowchart of an arch dam point cloud registration method based on deep learning and multi-target feature input;

[0073] Figure 2 This is a flow chart of the algorithm for extracting the contour features of the rock mass below the dam according to the present invention;

[0074] Figure 3 It is a schematic diagram of the local reference coordinate system constructed by the present invention;

[0075] Figure 4 Schematic diagram of the Transformer-based feature extractor of the present invention;

[0076] Figure 5 It is a schematic diagram of the extraction process of the dam components of the present invention;

[0077] Figure 6 Schematic diagram of the attention mechanism process of the present invention;

[0078] Figure 7 Schematic diagram of the improved MLP process of the present invention applied to point cloud data;

[0079] Figure 8 It is a schematic diagram of the feature extraction network process of the present invention;

[0080] Figure 9 Schematic diagram of the rigid registration network process of the present invention;

[0081] Figure 10 1. It is a layout diagram of the arch dam measuring stations in an embodiment of the present invention;

[0082] Figure 11 This is a visualization data diagram of different measuring stations in the dam area according to an embodiment of the present invention;

[0083] Figure 12 1 is a schematic diagram of visualization results of dam components extracted from some measuring stations in an embodiment of the present invention;

[0084] Figure 13 2 is a schematic diagram of the contour feature extraction result according to an embodiment of the present invention;

[0085] Figure 14 2. A comparison diagram of the contour extraction results of arch dam components in an embodiment of the present invention;

[0086] Figure 15 is a schematic diagram of point cloud registration results in different scenarios according to an embodiment of the present invention;

[0087] Figure 16 2 is a comparison chart of the registration results of four sets of completely overlapping point clouds using different methods in an embodiment of the present invention;

[0088] Figure 17 2. A comparison diagram of the registration results of four sets of partially overlapping point clouds using different methods according to an embodiment of the present invention;

[0089] Figure 18 is the registration recall rate of partially overlapping point cloud data under different RMSE thresholds in the embodiment of the present invention;

[0090] Figure 19 is the registration recall rate of partially overlapping point cloud data under different chamfer distance thresholds in the embodiment of the present invention. DETAILED DESCRIPTION

[0091] In order to facilitate those skilled in the art to understand and implement the present invention, each step of the method proposed in the present invention is described in detail below. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms also fall within the scope defined by the claims appended hereto.

[0092] Example

[0093] like Figure 1 As shown, the present invention discloses a method for arch dam point cloud registration based on deep learning and multi-target feature input, comprising the following steps:

[0094] Step S1: Arrange measurement stations on the arch dam, scan the surface of the arch dam using a ground-based three-dimensional laser scanner, and obtain point cloud data of the arch dam;

[0095] Step S2: constructing an improved point cloud segmentation neural network to obtain the segmentation result of the arch dam point cloud;

[0096] Based on the traditional PointNet++, sampling processing and local area combination are added to form a set abstraction layer. High-dimensional features are extracted through multiple set abstraction layers, and the final segmentation result is obtained through skip connections and reverse interpolation.

[0097] Step S3: designing a feature extractor to extract the contour features of the segmented arch dam components;

[0098] Step S4: design a rigid transformation estimation network and register the arch dam components and rock mass contours as input data.

[0099] Specifically, the improved point cloud segmentation neural network described in step S2 makes improvements to the neural network data output based on the traditional PointNet++ to adapt the point cloud segmentation. The improved segmentation process of the arch dam point cloud is as follows:

[0100] Step S21: sampling and processing of point cloud data;

[0101] The input point cloud data is sampled using the farthest point sampling, the sampled data is concatenated and denormalized, and the tensor position information is saved;

[0102] Step S22: performing local area combination on the downsampled points;

[0103] Set the scale of the domain points and use the ball query method to query the domain point set S within the radius R. The formula is expressed as:

[0104] ;

[0105] In the above formula, is the number of points in the neighborhood point set S; The size of the local area combination is set; Indicates upsampling of the point set S;

[0106] Step S23: Using PointNet to extract features in each local area combination;

[0107] When combining local areas, multi-scale grouping (MSG) is used. PointNet is used to extract features from areas of different radii, and then these features are spliced ​​together. The process of extracting features using PointNet is expressed as follows:

[0108] ;

[0109] In the above formula, Represents a multilayer perceptron with one hidden layer; is the output of PointNet; is the input of PointNet; It is a transformation network, which represents the rotation and translation of the point cloud; represents the first conversion network; It is the maximum pooling, which is used to obtain the global features of the point cloud;

[0110] Step S24: obtaining the final segmentation result through skip connection and reverse interpolation;

[0111] When performing point cloud segmentation tasks, the reverse interpolation upsampling method is used to upsample the features of the point cloud after the set abstraction layer, and the features of the corresponding layer of the encoder are directly spliced ​​into the decoder through skip connections to obtain the final segmentation result;

[0112] Step S25: trace back to the original tensor position information in step S21 according to the final segmentation result of step S24, restore the original coordinates, and eliminate the spatial deviation between the segmentation result after reverse interpolation and the original data.

[0113] Specifically, the feature extractor in step S3 has a built-in rock mass contour feature extraction algorithm, such as Figure 2 As shown in Figure 2, the calculation steps of the rock mass contour feature extraction algorithm are as follows:

[0114] Step S31, down-sampling the original point cloud and calculating the normal vectors of the remaining points;

[0115] Step S32: Apply the region growing algorithm to perform preliminary clustering on the point cloud, grouping points with similar normal vectors into one category, and then further refine the clustering results through the clustering algorithm to separate and extract clear independent planes and plane features;

[0116] Step S33: After extracting the plane features, for each point in the point cloud Calculate the normal vector and construct the tangent plane , point Neighborhood point set Project onto the tangent plane to form a new point set ;

[0117] Step S34: Figure 3 As shown, in the point set , select a reference point , and point As the origin, construct a local coordinate system , the local coordinate system is Direction Axis, normal vector for axis, for axis;

[0118] Step S35: Calculate point set Other points arrive Vector With the coordinate axis The clockwise angle , take the difference between the two adjacent angles to get the angle set ,in ; Find the set Maximum angle ,when Greater than threshold When , the point is considered as an edge point, and the boundary points of the independent plane are extracted;

[0119] Step S36: Figure 4 As shown in the figure, the improved Transformer model is used to extract the shape features of the dam point cloud. The relationship function of the vector attention mechanism in the improved Transformer model is y i The expression is as follows:

[0120] ;

[0121] ;

[0122] In the above formula, for A collection of fields of points; is the target point; for points in the neighborhood; 、 、 The query, key, and value in the traditional Transformer model; 、 is a multi-layer perceptron; represents the normalization function; Indicates positional encoding; and is the coordinate vector.

[0123] Furthermore, if Figure 5 As shown in the figure, the improved Transformer model network uses four attributes of point cloud coordinates, curvature, density and normal vector as the input of the model, where the normal vector is replaced by the normal vector feature. The calculation formula of the normal vector feature of the point cloud is as follows:

[0124] ;

[0125] In the above formula, is the number of points in the neighborhood; for point The normal vector characteristic degree of is a point in the neighborhood; for point Normal vector at ; is the normal vector of each point in the neighborhood.

[0126] Specifically, the structure of the rigid transformation estimation network in step S4 includes an attention mechanism layer, an improved MLP layer, a feature extraction layer, a rigid change estimation layer and a loss function;

[0127] like Figure 6 As shown in Figure 2, the attention mechanism layer focuses the model's computing resources on important information through attention allocation. The mathematical expression of the attention mechanism is as follows:

[0128] ;

[0129] ;

[0130] In the above formula, represents the channel attention map; is the activation function; MLP stands for multi-layer perceptron with one hidden layer; is the input feature; is the output of the attention mechanism; 、 、 Represent average pooling, maximum pooling and random pooling respectively;

[0131] like Figure 7 As shown in the figure, the improved MLP layer is based on ResMLP, and Point-ResMLP (P-RMLP) suitable for point cloud data is designed. It adopts a sequence-to-sequence architecture. The improved MLP layer contains a Gelu activation function, two MLPs and two affine transformations, and adds residual connections to ensure that deeper network structures can be fully trained. The improved MLP layer formula is expressed as follows:

[0132] ;

[0133] ;

[0134] In the above formula, It is the input of the improved MLP layer; and is a learnable weight vector; and is the main learnable weight matrix of this layer; Express Perform affine transformation; To vector Convert to a diagonal matrix; Y is the output of the improved MLP layer; Aff( ) is the affine transformation; GeLU( ) is the activation function;

[0135] like Figure 8As shown, the feature extraction layer constructs a more comprehensive point cloud feature for any point The radius is established The local neighborhood of , , extracted features Expressed as:

[0136] ;

[0137] In the above formula, For local neighborhood points within ( )minus coordinates of It is based on the point pair feature PPF and supplements the point cloud features in the local area, which can be expressed as:

[0138] ;

[0139] In the above formula, and They are points and The normal vector of Indicates the angle; for point and the distance between them; for norm; Indicates a point and The curvature difference, Indicates a point density.

[0140] Furthermore, suppose two 3D point sets and , and these two point sets can be transformed by an affine transformation {R, t}, where is the original point cloud, For the target point cloud, the rigid change estimation layer uses a matching matrix Record the matching of two point sets, where If you click Corresponding to point ,but ,otherwise , in order to Better mapping to Above, the following conditions should be met:

[0141] ;

[0142] In the above formula, I Point set the number of midpoints; J Point set The number of midpoints; R is the linear transformation matrix; t is the translation matrix; is an annealing parameter, the purpose of which is to make the distance greater than The points are considered as outliers;

[0143] Since the matching matrix M Subject to the constraints of the matrix rows and columns and To simplify the problem, the matching matrix Perform relaxation changes to obtain , , is an arbitrary symbol; then the deterministic annealing algorithm is used to solve the minimum value of formula (12), where Initialized as:

[0144] ;

[0145] In the above formula, is the annealing parameter, The smaller the initial value of , the higher the overall randomness, and the less likely it is to fall into a local minimum. It will gradually increase during the iterative calculation process. hour, will approach {0,1}; where the annealing parameter and Prediction is obtained through a secondary network.

[0146] Substituting the learned feature distance for the spatial distance in formula (13), we get:

[0147] ;

[0148] In the above formula, and are the features of the point cloud;

[0149] After completing feature extraction and and After the prediction of the annealing parameters, the Sinkhorn normalization process is performed, and the rows and columns are normalized alternately to obtain a double random matrix from all positive square matrices, and then the optimal distribution is obtained; then the rigid transformation is estimated using SVD, and for each original point cloud , calculate the target point cloud Corresponding coordinates :

[0150] ;

[0151] In the above formula, Point Cloud Corresponding point cloud coordinates; is the matching matrix Elements in Point Cloud The point in

[0152] The loss function is the actual transformation and prediction transform Considering the different overlaps of the registration data in the actual registration process, the Huber loss function is used, but there are many outliers and The influence of the norm will lead to fewer intrinsic values, so the increase in quadratic loss L is expressed as:

[0153] ;

[0154] In the above formula, is the actual rotation matrix; Point set The point in is the actual translation vector; is the predicted rotation matrix; is the predicted translation vector; It is a hyperparameter and needs to be set manually.

[0155] The technical effect of the method of the present invention is further illustrated below through a specific example of arch dam point cloud registration.

[0156] In this example, a Trimble TX8 terrestrial 3D laser scanner was used to perform a point cloud registration test on a prototype arch dam in Jiangxi Province. Three-dimensional point cloud data was obtained through multi-station scanning, resulting in complete and high-precision dam area data. Spherical targets were then used to complete multi-station cloud stitching. Figure 10 As shown in the example, 9 measuring stations are arranged on the dam crest road and 4 measuring stations are arranged on the downstream dam surface. The data volume of each measuring station is shown in Table 1 below. Figure 11 Shown are the corresponding station visualization data.

[0157] Table 1. Point cloud information of each measuring station

[0158] ;

[0159] ① Experimental environment of this example

[0160] This example is based on the PyTorch framework and runs on a server equipped with an Intel(R) Core™ i7 12700kf CPU, 128GB of RAM, an NVIDIA RTX 4090, CUDA 11.7, and cuDNN v8.5.0. It uses the Adam optimizer with an initial learning rate of 0.0001.

[0161] ②Extraction results of dam components

[0162] In this example, there are four main types of components used for point cloud registration: guardhouse, flood outlet, dam top road and rock mass. Figure 12 As shown in Figure 2, this example visualizes selected observation stations from both the dam crest and the dam floor. Overall, the extraction results are highly accurate, with the arch dam achieving higher accuracy than the rock mass below the dam. The highest extraction accuracy for the dam crest road reached 98.59%, with a mean Intersection Over Union (IoU) of 0.97. Specific extraction accuracies are shown in Table 2.

[0163] Table 2. Extraction accuracy of dam components

[0164] ;

[0165] ③Contour feature extraction results

[0166] In this example, a point cloud contour extraction network model based on the Transformer structure was constructed and applied to the three parts of the guardhouse, the flood outlet and the dam top road. In addition, a clustering-based plane boundary extraction method was used to extract the rock mass contour features. Figure 13 The following is the result of contour feature extraction. Figure 13 The points in medium gray are non-feature points, and the other points are contour feature points.

[0167] In this example, the proposed point cloud contour extraction method is compared with the point cloud contour feature extraction methods based on surface curvature, normal vector and density. The comparison results are shown in Figure 2. Figure 14 and as shown in Table 3 below.

[0168] Table 3. Accuracy of feature line extraction of four point clouds by different methods

[0169] ;

[0170] ④Registration accuracy

[0171] 1. Arch dam registration results

[0172] Figure 15 (a)-(c) in the figure are the original states of the clouds at different observation stations. Figure 15 (d)-(f) in the figure are the point cloud registration results, as shown in Table 4 below. Figure 15Registration results for the three scenes.

[0173] Table 4. Quantitative evaluation of point cloud registration accuracy in different scenarios

[0174] ;

[0175] Visually, the data of the three scenes all achieved good results. From the quantitative results, scene (a) has the best registration effect. The main reason for the difference in registration accuracy may be the components of this scene. Figure 15 The middle scene (a) contains the measured data from sites 7 and 8. The two measuring stations are located on the left and right piers of the spillway, respectively. The point cloud data focuses on the spillway and the dam crest road, with the dam body and vegetation as secondary components. Figure 15 Scene (b) shows the measured data from sites 1 and 2, which are located at the top of the guardhouse and the abutment on the right bank of the arch dam, respectively. The data is mainly composed of vegetation, guardhouses, and the dam crest road. Since the dam surface and vegetation are not characteristic parts, they are not involved in the registration work. In addition, the vegetation point cloud accounts for a large proportion in scene (b), resulting in fewer characteristic point clouds of the stations. In scene (a), the proportion of the dam surface point cloud is small and does not affect the extraction of the station cloud features. Scene (c) shows sites 12 and 13. Both stations are located on the bedrock downstream of the arch dam. The point cloud data mainly consists of the dam surface, rock mass, spillway, and vegetation. In this scene, the main components to be registered are the rock mass and spillway, and the extracted characteristic point cloud can well support the registration work.

[0176] like Figure 16 As shown in Figure 2, the results of registering four sets of completely overlapping point clouds (guard booth, dam top road, flood discharge outlet, rock mass) using different methods are shown. Figure 16 (a) is the result of the complete overlapping point cloud registration of the booth. Figure 16 (b) is the result of fully overlapping point cloud registration of the dam top road. Figure 16 Middle (c) is the registration result of the completely overlapping point cloud of the flood outlet. Figure 16 Middle (d) is the result of complete overlapping point cloud registration of rock mass. Figure 16 The first column shows the original positions of the four groups of point clouds, and the remaining columns show the registration results of 4PCS, CPD, FPFH, PCA, RANSAC and the method of the present invention. Figure 16 It can be seen that the registration effects of the method of the present invention, CPD, FPFH and RANSAC are all relatively ideal in terms of vision. Figure 17 As shown in Figure 2, the results of registering four sets of partially overlapping point clouds (guard booth, dam top road, spillway, rock mass) using different methods are shown. Figure 17 (a) is the result of the complete overlapping point cloud registration of the booth. Figure 17 (b) is the result of fully overlapping point cloud registration of the dam top road. Figure 17Middle (c) is the registration result of the completely overlapping point cloud of the flood outlet. Figure 17 (d) is the result of complete overlapping point cloud registration of rock mass. Figure 17 As can be seen, for the four sets of overlapping point cloud data, only CPD achieved good results for the spillway and rock mass point clouds, while the proposed method performed well for all data. Tables 5 and 6 below show the registration accuracy of different methods for four sets of fully and partially overlapping point clouds (guard booth, dam crest road, spillway, and rock mass), respectively. The proposed method achieved the lowest RMSE and chamfer distance among the various methods, demonstrating a clear accuracy advantage.

[0177] Table 5. RMSE and chamfer distance of four groups of completely overlapping point clouds under different methods (unit: m)

[0178] ;

[0179] Table 6. RMSE and chamfer distance of four groups of partially overlapping point clouds under different methods (unit: m)

[0180] ;

[0181] In this example, the recall rate of the registration method is also evaluated. Figure 18 and Figure 19 The following figure shows the registration recall rates for four sets of overlapping point cloud data: the guardhouse, dam crest road, spillway, and rock mass, at different RMSE and chamfer distance thresholds. The five methods, 4PCS, CPD, FPFH, PCA, and RANSAC, performed inconsistently across several data sets, and all exhibited low registration recall rates at lower thresholds. However, the method proposed in this paper achieved high registration recall rates for all four sets of point cloud data.

[0182] 2. Ablation Experiment

[0183] The present invention conducts a large number of ablation studies on the field test dataset to demonstrate the effectiveness of the introduced attention mechanism and feature extraction method. The results are shown in Table 7 below.

[0184] Table 7. Ablation study on the measurement data validation set

[0185] ;

[0186] In the table: CA is the channel attention mechanism; IM is the improved MLP; FE is the feature extractor.

[0187] Attention Mechanism: To verify the impact of the improved MLP and feature extraction network proposed in this invention on registration, these two modules were removed separately, and performance was evaluated using the output transformation R,t as the final estimate. Based on the second row of Table 7, the third and fourth rows demonstrate that both CA and IM can improve registration accuracy. The first and fifth rows demonstrate that the combined effects of CA and IM play a positive role in registration. Feature Extractor: The feature extractor proposed in this invention is one of the key data processing methods. To demonstrate its important role in registration, it was included in the ablation experiment. Comparing the first, second, and fifth rows of Table 7 shows that the feature extractor has a significant impact. Without the feature extractor, the maximum registration error (RMSE) reached 24.08m, almost indicating registration failure. Even with CA and IM, the registration error reached 13.05m (RMSE), indicating unsatisfactory registration results. However, with the addition of the feature extractor in the second, third, fourth, and sixth rows, registration accuracy was significantly improved.

[0188] In summary, this paper uses a prototype arch dam as an example, using RMSE, chamfer distance, and recall as performance evaluation metrics to compare five commonly used registration algorithms (4PCS, CPD, FPFH, PCA, and RANSAC). Experimental results demonstrate that the proposed method achieves superior registration results, both in terms of quantitative analysis of performance evaluation metrics and comparison of visual results. Registration results for multi-station arch dam point cloud data demonstrate that the proposed method is effective for large-scale arch dams. Ablation experiments demonstrate that the proposed feature extraction method is necessary and can serve as a reference for solving the difficult problem of point cloud registration of large-scale hydraulic structures in practical engineering.

[0189] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.

Claims

1. A method for arch dam point cloud registration based on deep learning and multi-target feature input, characterized in that: The following steps are involved: Step S1: Arrange measurement stations on the arch dam, scan the surface of the arch dam using a ground-based three-dimensional laser scanner, and obtain point cloud data of the arch dam; Step S2: constructing an improved point cloud segmentation neural network to obtain the segmentation result of the arch dam point cloud; Based on the traditional PointNet++, sampling processing and local area combination are added to form a set abstraction layer. High-dimensional features are extracted through multiple set abstraction layers, and the final segmentation result is obtained through skip connections and reverse interpolation. Step S3: designing a feature extractor to extract the contour features of the segmented arch dam components; The feature extractor has a built-in rock mass contour feature extraction algorithm. The calculation steps of the rock mass contour feature extraction algorithm are as follows: Step S31, down-sampling the original point cloud and calculating the normal vectors of the remaining points; Step S32: Apply the region growing algorithm to perform preliminary clustering on the point cloud, grouping points with similar normal vectors into one category, and then further refine the clustering results through the clustering algorithm to separate and extract clear independent planes and plane features; Step S33: After extracting the plane features, for each point in the point cloud Calculate the normal vector and construct the tangent plane , point Neighborhood point set Project onto the tangent plane to form a new point set ; Step S34: In the point set , select a reference point , and point As the origin, construct a local coordinate system , the local coordinate system is Direction Axis, normal vector for axis, for axis; Step S35: Calculate point set Other points arrive Vector With the coordinate axis The clockwise angle , take the difference between the two adjacent angles to get the angle set ,in ; Find the set Maximum angle ,when Greater than threshold When , the point is considered as an edge point, and the boundary points of the independent plane are extracted; Step S36: Use the improved Transformer model to extract the shape features of the dam point cloud. The relationship function of the vector attention mechanism in the improved Transformer model y i The expression is as follows: ； ； In the above formula, for A collection of fields of points; is the target point; for points in the neighborhood; 、 、 The query, key, and value in the traditional Transformer model; 、 is a multi-layer perceptron; represents the normalization function; Indicates positional encoding; and is the coordinate vector; Step S4: design a rigid transformation estimation network and register the arch dam components and rock mass contours as input data.

2. The arch dam point cloud registration method based on deep learning and multi-objective feature input according to claim 1 is characterized in that: The improved point cloud segmentation neural network described in step S2 is based on the traditional PointNet++ and improves the neural network data output to adapt to point cloud segmentation. The improved segmentation process of the arch dam point cloud is as follows: Step S21: sampling and processing of point cloud data; The input point cloud data is sampled using the farthest point sampling, the sampled data is connected and denormalized, and the tensor position information is saved; Step S22: performing local area combination on the downsampled points; Set the scale of the domain points and use the ball query method to query the domain point set S within the radius R. The formula is expressed as: ； In the above formula, is the number of points in the neighborhood point set S; The size of the local area combination is set; Indicates upsampling of the point set S; Step S23: Using PointNet to extract features in each local area combination; When combining local areas, multi-scale grouping (MSG) is used. PointNet is used to extract features from areas of different radii, and then these features are spliced ​​together. The process of extracting features using PointNet is expressed as follows: ； In the above formula, Represents a multilayer perceptron with one hidden layer; is the output of PointNet; is the input of PointNet; It is a transformation network, which represents the rotation and translation of the point cloud; represents the first conversion network; It is the maximum pooling, which is used to obtain the global features of the point cloud; Step S24: obtaining the final segmentation result through skip connection and reverse interpolation; When performing point cloud segmentation tasks, the reverse interpolation upsampling method is used to upsample the features of the point cloud after the set abstraction layer, and the features of the corresponding layer of the encoder are directly spliced ​​into the decoder through skip connections to obtain the final segmentation result; Step S25: trace back to the original tensor position information in step S21 according to the final segmentation result of step S24, restore the original coordinates, and eliminate the spatial deviation between the segmentation result after reverse interpolation and the original data.

3. The arch dam point cloud registration method based on deep learning and multi-objective feature input according to claim 1 is characterized in that: The improved Transformer model network uses four attributes of point cloud coordinates, curvature, density and normal vector as the input of the model, where the normal vector is replaced by the normal vector feature. The calculation formula of the normal vector feature of the point cloud is as follows: ； In the above formula, is the number of points in the neighborhood; for point The normal vector characteristic degree of is a point in the neighborhood; for point Normal vector at ; is the normal vector of each point in the neighborhood.

4. The arch dam point cloud registration method based on deep learning and multi-objective feature input according to claim 1 is characterized in that: The structure of the rigid transformation estimation network in step S4 includes an attention mechanism layer, an improved MLP layer, a feature extraction layer, a rigid change estimation layer and a loss function; The attention mechanism layer focuses the model’s computational resources on important information through attention distribution. The mathematical expression of the attention mechanism is as follows: ； ； In the above formula, represents the channel attention map; is the activation function; MLP stands for multi-layer perceptron with one hidden layer; is the input feature; is the output of the attention mechanism; 、 、 Represent average pooling, maximum pooling and random pooling respectively; The improved MLP layer, based on ResMLP, is designed to be suitable for point cloud data. It adopts a sequence-to-sequence architecture. The improved MLP layer contains a Gelu activation function, two MLPs and two affine transformations, and adds residual connections to ensure that deeper network structures can be fully trained. The improved MLP layer formula is expressed as follows: ； ； In the above formula, It is the input of the improved MLP layer; and is a learnable weight vector; and is the main learnable weight matrix of this layer; Express Perform affine transformation; Indicates that the vector Convert to a diagonal matrix; Y is the output of the improved MLP layer; Aff( ) is the affine transformation; GeLU( ) is the activation function; The feature extraction layer is used to construct a more comprehensive point cloud feature. The radius is established The local neighborhood of , , extracted features Expressed as: ； In the above formula, For local neighborhood points within minus coordinates; It is based on the point pair feature PPF and supplements the point cloud features in the local area, which can be expressed as: ； In the above formula, and They are points and The normal vector of Indicates the angle; for point and the distance between them; for norm; Indicates a point and The curvature difference, Indicates a point density.

5. The arch dam point cloud registration method based on deep learning and multi-objective feature input according to claim 4 is characterized in that: Assume two 3D point sets and , and these two point sets can be transformed by an affine transformation {R, t}, where is the original point cloud, For the target point cloud, the rigid change estimation layer uses a matching matrix Record the matching of two point sets, where If you click Corresponding to point ,but ,otherwise , in order to Better mapping to Above, the following conditions should be met: ； In the above formula, I Point set the number of midpoints; J Point set The number of midpoints; R is the linear transformation matrix; t is the translation matrix; is an annealing parameter, the purpose of which is to make the distance greater than The points are considered as outliers; Since the matching matrix M Subject to the constraints of the matrix rows and columns and To simplify the problem, the matching matrix Perform relaxation changes to obtain , , is an arbitrary symbol; then the deterministic annealing algorithm is used to solve the minimum value of formula (12), where Initialized as: ； In the above formula, is the annealing parameter, The smaller the initial value of , the higher the overall randomness, and the less likely it is to fall into a local minimum. It will gradually increase during the iterative calculation process. hour, will approach {0,1}; where the annealing parameter and Prediction is obtained through a secondary network; Substituting the learned feature distance for the spatial distance in formula (13), we get: ； In the above formula, and are the features of the point cloud; After completing feature extraction and and After the prediction of the annealing parameters, the Sinkhorn normalization process is performed, and the rows and columns are normalized alternately to obtain a double random matrix from all positive square matrices, and then the optimal distribution is obtained; then the rigid transformation is estimated using SVD, and for each original point cloud , calculate the target point cloud Corresponding coordinates : ； In the above formula, Point Cloud Corresponding point cloud coordinates of is the matching matrix Elements in Point Cloud The point in The loss function is the actual transformation and prediction transform Considering the different overlaps of the registration data in the actual registration process, the Huber loss function is used, but there are many outliers and The influence of the norm will lead to fewer intrinsic values, so the increase in quadratic loss L is expressed as: ； In the above formula, is the actual rotation matrix; Point set The point in is the actual translation vector; is the predicted rotation matrix; is the predicted translation vector; It is a hyperparameter and needs to be set manually.

Images