Aircraft assembly point cloud registration method and device based on LiDAR scanning data

Abstract

The invention discloses an aircraft assembly point cloud registration method and device based on LiDAR scanning data, and belongs to the field of three-dimensional point cloud processing. The method comprises the following steps: acquiring a source point cloud and a target point cloud of an airplane model; performing feature extraction and downsampling on the source point cloud and the target point cloud to obtain downsampling points, and jointly constructing a local patch by each super point and nearby K sampling points; calculating a prediction rigid body transformation matrix of the source point cloud and the target point cloud; calculating density consistency loss, attitude consistency loss, geometric consistency loss and point matching loss of the global prediction rigid body transformation matrix and the real rigid body transformation matrix, and comprehensive loss of the global prediction rigid body transformation matrix and the real rigid body transformation matrix, and optimizing the global prediction rigid body transformation matrix for point cloud registration by taking the minimum comprehensive loss as a target to obtain a target rigid body transformation matrix; and registering the source point cloud and the target point cloud according to the target rigid body transformation matrix. And high-efficiency and high-precision aircraft point cloud registration is realized.

Description

Technical Field

[0001] The present invention belongs to the field of three-dimensional point cloud data processing, and more specifically, relates to an aircraft assembly point cloud registration method and device based on LiDAR scanning data. Background Art

[0002] In the aircraft manufacturing process, accurate component alignment is the key to ensure assembly quality. Traditional manual measurement and vision-based alignment technologies often face problems such as insufficient measurement accuracy, long processing time, and difficulty in covering complex structures when dealing with large aircraft components.

[0003] In order to improve the accuracy and efficiency of the assembly process, LiDAR (laser radar) technology, as a non-contact 3D measurement method, can quickly collect point cloud data of complex objects. However, how to efficiently align the large amount of point cloud data generated by multi-view LiDAR scanning to ensure accurate alignment between the source point cloud and the target point cloud remains a challenging problem.

[0004] Existing point cloud registration technologies mainly rely on methods such as ICP (Iterative Closest Point Algorithm) based on geometric features, but these methods are often inefficient and have limited registration accuracy when facing large-scale, complex geometric structures and multi-view data. Therefore, the development of an efficient and accurate point cloud registration method, especially for the registration method of LiDAR scanning data of large aircraft components, is an urgent need in the aircraft manufacturing field. Summary of the invention

[0005] In view of the defects of the related art, the purpose of the present invention is to provide an aircraft assembly point cloud registration method and device based on LiDAR scanning data, aiming to solve the problems of low efficiency and limited registration accuracy when the existing point cloud registration technology is applied to aircraft assembly.

[0006] To achieve the above objectives, in a first aspect, the present invention provides an aircraft assembly point cloud registration method based on LiDAR scanning data, comprising:

[0007] S1, obtaining the source point cloud and target point cloud of the aircraft model through a multi-view LiDAR scanning device;

[0008] S2, extract features from the source point cloud and the target point cloud respectively;

[0009] S3, downsampling the acquired features to obtain downsampling points, selecting super points from the downsampling points, and then constructing each super point and nearby K sampling points into a local patch based on geometric proximity; wherein K is greater than or equal to 1;

[0010] S4, calculating the predicted rigid body transformation matrix from each local patch in the source point cloud to the corresponding local patch in the target point cloud, and selecting the predicted rigid body transformation matrix with the most inliers as the global predicted rigid body transformation matrix;

[0011] S5, calculating the density consistency loss, posture consistency loss, geometric consistency loss and point matching loss of the global prediction rigid body transformation matrix and the real rigid body transformation matrix, and the comprehensive loss composed of them, and optimizing the global prediction rigid body transformation matrix with the minimum comprehensive loss as the goal, to obtain the target rigid body transformation matrix;

[0012] S6. Align the source point cloud with the target point cloud according to the target rigid body transformation matrix.

[0013] Optionally, the multi-view LiDAR scanning device scans the aircraft model from a preset height, a preset angle, and a preset distance.

[0014] Optionally, before S2, also include:

[0015] Injecting random noise into the source point cloud and the target point cloud;

[0016] Density perturbation processing is performed on the source point cloud and the target point cloud to adjust the sampling density.

[0017] Optionally, the extracting features from the source point cloud and the target point cloud respectively includes:

[0018] The KPConv-FPN network is used to extract geometric features from the source point cloud and the target point cloud.

[0019] Optionally, after S3 and before S4, the following is also included:

[0020] T1. Constructing a matching score matrix according to the corresponding super-point features in the source point cloud and the target point cloud, and obtaining a normalized matching matrix through bidirectional normalization;

[0021] The matching score matrix is:

[0022]

[0023] Among them, f i P represents the i-th feature element in the source point cloud superpoint feature vector, f j Q Represents the jth feature element in the target point cloud super-point feature vector;

[0024] The normalized matching score matrix is:

[0025]

[0026] in, and denote the number of superpoints in the source point cloud and the target power source, respectively;

[0027] T2. Select the top N matching scores from the matching matrix in descending order. c Items, as matching pairs of corresponding superpoints in the source and target point clouds:

[0028]

[0029] in, is the super point in the source point cloud, is the super point in the target point cloud, Indicates the xth point in the source point cloud i A super point, Indicates the yth point in the target point cloud i super points, 1≤i≤N c ;

[0030] T3. Calculate the matching pairs of corresponding superpoints in the source point cloud and the target point cloud The local matching matrix of :

[0031]

[0032] in, Indicates the xth point in the source point cloud i The feature matrix of a superpoint, Indicates the yth point in the target point cloud i The feature matrix of a superpoint, Represents feature dimension;

[0033] T4, calculate the soft allocation matrix Z of the local matching matrix i , and select the corresponding super-point matching pairs in the final source point cloud and target point cloud through the mutual top-K algorithm:

[0034]

[0035] in, Indicates the xth point in the source point cloud i A local patch constructed from superpoints, Indicates the yth point in the target point cloud i The local patch is constructed by super points, mutual topk means the mutual selection top-K algorithm, Represents the soft assignment matrix Z i The elements in corresponding to the x and y coordinate directions;

[0036] Furthermore, the predicted rigid body transformation matrix in S4 is calculated using the corresponding local patches of superpoint matching pairs in the source point cloud and the target point cloud.

[0037] Optionally, the calculating the density consistency loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix includes:

[0038] Calculate the feature variance of the super points of the source point cloud and the target point cloud, and calculate the average feature variance of the super points of the source point cloud and the target point cloud respectively;

[0039] The density consistency loss is obtained by summing the average feature variance of the superpoints of the source point cloud and the average feature variance of the superpoints of the target point cloud.

[0040] Optionally, the predicted rigid body transformation matrix includes a predicted rotation matrix and a predicted displacement vector; and the step of calculating the posture consistency loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix includes:

[0041] Calculate the true rotation matrix and true translation vector according to the changes in height, angle, and distance during the process of scanning the source point cloud and the target point cloud by the multi-view LiDAR scanning device;

[0042] Calculate the error between the predicted rotation matrix, predicted translation vector of the source point cloud and the target point cloud and the true rotation matrix and true translation vector, which is recorded as the posture consistency loss.

[0043] Optionally, calculating the geometric consistency loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix includes:

[0044] The geometric distance between the source point cloud after the real transformation and the predicted transformation is calculated and recorded as the geometric consistency loss.

[0045] Optionally, calculating the point matching loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix includes:

[0046] In the distribution matrix Z i The negative log-likelihood loss is used as the point matching loss corresponding to a single point:

[0047]

[0048] The overall point matching loss during point cloud transformation is:

[0049]

[0050] Among them, N g represents the number of randomly sampled true superpoint matching pairs, represents the true matching pair of a set of downsampled points extracted, I i and J i Represents a set of mismatched downsampling points.

[0051] In a second aspect, the present invention further provides an aircraft assembly point cloud registration device based on LiDAR scanning data, comprising:

[0052] A scanning module, used to obtain source point cloud and target point cloud of the aircraft model through a multi-view LiDAR scanning device;

[0053] A feature extraction module is used to extract features from the source point cloud and the target point cloud respectively;

[0054] A sampling module, used to downsample the acquired features to obtain downsampled points, select superpoints from the downsampled points, and then construct each superpoint and nearby K sampling points as a local patch based on geometric proximity; wherein K is greater than or equal to 1;

[0055] A registration module is used to calculate the predicted rigid body transformation matrix from each local patch in the source point cloud to the corresponding local patch in the target point cloud, and select the predicted rigid body transformation matrix with the most internal point matches as the global predicted rigid body transformation matrix; a loss calculation module is used to calculate the density consistency loss, posture consistency loss, geometric consistency loss and point matching loss of the global predicted rigid body transformation matrix and the real rigid body transformation matrix, and the comprehensive loss composed of them, and optimize the global predicted rigid body transformation matrix with the goal of minimizing the comprehensive loss to obtain the target rigid body transformation matrix;

[0056] The result output module is used to align the source point cloud with the target point cloud according to the target rigid body transformation matrix.

[0057] Compared with the prior art, the above technical solution conceived by the present invention can achieve the following beneficial effects:

[0058] 1. The present invention provides an aircraft assembly point cloud registration method based on LiDAR scanning data. The aircraft model data obtained by LiDAR scanning has the problems of non-uniform distribution, large posture and non-unique features. This scheme performs multi-level feature extraction and aggregates the original point cloud into a point cloud containing super points, which not only retains the information of the original data, but also reduces the amount of data, thereby improving the efficiency of subsequent calculations; constructs a local patch containing super points and nearby sampling points, and obtains a more accurate predicted rigid body transformation matrix based on the local patch calculation; adopts the design of multiple loss functions, so that the model can adapt to the above-mentioned complex situations from different angles, and by comprehensively considering multiple loss functions, the model can learn more comprehensive features and laws; it helps to overcome the matching difficulties caused by large posture changes and non-unique features in aircraft scanning. Thereby solving the problems of low efficiency and limited registration accuracy existing in the prior art, and realizing high-efficiency and high-precision aircraft point cloud registration.

[0059] 2. The present invention provides an aircraft assembly point cloud registration method based on LiDAR scanning data. Each superpoint constructs a local block based on the shortest distance in the geometric space. The local block is formed by selecting the down-sampling point closest to the superpoint and calculating the relevant feature matrix. This construction method can quickly and accurately obtain local features and improve the efficiency of feature extraction. Based on the superpoint matching pairs and matching score matrices, the point correspondence is further extracted. From the superpoint to the local area and then to the specific point, the correspondence between the points in different views can be determined more accurately, thereby improving the accuracy of the entire point cloud registration.

[0060] 3. The present invention provides an aircraft assembly point cloud registration method based on LiDAR scanning data. When calculating losses, multiple losses are calculated separately: posture consistency loss handles posture changes, geometric consistency loss handles geometric shape consistency, density consistency loss handles data density unevenness, and point matching loss handles point correspondence. Comprehensively considering multiple loss functions, the model not only performs well on training data, but also can achieve higher registration accuracy when facing new aircraft scanning data or scanning data of different aircraft components.

[0061] 4. The present invention provides an aircraft assembly point cloud registration method based on LiDAR scanning data. Noise perturbation increases the adaptability of the data to different noises, improves the robustness of the model to noise, and enables it to better distinguish between real point cloud features and noise; the density destruction mode simulates actual density changes, enhances the model's ability to process point clouds of different densities, and avoids registration errors or inaccurate feature extraction due to density changes. The use of the above data enhancement strategy significantly improves the diversity of the data set, enhances the adaptability and robustness of the model to noise and density changes in LiDAR data, and improves the generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a flow chart of a method for aircraft assembly point cloud registration based on LiDAR scanning data provided by the present invention;

[0063] Figure 2 is a schematic diagram of a multi-view LiDAR scanning device provided by the present invention;

[0064] Figure 3 is a schematic diagram of an aircraft model to be scanned provided by the present invention;

[0065] Figure 4 is a schematic diagram of multi-view aircraft scanning registration provided by the present invention;

[0066] Figure 5 It is a visualization schematic diagram of the registration result of an aircraft assembly point cloud registration method based on LiDAR scanning data provided by the present invention;

[0067] Figure 6 It is a schematic diagram of the effects of different data enhancement strategies adopted by the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0069] The contents involved in the above embodiment are described below in conjunction with a preferred embodiment.

[0070] Embodiment 1

[0071] like Figure 1 As shown, the present invention provides an aircraft assembly point cloud registration method based on LiDAR scanning data, comprising:

[0072] S1, obtaining the source point cloud and target point cloud of the aircraft model through a multi-view LiDAR scanning device;

[0073] S2, extract features from the source point cloud and the target point cloud respectively;

[0074] S3, downsampling the acquired features to obtain downsampling points, selecting super points from the downsampling points, and then constructing each super point and nearby K sampling points into a local patch based on geometric proximity; wherein K is greater than or equal to 1;

[0075] S4, calculating the predicted rigid body transformation matrix from each local patch in the source point cloud to the corresponding local patch in the target point cloud, and selecting the predicted rigid body transformation matrix with the most inliers as the global predicted rigid body transformation matrix;

[0076] S5, calculating the density consistency loss, posture consistency loss, geometric consistency loss and point matching loss of the global prediction rigid body transformation matrix and the real rigid body transformation matrix, and the comprehensive loss composed of them, and optimizing the global prediction rigid body transformation matrix with the minimum comprehensive loss as the goal, to obtain the target rigid body transformation matrix;

[0077] S6. Align the source point cloud with the target point cloud according to the target rigid body transformation matrix.

[0078] During the aircraft assembly process, scans from different perspectives may have large angle differences and unclear similar features. Accurate super-point matching pairs and matching score matrices can improve the accuracy of point cloud registration and enable better alignment of scans from different perspectives.

[0079] like Figure 2 As shown, the multi-view LiDAR scanning device provided in this embodiment includes a medium-range mechanical laser radar, a height-adjustable tripod, a three-axis rotating gimbal, and an electric slide rail.

[0080] like Figure 3 As shown, the aircraft model provided in this embodiment includes three parts, the total length of the aircraft model is 3 meters, the wingspan is 1.85 meters, and the height is 0.65 meters. For multi-view data acquisition, the medium-range mechanical laser radar (Hesai Pandar XT32, the effective working range can reach 120 meters) used in this embodiment.

[0081] like Figure 4 As shown, the multi-view LiDAR scanning device scans the aircraft model from a preset height, a preset angle, and a preset distance. The laser radar is set at different positions to scan from multiple views (different X, Y, Z positions) to obtain complete three-dimensional point cloud data of the aircraft model. Specifically, the source point cloud and the target point cloud are collected at multiple angles by the multi-view LiDAR scanning device, specifically including scanning the aircraft model from different heights, angles, and distances to ensure complete coverage of the geometric structure of the aircraft model, especially the key areas that require high-precision alignment during the assembly process, such as complex curved surfaces, connection parts, and hole areas.

[0082] Each scan generates a set of 3D point cloud data. The source point cloud P and the target point cloud Q represent the point cloud data sets of the same part of the aircraft model from different perspectives. Each LiDAR device generates point cloud data containing 3D coordinate information. The point cloud data set is represented by P = {p i ∈R 3 |i=1,…,N} and Q={q i ∈R 3 |i=1,…,M}, where N and M are the number of points in the source point cloud and the target point cloud, respectively.

[0083] Multi-level feature extraction is performed on the source point cloud and the target point cloud. The feature extraction includes global geometric feature extraction, which is used to capture the overall geometric structure of the aircraft model, and local detail feature extraction, which is used to capture the local subtle structure of the aircraft model.

[0084] In the feature extraction process of the source point cloud and the target point cloud, superpoints are generated by step-by-step downsampling; local patches are constructed for each superpoint based on geometric distance, and the local patches contain K downsampled points closest to the superpoint. Superpoints are selected from the downsampled point cloud, and the geometric distance between each downsampled point and its nearest superpoint is calculated; for each downsampled point, its nearest superpoint is found, and superpoints with no corresponding downsampled points will be removed; for each superpoint, the K downsampled points closest to it are selected to form a local patch, which together reflect the geometric and feature information of the local area.

[0085] The predicted rigid body transformation matrix is ​​calculated through local patches in the source point cloud and the target point cloud to obtain multiple predicted rigid body transformation matrices. The point cloud data in the source point cloud is respectively brought into each predicted rigid body transformation matrix for calculation, and then verified with the point cloud data in the target point cloud. The predicted rigid body transformation matrix with the most internal point matches is selected as the global predicted rigid body transformation matrix. Then, according to the changes in height, angle and distance during the scanning of the source point cloud and the target point cloud by the multi-view LiDAR scanning device, the real rotation matrix and the real translation vector are calculated to obtain the real rigid body transformation matrix. Then, by calculating the density consistency loss, posture consistency loss, geometric consistency loss and point matching loss of the two transformation matrices, the global predicted rigid body transformation matrix is ​​optimized to obtain the target rigid body transformation matrix between the source point cloud and the target point cloud, and the alignment result is output.

[0086] Optionally, before S2, also include:

[0087] Injecting random noise into the source point cloud and the target point cloud;

[0088] Density perturbation processing is performed on the source point cloud and the target point cloud to adjust the sampling density.

[0089] This scheme adopts a specific data enhancement strategy to preprocess the acquired source point cloud and target point cloud; injects random noise into the source point cloud and target point cloud to simulate the measurement noise that may exist in the actual working environment; performs density perturbation processing on the point cloud data and adjusts the sampling density of the point cloud to enhance the system's ability to cope with data of different resolutions.

[0090] Two types of noise disturbances (uniform distribution and Gaussian distribution) and three density destruction modes (i.e., local density increase, local density reduction, and elimination) are used to enhance the data. Uniform distribution and Gaussian distribution introduce different noise distributions to each point in the point cloud; local density increase and local density reduction use k-nearest neighbors to randomly select local point clusters and increase or decrease the density of the clusters, respectively. This cropping method enhances data diversity by randomly deleting the selected local point clusters. Through the above data enhancement operations, the robustness and adaptability of this scheme under different working conditions are improved, ensuring that high-precision registration results can still be output in real environments.

[0091] Optionally, the extracting features from the source point cloud and the target point cloud respectively includes:

[0092] The KPConv-FPN network is used to extract geometric features from the source point cloud and the target point cloud.

[0093] The KPConv-FPN network is used to extract geometric features of the source point cloud and the target point cloud layer by layer to capture the global and local structural information of the point cloud. During the extraction process, the data of the source point cloud and the target point cloud are gradually downsampled to generate super points, which represent local geometric features.

[0094] The KPConv-FPN network is used for multi-level feature extraction to aggregate the original point cloud into a point cloud containing super points, which not only retains the information of the original data but also reduces the amount of data, thus improving the efficiency of subsequent calculations.

[0095] Step S2 specifically includes the following sub-steps:

[0096] S21, the input source point cloud is P = {p i ∈R 3 |i=1,…,N}, the target point cloud is Q={q i ∈R 3 |i=1,…,M};

[0097] S22. Use the KPConv-FPN network to perform multi-level feature extraction on the source point cloud and the target point cloud. The network is divided into a multi-layer convolution structure. Each layer gradually extracts the geometric features of the point cloud. Through the residual block (ResNet) and stride convolution, the point cloud generates a high-dimensional feature representation after layer-by-layer convolution.

[0098] S23. In the multi-layer convolution process, the geometric information of the source point cloud P and the target point cloud Q is extracted to obtain the downsampled point cloud and in, and Respectively represent the number of downsampling points, is the characteristic dimension.

[0099] S24. Select super points from the downsampled points to generate a super point cloud and in, and Respectively represent the number of super points; each super point has a feature representation and This process extracts the global geometric information of the point cloud while preserving the local detail features.

[0100] Optionally, after step S3 and before step S4, the following steps are further included:

[0101] T1. Constructing a matching score matrix according to the corresponding super-point features in the source point cloud and the target point cloud, and obtaining a normalized matching matrix through bidirectional normalization;

[0102] The feature matrix of the local patch constructed from the source point cloud is expressed as The feature matrix of the local patch constructed from the target point cloud is expressed as

[0103] m i,j It is a super point feature and The matching score matrix between is expressed as follows:

[0104]

[0105] Among them, f i P represents the i-th feature element in the source point cloud superpoint feature vector, f j Q Represents the jth feature element in the target point cloud super-point feature vector;

[0106] In order to eliminate false matches, bidirectional normalization is performed; the normalized matching score matrix is:

[0107]

[0108] in, and denote the number of superpoints in the source point cloud and the target power source, respectively.

[0109] T2. Select the top N matching scores from the matching matrix in descending order. c Items, as matching pairs of corresponding superpoints in the source and target point clouds:

[0110]

[0111] in, is the super point in the source point cloud, is the super point in the target point cloud, Indicates the xth point in the source point cloud i A super point, Indicates the yth point in the target point cloud i super points, 1≤i≤N c .

[0112] T3. Calculate the matching pairs of corresponding superpoints in the source point cloud and the target point cloud The local matching matrix of :

[0113]

[0114] in, Indicates the xth point in the source point cloud i The feature matrix of a superpoint, Indicates the yth point in the target point cloud i The feature matrix of a superpoint, Represents the feature dimension.

[0115] T4, calculate the soft allocation matrix Z of the local matching matrix i , and select the corresponding super-point matching pairs in the final source point cloud and target point cloud through the mutual top-K algorithm:

[0116]

[0117] in, Indicates the xth point in the source point cloud i A local patch constructed from superpoints, Indicates the yth point in the target point cloud i The local patch is constructed by super points, mutual topk means the mutual selection top-K algorithm, Represents the soft assignment matrix Z i The elements in correspond to the x and y coordinate directions.

[0118] For each selected superpoint matching pair, its corresponding local patch in the source point cloud and the target point cloud is used to calculate the predicted rigid body transformation moment. Specifically, the corresponding local rigid body transformation R i ,t i Computed via weighted singular value decomposition (SVD).

[0119] The weights of the downsampling point matching are based on the soft assignment matrix Z i The final transformation is determined by the transformation with the most inliers among all local rigid body transformations, as shown in the following formula:

[0120]

[0121] Where τ is the acceptance radius and C contains all matching pairs of local points:

[0122] The above local patch construction method effectively extracts geometric information, and uses the matching pairs obtained through local patch construction to predict the calculation of rigid body transformation, combining feature matching and rigid body transformation for more accurate point cloud registration.

[0123] The point cloud registration is optimized by calculating the density consistency loss, pose consistency loss, geometric consistency loss and point matching loss, and finally the rigid transformation between the source point cloud and the target point cloud is determined.

[0124] Optionally, the calculating the density consistency loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix includes:

[0125] Calculate the feature variance of the super points of the source point cloud and the target point cloud, and calculate the average feature variance of the super points of the source point cloud and the target point cloud respectively;

[0126] The density consistency loss is obtained by summing the average feature variance of the superpoints of the source point cloud and the average feature variance of the superpoints of the target point cloud.

[0127] The density consistency loss is used to ensure the density consistency of the source point cloud and the target point cloud in the local area. The density consistency loss function is:

[0128]

[0129]

[0130] in, and are the number of super points of the source point cloud and the target point cloud, respectively, K is the number of downsampled points contained in the local patch, and f ij is the feature vector of the jth point in the local patch of the i-th superpoint.

[0131] Optionally, the predicted rigid body transformation matrix includes a predicted rotation matrix and a predicted displacement vector; and the step of calculating the posture consistency loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix includes:

[0132] Calculate the real rotation matrix and real translation vector generated by the multi-view LiDAR scanning device during scanning;

[0133] Calculate the error between the predicted rotation matrix, predicted translation vector of the source point cloud and the target point cloud and the true rotation matrix and true translation vector, which is recorded as the posture consistency loss.

[0134] The pose consistency loss is used to compare the global pose difference between the source point cloud and the target point cloud. The pose consistency loss function is:

[0135]

[0136] Among them, R est and t est To predict the rotation matrix and translation vector, R gt and t gt are the true rotation matrix and the true translation vector.

[0137] The goal of the pose consistency loss is to minimize the difference between the predicted rigid body transformation and the true rigid body transformation. By optimizing the pose consistency loss, the pose difference is reduced to ensure that the global poses of the source point cloud and the target point cloud are consistent.

[0138] Optionally, the calculating the geometric consistency loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix includes:

[0139] The geometric distance between the source point cloud after the real transformation and the predicted transformation is calculated and recorded as the geometric consistency loss.

[0140] The geometric consistency loss is used to evaluate the difference in geometric structure between the source point cloud and the target point cloud. The geometric consistency loss function is:

[0141]

[0142] Among them, p i Represents a point in the source point cloud.

[0143] According to the changes in height, angle and distance during the scanning process of the source point cloud and the target point cloud by the multi-view LiDAR scanning device, the true rotation matrix and the true translation vector are calculated. The geometric error between the two sets of point clouds is evaluated, especially the difference in geometric shape of the local area; the geometric error is reduced by optimizing the geometric consistency loss, so as to ensure the geometric features of the two sets of point clouds match during the registration process.

[0144] Optionally, calculating the point matching loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix includes:

[0145] In the distribution matrix Z i The negative log-likelihood loss is used as the point matching loss corresponding to a single point:

[0146]

[0147] The overall point matching loss during point cloud transformation is:

[0148]

[0149] Among them, N g represents the number of randomly sampled true superpoint matching pairs, represents the true matching pair of a set of downsampled points extracted, I i and J i Represents a set of mismatched downsampling points.

[0150] The point matching loss is used to optimize the registration matching of each superpoint, ensuring the accuracy of point cloud registration by maximizing the number of matching pairs in the local rigid body transformation.

[0151] Furthermore, the comprehensive loss function combines pose consistency, geometric consistency, density consistency and point matching loss. By adjusting the weights of different loss items, the comprehensive loss can optimize the global registration effect.

[0152] The comprehensive loss function is:

[0153]

[0154] Among them, λ p , g , d and λ pm It is the weight of the corresponding loss term, which can adjust the registration accuracy according to the application scenario.

[0155] Furthermore, the aircraft assembly point cloud registration method based on LiDAR scanning data provided by the present invention is experimentally demonstrated to verify that it can achieve the effect of realizing high-precision three-dimensional data registration of large aircraft components during the assembly process.

[0156] To provide a comprehensive evaluation, experiments are conducted under two different settings: Experimental Group 1 and Experimental Group 2.

[0157] For Experimental Group 1, 70% of the scan data from the nose, fuselage, and tail are used for training, and the remaining 30% are used for testing. In Experimental Group 1, no data augmentation is applied to the dataset.

[0158] For Experimental Set 2, data from the nose and tail were used for training, and data from the fuselage were used for testing. Considering the significant structural differences between these parts and the limited data, data augmentation strategies were adopted during training to reduce the risk of overfitting.

[0159] To measure the average registration error, the error between the estimated and true transformations is calculated. After that, the mean isotropic error (MIE) of the rotation matrix and translation vector is calculated, denoted as MIE(R), MIE(t). In addition, the root mean square error (RMSE) of the source point cloud transformed using the estimated and true transformations is calculated.

[0160] Specifically, the entire system was implemented using PyTorch on a server using an NVIDIA RTX 3090 GPU. During training, the λ in the loss function p , g , d and λ pm They are set to 1, 1, 0.5 and 0.01 respectively. The Adam optimizer is used for training, with an initial learning rate of 1e-3 for experimental group 1 and 1e-4 for experimental group 2.

[0161] Our method is compared with other registration methods, including ICP, FGR, RANSAC, TEASER++, MAC, RIENet, and GeoTransformer. RIENet is an unsupervised learning-based method, while GeoTransformer is supervised. All learning-based networks are retrained on the same dataset. For ICP, RANSAC, and FGR, our method uses the Open3D version of the implementation, while the other methods use publicly released code.

[0162] Our method is first evaluated on LAD with partial overlap of experimental set 1, where the training and test datasets contain different scans from three parts. As shown in Table 1, our DPCNet outperforms traditional and learning-based methods in all metrics, and our results show significant improvements compared to the traditional method MAC. Since traditional methods rely on existing feature descriptors for feature extraction and registration, the lack of obvious features in aircraft scans limits their performance. Compared with learning-based methods, our method improves GeoTransformer by 14.7%, 30.6% and 16.2% in RMSE, MIE(R) and MIE(t). Its strong performance proves the effectiveness of our framework in recording LiDAR aircraft scans.

[0163] Further, experiments are conducted on a partially overlapping experimental group 2 to compare the generalization of different methods. Data from the nose and tail are used for training, and data from the fuselage is used for testing. The comparison results are shown in Table II. Our DPCNet consistently achieves the best performance in all indicators. Our scheme improves MIE(R) by 44.0% and MIE(t) by 21.8%, respectively. The results show the generalization of our framework when applied to different aircraft components. In addition, comparing the last row in Table I with the last row in Table II, our scheme improves MIE(t) on experimental group 1 by 11.2%. Although LAD in the experimental group 2 setting poses a greater challenge for testing, the implementation of the data augmentation strategy effectively improves the performance of our model.

[0164] Table 1 Comparison results of LAD dataset and experimental group 1

[0165] Method RMSE MIE(R) MIE(t) ICP 0.462 1.733 1.083 FGR 0.428 1.687 1.030 RANSAC 0.430 1.642 0.872 TEASER++ 0.578 1.701 0.915 MAC 0.563 1.838 1.023 RIENet 0.509 1.779 1.239 GeoTransformer 0.346 1.378 0.606 Ours 0.295 0.956 0.508

[0166] Table 2 Comparison results of LAD dataset and experimental group 2

[0167]

[0168]

[0169] right Figure 5 The scan data in the paper are qualitatively compared. Compared with the traditional methods ICP and RANSAC and the learning-based GeoTransformer method, the method of this scheme achieves the best alignment results. Although traditional methods such as ICP and RANSAC can be directly applied to real-world scenarios, their registration performance cannot meet the accuracy requirements of aircraft assembly. Profit-based methods, including GeoTransformer and DPCNet of this scheme, show more satisfactory alignment performance. Figure 2 As shown in the first two rows of , the lidar scans show uneven distribution and large pose variations. Our approach can achieve visualizations close to the ground truth, which also proves the effectiveness of our framework in recording aircraft scans.

[0170] exist Figure 6 The effectiveness of different data augmentation strategies is evaluated in . The experiments in this scheme are conducted under four conditions: without any data augmentation, only noise augmentation, only density augmentation, and applying both augmentations at the same time.

[0171] The lowest accuracy is achieved without data augmentation. This suggests that adding data augmentation during training can improve the registration accuracy of the method. The registration accuracy of only enhancing noise is better than that of only enhancing density. The highest registration accuracy is achieved when both types of augmentation are applied. These results suggest that combining different data augmentation strategies increases the diversity of training data and reduces overfitting of the model, thus helping the model achieve better registration accuracy.

[0172] We further evaluated the effects of various loss functions on LAD under experimental group 1. As shown in Table III, p , g and λ pm represents the pose consistency loss, geometric consistency loss and point matching loss. d represents the density consistency loss achieved on the local patch. p and λ g It performs well as a supervision loss to minimize the conversion error. d is an unsupervised loss that enables the model to extract density-invariant features. The combination of all losses results in the best registration result.

[0173] Table 3 Comparison results of different loss combinations

[0174] <![CDATA[λ p ]]> <![CDATA[λ p ]]> <![CDATA[λ d ]]> <![CDATA[λ pm ]]> RMSE MIE(R) MIE(t) √ 0.706 1.508 1.231 √ 0.843 1.639 1.452 √ 0.802 1.472 1.345 √ 0.927 1.234 1.378 √ √ √ √ 0.574 0.849 0.880

[0175] In summary, by conducting experiments on the same aircraft scanning data set, the method of the present invention is compared with traditional registration methods (such as iterative nearest point algorithm, etc.) and other learning-based registration methods. The root mean square error, overlap rate and other indicators of various registration methods in different test scenarios are quantitatively calculated. The method of the present invention can achieve more accurate registration results on aircraft scanning data with different viewing angles and different degrees of posture changes, highlighting its advantages in complex actual scenes. The method of the present invention respectively shows the registration accuracy results in four cases: no data enhancement, only noise enhancement, only density enhancement, and simultaneous application of two enhancements. Analyze how different enhancement strategies affect the performance of the model. Through a reasonable data enhancement strategy, the generalization ability of the model is improved, so that it can still maintain a high accuracy when facing actual data of different quality and characteristics. The method of the present invention analyzes the balance and synergy between different losses, and by adjusting the loss weight parameters, finds the best combination to achieve more accurate registration results.

[0176] The method of this scheme is applied to aircraft assembly quality inspection. The source point cloud and target point cloud of the aircraft to be inspected are obtained through a multi-view LiDAR scanning device. After the method described above, the registration result of the source point cloud and the target point cloud is output. Figure 5 As shown, the degree of overlap between the transformed source point cloud of the aircraft and the target point cloud of the aircraft in the registration result diagram indicates the assembly quality of the aircraft; the registration result diagram is judged, if the degree of overlap between the transformed source point cloud of the aircraft and the target point cloud of the aircraft reaches 95%, then the assembly quality of the aircraft is judged to be up to standard; if it is less than 95%, then the assembly quality of the aircraft is judged to be not up to standard and the aircraft needs to be reassembled.

[0177] This method can be applied to aircraft assembly quality inspection, accurately register point clouds for aircraft, obtain complete aircraft surface information, and thus detect the deformation of aircraft components to ensure aircraft assembly quality. It can also estimate and adjust the posture of the aircraft during assembly to improve assembly accuracy and efficiency. Furthermore, it can also be applied to the expansion of multi-view registration, for example, for quantitative measurement of aircraft skin gaps, posture estimation, and virtual assembly. It has the potential to be extended to large components in other industrial applications, providing them with accurate and efficient measurement and registration methods.

[0178] Embodiment 2

[0179] The present invention also provides an aircraft assembly point cloud registration device based on LiDAR scanning data, comprising:

[0180] A scanning module, used to obtain source point cloud and target point cloud of the aircraft model through a multi-view LiDAR scanning device;

[0181] A feature extraction module is used to extract features from the source point cloud and the target point cloud respectively;

[0182] A sampling module, used to downsample the acquired features to obtain downsampled points, select superpoints from the downsampled points, and then construct each superpoint and nearby K sampling points as a local patch based on geometric proximity; wherein K is greater than or equal to 1;

[0183] A registration module is used to calculate the predicted rigid body transformation matrix from each local patch in the source point cloud to the corresponding local patch in the target point cloud, and select the predicted rigid body transformation matrix with the most internal point matches as the global predicted rigid body transformation matrix; a loss calculation module is used to calculate the density consistency loss, posture consistency loss, geometric consistency loss and point matching loss of the global predicted rigid body transformation matrix and the real rigid body transformation matrix, and the comprehensive loss composed of them, and optimize the global predicted rigid body transformation matrix with the goal of minimizing the comprehensive loss to obtain the target rigid body transformation matrix;

[0184] The result output module is used to align the source point cloud with the target point cloud according to the target rigid body transformation matrix.

[0185] An aircraft assembly point cloud registration device based on LiDAR scanning data provided by an embodiment of the present invention can execute an aircraft assembly point cloud registration method based on LiDAR scanning data provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0186] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for aircraft assembly point cloud registration based on LiDAR scanning data, characterized in that: include: S1, obtaining the source point cloud and target point cloud of the aircraft model through a multi-view LiDAR scanning device; S2, extract features from the source point cloud and the target point cloud respectively; S3, downsampling the acquired features to obtain downsampling points, selecting super points from the downsampling points, and then constructing each super point and K nearby sampling points into a local patch based on geometric proximity; wherein K is greater than or equal to 1; S4, calculating the predicted rigid body transformation matrix from each local patch in the source point cloud to the corresponding local patch in the target point cloud, and selecting the predicted rigid body transformation matrix with the most inliers as the global predicted rigid body transformation matrix; S5, calculating the density consistency loss, posture consistency loss, geometric consistency loss and point matching loss of the global prediction rigid body transformation matrix and the real rigid body transformation matrix, and the comprehensive loss composed of them, and optimizing the global prediction rigid body transformation matrix with the goal of minimizing the comprehensive loss to obtain the target rigid body transformation matrix; S6. Align the source point cloud with the target point cloud according to the target rigid body transformation matrix.

2. The method according to claim 1, characterized in that The multi-view LiDAR scanning device scans the aircraft model from a preset height, a preset angle, and a preset distance.

3. The method according to claim 1, characterized in that Prior to S2, this also included: Injecting random noise into the source point cloud and the target point cloud; Density perturbation processing is performed on the source point cloud and the target point cloud to adjust the sampling density.

4. The method according to claim 1, characterized in that The feature extraction of the source point cloud and the target point cloud respectively comprises: The KPConv-FPN network is used to extract geometric features from the source point cloud and the target point cloud.

5. The method according to claim 1, characterized in that After S3 and before S4, it also includes: T1. Constructing a matching score matrix according to the corresponding super-point features in the source point cloud and the target point cloud, and obtaining a normalized matching matrix through bidirectional normalization; The matching score matrix is: Among them, f i P represents the i-th feature element in the source point cloud superpoint feature vector, f i Q Represents the jth feature element in the target point cloud super-point feature vector; The normalized matching score matrix is: in, and denote the number of superpoints in the source point cloud and the target power source, respectively; T2. Select the top N matching scores from the matching matrix in descending order. c Items, as matching pairs of corresponding superpoints in the source and target point clouds: in, is the super point in the source point cloud, is the super point in the target point cloud, Indicates the xth point in the source point cloud i A super point, Indicates the yth point in the target point cloud i super points, 1≤i≤N c ; T3. Calculate the matching pairs of corresponding superpoints in the source point cloud and the target point cloud The local matching matrix of : in, Indicates the xth point in the source point cloud i The feature matrix of a superpoint, Indicates the yth point in the target point cloud i The feature matrix of a superpoint, Represents feature dimension; T4, calculate the soft allocation matrix Z of the local matching matrix i , and select the corresponding super-point matching pairs in the final source point cloud and target point cloud through the mutual top-K algorithm: in, Indicates the xth point in the source point cloud i A local patch constructed from superpoints, Indicates the yth point in the target point cloud i The local patch is constructed by super points, mutual topk means the mutual selection top-K algorithm, Represents the soft assignment matrix Z i The elements in corresponding to the x and y coordinate directions; Furthermore, the predicted rigid body transformation matrix in S4 is calculated using the corresponding local patches of superpoint matching pairs in the source point cloud and the target point cloud.

6. The method according to claim 1, characterized in that The calculating the density consistency loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix comprises: Calculate the feature variance of points in local patches of the source point cloud and the target point cloud, and calculate the average feature variance of points in local patches of the source point cloud and the target point cloud respectively; The density consistency loss is obtained by summing the average feature variance of the points within the local patch of the source point cloud and the average feature variance of the points within the local patch of the target point cloud.

7. The method according to claim 1, characterized in that The predicted rigid body transformation matrix includes a predicted rotation matrix and a predicted displacement vector; the calculation of the posture consistency loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix includes: Calculate the true rotation matrix and true translation vector according to the changes in height, angle, and distance during the process of scanning the source point cloud and the target point cloud by the multi-view LiDAR scanning device; Calculate the error between the predicted rotation matrix, predicted translation vector of the source point cloud and the target point cloud and the true rotation matrix and true translation vector, which is recorded as the posture consistency loss.

8. The method according to claim 1, characterized in that Calculating the geometric consistency loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix includes: The geometric distance between the source point cloud after the real transformation and the predicted transformation is calculated and recorded as the geometric consistency loss.

9. The method according to claim 5, characterized in that Calculating the point matching loss between the global predicted rigid body transformation matrix and the real rigid body transformation matrix includes: In the distribution matrix Z i The negative log-likelihood loss is used as the point matching loss corresponding to a single point: The overall point matching loss during point cloud transformation is: Among them, N g represents the number of randomly sampled true superpoint matching pairs, represents the true matching pair of a set of downsampled points extracted, I i and J i Represents a set of mismatched downsampling points.

10. An aircraft assembly point cloud registration device based on LiDAR scanning data, characterized in that: include: A scanning module, used to obtain source point cloud and target point cloud of the aircraft model through a multi-view LiDAR scanning device; A feature extraction module is used to extract features from the source point cloud and the target point cloud respectively; A sampling module, used to downsample the acquired features to obtain downsampled points, select superpoints from the downsampled points, and then construct a local patch with each superpoint and K nearby sampling points based on geometric proximity; wherein K is greater than or equal to 1; A registration module is used to calculate the predicted rigid body transformation matrix from each local patch in the source point cloud to the corresponding local patch in the target point cloud, and select the predicted rigid body transformation matrix with the most internal point matches as the global predicted rigid body transformation matrix; a loss calculation module is used to calculate the density consistency loss, posture consistency loss, geometric consistency loss and point matching loss of the global predicted rigid body transformation matrix and the real rigid body transformation matrix, and the comprehensive loss composed of them, and optimize the global predicted rigid body transformation matrix with the goal of minimizing the comprehensive loss to obtain the target rigid body transformation matrix; The result output module is used to align the source point cloud with the target point cloud according to the target rigid body transformation matrix.

Images