Large aircraft complete machine appearance three-dimensional measurement method and device based on multi-component cooperation

Abstract

The invention relates to the technical field of aeronautical manufacturing, solves the technical problems that single measurement equipment is generally limited in measurement range, insufficient in capability of capturing details of a complex curved surface and susceptible to environmental factors, and particularly relates to a three-dimensional measurement method and device for the overall shape of a large aircraft based on multi-component collaboration. Comprising the following steps: acquiring contour point clouds and carrying out registration to obtain local contour point clouds of a whole machine under different local visual angles; and registering the local contour point clouds and constructing a sparse pose map, and registering the local contour point clouds acquired under different local view angles based on the sparse pose map to generate a complete overall contour point cloud of the large aircraft. According to the invention, the high-precision local point cloud data of the aircraft acquired by the measuring device can be registered, and the point cloud data of the surface of the large aircraft can be fully automatically generated, so that the high-precision and high-efficiency measurement of the large-size complex appearance can be realized, and the manufacturing and assembling quality of the aircraft can be further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of aviation manufacturing, and particularly to a three-dimensional measurement method and device for the overall shape of a large aircraft based on multi-component collaboration. Background Art

[0002] With the rapid development of the aviation industry, the design and manufacturing of large aircraft have continuously higher requirements for accuracy. During the overall aircraft manufacturing and assembly process, the three-dimensional measurement of the aircraft shape has become a crucial link, and its measurement accuracy directly affects the aerodynamic performance, structural strength, and assembly quality of the entire aircraft. However, due to the large volume and complex structure of the aircraft, traditional measurement methods (such as contact measurement or a single laser measurement device) are difficult to simultaneously meet the requirements of high precision, high efficiency, and economy.

[0003] In the prior art, a single measurement device usually has problems such as limited measurement range, insufficient ability to capture details of complex curved surfaces, and susceptibility to environmental factors. To solve these problems, the industry has gradually explored automated measurement technologies to achieve efficient three-dimensional measurement of the large aircraft shape through automated devices. However, existing automated measurement devices still have many deficiencies in practical applications. For example, the error differences of different measurement devices in the local coordinate system are significant, and it is difficult to calibrate the consistency of the measurement coordinate systems. At the same time, the overall stability and accuracy guarantee of the system still need to be improved. Summary of the Invention

[0004] Aiming at the deficiencies of the prior art, the present invention provides a three-dimensional measurement method and device for the overall shape of a large aircraft based on multi-component collaboration, which solves the technical problems that a single measurement device usually has a limited measurement range, insufficient ability to capture details of complex curved surfaces, and susceptibility to environmental factors.

[0005] To solve the above technical problems, the present invention provides the following technical solutions: A three-dimensional measurement method for the overall shape of a large aircraft based on multi-component collaboration, the method comprising the following steps:

[0006] S1. Obtain the contour point cloud of the overall aircraft at different local perspectives, and register the contour point cloud of the overall shape based on the local positioning reference to obtain the local contour point cloud of the overall aircraft;

[0007] S2. Register the local contour point clouds at different local perspectives to obtain the registration results of any two pairs of point cloud data containing the feature matching key points, and the registration result is a set of rigid transformation {R, t} between different point cloud data in the local contour point cloud, that is, the rotation matrix R and the translation vector t;

[0008] S3. Estimate the similarity of any two pairs of point cloud data based on key points, and construct a sparse pose graph G(V, E) using the similarity of any two pairs of point cloud data and the overlap prior during the scanning process;

[0009] S4. Based on the sparse pose graph G(V, E), register the local contour point clouds obtained from different local perspectives to generate the complete contour point cloud of the large aircraft.

[0010] Further, in step S2, the specific process includes the following steps:

[0011] S21. Preprocess the local contour point clouds from different local perspectives, and the preprocessing includes removing the useless noise points in the local contour point clouds;

[0012] S22. Calculate the normal direction and curvature of each point in the local contour point cloud, and use the downsampling combined with the normal direction and curvature to obtain the subset of the point cloud data to be registered into pairs, that is, the key points for feature matching;

[0013] S23. According to the normal information corresponding to the key points, generate the feature vectors of the key points by combining the RCS feature descriptor

[0014] S24. According to the feature vectors of the key points Use the NNSR feature matching method to generate the corresponding relationship set C of the key points Nodes ;

[0015] S25. According to the normal information corresponding to each point in the local contour point cloud, and calculate the FPFH feature vector FPFH(p i ) of each point in the local contour point cloud;

[0016] S26. According to the Euclidean distance between the coordinates of each point in the local contour point cloud and the coordinates of the key points, divide the points in the local contour point cloud into smaller local point clouds with different ranges;

[0017] S27. According to the corresponding relationship set C of the key points Nodes and the FPFH feature vector FPFH(p i ) of each point in the local point cloud corresponding to each key point, generate the rigid transformation between the corresponding relationships of each pair of key points through feature matching to obtain the rigid transformation set of the key points;

[0018] S28. Use all the global corresponding relationship sets C = {C1 ∪ C2 ∪ … ∪ C N}, from the rigid transformation set R = {R i |i = 1,..., N} and t = {t iSelect the most suitable rigid transformation from {i = 1,..., N} and as the rigid transformation between any two point cloud data.

[0019] Furthermore, in step S22, the specific process includes the following steps:

[0020] S221. Use K-nearest neighbor or radius search to obtain the neighborhood point set of any point p in the local shape contour point cloud i of Construct the covariance matrix C and perform eigenvalue decomposition to obtain the eigenvalues λ l , where:

[0021] The expression of the covariance matrix C is:

[0022]

[0023] The expression of eigenvalue decomposition is:

[0024] C·v l = λ l v l , l = 1, 2, 3

[0025] In the formula, is the centroid of the neighborhood point set O(p i ); k is the number of points in the neighborhood point set O(p i ); p j is the neighborhood point of point pi; T is the transpose symbol; v l is the eigenvector corresponding to the eigenvalue λ l ; D thres is the neighborhood radius;

[0026] S222. For the eigenvalues λ0 < λ1 < λ2 of the covariance matrix C, the eigenvector v0 corresponding to the smallest eigenvalue λ0 is the candidate normal direction. According to the view point v viewpoint coordinates, the normal vector n i can be obtained, that is:

[0027]

[0028] S223. Calculate the curvature σ l based on the eigenvalues λ i of the covariance matrix C. The calculation formula is:

[0029]

[0030] S224. According to the normal vector n i and the curvature σ iDivide the local contour point cloud into voxels and calculate the comprehensive score S of all points in each voxel i , and the calculation formula is:

[0031]

[0032] In the formula, n j is the normal vector of the neighborhood point p j ;

[0033] S225. Retain the k points with the highest scores. The points obtained after voxel downsampling are the key points and form the key point set Nodes P .

[0034] Furthermore, in step S23, the specific process includes the following steps:

[0035] S231. For each point in the key point set Nodes P , construct a local coordinate system using its normal vector and the weighted projection vector of the neighborhood points

[0036] S232. Transform the key point set Nodes P to the local coordinate system to obtain the surface and rotate it N times around the Z-axis of the local coordinate system rot , with the rotation angle of θ i each time, to generate a set of rotated surfaces

[0037] S233. Project each rotated surface onto the XY plane of the local coordinate system and extract the projected contour point set

[0038] S234. Centered at the point , divide the XY plane into 12 equally angled fan-shaped regions, and calculate the maximum radial distance d j of the contour point set Sector j in each fan-shaped region as the 12-dimensional contour signature f i , that is:

[0039]

[0040] In the formula, c k is the point in the contour point set Sector j in each fan-shaped region;

[0041] S235. Concatenate the contour signatures {f1, f2,..., f6} generated by 6 rotations in sequence to obtain a 72-dimensional RCS feature vector. The expression is:

[0042]

[0043] In the formula, Concat(·) represents the sequential concatenation operation.

[0044] Furthermore, in step S24, the specific process includes the following steps:

[0045] S241. Use the feature vectors of the key points in any pair of local shape contour point clouds P and Q as the source feature set and the target feature set

[0046] S242. For the source feature search for its nearest neighbor target feature and the second-nearest neighbor target feature in the target feature set and calculate the distances d1 and d2 between them, that is:

[0047]

[0048] S243. Calculate the matching score s assigned to each potential correspondence f (c i ), and the calculation formula is:

[0049]

[0050] S244. Traverse all source features to generate an initial correspondence set and generate a correspondence set C f (c i ) of key points according to the matching score s Nodes , and the expression is:

[0051] C Nodes ={c i ∈C|s f (c i )>τ}

[0052] In the formula, τ is the matching score threshold; C is the global correspondence set.

[0053] Furthermore, in step S25, the specific process includes the following steps:

[0054] S251. According to any point p in the local shape contour point cloud i and its neighborhood point p j 's normal vector n i 、n j Construct a local coordinate system, and calculate three angular parameters: the normal deviation angle α, the projection angle φ, and the pitch angle θ, where:

[0055] The calculation formula for the normal deviation angle α is:

[0056] α = arccos(n i , n j )

[0057] The calculation formula for the projection angle φ is:

[0058] φ = arctan2(y·(p j - p i ), x·(p j - p i ))

[0059] The calculation formula for the pitch angle θ is:

[0060]

[0061] In the formula, x and y are the unit vectors in the directions of the x-axis and y-axis in the local coordinate system based on point p i ;

[0062] S252. Divide each angular parameter into 11 statistical sub-intervals, generate three independent histograms, and splice them together as the feature histogram SPFH(p i );

[0063] S253. Perform weighted calculation on point p i and its neighborhood point set combined with the feature histogram SPFH(p i ) to obtain the FPFH feature vector FPFH(p i ) of each point. The calculation formula is:

[0064]

[0065] In the formula, K is the number of points in the neighborhood point set O(p i ) of point p i ; p k is the point with index k in the neighborhood point set; ε is a very small constant to prevent the denominator from being zero.

[0066] Furthermore, in step S27, the specific process includes the following steps:

[0067] S271. According to the key point pair Extract its local point cloud and the FPFH feature matrix of and and calculate the cost matrix Cos of cosine similarity i , that is:

[0068]

[0069] In the formula, is the feature dimension;

[0070] S272. According to the cost matrix Cos i , use the Sinkhorn algorithm to iteratively solve the soft assignment matrix Z i = Sinkhorn(C i , τ), and extract the matching point set C of high-confidence corresponding relationship point pairs through the mutual top-k screening strategy i ;

[0071] S273. According to the matching point set C i ={(p j , q k )}, calculate the centroid μ i of the matching point set C P and μ Q , and the calculation formula is:

[0072]

[0073] In the formula, p j , q k respectively represent the jth and kth points of the local point cloud and ;

[0074] S274. According to the centroids μ P and μ Q construct the centroid-removed coordinate matrices P c =[p j - μ P and Q c =[qk - μ Q , and calculate the covariance matrix to obtain the optimal rotation matrix R i and the optimal translation vector t i ;

[0075] S275. Based on the optimal rotation matrix R i and the optimal translation vector t i constitute the rigid transformation set R of key points R = {R i |i = 1,..., N} and t = {t i|i = 1, ..., N}.

[0076] Further, in step S3, the specific process includes the following steps:

[0077] S31. Extract the global feature using the NetVLAD layer according to the feature vector of the point cloud data P i under each perspective and normalize it;

[0078] S32. For any two pairs of point cloud data (P i , P j ), quantify the overlap degree s of the two point clouds through cosine similarity ij , and the calculation formula is:

[0079] s ij = (F i T F j + 1) / 2

[0080] In the formula, F i and F j are the global features corresponding to the two pairs of point cloud data (P i , P j ) respectively and T represents transpose;

[0081] S33. According to the overlap prior o i ∈ {0, 1} scanned by the two pairs of point cloud data (P j , P ij ), and the overlap degree s ij , calculate the similarity S i of any two pairs of point cloud data (P j , P ij ), and the calculation formula is:

[0082] S ij = o ij s ij

[0083] S34. According to the point cloud data P i under each perspective, construct the vertices V = {P i} in the sparse pose graph G(V, E). For each point cloud data P i , select the k adjacent point clouds with the highest similarity S ij to construct the edge set E, and the expression is:

[0084]

[0085] ​S35. On each edge (i, j) ∈ E of the sparse pose graph G(V, E), the value is the point cloud data P i and P j 's relative pose T ij =(R ij , t ij ).

[0086] Furthermore, in step S4, the specific process includes the following steps:

[0087] S41. Calculate the initial weight of the edges E in the sparse pose graph G(V, E), and update the value of the edges E through rotation synchronization and translation synchronization to obtain a new sparse pose graph G'(V', E');

[0088] S42. Recalculate the edge weights according to the new sparse pose graph G'(V', E'), and generate the final sparse pose graph G''(V'', E'') through six iterations;

[0089] S43. Generate a weight-based spanning tree according to the final sparse pose graph G''(V'', E'') and the corresponding weights, and apply the ICP algorithm to optimize the relative pose between the root node and its adjacent nodes through the spanning tree;

[0090] S44. Transform the adjacent nodes through the relative pose and merge them with the root node as a new root node to update the spanning tree, that is:

[0091] Transform the child node P child to the root node coordinate system through the transformation T child , that is:

[0092] P' child = R child P child + t child

[0093] And merge it with the root node as a new root node P' root = P root ∪ P' child ;

[0094] S45. Iterate the above two steps until only one node remains, that is, the complete overall shape contour point cloud of the large aircraft is obtained.

[0095] By means of the above technical solution, the present invention provides a three-dimensional measurement method and device for the overall shape of a large aircraft based on multi-component collaboration, which at least has the following beneficial effects:

[0096] 1. The three-dimensional measurement method for the overall shape of a large aircraft proposed by the present invention can register the high-precision local point cloud data of the aircraft obtained by the measuring device, and automatically generate the point cloud data on the surface of the large aircraft, so as to achieve high-precision and high-efficiency measurement of large-size complex shapes, thereby further improving the manufacturing and assembly quality of the aircraft.

[0097] 2. The three-dimensional measurement device for the overall shape of a large aircraft proposed by the present invention can automatically measure the shape of the large aircraft, achieve high-precision full-automatic measurement of the large-size complex aircraft shape, and obtain accurate local point cloud data of each part of the large aircraft. BRIEF DESCRIPTION OF THE DRAWINGS

[0098] The drawings described herein are used to provide a further understanding of the present application, and constitute a part of the present application. The schematic embodiments and descriptions thereof are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0099] Figure 1 is a flowchart of the three-dimensional measurement method for the overall shape of a large aircraft in the present invention;

[0100] Figure 2 is a schematic diagram of measuring the overall shape of a large aircraft in the present invention;

[0101] Figure 3 is a schematic diagram of the local shape contour point cloud in the present invention;

[0102] Figure 4 is a flowchart of the paired point cloud data registration in the present invention;

[0103] Figure 5 is a flowchart of the multi-viewpoint cloud data registration in the present invention;

[0104] Figure 6 is a schematic diagram of the overall shape contour point cloud in the present invention.

[0105] In the figure: 1. Moving device; 2. Local measuring device; 3. Auxiliary positioning device. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0106] In order to make the above objects, features and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the drawings and specific embodiments. Thereby, a full understanding of how the present application uses technical means to solve technical problems and achieve technical effects can be obtained and implemented accordingly.

[0107] This embodiment proposes a three-dimensional measurement method for the overall shape of a large aircraft based on multi-component collaboration, which can automatically measure the shape of the large aircraft and calculate the registration result between any two pairs of point cloud data based on the local measurement data obtained by automatic measurement. Then, the multi-viewpoint cloud registration result is calculated by combining the local measurement data and the paired point cloud registration result, and the coordinate system is unified to automatically obtain the three-dimensional data of the overall shape of the large aircraft. As Figure 1 shown, this method includes the following steps:

[0108] S1. Obtain the shape contour point cloud of the whole large aircraft from different local perspectives, and register the shape contour point cloud based on the local positioning reference to obtain the local shape contour point cloud of the whole aircraft, as Figure 3 shown. In this embodiment, the local positioning reference is given by a local positioning device, as Figure 2 shown, which consists of a tracking mobile device and a binocular camera. The tracking mobile device is an automated guided vehicle that can move freely in the horizontal direction, and a column is installed above it to realize the tracking and positioning of local measurement devices at different positions. The binocular camera is fixed to the column part, and its height can be adjusted based on local measurement devices at different positions, and it gives the relative pose of the local measurement device with respect to the local positioning device.

[0109] S2. Register the local shape contour point clouds from different local perspectives to obtain the registration result of any two pairs of point cloud data containing feature matching key points. The registration result is a set of rigid transformations {R, t} between different point cloud data in the local shape contour point cloud, that is, the rotation matrix R and the translation vector t. As Figure 4 shown, the specific process includes the following steps:

[0110] S21. Preprocess the local shape contour point clouds from different local perspectives. The preprocessing includes removing useless noise points in the local shape contour point clouds to improve the effect of subsequent processing. In this embodiment, conventional filtering methods are used to further optimize the local shape contour point clouds, retain meaningful points, and remove redundancy and noise at the same time.

[0111] S22. Calculate the normal direction and curvature of each point in the local shape contour point cloud, and use the downsampling combined with the normal direction and curvature to obtain a subset of the point cloud data to be registered into paired point clouds, that is, the key points for feature matching. The specific process includes the following steps:

[0112] S221. Use K-nearest neighbor or radius search to obtain any point p i in the local shape contour point cloud and construct a covariance matrix C and perform eigenvalue decomposition to obtain eigenvalues λ l , where:

[0113] The expression of the covariance matrix C is:

[0114]

[0115] The expression of eigen - decomposition is:

[0116] C·v l =λ l v l , l = 1, 2, 3

[0117] In the formula, is the centroid of the neighborhood point set O(p i ); k is the number of points in the neighborhood point set O(p i ); p j is the neighborhood point of point pi; T is the transpose symbol; v l is the eigen - vector corresponding to the eigenvalue λ l ; D thres is the neighborhood radius, with a value range of 0.05 - 0.2, which can be adjusted according to the point cloud density in this embodiment.

[0118] S222. For the eigenvalues λ0 < λ1 < λ2 of the covariance matrix C, the eigen - vector v0 corresponding to the minimum eigenvalue λ0 is the candidate of the normal direction. According to the view - point v viewpoint coordinates, the normal vector n i can be obtained, that is:

[0119]

[0120] S223. Calculate the curvature σ l based on the eigenvalues λ i of the covariance matrix C. The calculation formula is:

[0121]

[0122] S224. Divide the local shape contour point cloud into voxels according to the normal vector n i and the curvature σ i , and calculate the comprehensive score S i of all points in each voxel. The calculation formula is:

[0123]

[0124] In the formula, n j is the normal vector of the neighborhood point p j .

[0125] S225. Retain the k points with the highest scores. The points obtained after voxel down - sampling are the key points and form the key - point set Nodes P .

[0126] S23. Generate the feature vector of the key point by combining the normal information corresponding to the key point and using the RCS feature descriptor. The specific process includes the following steps:

[0127] S231. For each point in the key point set Nodes P Use its normal vector and the weighted projection vector of the neighborhood points to construct a local coordinate system In this embodiment, the z-axis of the local coordinate system is the normal direction of the point The x-axis takes the orthogonal projection of the point cloud scanning direction and the normal, and after normalization, it is obtained where v is the x-axis direction of the point cloud scanning sensor; the y-axis is calculated by cross product as y = z × x. where v scan is the x-axis direction of the point cloud scanning sensor; the y-axis is calculated by cross product as y = z × x.

[0128] S232. Transform the key point set Nodes P to the local coordinate system to obtain the surface and rotate it N times around the Z-axis of the local coordinate system rot each time by an angle of θ i to generate a set of rotated surfaces where the number of rotations N rot = 6, and the rotation angle θ i = 30°×i (i = 0, 1,..., 5).

[0129] S233. Project each rotated surface onto the XY plane of the local coordinate system and extract the set of contour points C i ;

[0130] S234. With the point as the center, divide the XY plane into 12 equally angled fan-shaped regions, and calculate the maximum radial distance d j of the contour point set Sector j in each fan-shaped region as the 12-dimensional contour signature f i , that is:

[0131]

[0132] where c k is the point in the contour point set Sector j in each fan-shaped region; Sectorj is the set of points in the contour point set C i in each fan-shaped region of the XY plane, j = 1, 2,..., 12;

[0133] S235. Concatenate the contour signatures {f1, f2,..., f6} generated by 6 rotations in order to obtain a 72-dimensional RCS feature vector The expression is:

[0134]

[0135] In the formula, Concat(·) represents the concatenation operation in order.

[0136] S24. According to the feature vectors of the key points Use the NNSR feature matching method to generate a set C of corresponding relationships of the key points Nodes . The specific process includes the following steps:

[0137] S241. Take the feature vectors of the key points in any pair of local shape contour point clouds P and Q as the source feature set and the target feature set

[0138] S242. For the source feature search for its nearest neighbor target feature and the second nearest neighbor target feature in the target feature set and calculate the distances d1 and d2 between them, that is:

[0139]

[0140] S243. Calculate the matching score s assigned to each potential correspondence f (c i ), and the calculation formula is:

[0141]

[0142] S244. Traverse all source features to generate an initial set of corresponding relationships and generate a set C of corresponding relationships of the key points according to the matching score s f (c i ), and the expression is: Nodes C

[0143] C Nodes ={c i ∈C|s f (c i )>τ}

[0144] In the formula, τ is the matching score threshold, and its value ranges from 0.6 to 0.8; C is the global set of corresponding relationships.

[0145] S25. According to the normal information corresponding to each point in the local shape contour point cloud, and combining the use of the FPFH feature descriptor as the calculation method, calculate the FPFH feature vector FPFH(p i ) of each point in the local shape contour point cloud. The specific process includes the following steps:

[0146] S251. According to any point p i and the neighborhood point p j in the local shape contour point cloud, construct a local coordinate system, and calculate three angular parameters: the normal deviation angle α, the projection angle φ, and the pitch angle θ, where: i 、n j Construct a local coordinate system, and calculate three angular parameters: the normal deviation angle α, the projection angle φ, and the pitch angle θ, where:

[0147] The calculation formula for the normal deviation angle α is:

[0148] α = arccos(n i , n j )

[0149] The calculation formula for the projection angle φ is:

[0150] φ = arctan2(y·(p j - p i ), x·(p j - p i ))

[0151] The calculation formula for the pitch angle θ is:

[0152]

[0153] In the formula, x and y are the unit vectors in the x-axis and y-axis directions in the local coordinate system based on the point p i , and the construction method of the local coordinate system is the same as that of the local coordinate system of the key point.

[0154] S252. Divide each angular parameter into 11 statistical sub-intervals, generate three independent histograms, and splice them together as the feature histogram SPFH(p i ) of the point;

[0155] S253. Perform weighted calculation on the point p i and its neighborhood point set in combination with the feature histogram SPFH(p i ) to obtain the FPFH feature vector FPFH(p i ) of each point. The calculation formula is:

[0156]

[0157] In the formula, K is the point pi Neighborhood point set The number of midpoints; p k is the point with index k in the neighborhood point set; ε is a very small constant to prevent the denominator from being zero.

[0158] S26, according to the Euclidean distance between the coordinates of each point in the local contour point cloud and the coordinates of the key point, the points in the local contour point cloud are divided into local point clouds with smaller ranges. The specific process includes the following steps:

[0159] S261, point cloud collection based on local shape contour With key point set The squared Euclidean distance between i,j :

[0160]

[0161] In the formula, Nodes i,d represents the coordinates of any key point in the key point set; P j,d Represents the coordinates of any point in the local contour point cloud set;

[0162] S262, based on the squared Euclidean distance dist i,j For each point p in the local contour point cloud j , assign the nearest key point index, the expression is:

[0163]

[0164] S263, dividing N groups of local point clouds according to the points in a group of local contour point clouds corresponding to each key point.

[0165] S27, according to the corresponding relationship set C of the key points Nodes The FPFH feature vector FPFH(p i ), generate the rigid transformation between each pair of key point correspondences through feature matching, and obtain the rigid transformation set of key points. The specific process includes the following steps:

[0166] S271, according to the key points Extract its local point cloud and The FPFH feature matrix and And calculate the cost matrix Cosine similarity i ,Right now:

[0167]

[0168] In the formula, is the feature dimension;

[0169] S272. According to the cost matrix Cos i , use the Sinkhorn algorithm to iteratively solve the soft assignment matrix Z i = Sinkhorn(C i , τ), and extract the matching point set C of high-confidence correspondence point pairs through the mutual top-k screening strategy i ;

[0170] S273. According to the matching point set C i ={(p j , q k )}, calculate the centroid μ i of the matching point set C P and μ Q . The calculation formula is:

[0171]

[0172] In the formula, p j , q k respectively represent the j-th and k-th points of the local point cloud and ;

[0173] S274. Construct the centroid-removed coordinate matrices P P and μ Q = [p c - μ j and Q P = [q c - μ k , and calculate the covariance matrix to obtain the optimal rotation matrix R i and the optimal translation vector t i . In this embodiment, the covariance matrix is decomposed by SVD, and the covariance matrix S = UΣV Τ , calculate the optimal rotation matrix R i = VU Τ and t i = μ Q - R i μ P .

[0174] S275. Based on the optimal rotation matrix R i and the optimal translation vector t i , construct the rigid transformation sets R = {R i | i = 1,..., N} and t = {t i | i = 1,..., N} of the key points.

[0175] ​S28. Utilize all global correspondence sets \(C = \{C1\cup C2\cup\cdots\cup C\}\), N and select the most suitable rigid transformation from the rigid transformation sets \(R = \{R\) i \(| i = 1,\cdots,N\}\) and \(t=\{t\) i \(| i = 1,\cdots,N\}\) as the rigid transformation between any two pairs of point cloud data. The specific process includes the following steps: and as the rigid transformation between any two pairs of point cloud data. The specific process includes the following steps:

[0176] S281. According to each candidate rigid transformation \((R\) i , \(t\) i ), calculate the number of inliers Inner i in the global correspondence set \(C\), that is:

[0177]

[0178] where \(\tau\) a represents the radius acceptable as an inlier, with a value of 0.15 mm; \(I\) is the indicator function, i.e., Iverson(·); is a pair of corresponding point clouds in the global correspondence set \(C\).

[0179] S282. Select the rigid transformation \((R\) i , \(t\) i ) with the largest number of inliers as the initial optimal solution, that is and perform iterative solution \(N\) r times to obtain the final rigid transformation In this embodiment, the initial optimal solution is used to calculate the inlier correspondences \(C\) among all correspondence point pairs inner , and then the optimal solution after one iteration is calculated from the inlier correspondences \(C\) inner through SVD decomposition Repeat the above process \(N\) r times to obtain the final rigid transformation

[0180] S3. Estimate the similarity of any two pairs of point cloud data based on key points, and construct a sparse pose graph \(G(V, E)\) using the similarity of any two pairs of point cloud data and the overlap prior during the scanning process, where the vertex \(V\) represents the point cloud data in the partial external contour point cloud, and the edge \(E\) represents the relative pose between the point cloud data corresponding to the two vertices connected by this edge, which is calculated by the pairwise registration method proposed in step S2. The specific process includes the following steps:

[0181] S31. Extract the global feature using the NetVLAD layer according to the feature vector i of the point cloud data \(P\) at each perspective And normalize it;

[0182] S32. For any two pairs of point cloud data (P i , P j ), quantify the overlap degree s of the two point clouds through cosine similarity ij , and the calculation formula is:

[0183] s ij =(F i T F j +1) / 2

[0184] In the formula, F i and F j are the global features corresponding to the two pairs of point cloud data (P i , P j ) respectively And T represents transpose;

[0185] S33. According to the overlap prior o i , P j ) scanned by the two pairs of point cloud data (P ij ∈{0, 1}, and the overlap degree s ij Calculate the similarity S i , P j ) of any two pairs of point cloud data (P ij , and the calculation formula is:

[0186] S ij =o ij s ij

[0187] S34. According to the point cloud data P i under each view, construct the vertices V={P i} in the sparse pose graph G(V, E). For each point cloud data P i , select the k adjacent point clouds with the highest similarity S ij to construct the edge set E, and the expression is:

[0188]

[0189] S35. On each edge (i, j)∈E of the sparse pose graph G(V, E), the value is the relative pose T i of the point cloud data P j and P ij =(R ij , t ij ). In this embodiment, the relative pose is the rigid transformation finally obtained in the above step S282, that is, (R ij , tij )。

[0190] S4. Register the local shape contour point clouds obtained from different local perspectives based on the sparse pose graph G(V, E) to generate the complete whole-aircraft shape contour point cloud of the large aircraft. As Figure 5 shown, in step S4, the specific process includes the following steps:

[0191] S41. Calculate the initial weight of the edge E in the sparse pose graph G(V, E), and update the value of the edge E through rotation synchronization and translation synchronization to obtain a new sparse pose graph G′(V′, E′). The specific process includes the following steps:

[0192] S411. According to the value T ij =(R ij , t ij ) of any edge (i, j) in the sparse pose graph G(V, E), and the global correspondence set C i and P j of the corresponding two pairs of point cloud data, calculate the inlier number as the pairwise registration quality score r ij , and the calculation formula is: ij i

[0193]

[0194] In the formula, (p i , p j ) = C ij is the corresponding point cloud in the global correspondence set C ij ; I is the indicator function, that is, Iverson(·); τ is the radius representing the acceptable inliers, with a value of 0.15 m;

[0195] S412. Combine the similarity score S ij of any edge (i, j) in the sparse pose graph G with the registration quality score r ij to calculate the initial weight

[0196] of the edge E in the sparse pose graph G(V, E) S413. Calculate the new rotation matrix using rotation synchronization according to the initial weight and the relative pose

[0197]

[0198] In the formula, R i , i = 1,..., N is the rotation matrix of the global pose of the point cloud P i , i = 1,.., N corresponding to the node with index i in the sparse pose graph G; T is the transpose symbol; R j, where \(j = 1,\cdots,N\) is the point cloud \(P\) with index \(j\) in the sparse pose graph \(G\). j , the rotation matrix of the global pose of \(j = 1,\cdots,N\). are two point clouds \(P\) corresponding to the edge \((i, j)\in E\) in the sparse pose graph \(G\). i and \(P\) j The rotation matrix of the initial relative pose between them; where the superscript is the iteration number, starting from 0.

[0199] S414. Calculate the new translation matrix using translational synchronization, that is:

[0200]

[0201] In the formula, \(t\) i , where \(i = 1,\cdots,N\) is the translation matrix of the global pose of the point cloud \(P\) corresponding to the node with index \(i\) in the sparse pose graph \(G\); \(t\) i , where \(i = 1,\cdots,N\); \(t\) j , where \(j = 1,\cdots,N\) is the translation matrix of the global pose of the point cloud \(P\) corresponding to the node with index \(j\) in the sparse pose graph \(G\); j , where \(j = 1,\cdots,N\). are two point clouds \(P\) corresponding to the edge \((i, j)\in E\) in the sparse pose graph \(G\). i and \(P\) j The translation matrix of the initial relative pose between them; where the superscript is the iteration number, starting from 0.

[0202] S415. Update the value of the edge \(E\) to obtain a new sparse pose graph \(G'(V', E')\), where

[0203]

[0204] In the formula, \(T\) is the transpose symbol; where the superscript is the iteration number, starting from 0.

[0205] S42. Recalculate the edge weights according to the new sparse pose graph \(G'(V', E')\) and iterate six times to generate the final sparse pose graph \(G''(V'', E'')\). The specific process includes the following steps:

[0206] S421. Calculate the rotation residual according to the value of the edge \(E\) in the sparse pose graph \(G'(V', E')\) that is: namely:

[0207]

[0208] In the formula, \(R\) i , where \(i = 1,\cdots,N\) and \(R\) j, where \(j = 1,\ldots,N\) are the point clouds \(P\) corresponding to the nodes with indices \(i\) and \(j\) in the sparse pose graph \(G\). i , where \(i = 1,\ldots,N\) and \(P\) j , where \(j = 1,\ldots,N\) are the rotation matrices of the global poses; \(\Delta(R_1,R_2)\) represents the angular difference between rotations \(R_1\) and \(R_2\); where the superscript is the iteration number, starting from 0.

[0209] S422. According to the rotation residual and the coefficient function of the iteration number \(m\) calculate the updated weight That is:

[0210]

[0211] S423. Loop and iterate steps S41 and S42 to generate the final sparse pose graph \(G''(V'',E'')\).

[0212] S43. According to the final sparse pose graph \(G''(V'',E'')\) and the corresponding weights, generate a weight-based spanning tree, and apply the ICP algorithm to optimize the relative pose between the root node and its adjacent nodes; the specific process includes the following steps:

[0213] S431. According to the sparse pose graph \(G''(V'',E'')\) and the edge weight \(w\) ij \(\in [0,1]\), sort the edge set \(E''\) in descending order of the edge weight \(w\). ij

[0214] S432. Traverse the sorted edges. If the two vertices connected by the edge belong to different sets, then add the edge to the spanning tree \(T(V\) T ,E T ). Specifically: Initialize the vertex set \(V\) of the spanning tree \(T(V\) T ,E T ) as \(V\) T \(=\{P_0|P_0\in V''\}\). Traverse the sorted edge set. If the two vertices connected by the edge do not all belong to the set \(V\) T , then add the edge to the edge set \(E\) of the spanning tree T , and add the vertices that do not belong to the set \(V\) T to the vertex set. Until all vertices in \(V''\) are added to \(V\) T , the algorithm ends.

[0215] S433. Denote the vertex weight as the sum of the weights of the adjacent edges Select the vertex with the highest weight as the root node \(P\) root , for each edge \((P\) T ,E T ) in the spanning tree \(T(V\) root ,P child), with the root node coordinate system as the reference, apply the ICP algorithm to optimize the relative pose of the child node to obtain the transformation T child , that is:

[0216]

[0217] In the formula, R and t are the rotation matrix and translation vector respectively.

[0218] S44. Transform adjacent nodes through relative poses and merge them with the root node as the new root node to update the generation tree, that is:

[0219] Transform the child node P child through the transformation T child to the root node coordinate system, that is:

[0220] P′ child = R child P child + t child

[0221] and merge it with the root node as the new root node P′ root = P root ∪ P′ child ;

[0222] S45. Loop and iterate the above two steps until only one node remains, that is, the complete overall shape contour point cloud of the large aircraft is obtained, as Figure 6 shown.

[0223] The three-dimensional measurement method for the overall shape of the large aircraft proposed by the present invention can register the high-precision aircraft local point cloud data obtained by the measurement device, automatically generate the point cloud data on the surface of the large aircraft, so as to realize the high-precision and high-efficiency measurement of the large-size complex shape, thereby further improving the manufacturing and assembly quality of the aircraft.

[0224] As Figure 2 shown, this embodiment also provides a device applied to the above three-dimensional measurement method for the overall shape of the large aircraft, including a motion device 1 and a local measurement device 2. The motion device 1 is set according to the position of the large aircraft and drives the local measurement device 2 to move to realize the scanning measurement of the large aircraft to obtain the shape contour point cloud of the large aircraft under the local perspective. An auxiliary positioning device 3, and the auxiliary positioning device 3 registers the shape contour point cloud obtained by the local measurement device 2 based on the local positioning reference by tracking the local measurement device 2 to obtain the local shape contour point cloud.

[0225] The motion device 1 includes a three-direction moving device arranged in the upper surface area of the large aircraft, including a top translation device that moves in two directions in the horizontal plane. An elevating device that moves in the vertical direction is connected to the top translation device. A partial measurement device 2 is installed at the tail end of the elevating device. It also includes an automatic guided vehicle (AGV vehicle) that moves freely on the horizontal plane, and an elevating device that moves in the vertical direction is arranged above it. Another partial measurement device 2 is installed at the tail end of the elevating device. The automatic guided vehicle corresponds to the area of the lower surface of the large aircraft.

[0226] The partial measurement device 2 includes a six-degree-of-freedom moving device installed at the tail end of the elevating device or the elevating equipment. It is an industrial robotic arm that provides a six-degree-of-freedom scanning perspective for the partial measurement device 2, enabling the partial measurement device 2 to collect point cloud data of the surface of the large aircraft under any surface. It also includes a surface point cloud acquisition device, which is a three-dimensional laser scanner and serves as a close-range terminal measurement device. It is carried by the industrial robotic arm and is used to collect point cloud data of the surface of the large aircraft.

[0227] The auxiliary positioning device 3 includes a tracking and moving device that realizes automatic acquisition for each partial acquisition area. It includes an automatic guided vehicle that moves freely in the horizontal direction, and a column is installed above it to realize the tracking and positioning of the partial measurement device 2 at different positions. It also includes a local positioning device, which is a binocular camera fixed to the column part. Its height can be adjusted based on the partial measurement device 2 at different positions, and it gives the relative pose of the partial measurement device 2 relative to the auxiliary positioning device 3.

[0228] Those of ordinary skill in the art can understand that all or part of the steps in the methods of the above embodiments can be completed by instructing relevant hardware through a program. Therefore, this application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) that contain computer-usable program codes.

[0229] The above embodiments have introduced the present invention in detail. Specific examples are used in this article to elaborate on the principles and implementation methods of the present invention. The descriptions of the above embodiments are only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A three-dimensional measurement method for the overall shape of a large aircraft based on multi-component collaboration, characterized in that, The method includes the following steps: S1. Obtain the outer contour point cloud of the whole large aircraft under different local perspectives, and register the outer contour point cloud based on the local positioning reference to obtain the local outer contour point cloud of the whole aircraft; S2. Register the local outer contour point clouds under different local perspectives to obtain the registration results of any two pairs of point cloud data including the key points for feature matching. The registration results are the set of rigid transformations {R, t} between different point cloud data in the local outer contour point cloud, that is, the rotation matrix R and the translation vector t; S3. Estimate the similarity of any two pairs of point cloud data according to the key points, and construct a sparse pose graph G(V, E) by using the similarity of any two pairs of point cloud data and the overlap prior during the scanning process; S4. Register the local outer contour point clouds obtained under different local perspectives based on the sparse pose graph G(V, E) to generate the complete outer contour point cloud of the large aircraft.

2. The three-dimensional measurement method for the overall shape of a large aircraft according to claim 1, wherein In step S2, the specific process includes the following steps: S21. Preprocess the local outer contour point clouds under different local perspectives. The preprocessing includes removing the useless noise points in the local outer contour point clouds; S22. Calculate the normal direction and curvature of each point in the local outer contour point cloud, and obtain the subset of the point cloud data to be registered as paired points by downsampling combined with the normal direction and curvature, that is, the key points for feature matching; S23. Generate the feature vector of the key point by combining the normal information corresponding to the key point and using the RCS feature descriptor S24. According to the feature vectors of the key points Use the NNSR feature matching method to generate the corresponding relationship set C of the key points Nodes ; S25. According to the normal information corresponding to each point in the local shape contour point cloud, and calculate the FPFH feature vector FPFH(p i ) of each point in the local shape contour point cloud; S26. Divide the points in the local outer contour point cloud into smaller local point clouds with different ranges according to the Euclidean distance between the coordinates of each point in the local outer contour point cloud and the coordinates of the key points; S27. According to the set C of corresponding relationships of key points Nodes and the FPFH feature vectors FPFH(p i ) of each point in the local point cloud corresponding to each key point, generate a rigid transformation between each pair of corresponding relationships of key points through feature matching to obtain a set of rigid transformations of key points; S28. Utilize all global correspondence sets C = {C1 ∪ C2 ∪ … ∪ C N}, and select the most suitable rigid transformation from the rigid transformation sets R = {R i | i = 1,..., N} and t = {t i | i = 1,..., N} and as the rigid transformation between any two pairs of point cloud data.

3. The three-dimensional measurement method for the overall shape of a large aircraft according to claim 2, wherein In step S22, the specific process includes the following steps: S221. Obtain any point p in the local shape contour point cloud by using K-nearest neighbor or radius search i and its neighborhood point set Construct a covariance matrix C and perform eigenvalue decomposition to obtain eigenvalues λ l , where: The expression of the covariance matrix C is: The expression of the eigen decomposition is: C·v l = λ l v l , l = 1, 2, 3 In the formula, is the centroid of the neighborhood point set O(p i ); k is the number of points in the neighborhood point set O(p i ); p j is the neighborhood point of point pi; T is the transpose symbol; v l is the eigenvector corresponding to the eigenvalue λ l ; D thres is the neighborhood radius; S222. For the eigenvalues λ0 < λ1 < λ2 of the covariance matrix C, the eigenvector v0 corresponding to the smallest eigenvalue λ0 is the candidate for the normal direction. According to the viewpoint v viewpoint coordinates, the normal vector n can be obtained i , that is: S223. Eigenvalue λ of covariance matrix C l Calculate curvature σ i , and the calculation formula is: S224. According to the normal vector n i and the curvature σ i divide the local shape contour point cloud into voxels and calculate the comprehensive score S of all points in each voxel i , and the calculation formula is: where n j is the normal vector of the neighborhood point p j ; S225. Retain the k points with the highest scores. The points obtained after voxel downsampling are the key points and form the key point set Nodes P .

4. The three-dimensional measurement method for the overall shape of a large aircraft according to claim 2, wherein In step S23, the specific process includes the following steps: S231. For each point in the key point set Nodes P a local coordinate system is constructed by using its normal vector and the weighted projection vector of neighborhood points ​ S232. Transform the key point set Nodes P to the local coordinate system to obtain a surface and rotate it around the Z-axis of the local coordinate system by N rot times, with the rotation angle of θ i each time, to generate a set of rotated surfaces S233. For each rotated surface project it onto the XY plane of the local coordinate system and extract the set of projected contour points S234. With the point as the center, divide the XY plane into 12 equally angled sector regions, and calculate the maximum radial distance d j of the contour point set Sector j as the 12-dimensional contour signature f i , that is: where c k is a point within the set of contour points Sector j in each sector region; S235. Concatenate the contour signatures {f1, f2,..., f6} generated by 6 rotations in sequence to obtain a 72-dimensional RCS feature vector The expression is: Where Concat(·) represents the sequential concatenation operation.

5. The three-dimensional measurement method for the overall shape of a large aircraft according to claim 2, wherein In step S24, the specific process includes the following steps: S241. Use the feature vectors of the key points in any pair of local shape contour point clouds P and Q as the source feature set and the target feature set S242. For the source feature Search for the target feature that is its nearest neighbor in the target feature set and the target feature that is the second nearest neighbor and calculate the distances d1 and d2 between them, i.e.: S243. Calculate each potential correspondence according to distances d1 and d2 The assigned matching score s f (c i ), and the calculation formula is: S244. Traverse all source features Generate an initial set of correspondence relationships And based on the matching score s of each correspondence f (c i ) Generate a set of correspondence relationships C for key points Nodes , the expression is: C Nodes = {c i ∈ C | s f (c i ) > τ} Where τ is the matching score threshold; C is the set of global correspondence relations.

6. The three-dimensional measurement method for the overall shape of a large aircraft according to claim 2, wherein In step S25, the specific process includes the following steps: S251. According to any point p in the local shape contour point cloud i and the neighborhood point p j with the normal vector n i 、n j construct a local coordinate system, and calculate three angular parameters: the normal deviation angle α, the projection angle φ, and the pitch angle θ, where: The calculation formula of the normal deviation angle α is: α = arccos(n i , n j ) The calculation formula of the projection angle φ is: φ=arctan2(y·(p j -p i ),x·(p j -p i )) The calculation formula of the pitch angle θ is: where x and y are unit vectors in the directions of the x-axis and y-axis respectively in the local coordinate system based on point p i ; S252. Divide each angular parameter into 11 statistical sub-intervals, generate three independent histograms, and splice them together as the feature histogram SPFH(p i ); S253. For point p i and its neighborhood point set Combine the signature of histograms of points SPFH(p i ) for weighted calculation to obtain the FPFH feature vector FPFH(p i ) of each point. The calculation formula is as follows: Where K is point p i Neighborhood point set The number of midpoints; p k is the point with index k in the neighborhood point set; ε is a very small constant to prevent the denominator from being zero.

7. The three-dimensional measurement method for the overall shape of a large aircraft according to claim 2, characterized in that In step S27, the specific process includes the following steps: S271. Extract its local point cloud according to the key point pair and the FPFH feature matrix of and calculate the cost matrix Cos of the cosine similarity that is i , namely:​ In the formula, is the feature dimension; S272. According to the cost matrix Cos i , use the Sinkhorn algorithm to iteratively solve the soft assignment matrix Z i = Sinkhorn(C i , τ), and extract the matching point set C of high-confidence corresponding relation point pairs through the mutual top-k screening strategy i ; S273. Calculate the centroid μ P and μ Q of the matching point set C i = {(p j , q k )} according to the matching point set C. The calculation formula is as follows: i ={(p j ,q k )} i P Q ​​​ where p j , q k represent the j-th and k-th points of the local point cloud and respectively; S274. According to the centroid μ P and μ Q Construct the centroid-removed coordinate matrices P c =[p j -μ P and Q c =[q k -μ Q , and calculate the covariance matrix Obtain the optimal rotation matrix R i and the optimal translation vector t i ; S275. Based on the optimal rotation matrix R i and the optimal translation vector t i to form a set of rigid transformations for key points R = {R i | i = 1,..., N} and t = {t i | i = 1,..., N}.

8. The three-dimensional measurement method for the overall shape of a large aircraft according to claim 2, characterized in that, In step S3, the specific process includes the following steps: S31. According to the point cloud data P under each perspective i of the feature vector use the NetVLAD layer to extract the global feature and normalize it; S32. For any two pairs of point cloud data (P i , P j ), the overlap degree s ij of the two point clouds is quantified by cosine similarity, and the calculation formula is: s ij =(F i T F j +1) / 2 Where F i and F j are the global features corresponding to two pairs of point cloud data (P i , P j ), respectively; and T represents the transpose. S33. Calculate the similarity S of any two pairs of point cloud data (P i , P j ) according to the overlap prior o ij ∈{0, 1} and the overlap degree s ij . The calculation formula is: i , P j ) as follows: ij ​ S ij = o ij s ij S34. According to the point cloud data P under each perspective i , construct the vertices V = {P in the sparse pose graph G(V, E) i}, for each point cloud data P i , select the similarity S ij The highest k adjacent point clouds are used to construct the edge set E, and the expression is: S35. On each edge (i, j) ∈ E of the sparse pose graph G(V, E), the value is the relative pose T i between the point cloud data P j and P ij = (R ij , t ij ).

9. The three-dimensional measurement method for the overall shape of a large aircraft according to claim 1, wherein In step S4, the specific process includes the following steps: S41. Calculate the initial weight of the edge E in the sparse pose graph G(V, E), and update the value of the edge E through rotation synchronization and translation synchronization of the initial weight to obtain a new sparse pose graph G′(V′, E′); S42. Recalculate the edge weights according to the new sparse pose graph G′(V′, E′), and generate the final sparse pose graph G″(V″, E″) through six iterative loops; S43. Generate a weight-based spanning tree according to the final sparse pose graph G″(V″, E″) and the corresponding weights, and optimize the relative pose by applying the ICP algorithm to the root node and its adjacent nodes through the spanning tree; S44. Transform the adjacent nodes through the relative pose and merge them with the root node as the new root node to update the spanning tree, that is: Transform the child node P child by the transformation T child to the root node coordinate system, i.e.: P′ child = R child P child + t child And merge it with the root node to form a new root node P' root = P root ∪ P' child ; S45. Iteratively repeat the above two steps until only one node remains, thus obtaining the point cloud of the complete external contour of the large aircraft.

10. An apparatus for the three-dimensional measurement method of the overall shape of a large aircraft according to any one of the above claims 1-9, characterized in that, Including: A motion device (1) and a local measurement device (2), where the motion device (1) is arranged according to the position of the large aircraft and drives the local measurement device (2) to move to scan and measure the large aircraft to obtain the point cloud of the external contour of the large aircraft from a local perspective; An auxiliary positioning device (3), which, by tracking the local measurement device (2), registers the point cloud of the external contour obtained by the local measurement device (2) based on the local positioning reference to obtain the point cloud of the local external contour.

Images