A feature vertex-based method for extracting repair quantity of an aviation buckling plate part

Abstract

A method for extracting repair quantities of aerospace fastener parts based on feature vertices is proposed. First, the fastener and fastener groove data are acquired and preprocessed to obtain point clouds of the fastener to be repaired and the fastener groove, as well as point clouds of the theoretical fastener and fastener groove. Then, dimensionality reduction processing of the point cloud data is performed. Next, feature extraction of the fastener and fastener groove is performed to obtain the feature vertices of the fastener and fastener groove. Then, spatial pose registration of the fastener and fastener groove is performed, followed by measurement of the closed joint of the fastener and fastener groove. Finally, the repair quantity of the fastener is calculated. This invention, based on the three-dimensional measurement data of aerospace fasteners, achieves high-precision registration of the complex spatial shape and spatial pose of the fastener groove, rapid measurement of the closed joint, and efficient calculation of the repair quantity, significantly improving the success rate and efficiency of a single repair of aerospace fasteners.

Description

Technical Field

[0001] This invention relates to the field of advanced aviation equipment technology, and specifically to a method for extracting repair quantities of aviation fastener-type parts based on feature vertices. Background Technology

[0002] The assembly process for advanced aviation equipment constitutes a significant portion of the overall manufacturing cycle and is a crucial factor limiting current production capacity. Among these components, the repair and fitting of surface fasteners is an indispensable part of the assembly process, currently relying on skilled manual labor for repeated assembly. For example, small parts such as fasteners and covers on aircraft wings require an average of 2-3 manual measurements and repairs, taking 2-3 hours. Medium-sized parts such as equipment bay doors, landing gear doors, and flap actuator covers on the aircraft fuselage require 3-4 manual measurements and repairs, taking 5-7 hours. Furthermore, connecting fasteners on the upper and lower wing surfaces require multiple people to repeatedly handle and repair them, exceeding 8 hours. Currently, the repair and fitting process for surface fasteners relies on manual measurement and experience-based judgment, which cannot obtain precise repair quantities for the entire fastener outline and lacks a systematic repair selection standard, resulting in inefficiency and low quality. Therefore, significantly improving the efficiency and quality of surface fastener repair and fitting can greatly increase the production capacity of advanced aviation equipment.

[0003] The approach to improving the efficiency and quality of fastener repair has shifted from the original manual, repetitive, one-by-one measurement and repair to batch automated and digital measurement methods. Algorithms related to 3D vision and point cloud processing can extract the geometric features of fastener parts and assemblies, and perform operations such as processing, filtering, and combining these features to ultimately achieve the measurement, positioning, and repair quantity calculation of fastener parts. Currently, some researchers have invented an auxiliary device for adjusting the allowance of cover-type parts. This device uses digital measuring equipment and optical projection equipment to project the amount of grinding on the cover-type parts and marks it with a marker pen. This aims to improve the problem of determining the amount and location of the adjustment. However, this device is cumbersome to build, has low precision, and cannot significantly improve the overall adjustment efficiency (Huang Ning, Zhang Yongliang, Han Ye, et al. Auxiliary device for adjusting the allowance of cover-type parts [P]. Liaoning Province: CN115431129A, 2022-12-06.). Other researchers have proposed a three-dimensional measured data-driven method for precise coordinated adjustment of large aircraft skeletons. This method first acquires the point cloud data of the aircraft skeleton and then registers it with the theoretical digital model using the features of the surfaces and holes. Then, it calculates the distance between the measured point cloud and the theoretical digital model, thereby calculating the compensation distance of the mating surface and the milling trajectory, ultimately achieving precise coordinated adjustment of the aircraft skeleton. However, this method is highly dependent on the part to be adjusted. The structural features of the parts cannot be realized for some parts with weak or complex features (Wang Jun, Wang Yuangeng, Chen Honghua et al. A method for precise coordination and repair of large aircraft skeleton driven by three-dimensional measured data [P]. Jiangsu Province: CN116204990B, 2023-07-07.). At the same time, the team also proposed a method for calculating the repair and processing path of aircraft skin based on tensor voting. The method uses tensor voting to select potential feature points of the point cloud, then classifies them to produce line features, and registers the line features of the measured point cloud with the theoretical point cloud to obtain the repair and processing route. However, the tensor voting link of this method lacks stability and robustness for different point cloud datasets ([1] Wang Jun, Wang Yuangeng, Li Zikuan et al. A method for calculating the repair and processing path of aircraft skin based on tensor voting [P]. Jiangsu Province: CN116541638B, 2023-09-12.).

[0004] To address the problems of the above methods, this patent proposes a method for extracting repair quantities of aerospace fastener parts based on feature vertices. It utilizes the point clouds of the theoretical fastener and its groove, along with their assembly information, as templates. The point cloud information of the fastener to be repaired and its groove are aligned in spatial pose using a point cloud registration algorithm. Furthermore, the repair quantity is calculated based on the edge information of both, thus achieving a highly efficient, high-quality, and low-human-factor-dependent fastener repair process. Currently, no corresponding literature has been published. Summary of the Invention

[0005] To overcome the above-mentioned technical shortcomings, the present invention aims to provide a method for extracting repair quantities of aviation fastener parts based on feature vertices. Based on the three-dimensional measurement data of aviation fasteners, it achieves high-precision registration of the spatial pose of complex spatial surfaces of fastener grooves, rapid measurement of closed seams, and efficient calculation of repair quantities, thereby significantly improving the success rate and efficiency of one-time repair of aviation fasteners.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for extracting repair quantities of aerospace fastener parts based on feature vertices includes the following steps:

[0008] Step 1), Acquisition and preprocessing of buckle plate and buckle plate groove data: The outer contour point cloud data of the buckle plate and buckle plate groove to be repaired is acquired based on a laser 3D scanner, and noise reduction filtering and data downsampling are performed to obtain the point cloud of the buckle plate and buckle plate groove to be repaired; The surface of the CAD model of the buckle plate and buckle plate groove with assembly relationship is sampled based on the PCL open source library to obtain its point cloud data, which is used as the theoretical buckle plate and buckle plate groove point cloud;

[0009] Step 2), Dimensionality reduction of point cloud data: Calculate the surface normals of the point clouds of the fastener to be repaired and the fastener groove, as well as the theoretical fastener and fastener groove point clouds obtained in Step 1), and use the PCA algorithm to solve for their respective principal normal vectors; through coordinate transformation, align the respective principal normal vectors with the Z-axis to obtain the optimal projection direction of the fastener and fastener groove mating boundary, and perform planar projection to achieve dimensionality reduction of point cloud data.

[0010] Step 3), Feature extraction of fasteners and fastener slots: The point clouds of the fasteners to be repaired and the theoretical fasteners obtained in Step 2) after dimensionality reduction are searched for their outer contours to achieve accurate extraction of the two-dimensional outer contour edges. The point clouds of the fastener slots to be repaired and the theoretical fastener slots obtained in Step 2) after dimensionality reduction are searched for their inner contours to achieve accurate extraction of the two-dimensional inner contour edges. Based on a voxel filtering algorithm, the two-dimensional inner and outer contour edges corresponding to the fasteners to be repaired, fastener slots, and theoretical fasteners and fastener slots are downsampled. Then, based on the Harris-3D feature vertex detection algorithm, the feature vertex query of the two-dimensional projected point clouds of the fasteners to be repaired, fastener slots, and theoretical fasteners and fastener slots is realized. Finally, based on a hash mapping algorithm, the two-dimensional vertex point clouds of the fasteners to be repaired, fastener slots, and theoretical fasteners and fastener slots are mapped back to three dimensions to achieve the extraction of edge feature vertices for each data point of the fasteners and fastener slots.

[0011] Step 4), Spatial pose registration of the fastener and fastener slot: For the vertices of the fastener and fastener slot to be repaired and the theoretical fastener and fastener slot obtained in Step 3), perform spatial geometric correspondence point group search and optimal point pairing matching to achieve accurate pairing of the vertices of the fastener to be repaired with the vertices of the theoretical fastener and the vertices of the fastener slot to be repaired with the vertices of the theoretical fastener slot. Based on the SVD algorithm, calculate the pose matrices of the two fasteners and the two fastener slots respectively to achieve coarse alignment of the spatial poses of the fastener to be repaired with the theoretical fastener and the fastener slot to be repaired with the theoretical fastener slot. Based on the ICP algorithm, perform fine registration on the two fasteners and the two fastener slots after coarse alignment to achieve fine alignment of the spatial poses of the fastener and fastener slot to be repaired with the theoretical fastener and fastener slot, thereby realizing the transformation of the fastener and fastener slot to be repaired from a random spatial pose to an assembled state.

[0012] Step 5), Measurement of the closed joint of the buckle and buckle groove: Based on the spatial pose transformation matrix of the point cloud of the buckle and buckle groove to be repaired in Step 4), the pose transformation of the inner and outer contour lines of the buckle and buckle groove in Step 3) is performed to obtain the respective edge lines of the buckle and buckle groove in the assembled state. The distance from each point of the outer contour line of the buckle to the inner contour line point of the buckle groove at the same position is calculated as the measurement of the joint between the buckle and the buckle groove.

[0013] Step 6), Calculation of panel repair amount: Establish a seam information database, define a mathematical model for calculating repair amount information, and pre-store multiple repair method templates in the seam information database; construct an information transmission interface between the panel and panel groove seam measurement algorithm and the repair amount database to realize real-time data interaction between the panel and panel groove seam measurement algorithm and the seam information database; store the seam information calculated in Step 5) in real time in the seam information database, calculate the repair amount using the repair information mathematical model, match it with the repair method templates in the seam information database, and finally return the repair amount information, repair method, and point cloud index of the position to be repaired for each position of the current panel.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0015] (1) This invention innovatively proposes a method for extracting the repair quantity of aviation buckle plate parts based on feature vertices, thereby breaking through the problem that the assembly of aviation equipment buckle plate parts is almost entirely done manually, resulting in low quality and low efficiency, and improving the production capacity of advanced aviation equipment.

[0016] (2) The feature vertex extraction method proposed in this invention is designed for complex spatial curved surfaces. It can quickly and efficiently obtain the feature vertices of complex spatial curved surfaces and can accurately describe the features and structure of different types of buckles.

[0017] (3) The point cloud registration method based on feature vertices proposed in this invention can achieve pose alignment between scanned point clouds and theoretical point clouds on complex spatial curved surfaces with high precision. Compared with other traditional point cloud registration algorithms, it has higher precision and robustness.

[0018] (4) The method for calculating the repair amount of the buckle plate proposed in this invention can accurately obtain the repair amount of all parts of the buckle plate at one time, thereby improving the success rate of workers in repairing the buckle plate in one go. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the process of the present invention.

[0020] Figure 2 Flowchart for dimensionality reduction and feature mapping of buckle panel and buckle panel groove point cloud data.

[0021] Figure 3 This is a flowchart of point cloud registration based on feature vertex matching.

[0022] Figure 4 The input and output diagrams are for aligning the spatial poses of the fastener to be repaired and the theoretical fastener; where (1) is the point cloud of the fastener to be repaired and the point cloud of the theoretical fastener; (2) is the feature edges and feature vertices of the theoretical fastener after dimensionality reduction; (3) is the feature edges and feature vertices of the fastener to be repaired after dimensionality reduction; and (4) is the registration result of the point cloud of the fastener to be repaired and the point cloud of the theoretical fastener.

[0023] Figure 5 The input and output diagrams are for aligning the spatial poses of the slot to be repaired and the theoretical slot; where (1) is the point cloud of the slot to be repaired and the point cloud of the theoretical slot; (2) is the feature edges and feature vertices of the theoretical slot after dimensionality reduction; (3) is the feature edges and feature vertices of the slot to be repaired after dimensionality reduction; and (4) is the registration result of the point cloud of the slot to be repaired and the point cloud of the theoretical slot.

[0024] Figure 6 The amount of trimming on one side of the five-corner buckle plate in this embodiment of the invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to embodiments and accompanying drawings. The present invention can be implemented in many different forms and should not be considered limited to the embodiments described herein. Rather, these embodiments are provided so that the present invention will be thoroughly and completely disclosed and will fully express the scope of the invention to those skilled in the art.

[0026] like Figure 1 As shown, a method for extracting repair quantities of aerospace fastener parts based on feature vertices includes the following steps:

[0027] Step 1), Acquisition and preprocessing of buckle plate and buckle plate groove data: The contour point cloud data of the buckle plate and buckle plate groove to be repaired is acquired using a handheld laser 3D scanner. Noise filtering and data downsampling are performed to remove redundant points and background noise, resulting in the point cloud of the buckle plate and buckle plate groove to be repaired. Surface sampling of the CAD model of the buckle plate and buckle plate groove with assembly relationships is performed using the PCL open-source library to obtain its point cloud data, which serves as the theoretical point cloud of the buckle plate and buckle plate groove. The specific steps are as follows:

[0028] Step 1.1) Use a handheld laser 3D scanner to scan and obtain the surface contour point cloud data of the actual fastener plate and fastener groove to be repaired, so as to simulate the actual operation scenario of the worker on site.

[0029] Step 1.2) Use the pass-through filtering algorithm to locate the specific three-dimensional space grid position of the background noise point, delete the point cloud data located therein, and complete the noise reduction filtering process of the physical point cloud of the buckle and buckle groove to obtain the point cloud of the buckle and buckle groove to be repaired.

[0030] Step 1.3) Using 3D modeling software, the theoretical digital model of the buckle and buckle groove is spatially assembled. Based on the PCL open source library program, the surface of the theoretical digital model of the buckle and buckle groove that retains the spatial assembly relationship is sampled to obtain the point cloud of the theoretical buckle and buckle groove.

[0031] Step 2), Point cloud data dimensionality reduction: Calculate the surface normals of the point clouds of the to-be-repaired fastener and fastener slot, as well as the theoretical fastener and fastener slot point clouds obtained in Step 1). Use the PCA algorithm to solve for their respective principal normal vectors. Align the principal normal vectors with the Z-axis through coordinate transformation to obtain the optimal projection direction of the fastener and fastener slot mating boundary. Perform planar projection to achieve dimensionality reduction of the point cloud data, facilitating subsequent algorithm processing. Figure 2 As shown, the specific steps are as follows:

[0032] Step 2.1): Based on the PCA algorithm, obtain the principal direction vectors of the buckle plate to be repaired, the buckle plate groove point cloud, and the theoretical buckle plate and buckle plate groove point cloud. The calculation process is as follows:

[0033] ① Let the point cloud dataset be P, where P represents a set containing n points, and each point p i From three-dimensional coordinates (x) i ,y i ,z i ) represents a unit vector Let X be the main direction vector of the point cloud data; construct the point cloud data P into a 3×n matrix X, where each column corresponds to the coordinates of a point;

[0034]

[0035] ② Calculate the centroid coordinates of the point cloud data The row mean of X:

[0036]

[0037] ③ Move the centroid of the point cloud data to the origin. The formula for this movement is as follows:

[0038]

[0039] In the above formula, X centered A matrix constructed for the 3D coordinates of the point cloud after the centroid has moved;

[0040] ④ Calculate the covariance matrix C of the point cloud data after movement. The calculation formula is as follows:

[0041]

[0042] ⑤ Perform eigenvalue decomposition on the covariance matrix C to obtain eigenvalues ​​λ1≥λ2≥λ3 and the corresponding eigenvector matrix V; where the column vectors of V are the eigenvectors. The principal direction vector is usually chosen as the eigenvector corresponding to the largest eigenvalue, i.e.

[0043] The geometric surface required for the projection of the buckle and buckle groove is determined by the principal direction vector of the point cloud. Both the inner and outer contours of the buckle can be expressed within this geometric surface.

[0044] Step 2.2): Calculate the spatial transformation matrix from the principal direction vector of each point cloud obtained in Step 2.1) to the Z-axis. Use the spatial transformation matrix to perform spatial transformation on the corresponding overall point cloud so that its principal direction vector is aligned with the Z-axis to obtain the optimal point cloud projection pose.

[0045] Step 2.3): Change the Z-coordinate value of each point in the point cloud to 0, that is, project it onto the XY plane, and set the projection point as p'. i (x' i ,y' i ,z' i This allows for dimensionality reduction of the data, and the projection equations are as follows:

[0046] x' i =x i y' i =y i z' i =0

[0047] Step 3), Feature extraction of the fastener and its groove: The point cloud of the fastener to be repaired and the theoretical fastener, obtained in Step 2), after dimensionality reduction, are searched for their outer contours to achieve accurate extraction of the two-dimensional outer contour edges. The point cloud of the groove to be repaired and the theoretical fastener, obtained in Step 2), after dimensionality reduction, are searched for their inner contours to achieve accurate extraction of the two-dimensional inner contour edges. Based on a voxel filtering algorithm, the two-dimensional inner and outer contour edges corresponding to the fastener, groove, and theoretical fastener are downsampled. Then, based on the Harris-3D feature vertex detection algorithm, the feature vertex query of the two-dimensional projected point cloud of the fastener, groove, and theoretical fastener is performed. Finally, based on a hash matching algorithm, the two-dimensional vertex point cloud of the fastener, groove, and theoretical fastener is mapped back to three dimensions to achieve extraction of the edge feature vertices of each data point of the fastener and groove. Figure 3 As shown, the specific steps are as follows:

[0048] Step 3.1) Use a latitude and longitude scanning method to extract the boundary of the point cloud data of the dimensionality-reduced buckle plate and buckle plate groove, so as to obtain the outer contour point cloud edge line of the buckle plate and the inner contour point cloud edge line of the buckle plate groove respectively.

[0049] Step 3.2) Use the voxel filtering algorithm to downsample the extracted point cloud edges to reduce redundant data without changing the original point cloud spatial coordinates, thereby improving the speed of subsequent data processing.

[0050] Step 3.3): Based on the Harris-3D feature vertex detection algorithm, extract the two-dimensional feature vertices of the edge lines of the point cloud of the buckle and buckle groove obtained after downsampling. These two-dimensional feature vertices correspond to the feature vertices in the original three-dimensional point cloud data of the buckle and buckle groove. The feature vertex extraction algorithm process is as follows:

[0051] ① Calculate the normal vector of each point based on the input discrete point cloud data, and then calculate the covariance matrix M of the points. 3d The calculation formula is as follows:

[0052]

[0053] Where n is the number of nearest neighbors of the search point, n x n y n z The components of the normal vector of the search point calculated from its nearest neighbors in the X, Y, and Z coordinates;

[0054] ② Calculate the characteristic vertex response value of the point based on the characteristic vertex response function. The calculation formula is as follows:

[0055] R = detM 3d-0.04(traceM 3d ) 2

[0056] In the above formula, R is the calculated characteristic vertex response value;

[0057] ③ Set a threshold for the response of a feature vertex. If the response value of a feature vertex at a certain point is greater than the threshold and it is also a local maximum point, then the point is identified as a feature vertex; if it is less than the threshold, then the point is not a feature vertex.

[0058] Step 3.4) Using a hash mapping algorithm, the feature vertex data of the two-dimensional edge lines of the buckle and buckle groove are indexed back into the point cloud data before the XY coordinate plane projection and principal direction vector transformation, to find the outer contour feature vertex of the three-dimensional buckle point cloud and the inner contour feature vertex of the three-dimensional buckle groove point cloud.

[0059] Step 3.5) The point cloud data of the buckle plate and buckle plate groove to be repaired and the point cloud data of the theoretical buckle plate and buckle plate groove model that has been spatially assembled are both processed in steps 3.1)-3.4).

[0060] Step 4), Spatial pose registration of the fastener and its slot: For the vertices of the fastener and slot to be repaired, and the theoretical fastener and slot obtained in Step 3), perform spatial geometric correspondence point group search and optimal point pairing matching to achieve accurate pairing of the vertices of the fastener to be repaired with the vertices of the theoretical fastener and the slot to be repaired with the vertices of the theoretical fastener. Based on the SVD algorithm, calculate the pose matrices of the two fasteners and the two slots respectively to achieve coarse alignment of the spatial poses of the fastener to be repaired with the theoretical fastener and the slot to be repaired with the theoretical fastener. Based on the ICP algorithm, perform fine registration on the coarsely aligned fasteners and slots to achieve fine alignment of the spatial poses of the fasteners to be repaired with the theoretical fastener and slots, thereby realizing the transformation of the fasteners to be repaired from random spatial poses to the assembled state, laying the foundation for subsequent repair quantity extraction; such as Figure 3 As shown, the specific steps are as follows:

[0061] Step 4.1): Search for corresponding point groups in spatial geometry and match optimal point pairs for the feature vertex point clouds of the buckle to be repaired and the theoretical buckle, and for the feature vertex point clouds of the physical buckle groove and the theoretical buckle groove, respectively. The search and matching process is as follows:

[0062] ① Define a type of point group for the extracted feature vertex point cloud: the point group contains three points, each of which is located at a different corner; search and obtain all non-repeating point groups of this type in the point cloud of the fastener to be repaired and the theoretical fastener, and the slot of the fastener to be repaired and the slot of the theoretical fastener.

[0063] ② Make a one-to-one correspondence between the buckle plate to be repaired and the theoretical buckle plate, and between the buckle plate groove to be repaired and the theoretical buckle plate groove, and find the geometrically corresponding point group. That is, the angle on the buckle plate or buckle plate groove where the point in the point group found in the point cloud to be repaired is the same as the angle on the buckle plate or buckle plate groove where the point in the point group found in the theoretical point cloud is located.

[0064] ③ Using the geometrically corresponding point group found in step ② as the data basis, calculate the distance and norm of each point in the point group, set the threshold of distance and norm, traverse the corresponding point group to find the specific spatial correspondence between each point in the point cloud point group to be repaired and each point in the theoretical point cloud point group, and complete the matching of the optimal point pair.

[0065] Step 4.2): Based on the optimal pair data and matching relationship obtained in Step 4.1), the SVD algorithm is used to calculate the coarse registration spatial transformation matrix M from the point cloud of the fastener to be repaired to the theoretical fastener point cloud and from the point cloud of the slot of the fastener to be repaired to the theoretical fastener slot point cloud, respectively. c1 and M c2 ;

[0066] Step 4.3), use the coarse registration spatial transformation matrix M c1 and M c2 Spatial transformations are performed on the point clouds of the fastener plate to be repaired and the slot of the fastener plate to be repaired, respectively, to achieve coarse spatial pose alignment between the fastener plate to be repaired and the theoretical fastener plate, and between the slot of the fastener plate to be repaired and the theoretical fastener slot. The alignment equations are as follows:

[0067]

[0068]

[0069] X' B =M c1 ×x B

[0070]

[0071]

[0072] X' BC =M c2 ×X BC

[0073] In the above formula, X B The coordinate matrix of the point cloud of the fastener to be repaired, (x Bi ,y Bi ,z Bi Let X' be the three-dimensional coordinates of its midpoint. B The coordinate matrix (x') is the point cloud of the fastener to be repaired, coarsely aligned to the theoretical fastener. Bi ,y' Bi ,z'Bi X represents the three-dimensional coordinates of its midpoint; BC The coordinate matrix of the point cloud of the fastener slot to be repaired, (x BCi ,y BCi ,z BCi Let X' be the three-dimensional coordinates of its midpoint. BC The coordinate matrix (x') is the point cloud of the fastener to be repaired, coarsely aligned to the theoretical fastener. BCi ,y' BCi ,z' BCi () represents the three-dimensional coordinates of its midpoint;

[0074] Step 4.4): Based on the point-to-point ICP algorithm, calculate the fine registration spatial transformation matrix M from the point cloud of the fastener to be repaired and the point cloud of the fastener slot to the theoretical fastener and the theoretical fastener slot, which have already undergone coarse spatial pose alignment. c3 and M c4 Each of them is then finely aligned in spatial pose, and the alignment equation is as follows:

[0075]

[0076]

[0077] X″ B =M c3 ×X″ B

[0078]

[0079]

[0080] X″ BC =M c4 ×X' BC

[0081] In the above formula, X″ BB The coordinate matrix (x″) is the point cloud of the fastener to be repaired, after precise alignment with the theoretical fastener. Bi ,y″ Bi ,z″ Bi Let X″ be the three-dimensional coordinates of its midpoint. BC The coordinate matrix (x″) is the point cloud of the fastener to be repaired, after precise alignment with the theoretical fastener. BCi ,y″ BCi ,z″ BCi ( ) represents the three-dimensional coordinates of its midpoint; this completes the transformation of the fastener to be repaired and its slot from a random spatial orientation to an assembled state, as shown in the diagram. Figure 4 and Figure 5 As shown, Figure 4 In (1), the initial pose random point cloud of the mating plate to be repaired and the theoretical point cloud of the mating plate are obtained. Then, after dimensionality reduction and feature extraction, such as... Figure 4 In steps (2) and (3), feature extraction of the edge lines and feature vertices is completed, and point cloud registration is finally achieved. The registration result is shown in the figure below. Figure 4 As shown in (4), it can be clearly seen that the point cloud of the fastener to be repaired and the point cloud of the theoretical fastener have been coarsely aligned in spatial pose. Figure 5 and Figure 4 Similarly, Figure 5 In (1), the point cloud of the mortise and tenon joint with random initial pose is shown, and the point cloud of the theoretical mortise and tenon joint is shown. After dimensionality reduction and feature extraction, the result is as follows: Figure 5 In steps (2) and (3), feature extraction of the edge lines and feature vertices is completed, and point cloud registration is finally achieved. The registration result is shown in the figure below. Figure 5 As shown in (4), it can be clearly seen that the point cloud of the buckle to be repaired and the point cloud of the theoretical buckle have been coarsely aligned in spatial pose.

[0082] Step 5), Measurement of the closed seam of the fastener and its groove: Based on the spatial pose transformation matrix of the point cloud of the fastener and its groove to be repaired in Step 4), the pose transformation of the inner and outer contour lines of the fastener and its groove in Step 3) is performed to obtain the respective edge lines of the fastener and its groove in the assembled state. The distance from each point on the outer contour line of the fastener to the inner contour line point of the groove at the same position is calculated as the measurement of the seam between the fastener and its groove; specifically:

[0083] Step 5.1): Based on the coarse spatial transformation matrix and fine spatial transformation matrix calculated in step 4), perform spatial transformation on the outer contour line of the fastener to be repaired and the inner contour line of the fastener groove to be repaired obtained by scanning using the latitude and longitude method in step 3) to obtain the closed gap of the fastener to be repaired and the fastener groove to be repaired in the spatial assembly state.

[0084] Step 5.2) Traverse each point in the point cloud of the outer contour of the fastener to be repaired, create its R neighborhood and binary tree for each point, and store all points in the R neighborhood (including the points in the inner contour of the fastener to be repaired) in the binary tree for subsequent searching.

[0085] Step 5.3): For each point in the outer contour of the buckle to be repaired, search for the nearest inner contour point of the buckle groove, calculate the Euclidean distance from the point to the nearest point, and use the calculated distance as the gap between the buckle and the buckle groove corresponding to the point, thereby completing the measurement of the gap between the buckle and the buckle groove.

[0086] Step 6), Calculation of panel repair amount: Establish a seam information database, define a mathematical model for calculating repair amount information, and pre-store multiple repair method templates in the seam information database; construct an information transmission interface between the panel and panel groove seam measurement algorithm and the repair amount database to achieve real-time data interaction between the panel and panel groove seam measurement algorithm and the seam information database; store the seam information calculated in Step 5) in real time in the seam information database, calculate the repair amount using the repair information mathematical model, match it with the repair method templates in the seam information database, and finally return the repair amount information, repair method, and point cloud index of the position to be repaired for each position of the current panel; specifically:

[0087] Step 6.1) Based on the digital model of the theoretical buckle plate and the theoretical buckle plate groove, use 3D modeling software to create three spatial assembly states: those that meet the assembly requirements, those that meet the assembly requirements through repair, and those that cannot meet the assembly requirements through repair. Measure the gap amount of each edge of different buckles in different spatial assembly states, and store all gap amounts in the seam information database for comparison with the measured gap amount in actual processing to determine the spatial assembly state of the actual object.

[0088] Step 6.2) Construct an information transmission interface between the panel and panel groove seam measurement algorithm and the repair quantity database, establish the correspondence between the gap amount of the panel and panel groove and the data in the repair quantity database, and realize real-time data transmission and interaction.

[0089] Step 6.3) For different shapes of buckle panels and buckle grooves, define the corresponding mathematical model for fitting amount calculation according to different spatial assembly states. Then, substitute the gap amount measured in actual processing and the determined spatial assembly state into the mathematical model for fitting amount calculation to calculate the fitting amount.

[0090] Step 6.4) Finally, the gap information, repair amount information, repair location, and repair method of each type of buckle and buckle groove are returned and displayed to guide the measurement and repair work of the buckle and buckle groove to be repaired.

[0091] The beneficial effects of this embodiment are as follows: In the process of comparing and analyzing measured data with theoretical numerical models, the principal direction vector is calculated using PCA (Principal Component Analysis). Using this vector as a medium, the optimal projection surface of the pentagonal buckle (groove) point cloud is obtained and projected. Feature vertex features are extracted from the two-dimensional projection surface and backtracked to accurately obtain the feature vertex point cloud of the three-dimensional buckle (groove). This innovative feature extraction method achieves coarse registration between the measured data and theoretical numerical model of the pentagonal buckle (groove). Then, the ICP algorithm is used to complete the fine registration between the measured data and theoretical numerical model of the pentagonal buckle (groove). While ensuring registration accuracy, this greatly improves the speed and robustness of the entire registration process. Finally, by calculating the closest distance from the edge of the pentagonal buckle point cloud to the edge of the pentagonal buckle groove point cloud, the repair amount of the pentagonal buckle is successfully obtained. Figure 6 As shown in Table 1, an accuracy of 0.1mm can be achieved, ultimately realizing the innovative integration of measurement and repair of pentagonal buckles.

[0092] Table 1. Extraction effect of pentagonal buckle panel repair quantity.

[0093]

[0094]

[0095] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless defined as herein.

[0096] It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for extracting repair quantities of aerospace fastener-type parts based on feature vertices, characterized in that, Includes the following steps: Step 1), Acquisition and preprocessing of buckle plate and buckle plate groove data: Obtain the outer contour point cloud data of the buckle plate and buckle plate groove to be repaired based on the laser 3D scanner, perform noise reduction filtering and data downsampling to obtain the point cloud of the buckle plate and buckle plate groove to be repaired; Perform surface sampling on the CAD model of the buckle plate and buckle plate groove with assembly relationship based on the PCL open source library to serve as the theoretical buckle plate and buckle plate groove point cloud. Step 2), dimensionality reduction of point cloud data: Calculate the surface normals of the point clouds of the fastener to be repaired and the fastener groove, as well as the theoretical fastener and fastener groove point clouds obtained in Step 1), and use the PCA algorithm to solve for their respective principal normal vectors; through coordinate transformation, align the respective principal normal vectors with the Z-axis to obtain the optimal projection direction of the fastener and fastener groove mating boundary, and perform planar projection to achieve dimensionality reduction of point cloud data. Step 3), Feature Extraction of Buckle Panels and Buckle Slots: The point clouds of the buckle panels to be repaired and the theoretical buckle panels obtained in Step 2) are subjected to outer contour search to accurately extract the two-dimensional outer contour edges of the point clouds. The point clouds of the buckle slots to be repaired and the theoretical buckle slots obtained in Step 2) are subjected to inner contour search to accurately extract the two-dimensional inner contour edges of the point clouds. Based on a voxel filtering algorithm, the two-dimensional inner and outer contour edges corresponding to the buckle panels to be repaired, buckle slots, and theoretical buckle panels and buckle slots are downsampled. Then, based on the Harris-3D feature vertex detection algorithm, the feature vertex query of the two-dimensional projected point clouds of the buckle panels to be repaired, buckle slots, and theoretical buckle panels and buckle slots is realized. Finally, based on a hash mapping algorithm, the two-dimensional vertex point clouds of the buckle panels to be repaired, buckle slots, and theoretical buckle panels and buckle slots are mapped back to three dimensions to realize the extraction of edge feature vertices of each data point of the buckle panels and buckle slots. Step 4) Spatial pose registration of the fastener and fastener slot: For the vertices of the fastener to be repaired, fastener slot and theoretical fastener and fastener slot obtained in Step 3), perform spatial geometric corresponding point group search and optimal point pairing matching to achieve accurate pairing of the vertices of the fastener to be repaired with the vertices of the theoretical fastener, and accurate pairing of the vertices of the fastener slot to be repaired with the vertices of the theoretical fastener slot. Based on the SVD algorithm, calculate the pose matrices of the two fasteners and the two fastener slots respectively to achieve coarse spatial pose alignment between the fastener to be repaired and the theoretical fastener, and between the fastener slot to be repaired and the theoretical fastener slot. Based on the ICP algorithm, the two fasteners and two fastener slots after coarse alignment are finely registered so that the fasteners and fastener slots to be repaired are finely aligned with the theoretical fasteners and fastener slots in spatial pose, thereby realizing the transformation of the fasteners and fastener slots to be repaired from random spatial pose to assembly state. Step 5), Measurement of the closed seam of the buckle and buckle groove: Based on the spatial pose transformation matrix of the point cloud of the buckle and buckle groove to be repaired in Step 4), the pose transformation of the inner and outer contour lines of the buckle and buckle groove in Step 3) is performed to obtain the respective edge lines of the buckle and buckle groove in the assembled state. The distance from each point of the outer contour line of the buckle to the inner contour line point of the buckle groove at the same position is calculated as the measurement of the seam between the buckle and the buckle groove. Step 6), Calculation of panel repair amount: Establish a seam information database, define a mathematical model for calculating repair amount information, and pre-store multiple repair method templates in the seam information database; construct an information transmission interface between the panel and panel groove seam measurement algorithm and the repair amount database to realize real-time data interaction between the panel and panel groove seam measurement algorithm and the seam information database; store the seam information calculated in Step 5) in real time in the seam information database, calculate the repair amount using the repair information mathematical model, match it with the repair method templates in the seam information database, and finally return the repair amount information, repair method, and point cloud index of the position to be repaired for each position of the current panel.

2. The method according to claim 1, characterized in that, Step 1) Specifically: Step 1.1) Use a handheld laser 3D scanner to scan and obtain the surface contour point cloud data of the actual object to be repaired, the buckle plate and the buckle plate groove; Step 1.2) Use the pass-through filtering algorithm to locate the specific three-dimensional space grid position of the background noise point, delete the point cloud data located in it, and obtain the point cloud of the buckle plate and buckle plate groove to be repaired; Step 1.3) Use 3D modeling software to spatially assemble the theoretical digital models of the buckle and buckle groove. Based on the PCL open source library program, perform surface sampling on the theoretical digital models of the buckle and buckle groove that retain the spatial assembly relationship to obtain the point cloud of the theoretical buckle and buckle groove.

3. The method according to claim 1, characterized in that, Step 2) specifically involves: Step 2.1) Based on the PCA algorithm, obtain the principal direction vectors of the buckle plate to be repaired, the buckle plate groove point cloud, and the theoretical buckle plate and buckle plate groove point cloud. The calculation process is as follows: ① Let the point cloud dataset be... ,in Indicates inclusion A set of points, each point From three-dimensional coordinates Represents a unit vector The principal direction vector of the point cloud data; [This refers to the point cloud data...] Build as matrix Each column corresponds to the coordinates of a point; ② Calculate the centroid coordinates of the point cloud data , , are the row averages of X; ③ Move the centroid of the point cloud data to the origin; ④ Calculate the covariance matrix C of the point cloud data after movement; ⑤ Regarding the covariance matrix Perform eigenvalue decomposition to obtain eigenvalues. and the corresponding eigenvector matrix ;in The column vectors are eigenvectors. The principal direction vector is chosen as the eigenvector corresponding to the largest eigenvalue, i.e. ; The geometric surface required for the projection of the buckle and buckle groove is determined by the principal direction vector of the point cloud. Both the inner and outer contours of the buckle are expressed within this geometric surface. Step 2.2) Calculate the spatial transformation matrix from the principal direction vector of each point cloud obtained in Step 2.1) to the Z-axis. Use the spatial transformation matrix to perform spatial transformation on the corresponding overall point cloud so that its principal direction vector is aligned with the Z-axis to obtain the optimal point cloud projection pose. Step 2.3) Change the Z coordinate value of each point in the point cloud to 0, that is, project the data onto the XY plane to complete the dimensionality reduction process. .

4. The method according to claim 1, characterized in that, Step 3) specifically involves: Step 3.1) Use a latitude and longitude scanning method to extract the boundary of the point cloud data of the dimensionality-reduced buckle plate and buckle plate groove, so as to obtain the outer contour point cloud edge line of the buckle plate and the inner contour point cloud edge line of the buckle plate groove respectively. Step 3.2) Use the voxel filtering algorithm to downsample the extracted point cloud edges; Step 3.3) Based on the Harris-3D feature vertex detection algorithm, extract the two-dimensional feature vertices of the edge lines of the point cloud of the buckle and buckle groove after downsampling. These two-dimensional feature vertices correspond to the feature vertices in the original three-dimensional point cloud data of the buckle and buckle groove. Step 3.4) Using a hash mapping algorithm, the feature vertex data of the two-dimensional edge lines of the buckle and buckle groove are indexed back into the point cloud data before the XY coordinate plane projection and principal direction vector transformation, to find the outer contour feature vertex of the three-dimensional buckle point cloud and the inner contour feature vertex of the three-dimensional buckle groove point cloud. Step 3.5) The point cloud data of the buckle plate and buckle plate groove to be repaired and the point cloud data of the theoretical buckle plate and buckle plate groove model that has been spatially assembled are both processed in steps 3.1)-3.4).

5. The method according to claim 1, characterized in that, Step 4) specifically involves: Step 4.1) Perform spatial geometric point cloud search and optimal point pair matching for the feature vertex point clouds of the buckle to be repaired and the theoretical buckle, and the feature vertex point clouds of the physical buckle groove and the theoretical buckle groove, respectively. The search and matching process is as follows: ① Define a type of point group for the extracted feature vertex point cloud: the point group contains three points, each of which is located at a different corner; search and obtain all non-repeating point groups of this type in the point cloud of the fastener to be repaired and the theoretical fastener, and the slot of the fastener to be repaired and the slot of the theoretical fastener. ② Make a one-to-one correspondence between the buckle plate to be repaired and the theoretical buckle plate, and between the buckle plate groove to be repaired and the theoretical buckle plate groove, and find the geometrically corresponding point group. That is, the angle on the buckle plate or buckle plate groove where the point in the point group found in the point cloud to be repaired is the same as the angle on the buckle plate or buckle plate groove where the point in the point group found in the theoretical point cloud is located. ③ Using the geometrically corresponding point group found in step ② as the data basis, calculate the distance and norm of each point in the point group, set the threshold of distance and norm, traverse the corresponding point group to find the specific spatial correspondence between each point in the point cloud point group to be repaired and each point in the theoretical point cloud point group, and complete the matching of the optimal point pair. Step 4.2), based on the optimal pair data and matching relationship obtained in Step 4.1), uses the SVD algorithm to calculate the coarse registration spatial transformation matrix from the point cloud of the fastener to be repaired to the theoretical fastener point cloud and from the point cloud of the slot of the fastener to be repaired to the theoretical fastener slot point cloud, respectively. and ; Step 4.3), using the coarse registration spatial transformation matrix and Spatial transformations are performed on the point clouds of the fastener plate to be repaired and the slot of the fastener plate to be repaired, respectively, to achieve coarse spatial pose alignment between the fastener plate to be repaired and the theoretical fastener plate, and between the slot of the fastener plate to be repaired and the theoretical fastener slot. The alignment equations are as follows: In the above formula, The coordinate matrix of the point cloud for the fastener to be repaired. Let be the three-dimensional coordinates of its midpoint. This is the coordinate matrix of the point cloud of the fastener to be repaired, after coarse alignment with the theoretical fastener. The three-dimensional coordinates of its midpoint; The coordinate matrix of the point cloud for the fastener slot to be repaired. Let be the three-dimensional coordinates of its midpoint. This is the coordinate matrix of the point cloud of the fastener to be repaired, after coarse alignment with the theoretical fastener. Here are the three-dimensional coordinates of its midpoint; Step 4.4) Based on the point-to-point ICP algorithm, calculate the fine registration spatial transformation matrix from the point cloud of the fastener to be repaired and the point cloud of the fastener slot to the theoretical fastener and the theoretical fastener slot, which have already undergone coarse spatial pose alignment, respectively. and Each of them is then finely aligned in spatial pose, and the alignment equation is as follows: In the above formula, This refers to the coordinate matrix of the point cloud of the fastener to be repaired, after precise alignment with the theoretical fastener. Let be the three-dimensional coordinates of its midpoint. This refers to the coordinate matrix of the point cloud of the fastener to be repaired, after precise alignment with the theoretical fastener. Let be the three-dimensional coordinates of its midpoint.

6. The method according to claim 1, characterized in that, Step 5) specifically involves: Step 5.1) Based on the coarse spatial transformation matrix and fine spatial transformation matrix calculated in Step 4), perform spatial transformation on the outer contour line of the fastener to be repaired and the inner contour line of the fastener groove to be repaired obtained by scanning using the latitude and longitude method in Step 3) to obtain the closed gap of the fastener to be repaired and the fastener groove to be repaired in the spatial assembly state. Step 5.2), traverse each point in the point cloud of the outer contour of the fastener to be repaired, create its R neighborhood and binary tree for each point, and store all points in the R neighborhood into the binary tree. Step 5.3) For each point in the outer contour of the buckle to be repaired, search for the nearest inner contour point of the buckle groove, calculate the Euclidean distance from the point to the nearest point, and use the calculated distance as the gap between the buckle and the buckle groove corresponding to the point, thereby completing the measurement of the gap between the buckle and the buckle groove.

7. The method according to claim 1, characterized in that, Step 6) specifically refers to: Step 6.1) Based on the digital model of the theoretical buckle plate and the theoretical buckle plate groove, use 3D modeling software to create three spatial assembly states: those that meet the assembly requirements, those that meet the assembly requirements through repair, and those that cannot meet the assembly requirements through repair. Measure the gap amount of each edge of different buckles in different spatial assembly states, and store all gap amounts in the seam information database for comparison with the measured gap amounts in actual processing to determine the spatial assembly state of the actual object. Step 6.2) Construct an information transmission interface between the panel and panel groove seam measurement algorithm and the repair quantity database, establish the correspondence between the gap amount of the panel and panel groove and the data in the repair quantity database, and realize real-time data transmission and interaction. Step 6.3) For different shapes of buckle panels and buckle grooves, define the corresponding mathematical model for fitting amount calculation according to different spatial assembly states. Then, substitute the gap amount measured in actual processing and the determined spatial assembly state into the mathematical model for fitting amount calculation to calculate the fitting amount. Step 6.4) Finally, the gap information, repair amount information, repair location, and repair method of each type of buckle and buckle groove are returned and displayed to guide the measurement and repair work of the buckle and buckle groove to be repaired.

Images