Method and System for Extracting Aircraft Skin Seam Features

Abstract

The present invention discloses a method and system for extracting the features of aircraft skin joints. The method for extracting the features of aircraft skin joints includes the following steps: obtaining a stripe projection feature region map of the joint; synthesizing a three-dimensional point cloud of the feature region; determining the feature center and the cross-section of the three-dimensional point cloud of the feature region in the three-dimensional point cloud of the feature region; projecting the point clouds near the feature center onto the cross-section of the three-dimensional point cloud of the feature region respectively, and respectively using the height difference mutation method and the maximum slope value method to identify the three-dimensional feature edges to obtain the edge point cloud of the feature center region; calculating the gap feature of the joint according to the edge point cloud of the feature center region obtained by the maximum slope value method; calculating the step difference feature of the joint according to the edge point cloud of the feature center region obtained by the height difference mutation method. The present invention proposes two methods for identifying the edge points of the three-dimensional point cloud of the aircraft skin, respectively realizing the edge point positioning of the step difference and the gap of the aircraft skin joint, and enhancing the robustness of the algorithm.

Description

Technical Field

[0001] The present invention relates to the technical field of precision measurement, and particularly to a method and system for extracting features of aircraft skin joints. Background Art

[0002] During the assembly process of an aircraft, due to factors such as machining accuracy, assembly accuracy, and environment, the assembly quality will inevitably deviate from the ideal characteristics, such as between skin joints, between the skin and the structure, and between fuselage sections. The assembly errors spread and accumulate through the production chain, ultimately directly affecting the stealth and aerodynamic performance of the aircraft. Therefore, it is of great significance to strictly control and check the assembly quality. The assembly quality is mainly closely related to elements such as tooling design, assembly sequence, positioning scheme, and shape measurement. Among them, the result of shape measurement is an important indicator to measure the assembly quality. A large number of measurements of skin joint structures are included in the shape measurement work, and relatively high measurement accuracy is required for this structure. Its quality assurance plays an important role in the aerodynamic characteristics, flight safety, flight cost, and stealth of fighter aircraft.

[0003] The traditional measurement of skin joint structures mainly relies on means such as inspection feeler gauges, special measuring tools, and manual observation. This detection method is based on the measurement results of analog quantities, and it is difficult to accurately describe the state of the joint structure. Moreover, it is difficult to meet the actual needs of manufacturing advanced aircraft in terms of information dimension, detection accuracy, and efficiency. When measuring a long skin joint, it is time-consuming and laborious. The gaps and steps of some complex structures cannot be measured because it is difficult to insert a feeler gauge. Moreover, this measurement method can only perform static measurement, cannot achieve real-time online measurement, and cannot quickly import measurement data into a computer for online analysis.

[0004] Visual measurement technology is widely used in precision measurement due to its advantages such as high precision, high efficiency, fast speed, and good repeatability, especially in the fields of industrial manufacturing, biomedicine, face recognition, cultural relic protection, etc. After years of substantial development, the accuracy of visual measurement technology has been able to reach the micron level and can be scanned and displayed in real time. Many industries have also equipped with machine vision scanners and corresponding three-dimensional algorithms to achieve different specific functions. The three-dimensional point cloud data of the aircraft skin joint area can be obtained by scanning the aircraft skin joint using structured light or grating projection technology, but what is obtained is the three-dimensional point cloud data of the entire scanning area. How to calculate the joint feature data (flush and gap) through algorithms is a crucial part in the three-dimensional measurement of the entire skin joint.

[0005] Many scientific research teams and companies have also studied the measurement of aircraft skin joints. The measurement method based on line laser scanning by Long Kun, Xie Qian, etc. uses line laser to scan and measure aircraft skin joints. According to the difference in the point cloud density obtained by the scanner, the characteristic regions are segmented, and then k-mean clustering is performed on the characteristic regions to determine the characteristics. Then, the critical points and boundary points are determined according to the height and slope differences of adjacent points. Finally, the gap and step difference are obtained by taking the difference. Xia Renbo, Chen Songlin, etc. from the Chinese Academy of Sciences used fringe projection technology to measure the characteristics of skin joints, located the joint position according to the correspondence between the three-dimensional point cloud data and the image, determined the edge points on both sides of the joint, calculated the equivalent of the joint characteristics, and improved the fringe projection measurement method, reducing the phase error and improving the adaptability of this method to the surface of the object to be measured. Oyeong Yi, etc. abroad designed an active ranging system based on binocular structured light images, combined the differential image with structured light modulation, reduced the influence of low illuminance on image processing, improved the detection efficiency, and achieved accurate measurement in low-light environments. Tran, etc. applied the monocular vision detection based on multi-line structured light to detect skin joints. This detection method reduces the influence of noise on joint feature extraction by eliminating unqualified images. There are also many technical products, such as the Laser Gauges measuring equipment of Origin Technologies in the United States, the Gocator2530 series of LMI Technologies Inc in Canada, the Gap Gun Pro developed by Third Dimension Software in the UK, the NEXTSENSE of Hexagon in Sweden, etc. They all apply vision measurement technology to measure aircraft skin joints and have the advantages of ease of use, portability, and ruggedness. Among them, measuring equipment such as Laser Gauges and Gap Gun Pro combines vision measurement technology with the measurement method of virtual gauges and has been applied to many airlines.

[0006] The measurement algorithms of different scanning methods are also different. Structured light scanning measurement uses a single-line laser for scanning, so the area scanned each time is very small. The three-dimensional characteristic data of the scanned area is obtained by directly processing the laser characteristic map obtained from each scan. The measurement range of grating projection scanning is relatively large, and the point cloud obtained from each scan is also dense. Extracting and calculating the characteristic data of the joint from numerous three-dimensional point cloud data is not like extracting the characteristic data of only one place from a single characteristic map in structured light scanning. Therefore, it is particularly important to study grating projection for measuring aircraft skin joints. However, there is little research on using grating projection to measure aircraft skin joints, and the existing algorithms have problems such as weak robustness, low measurement accuracy, and large errors when measuring curved joints. Summary of the Invention

[0007] The purpose of the present invention is to provide a method and system for extracting the seam features of an aircraft skin. By establishing a calculation model for aircraft skin feature data in the obtained three-dimensional point cloud of the overall aircraft skin seam, the edge points of the feature point cloud are extracted by the height difference mutation method and the maximum slope value method, so as to realize the extraction of the three-dimensional feature data of the aircraft skin, and achieve the purpose of high-precision detection and high-precision measurement of the seam.

[0008] The present invention provides a method for extracting the seam features of an aircraft skin, including the following steps:

[0009] Obtain the fringe projection feature area map of the seam;

[0010] Synthesize the three-dimensional point cloud of the feature area by combining the seam feature center in the fringe projection feature area map with the data of the feature area calibrated by the sensor; determine the feature center and the feature three-dimensional point cloud section in the three-dimensional point cloud of the feature area according to the seam feature center in the fringe projection feature area map;

[0011] Project the nearby point clouds of the feature center onto the feature three-dimensional point cloud section respectively, and use the height difference mutation method and the maximum slope value method respectively to perform three-dimensional feature edge recognition to obtain the edge point cloud of the feature center area;

[0012] According to the edge point cloud of the feature center area obtained by the maximum slope value method, calculate the gap feature of the seam; according to the edge point cloud of the feature center area obtained by the height difference mutation method, calculate the step difference feature of the seam.

[0013] Further, the obtaining of the fringe projection feature area map of the seam includes:

[0014] Obtain the original image of the grating projection area of the seam and the fringe projection feature map;

[0015] Extract and mark the seam feature center in the original image to obtain the feature area map;

[0016] Mask the fringe projection feature map according to the feature area map to obtain the fringe projection feature area map;

[0017] Perform phase calculation on the fringe projection feature area map, and synthesize the three-dimensional point cloud of the feature area by combining the marked data of the seam feature center with the data of the feature area calibrated by the sensor;

[0018] Determine the feature center and the feature three-dimensional point cloud section in the three-dimensional point cloud of the feature area according to the seam feature center in the feature area map.

[0019] Further, the extracting and marking the seam feature center in the original image to obtain the feature area map includes the following steps:

[0020] Denoise the original image;

[0021] Use the Steger algorithm to extract and mark the seam feature center in the denoised original image;

[0022] Dilate and mask the marked seam feature center to obtain a feature region map.

[0023] Furthermore, the seam feature center in the feature region map is the feature center in the three-dimensional point cloud of the feature region;

[0024] Determining the three-dimensional feature point cloud section of the feature region in the three-dimensional point cloud of the feature region according to the seam feature center in the feature region map includes:

[0025] Draw a perpendicular line at the feature center in the feature region map. The perpendicular line is perpendicular to the feature center line in the feature region map, and its two ends intersect the feature edge points on both sides of the feature center respectively;

[0026] Make a perpendicular plane along the perpendicular line, and this perpendicular plane is the three-dimensional feature point cloud section.

[0027] Furthermore, projecting the nearby point clouds of the feature center onto the three-dimensional feature point cloud section respectively includes:

[0028] Set the distance threshold from the nearby point clouds of the feature center to the three-dimensional feature point cloud section as λ, and calculate the distance d from the point cloud in the three-dimensional point cloud of the feature region to the three-dimensional feature point cloud section;

[0029] When the distance d from a certain point cloud in the nearby point clouds of the feature center to the three-dimensional feature point cloud section is less than or equal to the distance threshold λ, then project this point onto the three-dimensional feature point cloud section.

[0030] Furthermore, the method for respectively performing three-dimensional feature edge recognition using the height difference mutation method and the maximum slope value method to obtain the edge point cloud of the feature center region includes the following steps:

[0031] Taking the feature center as the origin and the point cloud arrangement direction as the x-axis, establish a plane rectangular coordinate system in the three-dimensional feature point cloud section;

[0032] In the plane rectangular coordinate system, use the height difference mutation method to determine the feature edge points on both sides of the feature center;

[0033] In the plane rectangular coordinate system, use the maximum slope value method to determine the feature edge points on both sides of the feature center.

[0034] Furthermore, using the height difference mutation method to determine the feature edge points on one side of the feature center includes;

[0035] Take a part of the total point cloud near the feature edge points on one side of the feature center as the starting point cloud data, and calculate the height difference between two adjacent point clouds in the starting point cloud data respectively, to obtain the set P = {|d1|, |d2|,..., |d m |}, calculate the average value of |d1|, |d2|,..., |d m | and the standard deviation is σ;

[0036] Take another part of the total point cloud near the feature edge points on one side of the feature center as the point cloud data to be determined, and calculate the height difference |d| between each point in the point cloud data to be determined and the next adjacent point respectively;

[0037] Use the three - standard - deviation rule to calculate the dispersion degree of each height difference |d| from the average value respectively;

[0038] When the height difference |d| between two adjacent points in the point cloud data to be determined satisfies then add the height difference |d| to the set P;

[0039] When the height difference |d| between a certain point and the next adjacent point in the point cloud data to be determined satisfies then determine this point as the feature edge point of the feature center.

[0040] Furthermore, using the maximum - slope method to determine the feature edge points on one side of the feature center includes the following steps:

[0041] Taking the feature center as the origin, and taking the point cloud arrangement direction as the positive x - axis direction, establish a plane rectangular coordinate system in the feature three - dimensional point cloud section;

[0042] Calculate the slope of the straight line formed by each point cloud in the near - point cloud of the feature edge points on one side of the feature center and the feature center point respectively;

[0043] Select the point with the largest absolute value of the slope of the straight line formed with the feature center point as the feature edge point of the feature center.

[0044] Furthermore, for the edge point cloud of the feature center region obtained by the maximum - slope - value method, calculating the gap feature of the seam includes:

[0045] Suppose the feature edge points on both sides of the feature center obtained by the maximum - slope - value method are (x1, y1) and (x2, y2) respectively, then the gap d of the seam is:

[0046] d = |x1 - x2| (1)

[0047] The edge point cloud of the characteristic center region obtained by the height difference mutation method is calculated to obtain the step difference feature of the seam, including:

[0048] Let the straight line equation of the straight line L1 fitted by the characteristic edge points on one side of the characteristic center obtained by the height difference mutation method and the nearby point cloud on one side of the characteristic center be:

[0049] A1x + B1y + C1 = 0 (2)

[0050] Let the straight line equation of the straight line L2 fitted by the characteristic edge points on the other side of the characteristic center obtained by the height difference mutation method and the nearby point cloud on the other side of the characteristic center be:

[0051] A2x + B2y + C2 = 0 (3)

[0052] Then the step difference h of the seam is:

[0053]

[0054]

[0055]

[0056] Among them, A1, B1, C1 are the coefficients of the straight line equation L1;

[0057] A2, B2, C2 are the coefficients of the straight line equation L2;

[0058] (x i , y i ) is a point on the straight line L1;

[0059] (x k , y k ) is a point on the straight line L2;

[0060] n and m are the numbers of points on the left and right sides in the characteristic region respectively;

[0061] is the average distance from the right-side points to the fitted straight line L1;

[0062] is the average distance from the left-side points to the fitted straight line L2.

[0063] The present invention provides an aircraft skin seam feature extraction system, including:

[0064] An image acquisition module for acquiring a stripe projection feature region map of the seam;

[0065] 3D point cloud data acquisition module, which is used to synthesize the 3D point cloud of the feature region by combining the seam feature center in the stripe projection feature region map with the data of the feature region calibrated by the sensor, and determine the feature center and the cross-section of the feature 3D point cloud in the 3D point cloud of the feature region according to the seam feature center in the stripe projection feature region map;

[0066] Edge point cloud acquisition module, which is used to project the point clouds near the feature center onto the cross-section of the feature 3D point cloud respectively, and use the height difference mutation method and the maximum slope value method respectively to identify the 3D feature edge and obtain the edge point cloud of the feature center region;

[0067] Feature data acquisition module, which is used to calculate the gap feature of the seam according to the edge point cloud of the feature center region obtained by the maximum slope value method, and calculate the step difference feature of the seam according to the edge point cloud of the feature center region obtained by the height difference mutation method.

[0068] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0069] The present invention proposes to locate the seam position of the 3D point cloud from the seam image feature, extract the feature center by using the Steger algorithm, realize the sub-pixel level positioning of the straight line and non-straight line feature centers, and then link the feature center to the feature center of the 3D point cloud to most accurately identify the 3D point cloud centers of the straight seam and the curve seam; the present invention projects the feature point cloud near the cross-section into the cross-section by setting a threshold, which not only excludes the point cloud in the non-feature region, but also minimizes the error caused by the square arrangement of the point cloud; the present invention proposes two edge point cloud extraction algorithms, including the height difference mutation method for identifying the step difference edge points and the maximum slope value method for identifying the gap edge points. The height difference mutation method improves the existing slope difference recognition algorithm and simplifies the steps of the algorithm for identifying mutation points; the maximum slope value method uses the slope change between the seam center and the feature point cloud to identify the feature edge points, and the absolute value of the slope between the feature edge points and the seam center is the largest. This edge point recognition algorithm is fast, can quickly locate the edge points, and can most identify the edge points where the last mutation occurs. The application of these two methods in the curve aircraft skin seam can well determine the perpendicular line perpendicular to the seam, and the effect is significantly better than other algorithms. Description of the Drawings

[0070] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention. In the drawings:

[0071] Figure 1 is the flow chart of the aircraft skin seam feature extraction method proposed by the present invention;

[0072] Figure 2It is a schematic diagram after extracting the feature center by using the Steger algorithm in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention;

[0073] Figure 3 It is a schematic diagram after extracting the feature center of the aircraft skin seam by using the Steger algorithm in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention;

[0074] Figure 4 It is a schematic diagram of the three-dimensional point cloud of the feature region in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention;

[0075] Figure 5 It is a schematic diagram after linking the two-dimensional image to the three-dimensional point cloud of the feature region in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention;

[0076] Figure 6 It is a schematic diagram of the point cloud arranged in a matrix in the three-dimensional point cloud of the feature region in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention;

[0077] Figure 7 It is a schematic diagram after projecting the point cloud data near the feature center region onto the section π in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention;

[0078] Figure 8 It is a schematic diagram of obtaining the edge point cloud of the feature center region by using the height difference mutation method in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention;

[0079] Figure 9 It is a schematic diagram of obtaining the edge point cloud of the feature center region by using the maximum slope method in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention;

[0080] Figure 10 It is a schematic diagram of the phase measurement profilometry measurement system in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention

[0081] Figure 11 It is a schematic diagram of customizing a straight seam in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention;

[0082] Figure 12 It is a schematic diagram of the aircraft skin seam in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention;

[0083] Figure 13 It is a schematic diagram of the selected measurement region in the embodiment of the aircraft skin seam feature extraction method proposed by the present invention;

[0084] Figure 14It is a schematic diagram of the step difference error and the gap error after obtaining the edge point cloud of the feature center region by using the height difference mutation method and the maximum slope method in the embodiment of the aircraft skin joint feature extraction method proposed by the present invention. Detailed implementation manners

[0085] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. However, it should be understood that the protection scope of the present invention is not limited by the specific implementation manners.

[0086] Embodiment

[0087] As Figures 1-14 shown, the aircraft skin joint feature extraction method includes the following steps:

[0088] Step 1: Obtain the stripe projection feature region map of the joint, including:

[0089] Step 1.1: Obtain the original image of the grating projection region of the joint and the stripe projection feature map.

[0090] Step 1.2: Extract and mark the joint feature center in the original image to obtain the feature region map, including:

[0091] Denoise the original image;

[0092] Use the Steger algorithm to extract and mark the joint feature center in the denoised original image. In the two-dimensional feature image, the Steger algorithm extracts the feature center mark along the center of the feature region, and the feature center can be accurate to the sub-pixel level. Use the Steger algorithm to extract the custom line and the feature center at the joint of the aircraft skin, as Figures 2-3 shown.

[0093] Dilate and mask the marked joint feature center to obtain the feature region map.

[0094] Step 1.3: Mask the stripe projection feature map according to the feature region map to obtain the stripe projection feature region map;

[0095] Step 2: Synthesize the joint feature center in the stripe projection feature region map with the data of the feature region calibrated by the sensor to form the feature region three-dimensional point cloud. Among them, perform region dilation and masking along the feature center line, and adjusting the dilation pixel value can extract the image at the feature and its edge, and this image corresponds to the three-dimensional feature region in the three-dimensional point cloud, as Figure 4 shown.

[0096] Among them, determining the feature center and the feature three-dimensional point cloud cross-section in the feature region three-dimensional point cloud according to the joint feature center in the stripe projection feature region map includes:

[0097] Step 2.1: Calculate the phase of the stripe projection feature region map, and synthesize the three-dimensional point cloud of the feature region by combining the marked data of the seam feature center and the data of the feature region calibrated by the sensor. Among them, the seam feature center in the feature region map is the feature center in the three-dimensional point cloud of the feature region.

[0098] The synthesized three-dimensional point cloud data is the three-dimensional point cloud data with only seam features, without the point cloud data of the non-feature region.

[0099] The pixel position of the feature center located by the Steger algorithm in the original image is the physical position of the three-dimensional point cloud of the feature region. Use this position relationship to determine the feature center and cross-section in the three-dimensional point cloud of the feature region. Among them, the seam feature center in the feature region map is the feature center in the three-dimensional point cloud of the feature region.

[0100] Step 2.2: Determine the feature center and the cross-section of the three-dimensional feature point cloud in the three-dimensional point cloud of the feature region according to the seam feature center in the feature region map, including:

[0101] Draw a perpendicular line at the feature center in the feature region map. The perpendicular line is perpendicular to the feature center line in the feature region map, and its two ends intersect at the feature edge points on both sides of the feature center respectively;

[0102] Make a perpendicular plane along the perpendicular line, and this perpendicular plane is the cross-section of the three-dimensional feature point cloud.

[0103] Step 4: Project the nearby point clouds of the feature center onto the cross-section of the three-dimensional feature point cloud respectively, and use the height difference mutation method and the maximum slope value method respectively to perform three-dimensional feature edge recognition to obtain the edge point cloud of the feature center region.

[0104] Step 4.1: Project the nearby point clouds of the feature center onto the cross-section of the three-dimensional feature point cloud respectively, including:

[0105] Set the distance threshold from the nearby point cloud of the feature center to the cross-section of the three-dimensional feature point cloud as λ, and calculate the distance d from the point cloud in the three-dimensional point cloud of the feature region to the cross-section of the three-dimensional feature point cloud;

[0106] When the distance d from a certain point in the nearby point cloud of the feature center to the cross-section of the three-dimensional feature point cloud is less than or equal to the distance threshold λ, project this point onto the cross-section of the three-dimensional feature point cloud.

[0107] The present invention proposes two different feature point edge recognition algorithms, the height difference mutation method and the maximum slope value method, and realizes the positioning of the edge points of the step difference and gap of the seam respectively according to these two methods.

[0108] Step 4.2: Use the height difference mutation method and the maximum slope value method respectively to perform three-dimensional feature edge recognition to obtain the edge point cloud of the feature center region, including the following steps:

[0109] Step 4.2.1: Taking the feature center as the origin, establish a plane rectangular coordinate system in the cross-section of the feature three-dimensional point cloud with the point cloud arrangement direction as the x-axis;

[0110] Step 4.2.2: In the plane rectangular coordinate system, use the height difference mutation method to determine the feature edge points on both sides of the feature center.

[0111] Among them, using the height difference mutation method to determine the feature edge points on one side of the feature center includes the following steps:

[0112] Taking a part of the total number of point clouds near the feature edge points on one side of the feature center as the starting point cloud data, calculate the height difference between two adjacent point clouds in the starting point cloud data respectively, and obtain the set P = {|d1|, |d2|,..., |d m |}, calculate the average value of |d1|, |d2|,..., |d m | and the standard deviation is σ; and the standard deviation is σ;

[0113] Taking another part of the total number of point clouds near the feature edge points on one side of the feature center as the to-be-determined point cloud data, calculate the height difference |d| between each point in the to-be-determined point cloud data and the next point adjacent to it respectively;

[0114] Use the three times standard deviation rule to calculate the dispersion degree of each height difference |d| and the average value respectively;

[0115] When the height difference |d| between two adjacent points in the to-be-determined point cloud data satisfies then add this height difference |d| to the set P;

[0116] When the height difference |d| between a certain point and the next point adjacent to it in the to-be-determined point cloud data satisfies then determine this point as the feature edge point of the feature center.

[0117] Step 4.2.3: In the plane rectangular coordinate system, use the maximum slope method to determine the feature edge points on both sides of the feature center.

[0118] Among them, using the maximum slope method to determine the feature edge points on one side of the feature center includes the following steps:

[0119] Taking the feature center as the origin, establish a plane rectangular coordinate system in the cross-section of the feature three-dimensional point cloud with the point cloud arrangement direction as the positive x-axis direction;

[0120] Calculate the slope of the straight line formed by each point cloud in the point clouds near the feature edge points on one side of the feature center and the feature center point respectively;

[0121] Select the point with the largest absolute value of the slope of the line formed with the feature center point as the feature edge point of the feature center.

[0122] Step 5: Calculate the gap feature of the seam based on the edge point cloud of the feature center region obtained by the maximum slope value method; calculate the step difference feature of the seam based on the edge point cloud of the feature center region obtained by the height difference mutation method, including:

[0123] Let the feature edge points on both sides of the feature center be (x1, y1) and (x2, y2) respectively, then the gap d of the seam is:

[0124] d = |x1 - x2| (1)

[0125] Fit a straight line L1 based on the nearby point cloud and the feature edge point on one side of the feature center, where the fitting straight line equation of the straight line L1 is:

[0126] A1x + B1y + C1 = 0 (2)

[0127] Fit a straight line L2 based on the nearby point cloud and the feature edge point on the other side of the feature center, where the fitting straight line equation of the straight line L2 is:

[0128] A2x + B2y + C2 = 0 (3)

[0129] Then the step difference h of the seam is:

[0130]

[0131]

[0132]

[0133] Among them, A1, B1, C1 are the coefficients of the straight line equation L1; A2, B2, C2 are the coefficients of the straight line equation L2; (x i , y i ) is the point on the straight line L1; (x k , y k ) is the point on the straight line L2; n, m are the numbers of points on the left and right sides in the feature region respectively; is the average distance from the right side points to the fitting straight line L1; is the average distance from the left side points to the fitting straight line L2.

[0134] In the present invention, by using the nearby point cloud of the feature center region and the edge point cloud of the feature center region obtained according to two methods, the features of the seam are calculated to obtain a three-dimensional data calculation model of the seam features. By bringing the collected seam data information into the three-dimensional data calculation model of the seam features, the seam feature calculation is realized.

[0135] In order to more intuitively show the establishment process of the three-dimensional feature calculation model in the non-linear features of the aircraft skin, the three-dimensional feature calculation process of the aircraft skin is selected here to illustrate steps 4-5.

[0136] like Figure 5 As shown, in the two-dimensional feature image, a vertical line is drawn at the feature center A, which is perpendicular to the feature center line and intersects the feature edges on both sides at points B and C. Then the two-dimensional image is linked to the three-dimensional point cloud, and a section π is made along the vertical line, which intersects the α, β, and γ planes at points A, B, and C. Point A is located at the center of the feature, and points B and C are located at the feature edges on both sides. However, due to the existence of the edge arc transition area in the three-dimensional point cloud, the three-dimensional feature value calculated directly using points B and C is not accurate, and points B and C need to be re-determined on the projection surface later. Since the arrangement of the extracted three-dimensional point cloud is in a square array, such as Figure 6 As shown, in the non-linear feature calculation model, not all point clouds fall in the straight line BC in an orderly manner, nor are they all in the plane ABC. If they are directly analyzed, certain errors will be caused.

[0137] To solve this problem, the nearby point cloud data can be projected onto the cross section π for analysis, such as Figure 7 As shown, a distance threshold λ is set, and the distance d from the surface point cloud to the section π is calculated. When d≤λ, the point is projected onto the section, otherwise, the point is not projected onto the section.

[0138] The determination of λ needs to minimize the point distance (Δ) error caused by the square array arrangement of the point cloud, while considering the occurrence of the situation where the points in the non-selected area are projected above the feature area after projection. In the straight feature area, the determination of λ does not have the latter problem. In the non-straight feature area, it only needs to be set The appropriate λ can be determined.

[0139] When the points within the threshold are projected onto plane π, a plane rectangular coordinate system is established with point A as the origin and the point cloud arrangement direction as the x-axis. Figure 8 As shown in the figure, the theoretical gap d is the difference between points B and C in the x-axis direction, and the step difference h is the distance between the two straight lines L1 and L2. However, due to the existence of the arc transition point cloud at the edge of the seam, direct straight line fitting will cause a large error. Therefore, it is necessary to first re-determine the endpoints of the straight lines on both sides of the seam, points B and C.

[0140] According to the characteristics of the arc edge, the present invention proposes two methods for determining points B and C: a height difference mutation method and a slope maximum value method. The following describes these two methods by taking determining point B as an example.

[0141] Height difference mutation method:

[0142] Starting from point D, select 1 / 10 of the total number of point clouds as the starting point clouds (from point D to point E), and calculate the set P of height differences between adjacent point clouds in the starting point clouds, P = {|d1|, |d2|,..., |d m |}, and calculate the average value of |d1|, |d2|,..., |d m | and the standard deviation is σ.

[0143] Starting from point E, calculate the height difference |d| between each point and the next point in turn.

[0144] Use the 3σ method to calculate the degree of dispersion between |d| and . If then the degree of mutation between this point and the next point is not large, and add |d| to the set P.

[0145] If then the degree of mutation between this point and the next point is very large, and determine this point as point B.

[0146] From Figure 8 it can be clearly seen that the height differences between adjacent points from the end point D to B1 are not large. However, between point B1 and point B2, due to the existence of the height difference, the height difference between the two points is also significantly greater than that between any two points from point D to point B1. Therefore, point B1 is determined as point B.

[0147] This method classifies the point clouds in the arc transition region as characteristic point clouds, is very sensitive to the height mutation in the arc transition region, and the maximum clearance value can be obtained for points B and C selected by this method.

[0148] Maximum slope method:

[0149] Starting from point A, traverse all the points on the left side of point A in turn, and calculate the absolute value of the slope |k1|, |k2|,..., |k n | between each point and point A. From Figure 9 it can be clearly seen that |k| from the end point D to point B2 is increasing, and |k| from point B2 to point F is decreasing. The point with the largest |k| value is point B2. Therefore, point B2 is determined as point B.

[0150] This method classifies the point clouds in the arc transition region as plane point clouds, can quickly detect point B2, is not sensitive to the height mutation in the arc transition region, and is not likely to misselect the points on the left side of point B2 as point B. The most accurate clearance value can be obtained for points B and C selected by this method.

[0151] After determining point B(x B , y B ), point C(x C , yC ) Fit a straight line L1: A1x + B1y + C1 = 0 to the point cloud between point D and point B, and fit a straight line L2: A2x + B2y + C2 = 0 to the point cloud between point C and point H. Then the gap is:

[0152] d = |x B -x C | (7)

[0153] Fit a straight line L1 based on the nearby point cloud and the feature edge points on one side of the feature center, where the fitting straight line equation of the straight line L1 is:

[0154] A1x + B1y + C1 = 0 (8)

[0155] Fit a straight line L2 based on the nearby point cloud and the feature edge points on the other side of the feature center, where the fitting straight line equation of the straight line L2 is:

[0156] A2x + B2y + C2 = 0 (9)

[0157] The step difference is: Then the step difference h of the seam is:

[0158]

[0159]

[0160]

[0161] Among them, A1, B1, C1 are the coefficients of the straight line equation L1; A2, B2, C2 are the coefficients of the straight line equation L2; (x i , y i ) is a point on the straight line L1; (x k , y k ) is a point on the straight line L2; n and m are the numbers of points on the left and right sides in the feature region respectively; is the average distance from the right-side points to the fitting straight line L1; is the average distance from the left-side points to the fitting straight line L2.

[0162] In the actual measurement of the aircraft skin, the middle of the seam is hollow, that is, the depth in the middle of the seam cannot be surveyed. However, after implicit calibration, when scanning the skin, the point cloud data in the middle of the seam exists, and this point cloud data exists in the form of a reference plane, that is, h = 0. Although the depth data in the middle of the seam cannot be surveyed, the step difference and gap on both sides can be calculated. For the point cloud data without the middle points of the seam, the slope difference mutation method is still applicable.

[0163] Embodiment 2

[0164] The present invention provides an aircraft skin seam feature extraction system, including:

[0165] An image acquisition module, configured to acquire a stripe projection feature region map of a seam;

[0166] A three-dimensional point cloud data acquisition module, configured to synthesize a three-dimensional point cloud of a feature region by using the seam feature center in the stripe projection feature region map and the data of the feature region calibrated by a sensor, and determine the feature center and the feature three-dimensional point cloud cross-section in the three-dimensional point cloud of the feature region according to the seam feature center in the stripe projection feature region map;

[0167] An edge point cloud acquisition module, configured to project the nearby point clouds of the feature center onto the feature three-dimensional point cloud cross-section respectively, and perform three-dimensional feature edge recognition by using the height difference mutation method and the maximum slope value method respectively to acquire the edge point cloud of the feature center region;

[0168] A feature data acquisition module, configured to calculate and obtain the gap feature of the seam according to the edge point cloud of the feature center region acquired by the maximum slope value method, and calculate and obtain the step difference feature of the seam according to the edge point cloud of the feature center region acquired by the height difference mutation method.

[0169] The method for extracting the features of the aircraft skin seam proposed by the present invention locates the seam position of the three-dimensional point cloud through the seam image features, extracts the feature center by using the Steger algorithm, realizes the sub-pixel level positioning of the straight line and non-straight line feature centers, and then links from the feature center to the feature center of the three-dimensional point cloud, and most accurately identifies the three-dimensional point cloud centers of the straight seam and the curve seam. In addition, the present invention completely establishes a three-dimensional feature calculation model applicable to the straight line segment and the curve segment of the aircraft skin seam, proposes the height difference mutation method for the extraction of the step difference edge points, and proposes the maximum slope value method for the extraction of the gap edge points, and realizes the extraction of the three-dimensional feature data of the aircraft skin according to these two algorithms. These two methods for identifying the edge points of the three-dimensional point cloud of the aircraft skin respectively locate the edge points of the step difference and the gap of the aircraft skin seam, enhancing the robustness of the algorithm. Among them, the height difference mutation method proposed for the extraction of the step difference edge points improves the existing slope difference recognition algorithm and simplifies the steps of the algorithm for identifying the mutation points. The maximum slope value method proposed for the extraction of the gap edge points uses the slope change between the seam center and the feature point cloud to identify the feature edge points, and the absolute value of the slope between the feature edge points and the seam center is the largest. This edge point recognition algorithm is fast, can quickly locate the edge points, and can identify the last mutated edge points to the maximum extent.

[0170] The technical solutions in the present invention will be further described below with reference to specific embodiments.

[0171] In order to verify the effectiveness and superiority of the proposed theoretical derivation and algorithm, an experiment is carried out by using a DLP projector of model DLP4500SL02 and a Basler camera of aca2440 - 75um. Figure 10A monocular phase measurement profilometry measurement system built for the laboratory, where the industrial camera and the projector are not on the same horizontal line. The high-precision displacement stage can move the measurement system in six directions: up, down, left, right, forward, and backward. The calibration plate has an accuracy of 0.005 mm, and the side length of the checkerboard is 6 mm. This measurement system can obtain the three-dimensional point cloud data of the entire scanned feature area in a single static scan.

[0172] In this embodiment, customized straight seams and aircraft skin seams are selected as the measurement objects. The straight seam diagram is as shown in Figure 11 shown, and the aircraft skin seam diagram is as shown in Figure 12 shown. In each of the straight seam and the aircraft skin seam, a section of the measurement area is selected to verify the accuracy of the two point-taking methods and the calculation model. As shown in Figure 13 shown, a vernier caliper is used to measure the measurement points in the left and right figures. The average seam step differences at the measurement points in the left figure are 1.000 mm and 0.600 mm respectively, and the average seam widths at the measurement points in the right figure are 2.998 mm and 1.999 mm respectively.

[0173] In the straight seam and the aircraft skin seam, 2 measurement areas are respectively selected, and 50 feature centers are evenly selected in each of the 4 measurement areas. The points in the point cloud where the feature centers are located are taken by two point-taking methods, and the three-dimensional feature data calculation model is used to calculate the selected point cloud data. The step difference errors and gap errors calculated by the two point-taking methods are as shown in Figure 14 shown. Except for the calculation error of the gap d caused by individual incorrect point-taking in the height difference mutation method, there are no other calculation errors. The measurement errors are specifically as follows:

[0174] Figure 14 (a) h = 1.000 mm, Figure 14 (b) h = 0.600 mm; Figure 14 (c) d = 2.998 mm, Figure 14 (d) d = 1.999 mm.

[0175] The measured data are divided into 5 groups, with 10 columns of point cloud in each group. The calculated average values and error analysis are shown in Table 1.

[0176] Table 1 Error Analysis

[0177]

[0178] From the data in the table, it can be seen that the average error of the step difference h calculated by the first point selection method is less than 0.040 mm, and the maximum error is less than 0.080 mm. The calculated gap d has a huge error due to the misselection of points B and C in individual groups, but the average error is still less than 0.600 mm. The average error of the step difference h calculated by the second point selection method is less than 0.090 mm, and the maximum error is less than 0.180 mm. The calculated gap result is better than the first point selection method, with an average error less than 0.015 mm and a maximum error less than 0.080 mm. Therefore, the step difference measured by the height difference mutation method and the gap measured by the slope maximum value method should be the final measurement results.

[0179] Compared with the existing classical three-dimensional feature calculation models, the algorithm proposed in the present invention has better robustness, especially in the three-dimensional measurement of curve seams. The present invention systematically establishes a three-dimensional data measurement model for curve seams for the first time, solves the problem of point cloud selection perpendicular to the curve seam, reduces the three-dimensional measurement of features to two-dimensional measurement, and proposes two algorithms for quickly determining feature edge points. The three-dimensional data of the seam can be well measured according to these two edge point determination algorithms.

[0180] Finally, it should be noted that the above disclosure is only a specific embodiment of the present invention. However, the embodiments of the present invention are not limited thereto, and any changes that can be thought of by those skilled in the art should fall within the protection scope of the present invention.

Claims

1. Method for extracting features of aircraft skin joints, characterized in that, Including the following steps: Obtain the stripe projection feature area map of the seam; Synthesize the 3D point cloud of the feature area by combining the seam feature center in the stripe projection feature area map with the data of the feature area calibrated by the sensor; determine the feature center and the feature 3D point cloud cross-section in the 3D point cloud of the feature area according to the seam feature center in the stripe projection feature area map; Project the nearby point clouds of the feature center onto the feature 3D point cloud cross-section respectively, and use the height difference mutation method and the maximum slope value method respectively to identify the 3D feature edges and obtain the edge point cloud of the feature center area; Calculate the gap feature of the seam according to the edge point cloud of the feature center area obtained by the maximum slope value method; calculate the step difference feature of the seam according to the edge point cloud of the feature center area obtained by the height difference mutation method; The obtaining of the stripe projection feature area map of the seam includes: Obtain the original image of the grating projection area of the seam and the stripe projection feature map; Extract and mark the seam feature center in the original image to obtain the feature area map; Mask the stripe projection feature map according to the feature area map to obtain the stripe projection feature area map; Perform phase calculation on the stripe projection feature area map, and synthesize the 3D point cloud of the feature area by combining the marked data of the seam feature center with the data of the feature area calibrated by the sensor; Determine the feature center and the feature 3D point cloud cross-section in the 3D point cloud of the feature area according to the seam feature center in the feature area map; The extracting and marking the seam feature center in the original image to obtain the feature area map includes the following steps: Denoise the original image; Use the Steger algorithm to extract and mark the seam feature center in the denoised original image; Dilate and mask the marked seam feature center to obtain the feature area map; The seam feature center in the feature area map is the feature center in the 3D point cloud of the feature area; The determining the feature 3D point cloud cross-section in the 3D point cloud of the feature area according to the seam feature center in the feature area map includes: Draw a perpendicular line at the feature center in the feature area map, the perpendicular line is perpendicular to the feature center line in the feature area map, and its two ends intersect at the feature edge points on both sides of the feature center respectively; Make a perpendicular plane along the perpendicular line, and this perpendicular plane is the feature 3D point cloud cross-section; The respectively using the height difference mutation method and the maximum slope value method to identify the 3D feature edges and obtain the edge point cloud of the feature center area includes the following steps: Take the feature center as the origin and establish a plane rectangular coordinate system in the feature 3D point cloud cross-section with the point cloud arrangement direction as the x-axis; In the plane rectangular coordinate system, use the height difference mutation method to determine the feature edge points on both sides of the feature center; In the plane rectangular coordinate system, use the maximum slope value method to determine the feature edge points on both sides of the feature center; Using the height difference mutation method to determine the feature edge points on one side of the feature center includes; Take a part of the total number of point clouds near the feature edge points on one side of the feature center as the starting point cloud data, and calculate the height difference between two adjacent point clouds in the starting point cloud data respectively, to obtain the set P = {|d1|, |d2|,..., |d m |}, calculate the average value of |d1|, |d2|,..., |d m | and the standard deviation is σ; Take another part of the point cloud in the total number of nearby point clouds of the feature edge points on one side of the feature center as the to-be-determined point cloud data, and calculate the height difference |d| between each point in the to-be-determined point cloud data and the next adjacent point respectively; Calculate the dispersion degree of each height difference |d| and the average value respectively using the three - standard - deviation rule ; When the height difference |d| between two adjacent points in the point cloud data to be determined satisfies then add the height difference |d| to the set P; When the height difference |d| between a certain point in the point cloud data to be determined and the next adjacent point to this point satisfies then this point is determined as the feature edge point of the feature center; Determining the feature edge points on one side of the feature center using the maximum slope method includes the following steps: Taking the feature center as the origin, establish a plane rectangular coordinate system in the cross-section of the feature three-dimensional point cloud with the point cloud arrangement direction as the positive x-axis direction; Calculate the slope of the line formed by each point cloud in the nearby point cloud of the feature edge points on one side of the feature center and the feature center point respectively; Select the point with the largest absolute value of the slope of the line formed with the feature center point as the feature edge point of the feature center; For the edge point cloud of the feature center region obtained according to the maximum slope value method, calculating the gap feature of the seam includes: Assume that the feature edge points on both sides of the feature center obtained according to the maximum slope value method are (x1, y1) and (x2, y2) respectively, then the gap d of the seam is: d = |x1 - x2| (1) For the edge point cloud of the feature center region obtained according to the height difference mutation method, calculating the step difference feature of the seam includes: Assume that the linear equation of the line L1 fitted by the feature edge points on one side of the feature center obtained according to the height difference mutation method and the nearby point cloud on one side of the feature center is: A1x + B1y + C1 = 0 (2) Assume that the linear equation of the line L1 fitted by the feature edge points on the other side of the feature center obtained according to the height difference mutation method and the nearby point cloud on the other side of the feature center is: A2x + B2y + C2 = 0 (3) Then the step difference h of the seam is: Where, A1, B1, C1 are the coefficients of the linear equation L1; A2, B2, C2 are the coefficients of the linear equation L2; (x i , y i ) is a point on the straight line L1; (x k , y k ) is a point on the straight line L2; n, m are the numbers of points on the left and right sides in the feature region respectively; is the average distance from the right-side point to the fitted straight line L1; is the average distance from the left point to the fitted line L2.

2. The method for extracting the characteristics of the aircraft skin joint according to claim 1, wherein: Projecting the nearby point cloud of the feature center onto the cross-section of the feature three-dimensional point cloud respectively, including: Setting the distance threshold from the nearby point cloud of the feature center to the cross-section of the feature three-dimensional point cloud as λ, and calculating the distance d from the point cloud in the three-dimensional point cloud of the feature region to the cross-section of the feature three-dimensional point cloud; When the distance d from a certain point cloud in the nearby point cloud of the feature center to the cross-section of the feature three-dimensional point cloud is less than or equal to the distance threshold λ, then project this point onto the cross-section of the feature three-dimensional point cloud.

3. An aircraft skin joint feature extraction system, characterized in that Including: An image acquisition module for acquiring the fringe projection feature region map of the seam; A three-dimensional point cloud data acquisition module for synthesizing the feature region three-dimensional point cloud with the seam feature center in the fringe projection feature region map and the data of the feature region calibrated by the sensor, and determining the feature center and the feature three-dimensional point cloud cross-section in the feature region three-dimensional point cloud according to the seam feature center in the fringe projection feature region map; An edge point cloud acquisition module for projecting the nearby point cloud of the feature center onto the cross-section of the feature three-dimensional point cloud respectively, and performing three-dimensional feature edge recognition using the height difference mutation method and the maximum slope value method respectively to obtain the edge point cloud of the feature center region; A feature data acquisition module for calculating the gap feature of the seam according to the edge point cloud of the feature center region obtained by the maximum slope value method, and calculating the step difference feature of the seam according to the edge point cloud of the feature center region obtained by the height difference mutation method; The acquisition of the fringe projection feature region map of the seam includes: Acquiring the original image of the grating projection region of the seam and the fringe projection feature map; Extract the center of the seam feature in the original image and mark it to obtain the feature region map; Mask the fringe projection feature map according to the feature region map to obtain the fringe projection feature region map; By performing phase calculation on the fringe projection feature region map and combining the marked data of the seam feature center with the data of the feature region calibrated by the sensor, synthesize the three-dimensional point cloud of the feature region; Determine the feature center and the cross-section of the feature three-dimensional point cloud in the three-dimensional point cloud of the feature region according to the seam feature center in the feature region map; The extracting the center of the seam feature in the original image and marking it to obtain the feature region map includes the following steps: Denoise the original image; Use the Steger algorithm to extract the center of the seam feature in the denoised original image and mark it; Dilate and mask the marked center of the seam feature to obtain the feature region map; The center of the seam feature in the feature region map is the feature center in the three-dimensional point cloud of the feature region; The determining the cross-section of the feature three-dimensional point cloud in the three-dimensional point cloud of the feature region according to the seam feature center in the feature region map includes: Draw a perpendicular line at the feature center in the feature region map. The perpendicular line is perpendicular to the feature center line in the feature region map, and its two ends intersect the feature edge points on both sides of the feature center respectively; Make a perpendicular plane along the perpendicular line, and this perpendicular plane is the cross-section of the feature three-dimensional point cloud; The respectively using the height difference mutation method and the maximum slope value method to identify the three-dimensional feature edges to obtain the edge point cloud of the feature center region includes the following steps: Take the feature center as the origin and the point cloud arrangement direction as the x-axis to establish a plane rectangular coordinate system in the cross-section of the feature three-dimensional point cloud; In the plane rectangular coordinate system, use the height difference mutation method to determine the feature edge points on both sides of the feature center; In the plane rectangular coordinate system, use the maximum slope value method to determine the feature edge points on both sides of the feature center; Using the height difference mutation method to determine the feature edge points on one side of the feature center includes; Take a part of the total number of point clouds near the feature edge points on one side of the feature center as the starting point cloud data, and calculate the height difference between two adjacent point clouds in the starting point cloud data respectively, to obtain the set P = {|d1|, |d2|,..., |d m |}, calculate the average value of |d1|, |d2|,..., |d m | and the standard deviation is σ; Take another part of the point cloud in the total number of nearby point clouds of the feature edge points on one side of the feature center as the point cloud data to be determined, and calculate the height difference |d| between each point in the point cloud data to be determined and the next adjacent point; Calculate the dispersion degree of each height difference |d| and the average value respectively using the three - standard - deviation rule ; When the height difference |d| between two adjacent points in the point cloud data to be determined satisfies then add the height difference |d| to the set P; When the height difference |d| between a certain point in the point cloud data to be determined and the next adjacent point to this point satisfies then this point is determined as a feature edge point of the feature center; Using the maximum slope value method to determine the feature edge points on one side of the feature center includes the following steps: Take the feature center as the origin and the point cloud arrangement direction as the positive x-axis to establish a plane rectangular coordinate system in the cross-section of the feature three-dimensional point cloud; Calculate the slope of the straight line formed by each point cloud in the nearby point clouds of the feature edge points on one side of the feature center and the feature center point respectively; Select the point with the largest absolute value of the slope of the straight line formed with the feature center point as the feature edge point of the feature center; The calculating the gap feature of the seam from the edge point cloud of the feature center region obtained according to the maximum slope value method includes: Suppose the feature edge points on both sides of the feature center obtained according to the maximum slope value method are (x1, y1) and (x2, y2) respectively, then the gap d of the seam is: d = |x1 - x2| (1) The calculating the step difference feature of the seam from the edge point cloud of the feature center region obtained according to the height difference mutation method includes: Let the straight-line equation of the feature edge point on one side of the feature center obtained according to the height difference mutation method and the straight line L1 fitted by the nearby point cloud on one side of the feature center be: A1x + B1y + C1 = 0 (2) Let the straight-line equation of the feature edge point on the other side of the feature center obtained according to the height difference mutation method and the straight line L1 fitted by the nearby point cloud on the other side of the feature center be: A2x + B2y + C2 = 0 (3) Then the step difference h of the seam is: where A1, B1, and C1 are the coefficients of the straight-line equation L1; A2, B2, and C2 are the coefficients of the straight-line equation L2; (x i , y i ) is a point on the straight line L1; (x k , y k ) is a point on the straight line L2; n and m are the numbers of points on the left and right sides in the feature region respectively; is the average distance from the right-side point to the fitted straight line L1; is the average distance from the left point to the fitted straight line L2.

Images