A visual inspection method for curved surface patterns of aircraft skin based on deformation analysis

Abstract

The application discloses a kind of based on deformation analysis's aircraft skin curved surface pattern visual inspection method, application equidistance feature mapping algorithm, under the premise that the geodesic distance between points is unchanged after curved surface bending deformation, two-dimensional plane is reduced to curved surface profile point cloud, overcome the error detection error caused by bending deformation and make data comparison more intuitive.By means of the form of small matrix product, the large geodesic distance matrix is approximated, and the idea of sampling and interpolation recovery of the profile point cloud is adopted, which greatly improves the calculation efficiency of the algorithm and reduces the memory usage while ensuring the final calculation accuracy.Based on the extraction of initial contour centerline according to the gray gradient, the sub-pixel accuracy level contour centerline is obtained, and the accurate matching of corresponding points is realized based on the phase information, which overcomes the large number of error matching problems caused by the dependence of traditional passive binocular algorithm on epipolar matching, and compared with the traditional probe, only one projector is added, which is high in realizability and simple in application.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of visual inspection, and more particularly, relates to a visual inspection method for a curved surface pattern of an aircraft skin based on deformation analysis. BACKGROUND

[0002] Large thin-walled parts in aircraft structures are usually machined by chemical milling. The key to the final precision of chemical milling parts is the machining precision of the chemical milling film marking line, which directly affects the correctness of the milling area. The traditional machining precision detection method of the chemical milling film marking line is to place the skin curve in a fixed position after marking, fix the machining curve and the template with a clamp, and manually judge whether the marking is qualified. This method cannot obtain the machining error data to optimize the marking process parameters. The method is inefficient, greatly affected by human factors, and difficult to ensure the accuracy of the machining precision detection.

[0003] Visual inspection technology, as a non-contact measurement method, is an effective method for detecting the machining precision of the chemical milling film marking line. By passive binocular vision reconstruction, the three-dimensional profile of the part processing pattern is reconstructed and compared with the standard template. This method is more intuitive than the template comparison method and can obtain error data. However, the traditional passive binocular method cannot accurately extract the center line of the measured processing pattern profile and complete the corresponding point matching. Since large skin curves are prone to bending deformation, the skin curve needs to be placed in a fixed position and fixed with a clamp during measurement, which affects the detection efficiency, and the curve still inevitably deforms at this time, introducing a large error for precision detection.

[0004] Therefore, there is an urgent need for a visual inspection method for the surface processing pattern of a skin curve, which can overcome the influence of curve deformation without the need for a clamp, improve the efficiency and accuracy of machining precision detection. SUMMARY

[0005] In view of the above defects or improvement needs of the prior art, the present application provides a visual inspection method for a curved surface pattern of an aircraft skin based on deformation analysis, thereby solving the technical problem that the detection precision and efficiency of the existing detection method need to be improved.

[0006] To achieve the above-mentioned purpose, according to the first aspect of the present application, a visual inspection method for a curved surface pattern of an aircraft skin based on deformation analysis is provided, comprising:

[0007] S1, according to the machining path of the skin curve, obtaining the standard point cloud of each machining pattern profile on the surface of the skin curve ;

[0008] S2, the left and right cameras simultaneously acquire uniform illumination maps and raster maps of each processing pattern. Sub-pixel precision contours of each processing pattern are extracted from the uniform illumination map of the left camera. Based on the phase information of the pixels on the sub-pixel precision contours in the absolute phase map of the left camera, the pixels are matched with the pixels in the absolute phase map of the right camera. The resulting matching point pairs are then used for 3D reconstruction to obtain the measurement point cloud of each processing pattern contour. This point cloud is then transformed to obtain the measurement point cloud of each processing pattern contour in the same coordinate system. The absolute phase map is calculated based on the raster map.

[0009] S3, respectively will be applied to and Rasterized sampling and The first operation is performed on the first point cloud to be processed to obtain a gridded standard point cloud. and measurement point cloud The first operation includes: constructing a triangular mesh based on the first point cloud to be processed and the first point cloud to be processed after homogenization, to obtain a meshed point cloud;

[0010] S4, respectively and The second operation is performed on the second point cloud to be processed to obtain standard planar point clouds of each processed pattern contour. Z b and measuring planar point clouds Z c The second operation includes: based on the geodesic distance matrix of the gridded point cloud, using an equidistant feature mapping algorithm to reduce the dimension of the second point cloud to a two-dimensional plane, and then performing interpolation processing to obtain a planar point cloud;

[0011] S5, Z c and Z b After registration and removal of incorrect matches, Z c Search and Z b Points in z bi The nearest point z ci The distance between the two is taken as the point. z bi Machining errors.

[0012] According to a second aspect of the present invention, a visual inspection device for aircraft skin surface patterns based on deformation analysis is provided, comprising:

[0013] The standard point cloud acquisition module is used to acquire standard point clouds of the contours of each processing pattern on the surface of the skin surface based on the processing path of the skin surface. ;

[0014] The measurement point cloud acquisition module is used to enable the left and right cameras to simultaneously acquire uniform illumination maps and raster maps of each processing pattern. It extracts the sub-pixel precision contours of each processing pattern from the uniform illumination map of the left camera. Based on the phase information of the pixels on the sub-pixel precision contours in the absolute phase map of the left camera, it matches the pixels with the pixels in the absolute phase map of the right camera to obtain matching point pairs. This matching point pair is then used for 3D reconstruction to obtain the measurement point cloud of each processing pattern contour, and coordinate transformation is performed to obtain the measurement point cloud of each processing pattern contour in the same coordinate system. The absolute phase map is calculated based on the raster map.

[0015] The first processing module is used to process the data respectively. and Rasterized sampling and The first operation is performed on the first point cloud to be processed to obtain a gridded standard point cloud. and measurement point cloud The first operation includes: constructing a triangular mesh based on the first point cloud to be processed and the first point cloud to be processed after homogenization, to obtain a meshed point cloud;

[0016] The second processing module is used to respectively process... and The second operation is performed on the second point cloud to be processed to obtain standard planar point clouds of each processed pattern contour. Z b and measuring planar point clouds Z c The second operation includes: based on the geodesic distance matrix of the gridded point cloud, using an equidistant feature mapping algorithm to reduce the dimension of the second point cloud to be processed to a two-dimensional plane to obtain a planar point cloud, and then performing interpolation processing;

[0017] The detection module is used to... Z c and Z b After registration and removal of incorrect matches, Z c Search and Z b Points in z bi The nearest point z ci The distance between the two is taken as the point. z bi Machining errors.

[0018] According to a third aspect of the present application, there is provided a visual inspection system for curved surface pattern of aircraft skin based on deformation analysis, comprising: a computer readable storage medium and a processor;

[0019] The computer readable storage medium is used to store executable instructions;

[0020] The processor is used to read the executable instructions stored in the computer readable storage medium, and execute the method according to the first aspect.

[0021] Overall, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0022] 1. The visual inspection method for curved surface pattern of aircraft skin based on deformation analysis provided by the present application applies an equidistance feature mapping algorithm to reduce the dimension of the curved surface contour point cloud to a two-dimensional plane under the premise that the geodesic distance between points after bending deformation is unchanged, thereby overcoming the error detection error caused by bending deformation and making data comparison more intuitive.

[0023] 2. The visual inspection method for curved surface pattern of aircraft skin based on deformation analysis provided by the present application approximates a large geodesic distance matrix in the form of a small matrix product, and adopts the idea of sampling and then interpolating and restoring the contour point cloud, thereby greatly improving the calculation efficiency of the algorithm and reducing the memory usage under the condition of ensuring the final calculation accuracy.

[0024] 3. The visual inspection method for curved surface pattern of aircraft skin based on deformation analysis provided by the present application extracts the initial contour center line based on the gray gradient, acquires the contour center line at the sub-pixel precision level based on the Steger algorithm, and realizes accurate matching of corresponding points based on phase information, thereby overcoming the problem of a large number of false matches caused by the dependence of traditional passive binocular algorithm on epipolar matching, and achieving high realizability and simple application compared with the traditional probe which only adds one projector.

[0025] 4. The visual inspection method for curved surface pattern of aircraft skin based on deformation analysis provided by the present application reconstructs the skin surface where the contour is located by using the spline interpolation method, obtains the uniformized curved surface point cloud by the interpolation method, constructs a complete curved surface triangular mesh, and solves the key problem of difficult accurate calculation of the geodesic distance between any two points in the contour by applying the MMP triangular mesh geodesic line algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 The visual inspection method for curved surface pattern of aircraft skin based on deformation analysis provided by the present application is shown in the flowchart;

[0027] Figure 2The (a) and (b) are a gray scale image and a raster image acquired at a measurement viewpoint;

[0028] Figure 3 The grid model of the profile points;

[0029] Figure 4 The algorithm flowchart for the curved profile point cloud to the plane profile;

[0030] Figure 5 The schematic diagram of the curved profile point cloud and the corresponding plane profile point cloud. DETAILED DESCRIPTION

[0031] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not used to limit the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0032] The embodiment of the present application provides a kind of based on deformation analysis's aircraft skin curved surface pattern visual inspection method, as shown in Figure 1 It includes:

[0033] S1, according to the machining path of skin surface, the standard point cloud of each machining pattern profile of the skin surface is obtained .

[0034] Specifically, according to the profile sampling of aircraft skin machining path, the contour standard point cloud model of laser engraving pattern (i.e. machining pattern) is obtained.

[0035] Read the machining file to extract the machining path, and obtain the coordinates of each machining point. Since the distribution of machining points is relatively sparse and the distance between two points is too large, the accuracy of subsequent registration is affected, so contour sampling is required to obtain complete point contour point cloud model. Put the machining point coordinates into the contour array, combine the contour shape constraint, traverse the contour array, when the machining point distance exceeds the set threshold and does not cross two contours, sampling is carried out, and a point is inserted into the contour array, otherwise no operation is performed, to obtain the complete contour standard point cloud model.

[0036] S2, left and right cameras synchronously acquire the uniform illumination image and the raster image of each machining pattern, and the sub-pixel accuracy contour of each machining pattern is extracted from the uniform illumination image of the left camera; according to the phase information of the pixel points on the sub-pixel accuracy contour in the absolute phase image of the left camera, the matching point pairs obtained by matching the pixel points with the pixel points in the absolute phase image of the right camera are matched to obtain the measurement point cloud of each machining pattern profile, and the measurement point cloud of each machining pattern profile in the same coordinate system is obtained by coordinate conversion ; wherein the absolute phase map is calculated from the grating pattern.

[0037] Preferably, the sub-pixel accuracy contour of each processing pattern is extracted from the uniform illumination map of the left camera, including:

[0038] The initial contour of each processing pattern is extracted from the uniform illumination map of the left camera, and the Steger centerline extraction algorithm or the gray gravity center method is used to process the initial contour to obtain the sub-pixel accuracy contour of each processing pattern.

[0039] Specifically, the skin surface gray scale map and the grating pattern are collected, as shown in (a) and (b) in Figure 2 , high-precision stereo matching and three-dimensional reconstruction are realized to obtain single contour point cloud data of each measurement viewpoint, and the measurement point cloud of the laser engraving pattern is spliced;

[0040] Wherein, the specific process of obtaining the measurement point cloud of the laser engraving pattern is: based on the skin surface contour point cloud measured by the line structured light, the partitioned viewpoint planning is carried out, based on the planning path, the machine arm carrying the camera and the projector is used to collect the uniform illumination gray scale map and the grating gray scale map of the corresponding contour under each viewpoint; after extracting the initial target contour, the Steger centerline extraction algorithm is used to obtain the sub-pixel accuracy contour, and the corresponding point matching is carried out combined with the phase information, the high-precision three-dimensional reconstruction of the measured contour is realized, and the measurement point cloud of the single contour is obtained; using the multi-camera pose measurement technology based on visual tracking, the measurement point cloud of the laser engraving pattern is obtained by accurately splicing the multiple measurement data of the measuring head, including:

[0041] S21, place the skin surface on the measurement plane, based on the contour point cloud measured by the line structured light, carry out partitioned viewpoint planning; based on the planning path, the machine arm carrying the measuring head moves to the position of the corresponding planning viewpoint in turn, projects the grating, and the left and right cameras acquire the uniform illumination gray scale map and the grating gray scale map of the corresponding contour under the viewpoint.

[0042] S22, carry out distortion correction and epipolar correction on the collected uniform illumination gray scale map and grating gray scale map. Based on the gray scale gradient segmentation and morphological processing, the initial target contour is extracted from the uniform illumination gray scale map of the left camera, the Steger centerline extraction algorithm is used to process the initial target contour, the sub-pixel accuracy contour is obtained, the corresponding point matching is carried out combined with the phase information, and the coordinates of the contour sub-pixel points i a (i.e. the pixel points on the sub-pixel accuracy contour) x , y ), the absolute phase value on the absolute phase map of the left camera is calculated i a ; then based on the absolute phase value, the absolute phase value on the first xMatch rows (where the x-axis points downwards and the y-axis points to the right) to find the match. i a Corresponding matching point i b ( i a and i b (The absolute phase values ​​of the two cameras are equal) to form matching point pairs, achieving high-precision 3D reconstruction of the measured contour and obtaining the measurement point cloud of each laser-etched pattern on the skin surface. The absolute phase maps of the left and right cameras are calculated based on the rectified grating grayscale maps of the left and right cameras.

[0043] S23 transforms the measurement point cloud of each laser-etched pattern to the same coordinate system.

[0044] The marker array (i.e., the rigid frame with the markers pasted on) is fixed to the probe. The coordinate system of the marker array is obtained through the scanning system. Then, the markers are tracked by a multi-camera measurement system (including multiple monocular cameras). Real-time pose measurement of the probe over a large range is achieved through the coupling of multiple monocular cameras. Finally, the probe measurement data is stitched together into the coordinate system of the multi-camera system.

[0045] S3, respectively will be applied to and Rasterized sampling and The first operation is performed on the first point cloud to be processed to obtain a gridded standard point cloud. and measurement point cloud The first operation includes: constructing a triangular mesh based on the first point cloud to be processed and the first point cloud to be processed after homogenization, to obtain a meshed point cloud.

[0046] Preferably, the homogenization process includes:

[0047] The surface containing the first point cloud to be processed is reconstructed using thin plate spline interpolation or polynomial interpolation, and a uniform point cloud (i.e., the first point cloud to be processed after homogenization) is obtained by interpolation on the surface.

[0048] Preferably, the step of constructing a triangular mesh based on the first point cloud to be processed and the homogenized point cloud to obtain a meshed point cloud includes:

[0049] Based on the first point cloud to be processed and the first point cloud to be processed after homogenization, the Bowyer-Watson algorithm is used to construct a triangular mesh to obtain a meshed point cloud.

[0050] Specifically, for the standard point cloud and the measured point cloud respectively, the surface is reconstructed and the homogenized point cloud data of the surface is obtained. The homogenized point cloud of the surface and the contour point cloud are combined to form a new point cloud and then meshed to obtain the meshed standard point cloud data and the measured point cloud.

[0051] The specific process for constructing meshed point cloud data is as follows: For both the standard contour point cloud and the measured contour point cloud, the following operations are performed: Surface fitting is performed based on the contour point cloud data. Discrete points are obtained on the surface through uniform interpolation, resulting in a homogenized point cloud of the surface. The contour point cloud is then uniformly sampled at equal intervals to obtain a sparse sampled point cloud. The contour sampled point cloud and the homogenized point cloud of the surface are combined to form a new point cloud. The Bowyer-Watson algorithm is used to triangulate the point cloud into a mesh, resulting in the following: Figure 3 The gridded point cloud data shown includes:

[0052] S31, specifically for standard contour point clouds and measuring contour point clouds The surface containing the contour is reconstructed using thin-plate spline interpolation or polynomial interpolation, and then uniform discrete point cloud data is obtained by interpolation on the surface. .

[0053] S32, to reduce subsequent computation and save memory space, standard point clouds were processed respectively. and measuring contour point clouds Raster sampling is performed to obtain a sparser contour point cloud. and .

[0054] S33, P c `and The point cloud was constructed using the Bowyer-Watson algorithm to create a triangular mesh, thus obtaining the meshed measurement contour point cloud data. Similarly, and The point cloud was constructed using the Bowyer-Watson algorithm to create a triangular mesh, thus obtaining a gridded standard contour point cloud data. .

[0055] S4, respectively and P c The second operation is performed on the second point cloud to be processed to obtain standard planar point clouds of each processed pattern contour. Z b and measuring planar point clouds Z c The second operation includes: based on the geodesic distance matrix of the gridded point cloud, using an equidistant feature mapping algorithm to reduce the dimension of the second point cloud to a two-dimensional plane and then performing interpolation processing to obtain a planar point cloud.

[0056] Preferably, the geodesic distance matrix wherein M is m a geodesic distance matrix of the m sampling points to each point in the second to-be-processed point cloud, with a size of n × m , n wherein N is the number of the second to-be-processed point cloud, the m sampling points are obtained by farthest point sampling on the grid point cloud;

[0057] the planar point cloud wherein Q is a Q matrix obtained by QR decomposition on , , J is a matrix with all elements being 1; and are respectively two largest eigenvalues and corresponding eigenvectors obtained by eigenvalue decomposition on , and R is an R matrix obtained by QR decomposition on , and are coincident parts.

[0058] Specifically, the geodesic distance matrices of the grid standard point cloud and the grid measured point cloud are respectively calculated , the sampled measured contour point cloud P c ` and the standard contour point cloud are reduced to a two-dimensional plane by using the isometric feature mapping algorithm, to obtain Z b ` and Z c `, and then interpolation processing is respectively performed on Z b ` and Z c ` to obtain the planar point cloud corresponding to Z b , and the planar point cloud corresponding to Z c .

[0059] Further, the geodesic distance between the grid points is calculated, and the specific process of constructing the geodesic distance matrix of the point cloud is as follows: based on the MMP triangular mesh geodesic line algorithm, the geodesic distance between the grid points is calculated, the geodesic distance matrix of the point cloud is constructed, based on the premise that the geodesic distance between the surface points is unchanged before and after the surface bending deformation, the surface point cloud is reduced to a plane by using the isometric feature mapping algorithm, and the geodesic distance between the points is kept unchanged, which is equivalent to unfolding the surface into a plane, as shown in Figure 4 , including (steps S41-S43 are respectively taken as processing objects) , :

[0060] S41, the furthest point sampling is performed on the grid contour point cloud to obtain m sampling points.

[0061] S42, based on the MMP triangular mesh geodesic line algorithm or CH algorithm, the geodesic distance of each source point to all points in m is calculated P` (if the processing object in S41 is , this is ; if the processing object in S41 is , this is ) to obtain m column vectors, which form a matrix n of size m × M , wherein n is the size of the point set P` .

[0062] S43, the known M matrix is used to approximate the large n × n geodesic distance matrix D in the form of small matrix multiplication to improve the calculation efficiency and reduce the memory usage.

[0063] Wherein, the reason for approximating the complete geodesic distance matrix D in the form of small matrix multiplication is that the calculation of the matrix D occupies a huge memory space and requires a large amount of calculation time, which is not feasible. Approximation can greatly improve the calculation efficiency at the expense of the allowable range of accuracy.

[0064] The D columns of the geodesic distance matrix m are selected to form a matrix m of size n× M , so only the geodesic distance of m source points to all points of the contour point cloud needs to be calculated. Considering the distribution characteristics of the contour point cloud and making the m source points as uniform as possible, the furthest point sampling is used to obtain m geodesic source points. For the m source points, the MMP triangular mesh geodesic line algorithm is used to calculate the geodesic distance of each source point to all other points, thereby constructing the matrix M .

[0065] The equidistance feature mapping algorithm is used to reduce the dimension of the curved surface point cloud to a plane, which specifically includes:

[0066] For the contour point set on the grid surface, the geodesic distance between any two points and the geodesic distance between two points , which represents the length of the shortest path between two points along the mesh surface, thereby constructing n n a geodesic distance matrix D where .

[0067] Specifically, approximating a large n n geodesic distance matrix D by means of a small matrix product form includes:

[0068] Let the approximated geodesic distance matrix satisfy the following objective function

[0069]

[0070] Based on the known part of the matrix D , the matrix M is obtained, and the matrix H is introduced, which functions to nullify the part of a matrix other than the first row. The objective function becomes T m

[0071]

[0072] The part of coinciding with is denoted as , and the objective function becomes

[0073]

[0074] The solution of the objective function is To reduce the influence of the smaller eigenvalues of the matrix , the matrix is subjected to eigenvalue decomposition, and the largest eigenvalues and the corresponding eigenvectors are selected, i.e.

[0075]

[0076] Thus, the large matrix D is approximated to the product form of a small matrix

[0077]

[0078] Specifically, the isometric feature mapping algorithm is used to reduce the dimensionality of the surface point cloud to a plane, specifically including:

[0079] For the reduced dimensionality plane point set , since the geodesic distance does not change, the shortest path between two points​​​​ and The Euclidean distance between them is ,Right now:

[0080]

[0081] To facilitate subsequent calculations, a centered matrix is ​​used. C Point set X To process,

[0082]

[0083] This moves the center of the point set to the origin, resulting in the point set. Z ,but .in

[0084]

[0085] in J It is a matrix in which all elements are 1.

[0086] Point set Z Convert to matrix form Z Calculate the inner product matrix

[0087]

[0088] Replacing the geodesic distance matrix with a product of smaller matrices, we have:

[0089]

[0090] To avoid directly performing eigenvalue decomposition on an n×n matrix, which reduces computational efficiency, the matrix... CM First QR break down:

[0091]

[0092] The inner product matrix becomes:

[0093]

[0094] right Perform eigenvalue decomposition:

[0095]

[0096] By reducing the dimensionality of point cloud data to a two-dimensional planar space, the two largest eigenvalues ​​and their corresponding eigenvectors are selected to form... ,but:

[0097]

[0098] Therefore, we obtain

[0099]

[0100] Thus, the curved surface contour point cloud and the corresponding plane point cloud data Z' are obtained, i.e., P c `and P b `perform the above operation to obtain Z b `and Z c `.

[0101] The plane point cloud data Z b `and Z c `are processed by using an interpolation method respectively to obtain the complete standard point cloud and the corresponding plane point cloud of the measurement point cloud Z b and Z c , which are used for point cloud registration and data comparison for evaluating the machining error of the contour. Z b and Z c .

[0102] Preferably, the interpolation processing includes:

[0103] finding , , in the first to-be-processed point cloud; wherein , , are the three points closest to , and is the point in the first to-be-processed point cloud before the gridding sampling;

[0104] let , solve the difference coefficient , , , , and interpolate the point in Z according to ; wherein , , are the points in the plane point cloud Z` closest to the point obtained by reducing the dimension of the second to-be-processed point cloud to a two-dimensional plane.

[0105] Specifically, the interpolation processing is performed on Z c `to obtain Zc For example, obtaining the planar point cloud corresponding to the complete contour points using interpolation specifically includes:

[0106] S51, for the complete contour point set before sampling P c Hit a point KdTree nearest neighbor search is used to find its location in the point set. P` The three closest points , , .

[0107] S52, Represented as a linear combination of three points, i.e. According to each point ( , , )of x , y , z Coordinates, list the corresponding system of equations, and solve for the interpolation coefficients for each point. , , .

[0108] Furthermore, = ,in .

[0109] S53, for the complete contour point set before sampling The corresponding set of plane mapping points Hit a point Interpolation coefficients can be used , , Represented as , , Corresponding plane points , , A linear combination. Traversal calculation yields the point set. Corresponding planar contour points The calculation formula is: That is, according to right Z c Points in ` Interpolation is performed to obtain Z c .

[0110] S5, Z c and Z bAfter registration and removal of incorrect matches, Z c Search and Z b Points in z bi The nearest point z ci The distance between the two is taken as the point. z bi Machining errors.

[0111] Preferably, the will Z b and Z c Registration and removal of incorrect matches, including:

[0112] Using ICP registration algorithm Z b and Z c After registration, if Z c any point in z ci If the distance between a target point and the target point is greater than a threshold, it is considered a mismatch and is removed; where the target point is... Z b Zhongyu z ci The point that is closest to you.

[0113] Specifically, the ICP registration algorithm is used to register the planar point cloud corresponding to the standard contour point cloud (i.e., the planar standard contour point cloud) and the planar point cloud corresponding to the measurement contour point cloud (i.e., the planar measurement contour point cloud), and to eliminate incorrect matches: for each point in the planar measurement contour point cloud, the nearest planar standard contour point in the planar standard contour point cloud is found. If the distance between the two exceeds a threshold, the planar measurement contour point is eliminated.

[0114] like Figure 5 As shown, for each point in the planar measurement contour point cloud, the nearest planar standard contour point in the planar standard contour point cloud is found, and the distance between the two is taken as the processing error value of that planar standard contour point. Data comparison is performed to evaluate the processing error of the contour.

[0115] The method provided by the application carries out profile sampling according to the machining path of the aircraft skin curved surface, to obtain a standard point cloud model of the profile; the profile point cloud of the skin curved surface measured by the line structured light is divided into sub-zones for view point planning, the surface gray scale map and the grating map of the skin curved surface are collected by the camera and the projector carried by the robot arm based on the view point path, high-precision stereo matching and three-dimensional reconstruction are realized, and single profile point cloud data of each measurement view point is obtained; the multi-camera pose measurement technology based on visual tracking is used to realize accurate splicing of the measurement data of the measuring head for multiple times, and the measurement point cloud of the laser engraving pattern is obtained; the curved surface is reconstructed and the uniform point cloud data of the curved surface is obtained for the standard point cloud and the measurement point cloud, the profile point cloud is sampled, and a new point cloud is formed and gridded based on the profile point cloud, the curved surface point cloud and the profile point cloud; the geodesic distance matrix of the standard point cloud and the measurement point cloud after sampling is calculated based on the MMP triangular mesh geodesic line algorithm, the profile point cloud data after sampling on the curved surface is reduced to a two-dimensional plane by using the equal distance feature mapping algorithm while keeping the geodesic distance unchanged; the plane point cloud corresponding to the standard point cloud and the measurement point cloud is obtained by using the interpolation method, and finally, the ICP registration algorithm is used to register the plane point cloud corresponding to the standard point cloud and the measurement point cloud, data comparison is performed, and the machining error of the profile is evaluated. The application can overcome the influence of the deformation of the skin curved surface, and realize fixture-free, automatic and high-precision detection of the laser engraving machining precision of the aircraft skin curved surface.

[0116] The aircraft skin curved surface pattern visual detection device based on deformation analysis provided by the application is described below, and the aircraft skin curved surface pattern visual detection device based on deformation analysis described below can be correspondingly referred to the aircraft skin curved surface pattern visual detection method based on deformation analysis described above.

[0117] The application provides an aircraft skin curved surface pattern visual detection device based on deformation analysis, comprising:

[0118] The standard point cloud acquisition module is configured to acquire standard point clouds of each machining pattern profile of the skin curved surface according to a machining path of the skin curved surface.

[0119] The measurement point cloud acquisition module is configured to synchronously acquire uniform illumination maps and grating maps of each machining pattern by using the left and right cameras, extract sub-pixel accuracy profiles of each machining pattern from the uniform illumination map of the left camera, perform three-dimensional reconstruction on matching point pairs obtained by matching pixel points on the sub-pixel accuracy profiles with pixel points in the absolute phase map of the right camera according to phase information of the pixel points in the absolute phase map of the left camera, and perform coordinate conversion on the measurement point clouds of each machining pattern profile to obtain the measurement point clouds of each machining pattern profile in the same coordinate system.

[0120] The first processing module is configured to respectively​​ and The first operation is performed on the first point cloud to be processed to obtain the meshed standard point cloud of each processing pattern outline. and measurement point cloud The first operation includes: homogenizing the first point cloud to be processed to obtain a uniform point cloud, and constructing a triangular mesh based on the first point cloud to be processed and the uniform point cloud to obtain a meshed point cloud.

[0121] The second processing module is used to process the data separately. and The second operation is performed on the second point cloud to be processed to obtain standard planar point clouds of each processed pattern contour. Z b and measuring planar point clouds Z c The second operation includes: based on the geodesic distance matrix of the gridded point cloud, using an equidistant feature mapping algorithm to reduce the dimension of the second point cloud to a two-dimensional plane, and then performing interpolation processing to obtain a planar point cloud;

[0122] The detection module is used to... Z c and Z b After registration and removal of incorrect matches, Z c Search and Z b Points in z bi The nearest point z ci The distance between the two is taken as the point. z bi Machining errors.

[0123] This invention provides a visual inspection system for aircraft skin surface patterns based on deformation analysis, comprising: a computer-readable storage medium and a processor;

[0124] The computer-readable storage medium is used to store executable instructions;

[0125] The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect.

[0126] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A visual inspection method for curved surface patterns of an aircraft skin based on morphing analysis, characterized in that, The interpolation processing comprises: S1, obtaining a standard point cloud P of each machining pattern contour of the skin surface according to the machining path of the skin surface b ; S2, the left and right cameras synchronously acquire the uniform illumination map and the grating map of each processing pattern, and extract the sub-pixel accuracy contour of each processing pattern from the uniform illumination map of the left camera; according to the phase information of the pixel points on the sub-pixel accuracy contour in the absolute phase map of the left camera, the pixel points in the absolute phase map of the right camera are matched to obtain a matching point pair, three-dimensional reconstruction is performed on the matching point pair to obtain a measurement point cloud of each processing pattern contour, and coordinate conversion is performed on the measurement point cloud to obtain a measurement point cloud P of each processing pattern contour in the same coordinate system c ; wherein, the absolute phase map is calculated according to the grating map; S3, respectively, grid sampling P b and P c to obtain P b `and P c `as the first to-be-processed point cloud, performing the first operation to obtain the grid standard point cloud P bw and the measurement point cloud P cw ; the first operation includes: constructing a triangular mesh based on the first to-be-processed point cloud and the first to-be-processed point cloud after the uniformization processing to obtain the grid point cloud; S4, respectively, P b `and P c `as the second to-be-processed point cloud to obtain the standard plane point cloud Z b and the measurement plane point cloud Z c of each machining pattern contour; the second operation comprises: according to the geodesic distance matrix of the gridded point cloud, using an equal distance feature mapping algorithm to reduce the second to-be-processed point cloud to a two-dimensional plane and then performing interpolation processing to obtain a plane point cloud; S5, Z c and Z b After registration and removal of incorrect matches, in Z c Searching for Z b point z in bi The nearest point z ci The distance between the two is taken as the point z. bi Machining errors.

2. The method of claim 1, wherein, The geodesic distance matrix Wherein M is a geodesic distance matrix of m sampling points to each point in the second to-be-processed point cloud, calculated based on an MMP triangular mesh geodesic line algorithm, with a size of n*m, n is the number of the second to-be-processed point cloud, and the m sampling points are obtained by farthest point sampling on the mesh point cloud. The planar point cloud wherein Q is a Q matrix obtained by QR decomposition of CM, J is a matrix with all elements being 1; Λ2 and V2 are respectively two largest eigenvalues and corresponding eigenvectors obtained by eigenvalue decomposition of CM, R is an R matrix obtained by QR decomposition of CM, M s is a part of M that coincides with M T .

3. The method of claim 2, wherein, The homogenization processing comprises: Finding p in the first to-be-processed point cloud i1 , p i2 , p i3 ; wherein p i1 , p i2 , p i3 are three points closest to p i , and p i is a point in the first to-be-processed point cloud before gridding sampling Let p i = α i p i1 + β i p i2 + γ i p i3 , solve the difference coefficient α i , β i , γ i of p i , according to z i = α i z i1 + β i z i2 + γ i z i2 , interpolate the points z i in Z; wherein z i1 , z i2 , z i3 are the points in the plane point cloud Z` obtained by reducing the dimension of the second to-be-processed point cloud to a two-dimensional plane, which are the closest to the point z i .

4. The method of claim 1, wherein, The first point cloud is reconstructed on a curved surface by using a thin plate spline interpolation method or a polynomial interpolation method, and the homogenized point cloud is interpolated on the curved surface. The sub-pixel accuracy contour of each machining pattern is extracted from the homogenized light map of the left camera, comprising:

5. The method of claim 1, wherein, The initial contour of each machining pattern is extracted from the homogenized light map of the left camera, and the Steger centerline extraction algorithm or the gray gravity center method is used to process the initial contour to obtain the sub-pixel accuracy contour of each machining pattern. The triangular mesh is constructed based on the first point cloud and the first point cloud after the homogenization processing, and the meshed point cloud is obtained, comprising:

6. The method of claim 5, wherein, The triangular mesh is constructed based on the first point cloud and the first point cloud after the homogenization processing by using the Bowyer-Watson algorithm, and the meshed point cloud is obtained. The computer readable storage medium and the processor; 7. The method of claim 1, wherein, The Z b and Z c performing registration and rejecting false matches, including: ICP registration algorithm is used to register Z b and Z c After registration, if the distance between any point z c ` in Z ci ` and its target point is greater than a threshold value, it is considered that there is an error matching, and it is eliminated; wherein the target point is the point in Z b closest to z ci `.

8. An apparatus for visual inspection of curved surface patterns of an aircraft skin based on morphing analysis, characterized in that, The computer readable storage medium is used for storing executable instructions; The standard point cloud acquisition module is configured to acquire a standard point cloud P of each machining pattern contour of the skin surface according to a machining path of the skin surface. b ; The measurement point cloud acquisition module is configured to synchronize the left and right cameras to acquire the uniform illumination map and the grating map of each machining pattern, extract the sub-pixel accuracy contour of each machining pattern from the uniform illumination map of the left camera, match the pixel points on the sub-pixel accuracy contour with the pixel points in the absolute phase map of the right camera according to the phase information of the pixel points in the absolute phase map of the left camera to obtain a matching point pair, perform three-dimensional reconstruction on the matching point pair to obtain the measurement point cloud of each machining pattern contour, and perform coordinate conversion on the measurement point cloud to obtain the measurement point cloud P of each machining pattern contour in the same coordinate system. c ; wherein the absolute phase map is calculated according to the grating map. The first processing module is used to process P respectively. b and P c P obtained by rasterization sampling b `and P c As the first point cloud to be processed, perform the first operation to obtain the meshed standard point cloud P. bw and measurement point cloud P cw The first operation includes: constructing a triangular mesh based on the first point cloud to be processed and the first point cloud to be processed after homogenization, to obtain a meshed point cloud; The second processing module is configured to perform a second operation on P b ` and P c ` as a second to-be-processed point cloud to obtain a standard plane point cloud Z b ` and a measurement plane point cloud Z c ` of each machining pattern contour; the second operation comprises: performing interpolation processing after dimension reduction of the second to-be-processed point cloud to a two-dimensional plane by using an equal-distance feature mapping algorithm according to a geodesic distance matrix of the gridded point cloud. a detection module for detecting Z c and Z b After registration and elimination of false matches, in Z c find the point z b in Z bi with the shortest distance to z ci , and take the distance between them as the machining error of the point z bi .

9. A visual inspection system for curved surface patterns of aircraft skin based on morphing analysis, characterized in that, The processor is used for reading the executable instructions stored in the computer readable storage medium, and executing the method according to any one of claims 1-7. ​ ​ ​

Images