A rivet convex-concave amount feature precise extraction method based on point cloud dimension reduction

Abstract

A rivet convex-concave amount feature accurate extraction method based on point cloud dimension reduction comprises the following steps: S1: calculating point cloud surface variation, and performing color mapping on the surface normal variation after mathematical processing; S2: adopting a method combining principal component analysis (PCA) projection technology and two-dimensional gridding to perform point cloud dimension reduction, and generating an image; S3: adopting a relevant image processing algorithm and an image point cloud mapping relationship to perform rivet region point cloud rough segmentation; and S4: based on hierarchical structure fitting, accurately extracting rivet head point cloud and skin point cloud, and calculating the rivet convex-concave amount.The present application can effectively and accurately and efficiently extract the rivet convex-concave amount feature from three-dimensional point cloud.

Description

Technical Field

[0001] The present invention relates to a method for accurately extracting convex and concave features of rivets based on point cloud dimensionality reduction. In particular, a method for accurately extracting convex and concave features of rivets is proposed by using point cloud gridding dimensionality reduction, a method for coarse segmentation of rivet area point clouds based on point cloud and image mapping, and a method for extracting rivets based on hierarchical structure fitting to achieve efficient and accurate feature extraction. The method belongs to the field of computer vision. Background Art

[0002] In aircraft manufacturing, riveting quality is a crucial factor affecting an aircraft's overall aerodynamic performance and fatigue life, making it essential to inspect riveting quality. Rivet convexity is a key indicator in riveting quality inspection. Traditional methods for inspecting rivet convexity rely primarily on visual or tactile inspection, which lacks quantitative data and results in low accuracy, efficiency, and reliability. Given the increasing level of automation and digitization in modern aircraft production, there is an urgent need to research automated and digitized methods for riveting quality inspection. Summary of the Invention

[0003] This paper addresses the low efficiency and accuracy of extracting rivet features and calculating rivet convexity and concavity from three-dimensional point cloud data. By proposing a method for accurately extracting rivet convexity and concavity based on point cloud dimensionality reduction, the paper improves detection efficiency by coarsely segmenting the rivet region through processing an image generated by point cloud dimensionality reduction and two-dimensional meshing. Furthermore, a hierarchical fitting approach accurately segments the rivet head point cloud from the skin point cloud, improving the accuracy of rivet convexity and concavity calculation.

[0004] The technical solution of the present invention is:

[0005] A method for accurately extracting rivet convex and concave features based on point cloud dimensionality reduction. The specific steps are as follows:

[0006] S1: Calculate the surface variation of the point cloud and perform color mapping on the surface normal variation after mathematical processing;

[0007] S2: Use principal component analysis (PCA) projection technology combined with two-dimensional gridding to reduce the dimension of point cloud and generate images;

[0008] S3: Use relevant image processing algorithms and image point cloud mapping relationships to perform rough segmentation of the rivet area point cloud;

[0009] S4: Based on hierarchical structure fitting, the rivet head point cloud and the skin point cloud are accurately extracted, and the rivet convexity and concavity are calculated.

[0010] The calculation of the point cloud surface variation and color mapping of the surface normal variation after mathematical processing include the following steps:

[0011] Step 1: Estimate the surface variation of the point cloud. Surface variation is an indicator that describes the local geometric features of a surface. It is similar to surface curvature and is more suitable for simplifying point sampling surfaces than curvature estimation based on function fitting. It is widely used in fields such as point cloud data simplification. For any point Pi(xi,yi,zi) in a discrete point cloud P, the set of points in its r neighborhood is:

[0012] N r (P i )={P l ,P2,P3,…P i ,…P m} (1)

[0013] Where: P i is a point in the point cloud; N r (P i ) is P i is the center; r is the point set in the spherical area with radius; m is the number of points in the neighborhood of r.

[0014] Calculate the center point of the neighborhood point set:

[0015]

[0016] Where: P i is a point in the point cloud; is the center point of the domain point set; m is the number of points in the neighborhood of r.

[0017] Construct the point cloud covariance matrix and solve it:

[0018]

[0019] Where: C is a symmetric semi-positive definite matrix; λ = {λ0,λ1,λ2} is the eigenvalue of C, and λ0≤λ1≤λ2; is λ j The corresponding eigenvector.

[0020] Define point P i The surface variation in the neighborhood of r is:

[0021]

[0022] Where: σ(P i ) characterizes the degree of flatness of a point in a local area, and its value range is (0 to 1 / 3). The smaller the value, the flatter the local area. When the value is 0, it represents a plane. λ0, λ1, and λ2 are the eigenvalues ​​of C in formula (3).

[0023] Step 2: Calculate the logarithm of the rivet surface variation. Since the curvature of the aircraft skin is very small and the riveted area is also very small, the surface variation in the riveted area is very small and not very distinct. Therefore, a logarithmic operation is performed on the surface variation:

[0024] The step S2 uses a method combining principal component analysis (PCA) projection technology with two-dimensional gridding to perform point cloud dimensionality reduction based on the calculation of the surface variation logarithm, and generates an image, which includes the following steps:

[0025] Step 1: Use principal component analysis (PCA) to determine the projection plane:

[0026] (1) Construct the covariance matrix of the entire point cloud P:

[0027]

[0028] Where: N is the number of point clouds, and the other variables have the same meanings as above. and is the matrix C P The eigenvectors corresponding to the two largest eigenvalues ​​represent the main direction of the point cloud and the direction vector, respectively, so we can and The plane where it is located is used as the point cloud projection plane. The eigenvector corresponding to the minimum eigenvalue λ0 is the plane normal vector.

[0029] (2) Calculate the projected point cloud coordinates:

[0030]

[0031] Where: Q i is the coordinate of the projected point, P i is a point in the original point cloud, is the center point of the point cloud, and U is a column-full rank matrix.

[0032] Step 2: Mesh the plane point cloud and map pixel information:

[0033] (1) For a two-dimensional point cloud Q = {Q i (x,y),i=1,2,...,N}, where N is the number of point clouds. First, calculate the number of grid rows and columns:

[0034]

[0035] Where: step is the grid step size, Row is the number of image pixel rows, Col is the number of columns, Y max 、Y min 、X max 、X minrespectively the maximum, minimum value of X, Y coordinate in point cloud Q, represents the upward rounding.

[0036] Then calculate the row and column number of any point Q i :

[0037]

[0038] In the formula: rowID, colID respectively represent the row and column number of point Q i : represents the downward rounding.

[0039] (2) Pixel information mapping: each pixel point of the two-dimensional image has only one RGB value corresponding to it, but after gridding, there may be the following three cases: only one point exists in a grid; multiple points exist in a grid; no point exists in a grid. When only one point exists in a grid, the RGB value corresponding to the surface variation of the point is taken as the pixel RGB value; when multiple points exist in a grid, the average value of the internal point cloud RGB values is calculated as the pixel RGB value; when no point exists in a grid, an eight-neighborhood average interpolation method is used for calculation, and the average value of the eight neighbor cells of the target cell (pixel, grid) is calculated to fill the empty cell.

[0040] The step S3 will generate the image using relevant image processing algorithms and image point cloud mapping relationship for rivet region point cloud rough segmentation, including the following steps:

[0041] Step 1, image-based rivet circle contour rough extraction:

[0042] (1) Grayscale the RGB image and perform Gaussian filtering to remove noise, and use the Canny algorithm to extract the edge contour.

[0043] (2) Calculate the pixel perimeter of the contour S. To ensure the accuracy of the fitting, we only select the circular contour with an arc greater than π / 4, so we limit the above contour features and retain the contours that meet the conditions.

[0044] (3) Perform Hough circle fitting on the extracted contour to obtain the center and radius of the rivet circle contour and each rivet circle

[0045] Step 2, rivet region point cloud rough segmentation based on point cloud and image mapping: after obtaining the rivet circle contour parameters (center position, radius), inverse map it back to the point cloud coordinate system, construct a ring-shaped circle bounding box, and extract the rivet region point cloud:

[0046] (1) Inverse gridding map the circle parameters in the image coordinate system to the projection plane, if the center of the circle in the image coordinate system is C img (x img , yimg ), with a radius of r img , since the origin of the image coordinate system is generally in the upper left corner, the projection coordinate system is:

[0047]

[0048] Where: C proj (x proj ,y proj ) is the coordinate of the center of the circle in the projection coordinate system, r proj is the radius in the projected coordinate system,

[0049] X min 、Y max , step have the same meaning as above.

[0050] (2) Through inverse projection, C proj (x proj ,y proj ) is mapped to the point cloud coordinate system:

[0051]

[0052] C p (x, y, z) are the coordinates of the center of the circle in the point cloud coordinate system, U R is the right inverse matrix of the projection matrix U mentioned above, U, Consistent with the above meaning, U T is the U transposed matrix.

[0053] (3) When mapping two-dimensional data back to three-dimensional space, information will be lost, so C p It will fall on a straight line passing through the true center of the circle and in the same direction as the normal vector of the projection plane. With this straight line as the axis, concentric annular cylinders are constructed with radii r proj +2mm, r proj / 2. Extract the point cloud inside the annular cylinder as the rivet area point cloud to complete the rivet area

[0054] Coarse segmentation of domain point clouds.

[0055] In step S4, a rivet convex-concave calculation model based on three-dimensional point cloud is designed, and then hierarchical fitting is performed according to the roughly extracted point cloud to obtain three parts, namely, rivet head point cloud, skin plane point cloud, and rivet outline point cloud, so as to accurately calculate the convex-concave amount.

[0056] Step 1: Design a calculation model for the convex and concave values ​​of rivets:

[0057]

[0058] Where: d flush is the convexoconcave amount of the rivet, d maxis the maximum value from the rivet head point cloud to the skin plane, d min It is the minimum value from the rivet head point cloud to the skin plane.

[0059] Step 2: Rivet layered fitting:

[0060] (1) Fit the curvature variation to a Gaussian distribution and use the value of μ+2σ as the threshold to extract the rivet contour point cloud;

[0061] (2) The extracted rivet contour points are fitted into a space circle and the rivet area is divided into two parts, the inner green is

[0062] The rivet head candidate point cloud, the outside is the skin candidate point cloud;

[0063] (3) Using the Ransac plane fitting algorithm, the rivet head candidate point cloud and the skin candidate point cloud are plane-fitted with appropriate bandwidths to obtain the layered rivet head point cloud and skin plane point cloud;

[0064] Step 3: Calculate the distance from the rivet head point cloud to the skin plane, and use the rivet convexity and concavity calculation model proposed in step 1 to calculate the rivet convexity and concavity.

[0065] The beneficial effects of the present invention are:

[0066] The present invention's method for accurately extracting rivet convexity and concavity features first calculates the surface variation of a point cloud and, after mathematical processing, performs color mapping on the surface normal variation. A method then combines principal component analysis (PCA) projection technology with two-dimensional gridding to reduce the point cloud dimensionality and generate an image. A related image processing algorithm is then used to perform a coarse segmentation of the rivet region point cloud. Finally, based on hierarchical structure fitting, the rivet head point cloud and the skin point cloud are accurately extracted, and the rivet convexity and concavity are calculated. This method can effectively address the problem of low accuracy and efficiency in calculating rivet convexity and concavity when dealing with large amounts of point cloud data. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 Graph of point cloud surface variation calculation results.

[0068] Figure 2 Graph of calculation results of the variational logarithm of the point cloud surface.

[0069] Figure 3 Schematic diagram of point cloud meshing.

[0070] Figure 4 Point cloud dimensionality reduction generates RGB images.

[0071] Figure 5 Image-based rough extraction process of rivet circle contour.

[0072] Figure 6Based on image point cloud inverse mapping rivet area point cloud rough segmentation schematic diagram.

[0073] Figure 7 Rivet convex-concave amount calculation model schematic diagram.

[0074] Figure 8 Rivet hierarchical fitting process diagram. DETAILED DESCRIPTION

[0075] The application will be further described in detail below in conjunction with the accompanying drawings and examples.

[0076] As Figures 1-5 shown.

[0077] A rivet convex-concave amount feature precise extraction method based on point cloud dimension reduction: comprising the following steps:

[0078] S1: Calculate the point cloud surface variation, and perform color mapping on the surface normal variation after mathematical processing;

[0079] S2: Use the principal component analysis (PCA) projection technology combined with the two-dimensional gridding method to perform point cloud dimension reduction, and generate an image;

[0080] S3: Use relevant image processing algorithms and image point cloud mapping relationships to perform rivet area point cloud rough segmentation;

[0081] S4: Based on hierarchical structure fitting, the rivet head point cloud and the skin point cloud are precisely extracted, and the rivet convex-concave amount is calculated.

[0082] Referring to Figures 1-2 : S1 calculates the point cloud surface variation, and performs color mapping on the surface normal variation after mathematical processing, comprising the following steps:

[0083] Step 1, as Figure 1 shown, point cloud surface variation estimation, the surface variation as an index to describe the local geometric features of the surface, has similarity with the surface curvature, and is more suitable for simplifying point sampling surfaces compared with curvature estimation based on function fitting, and is widely used in fields such as point cloud data simplification. For any point Pi(xi,yi,zi) in the discrete point cloud P, the point set in the r-neighborhood of Pi is:

[0084] N r (P i )={P l ,P2,P3,…P i ,…P m} (1)

[0085] In the formula: P i is a point in the point cloud; N r (P i ) is the r-neighborhood of P iis the center of the point set; r is the radius of the spherical region; m is the number of points in the r-neighborhood.

[0086] Calculate the center point of the neighborhood point set:

[0087]

[0088] In the formula: P i is a point in the point cloud; is the center point of the point set; m is the number of points in the r-neighborhood.

[0089] Construct the point cloud covariance matrix and solve:

[0090]

[0091] In the formula: C is a symmetric semi-definite matrix; λ = {λ0, λ1, λ2} are the eigenvalues of C, and λ0≤ λ1≤ λ2; is λ j The corresponding eigenvector.

[0092] Define the point P i The surface variation in the r-neighborhood is:

[0093]

[0094] In the formula: σ(P i ) represents the smoothness of the point in the local area, and the value range is (0~1 / 3). The smaller the value, the smoother the local area, and the value of 0 represents a plane; λ0, λ1, λ2 are the eigenvalues of equation (3) C.

[0095] Step 2, as shown in Figure 2 , the rivet surface variation logarithm is calculated. Since the curvature of the aircraft surface skin is very small, the riveting area is also very small, and the riveting area surface variation is very small and not much different, so the surface variation is logarithmically operated:

[0096] Referring to Figures 3-4 : S2 uses principal component analysis (PCA) projection technology combined with two-dimensional gridding method to reduce the dimension of point cloud based on the calculation of surface variation logarithm, and generates an image including the following steps:

[0097] Step 1, determine the projection plane by principal component analysis (PCA) method:

[0098] (1) Construct the covariance matrix of the whole point cloud P:

[0099]

[0100] In the formula: N is the number of whole point cloud, and the remaining variables have the same meaning as above. and is the matrix C P The eigenvectors corresponding to the two largest eigenvalues ​​represent the main direction of the point cloud and the direction vector, respectively, so we can and The plane where it is located is used as the point cloud projection plane. The eigenvector corresponding to the minimum eigenvalue λ0 is the plane normal vector.

[0101] (2) Calculate the projected point cloud coordinates:

[0102]

[0103] Where: Q i is the coordinate of the projected point, P i is a point in the original point cloud, is the center point of the point cloud, and U is a column-full rank matrix.

[0104] Step 2: Mesh the plane point cloud and map pixel information:

[0105] (1) Figure 3 As shown, for a two-dimensional point cloud Q = {Q i (x,y),i=1,2,...,N}, where N is the number of point clouds, first calculate

[0106] Calculate the number of grid rows and columns:

[0107]

[0108] Where: step is the grid step size, Row is the number of image pixel rows, Col is the number of columns, Y max 、Y min 、X max 、X min are the maximum and minimum values ​​of the X and Y coordinates in the point cloud Q, respectively. Indicates rounding up.

[0109] Then calculate any point Q i Row and column number:

[0110]

[0111] Where: rowID, colID represent point Q i The row and column number of the grid, Indicates rounding down.

[0112] (2) Figure 4As shown in the figure, pixel information mapping: Each pixel in a two-dimensional image has only one RGB value corresponding to it. However, after rasterization, there may be three situations: only one point in a grid; multiple points in a grid; or no point in the grid. For the above situations, when there is only one point in the grid, the RGB value corresponding to the surface variation of the point is used as the pixel RGB value; when there are multiple points in the grid, the average of the RGB values ​​of each point cloud within the grid is calculated as the pixel RGB value; when there are no points in the grid, the eight-neighborhood average interpolation method is used to fill the empty cell (pixel, grid) by calculating the average of the eight neighboring cells of the target cell.

[0113] Reference Figures 5-6 :S3 uses the relevant image processing algorithm and image point cloud mapping relationship to perform rough segmentation of the rivet area point cloud on the generated image, including the following steps:

[0114] Step 1: Figure 5 As shown, the rough extraction of rivet circle contour based on image:

[0115] (1) Grayscale the RGB image and perform Gaussian filtering to remove noise, and use the Canny algorithm to extract edge contours.

[0116] (2) Calculate the pixel perimeter of the contour as S. To ensure the accuracy of fitting, we only select circular contours with a radian greater than π / 4, so we restrict the above contour features and retain contours that meet the conditions.

[0117] (3) Perform Hough circle fitting on the extracted contour to obtain the rivet circle contour and the center and radius of each rivet circle

[0118] Step 2: Figure 6 As shown in the figure, the rivet area point cloud is roughly segmented based on point cloud and image mapping: after obtaining the rivet circle contour parameters (center position, radius), it is inversely mapped back to the point cloud coordinate system, and the annular circle bounding box is constructed to extract the rivet area point cloud:

[0119] (1) Inverse rasterize the circle parameters in the image coordinate system and map them to the projection plane. If the center of the circle is C in the image coordinate system, img (x img ,y img ), with a radius of r img , since the origin of the image coordinate system is generally in the upper left corner, the projection coordinate system is:

[0120]

[0121] Where: C proj (x proj ,y proj ) is the coordinate of the center of the circle in the projection coordinate system, r projis the radius in the projected coordinate system, X min 、Y max , step have the same meaning as above.

[0122] (2) Through inverse projection, C proj (x proj ,y proj ) is mapped to the point cloud coordinate system:

[0123]

[0124] Where: C p (x, y, z) are the coordinates of the center of the circle in the point cloud coordinate system, U R is the right inverse matrix of the projection matrix U mentioned above, U, Consistent with the above meaning, U T is the U transposed matrix.

[0125] Mapping two-dimensional data back to three-dimensional space will lose information, so C p It will fall on a straight line passing through the true center of the circle and in the same direction as the normal vector of the projection plane. With this straight line as the axis, concentric annular cylinders are constructed with radii r proj +2mm, r proj / 2. Extract the point cloud inside the annular cylinder as the rivet area point cloud and complete the rough segmentation of the rivet area point cloud.

[0126] Reference Figures 7-8 : In S4, a rivet convexity and concavity calculation model based on three-dimensional point cloud is designed, and then hierarchical fitting is performed according to the roughly extracted point cloud to obtain three parts: rivet head point cloud, skin plane point cloud, and rivet contour point cloud, to accurately calculate the convexity and concavity.

[0127] Step 1: Figure 7 As shown in the figure, the calculation model of rivet convexity and concaveness is designed:

[0128]

[0129] Where: d flush is the convexoconcave amount of the rivet, d max is the maximum value from the rivet head point cloud to the skin plane, d min It is the minimum value from the rivet head point cloud to the skin plane.

[0130] Step 2: Figure 8 As shown, the rivets are layered fitted:

[0131] (4) Fit the curvature variation to a Gaussian distribution and use the value of μ+2σ as the threshold to extract the rivet contour point cloud;

[0132] (5) The extracted rivet contour points are fitted into a space circle and the rivet area is divided into two parts, the inner green is

[0133] The rivet head candidate point cloud, the outside is the skin candidate point cloud;

[0134] (6) Using the Ransac plane fitting algorithm, the rivet head candidate point cloud and the skin candidate point cloud are plane-fitted with appropriate bandwidths to obtain the layered rivet head point cloud and skin plane point cloud;

[0135] Step 3: Calculate the distance from the rivet head point cloud to the skin plane, and use the rivet convexity and concavity calculation model proposed in step 1 to calculate the rivet convexity and concavity.

[0136] The parts not involved in the present invention are the same as the existing technology or can be implemented by using the existing technology.

Claims

1. A method for accurately extracting convex and concave features of rivets based on point cloud dimensionality reduction, characterized in that: The steps of the method for accurately extracting the convex and concave amounts of rivets are as follows: S1: Calculate the surface variation of the point cloud and perform color mapping on the surface normal variation after mathematical processing; S2: Use principal component analysis (PCA) projection technology combined with two-dimensional gridding to reduce the dimension of point cloud and generate images; S3: Use image processing algorithm and image point cloud mapping relationship to perform rough segmentation of rivet area point cloud; S4: Based on hierarchical structure fitting, the rivet head point cloud and the skin point cloud are accurately extracted, and the rivet convexity and concavity are calculated.

2. The method according to claim 1, wherein: Calculating the point cloud surface variation and color mapping the surface normal variation after mathematical processing includes the following steps: Step 1: Estimation of surface variation of point cloud. Surface variation is an indicator that describes the local geometric features of a surface and is similar to surface curvature. For any point Pi(xi,yi,zi) in a discrete point cloud P, the set of points in its r neighborhood is: N r (P i )={P l ,P2,P3,…P i ,…P m } (1) Where: P i is a point in the point cloud; N r (P i ) is P i is the center; r is the point set in the spherical area with radius; m is the number of points in the neighborhood of r; Calculate the center point of the neighborhood point set: Where: P i is a point in the point cloud; is the center point of the domain point set; m is the number of points in the neighborhood of r; Construct the point cloud covariance matrix and solve it: Where: C is a symmetric semi-positive definite matrix; λ = {λ0,λ1,λ2} is the eigenvalue of C, and λ0≤λ1≤λ2; is λ j The corresponding eigenvector; Define point P i The surface variation in the neighborhood of r is: Where: σ(P i ) characterizes the degree of flatness of the point in the local area, and its value range is (0~1 / 3). The smaller the value, the flatter the local area. When the value is 0, it represents a plane. λ0,λ1,λ2 are the eigenvalues ​​of C in formula (3); Step 2: Calculate the logarithm of the rivet surface variation. Since the curvature of the aircraft skin is very small and the riveted area is also very small, the surface variation in the riveted area is very small and not very distinct. Therefore, a logarithmic operation is performed on the surface variation:

3. The method according to claim 1, wherein: The step S2 uses a method combining principal component analysis (PCA) projection technology with two-dimensional gridding to perform point cloud dimensionality reduction on the basis of calculating the surface variation logarithm, and generates an image, which includes the following steps: Step 1: Use principal component analysis (PCA) to determine the projection plane: (1) Construct the covariance matrix of the entire point cloud P: Where: N is the number of the entire point cloud, and the other variables have the same meaning as formula (3); and is the matrix C P The eigenvectors corresponding to the two largest eigenvalues ​​represent the main direction of the point cloud and the direction vector, respectively, so we can and The plane where it is located is used as the point cloud projection plane; the eigenvector corresponding to the smallest eigenvalue λ2 is the plane normal vector; (2) Calculate the projected point cloud coordinates: Where: Q i is the coordinate of the projected point, P i is a point in the original point cloud, is the center point of the point cloud, and U is a column-full rank matrix; Step 2: Mesh the plane point cloud and map pixel information: (1) For a two-dimensional point cloud Q = {Q i (x,y),i=1,2,...,N}, where N is the number of point clouds. First, calculate the number of grid rows and columns: Where: step is the grid step size, Row is the number of image pixel rows, Col is the number of columns, Y max 、Y min 、X max 、X min are the maximum and minimum values ​​of the X and Y coordinates in the point cloud Q, respectively. Indicates rounding up; Then calculate any point Q i Row and column number: Where: rowID, colID represent point Q i The row and column number of the grid, Indicates rounding down; (2) Pixel information mapping: Each pixel in a two-dimensional image has only one RGB value corresponding to it. However, after rasterization, there are three situations: there is only one point in a grid; there are multiple points in a grid; there is no point in the grid; when there is only one point in the grid, the RGB value corresponding to the surface variation of the point is used as the pixel RGB value; when there are multiple points in the grid, the average value of the RGB values ​​of the internal point cloud is calculated as the pixel RGB value. When there is no point in the grid, the eight-neighborhood average interpolation method is used to calculate the target cell. Empty cells are filled by taking the average of the eight neighboring cells of the pixel and raster.

4. The method according to claim 1, wherein: The step S3 uses the image processing algorithm and the image point cloud mapping relationship to perform rough segmentation of the rivet area point cloud on the generated image, including the following steps: Step 1: Image-based rough extraction of rivet circle contour: (1) Grayscale the RGB image, perform Gaussian filtering to remove noise, and use the Canny algorithm to extract edge contours; (2) The pixel perimeter of the contour is calculated as S; to ensure the accuracy of the fitting, a circular contour with a radian greater than π / 4 is selected, so the above contour features are restricted and the contours that meet the conditions are retained; (3) Perform Hough circle fitting on the extracted contour to obtain the rivet circle contour and the center and radius of each rivet circle; Step 2: Rough segmentation of rivet area point cloud based on point cloud and image mapping: After obtaining the rivet circle contour parameters including the center position and radius, inversely map them back to the point cloud coordinate system, construct an annular circle bounding box, and extract the rivet area point cloud; (1) Inversely rasterize the circle parameters in the image coordinate system and map them to the projection plane. If the center of the circle in the image coordinate system is C img (x img ,y img ), with a radius of r img , since the origin of the image coordinate system is generally in the upper left corner, the projection coordinate system is: Where: C proj (x proj ,y proj ) is the coordinate of the center of the circle in the projection coordinate system, r proj is the radius in the projected coordinate system, X min 、Y max , step has the same meaning as above; (2) Through inverse projection, C proj (x proj ,y proj ) is mapped to the point cloud coordinate system: C p (x, y, z) are the coordinates of the center of the circle in the point cloud coordinate system, U R is the right inverse matrix of the projection matrix U in formula (6), U, Consistent with the above meaning, U T is the U transposed matrix; (3) When mapping two-dimensional data back to three-dimensional space, information will be lost, so C p It will fall on a straight line passing through the true center of the circle and in the same direction as the normal vector of the projection plane. With this straight line as the axis, concentric annular cylinders are constructed with radii r proj +2mm, r proj / 2. Extract the point cloud inside the annular cylinder as the rivet area point cloud to complete the rivet area Point cloud coarse segmentation.

5. The method according to claim 1, wherein: In step S4, the rivet head point cloud and the skin point cloud are accurately extracted based on hierarchical structure fitting, and the rivet convexity and concavity are calculated, which includes the following steps: Step 1: Design a calculation model for the convex and concave dimensions of rivets: Where: d flush is the convexoconcave amount of the rivet, d max is the maximum value from the rivet head point cloud to the skin plane, d min It is the minimum value from the rivet head point cloud to the skin plane; Step 2: Rivet layered fitting: (1) Fit the curvature variation to a Gaussian distribution and use the value of μ+2σ as the threshold to extract the rivet contour point cloud; (2) Perform spatial circle fitting on the extracted rivet contour points and divide the rivet area into two parts: the inner part is the rivet head candidate point cloud, and the outer part is the skin candidate point cloud; (3) Using the Ransac plane fitting algorithm, the rivet head candidate point cloud and the skin candidate point cloud are plane-fitted with appropriate bandwidths to obtain the layered rivet head point cloud and skin plane point cloud; Step 3: Calculate the distance from the rivet head point cloud to the skin plane, and use the rivet convexity and concavity calculation model proposed in step 1 to calculate the rivet convexity and concavity.

Images