A plate defect detection method based on three-dimensional point cloud

Abstract

The present invention discloses a plate defect detection method based on three-dimensional point cloud, which relates to the field of industrial defect detection. By filtering, coordinate conversion, plane fitting and other operations on the three-dimensional point cloud data of the plate collected at the production site, relevant parameters are measured, and after comparing with the set standard value and threshold, it is judged whether there are defects in size and flatness. Compared with the prior art, there is no need to use standard data samples as templates, and there is no need to train a preset neural network model. Defect detection in size and flatness can be achieved only based on the three-dimensional point cloud data collected on site, and the method has the advantages of high efficiency, high precision and low labor.

Description

Technical Field

[0001] The present invention relates to the field of industrial defect detection, and in particular to a plate defect detection method based on three-dimensional point cloud. Background Art

[0002] With the development of 3D machine vision technology, intelligent product defect detection technology based on 3D machine vision will be the future development trend. Sheet metal is a common stamping part with regular shape, which often has quality problems such as deformation and warping. In the actual production process, the size and flatness are usually required to have millimeter-level accuracy. The traditional detection method that only relies on workers' visual inspection cannot meet the accuracy requirements.

[0003] The machine vision-based stamping defect detection method proposed in CN 114187267A uses LOG operators of different scales to perform edge detection on stamping images, obtains multiple stamping edge images of different scales, and detects stamping defects by matching the actual coordinate positions of the edge pixels of the stamping with the standard positions of each standard edge pixel in the template image. The workpiece surface defect detection method based on deep learning proposed in CN 111415329A collects workpiece images under different backgrounds and lighting conditions, constructs a neural network model after preprocessing to obtain feature maps of different layers, and outputs the location information and category of the workpiece surface defects after feature pyramid fusion prediction.

[0004] However, the edge detection method requires template matching with the standard image, and the detection result depends to a certain extent on the image quality of the template and does not have independent detection capabilities. The detection method based on deep learning requires the collection of workpiece image data sets in advance and can only be used to detect actual products after training with a neural network model. When the data of relevant defect images is relatively small, the detection accuracy will have large errors. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the present invention proposes a plate defect detection method based on three-dimensional point cloud. By filtering, coordinate conversion, plane fitting and other operations on the three-dimensional point cloud data of the plate collected at the production site, the relevant parameters are measured, and after comparing with the set standard values ​​and thresholds, it is determined whether there are defects in size and flatness. The method has the advantages of high precision, high efficiency and low labor.

[0006] The technical solution adopted by the present invention is:

[0007] A plate defect detection method based on three-dimensional point cloud comprises the following steps:

[0008] S1: Set the selected area, use the filtering algorithm to filter the 3D point cloud data of the plate, and remove the noise of the data outside the selected area and the data within the selected area;

[0009] S2: using the filtered 3D point cloud data, segmenting it into several parts according to semantics, selecting the plate part according to semantic features, and obtaining the 3D point cloud data of the plate to be tested;

[0010] S3: converting the three-dimensional point cloud data of the plate to be tested into a three-dimensional point cloud matrix, calculating the eigenvalues ​​and eigenvectors of the three-dimensional point cloud matrix, creating a transformation matrix according to the eigenvectors, and performing coordinate transformation on the three-dimensional point cloud data of the plate;

[0011] S4: Dimension defect detection: Use the dimension detection algorithm to calculate the length, width and height of the plate according to the set accuracy, and compare them with the set standard values ​​of the actual length, width and height of the plate. If they exceed the threshold requirements, it is determined that there is a dimension defect;

[0012] S5: extracting the warping detection area; using a cutting algorithm to cut the three-dimensional point cloud data of the plate to obtain the three-dimensional point cloud data of the warping detection area;

[0013] S6: Fitting the plane model equation of the tilt detection area; using a plane fitting algorithm to fit the plane model where the three-dimensional point cloud data of the tilt detection area is located, and obtaining the plane model equation;

[0014] S7: Detection of warping defects; calculate the average distance from each point in the warping detection area to the plane model equation described in S6, and compare the obtained average distance value with the set standard value of the warping degree of the plate. If it exceeds the threshold requirement, it is determined that a warping defect exists.

[0015] The filtering algorithm is to filter the original three-dimensional point cloud data by conditional filtering and radius filtering in sequence, and includes the following steps:

[0016] S1.1: Conditional filtering: set range conditions for the X, Y, and Z coordinates of the 3D point cloud data, filter according to the range conditions, and remove useless points outside the set range;

[0017] S1.2: Radius filtering. For the three-dimensional point cloud data after conditional filtering, arbitrarily select a point in the three-dimensional point cloud data, select a sphere range with it as the center and a set value as the radius, calculate the distance from the points around the center of the sphere to the center of the sphere, and count the number of points falling within the sphere range. When the number is greater than or equal to the set threshold, retain the selected point; when the number is less than the set threshold, remove the selected point, traverse all the points in the three-dimensional point cloud data, and what remains is the three-dimensional point cloud data after noise removal.

[0018] The S3 specifically includes the following steps:

[0019] S3.1: The 3D point cloud data can be converted into an N-row 3-column matrix M according to the X, Y, and Z coordinates of each point. The coordinates of each point correspond to three values ​​in one row of the matrix M. Find the average value of all elements in M. For each value m in the matrix M ij Subtract the mean The zero-mean matrix M′ is obtained, as shown in equation (1) and equation (2):

[0020]

[0021]

[0022] S3.2: Calculate the covariance matrix C, as shown in formula (3):

[0023]

[0024] Where (M′) T represents the transformed rank matrix of the matrix M′;

[0025] S3.3: Establish the system of equations shown in formula (4) and calculate the eigenvalue λ and the corresponding eigenvector v of the covariance matrix C;

[0026]

[0027] Where λ1, λ2, λ3 are the first, second, and third eigenvalues ​​of the matrix C, and v1, v2, and v3 are the eigenvectors corresponding to λ1, λ2, and λ3 respectively;

[0028] S3.4: Set the feature vectors v1, v2, and v3 as the direction vectors of the X, Y, and Z axes in the coordinate system respectively;

[0029] S3.5: Create a transformation matrix based on the eigenvector. The transformation matrix is ​​defined as shown in formula (5):

[0030]

[0031] in, is a rotation matrix, each row of which corresponds to a direction vector on the X, Y, and Z axes; w1, w2, and w3 represent the offsets in the X, Y, and Z axes after the transformation, respectively. The default setting is 0, which means no offset; u represents the scaling factor of the transformation. The default setting is 1, which means no scaling.

[0032] S3.6: The three-dimensional point cloud data of the plate is transformed according to the transformation matrix. After the transformation, the directions of the three coordinate axes are: the X axis is horizontal to the right, the Y axis is vertically upward, and the Z axis is vertically outward.

[0033] The size detection algorithm is specifically as follows: traverse the three-dimensional coordinates of all points in the three-dimensional point cloud data, calculate a minimum rectangular bounding box containing all points and with sides parallel to the coordinate axis according to the range of their coordinate values, calculate the side length of the minimum rectangular bounding box and sort it, and obtain the length, width and height of the three-dimensional point cloud data of the plate.

[0034] The cutting algorithm specifically comprises the following steps:

[0035] S5.1: Obtain the two-dimensional coordinates of the four corner points of the plate on the X-axis and Y-axis, P1 (x1, y1), P2 (x2, y2), P3 (x3, y3), P4 (x4, y4), according to the vertex information of the rectangular bounding box;

[0036] S5.2: According to the actual border width L of the plate, obtain the two-dimensional coordinates of the X-axis and Y-axis of the four corner points of the cutting area P′1(x1+L,y1-L), P′2(x2+L,y2+L), P′3(x3-L,y3+L), P′4(x4-L,y4-L);

[0037] S5.3: A rectangular area is formed by the four points P′1, P′2, P′3, and P′4. Conditional filtering is performed on the three-dimensional point cloud data of the plate in the rectangular area. The three-dimensional point cloud data inside the rectangular area is removed, and the remaining three-dimensional point cloud data is the three-dimensional point cloud data of the warping detection area.

[0038] The plane fitting algorithm specifically includes the following steps:

[0039] S6.1: Randomly select the coordinates of three points Q1 (x1, y1, z1), Q2 (x2, y2, z2), Q3 (x3, y3, z3) in the three-dimensional point cloud data of the warping detection area;

[0040] S6.2: According to the coordinates of the three points Q1, Q2, and Q3, the equation group (6) is established to find the values ​​of the unknown coefficients A, B, C, and D, and the plane model equation is determined to be Ax+By+Cz+D=0:

[0041]

[0042] S6.3: Let (x i ,y i ,z i ) is the coordinate of any point in the warp detection area, and the average distance between all K points in the three-dimensional point cloud data of the warp detection area and the plane model equation obtained in S6.2 is calculated. Among them, the point-to-surface distance d i and the average distance for:

[0043]

[0044]

[0045] S6.4: Repeat S6.2 to S6.3 until the set number of iterations is reached, and then select the point to plane average distance The minimum plane equation model is the plane model equation that is finally fitted.

[0046] Beneficial technical effects

[0047] The present invention proposes a plate defect detection method based on three-dimensional point cloud. Compared with the prior art, it does not require standard data samples as templates, and does not require training of preset neural network models. It can realize defect detection in size and flatness based only on three-dimensional point cloud data collected on-site, and has the advantages of high efficiency, high precision and low labor. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flow chart of a plate defect detection method based on three-dimensional point cloud provided by an embodiment of the present invention;

[0049] Figure 2 Providing original three-dimensional point cloud data to be inspected for defects in the embodiment of the present invention;

[0050] Figure 3 Providing three-dimensional point cloud data of the plate to be inspected for the embodiment of the present invention;

[0051] Figure 4 The present invention provides three-dimensional point cloud data of a plate after coordinate transformation. DETAILED DESCRIPTION

[0052] An implementation mode of the present invention is described in detail below with reference to the accompanying drawings.

[0053] This embodiment provides a plate defect detection method based on three-dimensional point cloud, such as Figure 1 As shown, the following steps are included:

[0054] In this embodiment, the original three-dimensional point cloud data to be inspected for defects is as follows: Figure 2 As shown;

[0055] S1: Using a filtering algorithm to remove useless points and noise in the three-dimensional point cloud data to obtain filtered three-dimensional point cloud data, including the following steps:

[0056] S1.1: Conditional filtering: set range conditions for the X, Y, and Z coordinates of the 3D point cloud data, filter according to the range conditions, and remove useless points outside the set range.

[0057] S1.2: Radius filtering. For the three-dimensional point cloud data after conditional filtering, arbitrarily select a point in the three-dimensional point cloud data, select a sphere range with it as the center and a set value as the radius, calculate the distance from the points around the center of the sphere to the center of the sphere, and count the number of points falling within the sphere range. When the number is greater than or equal to the set threshold, retain the selected point; when the number is less than the set threshold, remove the selected point, traverse all the points in the three-dimensional point cloud data, and what remains is the three-dimensional point cloud data after noise removal.

[0058] S2: Segment the 3D point cloud data into several parts according to semantics, select the plate part according to the semantic information features, and obtain the 3D point cloud data of the plate to be tested, such as Figure 3 As shown;

[0059] S3: Performing PCA coordinate transformation on the three-dimensional point cloud data of the plate to be inspected, including the following steps:

[0060] S3.1: Convert the 3D point cloud data into an N-row, 3-column matrix M based on the X, Y, and Z coordinates of each point. The coordinates of each point correspond to three values ​​in one row of the matrix M. Find the average value of all elements in M For each value m in the matrix M ij Subtract the mean The zero-mean matrix M′ is obtained, as shown in equation (1) and equation (2):

[0061]

[0062]

[0063] S3.2: Calculate the covariance matrix C, as shown in formula (3), where (M′) T represents the transformed rank matrix of the matrix M′;

[0064]

[0065] S3.3: Establish the system of equations shown in formula (4) and calculate the eigenvalue λ and the corresponding eigenvector v of the covariance matrix C;

[0066]

[0067] It is found that λ1, λ2, λ3 are the first, second, and third eigenvalues ​​of the matrix C, and v1, v2, and v3 are the eigenvectors corresponding to λ1, λ2, and λ3 respectively.

[0068] S3.4: Set the feature vectors v1, v2, and v3 as the direction vectors of the X, Y, and Z axes in the coordinate system respectively;

[0069] S3.5: Create a transformation matrix based on the eigenvector. The transformation matrix is ​​defined as shown in formula (5):

[0070]

[0071] in, is a rotation matrix, each row of which corresponds to a direction vector on the X, Y, and Z axes; w1, w2, and w3 represent the offsets in the X, Y, and Z axes after the transformation, and are set to 0 by default to indicate no offset; u represents the scaling factor of the transformation, and is set to 1 by default to indicate no scaling.

[0072] S3.6: The three-dimensional point cloud data of the plate is transformed according to the transformation matrix. After the transformation, the directions of the three coordinate axes are: the X axis is horizontal to the right, the Y axis is vertically upward, and the Z axis is vertically outward. The three-dimensional point cloud data of the plate after the coordinate transformation is as follows: Figure 4 shown.

[0073] S4: traverse the three-dimensional coordinates of all points in the three-dimensional point cloud data, and calculate a minimum rectangular bounding box containing all points and with sides parallel to the coordinate axis according to the range of its coordinate values. Calculate the side length of the minimum rectangular bounding box and sort it to obtain the length, width and height of the three-dimensional point cloud data of the plate. After the length, width and height are obtained, they are compared with the set standard values ​​of the length, width and height of the plate. If they exceed the threshold requirements, it is determined that there is a size defect;

[0074] S5: extracting the warping detection area, including the following steps:

[0075] S5.1: Obtain the two-dimensional coordinates of the four corner points of the plate on the X-axis and Y-axis, P1 (x1, y1), P2 (x2, y2), P3 (x3, y3), P4 (x4, y4), according to the vertex information of the rectangular bounding box;

[0076] S5.2: According to the actual border width L of the plate, obtain the two-dimensional coordinates of the X-axis and Y-axis of the four corner points of the cutting area P′1(x1+L,y1-L), P′2(x2+L,y2+L), P′3(x3-L,y3+L), P′4(x4-L,y4-L);

[0077] S5.3: A rectangular area is formed by the four points P′1, P′2, P′3, and P′4. Conditional filtering is performed on the three-dimensional point cloud data of the plate in the rectangular area, and the three-dimensional point cloud data inside the rectangular area is removed. What remains is the three-dimensional point cloud data of the warping detection area.

[0078] S6: Fitting a plane model of the warping detection area, including the following steps:

[0079] S6.1: Randomly select the coordinates of three points Q1 (x1, y1, z1), Q2 (x2, y2, z2), Q3 (x3, y3, z3) in the three-dimensional point cloud data of the warping detection area;

[0080] S6.2: According to the coordinates of the three points Q1, Q2, and Q3, the equation group (6) is established to find the values ​​of the unknown coefficients A, B, C, and D, and the plane model equation is determined to be Ax+By+Cz+D=0;

[0081]

[0082] S6.3: Let (x i ,y i ,z i ) is the coordinate of any point in the warp detection area, and the average distance between all K points in the three-dimensional point cloud data of the warp detection area and the plane model equation obtained in S6.2 is calculated. Among them, the point-to-surface distance d i and the average distance for:

[0083]

[0084]

[0085] S6.4: Repeat S6.1 to S6.3 until the set number of iterations is reached, and then select the point to plane average distance The smallest plane equation model is the final fitted plane model equation. Set to the minimum average distance between points and surfaces d min .

[0086] S7: From S6.4, we can get the minimum average distance d between points and surfaces. min , d min The warping degree of the plate is compared with the set standard value. If it exceeds the threshold requirement, it is determined that a warping defect exists.

Claims

1. A plate defect detection method based on three-dimensional point cloud, characterized in that: The following steps are involved: S1: Set the selected area, use the filtering algorithm to filter the 3D point cloud data of the plate, and remove the noise of the data outside the selected area and the data within the selected area; S2: using the filtered 3D point cloud data, segmenting it into several parts according to semantics, selecting the plate part according to semantic features, and obtaining the 3D point cloud data of the plate to be tested; S3: converting the three-dimensional point cloud data of the plate to be tested into a three-dimensional point cloud matrix, calculating the eigenvalues ​​and eigenvectors of the three-dimensional point cloud matrix, creating a transformation matrix according to the eigenvectors, and performing coordinate transformation on the three-dimensional point cloud data of the plate; S4: Dimension defect detection: Use the dimension detection algorithm to calculate the length, width and height of the plate according to the set accuracy, and compare them with the set standard values ​​of the actual length, width and height of the plate. If they exceed the threshold requirements, it is determined that there is a dimension defect; S5: extracting the warping detection area; using a cutting algorithm to cut the three-dimensional point cloud data of the plate to obtain the three-dimensional point cloud data of the warping detection area; S6: Fitting the plane model equation of the tilt detection area; using a plane fitting algorithm to fit the plane model where the three-dimensional point cloud data of the tilt detection area is located, and obtaining the plane model equation; S7: detection of raised foot defects; The average distance from each point in the warping detection area to the plane model equation described in S6 is calculated, and the obtained average distance value is compared with the set standard value of the warping degree of the plate. If it exceeds the threshold requirement, it is determined that a warping defect exists.

2. The plate defect detection method based on three-dimensional point cloud according to claim 1, characterized in that: The filtering algorithm is to filter the original three-dimensional point cloud data by conditional filtering and radius filtering in sequence, and includes the following steps: S1.1: Conditional filtering: set range conditions for the X, Y, and Z coordinates of the 3D point cloud data, filter according to the range conditions, and remove useless points outside the set range; S1.2: Radius filtering. For the three-dimensional point cloud data after conditional filtering, arbitrarily select a point in the three-dimensional point cloud data, select a sphere range with it as the center and a set value as the radius, calculate the distance from the points around the center of the sphere to the center of the sphere, and count the number of points falling within the sphere range. When the number is greater than or equal to the set threshold, retain the selected point; when the number is less than the set threshold, remove the selected point, traverse all the points in the three-dimensional point cloud data, and what remains is the three-dimensional point cloud data after noise removal.

3. The plate defect detection method based on three-dimensional point cloud according to claim 1, characterized in that: The S3 specifically includes the following steps: S3.1: The 3D point cloud data can be converted into an N-row 3-column matrix M according to the X, Y, and Z coordinates of each point. The coordinates of each point correspond to three values ​​in one row of the matrix M. Find the average value of all elements in M. For each value m in the matrix M ij Subtract the mean The zero-mean matrix M′ is obtained, as shown in equation (1) and equation (2): S3.2: Calculate the covariance matrix C, as shown in formula (3): Where (M′) T represents the transformed rank matrix of the matrix M′; S3.3: Establish the system of equations shown in formula (4) and calculate the eigenvalue λ and the corresponding eigenvector v of the covariance matrix C; Where λ1, λ2, λ3 are the first, second, and third eigenvalues ​​of the matrix C, and v1, v2, and v3 are the eigenvectors corresponding to λ1, λ2, and λ3 respectively; S3.4: Set the feature vectors v1, v2, and v3 as the direction vectors of the X, Y, and Z axes in the coordinate system respectively; S3.5: Create a transformation matrix based on the eigenvector. The transformation matrix is ​​defined as shown in formula (5): in, is a rotation matrix, each row of which corresponds to a direction vector on the X, Y, and Z coordinate axes; w1, w2, and w3 represent the offsets in the X, Y, and Z axes after the transformation; u represents the scaling factor of the transformation; S3.6: The three-dimensional point cloud data of the plate is transformed according to the transformation matrix. After the transformation, the directions of the three coordinate axes are: the X axis is horizontal to the right, the Y axis is vertically upward, and the Z axis is vertically outward.

4. The plate defect detection method based on three-dimensional point cloud according to claim 3 is characterized in that: The w1, w2, and w3 are set to 0 by default, indicating no offset; the u represents the scaling factor of the transformation, which is set to 1 by default, indicating no scaling.

5. The plate defect detection method based on three-dimensional point cloud according to claim 1, characterized in that: The size detection algorithm is specifically as follows: traverse the three-dimensional coordinates of all points in the three-dimensional point cloud data, calculate a minimum rectangular bounding box containing all points and with sides parallel to the coordinate axis according to the range of their coordinate values, calculate the side length of the minimum rectangular bounding box and sort it, and obtain the length, width and height of the three-dimensional point cloud data of the plate.

6. The plate defect detection method based on three-dimensional point cloud according to claim 1, characterized in that: The cutting algorithm specifically comprises the following steps: S5.1: Obtain the two-dimensional coordinates of the four corner points of the plate on the X-axis and Y-axis, P1 (x1, y1), P2 (x2, y2), P3 (x3, y3), P4 (x4, y4), according to the vertex information of the rectangular bounding box; S5.2: According to the actual border width L of the plate, obtain the two-dimensional coordinates of the X-axis and Y-axis of the four corner points of the cutting area P′1(x1+L,y1-L), P′2(x2+L,y2+L), P′3(x3-L,y3+L), P′4(x4-L,y4-L); S5.3: A rectangular area is formed by the four points P′1, P′2, P′3, and P′4. Conditional filtering is performed on the three-dimensional point cloud data of the plate in the rectangular area. The three-dimensional point cloud data inside the rectangular area is removed, and the remaining three-dimensional point cloud data is the three-dimensional point cloud data of the warping detection area.

7. The plate defect detection method based on three-dimensional point cloud according to claim 1, characterized in that: The plane fitting algorithm specifically includes the following steps: S6.1: Randomly select the coordinates of three points Q1 (x1, y1, z1), Q2 (x2, y2, z2), Q3 (x3, y3, z3) in the three-dimensional point cloud data of the warping detection area; S6.2: According to the coordinates of the three points Q1, Q2, and Q3, the equation group (6) is established to find the values ​​of the unknown coefficients A, B, C, and D, and the plane model equation is determined to be Ax+By+Cz+D=0: S6.3: Let (x i ,y i ,z i ) is the coordinate of any point in the warp detection area, and the average distance between all K points in the three-dimensional point cloud data of the warp detection area and the plane model equation obtained in S6.2 is calculated. Among them, the point-to-surface distance d i and the average distance for: S6.4: Repeat S6.2 to S6.3 until the set number of iterations is reached, and then select the point to plane average distance The minimum plane equation model is the plane model equation that is finally fitted.

Images