Three-dimensional defect integrated detection method based on laser point cloud

Abstract

The invention discloses a laser point cloud-based three-dimensional defect integrated detection method, which comprises the following steps: point cloud preprocessing: carrying out pose adjustment on obtained point cloud data to ensure that reference plane data points are positioned on the same horizontal plane; segmenting a target surface point cloud based on an RANSAC algorithm; carrying out point cloud dimensionality reduction to extract target surface section data, fitting a reference straight line based on RANSAC, calculating mean square error and linearity features, and carrying out plane type or bending type identification by using an SVM (Support Vector Machine); mapping a gray level image by means of the height difference between the point cloud and the reference plane, and judging whether a head filling defect exists or not; calculating three characteristics of an average distance, a mean square error and a decision coefficient according to the target surface section data, performing bending classification by using an SVM, and calculating the bending amount; according to the method, the problems of classified identification and quantitative detection of three three-dimensional defects of qualified products, head filling, dead bending and bow bending on a wood ice cream stick and ice cream spoon production line are solved, product quality grading is facilitated, resource waste is reduced, and production efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional defect nondestructive detection, and in particular to a three-dimensional defect integrated detection method based on laser point cloud. Background Art

[0002] Ice cream sticks and ice cream scoops are commonly used products in the dairy industry. They are made of natural wood through a rotary cutting process and are prone to various types of defects. According to the defect characteristics of ice cream sticks, defects can be divided into three categories: surface grayscale defects, two-dimensional shape defects, and three-dimensional defects. Each type of defect also contains many different types of defects. These defects will directly affect food hygiene and food safety, and may even cause personal injury to users, and may also cause automatic production line shutdowns and other failures. In order to ensure safety of use and smooth production line processes, these defects need to be detected. In addition, in order to improve economic benefits, reduce waste and improve resource utilization, it is necessary to classify and identify defect types and quantitatively detect them, so that the defect types can be distinguished and quality graded according to the size of the defects, providing basic data support for intelligent product selection.

[0003] For the grayscale defects and two-dimensional shape defects on the surface of popsicle sticks, they are currently mainly detected with the help of machine vision technology, which basically realizes intelligent recognition and automatic selection, overcomes the shortcomings of manual inspection, and improves production efficiency and the reliability of inspection results.

[0004] The three-dimensional defects of popsicle sticks mainly include bow, dead bend and adze. The existing automated detection device relies on mechanical thickness measuring devices or beam sensors to convert three-dimensional defects into thickness to achieve indirect judgment of three-dimensional defects, and only judges bending defects, and cannot perform quantitative detection. The longitudinal bending detection mechanism of popsicle sticks developed based on the principle of light flux method (application number: CN01250579.X) can reduce the false detection rate of longitudinal bending of popsicle sticks from the original 20% to less than 3%, which has great economic benefits and can detect whether the outline size of the popsicle stick is out of tolerance, but cannot detect the size of the bending. The adze detection mechanism (application number: CN200820012542.0) is installed on the popsicle stick inspection machine to detect unqualified popsicle sticks with adze defects, but it cannot quantitatively detect the defect size and quality grading. Su Xiaohui (Research on Online Detection Method of Ice Scoop Appearance Defects, Shenyang University of Technology, 2020) collects images of the end faces of the ice scoop on both sides and judges the adze defects based on the image grayscale and edge detection operators. This method has strict requirements on lighting and imaging angles, and the processing algorithm has poor robustness, is prone to misjudgment, and is difficult to apply in engineering. The patent (application number: CN200920010450.3) proposes a visual inspection machine for ice lollies, in which only light source illumination and imaging camera devices are provided for adze detection, and no specific algorithm for adze detection is provided.

[0005] The related technologies related to visual inspection in the existing technology do not provide specific algorithm solutions; they can only perform rough inspection of single defects, cannot realize integrated inspection of multiple three-dimensional defects, and cannot realize defect classification and quantitative inspection. Summary of the invention

[0006] In order to solve the problems existing in the prior art, the purpose of the present invention is to provide an integrated three-dimensional defect detection method based on laser point cloud, so as to solve the problem of achieving qualified products and classification identification and quantitative detection of three three-dimensional defects such as adze heads, dead bends and bow bends on the production line of wooden popsicle sticks and ice cream scoops at one time.

[0007] To achieve the above object, the technical solution adopted by the present invention is: a three-dimensional defect integrated detection method based on laser point cloud, comprising the following steps:

[0008] Step 1: Preprocess the point cloud based on the RANSAC algorithm to adjust the pose of the acquired point cloud to ensure that the data points of the reference plane are on the same horizontal plane;

[0009] Step 2: point cloud segmentation to extract target surface point cloud;

[0010] Step 3, obtain the target surface cross-section data and perform RANSAC benchmark straight line fitting, calculate the mean square error and linearity characteristics, and use a binary classification algorithm to identify the target object as a plane or a curved class;

[0011] Step 4: For plane targets, the height difference between the point cloud at both ends of the target surface and the reference plane is mapped into a grayscale image to determine whether there is an adze defect, thereby realizing the identification of qualified and adze defects and the detection of defect quantity;

[0012] Step 5: For curved targets, the curved features are calculated from the target surface cross-section data, the SVM is used to identify the curved shape, and the amount of curvature is calculated.

[0013] As a further improvement of the present invention, the step 1 is specifically as follows:

[0014] Point cloud preprocessing uses the RANSAC algorithm to fit the point cloud to obtain the plane equation of the reference plane:

[0015] aX+bY+cZ+d=0 (1)

[0016] Find the new expression of Z with respect to X and Y: Z = -(aX + bY + d) / c, substitute the X and Y values ​​in the original point cloud data into the new expression to get Z', and subtract Z' from the Z value in the original point cloud to get Z new , and then convert the X, Y and Z values ​​in the original point cloud new Combine them together and adjust the posture to ensure that the reference plane coordinate points are on the same horizontal plane.

[0017] As a further improvement of the present invention, the step 2 is specifically as follows:

[0018] Point cloud segmentation is achieved by calculating all data points (X i ,Y i ,Z i ) to the reference plane i , determine the interior point (d i ≤T) and external points (d i >T), the threshold T and the number of iterations are determined according to the standard deviation of the plane distance and the ratio of the inner points. The distance formula is:

[0019]

[0020] Count the number of internal points and determine whether it is greater than the set ratio. If the ratio is not satisfied, the iteration returns, and if it is satisfied, the process ends, thus completing the point cloud segmentation and obtaining the target surface point cloud.

[0021] As a further improvement of the present invention, in step 3, the acquired target surface cross-sectional data (X) is classified based on the binary classification algorithm of the cross-sectional data. i ,Z i ) to perform RANSAC benchmark straight line fitting, calculate the mean square error and linearity characteristics, and use SVM to identify the target as a plane or a curved class; where the benchmark straight line equation is:

[0022]

[0023] The distance from the target surface section data to the reference straight line is:

[0024]

[0025] The formula for calculating the mean square error is:

[0026]

[0027] Linearity (LD) reflects the linearity of the distribution trend of the target surface cross-section data points. First, the distance standard deviation is calculated:

[0028]

[0029] In the formula, is the average distance.

[0030] Determine the maximum standard deviation σ max , the target surface section data points are statistically located in the interval Points within:

[0031]

[0032] Then the linearity is:

[0033] LD=N' / N (8).

[0034] As a further improvement of the present invention, in step 4, for a plane target, the height difference between the point cloud at both ends of the target surface and the reference plane is mapped to a grayscale image to determine whether there is an adze defect, thereby realizing the identification of qualified and adze defects and the detection of defect quantity; specifically, it includes:

[0035] (1) Use the RANSAC algorithm to fit the reference plane to the point clouds at both ends of the target surface;

[0036] (2) Since the data points in the adze area are lower than the reference plane, the data points on and above the reference plane are mapped to constant grayscale values, and the data points below the reference plane have their height differences linearly mapped to grayscale values;

[0037] (3) Based on the adze depth and size defined by industry standards, use the grayscale image to determine whether there is an adze defect; if it is greater than the defined value, it is judged as an adze and the actual defect size is output; otherwise, it is judged to be qualified.

[0038] As a further improvement of the present invention, step 5 is specifically as follows:

[0039] For curved targets, three bending features (i.e., average distance, mean square error, and coefficient of determination) are calculated from the target surface cross-section data, and SVM is used to determine whether the target is a bow or a dead bend, and the amount of bending is further calculated. Specifically, the following are included:

[0040] Calculate the distance h from the target surface section data point to the reference straight line i , then the average distance is:

[0041]

[0042] The mean square error (MSE) feature calculation formula is the same as formula (5);

[0043] Use the least squares method to fit the quadratic curve equation to the target surface cross-section data:

[0044]

[0045] The formula for calculating the coefficient of determination is:

[0046]

[0047] In the formula, is the mean of the actual observed values;

[0048] The average distance, mean square error and determination coefficient are selected as feature parameters, the feature data are calculated, and the input data is classified using SVM to identify dead bends or bow bends. If it is a bow bend, the least squares method is used to fit the reference straight line. If it is a dead bend, the RANSAC is used to fit the reference straight line. The maximum distance between the target surface section data and the reference straight line is calculated to be the amount of bending.

[0049] As a further improvement of the present invention, the target object is a wooden popsicle stick or a wooden ice cream scoop.

[0050] The beneficial effects of the present invention are:

[0051] 1. The present invention provides an integrated detection solution for three-dimensional defects of wooden popsicle sticks and ice cream scoops based on laser point cloud, which not only improves production efficiency but also saves equipment investment;

[0052] 2. The present invention can realize accurate identification and classification, quantitative detection and quality grading of various defects, thereby improving resource utilization, reducing waste and improving economic benefits;

[0053] 3. The present invention is a non-contact non-destructive testing method, which will not cause secondary damage to products such as popsicle sticks, and can improve the production stability of the assembly line. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flow chart of a three-dimensional defect integrated detection method based on laser point cloud of the present invention;

[0055] Figure 2 is the point cloud data of the ice cream stick sample after preprocessing in the embodiment of the present invention; wherein (a) is a qualified product, and (b) is an ice cream stick with a bow defect;

[0056] Figure 3 1 is the result of the segmentation of the point cloud region of the ice cream stick sample in the embodiment of the present invention; wherein (a) is a qualified product, (b) is an ice cream stick with a dead bend defect, (c) is an ice cream stick with a bow defect, and (d) is an ice cream stick with an adze defect;

[0057] Figure 4 is the surface cross-section point cloud data of the ice cream stick sample in the embodiment of the present invention; wherein (a) is a qualified product, and (b) is an ice cream stick with a bow defect;

[0058] Figure 5 1 is the RANSAC linear fitting result of the surface cross-section data of the ice cream stick sample in the embodiment of the present invention; wherein (a) is an ice cream stick with adze defect, and (b) is an ice cream stick with bow defect;

[0059] Figure 6 is the binary classification test result of the SVM model in the embodiment of the present invention;

[0060] Figure 7 1 is a grayscale image and recognition result of different ice cream stick adze defects in an embodiment of the present invention;

[0061] Figure 8 is the SVM bending classification test result in the embodiment of the present invention; wherein (a) is the training set test result, and (b) is the test set test result;

[0062] Fig. 9 1 is a schematic diagram of the quantitative detection results of bent ice cream stick samples in an embodiment of the present invention; wherein (a) is an ice cream stick with a dead bend defect, and (b) is an ice cream stick with a bow bend defect. DETAILED DESCRIPTION

[0063] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0064] Example 1

[0065] This embodiment takes a wooden popsicle stick as an example to illustrate a three-dimensional defect integrated detection method based on laser point cloud, including the following steps:

[0066] Step 1: The directly acquired 3D point cloud contains background point cloud in addition to the surface data of the popsicle stick. The background point cloud comes from the reference plane. To ensure that the data points of the reference plane are on the same horizontal plane, the acquired point cloud needs to be preprocessed.

[0067] Use the RANSAC algorithm to fit the point cloud data to get the plane equation of the reference plane:

[0068] aX+bY+cZ+d=0 (1)

[0069] Find an expression for Z in terms of X and Y:

[0070] Z=-(aX+bY+d) / c (2)

[0071] Substitute the X and Y values ​​of the original point cloud data into equation (2) to get Z', and subtract Z' from the Z value in the original point cloud to get Z new , and then convert the X, Y and Z values ​​in the original point cloud new Combine them together and adjust the posture to ensure that the reference plane coordinate points are on the same horizontal plane.

[0072] Step 2: To solve the point cloud segmentation problem of popsicle sticks with three-dimensional depth defects, a plane fitting segmentation method based on the RANSAC algorithm is provided.

[0073] Point cloud segmentation is achieved by calculating all data points (X i ,Y i ,Z i ) to the reference plane i, determine the interior point (d i ≤T) and external points (d i >T), the threshold T and the number of iterations are determined according to the standard deviation of the plane distance and the ratio of the inner points. The distance formula is:

[0074]

[0075] Count the number of internal points and determine whether it is greater than the set ratio. If the ratio is not satisfied, the iteration returns, and if it is satisfied, the process ends, thus completing the point cloud segmentation and obtaining the target surface point cloud.

[0076] Step 3: In order to reduce the difficulty of various defect classification and recognition algorithms and improve the reliability of classification results, the present invention proposes a binary classification algorithm based on target surface cross-sectional data to identify the flat or curved type of the tested popsicle stick.

[0077] (1) First, the 3D point cloud data of the target surface is reduced along the Y axis to obtain a 2D point set excluding the Y coordinate, thereby extracting the target surface point cloud cross-sectional data (X i ,Z i ).

[0078] (2) Reference straight line fitting

[0079] The RANSAC algorithm was used to analyze the cross-sectional data (X i ,Z i ) to fit the reference straight line, the straight line equation is:

[0080]

[0081] (3) Classification feature design

[0082] Calculate the cross-sectional data points (X i ,Z i ) to the reference straight line:

[0083]

[0084] Calculate the mean square error characteristics based on the above results:

[0085]

[0086] Linearity (LD) reflects the linearity of the distribution of cross-section data points. First, the distance standard deviation is calculated:

[0087]

[0088] In the formula, is the average distance.

[0089] Determine the maximum standard deviation σ max , the target surface section data points are statistically located in the interval Points within:

[0090]

[0091] Then the linearity is:

[0092] LD=N' / N (9)

[0093] (4) SVM plane and curved class recognition

[0094] Using mean square error (MSE) and linearity (LD) as input, the optimal parameters C and g are determined by grid search method. 70% of the samples are randomly selected as training sets and 30% as test sets. Gaussian RBF kernel function is used for plane and curved recognition.

[0095] Step 4: For plane targets, the present invention determines whether there is an adze defect by mapping the height difference between the point cloud at both ends of the target surface and the reference plane into a grayscale image, thereby realizing the identification of qualified and adze defects and the detection of defect quantity; specifically, it includes:

[0096] (1) Use the RANSAC algorithm to fit the reference plane to the point clouds at both ends of the target surface;

[0097] (2) Since the data points in the adze area are lower than the reference plane, the data points on and above the reference plane are mapped to constant grayscale values, and the data points below the reference plane have their height differences linearly mapped to grayscale values;

[0098] (3) Based on the adze depth and size defined by the industry standard, use the grayscale image to determine whether there is an adze defect; if it is greater than the defined value, it is judged as an adze and the actual defect size is output; otherwise, it is judged as qualified;

[0099] Step 5: For curved targets, three bending features (i.e., average distance, mean square error, and coefficient of determination) are calculated from the target surface cross-section data, and SVM is used to determine whether the target is a bow or a dead bend, and the amount of bending is further calculated; specifically, the following are included:

[0100] Calculate the distance h from the target surface section data point to the reference straight line i , then the average distance is:

[0101]

[0102] The mean square error (MSE) feature calculation formula is the same as formula (5);

[0103] Use the least squares method to fit the quadratic equation to the target surface cross-section data:

[0104]

[0105] The formula for calculating the coefficient of determination is:

[0106]

[0107] In the formula, is the mean of the actual observed values;

[0108] Use SVM to identify dead bends or bow bends; if it is a bow bend, use the least squares method to fit the reference straight line; if it is a dead bend, use RANSAC to fit the reference straight line; calculate the maximum distance between the target surface section data and the reference straight line, which is the amount of bending.

[0109] Example 2

[0110] A three-dimensional defect integrated detection method based on laser point cloud, the implementation process is as follows Figure 1 As shown, it mainly includes:

[0111] Step 1: Based on the point cloud preprocessing of the RANSAC algorithm, the acquired point cloud is adjusted to ensure that the data points of the reference plane are on the same horizontal plane. Figure 2 Shown is the pre-processed point cloud data.

[0112] Step 2: Point cloud segmentation extracts the target surface point cloud. Figure 3 Shown are point cloud data of different defect types and the surface point cloud of the popsicle stick segmented by fitting and difference calculation.

[0113] Step 3: Classify plane and curved targets. The three-dimensional data of the target surface is processed along the Y axis for dimensionality reduction. Figure 4 The cross-sectional data of the ice cream stick sample is shown. The RANSAC algorithm is used to fit the reference straight line equation, calculate the distance between each data point and the fitting straight line, and then calculate the mean square error (MSE) and linearity (LD) two features. Figure 5 Shown are the RANSAC straight line fitting results of the cross-section data of the popsicle stick sample (adze and bow target).

[0114] Several ice cream stick samples were selected, and the surface cross-section data were extracted to calculate the mean square error and linear fit. Under the same imaging conditions, 400 samples were randomly selected, including 4 categories (qualified, adze, bow, and dead bend), and finally formed a 400×2 feature matrix. Some features are shown in Table 1.

[0115] Table 1 Characteristic parameter calculation results (partial)

[0116]

[0117] The SVM classification model was used to classify the ice cream stick feature data, and the optimal parameters C = 64 and g = 0.5 were determined by grid search method. 70% of the samples were randomly selected as training sets and 30% as test sets, and the Gaussian RBF kernel function was used for binary classification. Figure 6 The following table shows the prediction results. The horizontal axis is the test sample number and the vertical axis is the category. The points represent the true category, the circles represent the predicted category, and the correctly classified points fall within the circles (see Figure 6 The error falls in the solid line box, and the error falls in the dotted line box. The statistical results show that the accuracy of the test set is 96.67%.

[0118] Step 4: Classification and detection of defects on flat popsicle sticks. For flat targets, the height difference between the point cloud at both ends of the target surface and the reference plane is mapped to a grayscale image to determine whether there is an adze defect. In actual operation, if the number of pixels counted exceeds the number of pixels set after quantization, the area is considered to be an adze area, otherwise, the popsicle stick is considered to be qualified. Figure 7 Shown are the results of adze defect recognition for different popsicle sticks.

[0119] Step 5: Bend classification and bend detection, select the average distance, mean square error and determination coefficient as feature parameters. Under the same experimental conditions, the above three feature parameters of the curved popsicle stick cross-section data were extracted, and a total of 200 groups of samples of dead bends and bow bends were obtained to form a 200×3 feature matrix, where "1" represents dead bends and "2" represents bow bends. The actual calculation results of some features are shown in Table 2.

[0120] Table 2 Calculation results of bending classification features (partial)

[0121]

[0122] The SVM classification model is applied to classify the popsicle stick feature data. 70% of the samples are randomly selected from the data set as the training set and 30% as the test set. The support vector machine and Gaussian RBF kernel function are used for classification, and the accuracy of the training set and the test set are both 100%. Figure 8 Shown are the test results.

[0123] If it is a bow bend, the least square method is used to fit the reference straight line. If it is a dead bend, the RANSAC method is used to fit the reference straight line. The maximum distance between the target surface section data and the reference straight line is the amount of bending. Fig. 9 Shown are the results of quantitative inspection of popsicle sticks with bending defects.

[0124] The above-mentioned embodiments only express the specific implementation of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A three-dimensional defect integrated detection method based on laser point cloud, characterized in that: The following steps are involved: Step 1: Preprocess the point cloud based on the RANSAC algorithm to adjust the pose of the acquired point cloud to ensure that the data points of the reference plane are on the same horizontal plane; Step 2: point cloud segmentation to extract target surface point cloud; Step 3, obtain the target surface cross-section data and perform RANSAC benchmark straight line fitting, calculate the mean square error and linearity characteristics, and use a binary classification algorithm to identify the target object as a plane or a curved class; Step 4: For plane targets, the height difference between the target surface point cloud and the reference plane is mapped into a grayscale image to determine whether there is an adze defect, thereby realizing the identification of qualified and adze defects and the detection of defect quantity; Step 5: For curved targets, the curved features are calculated from the target surface cross-section data, the SVM is used to identify the curved shape, and the amount of curvature is calculated.

2. The integrated three-dimensional defect detection method based on laser point cloud according to claim 1 is characterized in that: The step 1 is specifically as follows: Point cloud preprocessing uses the RANSAC algorithm to fit the point cloud to obtain the plane equation of the reference plane: aX+bY+cZ+d=0 (1) Find the new expression of Z with respect to X and Y: Z = -(aX + bY + d) / c, substitute the X and Y values ​​in the original point cloud data into the new expression to get Z', and subtract Z' from the Z value in the original point cloud to get Z new , and then convert the X, Y and Z values ​​in the original point cloud new Combine them together and adjust the posture to ensure that the reference plane coordinate points are on the same horizontal plane.

3. The integrated three-dimensional defect detection method based on laser point cloud according to claim 1 is characterized in that: The step 2 is specifically as follows: Point cloud segmentation is achieved by calculating all data points (X i , Y i , Z i ) to the reference plane i , determine the interior point (d i ≤T) and external points (d i >T), the threshold T and the number of iterations are determined according to the standard deviation of the plane distance and the ratio of the inner points. The distance formula is: Count the number of internal points and determine whether it is greater than the set ratio. If the ratio is not satisfied, the iteration returns, and if it is satisfied, the process ends, thus completing the point cloud segmentation and obtaining the target surface point cloud.

4. The integrated three-dimensional defect detection method based on laser point cloud according to claim 1 is characterized in that: In step 3, the acquired target surface cross-sectional data (X i ,Z i ) to perform RANSAC benchmark straight line fitting, calculate the mean square error and linearity characteristics, and use SVM to identify the target as a plane or a curved class; where the benchmark straight line equation is: The distance from the target surface section data to the reference straight line is: The formula for calculating the mean square error is: Linearity (LD) reflects the linearity of the distribution trend of the target surface cross-section data points. First, the distance standard deviation is calculated: In the formula, is the average distance. Determine the maximum standard deviation σ max , the target surface section data points are statistically located in the interval Points within: Then the linearity is: LD=N' / N (8).

5. The integrated three-dimensional defect detection method based on laser point cloud according to claim 1 is characterized in that: In step 4, for planar targets, the height difference between the point cloud at both ends of the target surface and the reference plane is mapped into a grayscale image to determine whether there is an adze defect, thereby achieving the identification of qualified and adze defects and the detection of defect quantity; specifically, the following steps are included: (1) Use the RANSAC algorithm to fit the reference plane to the target surface point cloud; (2) Since the data points in the adze area are lower than the reference plane, the data points on and above the reference plane are mapped to constant grayscale values, and the data points below the reference plane have their height differences linearly mapped to grayscale values; (3) Based on the adze depth and size defined by industry standards, use the grayscale image to determine whether there is an adze defect; if it is greater than the defined value, it is judged as an adze and the actual defect size is output; otherwise, it is judged to be qualified.

6. The integrated three-dimensional defect detection method based on laser point cloud according to claim 1, characterized in that: In step 5, for curved targets, three bending features (i.e., average distance, mean square error, and determination coefficient) are calculated from the target surface cross-section data, and SVM is used to determine whether the target is a bow or a dead bend, and the amount of bending is further calculated. Specifically, the following are included: Calculate the distance h from the target surface section data point to the reference straight line i , then the average distance is: The mean square error (MSE) feature calculation formula is the same as formula (5); Use the least squares method to fit the quadratic curve equation to the target surface cross-section data: The formula for calculating the coefficient of determination is: In the formula, is the mean of the actual observed values; Use SVM to identify dead bends or bow bends; if it is a bow bend, use the least squares method to fit the reference straight line; if it is a dead bend, use RANSAC to fit the reference straight line; calculate the maximum distance between the target surface section data and the reference straight line, which is the amount of bending.

7. The integrated three-dimensional defect detection method based on laser point cloud according to any one of claims 1 to 6, characterized in that: The target object is a wooden popsicle stick or a wooden ice cream scoop.

Images