Flaw detection method based on cooperation of linear array laser and mechanical arm

Abstract

The invention relates to the technical field of intelligent quality detection, in particular to a flaw detection method based on cooperation of linear laser and a mechanical arm, and the method comprises the steps: carrying out the all-directional scanning of the surface of a workpiece through the cooperation of the mechanical arm and the linear laser, and obtaining the depth data of the surface of the workpiece; then processing depth data to perform high-precision identification and positioning on flaws on the surface of the object; and performing preprocessing, surface area segmentation, curvature calculation and enhancement and multi-dimensional flaw detection on the depth data to realize automatic flaw detection. The device is matched with a mechanical arm and a linear array laser sensor, can adapt to workpieces of different shapes and different sizes, can remarkably improve the flaw detection efficiency in industrial production, reduces manual intervention, and reduces the workload of operators.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent quality inspection, and in particular to a defect inspection method based on the collaboration of linear laser and mechanical arm. Background Art

[0002] In modern industrial production, automated testing of product quality is a key link in improving production efficiency and ensuring product quality. With the continuous development of intelligent manufacturing, defect detection, as an important part of product quality management, is developing in the direction of high precision, high speed and intelligence. However, traditional defect detection methods mainly rely on manual visual inspection, which is inefficient, greatly affected by subjective factors, and difficult to meet the requirements of modern industry for high precision, high speed and mass production.

[0003] At present, the mainstream automated defect detection technologies can be roughly divided into the following two categories:

[0004] 1. Defect detection technology based on camera RGB images, such as automatic optical inspection (AOI) technology;

[0005] 2. Geometric defect detection technology based on linear laser point cloud.

[0006] Automatic optical inspection (AOI) technology uses optical imaging methods to obtain images of product surfaces and combines image processing algorithms for analysis to identify and classify defects on product surfaces. This technology performs well on certain flat or simple-structured workpieces, but still faces significant challenges when dealing with complex surfaces, tiny defects, or inspections under different lighting conditions.

[0007] Although existing technologies have improved the efficiency and accuracy of automated defect detection to a certain extent, there are still many deficiencies in coping with complex industrial scenarios and meeting the needs of high-speed real-time detection, as follows:

[0008] 1. Environmental sensitivity of automatic optical inspection (AOI) technology

[0009] AOI technology relies on optical imaging methods, so the image quality is easily affected by ambient light. In complex or changeable industrial environments, when the lighting conditions are inconsistent, the detection effect of the AOI system is often unstable, which may lead to missed detection or false detection. In addition, for workpieces with complex surfaces, AOI is difficult to effectively capture surface geometry changes, and the detection performance is limited.

[0010] 2. Automated Optical Inspection (AOI) is not sensitive to tiny defects

[0011] AOI technology mainly processes surface information based on RGB images and is not sensitive enough to small or local defects. In many industrial scenarios, some minor defects are difficult to distinguish in RGB images, especially when the surface color of the workpiece is complex. However, the depth data collected by linear laser can more clearly show the subtle geometric changes on the surface of the workpiece, thereby identifying defects that are difficult to capture in RGB images.

[0012] 3. Real-time limitations of linear laser detection technology

[0013] Most of the current defect detection technologies based on linear lasers rely on pre-trained deep learning models. Although these models perform well in certain scenarios, they often fail to meet the high requirements of real-time detection in industrial sites. Due to the long inference time of deep learning models, the system processing speed is not fast enough to support high-speed production line applications, resulting in low detection efficiency.

[0014] 4. Linear laser equipment is not flexible enough

[0015] In the prior art, linear laser equipment is usually fixed in position and has a fixed scanning area, which makes it difficult to adapt to workpieces with complex shapes and diverse structures. Since the laser equipment cannot flexibly adjust its position according to the shape of the workpiece, some workpieces with complex structures cannot be effectively scanned in certain areas, which in turn affects the comprehensiveness and accuracy of defect detection. Summary of the invention

[0016] The purpose of the present invention is to provide a defect detection method based on the collaboration of linear laser and robotic arm, which can complete the scanning and defect detection of workpieces through centralized control on the PC side, reduce the burden on operators, and improve detection efficiency and accuracy.

[0017] To achieve the above object, the present invention provides a defect detection method based on the collaboration of linear laser and robotic arm, comprising the following steps:

[0018] Step 1: PC controls the robot arm to start workpiece data collection;

[0019] Step 2: The robotic arm drives the linear laser sensor to scan, collect and return the execution results;

[0020] Step 3: The PC determines whether the workpiece data is complete. If the workpiece scanning data is successful, go to step 4, otherwise restart the acquisition process;

[0021] Step 4: Perform workpiece quality judgment. If the workpiece quality is unqualified, record the bad data and proceed to step 5;

[0022] If the workpiece quality is qualified, the workflow of the current workpiece is completed and the process goes to step 5;

[0023] Step 5: Repeat steps 1 to 4 until all workpieces are processed.

[0024] Optionally, the linear laser sensor is fixed at the end of the robotic arm, and its motion trajectory is consistent with that of the robotic arm.

[0025] Optionally, in step 4, a workpiece surface defect detection algorithm is used to perform quality judgment, and the process includes the following steps:

[0026] Step 4.1: Complete data preprocessing by missing value processing, boundary outlier detection and removal;

[0027] Step 4.2: Use connected component analysis to segment the workpiece surface;

[0028] Step 4.3: Combine differential curvature calculation and local curvature feature enhancement to obtain curvature data;

[0029] Step 4.4: Analyze curvature data and perform defect detection and analysis.

[0030] Optionally, the missing values ​​in step 4.1 are partial data loss or anomalies caused by the laser of the linear laser sensor not being correctly received by the receiver. The processing method is to mark these invalid points through mask processing and not participate in subsequent analysis; boundary anomaly points refer to depth mutation points in the data caused by uneven scanning or sensor noise.

[0031] Optionally, in step 4.2, surface segmentation is performed by determining whether the depths of adjacent points in the 8-connected neighborhood are close, and a number of surface regions are obtained as basic units for subsequent curvature calculation and defect detection.

[0032] Optionally, the surface curvature in step 4.3 is obtained based on the gradient and second-order derivative of the segmented surface depth matrix in the x and y directions, and the resolution of the feature is improved by calculating the curvature mean and standard deviation within a local window.

[0033] Optionally, the defect detection and analysis process includes the following steps:

[0034] Multi-scale feature extraction;

[0035] Use principal component analysis to reduce the dimensionality of multi-scale features;

[0036] Use connected region analysis to detect effective defect areas;

[0037] Fitting discrimination and trend analysis verify the effectiveness of defects.

[0038] Optionally, in the process of fitting discrimination and trend analysis, linear fitting or quadratic curve fitting is used to fit the edge points of each defect area, and the residual of the fitting is calculated.

[0039] The present invention provides a defect detection method based on the cooperation of linear laser and mechanical arm, through the cooperation of mechanical arm and linear laser, the surface of the workpiece is scanned in an all-round way to obtain the depth data of the workpiece surface, and then the depth data is processed to identify and locate the defects on the surface of the object with high precision; then the depth data is pre-processed, the surface area is segmented, the curvature is calculated and enhanced, and multi-dimensional defect detection is performed to realize automatic defect detection. The present invention cooperates with mechanical arm and linear laser sensor, can adapt to workpieces of different shapes and sizes, can significantly improve the efficiency of defect detection in industrial production, and also reduces manual intervention and reduces the workload of operators. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 It is a flow chart of a defect detection method based on the collaboration of linear laser and robotic arm of the present invention. DETAILED DESCRIPTION

[0042] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0043] See also Figure 1 The present invention provides a defect detection method based on the cooperation of linear laser and mechanical arm, comprising the following steps:

[0044] Step 1: PC controls the robot arm to start workpiece data collection;

[0045] Step 2: The robotic arm drives the linear laser sensor to scan, collect and return the execution results;

[0046] Step 3: The PC determines whether the workpiece data is complete. If the workpiece scanning data is successful, go to step 4, otherwise restart the acquisition process;

[0047] Step 4: Perform workpiece quality judgment. If the workpiece quality is unqualified, record the bad data and proceed to step 5;

[0048] If the workpiece quality is qualified, the workflow of the current workpiece is completed and the process goes to step 5;

[0049] Step 5: Repeat steps 1 to 4 until all workpieces are processed.

[0050] Specifically, the linear laser sensor is fixed at the end of the robotic arm, and its motion trajectory is consistent with that of the robotic arm.

[0051] The present invention embeds a defect detection algorithm on the workpiece surface in the process of judging the workpiece quality. First, the workpiece surface is scanned in all directions by the coordination of the robot arm and the linear laser to obtain the depth data of the workpiece surface. Then, the defects on the object surface can be identified and located with high precision by processing the depth data. Automated defect detection is achieved by preprocessing the depth data, segmenting the surface area, calculating and enhancing the curvature, and multi-dimensional defect detection.

[0052] The following is an explanation of the specific process. The workpiece data in defect detection is processed through the following process:

[0053] 1. Data Preprocessing

[0054] 1. Missing Value Processing

[0055] In depth data collection, some areas may be blocked by equipment, reflective surfaces, or other reasons, which may cause the laser emitted by the linear laser to be unable to be correctly received by the receiver, resulting in partial data loss or anomalies, and generating missing values. Such missing values ​​are recorded as NaN in the data. In order to avoid the interference of missing values ​​in subsequent analysis, the present invention marks these invalid points through mask processing. Assuming that the depth data matrix is ​​Z(x,y), the mask is defined as:

[0056]

[0057] Among them, outliers refer to extremely unreasonable depth values, such as abnormally high or low values ​​caused by sensor errors. These points are regarded as invalid points in subsequent processing and are not included in the analysis.

[0058] 2. Detection and removal of boundary outliers

[0059] Boundary outliers are depth mutation points in the data due to uneven scanning or sensor noise. These outliers significantly affect curvature calculation and defect detection, so they must be removed. In order to detect these outliers, the first-order difference method is used to calculate the depth difference between adjacent points.

[0060] Assuming the depth data is Z(x,y), the depth differences between adjacent points in the horizontal and vertical directions are:

[0061] ΔZ x =|Z(x+1,y)-Z(x,y)|, ΔZy =|Z(x,y+1)-Z(x,y)|.

[0062] When ΔZx or ΔZy exceeds the preset threshold ∈, these points are considered to be boundary outliers. To ensure that enough outlier data is removed, all points within a small range around the outlier (e.g., a 3×3 area) are marked as invalid points. This process can more effectively eliminate the influence of noise.

[0063] 2. Surface segmentation

[0064] After data preprocessing, the next step is to segment the surface area. The purpose of surface segmentation is to divide continuous points on the same surface into a region to facilitate subsequent curvature analysis. The connected component labeling algorithm is one of the commonly used surface segmentation methods. This algorithm determines whether adjacent points belong to the same surface by judging whether their depths are close.

[0065] 1. Connected Component Analysis

[0066] Assume there are two adjacent points Z(x1,y1) and Z(x2,y2), if the depth difference between them satisfies:

[0067] |Z(x1,y1)-Z(x2,y2)|≤∈,

[0068] Among them, ∈ is a preset threshold, indicating the acceptable range of depth value, and the two points are classified as the same connected region. Through the connected component labeling algorithm, the entire depth matrix is ​​divided into several connected regions, each of which corresponds to an independent surface. The algorithm usually uses an 8-connected neighborhood to process, that is, the relationship between a point and its 8 surrounding neighboring points is taken into account.

[0069] After connected component analysis, several surface regions can be obtained, which can be used as basic units for subsequent curvature calculation and defect detection.

[0070] 3. Curvature calculation and enhancement

[0071] Curvature is an important indicator of surface changes. By calculating the curvature, prominent areas on the surface (such as defects) can be found. The calculation of surface curvature requires the use of surface gradients and second-order derivatives.

[0072] 1. Differential curvature calculation

[0073] Let Z(x,y) be the segmented surface depth matrix. The curvature is calculated based on the gradient and second-order derivative of Z(x,y) in the x and y directions. First, calculate the gradient:

[0074]

[0075] Then calculate the second-order derivative:

[0076]

[0077] The calculation formula of surface curvature K(x,y) is:

[0078] K(x,y)=|Z xx |+|Z yy |.

[0079] The formula measures how much a surface changes in both the horizontal and vertical directions by calculating the curvature in those two directions. The greater the curvature value, the more dramatic the surface change.

[0080] 2. Local curvature feature enhancement

[0081] In order to enhance the characteristic performance of curvature, the resolution of the feature is improved by calculating the mean and standard deviation of the curvature in the local window. Assuming that the size of the local window is w×w, the calculation formula of the local mean μ(x,y) and the local standard deviation σ(x,y) is:

[0082]

[0083] By calculating the local curvature mean and standard deviation, more detailed surface variation information can be captured. This information helps to detect potential defects on the surface more accurately.

[0084] 4. Defect Detection and Analysis

[0085] After the curvature calculation is completed, the system enters the defect detection stage. The goal of defect detection is to identify abnormal areas on the surface by analyzing the curvature data. In order to improve the accuracy of detection, the present invention adopts a combination of multi-scale analysis method and principal component analysis (PCA).

[0086] 1. Multi-scale feature extraction

[0087] In order to adapt to defects of different sizes, this system uses a multi-scale feature extraction method. Specifically, filters of different scales are used to analyze the curvature data to capture defects of different sizes. Assuming that the scale of the filter is s and the direction is θ, the response of the filter can be expressed as:

[0088] F(x,y,s,θ)=∫∫K(x',y')G(x-x',y-y',s,θ)dx'dy',

[0089] Where G(x,y,s,θ) is a filter function used to extract high-frequency information of curvature at different scales and directions. Multi-scale filters can capture both small-scale and large-scale defect features, enhancing the system's ability to detect different types of defects.

[0090] 2. Principal Component Analysis (PCA)

[0091] In order to further reduce the feature dimension and improve computational efficiency, this system uses principal component analysis (PCA) to reduce the dimensionality of multi-scale features. The goal of PCA is to find a projection matrix W that projects the original multi-dimensional features F into a low-dimensional space and retains the most discriminative features. The specific formula is:

[0092] F PCA =W T F.

[0093] Where W is the projection matrix of the principal component and F is the multi-scale eigenvector. Through PCA dimensionality reduction, the system can reduce feature redundancy and extract the main feature components related to defects.

[0094] 3. Connected Region Analysis

[0095] After extracting multi-scale features and reducing the dimension, the system needs to identify possible defect areas. This step is achieved through the connected component marking algorithm. Assuming that the feature matrix is ​​FPCA(x,y), the connected components are marked to find the areas with similar eigenvalues. These connected areas are regarded as potential defects.

[0096] In order to filter out invalid areas, the present invention sets minimum and maximum area thresholds. Let the area of ​​the defective area be A(Ri), if its area meets the following conditions:

[0097] A min ≤A(R i )≤A max ,

[0098] This will prevent the detection of invalid areas that are too small or too large.

[0099] 4. Fitting discrimination and trend analysis

[0100] After the connected region analysis is completed, in order to further verify the validity of the defect, the present invention also uses fitting discrimination and trend analysis methods. For each detected defect area, the system attempts to fit its edge points using a linear or quadratic curve and calculate the residual of the fitting. If the residual is small, it means that the surface change of the area meets the expected defect morphology, and it is further confirmed to be a valid defect.

[0101] (1) Linear fitting

[0102] Assume that the edge point of the defect area is (xi, yi), and use linear fitting to estimate its change trend. The formula of linear fitting is:

[0103] y=ax+b,

[0104] Where a and b are fitting parameters. Solve for a and b using the least squares method and calculate the fitting residual Rlinea r :

[0105]

[0106] If the residual Rlinear is less than the preset threshold, it is considered that the change in the area conforms to the linear trend and may be a valid defect.

[0107] (2) Quadratic curve fitting

[0108] In order to adapt to more complex forms, this system also uses quadratic curve fitting. The quadratic fitting formula is:

[0109] y=ax 2 +bx+c,

[0110] Similarly, a, b and c are solved by the least squares method, and the residual Rquadratic is calculated:

[0111]

[0112] If the residual Rquadratic of the quadratic fit is smaller than the preset threshold and is significantly smaller than the residual of the linear fit, it is considered that the region has complex curvature changes, which is a more deterministic defect.

[0113] After being processed by the above algorithm, the data can quickly locate whether there are defects in the workpiece and the location of the defects.

[0114] Furthermore, in the embodiment, the present invention uses a linear laser sensor to work in conjunction with a robotic arm, and by fixing the linear laser at the end of the robotic arm, the consistency of its motion trajectory with the robotic arm is ensured, thereby achieving precise scanning. Specifically, a Mecha-Mind LNX-8080 linear laser sensor is used, which has 4096 scanning points, a scanning rate of 3.3-15kHz, an X-axis resolution of 0.0235mm, and a Z-axis resolution of 0.0005mm, which can ensure high precision and stability of workpiece data during scanning. The robotic arm uses the Kawasaki RS003N small and medium-sized general-purpose robot, which has a maximum arm span of 620mm and a repeatability accuracy of **±0.02mm**, allowing the robotic arm to stably and accurately perform scanning tasks under a high degree of freedom of 6 axes.

[0115] In terms of connection method, the linear laser sensor and the robotic arm are connected to the switch via a network cable, and the switch is then connected to the PC to achieve hardware interconnection and communication of the entire system.

[0116] It has been verified through the examples that the defect detection algorithm of the present invention can determine whether a workpiece has a defect and the defect position of the workpiece within 5 seconds after the optimization of the steps.

[0117] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A defect detection method based on the collaboration of linear laser and robotic arm, characterized in that: The following steps are involved: Step 1: PC controls the robot arm to start workpiece data collection; Step 2: The robotic arm drives the linear laser sensor to scan, collect and return the execution results; Step 3: The PC determines whether the workpiece data is complete. If the workpiece scanning data is successful, go to step 4, otherwise restart the acquisition process; Step 4: Perform workpiece quality judgment. If the workpiece quality is unqualified, record the bad data and proceed to step 5; If the workpiece quality is qualified, the workflow of the current workpiece is completed and the process goes to step 5; Step 5: Repeat steps 1 to 4 until all workpieces are processed.

2. The defect detection method based on the cooperation of linear laser and robot arm as claimed in claim 1, characterized in that: The linear laser sensor is fixed at the end of the mechanical arm, and its motion trajectory is consistent with that of the mechanical arm.

3. The defect detection method based on the cooperation of linear laser and robot arm as claimed in claim 2, characterized in that: In step 4, the workpiece surface defect detection algorithm is used to make quality judgments. The process includes the following steps: Step 4.1: Complete data preprocessing by missing value processing, boundary outlier detection and removal; Step 4.2: Use connected component analysis to segment the workpiece surface; Step 4.3: Combine differential curvature calculation and local curvature feature enhancement to obtain curvature data; Step 4.4: Analyze curvature data and perform defect detection and analysis.

4. The defect detection method based on the coordination of linear laser and robotic arm as claimed in claim 3, characterized in that: The missing values ​​in step 4.1 are partial data loss or anomalies caused by the laser of the linear laser sensor not being correctly received by the receiver. The processing method is to mark these invalid points through mask processing and exclude them from subsequent analysis; boundary anomalies refer to depth mutation points in the data caused by uneven scanning or sensor noise.

5. The defect detection method based on the cooperation of linear laser and robot arm as claimed in claim 3, characterized in that: In step 4.2, the surface is segmented by judging whether the depths of adjacent points in the 8-connected neighborhood are close, and a number of surface areas are obtained as basic units for subsequent curvature calculation and defect detection.

6. The defect detection method based on the cooperation of linear laser and robot arm as claimed in claim 3, characterized in that: The surface curvature in step 4.3 is obtained based on the gradient and second-order derivative of the segmented surface depth matrix in the x and y directions, and the resolution of the feature is improved by calculating the mean and standard deviation of the curvature within the local window.

7. The defect detection method based on the cooperation of linear laser and mechanical arm as claimed in claim 3, characterized in that: The defect detection and analysis process includes the following steps: Multi-scale feature extraction; Use principal component analysis to reduce the dimensionality of multi-scale features; Use connected region analysis to detect effective defect areas; Fitting discrimination and trend analysis verify the effectiveness of defects.

8. The defect detection method based on the cooperation of linear laser and robot arm as claimed in claim 7, characterized in that: In the process of fitting discrimination and trend analysis, linear fitting or quadratic curve fitting is used to fit the edge points of each defect area, and the residual of the fitting is calculated.

Images