A weld defect detection method based on three-dimensional point cloud

Abstract

The present invention provides a weld defect detection method based on three-dimensional point cloud, comprising: randomly selecting a pixel point from a depth image, taking it as the center and selecting a to-be-processed area; then taking the to-be-processed area as the center area, searching for adjacent areas; obtaining the depth mean of the adjacent areas; replacing the depth value of the pixel point at the center position of the to-be-processed area with the depth mean, filtering the center pixel point; traversing each pixel point of the depth image, adopting the same filtering processing method, completing the filtering of the entire depth image; subtracting the filtered depth image from the original depth image, traversing the depth difference of each pixel point, marking the pixel point whose depth difference is greater than a preset value I as a point in the defect area; obtaining the defect area, the method increases the depth difference between the defect area after filtering and the defect area of ​​the original image, further identifies the porosity defect or the spatter defect through parabola, depth and contrast judgment, and reduces the false alarm rate.

Description

Technical Field

[0001] The present invention relates to the field of weld defect detection, and in particular to a weld defect detection method based on three-dimensional point cloud. Background Art

[0002] In the metal parts processing process, welding process is an indispensable process for many metal products. The quality of welding directly affects the stability of the entire product. In the welding process, the following defects mainly exist: 1) Porosity defect: the gas in the welding pool does not have time to escape and stays in the weld to form defects; 2) Spatter defect: during welding, the electrode or wire is fused with the workpiece to be welded, and the arc force splashes part of the liquefied metal around the molten pool to form defects; 3) Deep pit defect: a defect caused by excessive assembly gap or too fast welding speed; 4) Weld through defect: during welding, the molten metal flows out from the back of the weld and separates from the weld to form a perforation. Among them, deep pits and weld through defects are both concave defects, with large size and prominent depth value information, which are easier to detect, while porosity and spatter defects are relatively random in size and easy to be misdetected or missed.

[0003] In order to evaluate the quality of the weld, it is necessary to perform defect detection on the welding area after or during welding. The existing defect detection methods mainly include manual detection, visual detection and other methods. The disadvantage of manual detection is that the cost is high, and the detection results are easily affected by factors such as fatigue or negligence of the inspector. Visual inspection can realize automatic detection, but due to the poor contrast and high noise of the weld image, common defect detection methods have the following problems: mean filtering - focusing on edge filtering of the defective area, the filtering effect on the middle part of the defective area is poor, so the recognition rate is low and the accuracy is poor; median filtering, Gaussian filtering and bilateral filtering - focusing on filtering the four corner points of the defective area, the filtering effect on the middle part of the defective area is also poor. Although the accuracy is improved compared to mean filtering, there are still problems of missed detection and false detection. At the same time, the algorithm is complex and time-consuming. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a weld defect detection method based on three-dimensional point cloud. The method proposes a new filtering method to filter the depth image of the weld area, thereby increasing the depth difference between the defect area after filtering and the defect area of ​​the original image, thereby accurately identifying the position of the defect area in the image; at the same time, it can also reduce the missed reporting rate; it has the characteristics of being fast, convenient and accurate, and is suitable for the identification and detection of various weld porosity defects.

[0005] The technical solution is as follows:

[0006] A weld defect detection method based on three-dimensional point cloud comprises the following steps:

[0007] Step 1: Scan the point cloud of the weld area and generate a depth image using the point cloud information;

[0008] Step 2: Filter the depth image in the following way:

[0009] Take any pixel point from the depth image, take it as the center, and select a local area of ​​the depth image, which is recorded as the selected area to be processed. The size of the selected area to be processed is set according to the empirical information of the defect size;

[0010] Then, taking the selected area to be processed as the central area, find the adjacent area connected to its position; average the depth values ​​of each pixel point in the adjacent area to obtain the depth mean; the depth value is the depth value of the pixel point in the original depth image;

[0011] The depth value of the pixel point at the center position of the selected area to be processed is replaced with the depth mean value, thereby filtering the center pixel point of the selected area to be processed;

[0012] Traverse each pixel of the depth image and use the same filtering method to complete the filtering of the entire depth image;

[0013] Step 3: Calculate the absolute value of the depth difference between each pixel on the filtered depth image and the original depth image, and mark the pixel whose absolute value of the depth difference is greater than a preset value I as a point in the defect area;

[0014] Mark each point in the defect area to obtain the location of the defect area.

[0015] Furthermore, the defective area is restored to the original depth image, and the defective area is judged using the following steps:

[0016] Taking the X-axis direction of the image coordinate system as the row direction, a parabola fitting is performed on each row of pixel points in the defect area;

[0017] If the fitted parabola opening size and parabola arc area meet the preset conditions, the corresponding pixel points are marked, otherwise, they are not retained;

[0018] Determine whether there are marked pixels. If so, count all the marked pixels and record them as the pore / splash defect area; if not, the defect area is other defects.

[0019] Furthermore, it is determined that the fitted parabola opening size and parabola arc area meet the preset conditions, using the following method:

[0020] According to the empirical information of pore defects or spatter defects, a parabolic opening size threshold A and a parabolic bow area threshold B are preset. When the fitted parabolic opening size is greater than the threshold A and the fitted parabolic bow area is greater than the threshold B, the preset conditions are met.

[0021] Furthermore, a least square method or a RANSAC method is used to perform parabola fitting on each row of pixels in the defect area.

[0022] Furthermore, the defective area is judged and identified using the following methods:

[0023] The defect area is restored to the original depth image, and then the depth mean of the first row of pixels and the depth mean of the last row of pixels in the defect area are calculated respectively, which are recorded as the first row mean and the last row mean respectively; the first row mean and the last row mean are subtracted from the minimum depth value in the defect area, and the difference is recorded as H1 and H2;

[0024] Determine whether (H1+H2) / 2 and H1 / H2 are both within a preset value range. If so, mark the defect area as a pore / splash defect area; if not, mark the defect area as other defect areas;

[0025] The preset numerical range is set according to the empirical information of the depth and symmetry of the porosity defect or the spatter defect.

[0026] Preferably, the selected area to be processed is a rectangular area;

[0027] The adjacent region includes a plurality of sub-regions connected to the to-be-processed selected region in the upper, lower and / or left and right directions, or the adjacent region is four neighborhoods or eight neighborhoods of the to-be-processed selected region.

[0028] Preferably, the length of a single side of the to-be-processed selected area is 0.5 mm to 2.5 mm; and the area of ​​a single sub-area in the adjacent area is 0.8 to 1.2 times the area of ​​the to-be-processed selected area.

[0029] Further, the selected area to be processed is a circular area;

[0030] The adjacent area is a circular or rectangular area with the same geometric center as the selected area to be processed, and its area is 1.2 to 2 times the area of ​​the selected area to be processed.

[0031] Preferably, the radius of the selected area to be processed is 0.5 mm to 2.5 mm.

[0032] This method proposes a new filtering method to filter the depth image of the weld area. This filtering method uses the depth mean of the external area that is not connected to the pixel point to be filtered to replace the original depth value, thereby increasing the depth difference between the defective area after filtering and the defective area of ​​the original image. After subtracting from the original image, the position of the defective area in the image is accurately identified; this process greatly reduces the extraction time of the defect candidate area and can realize real-time detection of defects; at the same time, it can also reduce the false negative rate;

[0033] Furthermore, by fitting a parabola to the image points in the defect area, the image points in the non-pore / splash defect area are eliminated, and the pore / splash defect area is accurately identified, thereby reducing the false alarm rate of defects;

[0034] At the same time, the type (pore / splash) of the defect area can be identified by judging the symmetry H1 / H2 and the defect depth (H1+H2) / 2 of the defect area.

[0035] This method is fast, convenient and accurate, and is suitable for the identification and detection of porosity defects in various welds.

[0036] Each filtering method processes the same weld area (800×470), and the comparison is as follows:

[0037] Filter Name Mean Filter Median Filter Gaussian filtering Bilateral filtering This method time consuming 10ms 30ms 32ms 30ms 15ms Minimum size of defect that can be detected 1.5mm 0.4mm 1.0mm 2.0mm 0.3mm

[0038] It can be seen from the above table that the processing method of the present invention is time-saving, can detect smaller defect areas, reduces the missed detection rate, and improves accuracy and detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1a is a grayscale image of the weld area in a specific implementation manner;

[0040] Figure 1b It is a point cloud diagram of the weld area in a specific implementation manner;

[0041] Figure 1c It is a depth diagram of the weld area in a specific implementation manner (partial enlargement);

[0042] Figure 2 It is a schematic diagram of marking the weld defect area after filtering in a specific implementation manner;

[0043] Figure 3 It is a schematic diagram comparing the depth information of pixels after being processed by different filtering methods in a specific implementation manner. DETAILED DESCRIPTION

[0044] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0045] A weld defect detection method based on three-dimensional point cloud comprises the following steps:

[0046] Step 1: Scan the point cloud of the weld area and generate a depth image using the point cloud information;

[0047] Step 2: Filter the depth image in the following way:

[0048] Take any pixel point from the depth image, take it as the center, and select a local area of ​​the depth image, which is recorded as the selected area to be processed. The size of the selected area to be processed is set according to the empirical information of the defect size;

[0049] Then, the selected area to be processed is taken as the central area to find the adjacent area connected to its position; the depth values ​​of each pixel in the adjacent area are averaged to obtain the depth mean; the depth value is the depth value of the pixel in the original depth image;

[0050] The depth value of the pixel at the center of the selected area to be processed is replaced by the depth mean value, so as to filter the center pixel of the selected area to be processed;

[0051] Traverse each pixel of the depth image and use the same filtering method to complete the filtering of the entire depth image;

[0052] like Figure 3 As shown, the difference between the depth value processed by this method and the depth value of the original image increases, thus performing step three;

[0053] Step 3: Calculate the absolute value of the depth difference between each pixel on the filtered depth image and the original depth image, and mark the pixel whose absolute value of the depth difference is greater than a preset value I as a point in the defect area;

[0054] Mark each point in the defect area to obtain the location of the defect area.

[0055] Among them, if the defect is a splash defect, the depth difference is a negative number; the depth difference obtained for other defects is a positive number.

[0056] In specific implementation, the selected area to be processed and the adjacent area can be set in the following ways:

[0057] Method 1: The selected area to be processed is a rectangular area;

[0058] The adjacent area includes a plurality of sub-areas connected to the to-be-processed selected area in the upper, lower and / or left and right directions, or the adjacent area is the eight neighborhoods of the to-be-processed selected area.

[0059] Since the defect is usually 0.3 to 2.5 mm, the single side length of the selected area to be processed is 0.3 mm to 2.5 mm; the area of ​​a single sub-area in the adjacent area is 0.8 to 1.2 times the area of ​​the selected area to be processed.

[0060] For ease of calculation, it is more preferably implemented as follows: the size of the selected area to be processed is (10 to 30 pixels) × (10 to 30 pixels), the adjacent area is the eight neighborhoods of the selected area to be processed, and the areas of the eight sub-areas are consistent with the area of ​​the selected area to be processed.

[0061] Method 2: The selected area to be processed is a circular area; its radius is 0.3mm to 2.5mm;

[0062] The adjacent area is: a circular or rectangular area with the same geometric center as the selected area to be processed, and its area is 1.2 to 2 times the area of ​​the selected area to be processed.

[0063] Since the cross section of the pore / splash defect is parabolic, in order to further optimize and determine the pore / splash defect area, the defect area is restored to the original depth image and the defect area is determined using the following steps:

[0064] Taking the X-axis direction of the image coordinate system as the row direction, a parabola fitting is performed on each row of pixel points in the defect area;

[0065] If the fitted parabola opening size and parabola arc area meet the preset conditions, the corresponding pixel points are marked, otherwise, they are not retained;

[0066] Determine whether there are marked pixels. If yes, count all the marked pixels and record them as the pore / spatter defect area; if no, the defect area is other defects. Other defects include deep pit defects and weld-through defects. Since the size of weld-through defects and deep pit defects is large, after the computer identifies the defects, the defect type can be distinguished and determined by naked eye observation.

[0067] Among them, the following method is used to determine whether the fitted parabola opening size and parabola bow area meet the preset conditions:

[0068] According to the empirical information of pore defects or spatter defects, a parabolic opening size threshold A and a parabolic bow area threshold B are preset. When the fitted parabolic opening size is greater than the threshold A and the fitted parabolic bow area is greater than the threshold B, the preset conditions are met.

[0069] During specific implementation, the least square method or the RANSAC method may be used to perform parabola fitting on each row of pixel points in the defect area.

[0070] In addition, in order to further determine the defect area and identify the defect type, the following methods are used to judge and identify the defect area:

[0071] The defect area is restored to the original depth image, and then the depth mean of the first row of pixels and the depth mean of the last row of pixels in the defect area are calculated respectively, which are recorded as the first row mean and the last row mean respectively; the first row mean and the last row mean are subtracted from the minimum depth value in the defect area, and the difference is recorded as H1 and H2;

[0072] It is used to calculate the depth (H1+H2) / 2 and symmetry H1 / H2 of the porosity defect or spatter defect;

[0073] Determine whether (H1+H2) / 2 and H1 / H2 are both within a preset value range. If so, mark the defect area as a pore / splash defect area; if not, mark the defect area as other defect areas;

[0074] The preset numerical range is set according to the empirical information of the depth and symmetry of the porosity defect or the spatter defect.

[0075] After step 3 is completed, only one of the parabola judgment method and the depth and contrast judgment method can be performed; or the two judgment methods can be performed synchronously / sequentially to jointly judge and identify the pore defect or spatter defect.

[0076] The following are exemplary scenarios:

[0077] A weld defect detection method based on three-dimensional point cloud comprises the following steps:

[0078] Step 1: Scan the weld area (such as Figure 1a ) point cloud (such as Figure 1b ), using point cloud information to generate depth images (such as Figure 1c );

[0079] Step 2: Filter the depth image in the following way:

[0080] Take any pixel point from the depth image, take it as the center, and select a local area of ​​the depth image, which is recorded as the to-be-processed selection area. The to-be-processed selection area is a rectangular area with a size of 15×20px;

[0081] Then, taking the selected area to be processed as the central area, find the adjacent area connected to its position; the adjacent area is the eight neighborhoods of the selected area to be processed, with a size of 20×30px;

[0082] The depth values ​​of each pixel in the adjacent area are averaged to obtain the depth mean; the depth value is the depth value of the pixel in the original depth image;

[0083] The depth value of the pixel at the center of the selected area to be processed is replaced by the depth mean value, so as to filter the center pixel of the selected area to be processed;

[0084] Traverse each pixel of the depth image and use the same filtering method to complete the filtering of the entire depth image;

[0085] Step 3: Obtain the absolute value of the depth difference between each pixel on the filtered depth image and the original depth image, and mark the pixel whose absolute value of the depth difference is greater than a preset value I as a point in the defect area; wherein the preset value I is set to: 8;

[0086] Mark each point in the defect area to obtain the location of the defect area.

[0087] The weld area is filtered using mean filtering, median filtering, Gaussian filtering, bilateral filtering and the present filtering method, and then subtracted from the original depth map; Figure 2 As shown, the pore defect (in the black frame) and the local background area around it are intercepted, and the depth difference of each pixel point is shown in the figure;

[0088] It can be seen from the figure that the depth difference obtained by the mean filter is the smallest, and it is difficult to mark the exact defect position after subtracting it from the original image; the depth difference after median filtering, Gaussian filtering, and bilateral filtering is improved, but points on the defect area will still be missed, resulting in incomplete defect areas; the depth difference after processing by this method is increased, and the extracted defect area is more accurate and complete.

[0089] The defect area obtained in step 3 is further restored to the original depth image, and the defect area is judged using the following steps:

[0090] Taking the X-axis direction of the image coordinate system as the row direction, perform RANSAC parabola fitting on each row of pixels in the defect area;

[0091] If the fitted parabola opening size and parabola arc area meet the preset conditions, the corresponding pixel points are marked, otherwise, they are not retained;

[0092] Determine whether there are marked pixels. If so, count all the marked pixels and record them as the pore / splash defect area; if not, the defect area is other defects.

[0093] Furthermore, the defect area obtained by annotation is restored to the original depth image, and then the depth mean of the first row of pixels and the depth mean of the last row of pixels in the defect area are calculated respectively, which are recorded as the first row mean and the last row mean respectively; the first row mean and the last row mean are subtracted from the minimum depth value in the defect area, and the difference is recorded as H1 and H2;

[0094] It is used to calculate the depth (H1+H2) / 2 and symmetry H1 / H2 of the porosity defect or spatter defect;

[0095] It is determined whether (H1+H2) / 2 and H1 / H2 are both within the preset value range. If so, the defect area is marked as a pore / splash defect area; if not, the defect area is marked as other defect areas.

[0096] The foregoing description of the specific exemplary embodiments of the present invention is presented for the purpose of illustration and description. The foregoing description is not intended to be exhaustive, nor is it intended to limit the present invention to the precise form disclosed, and it is apparent that many changes and variations are possible in light of the above teachings. The exemplary embodiments are selected and described to explain the specific principles of the present invention and its practical application, so that other technicians in the field can realize and utilize the various exemplary embodiments of the present invention and its different selected forms and modified forms. The scope of the present invention is intended to be limited by the appended claims and their equivalents.

Claims

1. A weld defect detection method based on three-dimensional point cloud, characterized in that: The following steps are involved: Step 1: Scan the point cloud of the weld area and generate a depth image using the point cloud information; Step 2: Filter the depth image in the following way: Take any pixel point from the depth image, take it as the center, and select a local area of ​​the depth image, which is recorded as the selected area to be processed. The size of the selected area to be processed is set according to the empirical information of the defect size; Then, taking the selected area to be processed as the central area, find the adjacent area connected to its position; average the depth values ​​of each pixel point in the adjacent area to obtain the depth mean; the depth value is the depth value of the pixel point in the original depth image; The depth value of the pixel point at the center position of the selected area to be processed is replaced with the depth mean value, thereby filtering the center pixel point of the selected area to be processed; Traverse each pixel of the depth image and use the same filtering method to complete the filtering of the entire depth image; Step 3: Calculate the absolute value of the depth difference between each pixel on the filtered depth image and the original depth image, and mark the pixel whose absolute value of the depth difference is greater than a preset value I as a point in the defect area; Mark each point in the defect area to obtain the location of the defect area.

2. The weld defect detection method based on three-dimensional point cloud according to claim 1, characterized in that: Restore the defect area to the original depth image and use the following steps to determine the defect area: Taking the X-axis direction of the image coordinate system as the row direction, a parabola fitting is performed on each row of pixel points in the defect area; If the fitted parabola opening size and parabola arc area meet the preset conditions, the corresponding pixel points are marked, otherwise, they are not retained; Determine whether there are marked pixels. If so, count all the marked pixels and record them as the pore / splash defect area; if not, the defect area is other defects.

3. The method according to claim 2, characterized in that: The method for judging whether the fitted parabola opening size and parabola arc area meet the preset conditions is: According to the empirical information of pore defects or spatter defects, a parabolic opening size threshold A and a parabolic bow area threshold B are preset. When the fitted parabolic opening size is greater than the threshold A and the fitted parabolic bow area is greater than the threshold B, the preset conditions are met.

4. The method according to claim 2, characterized in that: The least square method or RANSAC method is used to perform parabola fitting on each row of pixels in the defect area.

5. The method according to claim 1 or 2, characterized in that: Defective areas can be judged and identified using the following methods: The defect area is restored to the original depth image, and then the depth mean of the first row of pixels and the depth mean of the last row of pixels in the defect area are calculated respectively, which are recorded as the first row mean and the last row mean respectively; the first row mean and the last row mean are subtracted from the minimum depth value in the defect area, and the difference is recorded as H1 and H2; It is determined whether (H1+H2 / 2, H1 / H2 are both within the preset value range. If so, the defect area is marked as a pore / splash defect area; if not, the defect area is marked as other defect areas.

6. The method according to claim 1, characterized in that: The selected area to be processed is a rectangular area; The adjacent region includes a plurality of sub-regions that are connected to the selected area to be processed in the upper, lower and / or left and right directions, or the adjacent region is four neighborhoods or eight neighborhoods of the selected area to be processed.

7. The method according to claim 6, characterized in that: The single side length of the to-be-processed selected area is 0.5 mm to 2.5 mm; the area of ​​a single sub-area in the adjacent area is 0.8 to 1.2 times the area of ​​the to-be-processed selected area.

8. The method according to claim 1, characterized in that: The selected area to be processed is a circular area; The adjacent area is a circular or rectangular area with the same geometric center as the selected area to be processed, and its area is 1.2 to 2 times the area of ​​the selected area to be processed.

9. The method according to claim 8, characterized in that: The radius of the selected area to be processed ranges from 0.5 mm to 2.5 mm.

Images