A two-dimensional and three-dimensional collaborative weld defect detection method

Abstract

The application discloses a kind of two-dimensional three-dimensional collaborative weld defect detection methods, comprising the following steps: one, using line scanning system to weld scanning, obtain the line laser two-dimensional image of weld and carry out numbering to two-dimensional image, the line scanning system includes monocular camera and line laser;Two, two-dimensional pre-processing is carried out to the two-dimensional image obtained in step one, utilize the numbering of two-dimensional image to mark defect position, obtain the two-dimensional image with defect;Three, three-dimensional reconstruction is carried out to the two-dimensional image with defect obtained in step two, obtain three-dimensional model;Four, three-dimensional defect detection is carried out to three-dimensional model obtained in step three using the method of combining region growing based on curvature variation and surface fitting.The two-dimensional three-dimensional collaborative weld defect detection method of the application carries out two-dimensional pre-processing using two-dimensional information before three-dimensional reconstruction, greatly reduces the size of three-dimensional detection model, saves a lot of time, improves detection efficiency, can greatly reduce weld defect detection time.The way of combining region growing based on curvature variation and surface fitting is adopted, the integrity of the extracted defect is guaranteed.

Description

Technical Field

[0001] This invention relates to a two-dimensional and three-dimensional collaborative weld defect detection method, belonging to the field of weld detection technology. Background Technology

[0002] With the dramatic transformation of the automotive market, new energy vehicles are gradually replacing traditional gasoline-powered vehicles. Most new energy vehicles rely on battery modules, while also needing to meet the requirements of lightweight vehicle bodies. Therefore, battery casings are typically made of low-density, high-strength aluminum alloys. However, welding aluminum alloys is difficult, especially welding thin-plate aluminum alloys. Poor weld quality can lead to pinholes or bursts. Battery casing welding quality assessment is a crucial step in battery production, and traditional manual teaching methods are no longer sufficient for current industrial production needs. This paper focuses on an industrial scenario in the production process, researching surface reconstruction technology for detecting minute defects on the battery surface.

[0003] 2D image-based defect detection has been widely applied in industry. Suresh et al. used a linear CCD camera and a high-intensity line light source to detect surface defects on steel plates. Tsai et al. proposed a machine vision method to detect various color defects in low-contrast LCD panel images. Prasanna et al. proposed an algorithm with a partially fine-tuned multi-feature classifier to detect bridge deck cracks. Cruz et al. used a statistical method combining PCA and WMW to select the optimal features, and then used a neural network to classify and identify welding defects. Surface defect detection technology based on 2D images can quickly and effectively detect products with visible defects. However, since 2D images only contain grayscale and texture information of the product surface, they cannot directly reflect three-dimensional features such as the depth and height of defects. 2D images are also susceptible to complex factors such as uneven lighting and product surface materials. In particular, the detection accuracy for small-sized defects is limited, and the robustness of defect detection in complex industrial scenarios still needs improvement.

[0004] Traditional defect detection methods based on 3D point clouds can be broadly categorized into three types according to their principles: registration-based, fitting-based, and region-growing-based methods.

[0005] Registration-based methods primarily involve extracting 3D features from the point clouds of a standard workpiece and the workpiece under test for matching. The overlap of the two point clouds is achieved using Iterative Closest Point (ICP) or Normal Distribution Transform (NDT) algorithms, and the defect is located by calculating the deviation region. Naoki Akai et al. used 3D nondestructive testing scanning matching with experimentally determined uncertainties and landmark matching for localization.

[0006] Fitting-based defect detection methods primarily involve establishing a corresponding mathematical model for the product. We can fit the product surface point cloud data using the least squares method or random consistent sampling to obtain the corresponding model parameters, and then calculate the residual at each point to analyze the product surface quality. This method is suitable for product inspection with simple mathematical models. Fitting-based defect detection methods mainly establish a corresponding mathematical model for the workpiece, fit the workpiece surface point cloud data using least squares or random consistent sampling to obtain the corresponding model parameters, and then calculate the residual at each point to analyze the workpiece surface quality. This method is suitable for the inspection of workpieces with simple mathematical models. Xie et al. fitted the track surface point cloud using the least squares method to obtain its curve equation, and finally derived the track surface defect area based on the residual distribution of the curve points. Region-based segmentation algorithms start from a selected initial seed point and continuously merge the surrounding point cloud according to a given growth criterion to expand the segmented region. In 2006, Rabbani et al. used K-nearest neighbor or fixed radius search to find the nearest neighbor of each point and used this to fit a plane for normal vector estimation and residual value calculation. The residual value was used to describe the local flatness of the point. The seed points were sorted so that the algorithm would start growing from the point with the minimum residual value. The smoothness of the point cloud was constrained by the normal vector angle between adjacent points, thereby obtaining a relatively smooth segmentation surface.

[0007] Therefore, a new two-dimensional and three-dimensional collaborative weld defect detection method is needed to solve the above problems. Summary of the Invention

[0008] The purpose of this invention is to provide a two-dimensional and three-dimensional collaborative weld defect detection method to solve the problems mentioned in the background art.

[0009] A two-dimensional and three-dimensional collaborative weld defect detection method includes the following steps:

[0010] 1. The weld seam is scanned using a line scanning system to obtain a two-dimensional line laser image of the weld seam and the two-dimensional image is numbered. The line scanning system includes a monocular camera and a line laser.

[0011] 2. Perform two-dimensional preprocessing on the two-dimensional image obtained in step one, and use the number of the two-dimensional image to mark the defect location to obtain a two-dimensional image with defects.

[0012] 3. Perform 3D reconstruction on the defective 2D image obtained in step 2 to obtain a 3D model;

[0013] Fourth, a method combining region growing based on curvature change and surface fitting is used to perform three-dimensional defect detection on the three-dimensional model obtained in step three.

[0014] Furthermore, step two, the two-dimensional preprocessing, includes the following steps:

[0015] 21. Cropping the Region of Interest (ROI) in a 2D image to extract the region of interest;

[0016] 22. Dilate the region of interest obtained in step 21 and binarize it to obtain an intermediate image;

[0017] 23. Use the Steger algorithm to extract the centerline of the intermediate image obtained in step 22;

[0018] 24. Use the least squares method to perform curve fitting on the coordinates of the centerline extracted in step 23, and then use the curve fitting degree... Determine if there are defects and set a threshold. When the curve fit is good If the condition is met, the two-dimensional image is defective; otherwise, it is not defective.

[0019] Furthermore, in step 22, the region of interest obtained in step 21 is subjected to a 3-pixel dilation process.

[0020] Furthermore, in step 24, when L is less than 5, then L = 0.

[0021] Furthermore, in step 24, the coordinates of the centerline extracted in step 23 are used to perform a 5th power curve fitting using the least squares method.

[0022] Furthermore, in step 24, the curve fitting degree is calculated using the following formula:

[0023]

[0024] In the formula, L represents the minimum distance from all center points to the fitted curve.

[0025] Furthermore, step three also includes a 3D model downsampling process to obtain a downsampled 3D model.

[0026] Furthermore, in step three, the 3D model is divided into two regions based on the curvature changes: a curvature change threshold is set. Curvature change The region is labeled as the feature region P1, and the curvature changes The region is labeled as feature region P2.

[0027] Furthermore, step four, which combines region growing based on curvature changes with surface fitting, includes the following steps:

[0028] 31. Setting Growth Points and Growth Rules: Feature region P1 is assigned M growth points, and feature region P2 is assigned N growth points; the curvature threshold of feature region P1 is M1, and the curvature change neglect coefficient is M2; the curvature threshold of feature region P2 is N1, and the curvature change neglect coefficient is N2; where M>N, M1 <N1,M2<N2;

[0029] 32. Based on the growth points and growth rules set in step 31, perform region growth on the 3D point cloud to obtain a preliminary defective image;

[0030] 33. Apply the least squares method to the preliminary defect locations obtained in step 32 to perform surface fitting, and obtain the fitted defect image;

[0031] 34. Calculate the Euclidean distance K between the nearest point of each point in the preliminary defective image obtained in step 32 and the nearest point in the fitted defective image obtained in step 33, and set a distance threshold. ,when When that happens, that point is the defect point.

[0032] Beneficial effects: The two-dimensional and three-dimensional collaborative weld defect detection method of this invention utilizes two-dimensional information for two-dimensional preprocessing before three-dimensional reconstruction, which greatly reduces the size of the three-dimensional detection model, saves a significant amount of time, and improves detection efficiency, thus greatly reducing weld defect detection time. The method employs a combination of region growing based on curvature variation and surface fitting, ensuring the integrity of the extracted defects. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the linear scanning system.

[0034] Figure 2 This is a rendering of the reconstructed long-side weld.

[0035] Figure 3 A flowchart of a two-dimensional and three-dimensional collaborative weld defect detection method;

[0036] Figure 4 The 2D image after cropping the ROI;

[0037] Figure 5 The image is a two-dimensional image after dilation and binarization;

[0038] Figure 6 The image is a two-dimensional image after the center line has been extracted;

[0039] Figure 7 This is a graph showing the curve fitting results;

[0040] Figure 8 The results are for region growing and surface fitting.

[0041] Figure 9Extraction results for region growing and region growing with surface fitting;

[0042] Figure 10 The distribution of S-values ​​for all two-dimensional images;

[0043] Figure 11 This is the result of defect extraction using a purely 3D method;

[0044] Figure 12 This is a two-dimensional and three-dimensional collaborative method for weld defect detection. Detailed Implementation

[0045] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0046] Please see Figure 1 As shown, the line scanning system consists of a monocular camera and a line laser. It uses triangulation to reconstruct the weld seam. The battery weld seam is located on the outer contour of the upper surface, and the two long sides and two short sides are reconstructed separately. Based on camera calibration, line structure cursor calibration, and camera-robot system calibration, the captured two-dimensional images are used for three-dimensional reconstruction. This invention extracts one long side of the battery for three-dimensional reconstruction. Figure 2 This is a display image of the reconstructed long-side weld seam in Geomagic software.

[0047] The two-dimensional and three-dimensional collaborative weld defect detection method of the present invention mainly includes: two-dimensional preprocessing and three-dimensional detection.

[0048] Two-dimensional preprocessing is used to roughly determine the location of defects. The two-dimensional preprocessing uses the two-dimensional images before reconstruction and marks each two-dimensional image as a defective or defect-free image through steps such as ROI cropping, centerline extraction, and curve fitting, thereby obtaining the defect location distribution of the entire weld.

[0049] Please see Figure 3 As shown, 3D detection is more accurate than 2D methods and facilitates the acquisition of the specific morphology of defects. By obtaining the defect distribution through 2D preprocessing and then performing 3D detection only on the defective parts, the amount of images required for 3D reconstruction is reduced, thus decreasing the time spent on both 3D reconstruction and defect detection, and improving the overall efficiency of the detection. In this invention, defect detection employs a combination of region growing based on curvature changes and surface fitting, ensuring the completeness of the extracted defects.

[0050] The specific steps of two-dimensional preprocessing are as follows:

[0051] Two-dimensional information processing is much faster than three-dimensional reconstruction. By first using two-dimensional images to roughly determine the location of defects, and then performing three-dimensional reconstruction and defect extraction only on the defective locations, the efficiency of defect detection can be greatly improved.

[0052] Two-dimensional image processing steps:

[0053] First, the acquired camera image is cropped to extract the Region of Interest (ROI). Figure 4 Image (a) is a normal, defect-free two-dimensional image. Figure 4 (b) is a defective two-dimensional image;

[0054] The second step involves performing a 3-pixel dilation and binarization on the image to fully utilize the reflective point information, since reflective points can sometimes be very small. This operation effectively amplifies the weight of reflective points, making subsequent threshold determination more beneficial. Figure 5 In the image, (a) and (b) are the results of dilation and binarization of a defect-free two-dimensional image and a defective two-dimensional image, respectively.

[0055] The third step is to use the Steger algorithm to extract the center line of the image. Figure 6 (a) and (b) show the centerline extraction results for defect-free and defective centers, respectively.

[0056] Finally, the extracted centerline coordinates were fitted with a 5th-order curve using the least squares method, and the degree of curve fitting was used to determine whether there were small holes. Figure 7 In the figures (a) and (b), the curve fitting results are shown for a defect-free image and a defective image, respectively. It is clear from the two figures that the curve fitting results are worse when there are small holes.

[0057] Therefore, defective two-dimensional images can be well distinguished based on their different fits, and these defective images can be marked for subsequent analysis and processing.

[0058] The goodness of fit is represented by the sum of squares (S) of the minimum distances (L) from all center points to the fitted curve, with L values ​​less than 5 being ignored and treated as 0. . Figure 10 Let S represent the distribution of S values ​​for all two-dimensional images, with the vertical axis representing the sum of squares S and the horizontal axis num representing the image number.

[0059] from Figure 10 It can be seen that there are 9 regions with relatively large S values. A suitable threshold should be set. When S > If a defect is detected, 20 two-dimensional images prior to the defect and 20 images following the defect are selected for three-dimensional reconstruction and defect analysis to further accurately determine the size, shape, and location of the small hole. In this invention, a threshold is used. It is 1500.

[0060] The principle behind reducing detection time through 2D preprocessing is to reduce the size of the 3D model; therefore, the processing time of this method is directly related to the number of defects. We selected long-side welds with different numbers of defects for inspection. Table 1 compares the detection time of 2D preprocessing with that of direct 3D detection under different defect numbers. The operating environment was an AMD Ryzen 5 5600X6-Core Processor, and all times are single-threaded execution times. As can be seen from Table 1, in the experimental comparison of detection time with and without 2D preprocessing, the method combining 2D preprocessing reduces the direct 3D detection time to 1 / 2 to 1 / 3. This indicates that using 2D information for 2D preprocessing before 3D reconstruction significantly reduces the size of the 3D detection model, saves a lot of time, and improves detection efficiency.

[0061] Table 1. Detection time with or without two-dimensional preprocessing for different defect numbers.

[0062]

[0063] The specific steps for 3D defect extraction are as follows:

[0064] Defect extraction is essentially a region segmentation problem, and this invention uses the region growing method as a region segmentation method.

[0065] The point cloud obtained by line scanning has different densities along the x-axis and y-axis. Downsampling the 3D point cloud before region growing makes it more uniform and reduces its size. To avoid losing more detail, preprocessing is necessary. The point cloud is roughly divided into two regions based on its curvature variation. The region with larger curvature variation is marked as feature region P1, and a smaller downsampling parameter is set (0.04 in this invention); the region with smaller curvature variation is marked as P2, and a larger downsampling parameter is set (0.1 in this invention). Specifically, a curvature variation threshold is set. The curvature change is 0.034. The region is labeled as the feature region P1, and the curvature changes The region is labeled as feature region P2. This yields a downsampled point cloud, which accelerates point cloud processing while preserving sufficient detail.

[0066] The first step in region growth is the selection of growth points. We set different numbers of growth points for different regions, setting fewer growth points for region P2 and more growth points for region P1.

[0067] The growth rule is crucial to ensuring the effectiveness of the region growth method. Different growth rules with different thresholds should be selected for the growth points in regions P1 and P2. Region P1 is where defects are concentrated; setting a smaller curvature threshold and a more sensitive curvature change neglect coefficient can yield more detailed edge contours. Region P2 needs to maintain connectivity as much as possible, therefore, a larger curvature change coefficient is set. Specifically, in this invention, the curvature threshold for feature region P1 is 0.03, and the curvature change neglect coefficient is 2.8; the curvature threshold for feature region P2 is 0.04, and the curvature change neglect coefficient is 3.5. The curvature change neglect coefficient is calculated by comparing the normal phasor angle of points in a specified range of neighborhoods. If it is less than the normal phasor angle threshold, then the points in this neighborhood and the seed point are on the same plane. The curvature threshold is the maximum curvature set for region growth. If the curvature of a point is less than the curvature threshold, then this point is treated as a seed point and added to the seed sequence.

[0068] Region growing can extract defects with significant curvature changes. However, if a small portion of these defects has insignificant curvature changes, parts of the defect may grow together with normal regions, forming connected regions—a case of overgrowth. This will affect subsequent defect feature analysis. For example... Figure 8 As shown in (b), the small change in curvature at the lower edge of the defect causes the outer contour to grow into the interior of the defect.

[0069] This invention combines surface fitting analysis methods to supplement the region growing method.

[0070] Figure 8 Image (a) shows the reconstructed 3D model of the defective part, denoted as CloudA. The least squares method is used to fit a surface to the points near the defect, resulting in the fitted point cloud CloudB. The resulting image is shown below. Figure 8 As shown in (c), because surface fitting makes the point cloud model smoother overall, the original point cloud and the filtered point cloud will be misaligned in spatial distribution at the defect location, while they will almost overlap in other areas. A portion of the point cloud at the top of the defect can be extracted by the difference between the two.

[0071] Specific method: Using points in point cloud CloudA as indices, calculate the Euclidean distance K between each point in CloudA and the nearest point in point cloud CloudB. Set an appropriate distance threshold K0 based on the point cloud accuracy; points with an Euclidean distance K greater than the threshold K0 are considered defective points. In this invention, the distance threshold... =0.15. Figure 8 In the middle (d), the defects are extracted by the surface fitting method.

[0072] The Euclidean distance in three-dimensional space is In the formula, , Let be points in two three-dimensional spaces.

[0073] Combining surface fitting methods can effectively compensate for incomplete defect information caused by excessive region growth. The results extracted by the surface fitting method are also labeled as defects and added to the point cloud obtained from region growth. Figure 9 To extract results using surface fitting, this method can effectively address the issue of excessive region growth.

[0074] To verify the performance of the 3D detection method, samples with defects of different diameters were selected for testing. Table 2 shows the recognition rate of defects with different diameters. The recognition rate of defects with a diameter of 1 mm or more exceeded 90%, and the recognition rate of defects with a diameter of 0.5 mm or more was also above 80%. The experimental results indicate that the defect detection method based on the combination of region growing and surface fitting according to curvature variation has good robustness.

[0075] Table 2 Defect identification rate for different diameters

[0076]

[0077] Please see Figure 11 and Figure 12 As shown, this invention employs a combination of region growing and surface fitting to detect defects. To improve efficiency, a two-dimensional preprocessing method is introduced to assist in three-dimensional defect detection, achieving efficient and high-quality extraction of surface defects in battery welds. The core of the two-dimensional preprocessing is curve fitting of a single line laser image. The light reflection information from the line laser on the weld surface is used to roughly determine the defect location, and three-dimensional reconstruction and defect detection are performed only near the defect. Since two-dimensional detection is much faster than three-dimensional methods, two-dimensional preprocessing can significantly reduce defect detection time. Applying this invention to the detection of surface weld defects in new energy batteries accelerates the process of evaluating the welding quality of new energy battery casings.

Claims

1. A two-dimensional and three-dimensional collaborative method for weld defect detection, characterized in that, The steps include the following:

1. The weld seam is scanned using a line scanning system to obtain a two-dimensional line laser image of the weld seam and the two-dimensional image is numbered. The line scanning system includes a monocular camera and a line laser.

2. Perform two-dimensional preprocessing on the two-dimensional image obtained in step one, and use the number of the two-dimensional image to mark the defect location to obtain a two-dimensional image with defects. Step two, the two-dimensional preprocessing, includes the following steps: (21) Cropping the ROI of the two-dimensional image to extract the region of interest; (22) Dilate the region of interest obtained in step 21 and binarize it to obtain an intermediate image; (23) Use the Steger algorithm to extract the center line of the intermediate image obtained in step 22; (24) Use the least squares method to perform curve fitting on the coordinates of the centerline extracted in step 23, and use the curve fitting degree... Determine if there are defects and set a threshold. When the curve fit is good If the condition is met, the two-dimensional image is defective; otherwise, it is not defective.

3. Perform 3D reconstruction on the defective 2D image obtained in step 2 to obtain a 3D model; Fourth, a method combining region growing based on curvature change and surface fitting is used to perform three-dimensional defect detection on the three-dimensional model obtained in step three.

2. The two-dimensional and three-dimensional collaborative weld defect detection method as described in claim 1, characterized in that: In step 22, the region of interest obtained in step 21 is dilated by 3 pixels.

3. The two-dimensional and three-dimensional collaborative weld defect detection method as described in claim 1, characterized in that: In step 24, when L is less than 5, then set L=0.

4. The two-dimensional and three-dimensional collaborative weld defect detection method as described in claim 1, characterized in that: In step 24, the coordinates of the centerline extracted in step 23 are fitted to a fifth power curve using the least squares method.

5. The two-dimensional and three-dimensional collaborative weld defect detection method as described in claim 1, characterized in that: In step 24, the curve fitting degree is calculated using the following formula: In the formula, L represents the minimum distance from all center points to the fitted curve.

6. The two-dimensional and three-dimensional collaborative weld defect detection method as described in claim 1, characterized in that: Step three also includes a 3D model downsampling process to obtain a downsampled 3D model.

7. The two-dimensional and three-dimensional collaborative weld defect detection method as described in claim 1, characterized in that: In step three, the 3D model is divided into two regions based on the curvature changes: a curvature change threshold is set. Curvature change The region is labeled as feature region P1, and the curvature changes The region is labeled as feature region P2.

8. The two-dimensional and three-dimensional collaborative weld defect detection method as described in claim 7, characterized in that: Step four, the method combining region growing and surface fitting based on curvature variation, includes the following steps: (31) Setting growth points and growth rules: M growth points are set for feature region P1, and N growth points are set for feature region P2; the curvature threshold of feature region P1 is M1, and the curvature change neglect coefficient is M2; the curvature threshold of feature region P2 is N1, and the curvature change neglect coefficient is N2; where M>N, M1 <N1,M2<N2; (32) Based on the growth points and growth rules set in step 31, perform region growth on the three-dimensional point cloud to obtain a preliminary defective image; (33) The preliminary defective locations obtained in step 32 are fitted with surfaces using the least squares method to obtain the fitted defective image; (34) Calculate the Euclidean distance K between each point of the preliminary defective image obtained in step 32 and the nearest point of the fitted defective image obtained in step 33, and set a distance threshold. ,when If the time is right, then that point is the defect point.

Images