Weld joint forming visual inspection method based on surface topography three-dimensional reconstruction

Abstract

The invention discloses a method for performing visual inspection on surface forming quality characteristics of a welding seam by utilizing three-dimensional reconstruction, which comprises the following steps of: S1, performing uniform-speed scanning on the welding seam along the direction of a welding bead by using linear structured light emitted by a linear laser, and performing image acquisition on linear structured light stripes formed on the surface of the welding seam by using an industrial camera; s2, converting the acquired RGB image into a gray level image frame by frame, and carrying out image distortion correction and image denoising processing; s3, carrying out ROI (Region of Interest) positioning on the formed line structured light stripe image on the surface of the welding seam by adopting a pixel point gray level distribution calculation method; s4, carrying out image segmentation on the ROI region of the weld line structured light stripe image; and S5, carrying out sub-pixel-level center line extraction on the ROI of the weld line structured light stripe image by adopting a gray extreme value weighted centroid method. Automatic detection of welding seam forming quality characteristics such as laser welding and electric arc welding can be achieved, and the operation efficiency of a production line and the welding seam quality detection precision can be greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the intelligent manufacturing technology applied to the weld quality detection of arc welding, laser welding and the like, and particularly relates to a method for visually detecting the forming quality features of a weld surface by three-dimensional reconstruction. BACKGROUND

[0002] As a main connecting process for metal component processing, welding technology plays an irreplaceable role in many industrial fields, and its application ranges cover mechanical manufacturing, nuclear industry, aerospace, energy transportation, petrochemical industry, shipbuilding, automobile, and building and electronics. In actual welding, due to the selection of welding methods, the setting of process parameters, the complexity of the operating environment, and the skill level of the operators, and many other factors, the welding joints often have forming quality problems such as uneven weld width, undercut, and weld protrusions. Such problems can cause the effective load-bearing area of the weld to decrease, stress to concentrate, and assembly to be difficult during the use of the product, thereby directly affecting the strength, stability, and service life of the structure. With the development of science and technology, welding automation and intelligence are gradually improved, and welding robots are widely used in various welding occasions. However, the detection of the appearance quality of the weld still relies on manual measurement of the weld width, excess, and other parameters for judgment. This method has strong subjectivity, low efficiency, insufficient measurement accuracy, and limited recognition accuracy, and cannot meet the needs of modern production for high-precision and high-efficiency detection. SUMMARY

[0003] The purpose of the present application is to provide a weld forming visual detection method based on surface topography three-dimensional reconstruction, which can greatly improve the production line operation efficiency, improve the measurement accuracy of the weld forming quality features, and realize automatic measurement of the weld forming quality.

[0004] The weld forming visual detection method based on surface topography three-dimensional reconstruction comprises the following steps:

[0005] S1, the line laser emits line structured light to uniformly scan along the weld direction at a speed perpendicular to the weld, and at the same time, an industrial camera is used to collect the line structured light stripe formed on the weld surface;

[0006] S2, the collected RGB image is converted into a gray image frame by frame, the image distortion caused by the camera lens distortion is corrected using the camera intrinsic parameters and distortion coefficients, and the image is denoised using median filtering;

[0007] S3, the pixel point gray scale distribution calculation method is used to position the ROI of the weld seam surface forming line structure light stripe image;

[0008] S4, the image segmentation is carried out on the ROI region of the weld seam line structure light stripe image;

[0009] S5, the gray scale extreme weighted gravity center method is used to extract the center line of the ROI region of the weld seam line structure light stripe image at the sub-pixel level;

[0010] S6, the adaptive feature point extraction algorithm based on distance evolution is used to position the weld seam feature points of the center line of the ROI region of the weld seam line structure light stripe image;

[0011] S7, the three-dimensional coordinates of all feature points are obtained after the center line of the weld seam line structure light stripe image is calibrated, all the feature points are drawn in the three-dimensional space coordinates, and the three-dimensional reconstruction structure of the weld seam surface forming is obtained;

[0012] S8, the pixel coordinates of the feature points are converted into the camera coordinates in the camera coordinate system, the weld forming features are measured according to the feature point distribution, the corresponding forming feature information and defect feature information of the weld seam are obtained.

[0013] Further, the pixel point gray scale distribution calculation method is that the number of pixel point gray scale values above the threshold value in the row and column of the weld seam line structure light stripe gray scale image is calculated, the row coordinates of the first occurrence and the last occurrence of the gray scale threshold value are selected to represent the row boundary, and the column coordinates of the first occurrence and the last occurrence of the gray scale threshold value are selected to represent the column boundary, so as to position the ROI region.

[0014] Further, the image segmentation method uses the following calculation formula to obtain the best image segmentation effect:

[0015]

[0016] In the formula, p0(T) is the probability of the foreground pixel point in the weld seam line structure light stripe image, p1(T) is the probability of the background pixel point in the weld seam line structure light stripe image, μ0(T) is the average gray scale value of the foreground in the weld seam line structure light stripe image, and μ1(T) is the average gray scale value of the background in the weld seam line structure light stripe image.

[0017] Further, the gray scale extreme weighted gravity center method is that the gray scale extreme weighted gravity center method is used to extract the center line of the ROI region of the weld seam line structure light stripe image at the sub-pixel level.

[0018] S1, search the gray value extreme value of the weld line structured light fringe image ROI region column by column, select the five highest gray value points and record their coordinates (x j , y1), (x j , y2), (x j , y3), (x j , y4), (x j , y5);

[0019] S2, according to the above pixel point coordinates, the minimum value x min and the maximum value x max of the row coordinates are obtained, and the center position row coordinate of the above pixel points is obtained according to the following formula;

[0020]

[0021] S3, for the selected current column pixel point, the distance d i of the pixel point to the center position and the Gaussian weight w i of each pixel are calculated according to the following formula, respectively, and the closer to the center of the pixel point, the higher the weight, wherein, control the width of the Gaussian function;

[0022]

[0023]

[0024] S4, for the jth column, the weighted center is calculated by the following formula;

[0025]

[0026] S5, repeat steps S1 and S2, traverse all columns to obtain the center line position (x i , y c ) of each column, and form a center line point set as shown in the following formula.

[0027]

[0028] Further, the weld forming visual detection method based on surface topography three-dimensional reconstruction has the characteristics that the adaptive feature point extraction algorithm based on distance evolution specifically includes three steps of fitting the base material contour line, rough positioning and fine positioning.

[0029] Further, the fitting base material contour line has the characteristics that the following specific steps are adopted:

[0030] S1, extract the highest point A of the weld line structured light fringe image center line, that is, the point with the maximum y value, and divide the center line into Left and Right classes as the dividing line to fit respectively;

[0031] S2, randomly select two points from the center line data to calculate the slope and intercept of the straight line model;

[0032] S3, automatically select a suitable threshold T according to the data characteristics, calculate the error between each point on the center line and the model according to the model obtained in the previous step, and mark the data points with an error less than the threshold T as "inliers", and count the number of inliers corresponding to the current model;

[0033] S4, after multiple random sampling and evaluation until a certain number of iterations is reached, select the model with the most inliers as the best model, and perform least squares fitting on the inlier set of the best model to improve the fitting accuracy of the model, and obtain the final left fitting straight line L left and the right fitting straight line L right .

[0034] Further, the coarse positioning, characterized in that the following specific steps are adopted:

[0035] S1, calculate the distance of all pixel points on the center line of the weld line structured light stripe image to the fitting straight line L left and L right , determine the pixel point corresponding to the maximum distance as the weld vertex O, and extract feature points from the left and right sides respectively with O as the starting point.

[0036] S2, in the left weld feature point extraction, take O point as the starting point, traverse the pixel points on the left center line, calculate the vertical distance d of the pixel points to the fitting straight line L left point by point, and extract feature points based on the symbol mutation judgment method and the fluctuation count judgment method in parallel;

[0037] S3, start the recursive extraction mechanism, repeat the above steps to find new feature points;

[0038] S4, in the right feature point extraction, take O point as the starting point, traverse the pixel points on the right center line, and complete the feature point extraction using the same rules as the left side.

[0039] Further, the fine positioning, characterized in that the following specific steps are adopted:

[0040] S1, mark the last weld feature points on the left and right sides obtained by coarse positioning as W l and W r ;

[0041] S2, draw a perpendicular line from the O point to the base material far away from it, and the intersection points of the perpendicular line with the left and right fitting straight lines are recorded as J l and J r ;

[0042] S3, for all the center points between O and W l (W r ), get the point farthest from the straight line OW l (OW r ), recorded as T l (T r );

[0043] S4, if there is a weld feature point between O and W l (W r ), then all the points between adjacent feature points are recorded as point set D.

[0044] Further, the symbol mutation determination method, characterized in that: if the current point distance d and the previous point distance d pre change in sign, that is, d pre ≥ 0 > d or d pre ≤ 0 < d, the starting point of the sign change is recorded as a feature point P l .

[0045] Further, the wave count determination method, characterized in that: the wave counter continuously counts the pixel points whose |d| is less than the preset threshold value, and when the number of points continuously meeting the condition reaches the preset value, the first point in the counting interval is recorded as a feature point P l .

[0046] Compared with the prior art, the present application has the following beneficial effects:

[0047] (1) The method can reduce the labor load of the operator, greatly improve the operation efficiency of the production line, and meet the development needs of the intelligent transformation of modern welding manufacturing;

[0048] (2) The method can improve the measurement accuracy of the weld forming quality parameter, reduce the error caused by subjectivity, and increase the reliability of the quality detection of the welded structure;

[0049] (3) The method can realize automatic measurement of weld forming quality, and fill the gap in automatic measurement and quantization of defects such as uneven weld width, weld edge biting, and weld tumor;

[0050] (4) The method can be used for high-efficiency welding manufacturing and intelligent welding quality detection based on industrial robots. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 is a schematic diagram of the structure of a weld surface forming quality feature visual detection system based on three-dimensional reconstruction.

[0052] Figure 2is the filter and image segmentation result of the weld line structured light stripe image.

[0053] Figure 3 the result of extracting the weld line structured light stripe image center line by the gray extremum weighted barycenter method.

[0054] Figure 4 the weld feature point positioning result obtained by the adaptive feature point extraction algorithm based on distance evolution.

[0055] Figure 5 the three-dimensional reconstruction result of the weld surface forming structure.

[0056] Figure 6 the weld forming undercut defect feature extraction result.

[0057] In the figure, 1 is an industrial camera, 2 is a line laser, 3 is an image acquisition unit, 4 is a motor, 5 is a motion control unit, and 6 is a computer processing unit. DETAILED DESCRIPTION

[0058] The present application will be described in detail below in combination with the drawings and specific embodiments.

[0059] A weld forming visual detection method based on surface topography three-dimensional reconstruction, referring to Figure 1 The detection mechanism is composed of an industrial camera 1, a line laser 2, a motor 4, a motion control unit 5, and a computer processing unit 6, and adopts a slanting triangulation principle to automatically detect the weld forming surface quality features. The weld sample to be detected is a 2.0mm thick 6061 aluminum alloy butt joint plate, and the welding method adopted is a gas metal arc welding (MAG welding). The method comprises the following steps:

[0060] S1, the line laser 2 emits line structured light to uniformly scan along the weld direction at a right angle to the weld, and at the same time, the industrial camera 1 is set to an exposure time of 200ms to acquire images of the line structured light stripe formed on the weld surface;

[0061] S2, the acquired RGB images are converted into gray scale images frame by frame, the image distortion caused by the camera lens distortion is corrected using the camera intrinsic parameters and distortion coefficients, and the image is denoised using a median filter;

[0062] S3, the pixel gray scale distribution calculation method is used to position the ROI of the weld surface forming line structured light stripe image;

[0063] S4, the weld line structured light stripe image ROI region is subjected to image segmentation, and the obtained weld surface forming line structured light stripe image is as shown in Figure 2 ;

[0064] S5, using the grayscale extreme value weighting method, sub-pixel level centerline extraction is performed on the ROI region of the weld seam structured light stripe image. The resulting centerline of the weld seam structured light stripe image is as follows: Figure 3 As shown;

[0065] S6, an adaptive feature point extraction algorithm based on distance evolution is used to locate weld feature points in the center line of the ROI region of the weld seam structured light stripe image. The weld feature point localization results are as follows: Figure 4 As shown;

[0066] S7, after systematically calibrating the center line of the weld seam structured light stripe image, obtain the three-dimensional coordinates of all feature points. Plot all the obtained feature points in three-dimensional space coordinates to obtain the three-dimensional reconstructed structure of the weld seam surface formation, as shown in Figure 7. Figure 5 As shown;

[0067] S8, convert the pixel coordinates of the feature points to camera coordinates in the camera coordinate system, and measure the weld undercut defect features based on the feature point distribution to obtain the size distribution information of the corresponding left and right undercut defects of the weld. Figure 6 As shown.

[0068] The above embodiments are descriptions of typical preferred embodiments of the present invention, but the technical solutions of the present invention are not limited thereto. Any changes and modifications made by those skilled in the art based on the main technical concept of the present invention will fall within the technical scope to be protected by the present invention.

[0069] The visual inspection method for weld formation based on three-dimensional reconstruction of surface morphology proposed in this invention can be integrated with automated welding production lines, reducing the interference of manual labor intensity and human subjective will on welding quality inspection results, improving inspection efficiency and accuracy, and has broad prospects for promotion and application.

Claims

1. A visual inspection method for weld formation based on three-dimensional reconstruction of surface morphology, characterized in that, A mechanism consisting of a line laser, an industrial camera, a motor, a motion control unit, and a computer processing unit uses the principle of oblique-shot triangulation to automatically detect the surface quality characteristics of weld formation. The method includes the following steps: S1, the line laser emits line structured light to scan at a constant speed along the weld bead direction in a direction perpendicular to the weld seam. At the same time, an industrial camera is used to acquire images of the line structured light stripes formed on the weld seam surface. S2 converts the acquired RGB images frame by frame into grayscale images, uses camera intrinsic parameters and distortion coefficients to correct image distortion caused by camera lens distortion, and uses median filtering to denoise the images; S3, the pixel grayscale distribution calculation method is used to locate the ROI of the structured light stripe image of the weld surface forming line; S4, perform image segmentation on the ROI region of the weld seam structured light stripe image; S5, using the grayscale extreme value weighting method to extract the center line of the ROI region of the weld seam structured light stripe image at the subpixel level; S6, an adaptive feature point extraction algorithm based on distance evolution is used to locate the weld feature points in the center line of the ROI region of the weld seam structured light stripe image; S7. After systematically calibrating the center line of the weld seam structure light stripe image, obtain the three-dimensional coordinates of all feature points. Draw all the obtained feature points in the three-dimensional space coordinates to obtain the three-dimensional reconstruction structure of the weld seam surface. S8 converts the pixel coordinates of the feature points into camera coordinates in the camera coordinate system, and measures the weld formation features based on the feature point distribution to obtain the corresponding weld formation feature information and defect feature information.

2. The visual inspection method for weld formation based on three-dimensional reconstruction of surface morphology according to claim 1, characterized in that, The pixel grayscale distribution calculation method is as follows: calculate the number of pixels with grayscale values ​​above the threshold in the row and column of the grayscale image of the weld seam structure light stripe, select the row coordinates of the first and last occurrence of the grayscale threshold to represent the row boundary, and select the column coordinates of the first and last occurrence of the grayscale threshold to represent the column boundary, thereby locating the ROI region.

3. The visual inspection method for weld formation based on three-dimensional reconstruction of surface morphology according to claim 1, characterized in that, The image segmentation method uses the following calculation formula to obtain the best image segmentation effect: ; In the formula, p0(T) is the probability of a pixel belonging to the foreground in the structured light stripe image of the weld line, p1(T) is the probability of a pixel belonging to the background in the structured light stripe image of the weld line, μ0(T) is the average gray value of the foreground in the structured light stripe image of the weld line, and μ1(T) is the average gray value of the background in the structured light stripe image of the weld line.

4. The visual inspection method for weld formation based on three-dimensional reconstruction of surface morphology according to claim 1, characterized in that, The gray-level extreme value weighting method is as follows: S1, Search for extreme gray values ​​column by column in the ROI region of the weld seam structured light stripe image, select the five pixels with the highest gray values ​​and record their coordinates (x, y, y). j , y1), (x j , y2), (x j , y3), (x j , y4), (x j , y5); S2, based on the pixel coordinates mentioned above, obtain the minimum value x of the row coordinate. min and the maximum value x max The center position row coordinates of the above pixels can be obtained according to the following formula; ; S3, for the selected pixel in the current column, calculate the distance d from the pixel to the center position according to the following formula. i and the Gaussian weight w for each pixel i The closer the pixel is to the center, the higher its weight. Control the width of the Gaussian function; ; ; S4, calculate the weighted center of column j using the following formula; ; S5, Repeat steps S1 and S2, iterating through all columns to obtain the centerline position (x) of each column. i ,y c The centerline point set is formed as shown in the following formula: 。 5. The visual inspection method for weld formation based on three-dimensional reconstruction of surface morphology according to claim 1, characterized in that: The adaptive feature point extraction algorithm based on distance evolution specifically includes three steps: fitting the outline of the parent material, coarse positioning, and fine positioning.

6. The visual inspection method for weld formation based on three-dimensional reconstruction of surface morphology according to claim 5, characterized in that, in, The following specific steps are used to fit the contour line of the base material: S1. Extract the highest point A of the center line of the weld seam structured light stripe image, that is, the point with the largest y value. Use the column where point A is located as the dividing line to divide the center line into Left and Right classes and fit them respectively. S2, randomly select two points from the centerline data to calculate the slope and intercept of the straight line model; S3. Automatically select an appropriate threshold T based on the data characteristics, calculate the error between each point on the center line and the model based on the model obtained in the previous step, record data points with errors less than the threshold T as "inner points", and count the number of inner points corresponding to the current model. S4. After multiple random sampling and evaluations until a certain number of iterations are reached, the model with the most interior points is selected as the optimal model. The interior point set of the optimal model is then fitted using the least squares method to improve the model's fitting accuracy, resulting in the final left-side fitted line L. left And the fitted line L on the right right .

7. The visual inspection method for weld formation based on three-dimensional reconstruction of surface morphology according to claim 5, characterized in that, in, Coarse positioning is performed using the following specific steps: S1, calculate the line L from all pixels on the center line of the weld seam structured light stripe image to the fitted straight line. left and L right The distance is used to determine the pixel point corresponding to the maximum distance as the weld vertex O, and feature points are extracted to the left and right sides from this point. S2, in the extraction of feature points of the left weld, starting from point O, traverse the pixels on the left center line and calculate the distance from each pixel to the fitted line L. left The vertical distance d is calculated, and feature points are extracted in parallel based on the symbolic mutation determination method and the fluctuation counting determination method; S3, initiate the recursive extraction mechanism and repeat the above steps to find new feature points; S4. In the feature point extraction on the right side, starting from point O, traverse the pixels on the center line on the right side and use the same rules as on the left side to complete the feature point extraction.

8. The visual inspection method for weld formation based on three-dimensional reconstruction of surface morphology according to claim 5, characterized in that, in, For precise positioning, the following specific steps are adopted: S1, mark the last weld feature point on the left and right sides obtained from the coarse positioning as W. l W r ; S2, draw a perpendicular line from point O to the base material that is farther away from it. The intersection points of this perpendicular line with the fitted lines on the left and right sides are denoted as J, respectively. l and J r ; S3, for O and W l (W) r Find the distance from the center point of the line OW between all the centers of the two points. l (OW) r The largest point is denoted as T. l (T) r ); S4, if O and W l (W) r If there are weld feature points between adjacent feature points, then all points between adjacent feature points are denoted as point set D.

9. The visual inspection method for weld formation based on three-dimensional reconstruction of surface morphology according to claim 7, characterized in that, in, Symbol mutation determination method, including: if the distance d of the current point and the distance d of the previous point pre undergo a sign change, that is, d pre ≥0 > d or d pre ≤0 < d, then record the starting point of the sign change as the feature point P l .

10. The visual inspection method for weld formation based on three-dimensional reconstruction of surface morphology according to claim 7, characterized in that, in, The fluctuation counting method includes: continuously counting pixels whose |d| is less than a preset threshold using a fluctuation counter; when the number of consecutively satisfied points reaches a preset value, the first point in the counting interval is recorded as feature point P. l .

Images