Robust extraction method of centerline of structured light for fillet weld under complex background

Abstract

This invention relates to the field of machine vision technology and discloses a robust extraction method for the center line of structured light in fillet welds. It aims to solve the problem of missegmentation and loss of corner features caused by traditional algorithms relying on geometric connectivity segmentation under strong arc light and high reflectivity interference. The invention first obtains a light stripe backbone mask through multi-scale morphological filtering and clustering screening; secondly, it constructs a scattered data pool by performing global coarse extraction within the mask; subsequently, it uses a sequential random sampling consensus algorithm to adaptively extract the equations of the upper and lower main lines and calculate the theoretical intersection point through a "fit-stripping" iterative mechanism; finally, it extends along the main line towards the intersection point and performs independent pixel grayscale thresholding and sub-pixel fine extraction within a local window of the original image. This invention completely eliminates the risk of missegmentation, accurately repairing large-scale reflective fractures while perfectly preserving and reconstructing the sharp corner features of the fillet weld.

Description

Technical Field

[0001] This invention relates to the field of machine vision and image processing technology, specifically to a robust method for extracting the structure light centerline of a fillet weld in a complex background, applicable to visual weld tracking and three-dimensional shape measurement of industrial robots. Background Technology

[0002] In modern industrial automated welding, laser structured light-based visual sensing technology is key to achieving weld seam tracking. For fillet welds (V-shaped or L-shaped), the angular features formed by the structured light stripes at the weld root serve as the absolute reference for welding torch positioning.

[0003] In actual welding environments, there is intense arc light, large areas of metal spatter, and the inside of the corner joint is prone to mirror reflection.

[0004] These interferences cause severe local distortions, burrs, and even large-scale fractures in the extracted light stripe centerline.

[0005] Traditional centerline extraction algorithms (such as the extreme value method and the single gray-scale centroid method) are prone to failure or being misled by splash noise under strong interference.

[0006] To repair fractures and suppress noise, existing technologies often employ moving average or Savitzky-Golay (SG) smoothing filtering for post-processing.

[0007] However, while such local filtering algorithms smooth high-frequency noise, they are prone to smoothing the sharp corner features of fillet welds into rounded corners, resulting in the loss of root feature points.

[0008] Furthermore, conventional neighborhood tracking extension algorithms are prone to trajectory deviation or even tracking termination when encountering long-distance reflective breaks due to the lack of mathematical model constraints. Summary of the Invention

[0009] This invention aims to overcome the shortcomings of existing technologies and provide a robust method for extracting the structured light centerline of fillet welds under complex backgrounds. This method introduces a global RANSAC geometric fitting and independent pixel verification mechanism, accurately bridging large-scale fractures while preserving and enhancing the angle features of the weld. The specific implementation steps of the technical solution adopted by this invention to solve the technical problem are as follows:

[0010] Robust extraction methods for the centerline of structured fillet welds under complex backgrounds include:

[0011] Step S1: Image preprocessing and mask generation based on multi-scale morphology.

[0012] Let the acquired original fillet weld structured light grayscale image be... To remove the strong arc light background and highlight the structured light, a length of [length missing] was used. Horizontal linear structural elements Perform a top-hat transform on the image:

[0013]

[0014] in This represents the image after the top-hat transformation; This indicates the morphological opening operation.

[0015] To eliminate lateral noise caused by splashing, a length of [missing information] was used. Vertical line structural elements right Perform an opening operation to suppress the splatter light streaks and obtain a denoised image. :

[0016]

[0017] In the formula, and These are erosion and expansion operations, respectively.

[0018] Subsequently, a closing operation was used for bridging and repair, and a threshold was set for binarization to obtain the matrix. Perform connected component analysis and take the x-coordinate of the centroid of the connected component with the largest area. Based on this, traverse all connected components. If its centroid x-coordinate and area The region will be preserved if the following conditions are met, ultimately resulting in a pure light stripe backbone mask. .

[0019]

[0020] in, This indicates the set horizontal position tolerance; This indicates the set minimum area threshold.

[0021] Step S2: Global coarse extraction without prior knowledge. (In the mask) Within the effective area, the basic gray-level centroid method is used to calculate line by line to obtain the set of global coarse light stripe center points without segmentation. This step does not rely on geometric connectivity for hard segmentation, but instead constructs a global scatter plot data pool.

[0022] Step S3: Adaptive segmentation and global fitting based on sequence RANSAC. The set... As a global scatter plot data pool, let the linear model be... .

[0023] First iteration of fitting: Randomly select two points from the pool and calculate parameters. and Calculate the horizontal residuals from all points to the line:

[0024]

[0025] Among them, among them, and They represent the first The x and y coordinates of the scattered points; and Indicates the first The parameters of the linear model formed by sampling.

[0026] Will satisfy ( Points that fall below the in-point tolerance threshold are classified as in-points.

[0027] After a predetermined number of iterations, the set containing the largest number of local points is selected. Using the least squares method to... Perform parameter reestimation and solve the objective function to obtain the first principal parameters:

[0028]

[0029] And Remove from the global pool to obtain the remaining scatter set:

[0030]

[0031] Second iteration fitting: in The same sampling consensus algorithm is executed to extract the local point set of the second principal line and re-estimate the parameters.

[0032] Determining the Upper and Lower Segments: Compare the average ordinates of the two extracted main line intrapoints; the one with the smaller average is determined to be the upper segment trend line equation.

[0033]

[0034] The equation of the lower trend line is determined by the larger mean:

[0035]

[0036] Step S4: Solving for the intersection point using analytical geometry. Solve the equations of the two trend lines simultaneously to obtain the ordinate of the theoretical analytical intersection point of the fillet weld angle. :

[0037]

[0038] in, This represents the ordinate of the intersection point of the two lines in analytic geometric space.

[0039] Step S5: Trend-guided independent pixel verification and sub-pixel fine extraction. Taking the upper light stripe as an example, predict line by line from the end coordinate of the upper segment downwards along the trend line. The theoretical horizontal coordinate... for:

[0040]

[0041] In the original image without morphological destruction In China, with Create a local search window W centered on the element. Obtain the highest grayscale value within the local area of ​​the window. :

[0042]

[0043] Perform independent pixel threshold review: If ( (If the foreground grayscale threshold is used), then those that meet the criteria will be selected. A subset of pixels (with a high grayscale scaling factor) The judgment conditions are as follows:

[0044]

[0045] Calculate subpixel grayscale centroid:

[0046]

[0047] Among them, among them, This represents the calculated x-coordinate of the sub-pixel center point.

[0048] like If the predicted point is located in a real dark spot area, the current line is skipped, and the next line is examined. This mechanism eliminates the traditional drawback of terminating the extension upon encountering a dark area. An upward symmetrical operation is performed on the lower light stripe, ultimately outputting the complete fillet weld structured light centerline.

[0049] The beneficial effects of this invention are:

[0050] (1) Strong anti-interference and locked corner features: RANSAC global fitting is used to replace local smoothing, which fundamentally solves the mathematical contradiction that the traditional smoothing algorithm causes the fillet weld features to become blunt. Even if there is severe spatter and deformation, the straight line trend can be accurately reconstructed.

[0051] (2) Extremely high robustness of broken connection repair: The original image independent pixel review mechanism is introduced to refine the image. The trend line is allowed to cross isolated dark spots without being interrupted, which fills in large-scale reflective breaks while avoiding the erroneous extraction caused by traditional mindless extrapolation. Attached Figure Description

[0052] Figure 1 is a flowchart of a robust method for extracting the center line of a fillet weld structure under complex backgrounds, provided by an embodiment of the present invention.

[0053] Figure 2 is a grayscale image of the original fillet weld structure in an embodiment of the present invention.

[0054] Figure 3 shows the arc-removed image after horizontal top-hat filtering in an embodiment of the present invention.

[0055] Figure 4 shows the desplattered image after vertical opening operation processing according to an embodiment of the present invention.

[0056] Figure 5 shows the image after bridging repair using the closing operation in an embodiment of the present invention.

[0057] Figure 6 shows the image after binarization processing according to an embodiment of the present invention.

[0058] Figure 7 shows the pure mask image of the light stripe backbone obtained after clustering and screening in an embodiment of the present invention.

[0059] Figure 8 is a sequence of coarse light stripe center points obtained by coarse extraction within the mask area according to an embodiment of the present invention.

[0060] Figure 9 is a schematic diagram of the global robust piecewise fitting and theoretical intersection calculation principle based on RANSAC in an embodiment of the present invention.

[0061] Figure 10 shows the final light stripe centerline result after independent pixel threshold verification and fine extraction in the embodiment of the present invention. Detailed Implementation

[0062] The following example, using the extraction of the center of a light stripe during welding, which involves arc light, spatter noise, and reflective interference, is used to further describe the present invention in detail with reference to examples and accompanying drawings. However, the embodiments of the present invention are not limited thereto.

[0063] like Figure 1 As shown in the figure, this embodiment of the invention provides a robust method for extracting the structural light centerline of a fillet weld under complex backgrounds. The specific implementation steps are as follows:

[0064] Step S1: Image preprocessing and mask generation based on multi-scale morphology. As shown in Figure 2, the input original fillet weld structured light image contains large areas of strong arc light interference, metal spatter noise, and severe specular reflection at the root (corner) of the fillet weld.

[0065] First, a top-hat transform is applied to Figure 2 using a horizontal linear structuring element of length 30. As shown in Figure 3, the top-hat filter effectively removes large areas of strong arc light from the background, highlighting the thin strip-shaped structured light features.

[0066] Subsequently, to eliminate the remaining lateral splash noise in Figure 3, an opening operation was performed using a vertical line-type structuring element with a length of 20. As shown in Figure 4, the opening operation successfully suppressed the free splash points.

[0067] Next, a bridging operation was performed using a vertical line-type structural element with a length of 25, as shown in Figure 5. The bridging operation repaired some of the minor breaks caused by reflection.

[0068] Finally, Figure 5 is binarized by setting a threshold (e.g., 0.05) to obtain Figure 6;

[0069] Calculate the area and centroid x-coordinate of all connected components. Select the centroid x-coordinate of the connected component with the largest area as the benchmark, and retain effective connected components with an area greater than 10 that are within the horizontal position tolerance (e.g., 50 pixels). As shown in Figure 7, after clustering and filtering, a clean light stripe backbone mask that completely filters out spatter is finally generated.

[0070] Step S2: Global coarse extraction without prior knowledge. As shown in Figure 8, within the effective area of ​​the mask shown in Figure 7, the basic gray-level centroid method is used to calculate line by line to obtain the set of coarse light stripe center points (i.e., the point set in Figure 8). This embodiment does not perform segmentation operations to find the fracture gaps, but instead constructs all extracted points into a global scattered data pool.

[0071] Step S3: Adaptive segmentation and global fitting based on sequence RANSAC.

[0072] As shown in Figure 9, the sequence RANSAC algorithm is executed in the global scatter data pool:

[0073] First iteration extraction: Randomly sample and evaluate inliers in the pool, and identify the first main line containing the most inliers. After reestimating the parameters using the least squares method, remove all inliers corresponding to this main line from the scatter pool (as shown by the dark scatter points and dashed lines in the upper part of Figure 9).

[0074] Second iteration extraction: RANSAC is executed again on the remaining scatter points to lock in the second principal line and re-estimate the parameters (as shown by the dark scatter points and dashed line in the lower segment of Figure 9). The algorithm automatically ignores local deformation points and splash noise points outside the two principal lines (as shown by the gray scatter points in Figure 9, i.e., the eliminated outliers). By comparing the mean of the ordinates of the two extracted principal lines, the equations of the upper and lower trend lines are established respectively.

[0075] Step S4: Solving for the intersection point using analytical geometry theory. As shown in Figure 9, by simultaneously solving the equations of the upper and lower trend lines obtained in Step S3, the coordinates of the theoretical intersection point of the two lines in analytical geometry are obtained (marked with an asterisk at the intersection of the two dashed lines in Figure 9). This intersection point is not affected by actual physical reflective breaks and provides an absolutely reliable extension guide reference.

[0076] Step S5: Trend-guided independent pixel verification and sub-pixel fine extraction. Taking the upper light stripe as an example, extend downwards from the coordinates at the end of the upper segment. Predict the upper trend line in Figure 9 line by line until the theoretical intersection point. In the original, morphologically intact image (Figure 2), establish a local search window with a width of ±4 pixels centered on the predicted coordinates.

[0077] Perform independent pixel threshold review: Obtain the highest local grayscale value within the window. If the highest grayscale value is greater than or equal to the set foreground grayscale threshold (e.g., 55), then select pixels with grayscale values ​​greater than "highest grayscale value × 0.9" to participate in weighting and recalculate the sub-pixel grayscale centroid. If the highest grayscale value is less than 55, then determine that the predicted point is located in a dark spot area with no real light, skip the current row, and continue to review the next row.

[0078] The lower light bar performs an upward symmetrical operation. As shown in Figure 10, after the above review and fine extraction, the present invention not only accurately repairs the large-scale fracture caused by reflection (as shown by the solid dot connection in Figure 10), but also skips the absolute dark area due to the use of an independent pixel review mechanism, avoiding erroneous extraction. The final output fillet weld structure light center line perfectly retains the sharp corner features, exhibiting extremely high robustness.

[0079] The above examples are typical embodiments of the present invention. However, the embodiments of the present invention are not limited to the described embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention should be considered equivalent substitutions and are included within the protection scope of the present invention.

Claims

1. A robust method for extracting the structural centerline of a fillet weld under complex backgrounds, characterized in that, Includes the following steps: Step S1: Perform multi-scale morphological preprocessing and position tolerance-based clustering screening on the acquired original fillet weld structured light image to obtain the light stripe backbone mask; Step S2: Within the effective area of ​​the main light stripe mask, the set of global coarse light stripe center points is obtained using the gray-scale centroid method, and an unsegmented global scattered data pool is constructed. Step S3: The sequential random sampling consensus algorithm is used to iteratively fit and strip data from the global scatter data pool to adaptively obtain the upper and lower trend line equations of the fillet weld. Step S4: Combine the equations of the upper and lower trend lines, and calculate the theoretical analytical intersection coordinates of the fillet weld angle using analytical geometry; Step S5: Perform pixel-by-pixel prediction along the direction of the intersection of the trend line equation and the theoretical analysis point. Set a local search window at the prediction coordinates of the original image to perform independent pixel grayscale threshold review. Recalculate the sub-pixel grayscale centroid for the qualified pixels to complete the repair and fine extraction of the light stripe center line.

2. The extraction method according to claim 1, characterized in that, Step S1 specifically includes: applying a top-hat filter to the original image using horizontal linear structuring elements to remove large-area arc light; performing an opening operation using vertical linear structuring elements to remove lateral splashes; performing breakpoint bridging and binarization through a closing operation; calculating the area and centroid abscissa of all connected components, selecting the centroid abscissa of the connected component with the largest area as the benchmark, retaining effective connected components with horizontal position deviation within the tolerance range and area greater than a preset threshold, and generating the light stripe backbone mask.

3. The extraction method according to claim 1, characterized in that, The sequence random sampling consensus algorithm described in step S3 specifically includes: randomly sampling sample points from the global scatter data pool to construct a linear model, and counting the number of inliers with a horizontal residual less than a preset tolerance; after a specified number of iterations, selecting the model containing the most inliers as the first main line, and removing all inliers belonging to the first main line from the global scatter data pool; repeating the above iterative process to extract the second main line for the remaining scatter points after removal; calculating the mean ordinate of the inliers of the first main line and the second main line respectively, to adaptively determine the equations of the upper and lower trend lines.

4. The extraction method according to claim 1, characterized in that, The independent pixel grayscale threshold review in step S5 specifically involves: obtaining the highest local grayscale value of the original, unmorphologically processed image within the local search window; if the highest local grayscale value is greater than or equal to the set foreground grayscale threshold, the review is deemed successful; if the highest local grayscale value is less than the set foreground grayscale threshold, the predicted point is determined to be located in a dark area without light, the current point is skipped without interrupting the prediction extension, and the review of the next pixel on the prediction path continues.

5. The extraction method according to claim 4, characterized in that, When recalculating the subpixel grayscale centroid of the approved pixels, only pixels with grayscale values ​​greater than or equal to the product of the local highest grayscale value and the scaling factor within the local search window are selected for weighted calculation.

Images