Welding seam identification and feature point extraction method based on laser vision

Abstract

The invention belongs to the field of computer vision, and particularly relates to a laser vision-based welding seam identification and feature point extraction method, which comprises the following steps of: firstly, carrying out Gaussian filtering on an acquired welding seam image to suppress noise, and then extracting sub-pixel coordinates of a laser stripe center by utilizing an improved Steger algorithm; performing three-dimensional reconstruction by combining a laser plane equation participating in the camera, and generating an XZ plane image through sampling and interpolation; probabilistic Hough transform is applied to detect line segments, and the line segments are combined through angle and distance constraints to improve integrity; based on the structural features of the welding seam, combining line segments by utilizing dislocation and gap constraint, and screening out an effective welding seam line segment set through a prior angle; and finally, end points or cross points of line segments are extracted as feature points according to the welding seam type, and welding spot coordinates are stably output by utilizing an inter-frame tracking mechanism. Strong interference in the welding process can be effectively resisted, and online, real-time and high-precision recognition and feature point extraction of various types of welding seams are achieved.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically a method for weld seam recognition and feature point extraction based on laser vision. Background Technology

[0002] With the rapid development of my country's national economy, industrial automation is upgrading towards "intelligent manufacturing." Welding, as a core process in fields such as machinery, automobile manufacturing, and aerospace, faces continuously increasing demands for welding quality and production efficiency. Traditional manual welding suffers from low precision, poor stability, and high labor intensity, making it unable to meet the stringent standards of modern high-end equipment manufacturing. Automated welding using robots has become an ideal choice for an increasing number of enterprises.

[0003] Laser vision technology is widely used in weld seam tracking systems due to its advantages such as high real-time performance, high precision, and non-contact operation. Its core principle is to project laser stripes onto the workpiece surface, capture the deformed stripe images using a camera, and then calculate the three-dimensional information of the weld seam. However, in actual welding processes, complex conditions such as spatter, smoke obstruction, and strong arc light reflection can easily lead to problems such as broken laser stripe images, distorted grayscale distribution, and severe noise interference. Although these issues can be mitigated to some extent by adding hardware measures such as protective baffles and filters, image quality degradation is still difficult to avoid, posing a significant challenge to subsequent stripe center extraction, line segment recognition, and feature point localization.

[0004] Existing weld seam identification methods often suffer from problems such as incomplete line segment extraction, large feature point positioning deviations, and poor adaptability to different weld seam types when dealing with fractures and stripes with severe noise interference. These issues make it difficult to meet the robustness requirements of high-precision automated welding. Therefore, there is an urgent need for a weld seam identification and feature point extraction method that can effectively overcome the above difficulties and possess strong anti-interference capabilities, high accuracy, and wide adaptability. Summary of the Invention

[0005] The purpose of this invention is to provide a method with high accuracy and robustness that can process weld seam recognition and feature point extraction based on laser vision in real time, so as to solve the problems of incomplete line segment extraction and large feature point positioning deviation caused by laser stripe breakage and grayscale distribution distortion.

[0006] The technical solution adopted by this invention to achieve the above objectives is: a method for weld seam recognition and feature point extraction based on laser vision, comprising the following steps:

[0007] Step S1: Acquire weld seam images of the detection area using a camera, and apply Gaussian filtering to the weld seam images to obtain images that retain the edge details of the laser stripes;

[0008] Step S2: Use the improved Steger algorithm to extract the sub-pixel coordinates of the center of the laser stripe in the image;

[0009] Step S3: Combine the camera intrinsic parameter matrix and the laser plane parameters to solve the three-dimensional coordinates of the laser stripe center. By setting the sampling interval and performing linear interpolation on the z-axis coordinate, an xz plane image is generated.

[0010] Step S4: Apply probabilistic Hough transform to the xz-plane image to extract all line segments;

[0011] Step S5: Select line segments with the same angle and similar distance from all line segments in step S4 and merge them to improve the integrity of the line segments;

[0012] Step S6: Based on the structural characteristics of the weld type, combine the line segments by interpolation and gap constraints of adjacent line segments to initially form the target line segment combination of the weld;

[0013] Step S7: Obtain the angle of the weld target line segment combination. Based on the prior angle of the weld, retain the valid targets that are consistent with the welded structure to obtain the final set of valid weld line segments.

[0014] Step S8: Based on the weld type, calculate the endpoints of the line segments or the intersections of adjacent line segments as weld feature points; and use the inter-frame tracking mechanism to determine the weld coordinates.

[0015] Step S2 includes the following steps:

[0016] Step S2-1: Detect the column-direction local gray-level maxima in the Gaussian-filtered image:

[0017] The laser stripes exhibit a unimodal grayscale distribution along the y-axis. Local grayscale maxima are extracted for all v-coordinates of each u-coordinate. A (2n+1)×1 neighborhood window is used to filter the grayscale values. The following conditions must be met:

[0018]

[0019] and

[0020]

[0021] in, For index variables, If it is the grayscale threshold, then This refers to the local grayscale maximum.

[0022] Step S2-2: Calculate the discrete reciprocal:

[0023] For each candidate maximum point Take 5 consecutive pixels along the v-axis, that is: Their grayscale values ​​are as follows: , , , , The first derivative is calculated using the central difference method. With the second derivative ,Right now:

[0024]

[0025]

[0026] Step S2-3: Calculate the sub-pixel center coordinates:

[0027] Assume the grayscale distribution of the laser stripes along the v-axis follows a quadratic function. Its extreme point is the sub-pixel offset corresponding to the center of the laser stripe. Solve using derivatives; when hour, Obtain sub-pixel offset for:

[0028]

[0029] The coordinates of the center of the sub-pixel level laser stripe in column u are: .

[0030] In step S3, the step of combining the camera intrinsic parameter matrix and the laser plane parameters to solve for the three-dimensional coordinates of the laser stripe center includes the following steps:

[0031] Step S3-1: The camera intrinsic parameter matrix K is:

[0032]

[0033] in, Let be the normalized focal length of the camera in the x-direction. ; Let be the normalized focal length of the camera in the x-direction. ; The physical focal length of the camera; , The physical dimensions of a pixel in the x and y directions. The coordinates of the center pixel of the image;

[0034] Step S3-2: The equation of the laser plane in the camera coordinate system is:

[0035]

[0036] in, These are the equation coefficients for the laser plane;

[0037] Step S3-3: Three-dimensional coordinates of the laser stripe center for:

[0038]

[0039] in, The coordinates of the principal point of the image. Subpixel coordinate.

[0040] In step S3, the process of generating an xz-plane image by setting a sampling interval and performing linear interpolation on the z-axis coordinates specifically involves:

[0041] 1) Generate sampling points along the weld length direction, i.e., the X-axis, and generate a sampling point sequence at preset intervals: ;

[0042] 2) For each sampling point Find adjacent original 3D points and The interpolated z-coordinate is calculated using the interpolation formula, i.e.:

[0043]

[0044] 3) Pair each sampling point Considering a pixel in the xz plane, a two-dimensional xz plane image is generated by setting the image size and grayscale assignment rules, where the x-axis represents the weld length and the z-axis represents the height.

[0045] Step S4 includes the following steps:

[0046] Step S4-1: Randomly select a portion of the center points of the laser stripes from the xz plane image, and uniquely determine a straight line through the center points of the laser stripes;

[0047] Step S4-2: For each sampling point Calculate all possible The value is:

[0048]

[0049] in, The perpendicular distance from the origin to the line is... The angle between the line and the x-axis is calculated by adjusting the corresponding values ​​in the accumulator. The value is counted;

[0050] Step S4-3: Set a threshold and find the values ​​in the parameter space that have a count greater than that value. Values, each peak corresponds to a straight line in the image;

[0051] Step S4-4: Based on the detected line parameters By searching backwards from both ends of the line, we can find the endpoints of the line segment in the image that belong to the line and obtain the coordinates of the endpoints of the line segment.

[0052] Step S5 includes the following steps:

[0053] Step S5-1: For any two line segments , line segment The starting coordinates are The endpoint coordinates are line segment The starting coordinates are The endpoint coordinates are The direction angle of the two line segments , They are respectively:

[0054]

[0055]

[0056] Step S5-2: Line Segment From the two endpoints to the line segment distance , They are respectively:

[0057]

[0058]

[0059] Step S5-3: If the angle deviation satisfies: and distance , satisfy: ,and Then they are merged into a new line segment, forming a line segment set. ;

[0060] Step S5-4: For the line segment set Fit a straight line to each line segment as follows:

[0061]

[0062] in, The center point of the endpoints of the line segment set. Let be the unit direction vector of the line. Let be any point on the line;

[0063] New line segments merged The starting point ) and endpoint They are respectively:

[0064] ;

[0065] .

[0066] Step S6 specifically includes:

[0067] Step S6-1: Segment sorting. Sort the merged segment segments according to their x-coordinates and construct a segment linked list.

[0068] Step S6-2: Constraint judgment, traverse adjacent line segments , The calculated gap d and misalignment s are as follows:

[0069]

[0070]

[0071] Among them, line segment The starting coordinates are The endpoint coordinates are line segment The starting coordinates are The endpoint coordinates are ;

[0072] Step S6-3: If satisfied or Then, line segment combinations are performed.

[0073] Step S7 specifically includes:

[0074] Based on the weld type, calculate the included angle or absolute angle of adjacent line segments. If the following conditions are met: ;in, This is the actual angle of the weld. If the angle is a priori, then the combination of line segments is retained to obtain the effective weld line segment set.

[0075] In step S8, the step of calculating the endpoints of line segments or the intersections of adjacent line segments as weld feature points based on the weld type specifically involves:

[0076] For weld types of fillet joints and V-joints, the intersection of line segments is taken as the feature point, assuming the line segments... , their intersection for:

[0077]

[0078] For weld types such as lap, butt, or edge welds, the endpoints of the line segment are taken as feature points, and the line segment is known. Search for data points on the extension of the line segment to the left or right, and include consecutive points whose distance from the point to the line is less than or equal to a threshold into the line segment until no points meet the condition, then update the endpoint coordinates.

[0079] In step S8, determining the solder joint coordinates using the inter-frame tracking mechanism specifically involves:

[0080] Assume the coordinates of the feature points in the previous frame are The tracking window is The feature points of the current frame are satisfy:

[0081] ;

[0082] If there are m candidate points that meet the conditions within the tracking window Then feature points for: .

[0083] The present invention has the following beneficial effects and advantages:

[0084] 1. This invention can effectively suppress complex noise interference such as welding arc light, spatter, and fumes, accurately extract the sub-pixel-level center coordinates of laser stripes, and has excellent anti-interference performance.

[0085] 2. This invention combines Gaussian filtering with an improved Steger algorithm to effectively suppress complex noises such as welding arc light, spatter, and fumes, and achieves sub-pixel-level precise extraction of the center of laser stripes, laying a high-precision data foundation for core 3D reconstruction and feature recognition.

[0086] 3. This invention addresses the problem of easily broken laser stripes by designing a complete post-processing workflow for line segments (including line segment merging, combination based on misalignment and gap, and angle filtering), which can effectively repair incomplete line segments, improve the integrity of feature extraction, and exhibit strong adaptability to fluctuations in operating conditions.

[0087] 4. By introducing structural features based on weld type (such as intersections and endpoints) and prior angle knowledge, this method can adapt to a variety of common weld types such as corner joints, V-joints, lap joints, butt joints, and edge joints, breaking through the limitation of a single algorithm on weld type.

[0088] 5. The final inter-frame tracking mechanism of this invention utilizes the spatiotemporal continuity of weld seam features to effectively filter out instantaneous anomalies and noise interference, ensuring the continuity and stability of weld seam coordinate output, and further improving the reliability of the entire system.

[0089] 6. This invention integrates 3D reconstruction, image generation, line segment detection and optimization, feature extraction and tracking, achieving fully automated online operation. By combining probabilistic Hough transform and inter-frame tracking mechanisms, it significantly improves processing speed while ensuring accuracy, meeting the real-time trajectory guidance requirements of welding robots. Attached Figure Description

[0090] Figure 1 This is a flowchart of the weld seam identification and feature point extraction method of the present invention;

[0091] Figure 2 This is a schematic diagram illustrating the computational principle of the improved Steger algorithm in this invention.

[0092] Figure 3a This is a schematic diagram of the misalignment of adjacent line segments in this invention;

[0093] Figure 3b This is a schematic diagram of the gap between adjacent line segments in this invention. Detailed Implementation

[0094] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0095] like Figure 1 As shown, the laser vision-based weld seam recognition and feature point extraction method of this invention performs Gaussian filtering preprocessing on the acquired grayscale image to effectively suppress noise interference; an improved Steger algorithm is used to extract the laser stripe center; three-dimensional coordinate reconstruction is completed by combining the camera intrinsic parameter matrix and laser plane parameters; an xz plane image that can intuitively reflect the longitudinal contour of the weld seam is generated through adaptive sampling and linear interpolation technology; line segments in the xz plane image are detected using probabilistic Hough transform; line segments with consistent angles and close distances are merged to improve line segment integrity; based on the structural features of different weld seam types, line segment combinations are achieved through the misalignment and gap constraints of adjacent line segments to initially construct a set of target line segments for the weld seam; the direction angle of the line segment combination is calculated, and effective targets that meet the weld seam characteristics are screened using the prior angle range; the endpoints of line segments or the intersections of adjacent line segments are extracted as candidate feature points according to the weld seam type; finally, through an inter-frame tracking mechanism, instantaneous abnormal points are filtered out, and the accurate three-dimensional coordinates of the weld point are stably output. Figure 1 As shown, the specific steps include the following:

[0096] (1) Gaussian filtering preprocessing

[0097] Gaussian filtering was performed on the acquired grayscale image to effectively suppress noise while preserving the edge details of the laser stripes. The Gaussian filter kernel size was set to 3×11.

[0098] (2) Extracting the center of the laser stripe

[0099] The center of the laser stripe is extracted using the improved Steger algorithm. The calculation steps are as follows:

[0100] (2.1) Detecting local gray-level maxima in the column direction of the image after Gaussian filtering

[0101] The laser stripes exhibit a single-peak grayscale distribution along the y-axis. Therefore, by traversing the image column by column, local grayscale maxima are extracted for all v-coordinates of each u-coordinate. A (2n+1)×1 neighborhood window is used. If the grayscale value... satisfy,

[0102] (1)

[0103] and

[0104] (2)

[0105] In the formula, If it is the grayscale threshold, then This refers to the local grayscale maximum.

[0106] (2.2) Calculate the discrete reciprocal

[0107] For each candidate maximum point Take 5 consecutive pixels along the v-axis. Their grayscale values ​​are respectively , , , , The first derivative is calculated using the central difference method. With the second derivative ,

[0108] (3)

[0109] (4)

[0110] (2.3) Calculate the sub-pixel center coordinates

[0111] Assume the grayscale distribution of the laser stripes along the v-axis follows a quadratic function. Its extreme point is the sub-pixel offset corresponding to the center of the laser stripe. It can be solved using derivatives, when hour, The subpixel offset can be obtained:

[0112] (5)

[0113] Therefore, the coordinates of the sub-pixel level laser stripe center of this u column are:

[0114] (6)

[0115] (3) Generate the xz plane image

[0116] The xz-plane image is a crucial bridge connecting 3D coordinate reconstruction and line segment detection. Specific steps include:

[0117] (3.1) Calculate the three-dimensional coordinates of the center of the laser stripe

[0118] Assume the camera intrinsic parameter matrix is ​​K:

[0119] (7)

[0120] in, , , The physical focal length of the camera. , The physical dimensions of a pixel in the x and y directions, expressed in mm per pixel. Let be the coordinates of the image center pixel. The equation of the laser plane in the camera coordinate system is:

[0121] (8)

[0122] So, the three-dimensional coordinates of the center of the laser stripe for,

[0123] (9)

[0124] (3.2) Generation of xz-plane image

[0125] Sampling points are generated along the length of the weld, i.e., the x-axis. For each sampling point... Find adjacent original 3D points and The z-coordinate obtained by linear interpolation is,

[0126] (10)

[0127] The generated xz plane image corresponds to the weld length horizontally (x-axis), covering the entire weld from left to right, and to the weld height vertically (z-axis), decreasing from top to bottom.

[0128] (4) Line segment detection

[0129] The probabilistic Hough transform is applied to the xz-plane image to extract all line segments. The basic principle is to randomly sample a subset of data points, perform parameter space mapping and voting, and output the endpoints of the line segments. Specifically,

[0130] (4.1) Randomly sampled data points:

[0131] Randomly select a portion of the center points of the laser stripes from the xz plane image, and use these points to uniquely determine a straight line, avoiding traversing all data points and reducing the amount of computation.

[0132] (4.2) Parameter space mapping:

[0133] For each sampling point Calculate all possible value,

[0134] (11)

[0135] In the formula, The perpendicular distance from the origin to the line is... This is the angle between the line and the x-axis. In the accumulator, the corresponding... The value is counted.

[0136] (4.3) Parameter space peak detection

[0137] Set a threshold to find items in the parameter space whose count is greater than that value. The value represents a single line in the image, with each peak corresponding to a single line.

[0138] (4.4) Calculation of line segment endpoints

[0139] Based on the detected straight line parameters By searching backwards from both ends of the line, we can find the endpoints of the line segment in the image that belong to the line and obtain the coordinates of the endpoints of the line segment.

[0140] (5) Merging line segments

[0141] Merging line segments using angle and distance constraints improves their integrity. For any two line segments... , line segment The starting coordinates are The endpoint coordinates are line segment The starting coordinates are The endpoint coordinates are Its direction angle , They are respectively:

[0142] (12)

[0143] (13)

[0144] line segment From the two endpoints to the line segment distance , They are respectively:

[0145] (14)

[0146] (15)

[0147] If the angle deviation is satisfied, then they are merged into a new line segment. :

[0148] (16)

[0149] , satisfy:

[0150] (17)

[0151] (18)

[0152] in, Distance threshold

[0153] Assume a set of line segments They need to be merged, where each line segment is represented as... The starting point is The destination is The fitted straight line is:

[0154] (19)

[0155] In the formula, The center point of the endpoints of the line segment set. Let be the unit direction vector of the line. Let be any point on the straight line. Then the starting point of the merged new line segment is... ) and endpoint They are respectively:

[0156] (20)

[0157] (twenty one)

[0158] (6) Line segment combination

[0159] like Figures 3a-3b The diagram illustrates the misalignment and gap between adjacent line segments in this invention. Based on the structural characteristics of different weld types, this invention utilizes the misalignment and gap constraints of adjacent line segments to combine line segments, thus initially constructing a target line segment set for the weld. The steps are as follows:

[0160] (6.1) Segment sorting

[0161] Sort the merged line segments according to their x-coordinates and construct a linked list of line segments;

[0162] (6.2) Constraint Judgment

[0163] Traverse adjacent line segments , Calculate the gap d and the misalignment s.

[0164] (twenty two)

[0165] (twenty three)

[0166] If satisfied or ,in The gap threshold, If the misalignment threshold is reached, then line segment combinations are performed.

[0167] (7) Angle filtering

[0168] Based on the weld type, calculate the included angle or absolute angle of adjacent line segments. If satisfied,

[0169] (twenty four)

[0170] In the formula, This is the actual angle of the weld. If the angle threshold is used, then the combination of line segments is retained to obtain the effective weld line segment set.

[0171] (8) Calculate the characteristic points of the weld.

[0172] Based on the weld type, the endpoints of line segments or the intersections of adjacent line segments are extracted as candidate feature points. For weld types such as corner joints and V-joints, the intersections of line segments are taken as feature points, assuming the line segments... , their intersection for,

[0173] (25)

[0174] For weld types such as lap joints, butt joints, and edge welds, the endpoints of the line segment are taken as feature points, and the line segment is known. Search for data points on the extension of the line segment to the left / right, and include consecutive points that satisfy "distance from point to line ≤ threshold" into the line segment until no points meet the condition, then update the endpoint coordinates.

[0175] (9) Inter-frame tracking to determine solder joints

[0176] Weld point coordinates are determined using an inter-frame tracking mechanism. Based on the spatiotemporal continuity of weld seam features in consecutive image frames, noise interference and transient anomalies are filtered out to stably output the weld point coordinates. Assume the coordinates of the feature points in the previous frame are... The tracking window is The feature points of the current frame are To satisfy,

[0177] (26)

[0178] If there are m candidate points that meet the conditions within the tracking window Then feature points for:

[0179] (27)

[0180] Example 1:

[0181] This embodiment implements the invention in a typical robotic welding workstation. The system mainly consists of a six-degree-of-freedom welding robot, a laser vision sensor (including a line laser and an industrial camera), and an industrial control computer. The laser vision sensor is fixed approximately 50mm in front of the robot's end effector and moves with the robot.

[0182] System calibration:

[0183] Two key calibrations must be completed before welding can begin:

[0184] Camera intrinsic parameter calibration: Zhang Zhengyou calibration method was adopted, using a 9×6 checkerboard calibration board, 15 images with different poses were acquired, and the camera intrinsic parameter matrix K and distortion coefficients were calculated.

[0185] Hand-eye calibration and laser plane calibration: By controlling the robot's movement, the laser stripe is projected onto a standard block with known three-dimensional coordinates at different positions. Multiple sets of data are collected, and the equations (coefficients A, B, C, D) of the laser plane in the camera coordinate system and the transformation matrix between the sensor and the robot end effector are solved.

[0186] Specific implementation steps:

[0187] Step S1: Image acquisition and Gaussian filtering preprocessing

[0188] During the welding process, the camera continuously captures grayscale images of the weld area at a frame rate of 100fps. The image resolution is 1280×1024.

[0189] To suppress welding spatter and arc noise, Gaussian filtering was applied to the original image. In practice, the Gaussian filter kernel size was set to 3×11. This kernel size has been experimentally verified to effectively smooth noise in the y-direction (fringe normal) while preserving edge details in the x-direction (fringe direction) relatively well.

[0190] Step S2: Extract the center of the laser stripe using the improved Steger algorithm.

[0191] Step S2-1: Detect local gray-level maxima. Traverse the filtered image column by column (u-coordinate). For each column, use a (2×2+1)×1, i.e., 5×1, neighborhood window (n=2) to find local gray-level maxima. Gray-level threshold. Set to 60 (based on 8-bit grayscale image) to filter background noise.

[0192] Step S2-2: Calculate the derivatives. For each candidate point, take 5 consecutive pixels (v-2, v-1, v, v+1, v+2) along its normal direction (v-axis), and calculate the first derivative I' and the second derivative I'' using the central difference method.

[0193] Step S2-3: Calculate sub-pixel coordinates. According to the formula... Calculate the sub-pixel offset to obtain the sub-pixel coordinates of the center of the laser stripe column. .

[0194] Step S3: 3D Reconstruction and XZ Plane Image Generation

[0195] Steps S3-1 to S3-3: Solving for 3D coordinates. The sub-pixel coordinates obtained in step S2... Substituting the camera intrinsic parameter matrix K and the calibrated laser plane equation, the three-dimensional coordinates (X, Y, Z) of each laser point in the camera coordinate system are calculated using the principle of triangulation.

[0196] Step S3-4: Generate the XZ plane image. This is a crucial step in connecting the two-dimensional image with the three-dimensional line segment processing.

[0197] Step S4: Probabilistic Hough Transform Line Segment Detection

[0198] Apply the probabilistic Hough transform to the generated XZ plane image.

[0199] Specific parameter settings: Distance resolution for line detection 1 pixel, angular resolution The angle is 1°, and the accumulator threshold is 50 (meaning that at least 50 pixels are needed to vote for a line to be considered valid).

[0200] This step outputs a series of line segments, each represented by the coordinates of its endpoints.

[0201] Step S5: Merge line segments:

[0202] Steps S5-1 to S5-3: Merge and determine. Traverse all line segments, and for any two line segments... and Calculate its direction angle , and distance , .

[0203] Threshold setting: Angle threshold Set to 5°, distance threshold Set to 2 pixels. If both conditions are met... and and If so, then the two line segments should be merged.

[0204] Step S5-4: Merge and execute. For all line segments L that need to be merged, fit a new straight line using the least squares method, and take the minimum and maximum x-coordinates of the endpoints of all line segments as the starting and ending x-coordinates of the new line segment. Then, calculate the corresponding z-coordinates based on the fitted straight line equation to form a complete new line segment.

[0205] Step S6: Combination of line segments based on misalignment and gaps

[0206] Sort the merged line segments in ascending order of their x-coordinates.

[0207] Traverse adjacent line segments and calculate the gap d or misalignment s.

[0208] Threshold setting: Gap threshold Set to 15 pixels, misalignment threshold Set to 5 pixels. If d≤15 or s≤5, combine these two line segments into a larger "weld target line segment combination" to prepare for subsequent angle selection.

[0209] Step S7: Angle Filtering

[0210] Calculate the representative angle for each combination of line segments. Threshold applications based on prior knowledge:

[0211] For V-groove welds, the standard angle Set to 60°, angle threshold Set to 10°.

[0212] For fillet welds, standard angle Set to 90°, angle threshold Set to 10°.

[0213] like If so, retain the combination to form the final set of effective weld segments.

[0214] Step S8: Feature point extraction and inter-frame tracking

[0215] Feature point extraction:

[0216] For a V-groove, the intersection of the two line segments forming the groove is calculated as the feature point (i.e., the position where the welding torch should be aligned). The coordinates of the intersection point are obtained by solving the equations of the two lines.

[0217] For lap welds, the endpoint of the edge segment of the upper workpiece is taken as the feature point. By searching for neighboring points along the extension line of the segment at this endpoint, points that meet the conditions are included to update the endpoint coordinates, ensuring that the feature point is located at the true edge position.

[0218] Inter-frame tracking:

[0219] Set the tracking window radius N=5 pixels, that is, the window size is 11×11 pixels.

[0220] For the current frame, in the feature points of the previous frame Within the centered tracking window, search for all feature points belonging to the set of valid weld seam segments.

[0221] If k candidate points are found within the window, the final feature point coordinates of the current frame are their average value. This operation effectively smooths out feature point jumps caused by noise.

[0222] Through the steps in Example 1, the system can stably output sub-millimeter-level precision three-dimensional coordinates of weld feature points even in the presence of significant welding spatter and arc light interference. The robot controller uses these coordinates for real-time path correction, achieving high-quality automated welding. Testing showed a success rate of over 98% in recognizing various weld types, including V-grooves, corner joints, and lap joints, with a feature point positioning error of less than 0.2mm, fully meeting the application requirements of industrial sites.

[0223] In summary, and in conjunction with the embodiments, this invention overcomes the processing bottlenecks of traditional methods in the case of stripe breakage and noise interference. Through an adaptive line segment repair and combination mechanism, it ensures the integrity and reliability of feature extraction. It establishes an intelligent screening system based on prior knowledge, which significantly improves the accuracy and intelligence level of weld seam recognition. Finally, through inter-frame tracking technology, it achieves stable and smooth output of weld point coordinates, providing high-precision trajectory guidance for robotic welding.

[0224] Furthermore, this invention is not only applicable to conventional weld types such as V-joints, corner joints, and lap joints, but can also be extended to other industrial vision inspection scenarios requiring precise trajectory control. The widespread implementation of this technology will effectively improve welding automation levels and product quality, and has significant practical implications for promoting the upgrading of the intelligent manufacturing industry.

[0225] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or claims of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0226] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A laser vision-based weld seam recognition and feature point extraction method, characterized in that, The method comprises the following steps: Step S1: acquiring a weld seam image with a detection area by a camera, and performing Gaussian filtering on the weld seam image to obtain an image retaining edge details of laser stripes; Step S2: extracting sub-pixel coordinates of the center of the laser stripe in the image by using an improved Steger algorithm; Step S3: solving three-dimensional coordinates of the center of the laser stripe by combining a camera intrinsic matrix and laser plane parameters, generating an xz plane image by setting a sampling interval and performing linear interpolation on z-axis coordinates; Step S4: applying a probabilistic Hough transform to the xz plane image to extract all line segments; Step S5: screening and merging line segments with consistent angles and similar distances in all line segments in step S4 to improve the integrity of the line segments; Step S6: combining line segments based on the misplacement and gap of adjacent line segments to preliminarily form a weld seam target line segment combination based on the structural features of the weld seam type; Step S7: obtaining the angle of the weld seam target line segment combination, retaining effective targets consistent with the welding structure based on the prior angle of the weld seam, and obtaining a final effective weld seam line segment set; Step S8: calculating the intersection points of the end points of the line segments or adjacent line segments as weld seam feature points based on the weld seam type; and determining the weld point coordinates by using an inter-frame tracking mechanism. The step S2 comprises the following steps: 2.The laser vision based weld seam recognition and feature point extraction method according to claim 1, characterized in that, Step S2-1: detecting local gray maximum values of a column of the image after Gaussian filtering: And The laser fringe is single-peak gray scale distribution in y-axis direction. The local gray scale maximum points are extracted for all v coordinates of each u coordinate. A (2n+1) x 1 field window is adopted to screen the gray scale values satisfy the following conditions: Step S2-2: calculating a discrete reciprocal: wherein, is an index variable, is a gray scale threshold, then is the local gray scale maximum. Step S2-3: calculating a sub-pixel center coordinate: For each candidate maximum point , five consecutive pixel points are taken along the v-axis direction, namely: , whose gray values are respectively: , , , , , the first derivative and the second derivative are calculated by using the central difference method, namely: In step S3, the combination of the camera intrinsic matrix and the laser plane parameters to solve the three-dimensional coordinates of the center of the laser stripe comprises the following steps: Assume that the gray scale distribution of the laser stripe in the v-axis direction conforms to a quadratic function The extreme point of the quadratic function is the sub-pixel offset corresponding to the center of the laser stripe The derivative is solved; when , The sub-pixel offset is obtained ​ The sub-pixel level laser stripe center coordinates of the u column are: . 3.The laser vision based weld joint recognition and feature point extraction method according to claim 1, characterized in that, Step S3-1: the camera intrinsic matrix K is: Step S3-2: the equation of the laser plane in the camera coordinate system is: wherein, is the normalized focal length of the camera in the x direction, ; is the normalized focal length of the camera in the x direction, ; is the physical focal length of the camera; , is the physical size of a pixel in the x, y direction, is the image center pixel coordinate; In step S3, the xz plane image is generated by setting a sampling interval and performing linear interpolation on z-axis coordinates, specifically as follows: wherein are the equation coefficients of the laser plane, respectively; Step S3-3: Three-dimensional coordinates of the center of the laser stripe are: wherein, is the main point coordinate of the image, is the sub-pixel coordinate. 4.The laser vision based weld joint recognition and feature point extraction method according to claim 1, characterized in that, The step S4 comprises the following steps: 1) Along the length direction of the weld, i.e. X axis, to generate sampling points, to generate sampling point sequence at preset interval: ; 2) For each sampling point , find the neighboring original 3D points and , calculate the interpolated z coordinate using the interpolation formula, i.e. 3) Each pair of sampling points The pixel point in the xz plane is regarded as a pixel point in the xz plane, and a two-dimensional XZ plane image is generated by setting the image size and gray value assignment rule, wherein the x axis represents the weld length and the z axis represents the height. 5.The laser vision based weld joint recognition and feature point extraction method according to claim 1, characterized in that, Step S4-1: randomly selecting a part of laser stripe center points from the xz plane image, and uniquely determining a straight line through the laser stripe center points; The step S5 comprises the following steps: Step S4-2: For each sample point , all possible values are calculated as: wherein, is the perpendicular distance from the origin to the line, is the angle of the line with the x-axis, by counting the corresponding values in an accumulator; Step S4-3: Set a threshold, find out the values in the parameter space whose counts are greater than the threshold, each peak corresponds to a line in the image. Step S4-3: Set a threshold, find out the values in the parameter space whose counts are greater than the threshold, each peak corresponds to a line in the image. Step S4-4: According to the detected straight line parameters , the end point coordinates of the line segment are obtained by reversely searching the end points belonging to the straight line in the image. 6.The laser vision based weld joint recognition and feature point extraction method according to claim 1, characterized in that, The step S6 is specifically as follows: Step S5-1: obtaining the coordinates of the start point and the end point of the line segment , , the start point coordinate of the line segment , the end point coordinate of the line segment , the start point coordinate of the line segment , the end point coordinate of the line segment , and the direction angles , of the two line segments, respectively.​​ Step S5-2: Line segment Distance of two endpoints to line segment , respectively:​ Step S5-3: If the angle deviation satisfies: , and the distance , satisfies: , and ; then merge into a new line segment, forming a line segment set ; Step S5-4: For the set of line segments fitting a straight line to each line segment as: wherein is the center point of the set of line segment endpoints, is the unit direction vector of the straight line, is an arbitrary point on the straight line; the merged new line segment the start point ) and the end point ) respectively ； 。 7. The laser vision based weld recognition and feature point extraction method according to claim 1, characterized in that, Step S6-1: sorting the merged line segments according to the x coordinates, and constructing a line segment linked list; The step S7 is specifically as follows: Step S6-2: constraint judgment, traverse adjacent line segments , , the gap d and the misplacement s are calculated as: Wherein, the starting point coordinate of the line segment is , the terminal point coordinate is , the starting point coordinate of the line segment is , and the terminal point coordinate is ; Step S6-3: If or is satisfied, then line segment combination is performed. 8.The laser vision based weld joint recognition and feature point extraction method according to claim 1, characterized in that, In step S8, the calculation of the intersection points of the end points of the line segments or adjacent line segments as weld seam feature points based on the weld seam type is specifically as follows: Based on the weld type, the included angle of adjacent line segments or the absolute angle of the line segment is calculated , if the following is satisfied: ; wherein, is the actual angle of the weld, is the prior angle; the line segment combination is retained to obtain the set of effective weld line segments. 9.The laser vision based weld joint recognition and feature point extraction method according to claim 1, characterized in that, In step S8, the determination of the weld point coordinates by using the inter-frame tracking mechanism is specifically as follows: For the corner joint and V joint, the intersection point of the line segments is taken as the feature point, and the intersection point of the line segments is assumed to be: , , ​ For the weld type of lap, butt or edge, take the end points of the line segment as feature points, and known line segment , search for data points on the extension line of the left or right direction of the line segment, and include the continuous points that meet the point-to-line distance ≤ threshold into the line segment until there is no point that meets the condition, and update the end point coordinates. 10.The laser vision based weld joint recognition and feature point extraction method according to claim 1, characterized in that, ​ Assume that the coordinates of feature points of a previous frame are , a tracking window is , and the coordinates of feature points of a current frame are , which satisfy: ； If there are m qualified candidate points in the tracking window , the feature point is: .

Images