V-groove weld deviation detection method based on YOLO model and groove reconstruction

Abstract

The application discloses a V-shaped groove weld deviation detection method based on a YOLO model and groove reconstruction in the field of welding. First, a single YOLO model is trained to synchronously identify and intercept the left and right edge windows of a groove and the center window of a welding torch in a globally collected image of a welding area. Second, the groove edge window image is subjected to a gray scale transformation and a local adaptive threshold method, and the left and right edges of the groove are reconstructed in combination with a random sample consensus algorithm and a multi-level rollback mechanism. Then, the center of the groove is located by a midpoint recursion method based on the reconstructed groove edge, and the center of the welding torch is determined according to the center window of the welding torch. Finally, the deviation between the center of the groove and the center of the welding torch is calculated, and a two-level filtering mechanism combining Kalman filtering and moving average filtering is adopted to obtain a stable weld deviation value. The synchronous positioning of the double groove edges is realized by a single model, the arc light interference is effectively inhibited by combining the local adaptive threshold and the reconstruction method, and the robustness and precision of the weld deviation detection are improved.

Description

Technical Field

[0001] This invention belongs to the field of welding technology, specifically relating to a weld deviation detection method based on the YOLO model and groove reconstruction, which is suitable for online detection of the welding process of V-groove. Background Technology

[0002] In the welding process, efficient and high-precision detection of weld deviations is crucial for real-time control of welding quality. Currently, visual sensing technology is widely used for weld deviation detection, mainly divided into active vision and passive vision. Active vision methods require a laser as a background light source and often suffer from the problem of the detection point leading the welding torch; while passive vision methods do not require an external light source, are less expensive, and can acquire welding features synchronized with the arc position, giving them a significant advantage. However, in V-groove welding, the groove shape is affected by the processing angle, resulting in an uneven distribution of the background and target areas in the acquired welding images, posing a challenge to the accurate positioning and extraction of the groove edges.

[0003] Chinese patent application number 202510683095.X, entitled "Anti-interference visual detection method and system for weld bevel edge position based on dual YOLO model," discloses a bevel edge detection method based on a dual YOLO model. This method has the following shortcomings: 1) It requires two YOLO models to locate the bevel edge and the welding wire image area respectively, resulting in a complex system structure and high computational cost; 2) The image processing method used is not adaptable enough to edge extraction of V-shaped bevel images; 3) This method selects feature regions based on extracting feature points of the welding arc, but the welding arc has random fluctuations, which easily affect the positioning and detection accuracy. Chinese patent publication number CN115255565 B, entitled "Visual sensing detection method and application for narrow gap weld bevel edge based on global pattern recognition," proposes a passive visual sensing method for detecting the bevel edge position in narrow gap welding. Its disadvantages are: 1) The detection accuracy of the method of obtaining the bevel edge by the position of the electric arc is affected by the random fluctuation of the electric arc; 2) The bevel edge image processing method used is not well adapted to images such as V-shaped bevels where the background and the target have a large difference in proportion; 3) Only one side of the bevel edge can be located in a frame of image, and it is not possible to realize the synchronous detection of both sides of the bevel edge at the same time, resulting in low detection efficiency and poor synchronization. Summary of the Invention

[0004] The present invention aims to overcome the shortcomings of the prior art and provide a low-cost, easy-to-operate, and highly adaptable method for detecting weld deviation based on the YOLO model and groove reconstruction, thereby improving the real-time, efficient, adaptable, and high-precision detection performance of V-groove weld deviation.

[0005] The technical solution of the present invention includes the following steps:

[0006] Step 1): Acquire global images of the welding area, label the position coordinates of the left edge window of the bevel, the right edge window of the bevel, and the center window of the welding torch in all global images of the welding area, generate the original coordinate text dataset, input the original coordinate text dataset into the YOLO model for training, and form a trained single YOLO model.

[0007] Step 2): Online acquisition of a real-time global image of the welding area is input into the trained YOLO model to obtain the real-time left edge window image, right edge window image, and welding torch center window image; the real-time left edge and right edge of the bevel are reconstructed using the random sampling consensus algorithm.

[0008] Step 3): Extend the reconstructed left and right edge lines of the bevel to their respective left and right edge window areas. Within the left edge window, take the midpoint A of the left edge line within the window. Draw a horizontal line through midpoint A, intersecting the reconstructed right edge line at point B. Take the midpoint C of line segment AB. Within the right edge window, take the midpoint D of the right edge line within the window. Draw a horizontal line through midpoint D, intersecting the left edge line at point E. Take the midpoint F of line segment DE. Set the x-coordinate of the midpoint H of line segment CF as the bevel center position x. b ;

[0009] Step 4): Process the real-time image of the welding torch center to obtain the welding torch center position x. p ; Calculate the weld deviation sampling value Δw=x p -x b ;

[0010] Step 5): Repeat steps 2)-4) to obtain continuous weld deviation sampling values ​​Δw. Use a sliding window outlier filtering method to filter the continuous deviation sampling values ​​Δw to obtain the weld deviation detection value Δw. f .

[0011] Further, in step 2), grayscale transformation and threshold segmentation are performed on the real-time left edge window image and right edge window image of the bevel to obtain a threshold image containing only the black and white distribution of the left edge and right edge of the bevel.

[0012] Furthermore, the sliding window outlier data filtering includes a first-stage strongly smoothed Kalman filter and a second-stage moving average filter:

[0013] First stage: Strongly smoothed Kalman filter

[0014] Three parameters are set: minimum process noise, initial observation noise, and outlier detection threshold. When the difference between a sampled value Δw of a weld deviation and the predicted value exceeds the outlier detection threshold, it is determined to be an outlier, and the initial observation noise of the sampled value Δw of the weld deviation is adaptively increased.

[0015] Second level: Moving average filter

[0016] The observation noise is set with a sliding window length L. The weld deviation values ​​Δwk1, Δwk2, ..., Δwk are calculated by passing L consecutive weld deviation values ​​obtained through a strongly smoothed Kalman filter. L The average value is used as the final weld deviation detection value Δw. f .

[0017] Furthermore, the black pixels in the threshold image constitute the input candidate point set using the random sampling consensus algorithm. Two points are randomly selected from the candidate point set to determine a candidate line, and its slope k and intercept b are calculated. Then, the minimum distance from all other candidate points to the candidate line is calculated, and points with a distance less than a preset threshold are classified as interior points. The total number of interior points is traversed, and the candidate line containing the most interior points is selected as the final result of the bevel edge fitting. After the iteration, all corresponding interior points are collected to form a point set, and the least squares method is used to fit based on this point set to output the optimal slope and intercept. The absolute value of the slope k of the bevel edge line is reconstructed. When k satisfies the set value tanα0°, the left and right edge lines of the bevel are directly reconstructed to calculate the center position of the bevel; otherwise, refitting is performed.

[0018] Furthermore, we define the angle between the bevel edge and the horizontal line as greater than α0° as the normal case, corresponding to an absolute slope value greater than tanα0°. When the slope of the reconstructed left edge of the bevel is greater than or equal to tanα0° and the slope of the right edge of the bevel is less than or equal to tanα0°, the center position of the bevel is directly calculated from the reconstructed left and right edges of the bevel. Otherwise, if the slope of the reconstructed bevel edge in any frame does not meet this condition, automatic refitting is performed.

[0019] Furthermore, set the maximum number of refits N. max When the number of times the refit mode is activated is greater than N max If a valid bevel edge cannot be obtained afterward, a multi-level backtracking mechanism is activated to obtain the slope value of the bevel edge:

[0020] First-level initial rollback mechanism: When the number of welding images is less than or equal to the set N0 frames, the optimal slope value among the N0 bevel edge slope values ​​in the N0 frames is used as the slope value for reconstructing the bevel edge.

[0021] Second-level intermediate backoff mechanism: When the number of welding images exceeds N0 frames, the number of optimal slope values ​​exceeds the set N.k If there are 1, then calculate N respectively. k The average of the slope and intercept is used to reconstruct the bevel edge of the current frame;

[0022] The third-level ultimate rollback mechanism: If both the first-level and second-level rollback mechanisms fail, a default slope value is used. This default value is obtained by calculating the slope value of the bevel edge processing parameters.

[0023] Compared with existing similar technologies, the advantages and beneficial effects of this invention are mainly:

[0024] 1) Synchronous localization of dual-bevel edges based on a single YOLO model

[0025] This invention achieves rapid and accurate synchronous positioning of the edge regions on both sides of a weld bevel by constructing only a single YOLO model. Compared to traditional methods that rely on arc feature points for positioning, this invention significantly reduces the interference of random arc fluctuations on bevel edge positioning, and significantly improves the robustness and engineering applicability of the algorithm.

[0026] 2) Local adaptive segmentation and reconstruction method to suppress arc light interference

[0027] The local adaptive segmentation and reconstruction method proposed in this invention effectively suppresses the influence of arc light, spatter, and smoke on image feature extraction by fusing grayscale transformation and a local adaptive thresholding algorithm. Simultaneously, by employing a random sampling consensus algorithm and integrating a refitting mode with a multi-level backoff mechanism, stable reconstruction of the bevel edge is achieved, providing technical support for the accurate extraction of weld bevel edges and further improving the accuracy of welding feature recognition.

[0028] 3) Two-stage data filtering mechanism improves weld deviation detection accuracy

[0029] This invention employs a sliding window outlier data filtering method, based on a strongly smoothed Kalman filter and a moving average filter, for data filtering of weld deviation sampling values. This mechanism can effectively filter out noise and abnormal data during the detection process, improve the detection accuracy of weld deviations, thereby enhancing the control accuracy of the welding process and ultimately improving the weld formation quality. Attached Figure Description

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0031] Figure 1 This is a schematic diagram of a weld deviation visual sensing detection system for implementing the weld deviation detection method of the present invention;

[0032] Figure 2 This is a flowchart of the weld deviation detection method of the present invention;

[0033] Figure 3 A schematic diagram of the global image of the welding area when the welding torch is centered, as detected by the trained YOLO model.

[0034] Figure 4 A schematic diagram of the global image of the welding area when the welding torch deviates to the left, detected by the trained YOLO model.

[0035] Figure 5 A magnified view of a localized area used by a trained YOLO model to detect the center point of a bevel.

[0036] Figure 6 This is the process of weld image feature recognition, in which, Figure 6 (a) is a frame of a global image of the V-groove welding area. Figure 6 (b) The left and right edges of the bevel and the welding wire window for adaptive positioning. Figure 6 (c) and Figure 6 (d) are images of the left and right edge windows of the bevel, respectively, captured through the left edge window Class0 and the right edge window Class2. Figure 6 (e) and Figure 6 (f) are the left and right edge threshold images of the bevel after applying the Gaussian weighted local adaptive thresholding algorithm, respectively. Figure 6 (g) and Figure 6 The red lines in (h) represent the left and right edges of the bevel reconstructed using the random sampling consensus algorithm, combined with the refitting pattern and multi-level backoff mechanism. Figure 6 (i) Visualization result of global image fusion of the reconstructed edge and V-groove welding area;

[0037] Figure 7 The results of identifying welding features based on the YOLO model include the left and right edge windows of the bevel, the reconstructed left and right edges of the bevel, the feature points of the left and right edges of the bevel, the center line of the welding torch, and the center line of the bevel.

[0038] Figure 8 To compare the application of two different thresholding methods in binary images of bevel edges; Figure 8 (a) Image of the left edge window of the bevel generated for the YOLO model; Figure 8 (b) The result of the binary image of the left edge of the bevel obtained using the Otsu thresholding method; Figure 8 (c) is the binarized image result of the left edge of the bevel obtained by using the Gaussian weighted local adaptive thresholding algorithm;

[0039] Figure 9 This is an example of weld deviation detection.

[0040] In the figure: 1a. Left side wall of the bevel; 1b. Right side wall of the bevel; 1c. Bottom plate of the bevel; 2. Electric arc; 2b. Arc profile; 3a. Welding torch controller; 3b. Welding torch; 3c. Welding wire; 4. Centerline of the bevel; 5. Light reduction filter system; 6. Camera fixture; 7. Lens; 8. Image acquisition card; 9. Computer; 12. Global image of the welding area;

[0041] θ. The angle between the camera's centerline and the horizontal plane; V W Welding speed; P c The camera begins acquiring signals; Class0. Left edge window of the bevel; Class2. Right edge window of the bevel; △w. Weld deviation; w. Width of the global image of the welding area; h. Height of the global image of the welding area; w0. Width of the left edge window Class0 of the bevel; h0. Height of the left edge window Class0 of the bevel; w2. Width of the right edge window Class2 of the bevel; h2. Height of the right edge window Class2 of the bevel; Line0. Reconstructed left edge line of the bevel; Line2. Reconstructed right edge line of the bevel; A. Midpoint of the reconstructed left edge of the bevel within window Class0; B. Intersection of point A and Line2; C. Midpoint of line segment AB; D. Midpoint of the reconstructed right edge of the bevel within window Class2; E. Intersection of point D and Line0; F. Midpoint of line segment DE; H. Center point of the bevel, i.e., midpoint of line segment CF, located on bevel centerline 4. Detailed Implementation

[0042] like Figure 1 As shown, the weld deviation visual sensing detection system for implementing the weld deviation detection method of the present invention includes: a welding torch 3b, a welding torch controller 3a, a light reduction filter system 5, a camera fixture 6, a camera 7, an image acquisition card 8, and a computer 9. The welding specimen is prepared by machining and has a left bevel wall 1a, a right bevel wall 1b, and a bevel base plate 1c. The welding specimen is placed on a traveling machine tool and welded relative to the welding torch 3b at a welding speed V. w The camera 7, equipped with a light-reducing filter system 5, is positioned diagonally above the welding specimen. Its lens focuses on the end of the welding wire 3c and is arranged at an angle θ to the center line 4 of the welding groove. The camera 7 is fixedly connected to the welding torch 3b via a camera clamp 6, and connected via cables to the output of the welding torch controller 3a and the image acquisition card 8. The image acquisition card 8 is installed in the computer 9 to display and save the images acquired by the camera 7. The welding wire 3c passes through the welding torch 3b and, controlled by the welding torch controller 3a, generates an electric arc 2 at the groove of the specimen for welding. The welding torch controller 3a sends an image acquisition signal P. cThe camera 7 is triggered to acquire a global image of the welding area at a specified time. The image is then transmitted to the computer 9 via the image acquisition card 8. The computer 9 has a built-in weld deviation detection program. This program identifies welding features and calculates weld deviation parameters, then feeds the results back to the welding torch controller 3a via a communication interface or cable, so that the welding torch 3b can be controlled to weld on the test piece with optimized parameters. The light reduction filtering system 5 consists of a filter, a neutral density filter, and a UV filter.

[0043] like Figure 2 As shown, the following steps are used for weld deviation inspection:

[0044] Step 1: Building a single YOLO model

[0045] (1) Sample labeling: Before actual welding, the following is used Figure 1 The camera 7 shown acquires n frames of global images of the welding area, forming the input dataset. In these n frames, the arc is located at different positions on the bevel sidewalls, and the welding torch or wire is positioned centered, to the left, or to the right. Then, using an image annotation tool (e.g., LabelImg), the left edge window of the bevel, the right edge window of the bevel, the center window of the welding torch, or the window of the welding wire in all the global images of the welding area in the input dataset are manually annotated. The position coordinate information of each window is saved as a .txt file, generating a total of n sets of original coordinate text datasets.

[0046] (2) Model generation: Input n sets of original coordinate text datasets into the standard YOLO model for training, and set the number of iterations to n for each set. d The batch size is q, forming a trained YOLO model. The trained YOLO model can identify the left edge of the bevel, the right edge of the bevel, and the center window of the welding torch, and is used for adaptive positioning of the left edge of the bevel, the right edge of the bevel, and the welding wire window.

[0047] Step 2: Application of the YOLO model

[0048] use Figure 1 The camera 7 shown acquires a real-time global image of the welding area online, which is used as input to the trained YOLO model. The trained YOLO model outputs a real-time global image 12 of the welding area, as well as the window position coordinates of the left edge window Class0, the right edge window Class2, and the center window Class1 of the welding torch in the image. Figure 3 The image shown is a global image of the welding area when the welding torch is centered, output by the trained YOLO model (image 12). Figure 4The image shown is a global image 12 of the welding area when the welding torch is deflected to the left, output by the trained YOLO model. The width of the global image 12 is w, and the height is h; the width of the left edge window Class0 of the bevel is w0, and the height is h0; the width of the right edge window Class2 of the bevel is w2, and the height is h2. The arc profile in this global image 12 is 2b.

[0049] Step 3: Generation and processing of left and right edge window images of the bevel.

[0050] like Figure 3 and Figure 4 As shown, based on the window position coordinates, the left edge window image of the bevel, the right edge window image of the bevel, and the center window image of the welding torch are extracted from the global image 12 of the welding area. The left and right edge window images of the bevel are used to calculate the center position of the bevel, and the center window image of the welding torch is used to calculate the center position of the welding torch. The weld deviation value is obtained by the difference between the center position of the bevel and the center position of the welding torch.

[0051] Image processing is performed on the extracted left and right edge window images of the bevel to extract the left and right edges of the bevel. The image processing workflow includes grayscale transformation and threshold segmentation, which can be achieved using Otsu's algorithm or a Gaussian weighted local adaptive thresholding algorithm.

[0052] To eliminate the impact of welding interference on the extraction accuracy of the left and right edges of the bevel, the Random Sample Consensus Algorithm (RANSAC) is used to reconstruct them. Through this image processing method, a threshold image containing only black and white pixels representing the left and right edges of the bevel can be obtained. The black pixels in this threshold image constitute the input candidate point set for the RANSAC algorithm. The RANSAC algorithm randomly selects two points from the candidate point set, determines a candidate line based on these two points, and calculates its slope k and intercept b, recording them in a dictionary named line_info.

[0053] After obtaining the slope k and intercept b, the algorithm calculates the minimum distance from all other candidate points to the candidate line, classifying points with distances less than a preset threshold as inliers. Throughout the iteration process, the algorithm continuously tracks and iterates through the number of inliers contained in all candidate lines, selecting the candidate line with the most inliers as the final result of the bevel edge fitting. After the iteration ends, the algorithm sets out all inliers corresponding to the winning model and uses the least squares method to perform the final accurate fitting based on this pure point set, thereby outputting the optimal slope and intercept parameters to obtain the reconstructed left and right edges of the bevel.

[0054] The absolute value of the slope *k* of the reconstructed bevel edge line using the RANSAC algorithm is calculated. When *k* satisfies the set value *tanα0°*, the left and right edge lines of the bevel are directly reconstructed using the RANSAC algorithm to calculate the bevel center position; otherwise, a refit mode is initiated. Based on bevel processing parameters and numerous sample experiments, the angle between the bevel edge and the horizontal line is set to be greater than *α0°* as the normal condition, corresponding to an absolute slope value greater than *tanα0°*. When the slope of the reconstructed left edge line *line0* is greater than or equal to *tanα0°*, and the slope of the right edge line *line2* is less than or equal to *tanα0°*, the bevel center position is directly calculated using the reconstructed left and right edges. Otherwise, if the slope of the reconstructed bevel edge in any frame does not meet this condition, the refit mode is automatically initiated, and a maximum number of refits *N* is set. max When the number of times the refit mode is activated is greater than N. max If a valid bevel edge is still not obtained, a multi-level backtracking mechanism will be activated to obtain the slope value of the bevel edge and complete the reconstruction of the bevel edge.

[0055] The first-level initial rollback mechanism: When the number of welding images is less than or equal to the set N0 frames, the optimal slope value among the N0 bevel edge slope values ​​in the N0 frames is used as the slope value for reconstructing the bevel edge. The optimal slope value is the bevel edge slope value of the previous frame image.

[0056] Second-level intermediate backoff mechanism: When the number of welding images exceeds N0 frames, the number of optimal slope values ​​exceeds the set N. k If there are multiple N, then calculate the set N respectively. k The average of the slope and intercept is used to reconstruct the bevel edge of the current frame.

[0057] The third-level ultimate rollback mechanism: If the above two rollback mechanisms fail, the default slope value will be enabled. This default value is obtained by calculating the slope value of the bevel edge processing parameters.

[0058] This rollback mechanism significantly enhances the system's fault tolerance during the initial detection phase or when image quality is poor.

[0059] Step 4: Calculate the center position of the bevel

[0060] like Figure 5As shown, based on the above steps, the reconstructed left edge line (line0) and right edge line (line2) of the bevel are obtained. These lines are then extended to their respective left edge window (Class0) and right edge window (Class2) regions. In the left edge window (Class0), the midpoint A of the left edge line (line0) within the Class0 window is taken. A horizontal line (with a slope of 0) is drawn through midpoint A, intersecting the reconstructed right edge line (line2) at point B. The midpoint C of line segment AB is taken. Similarly, in the right edge window (Class2), the midpoint D of the right edge line (line2) within the Class2 window is taken. A horizontal line is drawn through midpoint D, intersecting the left edge line (line0) at point E. The midpoint F of line segment DE is taken. CF is connected, and its midpoint H is taken, where FH = HC. The coordinates of point H are obtained through midpoints C and F. The x-coordinate of the midpoint H of line segment CF is used as the x-coordinate of the bevel center position. b A vertical line is drawn from the center position H of the bevel, and this line is identified as the bevel centerline 4. Therefore, this invention, through the above-mentioned double horizontal construction of drawing horizontal lines twice and midpoint recursion, integrates the local position information of the left and right edges of the bevel with the overall geometric relationship to obtain the bevel center, effectively reducing the influence of unilateral fitting deviation and improving the accuracy of bevel center positioning and the robustness of the system.

[0061] When the left edge window Class0 and the right edge window Class2 of the bevel, located by the trained YOLO model, are at the same horizontal level, such as Figure 3 As shown, midpoints A and E may coincide, and points B and D may also coincide. In this case, the midpoint C of line segment AB coincides with the midpoint F of line segment DE, and this point is also the midpoint H of line segment CF. That is, points C, F, and H are the same. In this case, the x-coordinate of any one of these points can be used as the x-coordinate of the midpoint of the bevel. b The center point H of the bevel can also be determined by points C and F.

[0062] Apart from Figure 3 Except for the special case shown, midpoints A and E, and B and D, do not coincide. In this case, connect the midpoint C of line segment AB with the midpoint F of line segment DE to form line segment CF; take the midpoint H of this line segment such that FH = HC, and use the x-coordinate of point H as the x-coordinate of the bevel center. b ,like Figure 4 As shown, a vertical line is drawn through point H, and this line is defined as the bevel centerline 4, as shown in the enlarged view of Figure 5.

[0063] Step 5: Extract the center position of the welding torch

[0064] To obtain the welding torch centerline, the left and right contours of the welding wire are extracted by image processing of the captured image of the welding torch center window. The welding torch centerline 3c is obtained by calculating the centerline of the contours, or the midpoint of the horizontal coordinate of Class1 in the welding torch center window is taken, and a vertical line is drawn through this midpoint; this line is the welding torch centerline 3c. Figure 4 and Figure 5 As shown. The center position x of the welding torch is obtained by calculating the mean of the abscissa of the data on the centerline 3c of the welding torch. p .

[0065] Step 6: Obtain weld deviation

[0066] Based on the bevel center position x obtained in step 4 b The welding torch centerline position x obtained in step 5 p Subtracting the two values ​​yields the center position x of the bevel. b x with the center position of the welding torch p The deviation was calculated to obtain the sampled value of weld deviation Δw=x. p -x b .

[0067] Step 7: Repeat steps 2-6 to obtain continuous weld deviation sampling values ​​Δw.

[0068] Step 8: Filter the weld deviation sampling value Δw.

[0069] To improve the accuracy and stability of weld deviation sampling values ​​Δw, a sliding window outlier filtering method is used to filter continuous deviation sampling values ​​Δw, ultimately obtaining accurate weld deviation detection values ​​Δw. f The sliding window outlier data filtering includes a first-stage strongly smoothing Kalman filter and a second-stage moving average filter, specifically:

[0070] First stage: Strongly smoothed Kalman filter

[0071] The strongly smoothing Kalman filter is based on a state-space model and recursively filters the weld deviation sample value Δw through two steps: prediction and update. The strongly smoothing Kalman filter requires setting three parameters: the minimum process noise, the initial observation noise, and the outlier detection threshold. During the filtering process, if the difference between a current weld deviation sample value Δw and the predicted value exceeds the outlier detection threshold, the system determines it as an outlier and adaptively increases the initial observation noise of that weld deviation sample value Δw, thereby reducing the influence of outliers on the filtering result and achieving a strongly smoothing effect. The initial observation noise varies with the outlier; if the outlier is large, the initial observation noise increases; if the outlier is small, the initial observation noise decreases. To avoid a decrease in detection accuracy due to insufficient data in the initial stage of filtering, N is set... w Values, the first N wThe Kalman filter is not enabled for Δw; the Δw value is output directly without activating the strong smoothing Kalman filter.

[0072] Second level: Moving average filter

[0073] When the number of data is greater than N w The weld deviation sampling value Δw, after being processed by a strongly smoothed Kalman filter, needs to be further processed by a moving average filter. To reduce observation noise, a sliding window length L needs to be set. This is achieved by calculating L consecutive weld deviation values ​​Δwk1, Δwk2, ... Δwk obtained through the strongly smoothed Kalman filter. L The average value is used as the final weld deviation detection value Δw. f .

[0074] In summary, this invention accurately locates the welding feature window using the YOLO model, and extracts the bevel edge, bevel center, and welding torch center using image processing and edge reconstruction algorithms, thereby calculating the weld deviation. This method effectively solves the problems of difficult feature localization, extraction, and poor anti-interference ability in traditional visual inspection, providing a high-precision deviation detection means for weld tracking in automated welding processes.

[0075] The following provides an embodiment of the present invention and a verification example:

[0076] Example

[0077] This embodiment combines Figure 6 , Figure 7 and Figure 8 The specific implementation process and effects of the weld deviation detection method of the present invention are explained.

[0078] Step 1): Feature window recognition based on the YOLO model

[0079] In this embodiment, the pre-trained YOLO model is stored in computer 9. The YOLO model training parameters include: the number of iterations n. d =75, batch size q=1. Using... Figure 1 The system shown acquires a real-time global image of the V-groove welding area during welding, as shown below. Figure 6 As shown in (a), the width of the global image of the welding area is w=544 pixels and the height is h=544 pixels.

[0080] After inputting the global image of the welding area in this frame into the trained YOLO model, the model output is as follows. Figure 6(b) shows the identification results, including the left and right edges of the bevel and the welding wire window, adaptively located by the YOLO model. The green boxes in the figure mark the left edge window (class0), the welding torch center window (class1), and the right edge window (class2), respectively. The confidence scores for the three windows are 0.836, 0.839, and 0.796, respectively. The size range of the left edge window (class0) is 18–53 pixels wide and 38–266 pixels high; the size range of the right edge window (class2) is 11–38 pixels wide and 22–189 pixels high. When the window size detected by the model falls within the above range, it is considered a valid region.

[0081] Step 2): Generation and processing of bevel edge window images

[0082] Based on the coordinate information of the left edge window Class0 and the right edge window Class2 of the bevel, extract the following from the welding area image: Figure 6 (c) shows the left edge window image of the bevel and Figure 6 (d) shows the right edge window image of the bevel. To extract the bevel edge, the extracted window image is subjected to grayscale transformation, and then the Gaussian weighted local adaptive thresholding algorithm is used for image segmentation. The specific implementation of this algorithm is as follows: for each pixel (x, y) in the grayscale image, a local neighborhood window of size N×N is selected with it as the center, and the Gaussian weighted average value of the pixels in the neighborhood is calculated as the dynamic threshold T(x,y) of that point. In this embodiment, the size N of the local neighborhood window is 11 (N can be adjusted according to actual needs), and the fine-tuning constant C is set to 2. The grayscale image is binarized according to the dynamic threshold T(x,y): the pixel value is set to 255 if it is greater than the dynamic threshold T(x,y), and the value is set to 0 if it is less than or equal to the dynamic threshold T(x,y). The adaptive binary images of the left and right edges of the bevel after being processed by the Gaussian weighted local adaptive thresholding algorithm are shown below. Figure 6 (e) and Figure 6 As shown in (f), it can be clearly observed that the image in this state is composed of a large number of black noise points. The acquisition of the binary image lays the foundation for the accurate extraction of the bevel edge.

[0083] To verify the necessity of the Gaussian weighted local adaptive thresholding algorithm, this embodiment compares it with the Otsu thresholding method, and the results are as follows: Figure 8 As shown. For Figure 8 The left edge window image of the bevel shown in (a) is the result after conventional Otsu thresholding. Figure 8 As shown in (b), the edge information of the bevel is interfered with by the arc light, making it difficult to extract completely. The result processed using the Gaussian weighted local adaptive thresholding algorithm is as follows: Figure 8As shown in (c), the Gaussian weighted local adaptive thresholding algorithm effectively suppresses the interference of uneven lighting and local noise by calculating the Gaussian weighted mean of the local neighborhood for each pixel as a dynamic threshold, and clearly preserves the edge features of the bevel.

[0084] Step 3): Bevel edge reconstruction

[0085] against Figure 6 (e) and Figure 6 The binary image shown in (f) uses the Random Sample Consensus Algorithm (RANSAC) to reconstruct the bevel edges. This algorithm, through iterative random sampling and model validation, has the ability to filter outliers. Figure 6 (e) and Figure 6 The black pixels in (f) constitute the input candidate point set of the RANSAC algorithm.

[0086] The RANSAC algorithm randomly selects two points from the candidate point set. Based on these two points, a candidate line is determined, and its slope *k* and intercept *b* are calculated and recorded in a dictionary named *line_info*. After obtaining the slope and intercept information, the algorithm calculates the minimum distance from all other candidate points to this candidate line, classifying points with a distance less than a preset threshold of 2.0 pixels as inliers. Throughout the iteration process, the algorithm continuously tracks and iterates through the number of inliers contained in all candidate lines, selecting the candidate line with the most inliers as the final result for bevel edge fitting. After the iteration ends, the algorithm extracts all inliers corresponding to the winning model and uses the least squares method to perform a final accurate fit based on this pure point set, thereby outputting the optimal slope and intercept parameters to obtain the reconstructed left and right edges of the bevel, such as... Figure 6 (g) and Figure 6 As shown by the red line in (h), the final output Figure 6 The fused image shown in (i).

[0087] Step 4) Refitting Pattern and Multi-level Backoff Mechanism

[0088] To improve the robustness of edge reconstruction, this embodiment sets up a refitting mode. Based on beveling parameters and extensive sample experiments, the angle α0 between the beveling edge and the horizontal line is set to > 60° as the normal condition, corresponding to an absolute slope value greater than tanα0° = 1.732 (tan60° ≈ 1.732). This means the slope of the reconstructed left edge line (line0) is required to be ≥ 1.732, and the slope of the right edge line (line2) is required to be ≤ -1.732. If the slope of the reconstructed beveling edge in any frame does not meet this condition, the system will automatically activate the refitting mode. The maximum number of refits is set to N. max =3.

[0089] When the refit mode still fails to capture valid edges, the system will initiate a multi-level backoff mechanism:

[0090] The first-level initial rollback mechanism: When the number of welding images is less than or equal to N0 (N0=9) frames, the optimal slope value among the N0 bevel edge slope values ​​in the N0 frames is used as the slope value for reconstructing the bevel edge. The optimal slope value is the bevel edge slope value of the previous frame image.

[0091] Second-level intermediate backoff mechanism: When the number of welding images exceeds N0 frames, the number of optimal slope values ​​exceeds N. k (N) k =3), then calculate N respectively. k The average of the slope and intercept is used to reconstruct the bevel edge of the current frame.

[0092] The third-level ultimate rollback mechanism: If the above two rollback mechanisms fail, the default slope value will be enabled. This default value is obtained by calculating the slope value of the bevel edge processing parameters.

[0093] Step 5): Calculation of weld deviation

[0094] In this embodiment, the left and right edge windows of the bevel, located by the YOLO model in the global image of the welding area, are not at the same horizontal level. Figure 7 As shown, the slope of the reconstructed left edge line Line0 is 3.696, and the slope of the right edge line Line2 is -3.765. Connect CF and take its midpoint H such that FH = HC. The coordinates of point H are obtained through the midpoints C and F. The x-coordinate of the midpoint H obtained from the CF line segment is used as the x-coordinate of the bevel center position. b Following the double-horizontal construction and midpoint recursion method described in the specific implementation, the x-coordinate of the bevel center point H is calculated. b =267 pixels.

[0095] The width of the welding torch center window (Class 1) is 544 pixels, and its horizontal midpoint is 292 pixels. A vertical line drawn from this midpoint forms the welding torch centerline, with its horizontal position x... p =292 pixels.

[0096] Therefore, the weld deviation in the image of this embodiment is Δw=x p -x b =292-267=25 pixels.

[0097] Step 6): Repeat steps 1) through 5) to obtain multiple consecutive weld deviation sampling values ​​Δw. Then, for these multiple consecutive weld deviation sampling values ​​Δw, use a sliding window outlier filtering method to obtain the weld deviation detection value Δw. f .

[0098] This embodiment accurately locates the welding feature window using the YOLO model, and combines the Gaussian weighted local adaptive threshold algorithm, RANSAC edge reconstruction, refitting mode, and multi-level backoff mechanism to achieve stable detection of weld deviation sampling value Δw, verifying the effectiveness and robustness of the method of the present invention.

[0099] Verification Example

[0100] Through continuous welding experiments, the effectiveness of the method of the present invention in detecting weld deviations during dynamic welding was verified, and the improvement of detection accuracy by the sliding window outlier data filtering mechanism was highlighted.

[0101] The experiment used machined V-groove specimens with a groove length of 210 mm. Welding parameters were set as follows: welding current 250-300 A, welding voltage 27-29 V, wire extension 18 mm, wire diameter 1.2 mm, welding speed 240 mm / min, gas type / flow rate Ar-20%CO2, flow rate 20 L / min. The image acquisition system used a camera equipped with a 970 nm narrowband filter and a 30% transmittance neutral density filter. Camera parameters were set to aperture F11 and exposure time 0.3 ms.

[0102] To verify the effectiveness of the weld deviation detection method, a one-way deviation between the welding torch center and the bevel center was artificially set before welding. During welding, the welding torch initially deviated towards the right side wall of the bevel, and then gradually transitioned to deviating towards the left side wall, causing the welding torch center to exhibit a continuous changing trend relative to the bevel center. A total of 2561 frames of global images of the welding area were acquired during the entire welding process.

[0103] 1. Weld deviation detection results

[0104] Using the weld deviation detection results from the trained YOLO model, the aforementioned 2561 frames of images were processed frame by frame to obtain a sequence of weld deviation sampling values, such as... Figure 9 As shown by the black line. The test results show that the weld deviation value gradually transitions from positive to the right to negative to the left, with an overall deviation range of approximately +55 pixels to -31 pixels. This is consistent with the welding torch offset trend preset in the experiment, and preliminarily verifies the effectiveness of the detection method.

[0105] 2. Sliding window outlier data filtering

[0106] To eliminate abnormal sampling values ​​in individual frames caused by factors such as arc interference and splashing, and to improve the continuity and stability of detection results, this invention designs a two-stage sliding window outlier data filtering mechanism. The filtered result is as follows: Figure 9 As shown by the red curve in the middle. The specific implementation of this filtering mechanism is as follows:

[0107] First stage: Strongly smoothed Kalman filter

[0108] A recursive filtering process is applied to the deviation sample value Δw. The strongly smoothing Kalman filter is configured with three parameters: minimum process noise (1e-6), initial observation noise (2.0), and an outlier detection threshold of 5.0. During the filtering process, when the difference between a current weld deviation sample value Δw and the predicted value exceeds the outlier detection threshold of 5.0, it is determined to be an outlier, and the initial observation noise of that weld deviation sample value Δw is adaptively increased, thereby reducing the influence weight of outliers on the filtering result and achieving a strongly smoothing effect. The initial observation noise varies with the outlier; if the outlier is large, the initial observation noise increases; if the outlier is small, the initial observation noise decreases. To avoid a decrease in detection accuracy due to insufficient data in the initial stage of filtering, the first N... w (N) w =15) Δw values ​​are output directly without enabling Kalman filtering, and strong smoothing Kalman filtering is not enabled.

[0109] Second level: Moving average filter

[0110] When the number of data is greater than N w The weld deviation sampling value Δw, after being processed by a strongly smoothed Kalman filter, needs to be further processed by a moving average filter. The observation noise is set with a sliding window length L=5. The average of five consecutive weld deviation values ​​Δwk1, Δwk2, Δwk3, Δwk4, and Δwk5 obtained through the strongly smoothed Kalman filter is calculated as the final weld deviation detection value Δw. f During the filtering process, the moving average filter requires at least three frames of data to accumulate before calculating the average value; otherwise, it directly returns the current Kalman filter value. This mechanism further suppresses local fluctuations in the output sequence by using the arithmetic mean of the filtering results of the most recent five frames.

[0111] 3. Filtering Effect Analysis

[0112] contrast Figure 9 As can be seen from the two curves before and after filtering, the original deviation sampling value (black line) contains several peak abnormal values ​​caused by welding process disturbances, while the deviation value (red line) curve after two-stage filtering is smooth and continuous, effectively eliminating abnormal fluctuations and accurately reflecting the true trend of the change of the welding torch center relative to the bevel center.

[0113] The results show that the sliding window outlier data filtering method proposed in this invention, through the organic combination of Kalman filtering and moving average filtering, can effectively eliminate abnormal sampling values ​​in welding image sequences, improving the accuracy and stability of weld deviation detection. This method is particularly suitable for suppressing transient interference during the equipment startup phase in industrial vision systems, providing reliable data support for the real-time control of automated welding processes.

Claims

1. A method for detecting V-groove weld deviation based on YOLO model and groove reconstruction, characterized in that... Includes the following steps: Step 1): Acquire global images of the welding area, label the position coordinates of the left edge window of the bevel, the right edge window of the bevel, and the center window of the welding torch in all global images of the welding area, generate the original coordinate text dataset, input the original coordinate text dataset into the YOLO model for training, and form a trained single YOLO model. Step 2): Online acquisition of a real-time global image of the welding area is input into the trained YOLO model to obtain the real-time left edge window image, right edge window image, and welding torch center window image; the real-time left edge and right edge of the bevel are reconstructed using the random sampling consensus algorithm. Step 3): Extend the reconstructed left and right edge lines of the bevel to their respective left and right edge window areas. Within the left edge window, take the midpoint A of the left edge line within the window. Draw a horizontal line through midpoint A, intersecting the reconstructed right edge line at point B. Take the midpoint C of line segment AB. Within the right edge window, take the midpoint D of the right edge line within the window. Draw a horizontal line through midpoint D, intersecting the left edge line at point E. Take the midpoint F of line segment DE. Set the x-coordinate of the midpoint H of line segment CF as the bevel center position x. b ; Step 4): Process the real-time image of the welding torch center to obtain the welding torch center position x. p ; Calculate the weld deviation sampling value Δw=x p -x b ; Step 5): Repeat steps 2)-4) to obtain continuous weld deviation sampling values ​​Δw. Use a sliding window outlier filtering method to filter the continuous deviation sampling values ​​Δw to obtain the weld deviation detection value Δw. f .

2. The method for detecting V-groove weld deviation based on YOLO model and groove reconstruction according to claim 1, characterized in that: In step 2), grayscale transformation and threshold segmentation are used to process the real-time left edge window image and right edge window image of the bevel to obtain a threshold image containing only the black and white distribution of the left and right edges of the bevel.

3. The method for detecting V-groove weld deviation based on YOLO model and groove reconstruction according to claim 2, characterized in that: The black pixels in the threshold image constitute the input candidate point set using the random sampling consensus algorithm. Two points are randomly selected from the candidate point set to determine a candidate line, and its slope k and intercept b are calculated. Then, the minimum distance from all other candidate points to the candidate line is calculated, and points with a distance less than a preset threshold are classified as interior points. The total number of interior points is traversed, and the candidate line containing the most interior points is selected as the final result of the bevel edge fitting. After the iteration, all corresponding interior points are collected to form a point set, and the least squares method is used to fit based on this point set to output the optimal slope and intercept. The absolute value of the slope k of the bevel edge line is reconstructed. When k satisfies the set value tanα0°, the left and right edge lines of the bevel are directly reconstructed to calculate the center position of the bevel; otherwise, refitting is performed.

4. The method for detecting V-groove weld deviation based on YOLO model and groove reconstruction according to claim 3, characterized in that: The normal case is defined as the angle between the bevel edge and the horizontal line being greater than α0°, corresponding to an absolute slope value greater than tanα0°. When the slope of the reconstructed left edge of the bevel is greater than or equal to tanα0° and the slope of the right edge of the bevel is less than or equal to tanα0°, the center position of the bevel is directly calculated from the reconstructed left and right edges of the bevel. Otherwise, if the slope of the reconstructed bevel edge in any frame does not meet this condition, automatic refitting is performed.

5. The method for detecting V-groove weld deviation based on YOLO model and groove reconstruction according to claim 4, characterized in that: Set the maximum number of refits N. max When the number of times the refit mode is activated is greater than N max If a valid bevel edge cannot be obtained afterward, a multi-level backtracking mechanism is activated to obtain the slope value of the bevel edge: First-level initial rollback mechanism: When the number of welding images is less than or equal to the set N0 frames, the optimal slope value among the N0 bevel edge slope values ​​in the N0 frames is used as the slope value for reconstructing the bevel edge. Second-level intermediate backoff mechanism: When the number of welding images exceeds N0 frames, the number of optimal slope values ​​exceeds the set N. k If there are 1, then calculate N respectively. k The average of the slope and intercept is used to reconstruct the bevel edge of the current frame; The third-level ultimate rollback mechanism: If both the first-level and second-level rollback mechanisms fail, a default slope value is used. This default value is obtained by calculating the slope value of the bevel edge processing parameters.

6. The method for detecting V-groove weld deviation based on YOLO model and groove reconstruction according to claim 1, characterized in that: In step 5), the sliding window outlier data filtering includes a first-stage strongly smoothed Kalman filter and a second-stage moving average filter: First stage: Strongly smoothed Kalman filter Three parameters are set: minimum process noise, initial observation noise, and outlier detection threshold. When the difference between a sampled value Δw of a weld deviation and the predicted value exceeds the outlier detection threshold, it is determined to be an outlier, and the initial observation noise of the sampled value Δw of the weld deviation is adaptively increased. Second level: Moving average filter The observation noise is set with a sliding window length L. The weld deviation values ​​Δwk1, Δwk2, ..., Δwk are calculated by passing L consecutive weld deviation values ​​obtained through a strongly smoothed Kalman filter. L The average value is used as the final weld deviation detection value Δw. f .

7. The method for detecting V-groove weld deviation based on YOLO model and groove reconstruction according to claim 6, characterized in that: When the initial observation noise changes with outliers, if the outliers are large, the initial observation noise increases; if the outliers are small, the initial observation noise decreases.

8. The method for detecting V-groove weld deviation based on YOLO model and groove reconstruction according to claim 6, characterized in that: Set N w Values, the first N w The Δw value is directly output without enabling the strongly smoothed Kalman filter.

9. The method for detecting V-groove weld deviation based on YOLO model and groove reconstruction according to claim 2, characterized in that: Image segmentation is performed using a Gaussian weighted local adaptive thresholding algorithm: For each pixel (x, y) in the grayscale image, a local neighborhood window of size N×N is selected with the pixel as the center. The Gaussian weighted average value of the pixels in the neighborhood is calculated as the dynamic threshold T(x,y) for that point. The grayscale image is binarized according to the dynamic threshold T(x,y). Pixel values ​​greater than the dynamic threshold T(x,y) are set to 255, and values ​​less than or equal to the dynamic threshold T(x,y) are set to 0.

10. The method for detecting V-groove weld deviation based on YOLO model and groove reconstruction according to claim 1, characterized in that: In step 4), when processing the real-time welding torch center image, image processing is performed on the welding torch center window image to extract the left and right contours of the welding wire, and the center line of the contour is calculated to obtain the welding torch center line. Alternatively, the midpoint of the horizontal coordinate of the welding torch center window is taken, and a vertical line is drawn through this midpoint; this line is the welding torch center line. The welding torch center position x is obtained by calculating the mean of the horizontal coordinates of the data on the welding torch center line. p .

Images