A Passive Visual Weld Seam Tracking Method Based on Deep Learning Semantic Segmentation

Abstract

This invention discloses a passive visual weld seam tracking method based on deep learning semantic segmentation, comprising the following steps: establishing a vision system and acquiring weld seam images during PAW welding without the aid of external light; segmenting the weld seam images based on semantic segmentation to extract the arc light and weld seam components; labeling connected components in the segmented images to eliminate missegmented parts; determining the welding torch position based on the geometric features of the arc light and performing straight line fitting on the weld seam to calculate the offset between the two; and filtering the current result based on the previously calculated offset. The beneficial effects of this invention are: by constructing a BiseNetV2 network, accurate segmentation of the arc light and weld seam and accurate positioning of the welding torch are achieved; at the same time, the filtering method can obtain higher accuracy to meet the accuracy requirements of real-time weld seam tracking.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically a passive visual weld seam tracking method based on deep learning semantic segmentation. Background Technology

[0002] Plasma arc welding is an important branch of welding technology, offering easier control over the arc initiation point to control weld quality, and producing a more concentrated arc. In recent years, with continuous optimization of welding processes, plasma arc welding has been widely used in various industrial manufacturing sectors. However, when using plasma arc welding to weld spliced ​​plates, how to enable robots to automatically correct their trajectory during the welding process is a pressing technical problem. Plasma arc welding generates a strong arc light, and the weld seam itself has indistinct features, making weld seam positioning relatively difficult. Summary of the Invention

[0003] The purpose of this invention is to provide a passive visual weld seam tracking method based on deep learning semantic segmentation to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a passive visual weld seam tracking method based on deep learning semantic segmentation, comprising the following steps:

[0005] Step 1: Establish a vision system and acquire weld images during PAW welding without the aid of external light;

[0006] Step 2: Segment the weld seam image based on semantic segmentation and extract the arc light and weld seam portions;

[0007] Step 3: Perform connected component labeling on the segmented image to eliminate incorrectly segmented parts;

[0008] Step 4: Determine the welding torch position based on the geometric characteristics of the arc light and perform a straight line fitting on the weld seam to calculate the offset between the two.

[0009] Step 5: Filter the current result based on the previously calculated offset.

[0010] Furthermore, in step one, the vision system specifically includes a camera and a robotic arm. The vision system adopts a passive vision scheme, that is, it does not use an additional light source for illumination, and uses a neutral density filter to reduce the influence of arc light on image acquisition. The camera is set on the robotic arm at a 30-degree angle to the welding platform, so the camera and the robotic arm are relatively stationary, and the actual distance can be directly calculated from the acquired image through mapping.

[0011] Furthermore, in step two, multi-class segmentation is directly adopted, and BiseNetV2 is used as the network architecture to extract the features of the arc light and weld seam, achieving real-time high-precision segmentation of the weld seam and arc light parts of the image. The features include geometric shape and grayscale distribution details. In order to balance the uneven pixel ratio between the weld seam and arc light parts in the weld seam image, a hard example mining strategy is adopted during training to improve the segmentation effect. The introduced loss function is as follows:

[0012]

[0013] Among them, w k The weights of the categories are K and p, respectively. kn Let f be the grayscale value of pixel n of class k in gt. kn The corresponding predicted value, where N is the number of pixels selected for hard case mining.

[0014] Furthermore, in step three, based on the pixel grayscale spatial distribution information of the segmented weld seam portion of the image, the image is labeled with connected components. Based on the number of pixels contained in each connected component, the connected component with the largest number of pixels is selected as the weld seam location, thereby eliminating the influence introduced by missegmentation.

[0015] Furthermore, in step four, the actual position of the welding torch is determined within the segmented arc portion. The top of the arc, whose geometry is relatively stable during the welding process, is selected as the center selection area of ​​the welding torch. The weld seam is fitted using the least squares method to calculate the corresponding linear equation, as shown below:

[0016]

[0017] Where i is the pixel number, m is the total number of pixels, f(x) is the equation of the line, (x m ,y m Let f(x) be the pixel coordinates. Find the linear equation f(x) that minimizes the value of the above expression. This is the description of the final weld position. Substitute the selected welding torch position ordinate into the linear equation to calculate the offset between the weld and the welding torch.

[0018] Furthermore, in step five, when calculating the offset at the current moment, the offset calculated in the current frame is filtered based on the offsets calculated in the previous few frames. If the variance between the offset obtained in the current frame and the offsets in the previous M frames is small, mean filtering is used; otherwise, median filtering is used to improve tracking accuracy.

[0019] Furthermore, the range of M is 0-20 frames.

[0020] Compared with the prior art, the beneficial effects of the present invention are: by constructing the BiseNetV2 network, precise segmentation of arc light and weld seam and precise positioning of welding torch position are achieved; at the same time, the use of filtering methods can obtain higher accuracy to meet the accuracy requirements of real-time tracking of weld seam. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the welding operation state of the present invention.

[0023] Figure 2 This is a schematic diagram of the semantic segmentation network architecture adopted in this invention;

[0024] Figure 3 This is a schematic diagram illustrating the semantic segmentation effect of the present invention;

[0025] Figure 4 This is a schematic diagram of the image processing for segmented connected component labeling according to the present invention;

[0026] Figure 5 A schematic diagram illustrating the method for calculating the offset between the welding torch and the weld seam;

[0027] Figure 6 Error graph of the result obtained without using filtering method;

[0028] Figure 7 This is an error graph showing the results obtained after using the filtering method. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] Please see Figure 1 In this embodiment of the invention, a passive vision system based on deep learning semantic segmentation includes a high dynamic range camera and a robotic arm. The high dynamic range camera is fixed to the robotic arm by welding. The lens of the high dynamic range camera is equipped with an anti-splash lens, a neutral density filter, and a light source controller. The light source controller is used to adjust the light source at the weld seam, and the neutral density filter is used to reduce the impact of electric arc light on image acquisition.

[0031] Furthermore, the angle between the high dynamic range camera and the welding platform where the robotic arm is located is 30 degrees, which is beneficial for both shooting and welding. Compared to other angles, when the angle is greater than 30 degrees, the lens of the high dynamic range camera is closer to the top of the welding point, and the arc light shines directly on it, which is not conducive to image acquisition. Moreover, the temperature faced by the lens is also better, and the lens is more likely to be damaged. On the other hand, when the angle is less than 30 degrees, the tilt angle is too large, which is not conducive to welding.

[0032] A passive visual weld seam tracking method based on deep learning semantic segmentation includes the following steps:

[0033] Step 1: Acquire weld images during PAW welding using a passive vision system without the aid of external light;

[0034] Step 2: Segment the weld seam image based on semantic segmentation and extract the arc light and weld seam portions;

[0035] Step 3: Perform connected component labeling on the segmented image to eliminate incorrectly segmented parts;

[0036] Step 4: Determine the welding torch position based on the geometric characteristics of the arc light and perform a straight line fitting on the weld seam to calculate the offset between the two.

[0037] Step 5: Filter the current result based on the previously calculated offset.

[0038] In step one, the vision system adopts a passive vision scheme, that is, no additional light source is used for illumination. The light-reducing filter is used to reduce the influence of the arc light on image acquisition. The camera is set on the robotic arm at a 30-degree angle to the welding platform. Therefore, the camera and the robotic arm are relatively stationary, and the actual distance can be directly calculated from the acquired image through mapping.

[0039] like Figure 2 As shown, in step two, multi-class segmentation is directly adopted. BiseNetV2 is used as the network architecture to extract the features of arc light and weld seam, achieving real-time high-precision segmentation of the weld seam and arc light parts of the image. The features include geometric shape and grayscale distribution details. In order to balance the uneven pixel ratio between the weld seam and arc light parts in the weld seam image, a hard example mining strategy is adopted during training to improve the segmentation effect. The introduced loss function is shown below:

[0040]

[0041] Among them, w k The weights of the categories are K and p, respectively. kn Let f be the grayscale value of pixel n of class k in gt. knThe corresponding predicted value, where N is the number of pixels selected for hard case mining.

[0042] like Figure 3-4 As shown, in step three, the image is marked with connected components based on the pixel grayscale spatial distribution information of the segmented weld seam portion of the image. Based on the number of pixels contained in each connected component, the connected component with the largest number of pixels is selected as the weld seam location, thereby eliminating the influence introduced by missegmentation.

[0043] In step four, the actual position of the welding torch is determined within the segmented arc portion. The top of the arc, whose geometry is relatively stable during the welding process, is selected as the center area of ​​the welding torch. The weld seam is fitted using the least squares method to calculate the corresponding linear equation, as shown below:

[0044]

[0045] Where i is the pixel number, m is the total number of pixels, f(x) is the equation of the line, (x m ,y m Let 'x' be the pixel coordinates. The equation f(x) is the linear equation that minimizes the value of the above expression, which describes the final weld position. Substituting the selected welding torch position's ordinate into the linear equation, the offset between the weld and the welding torch is calculated, such as... Figure 5 As shown.

[0046] Finally, when calculating the offset at the current moment, the offset calculated in the current frame is filtered based on the offsets calculated in the previous 10 frames. If the variance between the offset obtained in the current frame and the offsets in the previous frames is small, mean filtering is used; otherwise, median filtering is used. This improves tracking accuracy. Figure 6 This is a schematic diagram of the error before filtering. Figure 7 This is a schematic diagram of the filtered error. (See attached image) Figure 7 As can be seen, the average error is within 0.07mm, which meets the accuracy requirements for real-time weld seam tracking.

[0047] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0048] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A passive visual weld seam tracking method based on deep learning semantic segmentation, characterized in that, Includes the following steps: Step 1: Establish a vision system and acquire weld images during PAW welding without the aid of external light; In step one, the vision system specifically includes a camera and a robotic arm. The vision system adopts a passive vision scheme. The camera lens is equipped with a neutral density filter. The camera is mounted on the robotic arm at a 30-degree angle to the welding platform. Step 2: Segment the weld seam image based on semantic segmentation and extract the arc light and weld seam portions; In step two, multi-class segmentation is directly adopted, and BiseNetV2 is used as the network architecture to extract features of arc light and weld seam. These features include geometric shape and grayscale distribution details. During training, a hard example mining strategy is used to improve the segmentation effect. The introduced loss function is shown below: ； in, Weights for each category For the number of categories, Let be the grayscale value of pixel n of class k in the real labeled image. For the corresponding predicted value, The number of pixels selected for difficult cases; Step 3: Perform connected component labeling on the segmented image to eliminate incorrectly segmented parts; Step 4: Determine the welding torch position based on the geometric characteristics of the arc light and perform a straight line fitting on the weld seam to calculate the offset between the two. In step four, the actual position of the welding torch is determined within the segmented arc portion. The top of the arc, whose geometry is relatively stable during the welding process, is selected as the center area of ​​the welding torch. The weld seam is fitted using the least squares method to calculate the corresponding linear equation, as shown below: ； Where i is the pixel number and m is the total number of pixels. The equation of the line is... Given the pixel coordinates, find the equation of the line that minimizes the value of the above expression. This describes the final weld position. Substitute the selected welding torch position's ordinate into the linear equation to calculate the offset between the weld and the welding torch. Step 5: Filter the current result based on the previously calculated offset.

2. The passive visual weld seam tracking method based on deep learning semantic segmentation according to claim 1, characterized in that: In step three, based on the pixel grayscale spatial distribution information of the segmented weld seam portion of the image, the image is labeled with connected components. Based on the number of pixels contained in each connected component, the connected component with the largest number of pixels is selected as the weld seam location.

3. The passive visual weld seam tracking method based on deep learning semantic segmentation according to claim 1, characterized in that: In step five, when calculating the offset at the current moment, the offset calculated in the current frame is filtered based on the offsets calculated in the previous few frames. If the variance between the offset obtained in the current frame and the offsets in the previous M frames is small, mean filtering is used; otherwise, median filtering is used.

4. The passive visual weld seam tracking method based on deep learning semantic segmentation according to claim 3, characterized in that: The range of M is 0-20 frames.

Images