Welding arc light filtering method based on generative adversarial network

Abstract

The invention relates to the technical field of industrial image processing, in particular to a welding arc light filtering method based on a generative adversarial network. The method comprises the following steps: acquiring a robot welding image, and generating gray-scale maps and edge maps of an arc light image and a non-arc light image by using a Canny edge detection algorithm to obtain an initial edge map and a gray-scale map sequence; based on the initial edge image and the gray-scale image sequence, training an edge enhancement network, and generating a complete edge image through adversarial loss and L1 loss; inputting the complete edge image and the arc light image into an arc light filtering network, filtering and reconstructing the texture of an arc light area by using a residual convolution block attention module, and generating a non-arc light image; and based on the non-arc light image and the complete edge image, performing evaluation by using image quality evaluation indexes such as SSIM and PSNR, and generating image quality feedback. The method can effectively recover the detail texture of the arc light area, and improves the visual monitoring quality.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial image processing, and in particular to a welding arc filtering method based on a generative adversarial network. Background Art

[0002] In modern industrial manufacturing, robot automatic welding is widely used in electronic manufacturing, mechanical processing, aerospace and other fields. Real-time visual monitoring during automatic welding is of great significance to improving welding quality and ensuring production safety. However, due to the high intensity and concentrated area of ​​arc light during welding, it is easy to cause overexposure of image sensors, interfere with the effective recognition of welding details by the visual system, and thus affect the judgment of welding quality and welding status.

[0003] At present, traditional arc filtering methods are mostly based on simple filtering and noise reduction techniques, but it is often difficult to achieve the ideal arc removal effect when faced with problems such as complex welding image backgrounds and blurred texture details in the arc area. In addition, although some image processing methods based on deep learning have certain noise filtering capabilities, they still have defects such as loss of arc area details and unclear texture recovery when processing welding images with complex backgrounds, which limits the application of machine vision in actual industrial environments.

[0004] Therefore, there is an urgent need for a welding arc filtering method that can adapt to complex backgrounds. By accurately filtering the arc light and clearly restoring the texture details of the arc light area, high-quality arc-free monitoring images can be provided for the welding process, thereby improving the accuracy and reliability of visual monitoring. Summary of the invention

[0005] The present invention provides a welding arc filtering method based on a generative adversarial network to solve the problem of how to accurately filter arc interference and restore the detailed texture of the arc area based on a welding image under a complex background through a generative adversarial network to improve the visual monitoring quality of the welding scene.

[0006] In order to solve the above technical problems, the present invention provides a welding arc filtering method based on a generative adversarial network, comprising:

[0007] Acquire robot welding images, and use the Canny edge detection algorithm to generate grayscale images and edge images of arc images and non-arc images to obtain initial edge images and grayscale image sequences;

[0008] Based on the initial edge map and grayscale image sequence, an edge enhancement network is trained to generate a complete edge map through adversarial loss and L1 loss;

[0009] Inputting the complete edge map and the arc light image into the arc light filtering network, filtering and reconstructing the texture of the arc light area through the residual convolution block attention module, and generating an arc light-free image;

[0010] Based on the arc-free image and the complete edge map, the arc-free image is evaluated using image quality evaluation indicators such as SSIM and PSNR to generate image quality feedback for optimizing parameters of the edge enhancement network and the arc filtering network.

[0011] Furthermore, the step of acquiring the robot welding image includes:

[0012] An original image in a welding operation scene is collected, and the original image is preprocessed to obtain an arc image containing an arc area and an arc-free image.

[0013] Furthermore, the step of generating the grayscale image includes:

[0014] The arc light image and the no-arc light image are converted into grayscale images to obtain an arc light grayscale image and a no-arc light grayscale image.

[0015] Furthermore, the steps of the Canny edge detection algorithm include:

[0016] An arc edge map and a non-arc edge map are generated based on the arc grayscale map and the non-arc grayscale map to obtain the initial edge map and grayscale map sequence.

[0017] Furthermore, the step of training the edge enhancement network includes:

[0018] The generator of the edge enhancement network is trained using adversarial loss and L1 loss to generate the complete edge map.

[0019] Furthermore, the step of inputting the complete edge map and the arc light image into the arc light filtering network includes:

[0020] The complete edge map and the arc light image are spliced ​​at channel level and then input into an arc light filtering network for feature reconstruction.

[0021] Furthermore, the arc light filtering network includes a residual convolution block attention module, which is used to enhance the texture details of the arc light area during the reconstruction process.

[0022] Furthermore, the training step of the edge enhancement network also includes:

[0023] The least squares generative adversarial loss (LSGAN) is introduced into the discriminator of the edge enhancement network to improve the stability of network training and optimize the edge clarity of the complete edge map.

[0024] Furthermore, the image quality evaluation step based on the arc-free image and the complete edge image includes:

[0025] The arc-free image is evaluated using structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) indicators to generate image quality feedback data.

[0026] Furthermore, the step of generating image quality feedback includes:

[0027] After calculating the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) of the arc-free image, the image quality feedback is used to adjust the learning rate of the edge enhancement network and the arc filtering network to improve the quality of subsequent arc-free image generation.

[0028] The key innovations of the present invention include: introducing a generative adversarial network and a residual convolution block attention module to perform fine texture filtering in the arc area to ensure that the edge and detail information is fully preserved; and adaptively optimizing network parameters through an image quality feedback mechanism to cope with different complex welding scenarios.

[0029] The following are its main beneficial effects:

[0030] (1) Improving image filtering accuracy: The present invention uses the Canny edge detection algorithm to obtain the initial edge map and grayscale image sequence, and trains based on the generative adversarial network to generate a complete edge map through adversarial loss and L1 loss. This step can accurately capture edge details under complex backgrounds and significantly improve image filtering accuracy.

[0031] (2) Enhanced detail recovery in arc light areas: The arc light image is processed by the residual convolutional block attention module (RCBAM). The present invention can specifically enhance the texture details of the arc light area during the reconstruction process and effectively filter out arc light interference. This module focuses on the local important areas of the image, avoiding the blurring of texture information by traditional methods, thereby ensuring the clarity of the output image.

[0032] (3) Adaptive parameter optimization: Image quality feedback data is generated using image quality evaluation indicators such as SSIM (structural similarity index) and PSNR (peak signal-to-noise ratio) to dynamically optimize the parameter settings of the edge enhancement network and arc filtering network. Through the feedback mechanism, this method can achieve adaptive adjustment in different welding scenarios to ensure the stability and quality of arc-free image generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic flow chart of a welding arc filtering method based on a generative adversarial network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0035] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0036] Reference Figure 1 , is a flow chart of a welding arc filtering method based on a generative adversarial network provided by an embodiment of the present invention, and the flow chart may at least include steps S100-S400:

[0037] S100, acquiring a robot welding image, and using a Canny edge detection algorithm to generate a grayscale image and an edge image of an arc image and a non-arc image, to obtain an initial edge image and grayscale image sequence;

[0038] S200, based on the initial edge map and grayscale map sequence, training the edge enhancement network, and generating a complete edge map through adversarial loss and L1 loss;

[0039] S300, inputting the complete edge map and the arc light image into an arc light filtering network, filtering and reconstructing the texture of the arc light area through a residual convolutional block attention module (RCBAM), and generating an arc light-free image;

[0040] S400. Based on the arc-free image and the complete edge map, the arc-free image is evaluated using image quality evaluation indicators such as SSIM and PSNR to generate image quality feedback for optimizing parameters of the edge enhancement network and the arc filtering network.

[0041] Step S100 at least includes steps S110-S130:

[0042] S110, acquiring a robot welding image, and extracting an arc image and a non-arc image from the welding image.

[0043] In step S110, the data at the input end is the original image set I generated during the welding operation. raw , which includes the overexposed area caused by the high-intensity arc generated during the welding process. By analyzing the original image set, each frame of the image is marked, and the image with obvious arc area is set as the arc image I arc , and the part of the image area without arc light is marked as arc-free image I non-arc This step ensures that the arcing area and the non-arcing area can be analyzed separately in the subsequent processing.

[0044] S120, performing grayscale conversion on the arc image and the non-arc image to obtain an arc grayscale image and a non-arc grayscale image.

[0045] In step S120, the arc image I obtained in step S110 is arc and arc-free image I non-arc As input data, the grayscale conversion function G(I) is applied respectively, which is realized by the following formula:

[0046] G(I)=0.299×I R +0.587×I G +0.114×I B

[0047] Among them, I R ,I G ,I B Represent the red, green and blue channels of the image respectively. After the conversion function, the arc image I arc and arc-free image I non-arc Generate arc grayscale image G respectively arc And the arc-free grayscale image G non-arc This grayscale conversion step makes the subsequent edge detection operation more efficient and reduces the computational redundancy caused by color information.

[0048] S130 , based on the arc light grayscale image and the non-arc light grayscale image, an edge image is generated using a Canny edge detection algorithm to obtain an initial edge image and a grayscale image sequence.

[0049] In step S130, the grayscale image G obtained from step S120 is detected using the Canny edge detection algorithm. arc and G non-arc Specifically, the initial edge map is obtained by the following steps:

[0050] Gaussian filtering smoothing: First, G arc and G non-arc Gaussian filtering is applied for smoothing, and the filter size is set to 5×5, with a standard deviation of σ = 2. The Gaussian filtering formula is as follows:

[0051]

[0052] Among them, G smooth Represents the smoothed grayscale image, where x and y are the horizontal and vertical coordinates of the image, ensuring that the image is suitable for edge detection after denoising.

[0053] Use the Sobel operator to smooth the G smooth Processing, calculation of horizontal direction and vertical direction The gradient of

[0054]

[0055] in, It is the gradient intensity map of arc light and no-arc light images, indicating the edge intensity of each pixel in the image.

[0056] Non-maximum suppression and hysteresis thresholding are used to remove unclear edges. Two thresholds, min Val = 50 and max Val = 100, are set, and edge pixels in the image are screened based on these thresholds to retain pixels with strong edge features.

[0057] After processing by the Canny edge detection algorithm, the initial edge map E of the arc is obtained. arc and no arc initial edge diagram E non-arc The above data G arc , G non-arc 、E arc 、E non-arc Together they constitute the initial edge map and grayscale image sequence, which are used as input in the next step S200 for further training and optimization of the edge enhancement network.

[0058] Step S200 at least includes steps S210-S230:

[0059] S210: Based on the initial edge map and grayscale map sequence, input them into an edge enhancement network to generate a complete edge map.

[0060] In step S210, the initial edge map and grayscale map sequence (including the arc grayscale map G arc , no arc grayscale image G non-arc 、Arc edge diagram E arc and no arc edge Figure E non-arc ) as input, and then passed to the generator and discriminator of the edge enhancement network. The edge enhancement network is based on the conditional generative adversarial network (Conditional GAN, CGAN) structure, and the generator input is (Garc ,E arc ) to generate a preliminary complete edge map in:

[0061]

[0062] here, represents the generator model of the edge-enhanced network, Represents the complete edge graph output by the generator. By inputting (G arc ,E arc ) and the edge image E without arc light non-arc As comparison data, the generator outputs a preliminary edge graph structure, laying the foundation for the subsequent optimization steps.

[0063] S220, training the edge enhancement network, optimizing the generator using adversarial loss and L1 loss, so that the generator generates a complete edge map.

[0064] In step S220, the generator of the edge enhancement network is further trained to optimize The edge details are restored. Specifically:

[0065] In order to stabilize the network training process, the adversarial loss function L in the form of Least-Square GAN (LSGAN) is used. adv :

[0066]

[0067] in, represents the discriminator of the edge enhancement network, Edge map for optimization generation With the real edge graph E arc , which enables the generator to generate more realistic edge graph structures.

[0068] To reduce With the real edge graph E arc The difference between the two introduces the L1 loss function To constrain the accuracy of image generation, it is defined as follows:

[0069]

[0070] By optimizing The generator is able to generate an edge map that is closer to the actual edge map. Effectively reduce blur in edge areas.

[0071] The final loss function combination for:

[0072]

[0073] Among them, λ adv and λ L1 are the weights of adversarial loss and L1 loss, respectively, to adjust the quality and details of the image output by the generator. This loss function continuously iterates and optimizes the generator during the training process, making the complete edge map it generates more accurate.

[0074] S230, based on the trained edge enhancement network, generate a complete edge map including arc area details.

[0075] In step S230, the generator optimized by S220 is Used to generate a complete edge graph. Specifically, G arc and E arc As the input of the generator, we get the final complete edge graph

[0076]

[0077] The detailed edge information of the arc light area can be understood as the complete edge structure recovered from the arc light image. The result will be directly passed to step S300 for further arc light filtering network processing to ensure the texture clarity and edge detail consistency of the final generated arc light-free image.

[0078] Step S300 at least includes steps S310-S330:

[0079] S310, inputting the complete edge map and the arc light image into an arc light filtering network.

[0080] In step S310, the complete edge map generated in step S230 is first With the original arc image I arc As input, the generator model of the arc filter network is passed In this step, the input data consists of the complete edge map generated by the edge enhancement network and the arc light image, and its input is defined as:

[0081]

[0082] Among them, X is the joint input data containing edge and arc information. This input data is processed by the multi-layer convolution of the network to provide sufficient feature basis for the subsequent residual convolution block attention module.

[0083] S320, based on the residual convolutional block attention module (RCBAM), the texture features of the arc area are filtered and reconstructed.

[0084] In step S320, the residual convolutional block attention module (RCBAM) is used to process the texture details of the joint input data X. The RCBAM module includes a channel attention module (CA) and a spatial attention module (SA), which respectively focus on the channel importance and spatial position of the feature map to form a sophisticated feature selection mechanism.

[0085] Specifically, the importance weight of each feature channel of the input data is first calculated, and the average value M of the feature map is obtained by global average pooling (GAP) and global maximum pooling (GMP). avg and the maximum value Μ max , and processed by MLP:

[0086] CA(X)=σ(MLP(Μ avg )+MLP(Μ max )

[0087] Among them, σ is the Sigmoid activation function, which is used to normalize the attention weight.

[0088] Furthermore, the feature map output by the channel attention module is further passed to the spatial attention module, and the importance of the feature position is extracted using the convolution kernel. The spatial attention module further removes the redundant information affected by the arc light by filtering the attention allocation of each feature position in space, and the output is defined as:

[0089] SA(X)=σ(Conv(X CA ))

[0090] Among them, X CA Represents the feature map after channel attention processing. The final residual convolution block reconstructs the texture features of the arc area by superimposing the original feature map X with the output processed by CA and SA.

[0091] S330, outputting the arc-free image generated by the arc filtering network to obtain an arc-free image.

[0092] In step S330, the data processed by RCBAM is passed to the last level of the arc light filtering network, and the final arc light-free image is generated through multi-layer convolution and deconvolution operations. It is expressed as:

[0093]

[0094] in, is the generator function of the arc filter network. Complete edge and texture detail information is retained, providing a basis for subsequent image quality evaluation and optimization.

[0095] Step S400 at least includes steps S410-S430:

[0096] S410 , based on the arc-free image and the complete edge map, use image quality evaluation indicators such as SSIM and PSNR to perform evaluation.

[0097] In step S410, the arc-free image generated in step S330 is With the complete edge graph As input, the image quality evaluation index is used for evaluation. The evaluation index of this step mainly includes structural similarity (SSIM) and peak signal-to-noise ratio (PSNR):

[0098] SSIM (Structural Similarity Index): SSIM is used to measure Compared with the ideal arc-free image I non-arc The structural similarity between them is defined as follows:

[0099]

[0100] in, and μ I For images with I non-arc The average value of and σ I is its variance, is the covariance of the two, C1 and C2 are constants.

[0101] PSNR Measurement with I non-arc The error degree between them is calculated as follows:

[0102]

[0103] Among them, MAX I Represents the maximum value of the image pixel, and MSE is the mean square error:

[0104]

[0105] The calculation results of SSIM and PSNR are used as the measurement standard of arc-free image quality, providing data support for the next step of analysis and feedback.

[0106] S420: Analyze the evaluation result to generate image quality feedback.

[0107] In step S420, the SSIM and PSNR index values ​​obtained in step S410 are analyzed to generate image quality feedback data. Specifically, by setting a threshold, it is determined whether the SSIM and PSNR values ​​meet the expected image quality requirements to form feedback parameters:

[0108] When the SSIM value is lower than the set threshold θ SSIM When , a feedback signal is generated to adjust the network parameters to enhance the structural similarity of the image. PSNR When the image quality feedback data ΔSSIM and ΔPSNR are generated, the feedback signal is generated to reduce the noise component in the image generation process. The image quality feedback data ΔSSIM and ΔPSNR will directly affect the parameter adjustment of the edge enhancement network and the arc light filtering network to improve the quality of image generation.

[0109] S430: Using the image quality feedback to optimize parameters of an edge enhancement network and an arc filtering network.

[0110] In step S430, the image quality feedback data ΔSSIM and ΔPSNR generated in step S420 are applied to the edge enhancement network. and arc-flash filtering network Optimize the parameters of . Adjust the learning rate of both according to the feedback and

[0111] When ΔSSIM>, increase the learning rate of the edge-enhanced network To enhance structural similarity.

[0112] When ΔPSNR>0, the convolutional layer parameters of the arc filter network are adjusted to reduce noise and improve image clarity.

[0113] Finally, the edge enhancement network after feedback optimization and arc-flash filtering network It can generate higher quality arc-free images and provide reliable image data for visual monitoring of welding scenes.

[0114] The key innovations of the present invention include: introducing a generative adversarial network and a residual convolution block attention module to perform fine texture filtering in the arc area to ensure that the edge and detail information is fully preserved; and adaptively optimizing network parameters through an image quality feedback mechanism to cope with different complex welding scenarios.

[0115] The following are its main beneficial effects:

[0116] (1) Improving image filtering accuracy: The present invention uses the Canny edge detection algorithm to obtain the initial edge map and grayscale image sequence, and trains based on the generative adversarial network to generate a complete edge map through adversarial loss and L1 loss. This step can accurately capture edge details under complex backgrounds and significantly improve image filtering accuracy.

[0117] (2) Enhanced detail recovery in arc light areas: The arc light image is processed by the residual convolutional block attention module (RCBAM). The present invention can specifically enhance the texture details of the arc light area during the reconstruction process and effectively filter out arc light interference. This module focuses on the local important areas of the image, avoiding the blurring of texture information by traditional methods, thereby ensuring the clarity of the output image.

[0118] (3) Adaptive parameter optimization: Image quality feedback data is generated using image quality evaluation indicators such as SSIM (structural similarity index) and PSNR (peak signal-to-noise ratio) to dynamically optimize the parameter settings of the edge enhancement network and arc filtering network. Through the feedback mechanism, this method can achieve adaptive adjustment in different welding scenarios to ensure the stability and quality of arc-free image generation.

[0119] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A welding arc filtering method based on generative adversarial network, characterized in that: The following steps are involved: Acquire robot welding images, and use the Canny edge detection algorithm to generate grayscale images and edge images of arc images and non-arc images to obtain initial edge images and grayscale image sequences; Based on the initial edge map and grayscale image sequence, an edge enhancement network is trained to generate a complete edge map through adversarial loss and L1 loss; Inputting the complete edge map and the arc light image into the arc light filtering network, filtering and reconstructing the texture of the arc light area through the residual convolution block attention module, and generating an arc light-free image; Based on the arc-free image and the complete edge map, the arc-free image is evaluated using image quality evaluation indicators such as SSIM and PSNR to generate image quality feedback for optimizing parameters of the edge enhancement network and the arc filtering network.

2. The welding arc filtering method according to claim 1, characterized in that: The step of obtaining the robot welding image comprises: An original image in a welding operation scene is collected, and the original image is preprocessed to obtain an arc image containing an arc area and an arc-free image.

3. The welding arc filtering method according to claim 1, characterized in that: The step of generating the grayscale image comprises: The arc light image and the no-arc light image are converted into grayscale images to obtain an arc light grayscale image and a no-arc light grayscale image.

4. The welding arc filtering method according to claim 1, characterized in that: The steps of the Canny edge detection algorithm include: An arc edge map and a non-arc edge map are generated based on the arc grayscale map and the non-arc grayscale map to obtain the initial edge map and grayscale map sequence.

5. The welding arc filtering method according to claim 1, characterized in that: The step of training the edge enhancement network comprises: The generator of the edge enhancement network is trained using adversarial loss and L1 loss to generate the complete edge map.

6. The welding arc filtering method according to claim 1, characterized in that: The step of inputting the complete edge map and the arc light image into the arc light filtering network comprises: The complete edge map and the arc light image are spliced ​​at channel level and then input into an arc light filtering network for feature reconstruction.

7. The welding arc filtering method according to claim 1, characterized in that: The arc light filtering network includes a residual convolution block attention module, which is used to enhance the texture details of the arc light area during the reconstruction process.

8. The welding arc filtering method according to claim 1, characterized in that: The training step of the edge enhancement network also includes: The least squares generative adversarial loss (LSGAN) is introduced into the discriminator of the edge enhancement network to improve the stability of network training and optimize the edge clarity of the complete edge map.

9. The welding arc filtering method according to claim 1, characterized in that: The image quality evaluation step based on the arc-free image and the complete edge image comprises: The arc-free image is evaluated using structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) indicators to generate image quality feedback data.

10. The welding arc filtering method according to claim 1, characterized in that: The step of generating image quality feedback comprises: After calculating the structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) of the arc-free image, the image quality feedback is used to adjust the learning rate of the edge enhancement network and the arc filtering network to improve the quality of subsequent arc-free image generation.

Images