Real-time robust weld seam image denoising method and system based on lightweight diffusion denoising model

Abstract

The application discloses a kind of real-time robust weld seam image denoising method and system based on light diffusion denoising model.Therein, denoising method includes the following steps: S1, noisy and noiseless weld seam image is collected, and weld seam image is processed to constitute data set;S2, construct teacher diffusion denoising model and student diffusion denoising model, and based on data set distillation training is carried out to obtain light diffusion denoising model;S3, deploy light diffusion denoising model, to carry out denoising to the weld seam image of real-time acquisition, to obtain the denoised weld seam image.The method of the application substantially reduces the model parameter amount while maintaining the denoising effect, substantially improves the inference speed, realizes the efficient real-time weld seam image denoising processing, meets the deployment demand of limited computing resource environment in industrial field.

Description

Technical Field

[0001] This invention relates to weld seam image processing technology for robots, specifically to a real-time robust weld seam image denoising method and system based on a lightweight diffusion denoising model. Background Technology

[0002] With the development of industrial automation, automated welding technology has become increasingly widespread, solving problems such as the high difficulty and operational risks associated with manual welding. Among these advancements, intelligent welding robots, which utilize real-time weld seam information to adjust the welding path and ensure welding quality, have been extensively researched and applied. This type of welding robot uses laser sensors to acquire weld seam information in real time. Weld seam tracking technology then processes this information to extract the coordinates of weld seam feature points, guiding the robot to complete the welding process. However, arc light and spatter during the welding process can induce significant noise in the weld seam images captured by the laser vision sensor, affecting the acquisition of weld seam feature points. Therefore, noise reduction of the weld seam image is crucial before applying a tracking system. Summary of the Invention

[0003] The primary objective of this invention is to overcome the shortcomings of the existing technology and provide a real-time robust weld image denoising method based on a lightweight diffusion denoising model. This real-time robust weld image denoising method based on a lightweight diffusion denoising model addresses the problem of real-time denoising of weld images with strong noise interference.

[0004] The second objective of this invention is to provide a real-time robust weld image denoising system based on a lightweight diffusion denoising model.

[0005] The first objective of this invention is achieved through the following technical solution: This real-time robust weld image denoising method based on a lightweight diffusion denoising model includes the following steps:

[0006] S1. Collect noisy and noiseless weld seam images, and process the weld seam images to form a dataset;

[0007] S2. Construct teacher diffusion denoising model and student diffusion denoising model, and perform distillation training based on dataset to enable the teacher diffusion denoising model to transfer knowledge to the student diffusion denoising model in order to obtain a lightweight diffusion denoising model.

[0008] S3. Deploy a lightweight diffusion denoising model to reduce noise in the real-time acquired weld seam images to obtain denoised weld seam images.

[0009] Following step S3, the following steps are also included:

[0010] S4. Construct a gray area detection module based on the HSV color space, and use the gray area detection module to identify gray areas in the noise-reduced weld image.

[0011] S5. Design an area-based regional filtering mechanism to process effective gray areas.

[0012] S6. Construct an intelligent fill module based on background sampling. The intelligent fill module achieves natural gray area replacement, thereby processing and optimizing the noise-reduced weld seam image.

[0013] Step S1 includes the following steps:

[0014] S11. The welding robot collects weld images of the same workpiece along the taught weld trajectory. When there is no welding, it obtains a noiseless first weld image sequence and a noisy second weld image sequence when welding. The weld images of the first weld image sequence and the second weld image sequence are made to correspond one-to-one.

[0015] S12. Preprocess the weld images of the first weld image sequence and the second weld image sequence;

[0016] S13. Further process the pre-processed weld image:

[0017] A vector is added to the weld feature points of each pair of corresponding weld images in the first and second weld image sequences. The direction of this vector is random and the value is less than 50 pixels. Weld images with a resolution of 300×300 are cropped with the end of the corresponding vector as the center. This results in a set of noisy cropped weld images and a set of clean cropped weld images. The cropped pairs of noisy and clean weld images are used as input to the dataset to form the dataset.

[0018] Step S2 includes the following steps:

[0019] S21. Based on the dataset, an open-source diffusion model is trained using a progressive training strategy to construct a teacher diffusion denoising model;

[0020] S22. Construct a student neural network, which includes an input processing layer, a first-time embedding processing layer, an encoding path, a decoding path, a detail enhancement layer, and an output enhancement layer.

[0021] S23. Construct student feature extraction blocks and integrate them into the student neural network;

[0022] S24. Based on the student neural network with integrated student feature extraction blocks, instantiate a student diffusion denoising model with the same structure as the teacher diffusion denoising model. Construct a knowledge distillation training framework with the goal of aligning the noise predictor output in the teacher diffusion denoising model with the noise predictor output in the student diffusion denoising model. Backpropagate to update the weight parameters of the student diffusion denoising model to realize knowledge transfer from the teacher diffusion denoising model to the student diffusion denoising model, so as to obtain a lightweight diffusion denoising model.

[0023] Step S21 includes the following steps:

[0024] S221. Pairs of noisy weld images and noiseless weld images are used as training inputs for the open-source diffusion model.

[0025] S222. Set the weights of the residual prediction loss and the noise prediction loss to be equal, and both use the L2 loss function. During training, adopt a dual-network independent optimization strategy. The residual prediction network and the noise prediction network use independent RAdam optimizers to update their parameters.

[0026] S223. In the early stage of training, a gradient pruning strategy is adopted to ensure the stability of training, and the gradient norm is limited to 1.0. At the same time, an exponential moving average strategy is used with a decay rate of 0.995, which is updated once every 10 iterations.

[0027] S224. Replace the convolution type, activation function and connection layer function in the forward propagation function of U-Net, delete some convolution and connection operations, and immediately retrain the open source diffusion model on a fixed dataset and hardware environment after each modification.

[0028] S225. Simultaneously test the PSNR index and inference time. By comparing the effects of different architecture configurations, the optimal combination of replacing the weight-balanced convolution in the block class of the basic component of forward propagation with conv2d convolution, replacing GroupNorm normalization with BatchNorm2d, and replacing the activation function with ReLU is finally identified to obtain the teacher diffusion denoising model.

[0029] In step S22, the construction process of the student neural network includes the following steps:

[0030] S221. Construct an enhanced input processing layer, perform preliminary feature extraction on the input image through 7×7 convolution operation, then process it through batch normalization and ReLU activation function, and then perform continuous processing of 3×3 convolution and 1×1 convolution through the input enhancement layer to obtain the enhanced initial feature map.

[0031] S222. Construct an enhanced second temporal embedding processing layer, convert the time step information into a high-dimensional feature vector through sinusoidal position encoding, and then perform linear transformation and GELU activation through a multilayer perceptron to obtain temporal conditional features;

[0032] S223. Construct the encoding path by continuously processing the initial feature map through three enhanced temporal embedding blocks, then downsampling through 4×4 convolution, then processing through three enhanced temporal embedding blocks, and finally obtaining a multi-scale encoded feature map through final downsampling and two temporal embedding blocks.

[0033] S224. Construct the decoding path, upsample the encoded feature map through 4×4 transposed convolution, then concatenate the upsampled feature map with the feature map of the same resolution of the encoding path through channels, and then obtain the decoding feature map through enhanced temporal embedding block processing.

[0034] S225. Construct a detail enhancement layer. Perform detail enhancement processing on the decoded feature map through two consecutive 3×3 convolutional layers. Then, further extract features through a combination of 1×1 convolution and 3×3 convolution to obtain an enhanced detail feature map.

[0035] S226. Construct an output enhancement layer, perform final feature extraction on the detail feature map through continuous processing of 3×3 convolution and 1×1 convolution, and then output the prediction result feature map with the same number of channels as the input image through a 1×1 convolution layer.

[0036] Step S23 includes the following steps:

[0037] S231. Construct a residual connection layer. Adjust the number of channels of the input feature map through 1×1 convolution operation. When the number of input and output channels are different, perform channel conversion. When they are the same, connect them directly to obtain the residual feature map.

[0038] S232. Construct a first temporal embedding processing layer, convert temporal embedding information into scaling and offset parameters through a linear layer, and then convert one-dimensional temporal features into two-dimensional parameters that match the feature map through dimensional expansion to obtain temporal modulation parameters.

[0039] S233. Construct the first convolutional block, perform preliminary feature extraction on the input feature map through 3×3 convolution operation, then perform batch normalization, apply scaling and offset parameters to the feature map through temporal conditional modulation, and finally process it through the ReLU activation function to obtain the temporally modulated feature map.

[0040] S234. Construct a second convolutional block, perform feature fusion on the temporal modulation feature map through 1×1 convolution operation, then perform batch normalization, and then process it through the GELU activation function to obtain the fused feature map.

[0041] S235. Construct the third convolutional block and process the fused feature map through depthwise separable convolution. First, spatial features are extracted through 3×3 depthwise convolution, then channel features are fused through 1×1 point convolution, and finally, batch normalization and ReLU activation function are applied to obtain the depthwise separable convolutional feature map.

[0042] S236. Construct a feature enhancement layer. Further feature extraction and enhancement are performed on the depthwise separable convolutional feature map through continuous processing of 3×3 convolution and 1×1 convolution. Then, the enhanced feature map is obtained through batch normalization and ReLU activation function.

[0043] S237. Construct a residual fusion layer, add the enhanced feature map and the residual feature map element by element, retain the original information through residual connections, and finally output the enhanced time-aware feature map to form the student diffusion denoising model.

[0044] Step S24 includes the following steps:

[0045] S241. Based on the student neural network with integrated student feature extraction blocks, instantiate a student diffusion denoising model with the same structure as the teacher diffusion denoising model. Its initial dimension is 32 and the dimension multiple sequence is (1, 2, 4).

[0046] S242, with a fixed time value [0.5, 0.5], directly call the function of the noise predictor in the teacher diffusion denoising model, perform single-step inference, and use the output as the input value of the loss function in distillation training;

[0047] S243. Use the MSE loss function to calculate the difference between the noise predictor output in the teacher diffusion denoising model and the noise predictor output in the student diffusion denoising model. Then, use the Adam optimizer to backpropagate based on the loss gradient to update the weight parameters of the student diffusion denoising model.

[0048] S254. Using the noisy image as input, train for 20 epochs, progressively adjust the weight parameters of the student diffusion denoising model, with the goal of aligning the noise predictor outputs in the teacher diffusion denoising model and the student diffusion denoising model. Backpropagate to update the weight parameters of the student diffusion denoising model, realizing knowledge transfer from the teacher diffusion denoising model to the student diffusion denoising model, and obtaining a lightweight diffusion denoising model.

[0049] A real-time robust weld seam image denoising system based on a lightweight diffusion denoising model includes:

[0050] The data acquisition module is used to acquire weld images with and without noise, and to process the weld images to form a dataset.

[0051] The model building and training module is used to build teacher diffusion denoising models and student diffusion denoising models, and to perform distillation training based on the dataset to obtain a lightweight diffusion denoising model.

[0052] The execution module is used to deploy a lightweight diffusion denoising model to denoise the real-time acquired weld seam images, and to optimize the denoised weld seam images to output weld seam images that meet the accuracy requirements.

[0053] The second objective of this invention is achieved through the following technical solution: real-time robust weld image denoising based on a lightweight diffusion denoising model, comprising:

[0054] The data acquisition module is used to acquire weld images with and without noise, and to process the weld images to form a dataset.

[0055] The model building and training module is used to build teacher diffusion denoising models and student diffusion denoising models, and to perform distillation training based on the dataset to obtain a lightweight diffusion denoising model.

[0056] The execution module is used to deploy a lightweight diffusion denoising model to denoise the real-time acquired weld seam images, and to optimize the denoised weld seam images to output weld seam images that meet the accuracy requirements.

[0057] The present invention has the following advantages over the prior art:

[0058] This invention trains a lightweight diffusion denoising model by combining a teacher diffusion denoising model with a student diffusion denoising model. This lightweight diffusion denoising model significantly reduces the number of model parameters while maintaining the denoising effect, greatly improves the inference speed, and achieves efficient and real-time weld seam image denoising processing, meeting the deployment needs of industrial sites in environments with limited computing resources.

[0059] The method of the present invention is a weld image post-processing method based on HSV color space detection and background sampling filling. It can process and optimize the weld image after denoising by the lightweight diffusion denoising model, further improve the imaging quality and detail of the weld image, and ensure that the final output image meets the accuracy requirements. Attached Figure Description

[0060] Figure 1 This is a flowchart of the real-time robust weld image denoising method based on a lightweight diffusion denoising model of the present invention.

[0061] Figure 2 This is a diagram of the backpropagation stepwise denoising process of the lightweight diffusion denoising model of the present invention for denoising weld seam images.

[0062] Figure 3This is a flowchart of the construction process of the student neural network of the present invention.

[0063] Figure 4 This is a flowchart illustrating the construction process of the student feature extraction block of the present invention.

[0064] Figure 5 A schematic diagram of the data acquisition module of this invention.

[0065] Among them, 1-supporting welding equipment; 2-welding robot; 3-welding torch; 4-laser vision sensor; 5-workpiece; 6-welding workbench; 7-industrial control computer; 8-control cabinet. Detailed Implementation

[0066] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0067] like Figure 1 and Figure 2 As shown, this embodiment of the real-time robust weld image denoising method based on a lightweight diffusion denoising model includes the following steps:

[0068] S1. Collect noisy and noiseless weld seam images, and process the weld seam images to form a dataset;

[0069] Step S1 includes the following steps:

[0070] S11. The welding robot collects weld images of the same workpiece along the taught weld trajectory. When there is no welding, it obtains a noiseless first weld image sequence and a noisy second weld image sequence when welding. The weld images of the first weld image sequence and the second weld image sequence are made to correspond one-to-one.

[0071] S12. Preprocess the weld images of the first weld image sequence and the second weld image sequence;

[0072] S13. Further process the pre-processed weld image:

[0073] A vector is added to the weld feature points of each pair of corresponding weld images in the first and second weld image sequences. The direction of this vector is random and the value is less than 50 pixels. Weld images with a resolution of 300×300 are cropped with the end of the corresponding vector as the center. This results in a set of noisy cropped weld images and a set of clean cropped weld images. The cropped pairs of noisy and clean weld images are used as input to the dataset to form the dataset.

[0074] Specifically, the process of adding a vector to the weld feature points of each pair of corresponding weld images in the first and second weld image sequences is to increase the sample size of the dataset. The cropped image contains only the region of interest containing weld feature information. The cropped, noise-free clean image is used as a sample of the dataset, and the cropped, noisy weld image is used as the input of the dataset. Together, they constitute the dataset used to train the diffusion model.

[0075] S2. Construct teacher diffusion denoising model and student diffusion denoising model, and perform distillation training based on dataset to enable the teacher diffusion denoising model to transfer knowledge to the student diffusion denoising model in order to obtain a lightweight diffusion denoising model.

[0076] Step S2 includes the following steps:

[0077] S21. Based on the dataset, an open-source diffusion model is trained using a progressive training strategy to construct a teacher diffusion denoising model;

[0078] Step S21 includes the following steps:

[0079] S221. Pairs of noisy weld images and noiseless weld images are used as training inputs for the open-source diffusion model.

[0080] S222. Set the weights of the residual prediction loss and the noise prediction loss to be equal, and both use the L2 loss function. During training, adopt a dual-network independent optimization strategy. The residual prediction network and the noise prediction network use independent RAdam optimizers to update their parameters.

[0081] S223. In the early stages of training, a gradient pruning strategy is used to ensure training stability, with the gradient norm limited to 1.0. At the same time, an exponential moving average strategy is used with a decay rate of 0.995, updated once every 10 iterations to ensure the stability of the open-source model's convergence.

[0082] S224. Replace the convolution type, activation function and connection layer function in the forward propagation function of U-Net, delete some convolution and connection operations, and immediately retrain the open source diffusion model on a fixed dataset and hardware environment after each modification.

[0083] S225. Simultaneously test the PSNR index and inference time. By comparing the effects of different architecture configurations, the optimal combination of replacing the weight-balanced convolution in the block class of the basic component of forward propagation with conv2d convolution, replacing GroupNorm normalization with BatchNorm2d, and replacing the activation function with ReLU is finally identified to obtain the teacher diffusion denoising model.

[0084] like Figure 3As shown, S22, construct a student neural network, which includes an input processing layer, a first-time embedding processing layer, an encoding path, a decoding path, a detail enhancement layer, and an output enhancement layer;

[0085] like Figure 3 As shown, in step S22, the process of constructing the student neural network includes the following steps:

[0086] S221. Construct an enhanced input processing layer, perform preliminary feature extraction on the input image through 7×7 convolution operation, then process it through batch normalization and ReLU activation function, and then perform continuous processing of 3×3 convolution and 1×1 convolution through the input enhancement layer to obtain the enhanced initial feature map.

[0087] S222. Construct an enhanced second temporal embedding processing layer, convert the time step information into a high-dimensional feature vector through sinusoidal position encoding, and then perform linear transformation and GELU activation through a multilayer perceptron to obtain temporal conditional features;

[0088] S223. Construct the encoding path by continuously processing the initial feature map through three enhanced temporal embedding blocks, then downsampling through 4×4 convolution, then processing through three enhanced temporal embedding blocks, and finally obtaining a multi-scale encoded feature map through final downsampling and two temporal embedding blocks.

[0089] S224. Construct the decoding path, upsample the encoded feature map through 4×4 transposed convolution, then concatenate the upsampled feature map with the feature map of the same resolution of the encoding path through channels, and then obtain the decoding feature map through enhanced temporal embedding block processing.

[0090] S225. Construct a detail enhancement layer. Perform detail enhancement processing on the decoded feature map through two consecutive 3×3 convolutional layers. Then, further extract features through a combination of 1×1 convolution and 3×3 convolution to obtain an enhanced detail feature map.

[0091] S226. Construct an output enhancement layer, perform final feature extraction on the detail feature map through continuous processing of 3×3 convolution and 1×1 convolution, and then output the prediction result feature map with the same number of channels as the input image through a 1×1 convolution layer.

[0092] S23. Construct student feature extraction blocks and integrate them into the student neural network;

[0093] like Figure 4 As shown, step S23 includes the following steps:

[0094] S231. Construct a residual connection layer. Adjust the number of channels of the input feature map through 1×1 convolution operation. When the number of input and output channels are different, perform channel conversion. When they are the same, connect them directly to obtain the residual feature map.

[0095] S232. Construct a first temporal embedding processing layer, convert temporal embedding information into scaling and offset parameters through a linear layer, and then convert one-dimensional temporal features into two-dimensional parameters that match the feature map through dimensional expansion to obtain temporal modulation parameters.

[0096] S233. Construct the first convolutional block, perform preliminary feature extraction on the input feature map through 3×3 convolution operation, then perform batch normalization, apply scaling and offset parameters to the feature map through temporal conditional modulation, and finally process it through the ReLU activation function to obtain the temporally modulated feature map.

[0097] S234. Construct a second convolutional block, perform feature fusion on the temporal modulation feature map through 1×1 convolution operation, then perform batch normalization, and then process it through the GELU activation function to obtain the fused feature map.

[0098] S235. Construct the third convolutional block and process the fused feature map through depthwise separable convolution. First, spatial features are extracted through 3×3 depthwise convolution, then channel features are fused through 1×1 point convolution, and finally, batch normalization and ReLU activation function are applied to obtain the depthwise separable convolutional feature map.

[0099] S236. Construct a feature enhancement layer. Further feature extraction and enhancement are performed on the depthwise separable convolutional feature map through continuous processing of 3×3 convolution and 1×1 convolution. Then, the enhanced feature map is obtained through batch normalization and ReLU activation function.

[0100] S237. Construct a residual fusion layer, add the enhanced feature map and the residual feature map element by element, retain the original information through residual connections, and finally output the enhanced time-aware feature map to form the student diffusion denoising model.

[0101] S24. Based on the student neural network with integrated student feature extraction blocks, instantiate a student diffusion denoising model with the same structure as the teacher diffusion denoising model. Construct a knowledge distillation training framework. With the output of the noise predictor in the teacher diffusion denoising model and the output of the noise predictor in the student diffusion denoising model as the target, backpropagate to update the weight parameters of the student diffusion denoising model, realize the knowledge transfer from the teacher diffusion denoising model to the student diffusion denoising model, and obtain a lightweight diffusion denoising model.

[0102] Step S24 includes the following steps:

[0103] S241. Based on the student neural network with integrated student feature extraction blocks, instantiate a student diffusion denoising model with the same structure as the teacher diffusion denoising model. Its initial dimension is 32 and the dimension multiple sequence is (1, 2, 4).

[0104] S242, with a fixed time value [0.5, 0.5], directly call the function of the noise predictor in the teacher diffusion denoising model, perform single-step inference, and use the output as the input value of the loss function in distillation training;

[0105] S243. Use the MSE loss function to calculate the difference between the noise predictor output in the teacher diffusion denoising model and the noise predictor output in the student diffusion denoising model. Then, use the Adam optimizer to backpropagate based on the loss gradient to update the weight parameters of the student diffusion denoising model.

[0106] The formula for calculating the MSE loss function is as follows:

[0107]

[0108] In the above formula, It is the loss value. y is the output tensor of the student model for the i-th sample. (i) It is the target output tensor of the teacher model for the i-th sample, where N is the batch size, C is the number of channels, H is the height, and W is the width.

[0109] The optimizer's parameter update formula is:

[0110]

[0111] In the above formula, θ t Here are the parameters for the t-th iteration, and α is the learning rate, α = 10. -4 ∈ is a numerical stability constant, ∈ = 10 -8 m t It is the first moment estimate of the gradient, v t It is the second-order gradient moment estimate; the first-order gradient moment estimate m t Represented as:

[0112] m t =β1·m t-1 +(1-β1)·g t

[0113] In the above equation, β1 is the Adam parameter, β1 = 0.9, g t The gradient of the distillation loss with respect to the parameters is expressed as:

[0114]

[0115] The gradient second moment estimate vt Represented as:

[0116]

[0117] In the above formula, β2 is the Adam parameter, β2 = 0.999.

[0118] S244. Using the noisy image as input, train for 20 epochs, progressively adjust the weight parameters of the student diffusion denoising model, with the goal of aligning the noise predictor outputs in the teacher diffusion denoising model and the student diffusion denoising model. Backpropagate to update the weight parameters of the student diffusion denoising model, realizing knowledge transfer from the teacher diffusion denoising model to the student diffusion denoising model, and obtaining a lightweight diffusion denoising model.

[0119] Under the same hardware conditions, the lightweight diffusion denoising model and the teacher diffusion denoising model were made to perform the same weld image denoising task. The experimental results show that the inference speed is improved by 1.2 times (44fps) compared with the original model, and the model performance is improved by 3.1dB. This shows that the lightweight diffusion denoising model construction method based on knowledge distillation proposed in this invention can denoise weld images in real time.

[0120] Specifically, this diffusion denoising model is built on an open-source diffusion model, which includes a dual U-Net denoising network and a temporal embedding module. It learns the prediction of image residuals and noise through the forward diffusion process and gradually restores image details from the degraded image during the backward diffusion process.

[0121] The forward diffusion process follows a Markov chain process and can be represented as:

[0122] I t =I t-1 +α t I res +β t ∈ t

[0123] In the above formula, I t It is the degraded image at the current time step, I t-1 It is the degraded image from the previous time step, I res It is a residual image, ∈ t It is random noise, α t It is the cumulative residual coefficient, α t ∈[0,1], β t It is the cumulative noise figure.

[0124] The reverse diffusion process follows a Markov chain process and can be represented as:

[0125]

[0126] In the above formula, I t-1 It is the image restored at the next time step, I t This is the image restored at the current time step. It is the residual of the neural network prediction. It's noise in the neural network prediction. It is the increment of the cumulative residual coefficient. It is the increment of the cumulative noise figure.

[0127] This open-source diffusion model employs a conditional generation architecture. The dual U-Net denoising network comprises a residual prediction network and a noise prediction network. The first U-Net is specifically responsible for predicting the residual difference between the input weld image and the target image, while the second U-Net is specifically responsible for predicting the noise components added during the diffusion process. An image-to-image conversion mode is used to optimize input channel computation. The prediction target is set as a joint prediction of residuals and noise. The testing mode is configured as joint prediction of residuals and noise. The L2 loss function is used for training and optimization.

[0128] The loss function for the predicted residual is:

[0129]

[0130] In the above formula, It is the residual loss value. It is the predicted residual, I res It is the actual residual.

[0131] The loss function for the predicted noise is:

[0132]

[0133] In the above formula, This is the noise loss value. It is the predicted noise, I res It is real noise.

[0134] In the construction of the lightweight diffusion denoising model of this invention, targeted architecture optimization and condition configuration based on the open-source diffusion model provide an effective technical solution for real-time denoising of weld seam images with strong noise interference. By modifying the functions in the basic module of the model, the performance and efficiency of the model for weld seam image denoising are significantly improved.

[0135] Specifically, during the training phase, the lightweight diffusion denoising model takes paired weld seam images (first and second weld seam image sequences) from the dataset as input. It uses the L2 loss function to simultaneously optimize the residual prediction loss and noise prediction loss. The teacher and student diffusion denoising models each employ independent RAdam optimizers for parameter updates. A standard noise intensity is used as the noise scheduler. During the inference phase, the model starts with the degraded image and progressively removes noise and restores image details through collaborative prediction using dual U-Nets, ultimately achieving accurate weld seam image restoration. This provides an effective technical solution for weld seam image processing tasks. This approach improves PSNR by 2.2 dB and inference speed by 97.6%, significantly enhancing the model's practicality and deployment efficiency.

[0136] S3. Deploy a lightweight diffusion denoising model to denoise the real-time acquired weld images and obtain denoised weld images.

[0137] S4. Construct a gray area detection module based on the HSV color space. This module identifies gray areas in the denoised weld seam image. Specifically, the gray area detection module includes HSV color space conversion, conservative thresholding, and morphological preprocessing. The input BGR color image is converted to the HSV color space. Conservative thresholds are set with a lower bound of [0,0,80] and an upper bound of [180,40,180] to avoid misjudging white areas as gray. The HSV image is then filtered by color range to obtain a preliminary gray area mask. Morphological preprocessing using a 2×2 open kernel removes small noise and avoids misjudging white stripe edges, resulting in a smooth gray area mask.

[0138] S5. Design an area-based region filtering mechanism to process effective gray areas. Specifically, perform contour detection on the smooth gray area mask to obtain the contour information of all gray areas. By setting an area threshold of 800 pixels, perform area filtering on the detected contours and retain only the contours with an area greater than the threshold. Redraw the filtered effective contours onto the mask to obtain the final mask containing only effective gray areas.

[0139] S6. Construct an intelligent filling module based on background sampling. This module achieves natural gray area replacement, thereby optimizing the noise-reduced weld image. Specifically, the grayscale image is binarized to identify high-brightness white areas. Morphological dilation is performed using a 5×5 dilation kernel to avoid sampling pixels near white stripes. The dilated white area mask is inverted to obtain a background area mask far from the white stripes. A brightness threshold of 50 is set to filter dark pixels in the background area to ensure that the filling pixels match the background color. Pixels are randomly selected from the dark background pixels to replace pixels in the gray area, achieving natural area filling. When no suitable dark background pixels are available, pure black is used as a backup filling scheme to ensure the integrity of the processing.

[0140] The second objective of this invention is achieved through the following technical solution: a real-time robust weld image denoising system based on a lightweight diffusion denoising model, comprising:

[0141] The data acquisition module is used to acquire weld images with and without noise, and to process these images to form a dataset; specifically, such as... Figure 5 As shown, the data acquisition module of the present invention mainly consists of a matching welding equipment, a welding robot, a welding torch, a laser vision sensor, a workpiece, a welding workbench, an industrial control computer, and a control cabinet.

[0142] The model building and training module is used to build teacher diffusion denoising models and student diffusion denoising models, and to perform distillation training based on the dataset to obtain a lightweight diffusion denoising model. The execution module is used to deploy the lightweight diffusion denoising model to denoise the real-time acquired weld seam images, and to optimize the denoised weld seam images to output weld seam images that meet the accuracy requirements.

[0143] The above-described specific embodiments are preferred embodiments of the present invention and are not intended to limit the present invention. Any other changes or equivalent substitutions made without departing from the technical solution of the present invention are included within the protection scope of the present invention.

Claims

1. A real-time robust weld image denoising method based on a lightweight diffusion denoising model, characterized in that, Includes the following steps: S1. Collect noisy and noiseless weld seam images, and process the weld seam images to form a dataset; S2. Construct teacher diffusion denoising model and student diffusion denoising model, and perform distillation training based on dataset to enable the teacher diffusion denoising model to transfer knowledge to the student diffusion denoising model in order to obtain a lightweight diffusion denoising model. Step S2 includes the following steps: S21. Based on the dataset, an open-source diffusion model is trained using a progressive training strategy to construct a teacher diffusion denoising model; Step S21 includes the following steps: S221. Pairs of noisy weld images and noiseless weld images are used as training inputs for the open-source diffusion model. S222. Set the weights of the residual prediction loss and the noise prediction loss to be equal, and both use the L2 loss function. During training, adopt a dual-network independent optimization strategy. The residual prediction network and the noise prediction network use independent RAdam optimizers to update their parameters. S223. In the early stage of training, a gradient pruning strategy is adopted to ensure the stability of training, and the gradient norm is limited to 1.

0. At the same time, an exponential moving average strategy is used with a decay rate of 0.995, which is updated once every 10 iterations. S224. Replace the convolution type, activation function and connection layer function in the forward propagation function of U-Net, delete some convolution and connection operations, and immediately retrain the open source diffusion model on a fixed dataset and hardware environment after each modification. S225. Simultaneously test the PSNR index and inference time. By comparing the effects of different architecture configurations, the optimal combination of replacing the weight-balanced convolution in the block class of the basic component of forward propagation with conv2d convolution, replacing GroupNorm normalization with BatchNorm2d, and replacing the activation function with ReLU is finally identified to obtain the teacher diffusion denoising model. S22. Construct a student neural network, which includes an input processing layer, a first-time embedding processing layer, an encoding path, a decoding path, a detail enhancement layer, and an output enhancement layer. S23. Construct student feature extraction blocks and integrate them into the student neural network; Step S23 includes the following steps: S231. Construct a residual connection layer. Adjust the number of channels of the input feature map through 1×1 convolution operation. When the number of input and output channels are different, perform channel conversion. When they are the same, connect them directly to obtain the residual feature map. S232. Construct a first temporal embedding processing layer, convert temporal embedding information into scaling and offset parameters through a linear layer, and then convert one-dimensional temporal features into two-dimensional parameters that match the feature map through dimensional expansion to obtain temporal modulation parameters. S233. Construct the first convolutional block, perform preliminary feature extraction on the input feature map through 3×3 convolution operation, then perform batch normalization, apply scaling and offset parameters to the feature map through temporal conditional modulation, and finally process it through the ReLU activation function to obtain the temporally modulated feature map. S234. Construct a second convolutional block, perform feature fusion on the temporal modulation feature map through 1×1 convolution operation, then perform batch normalization, and then process it through the GELU activation function to obtain the fused feature map. S235. Construct the third convolutional block and process the fused feature map through depthwise separable convolution. First, spatial features are extracted through 3×3 depthwise convolution, then channel features are fused through 1×1 point convolution, and finally, batch normalization and ReLU activation function are applied to obtain the depthwise separable convolutional feature map. S236. Construct a feature enhancement layer. Further feature extraction and enhancement are performed on the depthwise separable convolutional feature map through continuous processing of 3×3 convolution and 1×1 convolution. Then, the enhanced feature map is obtained through batch normalization and ReLU activation function. S237. Construct a residual fusion layer, add the enhanced feature map and the residual feature map element by element, retain the original information through residual connections, and finally output the enhanced time-aware feature map to form the student diffusion denoising model. S24. Based on the student neural network with integrated student feature extraction blocks, instantiate a student diffusion denoising model with the same structure as the teacher diffusion denoising model, construct a knowledge distillation training framework, take the output of the teacher diffusion denoising model and the output of the student diffusion denoising model as the target, backpropagate to update the weight parameters of the student diffusion denoising model, realize the knowledge transfer from the teacher diffusion denoising model to the student diffusion denoising model, and obtain a lightweight diffusion denoising model. S3. Deploy a lightweight diffusion denoising model to reduce noise in the real-time acquired weld seam images to obtain denoised weld seam images.

2. The real-time robust weld image denoising method based on a lightweight diffusion denoising model according to claim 1, characterized in that, Following step S3, the following steps are also included: S4. Construct a gray area detection module based on the HSV color space, and use the gray area detection module to identify gray areas in the noise-reduced weld image. S5. Design an area-based regional filtering mechanism to process effective gray areas. S6. Construct an intelligent fill module based on background sampling. The intelligent fill module achieves natural gray area replacement, thereby processing and optimizing the noise-reduced weld seam image.

3. The real-time robust weld image denoising method based on a lightweight diffusion denoising model according to claim 1, characterized in that, Step S1 includes the following steps: S11. The welding robot collects weld images of the same workpiece along the taught weld trajectory. When there is no welding, it obtains a noiseless first weld image sequence and a noisy second weld image sequence when welding. The weld images of the first weld image sequence and the second weld image sequence are made to correspond one-to-one. S12. Preprocess the weld images of the first weld image sequence and the second weld image sequence; S13. Further process the pre-processed weld image: A vector is added to the weld feature points of each pair of corresponding weld images in the first and second weld image sequences. The direction of this vector is random and the value is less than 50 pixels. Weld images with a resolution of 300×300 are cropped with the end of the corresponding vector as the center. This results in a set of noisy cropped weld images and a set of clean cropped weld images. The cropped pairs of noisy and clean weld images are used as input to the dataset to form the dataset.

4. The real-time robust weld image denoising method based on a lightweight diffusion denoising model according to claim 1, characterized in that, Step S21 includes the following steps: S221. Pairs of noisy weld images and noiseless weld images are used as training inputs for the open-source diffusion model. S222. Set the weights of the residual prediction loss and the noise prediction loss to be equal, and both use the L2 loss function. During training, adopt a dual-network independent optimization strategy. The residual prediction network and the noise prediction network use independent RAdam optimizers to update their parameters. S223. In the early stage of training, a gradient pruning strategy is adopted to ensure the stability of training, and the gradient norm is limited to 1.

0. At the same time, an exponential moving average strategy is used with a decay rate of 0.995, which is updated once every 10 iterations. S224. Replace the convolution type, activation function and connection layer function in the forward propagation function of U-Net, delete some convolution and connection operations, and immediately retrain the open source diffusion model on a fixed dataset and hardware environment after each modification. S225. Simultaneously test the PSNR index and inference time. By comparing the effects of different architecture configurations, the optimal combination of replacing the weight-balanced convolution in the block class of the basic component of forward propagation with conv2d convolution, replacing GroupNorm normalization with BatchNorm2d, and replacing the activation function with ReLU is finally identified to obtain the teacher diffusion denoising model.

5. The real-time robust weld image denoising method based on a lightweight diffusion denoising model according to claim 1, characterized in that, Step S24 includes the following steps: S241. Based on the student neural network with integrated student feature extraction blocks, instantiate a student diffusion denoising model with the same structure as the teacher diffusion denoising model. Its initial dimension is 32 and the dimension multiple sequence is (1, 2, 4). S242, with a fixed time value [0.5, 0.5], directly call the function of the noise predictor in the teacher diffusion denoising model, perform single-step inference, and use the output as the input value of the loss function in distillation training; S243. Use the MSE loss function to calculate the difference between the noise predictor output in the teacher diffusion denoising model and the noise predictor output in the student diffusion denoising model. Then, use the Adam optimizer to backpropagate based on the loss gradient to update the weight parameters of the student diffusion denoising model. S254. Using the noisy image as input, train for 20 epochs, progressively adjust the weight parameters of the student diffusion denoising model, with the goal of aligning the noise predictor outputs in the teacher diffusion denoising model and the student diffusion denoising model. Backpropagate to update the weight parameters of the student diffusion denoising model, realizing knowledge transfer from the teacher diffusion denoising model to the student diffusion denoising model, and obtaining a lightweight diffusion denoising model.

6. A real-time robust weld image denoising system based on a lightweight diffusion denoising model, characterized in that, The method for implementing the real-time robust weld image denoising based on the lightweight diffusion denoising model as described in claims 1-5 includes: The data acquisition module is used to acquire weld images with and without noise, and to process the weld images to form a dataset. The model building and training module is used to build teacher diffusion denoising models and student diffusion denoising models, and to perform distillation training based on the dataset to obtain a lightweight diffusion denoising model. The execution module is used to deploy a lightweight diffusion denoising model to denoise the real-time acquired weld seam images, and to optimize the denoised weld seam images to output weld seam images that meet the accuracy requirements.

Images