X-ray welding seam image recovery method based on deep learning

Abstract

The invention provides an X-ray welding seam image recovery method based on deep learning. The X-ray welding seam image recovery method comprises the steps of making a welding seam image recovery data set; a single-stage improved U-Net network is used for image restoration, the improved U-Net network has three layers, and each layer comprises a layer normalization calculation module, a convolution layer module, an activation function module and a simple gating attention mechanism module; obtaining a recovered image through the trained U-Net network model, carrying out loss function calculation on the recovered image and an original image in a training set, and adjusting parameters of the U-Net network model according to a loss function, so that the network can achieve a better recovery effect; testing the model; and performing complete weld seam image recovery by using the qualified model. According to the invention, the simple gating attention module is utilized to effectively improve the feature screening capability, improve the image recovery effect, reduce network parameters and achieve a better recovery effect; a U-Net method in a single-stage network is used for improvement, and the requirements for high definition and high details of an X-ray welding seam image are met.

Description

Technical Field

[0001] The present invention belongs to the technical field of weld image processing, and in particular relates to an X-ray weld image restoration method based on deep learning. Background Art

[0002] As the main method of connecting two workpieces, welding technology is widely used in aerospace, machinery manufacturing, energy transportation, microelectronics and other fields. With the development of welding technology at home and abroad, the requirements for welding quality are getting higher and higher. However, due to the influence of the welding environment and the welding process, many types of defects will inevitably occur, such as pores, slag inclusions, cracks, incomplete penetration, and incomplete fusion. These defects will cause the strength of the weld connection area to decrease and stress concentration, thus leading to serious safety accidents. Usually, the weld defect situation is obtained by analyzing and processing the weld imaging data. From the perspective of the X-ray weld imaging field, noise will be generated when converted into a digital image, which will affect the detection results and cause the problem of missed detection and false detection. The traditional filtering method has no obvious effect on noise removal and affects the perception of the human eye. In order to solve the above technical problems, the present invention designs an X-ray weld image restoration method based on deep learning to obtain clearer weld image data. Summary of the invention

[0003] In view of the shortcomings in the prior art, the present invention provides an X-ray weld image restoration method based on deep learning, which effectively solves the problem that traditional filtering methods have poor noise removal effect.

[0004] The present invention achieves the above technical objectives through the following technical means.

[0005] A deep learning-based X-ray weld image restoration method includes the following process:

[0006] Step 1: Create a cropped weld image restoration dataset;

[0007] Step 2: Use the single-stage improved U-Net network for image restoration;

[0008] Step 3: Test the U-Net network model trained in step 2;

[0009] Step 4: Use the qualified model to restore the complete weld image.

[0010] Furthermore, the specific process of step 1 is as follows:

[0011] Step 1.1: First, collect the weld image defect dataset. Select image data from the open source dataset GDXray and the self-collected weld image defect dataset. The pixel size of the selected image is 2000×500. The selected image is randomly cropped according to the fixed size and cropped into a square image of 512×512;

[0012] Step 1.2: Then analyze the noise of the cropped weld image. The noise is composed of Poisson noise and Gaussian noise. Poisson noise cannot be reproduced in a simulated environment. Therefore, the method of simulating Gaussian noise is used to add noise to the weld image. Perform noise simulation on the weld image. Add different levels of Gaussian noise σ' to the original image. The value range is 0 to 75 to simulate the imaging effects in different scenes to form a paired weld image restoration data set. The test set noise levels include σ'=25, 50, and 75 to form a weld image restoration test set. Then divide the data set into a training set and a validation set according to a certain ratio.

[0013] Furthermore, the specific process of step 2 is as follows:

[0014] Step 2.1: Use the noisy image in the training set as the network input and input the image into the improved deep neural network U-Net network; the improved U-Net network is divided into a decoding part and an encoding part, with a total of three layers. Feature extraction and retention are performed in each layer, and the Skip onnection operation is performed with the upsampling part to add the two feature maps element by element to retain the effective information in the image;

[0015] Step 2.2: Finally, the restored image is obtained by improving the U-Net network, and the loss function Loss is calculated with the original image in the training set:

[0016]

[0017] Among them, X i , R i , Y belongs to the input dataset D11, the size is N×C×H×W, N is the batch size, C is the number of channels, H is the height of the image, and W is the width of the image; X i represents the input image, R i represents the output image, Y represents the real image, i.e., the clear image; PSNR is the peak signal-to-noise ratio calculation method;

[0018] After multiple trainings, the parameters in the U-Net network model are adjusted according to the loss function so that the network can achieve better recovery effects.

[0019] Furthermore, in the improved U-Net network, each layer includes layer normalization calculation, convolution layer, activation function, and simple gated attention mechanism module; the simple gated attention mechanism module consists of two branches, including a context retention branch and an attention branch. The attention branch is composed of an attention mechanism. After the feature map is input, it is first divided into two halves, and one half of the feature map is extracted by the channel attention method. After the channel attention method, it is element-by-element multiplication with the original other half of the feature map. The simple gated attention mechanism is expressed as:

[0020] Gate(X1,X2,ρ,F)=F(X1)⊙ρ(X2)

[0021] Among them, X1 and X2 represent feature maps, i.e., network inputs, and F is the channel attention module; F(X1) represents the feature map X1 after the channel attention mechanism; ρ represents the activation function GELU, and ρ(X2) represents the feature map X2 after the activation function;

[0022] The channel attention mechanism is expressed as:

[0023] F(X)=X*σ(W2max(0,W1pool(X)))

[0024] Among them, F(X) represents the channel attention mechanism function; X represents the feature map, pool represents the global average pooling operation that aggregates spatial information into the channel; σ is a nonlinear activation function Sigmoid, W1 and W2 are fully connected layers, and ReLU is used between the two fully connected layers; * is the multiplication operation of the channel layer.

[0025] Furthermore, in step 4, the overall weld image is first decomposed into image blocks of corresponding sizes according to the improved U-Net network, and then the image blocks are sequentially sent to the improved U-Net network model to obtain multiple restored image blocks, and finally all the restored image blocks are spliced ​​in sequence to obtain a complete X-ray weld denoised image; in the above process, if the image block cannot be decomposed into a fixed size, the remaining part is filled with the image by zero padding.

[0026] The present invention has the following beneficial effects:

[0027] The improved U-Net network of the present invention, in which the simple gated attention module can effectively improve the feature screening ability, improve the image restoration effect, reduce network parameters, and finally achieve a better restoration effect; in order to obtain a clear image, a deep learning method is used to denoise the X-ray weld image; the present invention uses the improved U-Net method in a single-stage network to meet the requirements of high definition and high detail of X-ray weld images, and is widely used in the industrial field.

[0028] In addition, since the number of X-ray images in the currently publicly known GDXray dataset is very small, it cannot meet the training of this subsequent model, and the self-collected dataset contains more images and more typical defects. Therefore, the present invention combines the two, and the combined data volume can train the model more effectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Before and after images of simulated noise for the open source dataset;

[0030] Figure 2 Before and after images of simulated noise for an autonomously collected weld image defect dataset;

[0031] Figure 3 To improve the U-Net network diagram;

[0032] Figure 4 To improve the detailed schematic diagram of each layer module in the U-Net network;

[0033] Figure 5 This is a schematic diagram of a simple gated attention mechanism module;

[0034] Figure 6 Schematic diagram of denoising effect; DETAILED DESCRIPTION

[0035] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0036] The X-ray weld image restoration method based on deep learning of the present invention comprises the following process:

[0037] Step 1: Create a weld image restoration dataset. The specific steps are as follows:

[0038] Step 1.1: First, collect the weld image defect dataset from Figure 1 (a) shows the open source dataset GDXray and Figure 2 Image data is selected from the self-collected weld image defect dataset shown in (a), and the pixel size of the selected image is about 2000×500. The selected image is randomly cropped according to a fixed size and cropped into a square image of size 512×512.

[0039] Step 1.2: Then analyze the noise of the cropped weld image. The noise is mainly composed of Poisson noise and Gaussian noise. Poisson noise cannot be reproduced in the simulation environment, so the method of simulating Gaussian noise is used to add noise to the weld image. Perform noise simulation on the weld image and add different levels of Gaussian noise σ' to the original image, such as Figure 1 (b) Figure 2As shown in (b), the value range is 0 to 75 to simulate the imaging effects in different scenes, forming a paired weld image restoration dataset D1, and the test set noise levels include σ'=25, 50, and 75, forming a weld image restoration test set E; then the dataset D1 is divided into a training set D11 and a verification set D12 according to a 9:1 ratio.

[0040] Step 2: Use the single-stage improved U-Net network to restore the image. The specific steps are as follows:

[0041] Step 2.1: Use the noisy image in the training set D11 as the network input, such as Figure 3 As shown, the image is input into the improved deep neural network U-Net network; the improved U-Net network is divided into a decoding part and an encoding part, with a total of three layers. The modules of each layer are improved modules. Features are extracted and retained in each layer, and the Skiponnection operation is performed with the subsequent upsampling part to add the two feature maps element by element to retain the effective information in the image.

[0042] The noisy images in the training set D11 are input into the improved U-Net network, which includes layer normalization calculation, convolution layer, activation function, and simple gated attention mechanism modules. Figure 4 Shown Figure 3 The specific composition of each module in the U-Net network is improved in this paper, where LayerNorm is the layer normalization calculation, conv is the traditional convolution operation, dconv is the depthwise separable convolution, GELU is the activation function, and SGAU is the simple gated attention mechanism. is element-by-element addition. Figure 5 As shown in , the simple gated attention mechanism module consists of two branches, including a context retention branch and an attention branch. The attention branch is composed of an attention mechanism, such as Figure 4 As shown in the figure, Input is the feature map input, Avgpool represents the average pooling operation, conv is the convolution operation, ReLu is the activation function, ⊙ represents element-by-element multiplication, and output is the output feature map after feature fusion; the simple gated attention mechanism module is: after the feature map is input, the feature map is first divided into two halves, and one half of the feature map extracts information through the channel attention method, that is, the global average pooling layer, 1×1 convolution layer and activation function are composed.

[0043] After performing element-wise multiplication with the other half of the original feature map through the channel attention method, SGAU can be expressed as:

[0044] Gate(X1,X2,ρ,F)=F(X1)⊙ρ(X2)

[0045] Among them, X1 and X2 represent feature maps, F is the channel attention module; F(X1) represents the feature map X1 after the channel attention mechanism; ρ represents the activation function GELU, and ρ(X2) represents the feature map X2 after the activation function.

[0046] The channel attention mechanism can be expressed as:

[0047] F(X)=X*σ(W2max(0,W1pool(X)))

[0048] Among them, F(X) represents the channel attention mechanism function; X represents the feature map, pool represents the global average pooling operation that aggregates spatial information into the channel. σ is a nonlinear activation function Sigmoid, W1 and W2 are fully connected layers, and ReLU is used between the two fully connected layers; * is the multiplication operation of the channel layer.

[0049] Step 2.2: Finally, the restored image is obtained through the deep neural network, and the loss function is calculated with the original image in the training set. The loss function is the PSNR function, that is:

[0050]

[0051] Among them, X i , R i , Y belongs to the input dataset D11, the size is N×C×H×W, N is the batch size, C is the number of channels, H is the height of the image, and W is the width of the image; X i represents the input image, R i represents the output image, Y represents the real image, i.e., the clear image; PSNR is the peak signal-to-noise ratio calculation method;

[0052] After multiple trainings, the parameters in the U-Net network model are adjusted according to the loss function so that the network can achieve better recovery effects.

[0053] Step 3: Model testing: Save the trained weights (i.e., the parameters of the U-Net network model trained in step 2), and send the noise images of different levels in the test set E as input into the improved U-Net network model for restoration, and obtain the image restoration results under different noise levels.

[0054] Step 4: Use the qualified model to restore the complete weld image: De-noise the complete weld image. First, decompose the entire image into image blocks of corresponding sizes according to the improved U-Net network, then send the image to the improved U-Net network model in sequence to obtain restored image blocks one by one, and finally splice all the image blocks in order to obtain a complete X-ray weld denoised image. If the image block cannot be decomposed into a fixed size, the remaining part is filled with zeros.

[0055] The specific image restoration effects are as follows: Figure 6 As shown, Figure 6 (a) is the denoising effect (image restoration effect) for the open source dataset. Figure 6 (b) is the denoising effect (image restoration effect) of the self-collected weld image defect dataset.

[0056] The embodiments are preferred implementations of the present invention, but the present invention is not limited to the above-mentioned implementations. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essential content of the present invention belong to the protection scope of the present invention.

Claims

1. A deep learning-based X-ray weld image restoration method, characterized in that: The process includes the following: Step 1: Create a cropped weld image restoration dataset; Step 2: Use the single-stage improved U-Net network for image restoration; Step 3: Test the U-Net network model trained in step 2; Step 4: Use the qualified model to restore the complete weld image.

2. The X-ray weld image restoration method based on deep learning according to claim 1, characterized in that: The specific process of step 1 is as follows: Step 1.1: First, collect the weld image defect dataset, select image data of corresponding pixel size from the open source dataset GDXray and the self-collected weld image defect dataset, randomly crop the selected images according to a fixed size, and crop square images; Step 1.2: Then analyze the noise of the cropped weld image. The noise is composed of Poisson noise and Gaussian noise. Poisson noise cannot be reproduced in a simulated environment. Therefore, the method of simulating Gaussian noise is used to add noise to the weld image. Perform noise simulation on the weld image. Add different levels of Gaussian noise σ' to the original image. The value range is 0 to 75 to simulate the imaging effects in different scenes to form a paired weld image restoration data set. The test set noise levels include σ'=25, 50, and 75 to form a weld image restoration test set. Then divide the data set into a training set and a validation set according to a certain ratio.

3. The X-ray weld image restoration method based on deep learning according to claim 2 is characterized in that: The specific process of step 2 is as follows: Step 2.1: Take the noisy image in the training set as the network input and input the image into the improved deep neural network U-Net network; The improved U-Net network is divided into a decoding part and an encoding part, with a total of three layers. Features are extracted and retained in each layer, and the Skip connection operation is performed with the upsampling part to add the two feature maps element by element to retain the effective information in the image. Step 2.2: Finally, the restored image is obtained by improving the U-Net network, and the loss function Loss is calculated with the original image in the training set: Among them, X i , R i , Y belongs to the input dataset D11, the size is N×C×H×W, N is the batch size, C is the number of channels, H is the height of the image, and W is the width of the image; X i represents the input image, R i represents the output image, Y represents the real image, i.e., the clear image; PSNR is the peak signal-to-noise ratio calculation method; After multiple trainings, the parameters in the U-Net network model are adjusted according to the loss function so that the network can achieve better recovery effects.

4. The X-ray weld image restoration method based on deep learning according to claim 3 is characterized in that: In the improved U-Net network, each layer includes layer normalization calculation, convolution layer, activation function, and simple gated attention mechanism module; The simple gated attention mechanism module consists of two branches, including the context preservation branch and the attention branch. The attention branch is composed of the attention mechanism. After the feature map is input, it is first divided into two halves. One half of the feature map extracts information through the channel attention method, and then performs element-by-element multiplication with the original other half of the feature map after the channel attention method. The simple gated attention mechanism is expressed as: Gate(X1,X2,ρ,F)=F(X1)⊙ρ(X2) Among them, X1 and X2 represent feature maps, i.e., network inputs, and F is the channel attention module; F(X1) represents the feature map X1 after the channel attention mechanism; ρ represents the activation function GELU, and ρ(X2) represents the feature map X2 after the activation function; The channel attention mechanism is expressed as: F(X)=X*σ(W2max(0,W1pool(X))) Among them, F(X) represents the channel attention mechanism function; X represents the feature map, pool represents the global average pooling operation that aggregates spatial information into the channel; σ is a nonlinear activation function Sigmoid, W1 and W2 are fully connected layers; * is the multiplication operation of the channel layer.

5. The X-ray weld image restoration method based on deep learning according to claim 2, characterized in that: In the step 4, the overall weld image is first decomposed into image blocks of corresponding sizes according to the improved U-Net network, and then the image blocks are sequentially sent to the improved U-Net network model to obtain multiple restored image blocks, and finally all the restored image blocks are spliced ​​in sequence to obtain a complete X-ray weld denoised image; in the above process, if the image block cannot be decomposed into a fixed size, the remaining part is filled with the image by zero padding.

Images