A X-ray weld defect image reconstruction method based on improved SRGAN

Abstract

The present invention provides an X-ray weld defect image reconstruction method based on improved SRGAN, which relates to the technical field of weld defect image reconstruction. The specific steps include: S1. constructing an X-ray weld defect image dataset HRdata, including low-resolution-high-resolution images SRdata, and obtaining a number of 640×640 weld defect images by performing image reorganization processing on the HRdata dataset; S2. using a residual block structure N-SPPF to replace the residual block in the SRGAN network generator structure, and using a ShuffleNet v2 network-based content loss to replace the VGG network-based content loss in the SRGAN network; S3. using a GAMattent ion attention mechanism to replace the PReLU activation function layer and the BN layer in the original SRGAN network generator structure. The present invention optimizes the network training process by improving SRGAN, so that the feature representation of the reconstruction result is more prominent, and provides an important premise and guarantee for further detection, recognition and analysis of super-resolution reconstructed images.

Description

Technical Field

[0001] The present invention relates to the technical field of weld defect image reconstruction, and in particular to an X-ray weld defect image reconstruction method based on improved SRGAN. Background Art

[0002] In the prior art, in large-span space structures, the safety of the connection at the node plays an important role in the entire structure. Different types of rods are connected by welds, bolts, etc. to form a whole. The weld connection has the advantages of high rigidity, good sealing, and no damage to the cross section. It is widely used in large-span space structures. However, in the actual welding process, it is affected by the welding environment, welding process, etc., resulting in a decrease in welding quality, which can easily lead to engineering accidents. Non-destructive testing of welds at welding points includes infrared testing, magnetic particle testing, and X-ray testing. Compared with other testing methods, X-ray testing can directly display the internal structure and defect information of the weld through images, so as to better identify and locate welding defects.

[0003] However, there are still some shortcomings: First, the high-frequency information of the image is obtained through basis functions or interpolation kernels, and then the image super-resolution reconstruction is performed. This method is directly applied to weld defect images, which will result in obvious blockiness and shadows, which is not conducive to image judgment; Second, there is still a difference between the image generated by the SRGAN network and the expected image. During the training process, the images used are natural images. Compared with weld images, natural images have richer feature information, but for weld images, there is less feature information in the image, and super-resolution reconstruction cannot be performed directly. Summary of the invention

[0004] The object of the present invention is to provide an X-ray weld defect image reconstruction method based on improved SRGAN to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] An X-ray weld defect image reconstruction method based on improved SRGAN, the specific steps include:

[0007] S1. Construct an X-ray weld defect image dataset HRdata, including low-resolution and high-resolution images, and obtain several 640×640 weld defect images by performing image reorganization processing on the GDXray dataset;

[0008] S2. The residual block structure N-SPPF is used to replace the residual block in the generator structure of the SRGAN network, and the content loss based on the ShuffleNet v2 network is used to replace the content loss based on the VGG network in the SRGAN network;

[0009] S3. The GAMattention mechanism is used to replace the PReLU activation function layer and BN layer in the original SRGAN network generator structure;

[0010] S4. Replace the BN layer-LeakyReLU layer in the convolution layer-BN layer-LeakyReLU layer of the original SRGAN network discriminator structure with the GAMattention mechanism;

[0011] S5. Remove the BN layer and activation function layer in the original SRGAN discriminator network, add the Att module, and replace the LeakyReLU layer in the original SRGAN network discriminator structure with the SoftPlus layer;

[0012] S6. Replace the first and last activation functions of the original SRGAN discriminator network, Leaky ReLU, with SoftPlus functions;

[0013] S7. Increase the number of convolutional layers in the original SRGAN network discriminator structure from 1024 to 2048, and increase the discriminator network depth to 1024;

[0014] S8. Build an X-ray weld defect image generation adversarial network LAAR-SRGAN based on loss function, activation function, attention mechanism, residual block, and network layer depth;

[0015] S9. The constructed LAAR-SRGAN network is used to perform model training on the constructed X-ray weld defect dataset, and the trained network model is used to perform super-resolution reconstruction on the low-resolution X-ray weld defect image;

[0016] S10. Evaluate the effect of reconstructed X-ray weld defect images.

[0017] Furthermore, the 650 weld defect image sets SRdata obtained by image reconstruction processing were down-sampled by Bicubic×4 to obtain LR images, and the data set was named LRdata. The weld defect images of the HRdata data set were divided into the first training set, the first validation set, and the first test set, and the division ratio of first training set: first validation set: first test set was 8:1:1. The weld defect images of the LRdata data set were divided into the second training set, the second validation set, and the second test set, and the division ratio of second training set, second validation set, and second test set was 8:1:1. The first training set, the first validation set, the second training set, and the second validation set were used for model testing, and the second test set was used for model testing. The defect image reconstruction effect was tested.

[0018] Furthermore, the content loss based on the ShuffleNet v2 network is used to replace the content loss based on the VGG network in the SRGAN network, and the formula is as follows:

[0019]

[0020] Among them, L SR is the super-resolution reconstruction loss value, To obtain the content loss value based on the ShuffleNet v2 network, It is the super-resolution reconstruction loss value of the VGG network.

[0021] Furthermore, the GAMattention mechanism is used to replace the PReLU activation function layer and BN layer in the original SRGAN network generator structure, based on the following formula:

[0022]

[0023] Among them, Mc, Ms are channel attention features and spatial attention features, F1∈R C×H×W is the input feature map, F2 is the intermediate state, and F3 is the output. Represents element-wise multiplication.

[0024] Furthermore, the process of replacing the BN layer-LeakyReLU layer in the convolution layer-BN layer-LeakyReLU layer in the original SRGAN network discriminator structure with the GAMattention attention mechanism is as follows:

[0025] Global average pooling: Perform global average pooling on the feature map to reduce the spatial dimension of the feature map to 1;

[0026] Two fully connected layers: The feature vector after global average pooling is input into two fully connected layers, where the first fully connected layer is used to reduce the dimension and the second fully connected layer is used to generate attention weights;

[0027] Sigmoid activation function: Apply the Sigmoid activation function to limit the attention weight between 0 and 1;

[0028] Attention weighting: Use attention weights to weight the original feature map, enhance the response of important features, and suppress irrelevant features.

[0029] For the output X of each convolutional layer, the replacement process is as follows:

[0030] Input the output X into the GAM module to obtain the attention-weighted output X′;

[0031] Use the replaced output X′ to continue the subsequent convolution operation;

[0032] In the replaced discriminator structure, the output X′ of the GAM module can be expressed as:

[0033] X′=X·σ{W2·ReLU[W1·GAP(X)]}

[0034] Where: GAP(X) is the global average pooling operation, W1 and W2 are the weight matrices of the two fully connected layers, and σ is the Sigmoid activation function.

[0035] Furthermore, the BN layer and activation function layer in the middle of the original SRGAN discriminator network are removed, the Att module is added, and the LeakyReLU layer in the original SRGAN network discriminator structure is replaced with the SoftPlus layer. The formula is as follows:

[0036] SoftPlus(x)=ln(1+e x )

[0037] Where x is the input value.

[0038] Furthermore, the process of replacing the first and last activation functions of the original SRGAN discriminator network, Leaky ReLU, with SoftPlus functions is as follows:

[0039] Find the first activation function of the SRGAN discriminator network and replace the Leaky ReLU function with the SoftPlus function; find the last activation function of the SRGAN discriminator network and replace the Leaky ReLU function with the SoftPlus function, ensuring that the replaced SoftPlus function is correctly connected to the adjacent layers (convolutional layers, fully connected layers).

[0040] Furthermore, the process of building an X-ray weld defect image generation adversarial network LAAR-SRGAN based on loss function, activation function, attention mechanism, residual block, and network layer depth is as follows:

[0041] Residual Block: Residual Blocks are usually used, including convolutional layers, batch normalization and activation functions. The generator usually uses ReLU or Leaky ReLU activation functions.

[0042] Increase image resolution through deconvolution and sub-pixel convolution methods;

[0043] x out =x in +F(x in )

[0044] Among them, x in is the original feature input to the residual block, xout is the output feature obtained after the residual block processing, F(x in ) is a small convolutional network, where F(x in ) contains convolution, BN and activation functions,

[0045] Convolutional layer: a series of convolutional layers to extract features;

[0046] Activation functions: including Leaky ReLU and SoftPlus;

[0047] Attention mechanism: add self-attention mechanism to emphasize key features;

[0048] The adversarial loss usually adopts the cross entropy loss;

[0049] Discriminator D loss formula:

[0050]

[0051] Among them, D(x) is the judgment of the discriminator D on the real data x, and G(z) is the false data generated by the generator after the input noise z is given. is the expected value of the real data x, is the expected value of the generator input noise z;

[0052] Generator loss formula:

[0053]

[0054] Where D[G(z)] is the output of the discriminator D for the sample G(z) generated by the generator;

[0055] Perceptual loss ensures the visual similarity between generated images and real images by comparing the distance in high-dimensional feature space. The perceptual loss is the activation value distance in the VGG network:

[0056]

[0057] Among them, Φ i [G(z)] is the activation value of the i-th layer of the VGG network, Φ i (x) is the activation value of the i-th layer of the real data x in the VGG network;

[0058] Self-attention formula:

[0059] A=softmax(Q·K T )

[0060] Among them, A is the attention matrix, Q is the query matrix, K is the key matrix, and T is the transpose operation.

[0061] Furthermore, the constructed LAAR-SRGAN network is used to train the constructed X-ray weld defect dataset. The process of super-resolution reconstruction of low-resolution X-ray weld defect images using the trained network model is as follows:

[0062] The formula for adversarial loss is:

[0063] Ladv=-Ex hr ~p hr [logD(x hr )]-Exlr~p lr log{1-D[G(x lr )]}

[0064] Among them, Ladv is the confrontation loss, -Ex hr For high-resolution images, p hr [logD(x hr )] is the probability of the real image, Exlr is the image generated by the generator, p lr log{1-D[G(x lr )]} is the probability of the image generated by the generator, D(x hr ) is the output of the discriminator for high-resolution images, D[G(x lr )] is the output of the discriminator for the synthetic high-resolution image generated by the low-resolution input generated by the generator;

[0065] Perceptual loss: measures the difference between generated images and real high-resolution images through features extracted by networks such as VGG:

[0066]

[0067] Among them, Lperceptual is the perceptual loss, Φ i [G(x lr )] is the feature of the i-th layer of the image generated by the generator, Φ i (x hr ) is the feature of the real high-resolution image at the i-th layer;

[0068] The total generator loss formula is:

[0069] LG=Ladv+λ perceptual Lperceptual

[0070] Among them, λ perceptual is the weight parameter of perceptual loss;

[0071] Use the LRdata and HRdata datasets to train the LAAR-SRGAN network, and update the parameters of the generator by optimizing the generator's loss function;

[0072] The trained LAAR-SRGAN model is used to perform super-resolution reconstruction on new low-resolution images. Given a low-resolution image, the generator network is used to obtain the corresponding high-resolution image.

[0073] Furthermore, the process of evaluating the effect of the reconstructed X-ray weld defect image is as follows:

[0074] Peak signal-to-noise ratio and structural similarity are used to evaluate the effect of the image.

[0075] Among them, the formula for peak signal-to-noise ratio is:

[0076]

[0077] The formula for structural similarity is:

[0078]

[0079] SSIM(x,y)=[l(x,y) α c(x,y) β s(x,y) γ ]

[0080] Where MSE is the mean square error, W is the number of rows, H is the number of columns, and X(i,j) R,G,B is the pixel value of the original image at position (i, j), Y(i, j) R,G,B To reconstruct the pixel value of the image at position (i, j), PSNR is the peak signal-to-noise ratio, X MAX is the maximum value of image pixels, SSIM is structural similarity, l is brightness, c is contrast, s is structure, c1, c2 and c3 are constant correction coefficients, c3 = c2 / 2, where μ x is the average value of x, μ y is the mean value of y. is the variance of x, is the variance of y, σ x is the standard deviation of x, σ y is the standard deviation of y, σ xy is the oblique variance of x and y, x is the pixel value of the original image, y is the pixel value of the reconstructed image, α is the factor coefficient of brightness, β is the factor coefficient of contrast, γ is the factor coefficient of structure, α=β=γ=1;

[0081] When PSNR ≥ 30, SSIM ≈ 1, it means that the reconstructed X-ray weld defect image has good effect;

[0082] When PSNR<30, SSIM≈0, it means that the reconstructed X-ray weld defect image is poor.

[0083] Compared with the prior art, the present invention has the following beneficial effects:

[0084] The ShuffleNet v2 network is used as the loss function to calculate the content loss in the training process. By comparing the original SRGAN network, the ShuffleNet v2 convolutional neural network is used as the loss function. During the training process, more abundant defect feature information can be extracted. The GAMattention attention mechanism is used to replace the PReLU activation function layer and BN layer in the original SRGAN network generator structure, so that the improved generator can integrate more image information, combine low-frequency details from feature maps of different scales with high-level semantics, and extract features at all scales to achieve accurate reconstruction of perceptual consistency, thereby generating high-quality images. The LeakyReLU activation function is replaced by the SoftPlus activation function to further improve the generalization ability of the algorithm. The BN layer and some activation function layers are removed, the GAMattention attention mechanism is added, and the number of network layers is deepened, so that the discriminator can better distinguish HR images from SR images. Therefore, the improved SRGAN optimizes the network training process, makes the feature representation of the reconstruction result more prominent, and provides an important premise and guarantee for further detection, recognition and analysis of super-resolution reconstructed images. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 It is a flow chart of the overall method of the present invention;

[0086] Figure 2 The GDXray weld image reconstruction method of the present invention;

[0087] Figure 3 This is a schematic diagram of the HRdata weld defect image of the present invention;

[0088] Figure 4 This is a schematic diagram of the LRdata weld defect image of the present invention;

[0089] Figure 5 This is a schematic diagram of the N-SPPF component modules of the present invention;

[0090] Figure 6 It is a flow chart of the GAMattention method of the present invention;

[0091] Figure 7 This is a schematic diagram of the improved network structure of the SRGAN generator of the present invention;

[0092] Figure 8 This is a schematic diagram of the network structure of the SRGAN discriminator of the present invention;

[0093] Fig. 9This is a schematic diagram of the LAAR-SRGAN network structure of the present invention;

[0094] Fig.10 This is a schematic diagram of the LAAR-SRGAN reconstruction effect of the present invention. DETAILED DESCRIPTION

[0095] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0096] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0097] Example:

[0098] See also Figures 1 to 10 , the present invention provides a technical solution:

[0099] An X-ray weld defect image reconstruction method based on improved SRGAN, such as Figure 1 As shown, the specific steps include:

[0100] S1. Construct an X-ray weld defect image dataset HRdata, including low-resolution and high-resolution images, and obtain several 640×640 weld defect images by performing image reorganization processing on the GDXray dataset;

[0101] S2. The residual block structure N-SPPF is used to replace the residual block in the generator structure of the SRGAN network, and the content loss based on the ShuffleNet v2 network is used to replace the content loss based on the VGG network in the SRGAN network;

[0102] S3. The GAMattention mechanism is used to replace the PReLU activation function layer and BN layer in the original SRGAN network generator structure;

[0103] S4. Replace the BN layer-LeakyReLU layer in the convolution layer-BN layer-LeakyReLU layer of the original SRGAN network discriminator structure with the GAMattention mechanism;

[0104] S5. Remove the BN layer and activation function layer in the original SRGAN discriminator network, add the Att module, and replace the LeakyReLU layer in the original SRGAN network discriminator structure with the SoftPlus layer;

[0105] S6. Replace the first and last activation functions of the original SRGAN discriminator network, Leaky ReLU, with SoftPlus functions;

[0106] S7. Increase the number of convolutional layers in the original SRGAN network discriminator structure from 1024 to 2048, and increase the discriminator network depth to 1024;

[0107] S8. Build an X-ray weld defect image generation adversarial network LAAR-SRGAN based on loss function, activation function, attention mechanism, residual block, and network layer depth;

[0108] S9. The constructed LAAR-SRGAN network is used to perform model training on the constructed X-ray weld defect dataset, and the trained network model is used to perform super-resolution reconstruction on the low-resolution X-ray weld defect image;

[0109] S10. Evaluate the effect of reconstructed X-ray weld defect images.

[0110] In this embodiment, the improved SRGAN includes a residual, a PReLU activation function layer and a BN layer. The attention mechanism is based on the CBAM submodule and adopts a sequential channel-space attention mechanism.

[0111] Based on the above embodiments, Figure 2 , Figure 3 and Figure 4As shown in the figure, the weld images of the GDXray dataset are reorganized to obtain 650 X-ray weld images with a size of 640×640. The 650 datasets are used as HRdata. Since low-resolution images need to be input into the generator network during the model training process, the 650 weld defect image sets SRdata obtained by image reorganization are down-sampled by Bicubic×4 to obtain LR images. The dataset is named LRdata. The weld defect images of the HRdata dataset are divided into the first training set, the first validation set, and the first test set. The division ratio is first training set: first validation set: first test set = 8:1:1; the weld defect images of the LRdata dataset are divided into the second training set, the second validation set, and the second test set. The division ratio is second training set, second validation set, and second test set = 8:1:1. The first training set, the first validation set, the second training set, and the second validation set are used for model testing, and the second test set is used for model testing to test the defect image reconstruction effect.

[0112] Based on the above embodiments, Figure 5 As shown in the figure, N-SPPF consists of 8 GhostConv modules and 3 MaxPool2d modules. The input information is x. Within the network structure, the addition operations of x1, x2, x3, x4, and x5 are implemented, and the addition operations of y1, y2, and y3 are implemented, which improves the model's ability to extract defect features of X-ray weld defect images.

[0113] On the basis of the above embodiment, the content loss based on the ShuffleNet v2 network is used to replace the content loss based on the VGG network in the SRGAN network, and the formula is as follows:

[0114]

[0115] Among them, L SR is the super-resolution reconstruction loss value, To obtain the content loss value based on the ShuffleNet v2 network, It is the super-resolution reconstruction loss value of the VGG network.

[0116] Based on the above embodiments, Figure 6 and Figure 7 As shown in the figure, the GAMattention attention mechanism is used to replace the PReLU activation function layer and BN layer in the original SRGAN network generator structure. The formula is as follows:

[0117]

[0118] Among them, Mc, Ms are channel attention features and spatial attention features, F1∈RC×H×W is the input feature map, F2 is the intermediate state, and F3 is the output. Represents element-wise multiplication.

[0119] The process of replacing the BN layer-LeakyReLU layer in the convolution layer-BN layer-LeakyReLU layer in the original SRGAN network discriminator structure with the GAMattention attention mechanism is as follows:

[0120] Global average pooling: Perform global average pooling on the feature map to reduce the spatial dimension of the feature map to 1;

[0121] Two fully connected layers: The feature vector after global average pooling is input into two fully connected layers, where the first fully connected layer is used to reduce the dimension and the second fully connected layer is used to generate attention weights;

[0122] Sigmoid activation function: Apply the Sigmoid activation function to limit the attention weight between 0 and 1;

[0123] Attention weighting: Use attention weights to weight the original feature map, enhance the response of important features, and suppress irrelevant features.

[0124] For the output X of each convolutional layer, the replacement process is as follows:

[0125] Input the output X into the GAM module to obtain the attention-weighted output X′;

[0126] Use the replaced output X′ to continue the subsequent convolution operation;

[0127] In the replaced discriminator structure, the output X′ of the GAM module can be expressed as:

[0128] X′=X·σ{W2·ReLU[W1·GAP(X)]}

[0129] Where: GAP(X) is the global average pooling operation, W1 and W2 are the weight matrices of the two fully connected layers, and σ is the Sigmoid activation function.

[0130] Based on the above embodiments, Figure 8 As shown in the figure, the BN layer and activation function layer in the middle of the original SRGAN discriminator network are removed, the Att module is added, and the LeakyReLU layer in the original SRGAN network discriminator structure is replaced with the SoftPlus layer. The formula is as follows:

[0131] SoftPlus(x)=ln(1+e x )

[0132] Where x is the input value;

[0133] The process of replacing the first and last activation functions of the original SRGAN discriminator network, Leaky ReLU, with the SoftPlus function is as follows:

[0134] Find the first activation function of the SRGAN discriminator network (usually after the first convolutional layer), and replace the LeakyReLU function with the SoftPlus function; find the last activation function of the SRGAN discriminator network (usually after the last convolutional layer), and replace the Leaky ReLU function with the SoftPlus function; ensure that the replaced SoftPlus function is correctly connected to the adjacent layers (convolutional layer, fully connected layer).

[0135] Based on the above embodiments, Figure 8 and Fig. 9 As shown in the figure, the process of building an X-ray weld defect image generation adversarial network LAAR-SRGAN based on loss function, activation function, attention mechanism, residual block, and network layer depth is as follows:

[0136] Residual Block: Residual Blocks are usually used, including convolutional layers, batch normalization and activation functions. The generator usually uses ReLU or Leaky ReLU activation functions.

[0137] Increase image resolution through deconvolution and sub-pixel convolution methods;

[0138] x out =x in +F(x in )

[0139] Among them, x in is the original feature input to the residual block, x out is the output feature obtained after the residual block processing, F(x in ) is a small convolutional network, where F(x in ) contains convolution, BN and activation functions,

[0140] Convolutional layer: a series of convolutional layers to extract features;

[0141] Activation functions: including Leaky ReLU and SoftPlus;

[0142] Attention mechanism: add self-attention mechanism to emphasize key features;

[0143] The adversarial loss usually adopts the cross entropy loss;

[0144] Discriminator D loss formula:

[0145]

[0146] Among them, D(x) is the judgment of the discriminator D on the real data x, and G(z) is the false data generated by the generator after the input noise z is given. is the expected value of the real data x, is the expected value of the generator input noise z;

[0147] Generator loss formula:

[0148]

[0149] Where D[G(z)] is the output of the discriminator D for the sample G(z) generated by the generator;

[0150] Perceptual loss ensures the visual similarity between generated images and real images by comparing the distance in high-dimensional feature space. The perceptual loss is the activation value distance in the VGG network:

[0151]

[0152] Among them, Φ i [G(z)] is the activation value of the i-th layer of the VGG network, Φ i (x) is the activation value of the i-th layer of the real data x in the VGG network;

[0153] Self-attention formula:

[0154] A=softmax(Q·K T )

[0155] Among them, A is the attention matrix, Q is the query matrix, K is the key matrix, and T is the transpose operation.

[0156] Based on the above embodiments, Fig. 9 As shown, the constructed LAAR-SRGAN network is used to train the model of the constructed X-ray weld defect dataset. The process of super-resolution reconstruction of low-resolution X-ray weld defect images by the trained network model is as follows:

[0157] The formula for adversarial loss is:

[0158] Ladv=-Ex hr ~p hr [logD(x hr )]-Exlr~p lr log{1-D[G(x lr )]}

[0159] Among them, Ladv is the confrontation loss, -Ex hr For high-resolution images, phr [logD(x hr )] is the probability of the real image, Exlr is the image generated by the generator, p lr log{1-D[G(x lr )]} is the probability of the image generated by the generator, D(x hr ) is the output of the discriminator for high-resolution images, D[G(x lr )] is the output of the discriminator for the synthetic high-resolution image generated by the low-resolution input generated by the generator;

[0160] Perceptual loss: measures the difference between generated images and real high-resolution images through features extracted by networks such as VGG:

[0161]

[0162] Among them, Lperceptual is the perceptual loss, Φ i [G(x lr )] is the feature of the i-th layer of the image generated by the generator, Φ i (x hr ) is the feature of the real high-resolution image at the i-th layer;

[0163] The total generator loss formula is:

[0164] LG=Ladv+λ perceptual Lperceptual

[0165] Among them, λ perceptual is the weight parameter of perceptual loss;

[0166] Use the LRdata and HRdata datasets to train the LAAR-SRGAN network, and update the parameters of the generator by optimizing the generator's loss function;

[0167] The trained LAAR-SRGAN model is used to perform super-resolution reconstruction on new low-resolution images. Given a low-resolution image, the generator network is used to obtain the corresponding high-resolution image.

[0168] Based on the above embodiments, Fig.10 As shown in the figure, the process of evaluating the effect of the reconstructed X-ray weld defect image is as follows:

[0169] Peak signal-to-noise ratio and structural similarity are used to evaluate the effect of the image.

[0170] Among them, the formula for peak signal-to-noise ratio is:

[0171]

[0172] The formula for structural similarity is:

[0173]

[0174] SSIM(x,y)=[l(x,y) α c(x,y) β s(x,y) γ ]

[0175] Where MSE is the mean square error, W is the number of rows, H is the number of columns, and X(i,j) R,G,B is the pixel value of the original image at position (i, j), Y(i, j) R,G,B To reconstruct the pixel value of the image at position (i, j), PSNR is the peak signal-to-noise ratio, X MAX is the maximum value of image pixels, SSIM is structural similarity, l is brightness, c is contrast, s is structure, c1, c2 and c3 are constant correction coefficients, c3 = c2 / 2, where μ x is the average value of x, μ y is the mean value of y. is the variance of x, is the variance of y, σ x is the standard deviation of x, σ y is the standard deviation of y, σ xy is the oblique variance of x and y, x is the pixel value of the original image, y is the pixel value of the reconstructed image, α is the factor coefficient of brightness, β is the factor coefficient of contrast, γ is the factor coefficient of structure, α=β=γ=1;

[0176] When PSNR ≥ 30, SSIM ≈ 1, it means that the reconstructed X-ray weld defect image has good effect;

[0177] When PSNR<30, SSIM≈0, it means that the reconstructed X-ray weld defect image is poor.

[0178] Generally speaking, a PSNR value above 30 dB is considered to be a high-quality image, while a higher SSIM value indicates that the image is closer to the original image in structure and content. That is, within a certain range, the closer the PSNR is to 100 and the closer the SSIM is to 1, the better the reconstructed X-ray weld defect image will be.

[0179] In order to verify the above conclusions, based on the calculation of the peak signal-to-noise ratio and structural similarity, the peak signal-to-noise ratio and structural similarity are counted and the mean is calculated, which are Avg_PSNR and Avg_SSIM respectively.

[0180] Using ShuffleNet v2 as the feature extraction network, compared with the original SRGAN network, the PSNR value increased by 0.29dB, the SSIM value increased by 0.004, the Avg_PSNR value increased by 2.69dB, and the Avg_SSIM value increased by 0.023; the SRGAN model training results using SoftPlus are PSNR / SSIM values ​​of 35.43dB / 0.950, and avg_PSNR / avg_SSIM values ​​of 30.72dB / 0.929. Compared with the original SRGAN network, the PSNR value increased by 0.83dB, the SSIM value increased by 0.001, the avg_PSNR value increased by 1.35dB, and the avg_SSIM value increased by 0.004; using SoftPlus_ShuffleNet The SRGAN model v2 has a PSNR / SSIM value of 39.54dB / 0.951 and an avg_PSNR / avg_SSIM value of 34.28dB / 0.948 when trained on the X-ray weld defect dataset. Compared with the original SRGAN network, the PSNR value increased by 4.94dB, the SSIM value increased by 0.002, the avg_PSNR value increased by 4.91dB, and the avg_SSIM value increased by 0.023; the PSNR value of the LAAR-SRGAN network model is 41.63dB, the SSIM value is 0.958, the avg_PSNR value is 37.02dB, and the avg_SSIM value is 0.956. Compared with the SRGAN model, the PSNR value increased by 7.03dB, the SSIM value increased by 0.009, the avg_PSNR value increased by 7.65dB, and the avg_SSIM value increased by 0.031. Therefore, compared with the SRGAN network model, the reconstructed defect feature information of the LAAR-SRGAN network model is clearer and the image features in details are richer.

[0181] The specific values ​​of α, β and γ in the formula are generally determined by technical personnel in this field according to actual conditions. The essence of this formula is a comprehensive analysis based on weighted summation. Technical personnel in this field collect multiple groups of sample data and set corresponding preset proportional coefficients for each group of sample data. The preset proportional coefficients and the collected sample data are substituted into the formula. Through repeated experiments and parameter adjustments, the accuracy of the model output and the rationality of the results are observed, these factor coefficients are gradually adjusted, and the performance and effect of the model under different parameter settings are compared to find the optimal coefficient combination. The calculated factor coefficients are screened and averaged to obtain the values ​​of α, β and γ.

[0182] The specific values ​​of α, β and γ in the formula are generally determined by technical personnel in this field according to actual conditions. The essence of this formula is a comprehensive analysis based on weighted summation. Technical personnel in this field collect multiple groups of sample data and set corresponding preset proportional coefficients for each group of sample data. The preset proportional coefficients and the collected sample data are substituted into the formula. Through repeated experiments and parameter adjustments, the accuracy of the model output and the rationality of the results are observed, these factor coefficients are gradually adjusted, and the performance and effect of the model under different parameter settings are compared to find the optimal coefficient combination. The calculated factor coefficients are screened and averaged to obtain the values ​​of α, β and γ.

[0183] In addition, the size of the preset factor coefficient is to quantify each parameter to obtain a specific value. In order to facilitate subsequent comparison, the size of the coefficient depends on the amount of sample data and the preset proportional coefficient initially set by technical personnel in this field for each set of sample data. It is not unique as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0184] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0185] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0186] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0187] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An X-ray weld defect image reconstruction method based on improved SRGAN, characterized in that: The specific steps include: S1. Construct an X-ray weld defect image dataset HRdata, including low-resolution and high-resolution images; S2. The residual block structure N-SPPF is used to replace the residual block in the generator structure of the SRGAN network, and the content loss based on the ShuffleNet v2 network is used to replace the content loss based on the VGG network in the SRGAN network; S3. The GAMattention mechanism is used to replace the PReLU activation function layer and BN layer in the original SRGAN network generator structure; S4. Replace the BN layer-LeakyReLU layer in the convolution layer-BN layer-LeakyReLU layer of the original SRGAN network discriminator structure with the GAMattention mechanism; S5. Replace the LeakyReLU layer in the original SRGAN network discriminator structure with the SoftPlus layer; S6. Increase the number of convolutional layers of the discriminator in the original SRGAN network from 1024 to 2048; S7. Build an X-ray weld defect image generation adversarial network LAAR-SRGAN based on loss function, activation function, attention mechanism, residual block, and network layer depth; S8. The constructed LAAR-SRGAN network is used to perform model training on the constructed X-ray weld defect dataset, and the trained network model is used to perform super-resolution reconstruction on the low-resolution X-ray weld defect image; S9. Evaluate the effect of reconstructed X-ray weld defect images; N-SPPF consists of 8 GhostConv modules and 3 MaxPool2d modules. The input information is x. Within the network structure, the addition operations of x1, x2, x3, x4, and x5 are implemented, and the addition operations of y1, y2, and y3 are implemented. x1, x2, x3, x4, x5, y1, y2, and y3 are feature maps after passing through different modules.

2. The X-ray weld defect image reconstruction method based on improved SRGAN according to claim 1 is characterized in that: The weld defect image dataset has 650 images. The 650 weld defect image sets obtained by image reorganization are down-sampled by Bicubic×4 to obtain LR images, and the dataset is named LRdata. The HRdata weld defect image dataset is divided into a first training set, a first validation set, and a first test set, and the division ratio is first training set: first validation set: first test set = 8:1:

1. The weld defect images of the LRdata dataset are divided into a second training set, a second validation set, and a second test set, and the division ratio is second training set, second validation set, and second test set = 8:1:

1. The first training set, the first validation set, the second training set, and the second validation set are used for model testing, and the second test set is used for model testing, and the defect image reconstruction effect test is performed.

3. The X-ray weld defect image reconstruction method based on improved SRGAN according to claim 1 is characterized in that: The GAMattention attention mechanism is used to replace the PReLU activation function layer and BN layer in the original SRGAN network generator structure. The formula is as follows: Among them, Mc, Ms are channel attention features and spatial attention features, F1∈R C×H×W is the input feature map, F2 is the intermediate state, and F3 is the output. Represents element-wise multiplication.

4. The X-ray weld defect image reconstruction method based on improved SRGAN according to claim 3 is characterized in that: The process of replacing the BN layer-LeakyReLU layer in the convolution layer-BN layer-LeakyReLU layer in the original SRGAN network discriminator structure with the GAMattention attention mechanism is as follows: Global average pooling: Perform global average pooling on the feature map to reduce the spatial dimension of the feature map to 1; Two fully connected layers: The feature vector after global average pooling is input into two fully connected layers, where the first fully connected layer is used to reduce the dimension and the second fully connected layer is used to generate attention weights; Sigmoid activation function: Apply the Sigmoid activation function to limit the attention weight between 0 and 1; Attention weighting: Use attention weights to weight the original feature map; For the output X of each convolutional layer, the replacement process is as follows: Input the output X into the GAM module to obtain the attention-weighted output X′; Use the replaced output X′ to continue the subsequent convolution operation; In the replaced discriminator structure, the output X′ of the GAM module is expressed as: X′=X·σ{W2·ReLU[W1·GAP(X)]} Where: GAP(X) is the global average pooling operation, W1 and W2 are the weight matrices of the two fully connected layers, and σ is the Sigmoid activation function.

5. The X-ray weld defect image reconstruction method based on improved SRGAN according to claim 4 is characterized in that: Replace the LeakyReLU layer in the original SRGAN network discriminator structure with the SoftPlus layer based on the following formula: SoftPlus(x)=ln(1+e x ) Where x is the input value.

6. The X-ray weld defect image reconstruction method based on improved SRGAN according to claim 5 is characterized in that: The process of evaluating the effect of reconstructed X-ray weld defect images is as follows: Peak signal-to-noise ratio and structural similarity are used to evaluate the effect of the image. Among them, the formula for peak signal-to-noise ratio is: The formula for structural similarity is: SSIM(x,y)=[l(x,y) α ·c(x,y) β ·s(x,y) γ ] Where MSE is the mean square error, W is the number of rows, H is the number of columns, and X(i,j) R,G,B is the pixel value of the original image at position (i, j), Y(i, j) R,G,B To reconstruct the pixel value of the image at position (i, j), PSNR is the peak signal-to-noise ratio, X MAX is the maximum value of image pixels, SSIM is structural similarity, l is brightness, c is contrast, s is structure, c1, c2 and c3 are constant correction coefficients, c3 = c2 / 2, where μ x is the average value of x, μ y is the mean value of y. is the variance of x, is the variance of y, σ x is the standard deviation of x, σ y is the standard deviation of y, σ xy is the oblique variance of x and y, x is the pixel value of the original image, y is the pixel value of the reconstructed image, α is the factor coefficient of brightness, β is the factor coefficient of contrast, γ is the factor coefficient of structure, α=β=γ=1; When PSNR ≥ 30, SSIM ≈ 1, it means that the reconstructed X-ray weld defect image has good effect; When PSNR<30, SSIM≈0, it means that the reconstructed X-ray weld defect image is poor.

Images