A method of introducing a weld defect detection in x-ray image reconstruction and a system thereof

Abstract

The application discloses a kind of weld defect detection methods and systems of introducing X-ray image reconstruction.The present application is to obtain the X-ray image of the weld of workpiece, and it is pretreated;Image reconstruction is carried out to the image after pretreatment using Style-Pix2pix, and the image after reconstruction is obtained;Defect detection is carried out to the image after reconstruction using MO-YOLO, and the defect category and position information are obtained.In the image preprocessing stage, the weld X-ray image is reconstructed by the Style-Pix2pix model, the image quality is enhanced, and the feature confusion problem caused by different window levels and window widths is effectively solved, providing a basis for subsequent defect detection.In the defect detection stage, the MO-YOLO model is optimized according to the characteristics of the weld defect, so that it is more suitable for the weld defect detection scene, enhances the recognition ability of various defects in the weld image, and improves the recognition ability of different types and sizes of defects.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image detection, and particularly relates to a welding seam defect detection method and system introducing X-ray image reconstruction. BACKGROUND

[0002] In pipe production, welding is a very critical link, but the welding process is prone to produce various defects, such as pores, pits, undercut, etc. These defects can greatly affect the carrying capacity and durability of the product, so timely detection and processing are crucial. For a long time, the detection of welding seam defects has relied on manual detection, and this traditional method has great limitations. First, manual detection is easily affected by subjective factors and individual differences, resulting in unstable detection accuracy. Second, with the expansion of production scale and the improvement of production speed, the efficiency of manual detection gradually becomes a bottleneck restricting the production process, limiting the expansion of production scale. In order to overcome these problems, in recent years, automated technologies based on deep learning have been gradually introduced to achieve more intelligent and efficient production operations. With the continuous promotion of Industry 4.0 and intelligent manufacturing, welding seam defect detection urgently needs to be automated and intelligent.

[0003] The surface defects of pipe welding seams not only have various types, irregular defect shapes, and large scale variation ranges, but also are subject to factors such as shooting equipment and exposure time, so the model is required to have good representation ability. Most of the existing defect detection techniques based on deep learning use target detection methods, which only aim to generate defect bounding boxes and cannot accurately locate the positions of small defects, so defect detection methods with finer detection granularity need to be researched. Current researches on welding seams include welding seam region segmentation, defect classification and identification, and automated welding seam defect detection systems. Because welding technology has existed for a long time, research on welding seam images has been ongoing for a long time. Before 2010, due to the limitation of computing power, mathematical methods were mainly used for processing welding seam images and detecting defects in welding seam images. In recent years, due to the improvement of computing power, using deep learning methods to perform this task has become the mainstream. SUMMARY

[0004] The present application aims to overcome the deficiencies of the prior art and provide a welding seam defect detection method and system introducing X-ray image reconstruction.

[0005] The present application is implemented as follows. In a first aspect, the present application provides a welding seam defect detection method introducing X-ray image reconstruction, which comprises:

[0006] An X-ray image containing a welding seam of a workpiece is acquired and preprocessed;

[0007] The image is reconstructed by using a style transfer generative adversarial network Style-Pix2pix.

[0008] The defect detection model MO-YOLO is used for defect detection on the reconstructed image to obtain defect category and position information.

[0009] The style transfer generative adversarial network Style-Pix2pix includes a generator and a discriminator.

[0010] In a second aspect, the present application provides a weld defect detection system, which comprises:

[0011] A data acquisition module is responsible for acquiring an X-ray image containing a workpiece weld and pre-processing the same.

[0012] An image reconstruction module is responsible for reconstructing the pre-processed image by using a style transfer generative adversarial network Style-Pix2pix to obtain a reconstructed image.

[0013] A defect detection module is responsible for defect detection on the reconstructed image by using a global adaptive defect detection model MO-YOLO to obtain defect category and position information.

[0014] In a third aspect, the present application provides a computer readable storage medium having a computer program stored thereon.

[0015] In a fourth aspect, the present application provides a computing device comprising a memory and a processor.

[0016] The present application has at least the following advantages:

[0017] The weld defect detection method provided by the present application improves the precision and efficiency of weld defect detection through the improvement of image preprocessing and defect detection model. In the image preprocessing stage, the Style-Pix2pix model can reconstruct the weld X-ray image, enhance the image quality, effectively solve the feature confusion problem caused by different window levels and window widths, and provide a better foundation for subsequent defect detection. In the defect detection stage, the MO-YOLO model is optimized according to the characteristics of the weld defects, so that it is more suitable for the weld defect detection scene, enhances the recognition ability of various defects in the weld image, and improves the recognition ability of different types and sizes of defects. The results show that the method of the present application has achieved good results on multiple data sets, has higher detection accuracy and better performance compared with traditional methods and other comparative models, and can be effectively applied to weld defect detection in industrial production, improve production efficiency, and reduce product loss caused by defects. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 is the network architecture diagram of the style transfer generative adversarial network (Style-Pix2pix) provided by the embodiment of the present application;

[0020] Figure 2 is the network architecture diagram of the global selective kernel attention mechanism module GSKA provided by the embodiment of the present application;

[0021] Figure 3 is the network architecture diagram of the orthogonal kernel attention module OKA provided by the embodiment of the present application;

[0022] Figure 4 is the network architecture diagram of the rearrangement up-sampling module provided by the embodiment of the present application;

[0023] Figure 5 is the network architecture diagram of the multi-scale discriminator based on PatchGAN provided by the embodiment of the present application;

[0024] Figure 6 is the network architecture diagram of the MO-YOLO network provided by the embodiment of the present application;

[0025] Figure 7 is the network architecture diagram of the global adaptive attention detection head provided by the embodiment of the present application;

[0026] Figure 8 A network architecture diagram of a dynamic bottleneck module provided for an embodiment of the present application is provided.

[0027] Figure 9 A network architecture diagram of a multi-scale intra-feature interaction module based on an attention mechanism provided for an embodiment of the present application is provided.

[0028] Figure 10 Comparison of prediction results of various models. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application.

[0030] The weld defect detection method provided by the embodiment of the present application introduces X-ray image reconstruction, and includes the following steps:

[0031] Step 1, an X-ray image containing a workpiece weld is obtained, preprocessed, cut into small pictures, and standardized.

[0032] In an actual weld production scene, the obtained weld image is stored in a DCM format, and these images are obtained by X-ray photography. Since the DCM image cannot be directly used for subsequent deep learning model processing, 16-bit grayscale images are first extracted from the DCM file. Considering that there are many problems in the original DCM image after parsing, such as large differences in defect features under different window positions and window widths, inconsistent window positions and window widths of images taken by different devices, and the like, direct use will bring difficulties to defect detection. Therefore, the extracted 16-bit grayscale images are cut into small pictures to ensure that the small pictures cut on the weld and the base plate have consistent styles, can clearly display defects, and have obvious differences between defects and backgrounds, so that the influence of the above factors can be reduced and subsequent processing is facilitated. The small pictures after cutting are standardized to unify the image size to the required size for model input, for example, in subsequent experiments, when the Style-Pix2pix model is trained, the image size is adjusted to 256x256 pixels; when the MO-YOLO model is trained, the image size is adjusted to 640x640 pixels. At the same time, the pixel values are mapped to a specific interval, such as [0, 255], to ensure the stability and consistency of the data, and to provide a good data foundation for subsequent image reconstruction and defect detection.

[0033] Step 2, the small pictures after preprocessing are reconstructed by using a style transfer generative adversarial network (Style-Pix2pix).

[0034] As shown in Figure 1 , the style transfer generative adversarial network (Style-Pix2pix) includes a generator and a discriminator.

[0035] (1) The generator adopts the Unet basic framework, including an encoder and a decoder;

[0036] (1.1) The encoder includes N cascaded coding layers and the decoder includes N cascaded decoding layers; for layers 1 to N-1, coding layers and decoding layers located in the same layer adopt a skip connection of a cascaded global selective kernel attention mechanism module (GSKA); coding layers and decoding layers located in layer N adopt a skip connection of a cascaded orthogonal kernel attention module (OKA).

[0037] (1.1.1) The Global Selective Kernel Attention (GSKA) mechanism module is inspired by selective kernel attention. By changing the feature aggregation method, it first concatenates features of different scales and then uses 1×1 convolution to aggregate features, thus effectively avoiding the information loss problem of the original attention module when fusing features. In the Pix2pix generator, skip connections connect low-level features with high-level features, which helps the model retain the texture and details of the original image, prevent image distortion, and alleviate the gradient vanishing problem. However, the original fusion method has limitations. Here, GSKA is used to improve it, increasing learnability and enabling better selection of features that are beneficial to image generation. This mechanism is placed between the encoder and decoder of each layer of U-Net to strengthen the extracted high-level features. Experimental results show that it has significant improvements in detail metrics MSE and SSIM, as well as in the structure representation metric PSNR, effectively optimizing the details and overall structure of the generated image. For example, when processing weld images with porosity defects, GSKA can more accurately capture the texture details around the defects, resulting in a clearer image with higher similarity to the real image. See Appendix for details. Figure 2 The implementation process of GSKA is as follows:

[0038]

[0039]

[0040]

[0041]

[0042] in, This represents the output feature map of the i-th coding layer, where i = 1, 2, ..., N-1; Represents a 3×3 convolution. Represents a 5×5 convolution; For the merge operation, for convolution, with a normalization layer and a ReLU activation function; 、 is the weight of the linear layer, 、 is the offset of the linear layer, ReLU is the ReLU activation function, and Softmax is the normalization operation. represents the output result of the current global selective kernel attention mechanism module GSKA.

[0043] (1.1.2) In the innermost layer of the Pix2pix generator, the encoding module processes the deepest features in the model, which contain high-level semantic information but lack spatial details, and improper processing can easily lead to problems in the output image. The orthogonal kernel attention module OKA is placed here, which, while retaining the advantages of large separable kernel convolution attention, pays more attention to the fusion of global information and enhances the model's ability to utilize global information, effectively refining the global information of the generated image. However, due to the use of dilated convolution, the convolution layer weights for extracting features are not continuous, which to some extent will lose some details, causing fluctuations in the experimental results in terms of MSE and PSNR indicators, but overall it plays a positive role in optimizing the structure of the generated image. For example, when processing some complex weld structure images, the OKA module can better integrate features at different levels, making the global structure of the generated image more reasonable, although there may be some loss in details, but the overall defect positioning and structure presentation effect is better. Specifically, see the attached Figure 3 , the implementation process of OKA is as follows:

[0044]

[0045]

[0046]

[0047]

[0048] wherein represents the output feature map of the last encoding layer; is the operation of assigning the channel number to the length and width; is a convolution with a kernel size of ; is a convolution with a kernel size of ; is a convolution, is a dilated convolution with a kernel size of ; is a dilated convolution with a kernel size of 3 ; is the operation of Feature movement in the long and wide dimensions to the operation on the channel.

[0049] 、 、 、 、 The normalization layer and the ReLU activation function are followed to improve the nonlinear expression ability of the module.

[0050] All encoding layer structures are the same, using LeakyReLU, convolution block, normalization in series;

[0051] (1.2) The first N-1 decoding layer structures are the same, using ReLU, transpose convolution block, normalization; the last decoding layer uses the rearranged upsampling module;

[0052] (1.2.1) The rearranged upsampling module combines convolution module and pixel rearrangement module, overcoming the shortcomings of traditional transpose convolution in recovering high-frequency information. Pix2pix generator uses transpose convolution when upsampling, which has the problems of being able to learn only fixed upsampling mode, being difficult to recover texture and edge high-frequency information, and being easy to introduce checkerboard effect. The rearranged upsampling module reduces the amount of calculation by first convolution and then pixel rearrangement, and is more stable during training, which can select a more suitable pixel arrangement from the redundant information of the intermediate features, effectively improving the quality of the final generated image. Taking the common undercut defect in the weld image as an example, the rearranged upsampling module can restore the edge details of the undercut more clearly, avoiding the blur and checkerboard effect that may occur in transpose convolution, making the generated image more consistent with the real situation. Specifically, referring to FIG. 6, the implementation process of the rearranged upsampling module is as follows: Figure 4

[0053]

[0054] wherein, is the input feature, = or , represents the result of the output of the first layer encoder processed by the global selective kernel attention mechanism module GSKA, represents the output of the second to last decoding layer, is the ReLU activation function, is a convolution with a kernel size of 3, P is the image rearrangement operation, is the Tanh activation function.

[0055] ​(1.3) To further enhance the model's ability to perceive the global structure and details of the image, MSE loss and SSIM loss are introduced into the loss function of the generator. SSIM loss is used to enhance the similarity of brightness, contrast and overall structure between the generated image and the real image. For weld images, which contain defect areas, SSIM loss can enhance the ability to fix the weld edge in the target image, so that the defect area is more obviously different from the background. For example, when detecting small defects, SSIM loss can make the model pay more attention to the overall shape and edge of the defect, avoiding ignoring the true characteristics of the defect due to the interference of local details. At the same time, increasing SSIM loss can also make the generated image more consistent with the way the human eye perceives, making up for the shortcoming of MSE loss that only focuses on details. MSE loss focuses on reducing the pixel-level error between the real image and the false image. Although the weld image is divided into weld area and base plate area, the difference between the two areas is relatively obvious, but the texture changes little in these two areas, only some differences between the defect and the pixel points. MSE loss can smooth the output, which is suitable for the reconstruction task of weld images.

[0056] The formula of MSE loss is formula (10), and the formula of SSIM loss is formula (11).

[0057]

[0058]

[0059]

[0060] In the formula, represents the final loss value, is the GAN loss, is the MSE loss, is the L1 loss, is the SSIM loss. represents the image generated by the generator, is the generator, is the original image input to the generator, is the target image, indicates the mean square error between features, indicates the mean value of the mean square error, represents the final loss value. and are the average pixel values of and , respectively, and are the variances of features and , indicating the contrast, is the feature and covariance, representing the similarity of the structure, and are two constants, is the final loss value.

[0061] Through the introduction of these two loss functions (MSE loss and SSIM loss), the generator is effectively guided to output higher quality images. By adjusting the weights of each loss term, such as setting the MSE loss weight to 80, it helps the generated image to maintain consistency with the target image in low-dimensional features, avoiding the generated image from being too blurred; setting the weight of the SSIM loss to 30 allows the image details to be focused on while also paying attention to the overall structure of the image. Because the structure of the weld image is relatively not very complex, the weight of the SSIM loss here is slightly lower than that of the MSE loss, in order to achieve the effect of balanced optimization.

[0062] (2) The discriminator adopts a multi-scale discriminator based on PatchGAN. The ordinary discriminator usually only judges one dimension of the generated image, so it may not be sensitive to the changes in the global structure. The multi-scale discriminator can make judgments in different dimensions, while paying attention to the global structure and local details of the generated image, so that the generator can use multiple levels of information for optimization to generate more realistic samples closer to the real image. In terms of stability, a single-scale discriminator may converge too early, making it difficult for the generator to continue to optimize. By introducing a multi-scale discriminator, the generator and discriminator can compete in different scales during training, which can gradually optimize the generator and make the adversarial nature of the training process more balanced.

[0063] In order to enhance the quality of the reconstructed weld image, the multi-scale discriminator based on PatchGAN utilizes the ability of PatchGAN to achieve multi-scale evaluation of the weld image. For details, see the attached Figure 5, which includes n layers of branches, a stack operation layer, each layer of branch includes a PatchGAN discriminator; the input of the first layer of branch is the image generated by the generator, the input of the 2nd to n layer of branch is the result of the average pooling of the input of the last layer; when downsampling is performed using average pooling, the value taken is the average value of the local area, which avoids the dependence on the local maximum value. Therefore, it can capture the overall features of the image, not just some local mutations. Average pooling also helps to eliminate noise and irregular changes in the image, making the discrimination result more stable and reliable. For example, there may be some noise points in the weld image due to the shooting or transmission process, and average pooling can reduce the interference of these noise on the discrimination result to a certain extent. The stack operation layer performs stack operation on the outputs of all layer branches; using stack operation to perform the final feature aggregation operation, first, the features extracted at different scales can be preserved, ensuring the richness of the features. It can also ensure that the features at each scale still maintain independence when merging, avoiding information interference between different scales, which can be processed and analyzed separately. This means that the model can more comprehensively consider the features of the image at different scales, thus making more accurate judgments on the authenticity of the generated image. For example, when detecting small defects, small-scale features can provide detailed information about the defects, while large-scale features can help determine the location of the defects in the entire weld image and their relationship with other parts. The specific implementation process is as follows:

[0064]

[0065]

[0066]

[0067] wherein, is an average pooling operation, represents the input of the i-th layer branch, ; is a PatchGAN discriminator; is a stack operation, is the output result of the discriminator.

[0068] Step 3, using a global self-adaptive defect detection model (MO-YOLO) to detect defects in the reconstructed image X ori , and obtain the defect category and position information.

[0069] Referring to the accompanying Figure 6 ​, the global adaptive defect detection model (MO-YOLO) comprises a first lightweight downsampling module LiteConv, four lightweight dynamic energy gathering feature tuple modules connected in series, an attention-based multi-scale internal feature interaction module AMIFI, two upsampling dynamic selection bottleneck modules, two lightweight sampling connection selection modules, and three global adaptive attention detection heads GAA-Detect; the input of the first lightweight downsampling module LiteConv is a reconstructed image output by Style-Pix2pix, and the output end is connected to the input end of the first lightweight dynamic energy gathering feature tuple module;

[0070] The lightweight dynamic energy gathering feature tuple module comprises a second lightweight downsampling module LiteConv and a first dynamic selection bottleneck module DSB-C2f based on conditional convolution connected in series.

[0071] The upsampling dynamic selection bottleneck module comprises an upsampling module, a Concat operation, and a second dynamic selection bottleneck module DSB-C2f based on conditional convolution connected in series; the output end of the attention-based multi-scale internal feature interaction module AMIFI is connected to the upsampling module of the first upsampling dynamic selection bottleneck module, and the input end of the Concat operation is connected to the output end of the third lightweight dynamic energy gathering feature tuple module and the output end of the upsampling module; the output end of the second dynamic selection bottleneck module DSB-C2f based on conditional convolution of the first upsampling dynamic selection bottleneck module is connected to the upsampling module of the second upsampling dynamic selection bottleneck module, and the input end of the Concat operation is connected to the output end of the second lightweight dynamic energy gathering feature tuple module and the output end of the upsampling module;

[0072] The lightweight sampling connection selection module comprises a third lightweight downsampling module LiteConv, a Concat operation, and a third dynamic selection bottleneck module DSB-C2f based on conditional convolution connected in series; the output end of the second upsampling dynamic selection bottleneck module is connected to the input end of the third lightweight downsampling module LiteConv of the first lightweight sampling connection selection module, the output end of the third lightweight downsampling module LiteConv is connected to the input end of the Concat operation, and the output end of the second dynamic selection bottleneck module DSB-C2f based on conditional convolution in the first upsampling dynamic selection bottleneck module is connected to the input end of the Concat operation; the output end of the first lightweight sampling connection selection module is connected to the input end of the third lightweight downsampling module LiteConv of the first lightweight sampling connection selection module, the output end of the third lightweight downsampling module LiteConv is connected to the input end of the Concat operation, and the output end of the attention-based multi-scale internal feature interaction module AMIFI is connected to the input end of the Concat operation;

[0073] An input end of a first global adaptive attention detection head GAA-Detect is connected to an output end of the second up-sampling dynamic selection bottleneck module;

[0074] An input end of a second global adaptive attention detection head GAA-Detect is connected to an output end of the first light sampling connection selection module;

[0075] An input end of a third global adaptive attention detection head GAA-Detect is connected to an output end of the second light sampling connection selection module;

[0076] The first, second and third light down-sampling modules LiteConv have the same architecture and each include two convolutional layers. The light down-sampling module skillfully uses a two-layer convolutional structure to seamlessly splice three-fourths of the features processed by single convolution with one-fourth of the features processed by double convolution. This ingenious design, on the one hand, greatly reduces the spatial occupation of the feature map, effectively reduces the computational load of each convolutional layer, and significantly speeds up the inference process of the down-sampling module; on the other hand, it successfully retains rich and diverse feature representation, greatly enhancing the model's ability to capture and extract the characteristics of the weld defects. In the feature pyramid structure of YOLOv8, this module further expands the deep feature expression dimension, significantly improves the generalization performance of the model, and enables it to comfortably cope with the complex conditions presented by the weld defects, such as diverse shapes, different scales and shapes. The specific implementation process is as follows:

[0077]

[0078]

[0079] In formula (15) represents the input feature; is a convolutional layer with a kernel size of 1 and a step size of 1, which includes a normalization layer and a ReLU activation function, and the output channel number is three-fourths of the set channel number; is the output feature.

[0080] In formula (16) is the output of formula (15), is a second convolutional layer with a kernel size of 5 and a step size of 1, with a normalization layer and a ReLU activation function, and the output channel number is one-fourth of the set channel number; is the output of the module.

[0081] The first, second and third dynamic selection bottleneck module based on conditional convolution (DSB-C2f) has the same architecture. In the key calculation of routing information, the Top-K strategy is introduced innovatively, and the last convolution layer of the bottleneck is replaced with a dynamic selection convolution. In the architecture of YOLOv8, the C2f module mainly undertakes the important task of feature transfer, but the fixed size of the convolution kernel in the bottleneck makes it difficult to achieve the best expression effect according to the unique characteristics of the features. The DSB module uses conditional convolution technology to accurately calculate the routing weights of each "expert" (there are 4 "experts" in the figure) through a series of ordered steps such as adaptive average pooling, linear layer, ReLU activation function and Softmax operation, and selects the largest K values as the final routing information input to the conditional convolution, and then calculates the convolution kernel that best fits the current features. This advanced calculation method greatly enriches the information contained in the module, effectively avoids the redundancy of routing information, and significantly enhances the expression ability of the module. After the dynamic selection convolution, the SiLU activation function is used instead of the traditional ReLU activation function. The SiLU function can exhibit excellent performance in reducing information loss and effectively prevent the problem of gradient disappearance during training. However, due to the relatively complex structure of the module itself, the complexity of the model in the local area increases after introducing this module, which leads to a certain degree of decline in recall rate, but it performs well in precision, which can significantly reduce the number of false identifications of defects. See Fig. 8, the specific implementation process is as follows: Figure 8

[0082]

[0083]

[0084]

[0085] In formula (17), represents the input feature; represents the output of the dynamic selection convolution; represents the final output; is a conditional convolution that receives input features and routing information as parameters; is the top k weights; and are the corresponding activation functions, respectively; is a linear layer; is an adaptive average pooling; and are the corresponding activation function and normalization layer, respectively. ​

[0086] Referring to the accompanying drawings Figure 9 The attention-based multi-intrascale feature interaction module (AMIFI) aims to effectively compensate for the information loss defect in the information processing process of the spatial pyramid pooling layer, and its construction basis is the Transformer encoder, and a multi-scale two-dimensional sine cosine position encoding is introduced. The AMIFI module can more efficiently promote the information flow and interaction within the feature pyramid, and effectively improve the accuracy of the model prediction result. The specific implementation process is:

[0087] 1. According to the output features of the fourth lightweight dynamic energy gathering feature tuple module, a plurality of two-dimensional sine cosine encodings of different scales are obtained through a multi-scale two-dimensional sine cosine encoding module. For example, the module provides three two-dimensional sine cosine encodings of different scales, and the temperature coefficients are 10000, 5000 and 2000 respectively.

[0088] 2. Each two-dimensional sine cosine encoding is multiplied by a corresponding learnable weight coefficient for fusion.

[0089] 3. The fused results are stacked in the 0th dimension.

[0090] 4. The stacked results are summed.

[0091] 5. The sum result is used as the position encoding.

[0092] 6. The position encoding is input into N multi-head attention modules for processing to obtain the final output. The multi-head attention module includes multi-head attention, addition & normalization, feedforward network, addition & normalization.

[0093] The two-dimensional sine cosine encoding receives the length and width of the output features of the fourth lightweight dynamic energy gathering feature tuple module, a plurality of different temperature coefficients, and a position encoding dimension parameter to generate a position encoding, and the implementation process is:

[0094]

[0095] In formula (19), represents the final output, and is the length and width of the output features of the fourth lightweight dynamic energy gathering feature tuple module, is the temperature coefficient, is the two-dimensional sine cosine encoding, represents the configuration temperature coefficient corresponding to the learnable weight coefficient.

[0096] The global adaptive attention detection head GAA-Detect introduces a global adaptive attention mechanism, strictly follows the core design idea of the encoder and the decoder, and deeply mines and efficiently extracts feature information. At the starting position of the module, a global adaptive attention module is carefully arranged for reinforced processing of the channel dimension and the spatial dimension, which can accurately highlight the key features of the target object and effectively weaken the interference influence of the background information. At the same time, the original convolution structure of the detection head is reasonably optimized and adjusted, and the original two 3x3 convolutions are innovatively changed into a combination form of two 1x1 convolutions and one 3x3 convolution. Among them, the first 1x1 convolution is used to reduce the number of channels, thereby reducing the calculation load of the subsequent 3x3 convolution, and effectively removing redundant channels; the second 3x3 convolution is focused on feature extraction and down-sampling operation; and the last 1x1 convolution is responsible for restoring the number of channels to an appropriate level. This carefully designed structural adjustment gives the detection head the powerful ability to handle more complex features, significantly reduces the calculation amount, and greatly improves the detection performance. In addition, through the innovative improvement of the calculation method, the calculation efficiency is effectively improved, which effectively guarantees the independence of the picture in the spatial and channel information levels, thereby effectively strengthening the image features used for detection.

[0097] Specifically, referring to the accompanying drawings Figure 7 , the GAA-Detect includes a global adaptive attention module, a first branch, and a second branch, and the implementation process is as follows:

[0098] The global adaptive attention module is to calculate the weight map of the input feature through channel attention and spatial attention, and then the input feature is weighted with the weight map in a residual connection manner to obtain the strengthened feature.

[0099] The first branch is responsible for predicting the formula of the candidate box (anchor box loss) part as shown in formula (22):

[0100]

[0101] In the formula, is the strengthened feature output by the global adaptive attention module, is a combination of a two-dimensional convolution, a normalization layer, and an activation function, the convolution kernel size is 1, the step is 1, and the padding is 1; and are the same , the convolution kernel, step, and padding of are 3, 2, and 1 respectively; For two-dimensional convolution, the convolution kernel, step and padding are all 1. For the output of the first branch, that is, the anchor frame loss.

[0102] The formula of the part responsible for predicting the object category (class loss) of the second branch is as formula (23).

[0103]

[0104] In the formula For the output of the second branch, that is, the class loss.

[0105] The present embodiment compares the Style-Pix2pix network of the present application with the classic networks DNCNN, Uformer, RED-CNN, BRDNet and Pix2pix, and evaluates the indicators of the models on the GDXRay dataset and the weld image reconstruction dataset.

[0106] Table 1 Results in the GDXray weld dataset

[0107]

[0108] Table 1 is the result of each model on the GDXRay weld dataset, wherein the MSE of the Style-Pix2pix proposed in the present application is 0.3703, which is 0.2942 lower than that of the benchmark model Pix2pix, and the MSE index of Pix2pix is 0.6645. The PSNR is 2.84 higher than that of the benchmark model, and the benchmark model is 32.25. The SSIM of Pix2pix is 0.9298, and the Style-Pix2pix is 0.0429 higher than that. The generator of Style-Pix2pix increases two attention mechanisms on the basis of the Pix2pix generator, which can help the model to extract the features conducive to generating the target image from the original image, and uses the rearrangement upsampling module in the outermost decoder part to arrange the input features and select the most suitable arrangement. The present application improves the generator in the direction of reducing the amount of information, and the information of the target image can be found in the original image. The MSE loss and SSIM loss are also introduced in the loss function of the generator, and a multi-scale discriminator based on PatchGAN is introduced. These two improvements can also help the generator to optimize the quality of the generated image in the overall structure and the details, and based on the strong discrimination ability of the multi-scale discriminator, the optimization time of the generator can be increased in the later stage of the training process, so that the generated image performs better in details.

[0109] Table 2 Results of the weld image preprocessing dataset

[0110]

[0111] The experimental results of each model on the self-built weld image preprocessing dataset are shown in Table 2. The MSE of Style-Pix2pix is 34.02, the PSNR is 33.30, and the SSIM is 0.9123. Compared with the benchmark model Pix2pix, the MSE of Pix2pix is 47.94, which is 13.92 lower than that of the model. The PSNR of Pix2pix is 31.81, which is 1.49 higher than that of the model. The SSIM of the model is 0.0307 higher than that of Pix2pix. The Style-Pix2pix model achieves better results than other models, with all indicators higher than those of other models. The task to be done on this dataset is image denoising, which requires the image structure to be the same as the original image, and the defect area to be highlighted with strong contrast with the background, while the texture of the defect remains unchanged. Therefore, this task is more complex than the GDXRay weld dataset. Because Style-Pix2pix improves the network structure and discriminator, the quality of the generated image is also greatly improved.

[0112] The prediction results of each model for the same picture are shown in Figure 10 , where the top left corner is the original image, the right is the target image, and the rest are the prediction results of each model, which are labeled below the picture. Figure 10 The picture shown has an undercut defect on the left side of the middle part of the image. In the predicted image of Pix2pix, it can be found that the defect area is relatively blurred compared to the target image, and the brightness of the defect center is not enough. On the right side of the weld area, most of the area is occupied by gray, while the boundary between gray and black in the original image is relatively obvious. At the lower right position of the weld, there is a white spot in the original image, which is also restored to white in the generated result of Pix2pix, while the position is processed as background in the target image, which is black. Style-Pix2pix restores this point. The reconstruction of the defect area by DNCNN is also relatively blurred, and the color of the motherboard area is not clearly distinguished from the color of the weld area. The white spot at the lower right is not processed as background, and the result is worse than that of Style-Pix2pix. The reconstruction effect of Uformer on the image is better than that of other models, reaching a similar effect to Style-Pix2pix. RED-CNN handles the defect and weld area well, but does not completely process the white spot as background, but as a more obvious black color. BRD-Net handles the weld area well, but the defect area is processed relatively blurred, and the white spot is also processed towards a brighter direction, failing to capture the optimization direction from the original image to the target image. In addition, in the middle part of the image, there is a regular non-defect area below the weld area, and this model restores the edge of this area poorly.

[0113] The performance of the MO-YOLO model is compared with YOLOv5, YOLOv6, YOLOv8, YOLO11, RT-DETR, and Faster-RCNN through experiments on the NEU-DET dataset and a self-built weld defect detection dataset. The weld defect detection dataset has a total of 5209 images, which are divided into a training set, a validation set, and a test set according to an 8:1:1 ratio. The number of images in the training set, the validation set, and the test set is 4167, 521, and 521, respectively. The defect types are divided into eight categories, namely single wire detection line, pit, plate injury, gas hole, undercut, misalignment, tungsten clamping, and external concave. The number of instances of each defect type is 520, 869, 1603, 1186, 1571, 671, 614, and 330, respectively, and the total number of instances is 7364.

[0114] Table 2: Results of each model on the NEU-DET dataset

[0115]

[0116] The performance of each model on the NEU-DET dataset is shown in Table 1. From the table, it can be concluded that MO-YOLO performs better than other models in terms of AP, AR, F1_score, and mAP50, and has a moderate level of FPS and a low FLOPS, which can meet the needs of some edge devices, and achieves a balance between precision and speed. The AP of MO-YOLO is 74.9%, which is 5.6% higher than that of the benchmark model YOLOv8s. The AR is 72.1%, which is 1.8% higher than that of the benchmark model. The F1_score is 73.5%, which is 3.7% higher than that of the benchmark model. The mAP50 is 77.1, which is 2.4% higher than that of the benchmark model. The FPS is 145, which is 99 frames lower than that of the benchmark model. The FLOPS is 38G, which is 10G higher than that of the benchmark model. MO-YOLO improves four modules such as LiteConv and DAA-Detect compared with YOLOv8s, which proves the effectiveness of these modules combined together. However, the addition of these modules also increases the depth of the model, such as LiteConv which changes one layer of convolution to two layers, and GAA-Detect which adds attention mechanism before the detection head, which reduces the speed of model inference. Therefore, the FPS of MO-YOLO is 99 frames lower than that of YOLOv8s. Similarly, these modules also increase the computational complexity of model inference, so the FLOPS increases by 10G, of which GAA-Detect contributes the most.

[0117] Table 2: Results of each model on the self-built weld defect detection dataset

[0118]

[0119] The performance of each model on the self-built data set is shown in Table 2. MO-YOLO achieves the best performance in the precision indicators AP, F1_score and mAP50 compared with other models, and is relatively poor in the indicators of AR, FPS and FLOPS. Among them, the AP of MO-YOLO is 84.7%, which is 3.8% higher than the benchmark model. The AR is 82.7%, which is 12.7% higher than the benchmark model. The F1_score is 83.7%, and the benchmark model is 75.1%, which is 8.6% higher. The mAP50 is 87.2%, which is 17.5% higher than the YOLOv8s. The frame number of MO-YOLO is 90 frames, which is 35 frames lower than the YOLOv8s model, but the precision is greatly improved. Compared with other YOLO models, the FLOPS of MO-YOLO is higher, which is 38G, 10G higher than the YOLOv8s model. Except for Faster-RCNN, MO-YOLO is better than other models in the precision indicators. The decline in the speed indicators FPS and FLOPS is within an acceptable range, although it is not the highest, but they are all within a relatively good range.

[0120] The embodiment also provides a weld defect detection system, which comprises:

[0121] A data acquisition module is responsible for acquiring an X-ray image of a workpiece weld and pre-processing the same;

[0122] An image reconstruction module is responsible for image reconstruction on the pre-processed image by using a style transfer generative adversarial network Style-Pix2pix to obtain a reconstructed image;

[0123] A defect detection module is responsible for defect detection on the reconstructed image by using a global self-adaptive defect detection model MO-YOLO to obtain defect category and position information.

[0124] In addition, those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CDROM, optical storage, etc.) containing computer usable program code.

[0125] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0126] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0127] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flow or blocks Figure 1 one or more flow or blocks

[0128] The above described embodiments are merely intended to illustrate the technical solutions of the present application, but not to limit the same; even though the above detailed description has been made with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some of the technical features can be replaced equivalently; and such modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A weld defect detection method incorporating X-ray image reconstruction, characterized by, The method comprises: acquiring an X-ray image containing a workpiece weld, and preprocessing the X-ray image; reconstructing the preprocessed image by using a style transfer generative adversarial network Style-Pix2pix to obtain a reconstructed image; detecting defects in the reconstructed image by using a global adaptive defect detection model MO-YOLO to obtain defect categories and position information; The style transfer generative adversarial network Style-Pix2pix comprises a generator and a discriminator; the generator adopts an Unet basic framework and comprises an encoder and a decoder; the encoder comprises N encoding layers connected in series, and the decoder comprises N decoding layers connected in series; for the 1st to (N-1)th layers, the encoding layers and the decoding layers at the same layer adopt a skip connection of a global selective kernel attention mechanism module GSKA; and for the Nth layer, the encoding layers and the decoding layers adopt a skip connection of an orthogonal kernel attention module OKA. The implementation process of the global selective kernel attention mechanism module GSKA is as follows: ； ； ； ； wherein, represents the output feature map of the i-th encoding layer, i = 1, 2, …, N-1; represents a 3x3 convolution, represents a 5x5 convolution; is a merge operation, is convolution, equipped with a normalization layer and a ReLU activation function; , is a weight of a linear layer, , is a bias of a linear layer, ReLU is a ReLU activation function, and Softmax is a normalization operation; represents an output result of the current global selective kernel attention mechanism module GSKA. The implementation process of the orthogonal kernel attention module OKA is as follows: ； ； ； ； in This represents the output feature map of the last coding layer; It is The operation of allocating channel numbers to length and width; The convolution kernel is Convolution of size, The kernel size is Convolution; for convolution, The kernel size is dilated convolution, For a convolution kernel size of 3 dilated convolution; To be The operation of moving features in the length and width dimensions to the channel; The global adaptive defect detection model MO-YOLO comprises a first lightweight down-sampling module LiteConv, four lightweight dynamic energy aggregation feature tuple modules connected in series, an attention-based multi-scale internal feature interaction module AMIFI, two up-sampling dynamic selection bottleneck modules, two lightweight sampling connection selection modules, and three global adaptive attention detection heads GAA-Detect; the first lightweight down-sampling module LiteConv takes the reconstructed image output by the Style-Pix2pix as input, and the output end is connected to the input end of the first lightweight dynamic energy aggregation feature tuple module; the attention-based multi-scale internal feature interaction module AMIFI takes the output end of the last lightweight dynamic energy aggregation feature tuple module as input; The lightweight dynamic energy aggregation feature tuple module comprises a second lightweight down-sampling module LiteConv and a first dynamic selection bottleneck module DSB-C2f connected in series; The up-sampling dynamic selection bottleneck module comprises an up-sampling module, a Concat operation, and a second dynamic selection bottleneck module DSB-C2f connected in series; the up-sampling module of the first up-sampling dynamic selection bottleneck module is connected to the output end of the attention-based multi-scale internal feature interaction module AMIFI, the input end of the Concat operation is connected to the output end of the third lightweight dynamic energy aggregation feature tuple module and the output end of the up-sampling module; the up-sampling module of the second up-sampling dynamic selection bottleneck module is connected to the output end of the second dynamic selection bottleneck module DSB-C2f of the first up-sampling dynamic selection bottleneck module, and the input end of the Concat operation is connected to the output end of the second lightweight dynamic energy aggregation feature tuple module and the output end of the up-sampling module; The light sampling connection selection module comprises a third light down-sampling module LiteConv, a Concat operation, and a third dynamic selection bottleneck module DSB-C2f based on conditional convolution connected in series; an input end of the third light down-sampling module LiteConv of the first light sampling connection selection module is connected with an output end of the second up-sampling dynamic selection bottleneck module, and an input end of the Concat operation is connected with an output end of the third light down-sampling module LiteConv and an output end of the second dynamic selection bottleneck module DSB-C2f based on conditional convolution in the first up-sampling dynamic selection bottleneck module; an input end of the third light down-sampling module LiteConv of the first light sampling connection selection module is connected with an output end of the first light sampling connection selection module, and an input end of the Concat operation is connected with an output end of the third light down-sampling module LiteConv and an output end of the attention-based multi-scale internal feature interaction module AMIFI; an input end of the first global adaptive attention detection head GAA-Detect is connected with an output end of the second up-sampling dynamic selection bottleneck module; an input end of the second global adaptive attention detection head GAA-Detect is connected with an output end of the first light sampling connection selection module; an input end of the third global adaptive attention detection head GAA-Detect is connected with an output end of the second light sampling connection selection module.

2. The method of claim 1, wherein, The first N-1 decoding layers have the same structure, and each decoding layer adopts ReLU, a transpose convolution block, and normalization; the last decoding layer adopts a rearranged up-sampling module; The implementation process of the rearranged up-sampling module is as follows: ； wherein, is the input feature, or , represents the result of the output of the first layer encoder processed by the global selective kernel attention mechanism module GSKA, represents the output of the second-to-last decoding layer, is a ReLU activation function, is a convolution with a kernel size of 3, is an image rearrangement operation, is a Tanh activation function.​ 3. The method of claim 1, wherein, The discriminator adopts a multi-scale discriminator based on PatchGAN, and comprises n layer branches and a stack operation layer, each layer branch comprises a PatchGAN discriminator; the input of the first layer branch is an image generated by the generator, the input of the 2nd to n layer branches is the result after average pooling of the input of the last layer; the stack operation layer performs stack operation on the outputs of all layer branches.

4. The method of claim 1, wherein, The global adaptive attention detection head GAA-Detect comprises a global adaptive attention module, a first branch, and a second branch, and the implementation process is as follows: The global adaptive attention module is to calculate the weight map of the input feature map through channel attention and spatial attention, and then the input feature is weighted with the weight map in a residual connection manner to obtain the reinforced feature; The implementation process of the first branch is as follows: ； In the formula, reinforced features output by the global adaptive attention module, , , is a combination of a two-dimensional convolution, a normalization layer and an activation function; The convolution kernel size of is 1, the step is 1, and the padding is 1; The convolution kernel, step, and padding of are 3, 2, and 1, respectively; is a two-dimensional convolution, whose convolution kernel, step, and padding are all 1; is the output of the first branch; The implementation process of the second branch is as follows: ； In the formulae is the output of the second branch.

5. A system for detecting defects in a weld according to any one of claims 1 to 4, characterised in that, The system comprises: a data acquisition module, which is responsible for acquiring an X-ray image containing a workpiece weld and pre-processing the same; an image reconstruction module, which is responsible for performing image reconstruction on the pre-processed image by using a style transfer generative adversarial network Style-Pix2pix to obtain a reconstructed image; a defect detection module, which is responsible for performing defect detection on the reconstructed image by using a global self-adaptive defect detection model MO-YOLO to obtain defect category and position information.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed in the computer, the computer executes the method of any one of claims 1-4.

7. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and the processor executes the executable code to implement the method in any one of claims 1-4.

Images