X-ray welding seam image defect detection method based on deep learning

Abstract

The invention provides an X-ray welding seam image defect detection method based on deep learning, which comprises the following steps: firstly, making a welding seam defect detection data set, then dividing the data set, and then carrying out defect feature extraction and detection by utilizing an improved Gold-YOLO method. The Gold-YOLO backbone network adopts an OfficientRep network and is composed of RepVGG modules based on structure reparatization, and the network characterization capability is enhanced by adopting a multi-branch feature extraction method; the method comprises the following steps: carrying out feature aggregation on a bottom-layer feature map, carrying out information fusion on the feature map after feature aggregation by adopting an InceptionNext module, and by using a rapid self-attention mechanism, replacing a normalization layer and an activation function, reducing the size of the feature map and accelerating matrix operation in the self-attention mechanism, so as to achieve the purpose of accelerating the reasoning speed. And finally, a fast two-way gating mechanism is used as a feedforward neural network, so that the precision is ensured while information is screened.

Description

Technical Field

[0001] The present invention belongs to the technical field of weld defect detection, and in particular relates to an X-ray weld image defect detection method based on deep learning. Background Art

[0002] As the main method of connecting two workpieces, welding technology is widely used in aerospace, machinery manufacturing, energy transportation, microelectronics and other fields. With the development of welding technology at home and abroad, the requirements for welding quality are getting higher and higher, but due to the influence of welding environment and welding process, many types of defects will inevitably occur, such as pores, slag inclusions, cracks, incomplete penetration, lack of fusion, etc. These defects will lead to a decrease in the strength of the weld connection area and stress concentration, thus leading to serious safety accidents.

[0003] X-ray weld image defect detection has gone through three stages: traditional methods, machine learning methods, and deep learning methods. The traditional method is based on grayscale analysis of images, but this method needs to re-explore suitable thresholds for weld radiographic images in different scenes and with different grayscale distributions, and has obvious limitations. Research based on machine learning mainly focuses on weld defect classification tasks, including artificial neural networks of multi-layer perceptrons, K nearest neighbors, C-means clustering, support vector machines, and fuzzy algorithms. With the rapid development of deep learning, the classification task of X-ray weld image defects has been transformed into a detection task, and mainstream models include Faster RCNN, Mask RCNN, CenterNet, YOLO, SSD, RetinaNet, etc. However, the weld defect detection method based on deep learning is very similar to the conventional visual task method, lacks targeted improvement strategies, and has the current status of slow detection speed, low detection accuracy, and inability to meet the needs of industrial backgrounds. Therefore, the present invention uses the latest Gold-YOLO method as the baseline model, and adopts feature aggregation, fast attention mechanism and fast two-way gating mechanism for improvement, which effectively improves the detection accuracy and maintains a high speed, meets the needs of X-ray weld defect detection, and is widely used in the industrial field. Summary of the invention

[0004] In view of the shortcomings in the prior art, the present invention provides an X-ray weld image defect detection method based on deep learning, which solves the problems of slow detection speed and low detection accuracy existing in traditional methods.

[0005] The present invention achieves the above technical objectives through the following technical means.

[0006] A deep learning-based X-ray weld image defect detection method includes the following process:

[0007] Step 1: Create a weld defect detection dataset;

[0008] Step 2: Construct an improved Gold-YOLO method for defect feature extraction and detection.

[0009] Furthermore, the specific process of step 1 is as follows:

[0010] Step 1.1: First, obtain the original weld image data and filter it to remove noise;

[0011] Step 1.2: Crop the weld image, screen the cropped weld image, and mark n types of weld defects, and finally obtain a y*y size weld defect dataset containing n types;

[0012] Step 1.3: Then divide the weld defect dataset into training set, validation set, and test set.

[0013] Furthermore, the specific process of step 2 is as follows:

[0014] Step 2.1: Use the training set as network input and input it into the improved Gold-YOLO backbone network model;

[0015] Step 2.2: The Gold-YOLO backbone network uses the EfficientRep network, which is composed of the RepVGG module based on structural reparameterization and uses a multi-branch feature extraction method to enhance the network representation capability;

[0016] Step 2.3: Use the method of structural reparameterization to make the branch network lightweight. During the inference process, delete the 1×1 convolution and residual connection in the network branch. During the training process, the backbone network uses RepBlock and RepConv modules, and only uses RepConv modules during the inference process.

[0017] Step 2.4: The multi-scale features B2, B3, B4 and B5 obtained in step 2.3 are input into the feature aggregation module for feature fusion. The feature aggregation module is divided into a low-order feature aggregation module and a high-order feature aggregation module;

[0018] Step 2.5: The feature aggregation map is passed into the information fusion module. Two different methods are used in the low-level and high-level information fusion. In the low-level, the latest InceptionBlock is introduced as the information fusion module; InceptionBlock consists of 4 branches, including 3×3 deep convolution to avoid large convolution kernels from reducing the training speed, and decoupling a large convolution kernel Kh×Kw into two small convolution kernels 1×Kw and Kh×1, and finally using residual connection to ensure context information; in high-level information fusion, after aggregating the feature map information based on the P5 layer, a fast self-attention mechanism is used for information fusion;

[0019] Step 2.6: After the information is aggregated through the two low-order and high-order modules, it is passed to the interference module. The interference module first fuses the local information with the global information. The effective features can be strengthened and the redundant features can be suppressed through element-wise multiplication. Then the residual connection is used to ensure the propagation of contextual information.

[0020] Step 2.7: The detection head uses a decoupled detection head and adopts anchor frame assisted training; the decoupled detection head finally outputs the four coordinates of the defect, the defect type and the confidence, and then the four coordinates are connected through the post-processing process of the network to obtain the detection result.

[0021] Furthermore, in step 2.4, the purpose of the low-order feature aggregation module is to obtain the texture and edge features of the underlying feature map and aggregate the features of the four feature layers output by the backbone network. The formula of the low-order feature aggregation module is as follows:

[0022] F4,F5(H,W)=billinear([B4,B5])

[0023] F2(H,W)=average_pool(B2)

[0024] F concat =Concat(F2,B3,F4,F5)

[0025] Among them, billinear represents the bilinear interpolation method, H and W represent the size of the B3 layer feature map, average_pool represents the average pooling downsampling operation; F4 and F5 represent the feature maps of B4 and B5 respectively expanded by the bilinear interpolation method; F2 represents the B2 feature map processed by the average pooling layer; F concat Represents the concatenated feature map; Concat means the concatenation operation integrates four feature maps into one feature map.

[0026] Furthermore, in step 2.4, the purpose of the high-order feature aggregation module is to obtain semantic information in the feature map, and to perform feature aggregation based on the P5 layer. The formula of the high-order feature aggregation module is as follows:

[0027] P3,P4(H,W)=average_pool(P3,P4)

[0028] Among them, H and W are the height and width of the feature map of the P5 layer; P3, P4, and P5 are high-order feature maps respectively; average_pool represents the average pooling downsampling operation.

[0029] Furthermore, in step 2.6, in order to ensure the consistency of the size of the feature map, the feature map is adjusted using the pooling layer and bilinear interpolation method. The specific calculation formula is as follows:

[0030]

[0031] Among them, Avg / Bilinear respectively represent average pooling downsampling and bilinear interpolation to scale feature maps of different sizes. represents the fully connected layer, X global represents the global feature, X local Represents local features, represents the addition of elements, Represents element-wise multiplication.

[0032] The present invention has the following beneficial effects:

[0033] The present invention improves the Gold-YOLO network. The backbone network adopts the EfficientRep network, which is composed of the RepVGG module based on structural reparameterization. The method of multi-branch feature extraction is used to enhance the network's characterization ability. In view of the low precision problem, feature aggregation is performed on the underlying feature map, and then the latest InceptionNext module is used to fuse the feature map after feature aggregation. Secondly, a fast self-attention mechanism is used. By replacing the normalization layer and activation function and reducing the size of the feature map, the matrix operation in the self-attention mechanism is accelerated to achieve the purpose of accelerating the reasoning speed. Finally, a fast two-way gating mechanism is used as a feedforward neural network to ensure accuracy while screening information. The method of the present invention has been improved in detection accuracy, maintained at a high speed, and has also been improved in small target detection evaluation indicators, meeting the requirements of X-ray weld image defect detection for accuracy and speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The original image of the weld defect of the present invention;

[0035] Figure 2 Schematic diagram of different types of weld defects according to the present invention;

[0036] Figure 3 Schematic diagram of the improved Gold-YOLO network

[0037] Figure 4 This is a schematic diagram of the Efficient backbone network;

[0038] Figure 5 Schematic diagram of low-order feature aggregation network;

[0039] Figure 6 Schematic diagram of high-order feature aggregation network;

[0040] Figure 7 This is a schematic diagram of the InceptionBlock module;

[0041] Figure 8 Schematic diagram of the fast self-attention module;

[0042] Fig. 9 It is the schematic diagram of the interference module;

[0043] Fig.10 This is a schematic diagram of the decoupling detection head module;

[0044] Fig.11 Schematic diagram of network accuracy before and after improvement, where (a) is the confusion matrix of the baseline model and (b) is the confusion matrix of the improved model;

[0045] Fig.12 This is the weld defect detection effect diagram. DETAILED DESCRIPTION

[0046] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments, but the protection scope of the present invention is not limited thereto.

[0047] See attached Figures 1 to 12 The X-ray weld image defect detection method based on deep learning of the present invention comprises the following process:

[0048] Step 1: Create a weld defect detection dataset. The specific steps are as follows:

[0049] Step 1.1: First, obtain the original weld image data, preprocess the original weld image data, filter the original weld image size w*h, and use the Gaussian filtering method to remove noise from the weld image.

[0050] Step 1.2: Crop the weld image, randomly crop the original weld image size w*h to a size of y*y, screen the cropped image, and annotate n types of weld defects, and finally obtain a weld defect dataset D of size y*y containing n types;

[0051] Step 1.3: Then divide the data set D into training set D1, validation set D2, and test set D3 according to the ratio of a:b:c.

[0052] Step 2: Construct an improved Gold-YOLO method to extract and detect defect features. The specific steps are as follows;

[0053] Step 2.1: Use the training set D1 as the network input to the improved Gold-YOLO backbone network model;

[0054] Step 2.2: The backbone network of Gold-YOLO uses the EfficientRep network, which is composed of RepVGG modules based on structural reparameterization and uses a multi-branch feature extraction method; compared with a single-branch convolutional network, a multi-branch network can better enhance the representation ability of the network. The formula of the RepVGG training module is as follows:

[0055]

[0056] Step 2.3: Use the method of structural reparameterization to make the branch network lightweight. During the inference process, delete the 1×1 convolution and residual connection in the network branch. During the training process, the backbone network uses RepBlock and RepConv modules, and only uses RepConv modules during the inference process.

[0057] Step 2.4: The multi-scale features B2, B3, B4 and B5 obtained in step 2.3 are used as input and input into the feature aggregation module for feature fusion; the feature aggregation module is divided into a low-order feature aggregation module and a high-order feature aggregation module. The purpose of the low-order feature aggregation module is to obtain the texture and edge features of the underlying feature map and aggregate the features of the four feature layers output by the backbone network. The formula of the low-order feature aggregation module is as follows:

[0058] F4,F5(H,W)=billinear([B4,B5])

[0059] F2(H,W)=average_pool(B2)

[0060] F concat =Concat(F2,B3,F4,F5)

[0061] Among them, billinear represents the bilinear interpolation method, H and W represent the size of the B3 layer feature map, average_pool represents the average pooling downsampling operation; F4 and F5 represent the feature maps of B4 and B5 respectively expanded by the bilinear interpolation method; F2 represents the B2 feature map processed by the average pooling layer; F concat Represents the concatenated feature map; Concat represents the concatenation operation that integrates four feature maps into one feature map;

[0062] like Figure 6 As shown in the figure, P3, P4, and P5 are high-order feature maps, which contain rich semantic information. The purpose of the high-order feature aggregation module is to obtain the semantic information in the feature map. Therefore, feature aggregation is performed based on the P5 layer. The formula of the high-order feature aggregation module is as follows:

[0063] P3,P4(H,W)=average_pool(P3,P4)

[0064] Among them, H and W are the height and width of the feature map of the P5 layer;

[0065] Step 2.5: The feature aggregation graph is passed into the information fusion module. Two different methods are used in the information fusion of the low stage and the high stage. In the low stage, the latest InceptionBlock is introduced as the information fusion module; InceptionBlock consists of 4 branches, including 3×3 deep convolution to avoid large convolution kernels from reducing the training speed, and decoupling a large convolution kernel Kh×Kw into two small convolution kernels 1×Kw and Kh×1, and finally using residual connection to ensure context information;

[0066] In high-level information fusion, after aggregating feature map information based on the P5 layer, a fast self-attention mechanism is used to perform information fusion. The self-attention mechanism has a good effect in capturing long-distance feature dependencies and ensuring context information.

[0067] Step 2.6: After aggregating information through two low-order and high-order modules, the information is passed to the interference module in the network. The interference module first fuses local information with global information. By element-wise multiplication, effective features can be strengthened and redundant features can be suppressed. Then, residual connections are used to ensure the propagation of contextual information. In order to ensure the consistency of the size of the feature map, the pooling layer and bilinear interpolation method are used to adjust the feature map. The specific calculation formula is as follows:

[0068]

[0069] Among them, Avg / Bilinear respectively represent average pooling downsampling and bilinear interpolation to scale feature maps of different sizes. represents the fully connected layer, X global represents the global feature, X local Represents local features, represents the addition of elements, Represents element-wise multiplication.

[0070] Step 2.7: The detection head uses a decoupled detection head and adopts anchor frame assisted training. During the training process, an additional anchor frame-based detection head will be added in the classifier and regressor respectively. The two different methods are learned in independent loss functions respectively, and the information trained by the auxiliary head is integrated into the branch without anchor frame. Finally, in the inference stage, the auxiliary head will not participate, which ensures that the network is based on the fact that there is no anchor frame, which helps to improve the detection accuracy.

[0071] The above decoupled detection head can finally output the four coordinates of the defect, the defect type and the confidence level. Then, the four coordinates are connected through the post-processing process of the network to obtain the following: Fig.12 The detection results shown are used to realize defect feature extraction and detection.

[0072] The network model is tested on the test set D3. Fig.12 The experimental results show that compared with the baseline network, the improved method has correspondingly improved the accuracy for the four defect types and achieved a higher detection level.

[0073] The embodiments are preferred implementations of the present invention, but the present invention is not limited to the above-mentioned implementations. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essential content of the present invention belong to the protection scope of the present invention.

Claims

1. A method for detecting defects in X-ray weld images based on deep learning, characterized in that: The process includes the following: Step 1: Create a weld defect detection dataset; Step 2: Construct an improved Gold-YOLO method for defect feature extraction and detection.

2. The X-ray weld image defect detection method based on deep learning according to claim 1 is characterized in that: The specific process of step 1 is as follows: Step 1.1: First, obtain the original weld image data and filter it to remove noise; Step 1.2: Crop the weld image, screen the cropped weld image, and mark n types of weld defects, and finally obtain a y*y size weld defect dataset containing n types; Step 1.3: Then divide the weld defect dataset into training set, validation set, and test set.

3. The X-ray weld image defect detection method based on deep learning according to claim 1 is characterized in that: The specific process of step 2 is as follows: Step 2.1: Use the training set as network input and input it into the improved Gold-YOLO backbone network model; Step 2.2: The Gold-YOLO backbone network uses the EfficientRep network, which is composed of the RepVGG module based on structural reparameterization and uses a multi-branch feature extraction method to enhance the network representation capability; Step 2.3: Use the method of structural reparameterization to make the branch network lightweight. During the inference process, delete the 1×1 convolution and residual connection in the network branch. During the training process, the backbone network uses RepBlock and RepConv modules, and only uses RepConv modules during the inference process. Step 2.4: The multi-scale features B2, B3, B4 and B5 obtained in step 2.3 are input into the feature aggregation module for feature fusion. The feature aggregation module is divided into a low-order feature aggregation module and a high-order feature aggregation module; Step 2.5: The feature aggregation graph is passed into the information fusion module. Two different methods are used in the information fusion of the low stage and the high stage. In the low stage, the latest InceptionBlock is introduced as the information fusion module; InceptionBlock consists of 4 branches, including 3×3 deep convolution to avoid large convolution kernels from reducing the training speed, and decoupling a large convolution kernel Kh×Kw into two small convolution kernels 1×Kw and Kh×1, and finally using residual connection to ensure context information; In high-level information fusion, after aggregating feature map information based on the P5 layer, a fast self-attention mechanism is used to perform information fusion; Step 2.6: After the information is aggregated through the two low-order and high-order modules, it is passed to the interference module. The interference module first fuses the local information with the global information. The effective features can be strengthened and the redundant features can be suppressed through element-wise multiplication. Then the residual connection is used to ensure the propagation of contextual information. Step 2.7: The detection head uses a decoupled detection head and adopts anchor frame assisted training; the decoupled detection head finally outputs the four coordinates of the defect, the defect type and the confidence, and then the four coordinates are connected through the post-processing process of the network to obtain the detection result.

4. The X-ray weld image defect detection method based on deep learning according to claim 3 is characterized in that: In step 2.4, the purpose of the low-order feature aggregation module is to obtain the texture and edge features of the underlying feature map and aggregate the features of the four feature layers output by the backbone network. The formula of the low-order feature aggregation module is as follows: F4,F5(H,W)=billinear([B4,B5]) F2(H,W)=average_pool(B2) F concat =Concat(F2,B3,F4,F5) Among them, billinear represents the bilinear interpolation method, H and W represent the size of the B3 layer feature map, average_pool represents the average pooling downsampling operation; F4 and F5 represent the feature maps of B4 and B5 respectively expanded by the bilinear interpolation method; F2 represents the B2 feature map processed by the average pooling layer; F concat Represents the concatenated feature map; Concat means the concatenation operation integrates four feature maps into one feature map.

5. The X-ray weld image defect detection method based on deep learning according to claim 3 is characterized in that: In step 2.4, the purpose of the high-order feature aggregation module is to obtain semantic information in the feature map and perform feature aggregation based on the P5 layer. The formula of the high-order feature aggregation module is as follows: P3,P4(H,W)=average_pool(P3,P4) Among them, H and W are the height and width of the feature map of the P5 layer; P3, P4, and P5 are high-order feature maps respectively; average_pool represents the average pooling downsampling operation.

6. The X-ray weld image defect detection method based on deep learning according to claim 3 is characterized in that: In step 2.6, in order to ensure the consistency of the feature map size, the pooling layer and bilinear interpolation method are used to adjust the feature map. The specific calculation formula is as follows: Among them, Avg / Bilinear respectively represent average pooling downsampling and bilinear interpolation to scale feature maps of different sizes. represents the fully connected layer, X global represents the global feature, X local Represents local features, represents the addition of elements, Represents element-wise multiplication.

Images