Weld joint DR image defect intelligent identification method based on local-global efficient modeling improved YOLOv5

Abstract

The invention discloses a weld DR image defect intelligent identification method based on local-global efficient modeling improved YOLOv5. According to the method, an iRMBConv efficient feature extraction module is designed, and the module combines the technical advantages of dynamic global modeling and static local information fusion. In the Backbone network, the iRMBConv module can fully extract the multi-scale defect features of the weld DR image, and the recognition precision of the defects of different scales can be improved. Besides, in the Neck network, a context sensing module CSM is designed, the characteristics of different receptive fields are obtained by adopting expansion convolution with different expansion rates, the characteristic information of the tiny defects in the weld DR image is effectively enhanced, and then the recognition precision of the model on the tiny defects is improved. The intelligent identification model has high identification precision and good robustness, is suitable for the industrial fields of aviation, aerospace, nuclear industry, special equipment and the like, and promotes the application and deployment of the weld DR image defect intelligent identification technology.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and digital radiographic imaging detection, and designs a weld digital radiographic image defect intelligent recognition technology, specifically a weld DR (Digital Radiography) image defect intelligent recognition method based on local-global efficient model building and improved YOLOv5. Background Art

[0002] As a basic workpiece connection method, welding is widely used in manufacturing, aerospace, nuclear industry and construction. Its quality has a direct and important impact on the performance of the structure, so it is very important to detect internal defects in welds. At present, the commonly used weld nondestructive testing (NDT) methods are radiographic testing (RT) and ultrasonic testing (UT). Among them, X-ray testing uses X-rays to penetrate the workpiece and form an image on the detector. It reveals the internal and external structure and defects of the inspected workpiece by detecting the absorption degree of X-rays. It is widely used in weld nondestructive testing. In the traditional film evaluation method, the defect evaluation of weld radiographic detection images mainly relies on subjective judgment by personnel with professional qualifications. This method is easily affected by subjective factors such as the eyesight and fatigue of the evaluator, resulting in inconsistency in the evaluation results. In addition, the efficiency of manual defect evaluation is low and it is difficult to meet the growing detection needs. Therefore, in order to reduce labor costs and improve detection efficiency, it is necessary to develop an intelligent recognition method for weld defect targets based on artificial intelligence.

[0003] With the rapid development of artificial intelligence technology, major breakthroughs have been made in the field of defect detection, which has promoted the development of a series of intelligent defect assessment methods based on machine learning. Traditional machine learning algorithms require manual design of feature vectors based on data distribution, making them only applicable to datasets with specific targets in specific environments. In addition, it remains a challenge to design effective image features for classification models in complex welding environments. Deep learning is a machine learning technology based on artificial neural networks. Defect recognition methods driven by deep learning can automatically extract deep features of defects from raw images, eliminating the complexity and inaccuracy associated with manually designed feature extraction methods. In addition, it can utilize a large number of images for end-to-end training, thereby enhancing the generalization ability and robustness of defect recognition models. In the field of weld defect recognition, this method has demonstrated excellent performance.

[0004] YOLOv5 is a lightweight network widely used in industrial target detection tasks. YOLOv5 mainly includes Backbone network, Neck network and Head network. The Backbone network adopts the CSPDarknet53 network structure, which is used to extract features of the input image. The Neck network includes the PANet module and the FPN module for further feature extraction and fusion. The Head network adopts a lightweight single-stage detector based on the Anchor Box design, which is responsible for generating the prediction box and category probability of target detection.

[0005] Defects in welds, such as pores, inclusions, cracks, lack of penetration, and lack of fusion, present multiple scale characteristics. The YOLOv5 target detection algorithm performs well in identifying larger scale weld defects and can meet basic detection needs. However, the detection of small scale defects faces the problem of missed detection. Therefore, it is necessary to optimize the existing YOLOv5 network model to improve the accurate recognition of multi-scale weld defects. Summary of the invention

[0006] In view of the difficulty of multi-scale defect recognition in welds in the above-mentioned prior art, the present invention provides a weld DR image defect intelligent recognition method based on the local-global efficient model building and improved YOLOv5, constructs the iRMB_Conv module in the Backbone network for dynamic global feature modeling and static local feature fusion, and constructs the context-aware module CSM in the Neck network, which can effectively enhance the feature information of small-scale defects, thereby helping to improve the accuracy of multi-scale defect recognition. The present invention has highly accurate defect recognition capability and recall rate, and provides strong support for the development of intelligent recognition technology for weld defects.

[0007] In order to solve the above technical problems, the present invention proposes a weld DR image defect intelligent recognition method based on local-global efficient model building to improve YOLOv5, including the following steps:

[0008] Step 1: Collect DR images of welds containing different types of weld defects and mark the weld defects;

[0009] Step 2: Enhance the labeled data by rotation, mirroring, and cropping, and randomly divide the enhanced labeled data into training set, validation set, and test set in a ratio of 8:1:1;

[0010] Step 3: On the Pytorch platform, build a weld DR image defect intelligent recognition network model based on the improved YOLOv5 with local-global efficient model building;

[0011] Step 4: Input the data of the training set into the model constructed in step 3 for training, and use the validation set to verify the model; after each Epoch, calculate the loss and update the model parameters through back propagation; the training stops when the preset Epoch number is reached, and the trained network model is saved; and the generalization performance of the trained network model is evaluated using the test set;

[0012] Step 5: Input the DR image of the weld to be identified into the network model, output the detection result through model reasoning, and realize intelligent identification of weld defects.

[0013] Furthermore, the weld DR image defect intelligent identification method of the present invention comprises:

[0014] The specific contents of step 1 include:

[0015] 1-1) Carry out weld DR experiment according to NBT 47013.11-2015 standard, and the sensitivity value, resolution, gray value and normalized signal-to-noise ratio of weld DR image shall at least meet the requirements of AB level detection technology level;

[0016] 1-2) Use Image J image processing software to adjust the window width and window position of the weld DR image, and export the image in JPG format;

[0017] 1-3) Use labelme image annotation tool to annotate 5 types of weld defects in JPG format weld DR image, and generate json format annotation file; the 5 types of weld defects are pores, inclusions, cracks, lack of penetration and lack of fusion, and PO, SL, CRK, LP and LF are used to represent the weld defect categories of pores, inclusions, cracks, lack of penetration and lack of fusion;

[0018] 1-4) Convert the json format annotation file into xml format.

[0019] In step 2, when enhancing the annotated data, the angles of rotation enhancement are 90°, 180° and 270°; the mirror enhancement includes horizontal mirroring and vertical mirroring; the cropping enhancement is centered on the defect center, and the cropping size is randomly selected within the range of 900-1200 pixels; the rotation enhancement and mirror enhancement methods are applied in a random manner, and the cropping enhancement method is a mandatory application method.

[0020] The specific contents of step 3 include:

[0021] 3-1) Construct an iRMB_Conv module for feature extraction in the Backbone network. This module is implemented by connecting M iRMB modules in series with a 3×3 convolution layer. The core components of the iRMB module include the Expanded Window Multi-Head Self-Attention (EW-MHSA) module, the Depth-Wise Convolution (DW-Conv) with integrated short connections, a 1×1 convolution layer, and a residual connection. In the EW-MHSA module, a prerequisite judgment step is first performed to determine whether to perform the Self-Attention (Self-Attention) operation. If it is determined that the Self-Attention operation is not to be performed, the input features are directly passed through a channel-expanded 1×1 convolution operation, and the convolution result is used as the output of the EW-MHSA module. Conversely, if the Self-Attention operation is required, the input features of the EW-MHSA module are first subjected to preliminary convolution processing. Subsequently, the convolved features are split into matrices K and Q through the Split operator. Next, a dot product operation is performed on K and Q, and the Softmax function is applied to obtain the attention weight (AttentionMap). Finally, the attention weight is weighted with the original input features of the EW-MHSA module, and a 1×1 convolution operation with channel expansion is performed. The result is the output feature of the EW-MHSA module. The specific expression of the iRMB module is:

[0022] Y 1 =EW-MHSA(Y)

[0023] Y 2 =DW-Conv(Y 1 )+Y 1

[0024]

[0025] Among them, Y is the input feature of the iRMB module, is the output feature of the iRMB module, acting on Y 2 The above is a 1×1 convolutional layer with channel contraction;

[0026] 3-2) Instantiate multiple iRMB_Conv modules through input channels, prerequisite judgment settings, number of multi-head attention, iRMB module depth M, output channels, and DW-Conv convolution kernels; 4 to 5 instantiated iRMB_Conv modules can be included in the Backbone network for feature extraction;

[0027] Preferably, the number of instantiations is 4, the input channels of the 4 iRMB_Conv modules are 24, 32, 48 and 120 respectively, the prerequisite judgment settings are No, No, Yes and Yes respectively, the number of multi-head attentions is 16, 16, 20 and 20 respectively, the iRMB module depth M is 3, 3, 9 and 3 respectively, the output channels of the 4 iRMB_Conv modules are 32, 48, 120 and 200 respectively, and the depth separation convolution DW-Conv convolution kernel sizes are 3, 3, 5 and 5 respectively; further, the output channel of the 3×3 convolution layer in the iRMB_Conv module is the same as the output channel of the module, which is set to 32, 48, 120 and 200 respectively.

[0028] 3-3) Constructing a context-aware module CSM, including extracting multi-scale features F through three convolutional layers with different dilation rates 1 、F 2 and F 3 ; Use 1×1 convolution to adjust its dimension and get F′ 1 , F′ 2 and F′ 3 , and transform F′ along the channel dimension 1 , F′ 2 and F′ 3 The concatenated features are concatenated into a new feature; the concatenated features are normalized by 1×1 convolution and Softmax operations, and are split into three parts in the channel dimension as multi-scale features F 1 、F 2 and F 3 Finally, the multi-scale feature F 1 、F 2 and F 3 The context fusion feature finally obtained by multiplying the features with their respective weights and then adding them is the output feature of the context perception module;

[0029] 3-4) Replace the 13th layer C3 module in the Neck part with the above-mentioned context-aware module CSM, so as to provide fully integrated feature information for the defect classification and bounding box regression tasks in the downstream Head part.

[0030] The specific contents of step 4 include:

[0031] 4-1) Set hyperparameters, including: batch size is 4, optimizer is stochastic gradient descent SGD, initial learning rate is 0.01, total training cycle epoch is 200, input image size is 640×640;

[0032] 4-2) In the model training phase, the training set data is input into the model, and the weights in the model are adjusted through the back-propagation and gradient descent optimization algorithms, so that the model can learn useful features and patterns from the data;

[0033] 4-3) The validation set is used to evaluate the performance of the model and adjust the hyperparameters during the training process; at the end of each Epoch, the model will be evaluated on the validation set to determine whether the model is overfitting or underfitting, so as to select the best hyperparameters.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] (1) The present invention improves YOLOv5 and designs an efficient feature extraction module iRMB_Conv in the Backbone network. The module takes into account the advantages of dynamic global modeling and static local information fusion, and can effectively increase the receptive field of the model.

[0036] (2) The Backbone network constructed based on the iRMB_Conv module in the present invention can make full use of the excellent processing ability of convolutional neural networks in the early stages of visual tasks and the advantages of Transformer in extracting long-distance features, fully extract the multi-scale defect features of welds, and is beneficial to the intelligent identification of multi-scale defects.

[0037] (3) In the Neck network, the present invention designs a context-aware module CSM, obtains features of different receptive fields through dilated convolutions with different dilation rates, uses cascade operations to obtain spatial adaptive weights, and obtains context information features through weighted summation.

[0038] (4) The present invention introduces the CSM module into the deep feature fusion part of the Neck network, which enhances the ability to capture the characteristic information of tiny defects in the weld, thereby improving the accuracy of the intelligent recognition algorithm for tiny defects in the weld. Therefore, the present invention improves the recognition accuracy of multi-scale defects as a whole.

[0039] (5) The present invention uses deep learning technology to realize intelligent recognition of weld defects. The designed recognition model can effectively extract multi-scale defect features and enhance the characteristics of small defects, significantly reducing the missed detection rate. Therefore, this model has high recognition accuracy and strong robustness, providing reliable technical support for intelligent recognition technology of weld defects in industrial fields such as aviation, aerospace, nuclear industry, and special equipment, and promoting the application and implementation of related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a network structure diagram for realizing the weld DR image defect intelligent recognition method of the present invention based on the local-global efficient model building to improve YOLOv5;

[0041] Figure 2 It is a schematic diagram of the structure of the iRMB_Conv module used for feature extraction in the present invention;

[0042] Figure 3 It is a schematic diagram of the structure of a context-aware module CSM module used for feature enhancement in the present invention;

[0043] Figure 4(a) to Figure 4(i) It is the training result diagram of the present invention:

[0044] Figure 5-1 and Figure 5-2 All of them are defect recognition result diagrams of the present invention, where:

[0045] Figure 5-1 Among them, (a) is the defect recognition result diagram of YOLOv5, (b) is the defect recognition result diagram of YOLOv7, (c) is the defect recognition result diagram of YOLOv8, (d) is the defect recognition result diagram of YOLOv9, and (e) is the defect recognition result diagram of YOLOv5 improved by the model of the present invention;

[0046] Figure 5-2 To express clearly Figure 5-1 The relevant data in Figure 5-1 A local enlarged schematic diagram of the dotted line area shown in FIG. DETAILED DESCRIPTION

[0047] The design concept of the weld DR image defect intelligent recognition method based on the local-global efficient model building and improved YOLOv5 proposed in the present invention is: the method designs an iRMB_Conv efficient feature extraction module, which combines the technical advantages of dynamic global modeling and static local information fusion. In the Backbone network, the iRMB_Conv module can fully extract the multi-scale defect features of the weld DR image, which helps to improve the recognition accuracy of defects of different scales. In addition, in the Neck network, the present invention designs a context-aware module CSM, which uses dilated convolutions with different dilation rates to obtain features of different receptive fields, effectively enhancing the feature information of tiny defects in the weld DR image, thereby improving the model's recognition accuracy for tiny defects. The intelligent recognition model has high recognition accuracy and good robustness, and is suitable for industrial fields such as aviation, aerospace, nuclear industry, and special equipment, and promotes the application and deployment of intelligent recognition technology for weld DR image defects.

[0048] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but the following embodiments are by no means intended to limit the present invention in any way.

[0049] Example

[0050] The present invention proposes a weld DR image defect intelligent recognition method based on local-global efficient model building to improve YOLOv5, comprising the following steps:

[0051] Step 1: Data collection: Collect DR images of welds with different types of defects and mark the defects;

[0052] Step 2: Dataset creation: The labeled data is enhanced by rotation, mirroring, and cropping, and the enhanced labeled data is randomly divided into training set, validation set, and test set in a ratio of 8:1:1;

[0053] Step 3: Model construction: On the open source deep learning framework Pytorch platform, a weld defect intelligent recognition network model based on the improved YOLOv5 with local-global efficient model building is constructed;

[0054] Step 4, model training: input the training set data into the model built in step 3 for training, and use the validation set to validate the model. After each epoch (the model completes a complete traversal of the entire training data set), calculate the loss and update the model parameters through back propagation. Training stops when the preset number of epochs is reached, and finally saves the network model with the best performance;

[0055] Step 5: Model testing and reasoning: Use the test set to evaluate the generalization performance of the network model saved in step 4. The main purpose of model reasoning is to apply the network model to weld DR images in actual engineering for intelligent defect recognition.

[0056] The above steps are described as follows:

[0057] Step 1: The specific steps of data collection are as follows:

[0058] 1-1) Carry out weld DR test according to NBT 47013.11-2015 (Nondestructive testing of pressure equipment Part 11: X-ray digital imaging testing) standard. The sensitivity value, resolution, gray value and normalized signal-to-noise ratio of the weld DR image shall at least meet the requirements of AB level detection technology level;

[0059] 1-2) Use Image J (image processing) software to adjust the window width and window position of the weld DR image collected in the experiment, and export the image in JPG format;

[0060] 1-3) Labelme (image annotation tool) is used to annotate the five defects in the weld DR image in JPG format to generate a json file corresponding to the DR image. The five defects are pores, inclusions, cracks, lack of penetration and lack of fusion. PO, SL, CRK, LP and LF are used to represent the defect categories of pores, inclusions, cracks, lack of penetration and lack of fusion.

[0061] 1-4) Convert the json format annotation file generated by labelme into xml format to conform to the annotation file specification of the VOC dataset.

[0062] Step 2: Dataset preparation includes:

[0063] 2-1) Rotation Enhancement Randomly select a rotation angle from a predetermined rotation angle set for data enhancement, the rotation angle set includes 90°, 180° and 270°; rotate the DR image to be enhanced according to the selected rotation angle, and save the rotated image as a new DR image; in the rotated new image, calculate the coordinate value of the defect, and update the coordinate information in the xml annotation file corresponding to the DR image to be enhanced based on the coordinate value to generate a new xml annotation file corresponding to the new image;

[0064] 2-2) Mirror enhancement includes horizontal mirroring and vertical mirroring; randomly selecting a mirroring method to perform a mirroring operation on the image to be enhanced, and saving it as a new DR image; in the new image after the mirroring operation, calculating the coordinate value of the defect, and based on the coordinate value, updating the coordinate information in the xml annotation file corresponding to the DR image to be enhanced, so as to generate a new xml annotation file corresponding to the new image;

[0065] 2-3) Cropping enhancement is centered on the defect center, and the cropping size is randomly selected within the range of 900-1200 pixels. The DR image to be enhanced is cropped, and the cropped image is saved as a new DR image; the coordinate value of the defect is calculated in the cropped new image, and based on the coordinate value, the coordinate information in the XML annotation file corresponding to the DR image to be enhanced is updated to generate a new XML annotation file corresponding to the new image;

[0066] 2-4) The rotation enhancement and mirror enhancement methods are applied randomly, with a probability of 60% of random application, and the cropping enhancement method is a mandatory application method; the final data enhancement methods include four combinations of cropping enhancement, random rotation enhancement combined with cropping enhancement, random mirror enhancement combined with cropping enhancement, and random rotation enhancement combined with random mirror enhancement and cropping enhancement; the DR images of the original experiment and their corresponding xml annotation files are processed by the above four data enhancement methods in turn to generate the final enhanced data;

[0067] 2-5) The enhanced VOC format dataset is randomly divided into training set, validation set and test set in a ratio of 8:1:1, and the data file names of each part are saved in train.txt, val.txt and test.txt files respectively. The weld defect categories, labels and number of defects in each category in the dataset are shown in Table 1.

[0068] Table 1

[0069]

[0070]

[0071] Step 3: Improve the weld defect intelligent recognition network model of YOLOv5 based on local-global efficient model building, such as Figure 1 As shown, where N defect is the number of defect categories, N defect =5, specifically including the following steps:

[0072] 3-1) Construct the iRMB_Conv module for feature extraction in the Backbone network, such as Figure 2 As shown in Figure 1, this module is implemented by connecting M iRMB modules in series with a 3×3 convolution layer. The core components of the iRMB module include the Expanded Window Multi-Head Self-Attention (EW-MHSA) module, the Depth-Wise Convolution (DW-Conv) with integrated short connections, the 1×1 convolution layer, and the residual connection. In the EW-MHSA module, a prerequisite judgment step is first performed to determine whether to perform the Self-Attention (Self-Attention) operation. If the Self-Attention operation is determined not to be performed, the input features are directly passed through a channel-expanded 1×1 convolution operation, and the convolution result is used as the output of the EW-MHSA module. Conversely, if the Self-Attention operation is required, the input features of the EW-MHSA module are first subjected to preliminary convolution processing. Subsequently, the convolved features are split into matrices K and Q through the Split operator. Next, a dot product operation is performed on K and Q, and the Softmax function is applied to obtain the attention weight (Attention Map). Finally, the attention weights are weighted with the original input features of the EW-MHSA module, and then a 1×1 convolution operation with channel expansion is performed. The result is the output feature of the EW-MHSA module. The specific expression of the iRMB module is:

[0073] Y 1 =EW-MHSA(Y)

[0074] Y 2 =DW-Conv(Y 1 )+Y 1

[0075]

[0076] Among them, Y is the input feature of the iRMB module, is the output feature of the iRMB module, acting on Y 2 The above is a 1×1 convolutional layer with channel contraction;

[0077] 3-2) Instantiate multiple iRMB_Conv modules through input channels, prerequisite judgment settings, number of multi-head attention, iRMB module depth M, output channels and DW-Conv convolution kernels; specifically, instantiate 4 iRMB_Conv modules, whose input channels are 24, 32, 48 and 120 respectively, the prerequisite judgment settings are No, No, Yes and Yes respectively, the number of multi-head attention is 16, 16, 20 and 20 respectively, the iRMB module depth M is 3, 3, 9 and 3 respectively, the output channels are 32, 48, 120 and 200 respectively, and the DW-Conv convolution kernel size is 3, 3, 5 and 5 respectively; further, the output channel of the 3×3 convolution layer in the iRMB_Conv module is the same as the output channel of the module, which is set to 32, 48, 120 and 200 respectively; the instantiated 4 iRMB_Conv modules are used for feature extraction in the Backbone network, such as Figure 1 As shown;

[0078] 3-3) Construct a context-aware module (CSM), such as Figure 3 First, three convolutional layers with different dilation rates are used to extract multi-scale features F. 1 、F 2 and F 3 , and then use 1×1 convolution to adjust its dimensions to get F 1 ′、F 2 ′ and F 3 ′, and along the channel dimension direction F 1 ′、F 2 ′ and F 3 ′ is concatenated into a new feature, where the three expansion rates are set to 1, 3, and 5 respectively. Next, the concatenated features are normalized by 1×1 convolution and Softmax operations, and the Split operator is used to split the normalized weights into W in the channel dimension. 1 , W 2 and W 3 , and are taken as multi-scale features F 1、F 2 and F 3 Finally, these features are multiplied by their respective weights and added together to obtain the final context fusion feature, which is W 1 ·F 1 +W 2 ·F 2 +W 3 ·F 3 ;

[0079] 3-4) Replace the 13th layer C3 module in the Neck part of the YOLOv5 model with the context-aware module CSM to fully mine the contextual information of weld DR image defects and provide fully integrated feature information for the defect classification and bounding box regression tasks in the downstream Head part.

[0080] Step 4: Model training includes:

[0081] 4-1) Model hyperparameter settings, including batch size of 4, optimizer of stochastic gradient descent SGD, initial learning rate of 0.01, total training cycle epoch set to 200, and input image size of 640×640;

[0082] 4-2) In the model training phase, the training set data is input into the model, and the weights in the model are adjusted through the back-propagation and gradient descent optimization algorithms, so that the model can learn useful features and patterns from the data;

[0083] 4-3) The validation set is used to evaluate the performance of the model and adjust the hyperparameters during the training process. At the end of each Epoch, the model is evaluated on the validation set to determine whether the model is overfitting or underfitting and to help select the best hyperparameters.

[0084] In this embodiment, the weld defect intelligent recognition network model based on the improved YOLOv5 of the local-global efficient model building is trained by using the training set. The training results are as follows: Figure 4(a) to Figure 4(i)As shown, Figure 4(a) is the bounding box regression loss curve of the 0-200Epoch training set; Figure 4(b) is the target confidence loss curve of the 0-200Epoch training set; Figure 4(c) is the classification loss curve of the 0-200Epoch training set; Figure 4(d) is the bounding box regression loss curve of the 0-200Epoch validation set; Figure 4(e) is the target confidence loss curve of the 0-200Epoch validation set; Figure 4(f) is the classification loss curve of the 0-200Epoch validation set; Figure 4(g) is the average accuracy (mAP@0.5) curve of the 0-200Epoch model; Figure 4(h) is the precision (Precision) curve of the 0-200Epoch model; Figure 4(i) is the recall (Recall) curve of the 0-200Epoch model.

[0085] In this embodiment, the improved YOLOv5 weld defect intelligent recognition network model based on local-global efficient construction is trained by using the training set.

[0086] The training set is used to train YOLOv5, YOLOv7, YOLOv8 and YOLOv9 respectively, and the models are verified in the validation set. The comparison of experimental results of different models is shown in Table 2.

[0087] Table 2

[0088] Model mAP@0.5(%) Precise(%) Recall(%) YOLOv5 86.5 87.5 83.5 YOLOv7 54.5 56.4 58.7 YOLOv8 76.7 82.8 68.5 YOLOv9 81.3 82.8 74.1 Model of the present invention 91.5 88.3 87.3

[0089] As shown in Table 2, the average accuracy (mAP@0.5), precision (Precision) and recall (Recall) of weld DR image defect recognition of the optimized model of YOLOv5 of the present invention are 91.5%, 88.3% and 87.3%, respectively, which is significantly better than other models. In particular, compared with the original YOLOv5 model, the average accuracy (mAP@0.5), precision (Precision) and recall (Recall) of the optimized model of the present invention are increased by 5.0%, 0.8% and 3.8%, respectively.

[0090] Step 5: Model testing and reasoning mainly include:

[0091] 5-1) Evaluate the generalization performance of the network model saved in step 4 by using the test set;

[0092] 5-2) The main purpose of model reasoning is to apply the network model to weld DR images in actual engineering for intelligent defect recognition;

[0093] 5-3) Input a weld DR image to be identified into YOLOv5, YOLOv7, YOLOv8, YOLOv9 and the model of the present invention respectively, and output the detection results, such as Figure 5-1and Figure 5-2 As shown in the figure, it can be seen that the YOLOv5 and YOLOv7 models misclassify the pore (PO) defect on the far left of the weld as an inclusion (SL) defect, the YOLOv8 model misses the pore (PO) defect on the far left, and the YOLOv9 model misclassifies the low-density inclusion (SL) defect in the middle of the weld as a pore (PO) defect. The YOLOv5 optimization model proposed in the present invention can effectively identify the defects in the weld of the DR image, and the defect classification is accurate.

[0094] Although the present invention has been described above in conjunction with the accompanying drawings, the present invention is not limited to the above-mentioned specific embodiments, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can make many improvements and changes without departing from the purpose of the present invention, which are all within the protection of the present invention.

Claims

1. A weld DR image defect intelligent recognition method based on local-global efficient model building to improve YOLOv5, comprising the following steps: Step 1: Collect DR images of welds containing different types of weld defects and mark the weld defects; Step 2: Enhance the labeled data by rotation, mirroring, and cropping, and randomly divide the enhanced labeled data into training set, validation set, and test set in a ratio of 8:1:1; Step 3: On the Pytorch platform, build a weld DR image defect intelligent recognition network model based on the improved YOLOv5 with local-global efficient model building; Step 4: Input the data of the training set into the model constructed in step 3 for training, and use the validation set to verify the model; after each Epoch, calculate the loss and update the model parameters through back propagation; the training stops when the preset Epoch number is reached, and the trained network model is saved; and the generalization performance of the trained network model is evaluated using the test set; Step 5: Input the DR image of the weld to be identified into the network model, output the detection result through model reasoning, and realize intelligent identification of weld defects.

2. The method for intelligently identifying weld DR image defects according to claim 1, characterized in that: The specific contents of step 1 include: 1-1) Carry out weld DR experiment according to NBT 47013.11-2015 standard, and the sensitivity value, resolution, gray value and normalized signal-to-noise ratio of weld DR image shall at least meet the requirements of AB level detection technology level; 1-2) Use Image J image processing software to adjust the window width and window position of the weld DR image, and export the image in JPG format; 1-3) Use labelme image annotation tool to annotate 5 types of weld defects in JPG format weld DR image, and generate json format annotation file; the 5 types of weld defects are pores, inclusions, cracks, lack of penetration and lack of fusion, and PO, SL, CRK, LP and LF are used to represent the weld defect categories of pores, inclusions, cracks, lack of penetration and lack of fusion; 1-4) Convert the json format annotation file into xml format.

3. The method for intelligently identifying weld DR image defects according to claim 1, characterized in that: In step 2, the annotation data is enhanced, and the rotation enhancement angles are 90°, 180°, and 270°; the mirror enhancement includes horizontal mirroring and vertical mirroring; the cropping enhancement is centered on the defect center, and the cropping size is randomly selected within the range of 900-1200 pixels; Among them, the rotation enhancement and mirror enhancement methods are applied in a random manner, and the cropping enhancement method is an application method.

4. The method for intelligently identifying weld DR image defects according to claim 1, characterized in that: The specific contents of step 3 include: 3-1) Constructing an iRMB_Conv module for feature extraction in the Backbone network, which is implemented by connecting M iRMB modules in series with a 3×3 convolutional layer; the core components of the iRMB module include an extended window multi-head self-attention EW-MHSA module, a deep separation convolution DW-Conv with integrated short connections, a 1×1 convolutional layer, and a residual connection; in the EW-MHSA module, a prerequisite judgment step is first performed to determine whether to perform a self-attention operation; If it is determined that the Self-Attention operation is not to be performed, the input features are directly subjected to a 1×1 convolution operation with a channel expansion, and the convolution result is used as the output of the EW-MHSA module; conversely, if the Self-Attention operation is to be performed, the input features of the EW-MHSA module are first subjected to preliminary convolution processing; then, the convolved features are split into a matrix K and a matrix Q through the Split operator; then, a dot product operation is performed on K and Q, and the Softmax function is applied to obtain the attention weight; finally, the attention weight is weighted with the original input features of the EW-MHSA module, and a 1×1 convolution operation with a channel expansion is performed, and the result is the output feature of the EW-MHSA module; the specific expression of the iRMB module is: Y1=EW-MHSA(Y) Y2=DW-Conv(Y1)+Y1 Among them, Y is the input feature of the iRMB module, is the output feature of the iRMB module, and a 1×1 convolutional layer with channel contraction is applied to Y2; 3-2) Instantiate multiple iRMB_Conv modules through input channels, prerequisite judgment settings, number of multi-head attention, iRMB module depth M, output channels, and DW-Conv convolution kernels; include 4 to 5 instantiated iRMB_Conv modules in the Backbone network for feature extraction; 3-3) Constructing a context-aware module CSM, including extracting multi-scale features F1, F2 and F3 through three convolutional layers with different expansion rates; adjusting their dimensions using 1×1 convolution respectively to obtain F′1, F′2 and F′3, and concatenating F′1, F′2 and F′3 into a new feature along the channel dimension; generating normalized weights for the concatenated features through 1×1 convolution and Softmax operations, and splitting them into three parts in the channel dimension, which are used as adaptive weights of the multi-scale features F1, F2 and F3 respectively; finally, multiplying the multi-scale features F1, F2 and F3 by their respective weights and then adding them together, and the final context fusion feature is the output feature of the context-aware module CSM; 3-4) Replace the 13th layer C3 module in the Neck part with the above-mentioned context-aware module CSM, so as to provide fully integrated feature information for the defect classification and bounding box regression tasks in the downstream Head part.

5. The method for intelligently identifying weld DR image defects according to claim 4 is characterized in that: In step 3-2), the Backbone network includes 4 instantiated iRMB_Conv modules; the input channels of the 4 iRMB_Conv modules are 24, 32, 48 and 120, respectively; the prerequisite judgment settings are No, No, Yes and Yes, respectively; the number of multi-head attention is 16, 16, 20 and 20, respectively; the iRMB module depth M is 3, 3, 9 and 3, respectively; the output channels of the 4 iRMB_Conv modules are 32, 48, 120 and 200, respectively; the convolution kernel sizes of the deep separation convolution DW-Conv are 3, 3, 5 and 5, respectively; The output channels of the 3×3 convolutional layer in the iRMB_Conv module are the same as the output channels of the module, which are set to 32, 48, 120 and 200 respectively.

6. The method for intelligently identifying weld DR image defects according to claim 1, characterized in that: The specific contents of step 4 include: 4-1) Set hyperparameters, including: batch size is 4, optimizer is stochastic gradient descent SGD, initial learning rate is 0.01, total training cycle epoch is 200, input image size is 640×640; 4-2) In the model training phase, the training set data is input into the model, and the weights in the model are adjusted through the back-propagation and gradient descent optimization algorithms, so that the model can learn useful features and patterns from the data; 4-3) The validation set is used to evaluate the performance of the model and adjust the hyperparameters during the training process; at the end of each Epoch, the model will be evaluated on the validation set to determine whether the model is overfitting or underfitting, so as to select the best hyperparameters.

Images