Electric arc additive manufacturing microstructure defect detection method based on YOLOv5 improvement

Abstract

The invention provides an electric arc additive manufacturing microstructure defect detection method based on YOLOv5 improvement. The electric arc additive manufacturing microstructure defect detection method comprises the following steps: step 1, establishing a test data set; step 2, constructing a neural network model: adjusting and configuring corresponding hyper-parameters, establishing a neural network model of YOLOv5, and improving the neural network model of YOLOv5; step 3, training to optimize a neural network model of YOLOv5; the YOLOv5 neural network model is trained each time, and a corresponding weight file is generated; and step 4, outputting a test result: storing the trained weight file in the network model of the YOLOv5, testing the detection speed of the YOLOv5 in a prediction program based on the trained weight file, and outputting the test result. The target object can be detected and positioned more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning target detection, and in particular to an arc additive manufacturing microstructure defect detection method based on improved YOLOv5. Background Art

[0002] WAAM is a metal additive manufacturing technology based on arc welding. Compared with other metal additive manufacturing technologies, it has high material utilization and low manufacturing cost, and has broad application prospects in aerospace, automobile manufacturing, shipbuilding, energy and other fields. However, since WAAM involves complex heat and mass transfer processes in multi-layer stacking and electric deposition, WAAM parts often have surface and microscopic defects (such as surface pores, spatter, weld nodules, and internal microscopic pores, cracks, unfused and slag inclusions, etc.). These defects will lead to a decrease in the mechanical properties of the parts, affect their reliability and service life, and even cause the parts to be scrapped. Rapid and accurate detection and identification of surface and microscopic defects, as well as characterization and analysis, are important steps for subsequent optimization and improvement of process parameters and suppression and elimination of defects.

[0003] Traditional WAAM parts macro defect detection methods mainly include visual inspection and machine vision inspection, while internal micro structural defect detection methods mainly include ultrasonic phased array detection, eddy current detection, radiographic detection, metallographic image detection, etc. The above macro and micro defect detection methods based on traditional machine vision and professional equipment only provide a means of defect extraction or imaging, and the characterization and identification of defects mainly rely on manual experience, which greatly reduces the accuracy and efficiency of identification.

[0004] With the innovation of image processing technology and the advancement of deep learning technology, the target defect detection method combining image recognition and deep learning has become an effective technical approach. Through the systematic training of the deep learning model, this method can automatically identify and extract defect information and accurately identify the defect type, thereby achieving rapid and accurate detection of macro and micro defects in the WAAM process.

[0005] At present, there are RCNN, SSD, YOLO and other methods based on deep learning in the field of target detection. Among them, YOLO is the most widely used. By modifying its attention mechanism and loss function, it can make up for its lack of precision and accuracy in small target detection. Therefore, a method for arc additive microstructure defect detection based on the improved YOLOv5 algorithm is proposed. Summary of the invention

[0006] In order to solve the problem that arc additive manufacturing microstructure defects rely on manual experience and have low recognition accuracy and efficiency, a method for arc additive manufacturing microstructure defect detection based on an improved YOLOv5 algorithm is proposed.

[0007] In order to achieve the above technical features, the purpose of the present invention is to achieve: a method for detecting microstructure defects in arc additive manufacturing based on improved YOLOv5, comprising the following steps: Step 1: Create a test dataset: Collect arc additive manufacturing microstructure defect image data, and then perform data enhancement on different types of pore defect images through geometric transformation to obtain a test data set consisting of images to be tested; Step 2: Build a neural network model: Adjust and configure the corresponding hyperparameters, build the YOLOv5 neural network model, and improve the YOLOv5 neural network model; Step 3: Train to optimize the YOLOv5 neural network model: After data augmentation, the images of different types of pore defects are input into the YOLOv5 neural network model, and the YOLOv5 neural network model is trained to optimize the YOLOv5 neural network model and generate images; each time the YOLOv5 neural network model is trained, a corresponding weight file is generated; Step 4, output of test results: Save the trained weight file to the YOLOv5 network model. Then, based on the trained weight file, test the detection speed of YOLOv5 in the prediction program and output the test results.

[0008] In the step 2, the specific process of improving the neural network model of YOLOv5 is: replacing the original CSPDarkNet53 backbone network with a lightweight backbone network; replacing the original PANet with DPAN; and replacing the Conv module and C3 module in the neck network with the GhostConv module and the GC3ECA module.

[0009] The lightweight backbone network consists of GhostConv module, PConv module and ESPP module, which optimizes the number of parameters and GFLOPs value of the model; Conv in YOLOv5 is replaced by GhostConv, which reduces the number of parameters, FLOPs value and model size, and improves the inference speed; The C3 module in the backbone network was replaced with the GC3ECA module, and the network depth of the GC3ECA module in the model was increased to ensure the accuracy of the model; The SPPF module is replaced by the ESPP module to enhance the extraction of feature layer information.

[0010] The ESPP module retains the basic framework of SPPCSPC and introduces the PConv module at the front end of the spatial pyramid pooling input.

[0011] Change the SPP structure in the SPPSPC structure to an SPPF structure.

[0012] DPAN adds additional cross-layer connections and PAN networks on the basis of PANet to achieve extended-layer fusion of feature information.

[0013] The YOLOv5 neck network first receives information from the backbone network and performs up and down sampling operations through PANet to obtain three temporary feature layers. Through cross-layer connections, the feature information from the previous convolutional layer is fused with the temporary feature layer; then, the merged feature layer is subsequently passed to the new PAN layer to implement two-cycle PAN operations; through the above operations, DPAN is able to better aggregate more features and retain detailed information in the low-resolution feature layer.

[0014] The C3 module is replaced by the GC3ECA module as follows: By combining the ECA attention mechanism, the BottleNeck in the C3 module is improved to ECA_BottleNeck to enhance the module's perception of channel information; then, the Conv module at the input end of the C3 module is replaced by the GhostConv module.

[0015] The present invention has the following beneficial effects: 1. Compared with other YOLO algorithms, the improved YOLOv5 of the present invention has improved the accuracy of object detection and can detect and locate target objects more accurately.

[0016] 2. Although the accuracy of the present invention has been improved, the improved Yolov5 can still maintain a faster detection speed while ensuring higher accuracy, which is very important for real-time applications.

[0017] 3. Compared with the YOLOv5 and YOLOv4 algorithms, the improved YOLOv8 of the present invention has a smaller model and can be applied in many places. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0019] Figure 1 Schematic diagram showing the improved YOLOv5 model in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the network structure of ESPP in an embodiment of the present invention; (a) shows the schematic diagram of the structure of SPPCSP; (b) shows the schematic diagram of the structure of ESPP.

[0021] Figure 3 A schematic diagram of the PANet and DPAN network structures in an embodiment of the present invention; (a) shows the PAN structure; (b) shows the dotted and solid lines that form a loop between the feature map and the PAN network; (c) shows the structural diagram of the feature extraction of PAN.

[0022] Figure 4 A schematic diagram of a GC3ECA network structure in an embodiment of the present invention; (a) C3 structure; (b) Bottleneck structure in C3 structure with residual connections when add is True and no residual connections when False; (c) Improved GC3ECA structure; (d) ECA_BottleNeck structure in GC3ECA. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0024] Embodiment 1: Reference Figure 1 , an improved arc additive manufacturing microstructure defect detection method based on YOLOv5 includes the following steps: Step 1: Establish a test data set: collect arc additive manufacturing microstructure defect image data, and then enhance the image data of two types of defect images, pores and unfused pores, through geometric transformation, so as to obtain a test data set consisting of 5,000 images to be tested; Step 2: Build a neural network model: adjust and configure the corresponding hyperparameters to build the YOLOv5 neural network model, and improve the YOLOv5 neural network model; Step 3: training to optimize the network model of YOLOv5: inputting two data-enhanced arc additive manufacturing microstructure defect images of pores and unfused pores into the network model of YOLOv5, training the network model of YOLOv5 to optimize the network model of YOLOv5 and generate detection images; the network model of YOLOv5 is trained, and a corresponding weight file is generated each time the training is performed; Step 4: Save the trained weight file to the YOLOv5 network model. Then, based on the trained weight file, test the detection speed of YOLOv5 in the prediction program and output the prediction results.

[0025] As a preferred embodiment of the present application, in step 2, the specific process of improving the neural network model of YOLOv5 is as follows: the original CSPDarkNet53 network is replaced by a self-designed lightweight backbone network, and the original DPAN is replaced by a DPAN dual-path aggregation network; in the neck network, the Conv module is replaced by GhostConv, and the C3 module is replaced by a GC3ECA module.

[0026] In step 2: The lightweight backbone network optimizes the number of model parameters and GFLOPs value through the GhostConv module, PConv module and ESSP module, and improves the speed of extracting model feature information.

[0027] It should be noted that the GhostConv module first uses a small number of convolution kernels to extract features from the input feature map, then further performs a cheaper linear transformation φ on this part of the feature map, and finally generates the final feature map through a splicing operation. This method reduces the learning cost of non-critical features. By combining a small number of convolution kernels and cheaper linear change operations to replace the traditional convolution method, the demand for computing resources is effectively reduced without affecting the performance of the model. PConv uses a special memory access method to reduce computational complexity and memory overhead. Specifically, PConv applies continuous or regular memory access methods to convolution operations, and regards the first or last continuous channel as a representative of the entire feature map for calculation. This method not only reduces redundant calculations and memory accesses of neural networks, but also extracts spatial features more efficiently.

[0028] Furthermore, the ESPP module retains the basic framework of SPPCSPC while introducing the PConv module at the front end of the spatial pyramid pooling input, thereby reducing the number of model parameters and improving the inference speed. In addition, in order to optimize the SPPCSPC structure, the SPP structure is changed to the SPPF structure because the SPPF structure performs better in terms of accuracy and speed.

[0029] Furthermore, the DPAN network adds additional cross-layer connections on the basis of PANet, connects adjacent layers in the PAN layer through the DWConv module and the Concat module, and passes them to the subsequent PAN network to realize the re-fusion extraction of detail information in the feature information. Specifically, the neck network first receives information from the backbone network, and performs up and down sampling operations through PANet to obtain three temporary feature layers. Through cross-layer connections, the feature information from the previous convolutional layer is fused with the temporary feature layer. This process is designed to retain more detailed information and prevent information loss. Then, the merged feature layer is subsequently passed to the new PAN layer for implementing two cycles of PAN operations. Through the above operations, DPAN can better aggregate more features and retain detailed information in the low-resolution feature layer. At the same time, the DWConv module is inserted on the output connection of the temporary feature layer to extract local detail features. The DWConv module can reduce the computational load of the model, alleviate information loss, and further optimize the model performance. By introducing the DPAN structure, more efficient feature extraction is achieved, and the accuracy of the micropore defect model detection of the WAAM specimen cross section is significantly improved.

[0030] Furthermore, in the neck network, the GhostConv module replaces the original Conv module, and the GC3ECA module replaces the original C3 module. The core idea of ​​the ECA attention mechanism is to introduce a channel attention mechanism in the convolution operation to capture the relationship between different channels, thereby improving the ability of feature representation. The goal of the channel attention mechanism is to adaptively adjust the weights of channel features so that the network can better focus on important features and suppress unimportant features. Through this mechanism, ECANet can effectively enhance the representation ability of the network without increasing too many parameters and computational costs.

[0031] In summary, the GhostConv module and PConv module are introduced to redesign the lightweight backbone network, and the SPPF module of the original model is redesigned and named as the ESPP module. Through the design of the lightweight backbone network, the model parameters and GFLOPs value are reduced and the detection accuracy and speed of the model are improved. In order to further improve the network's ability to extract and fuse small features, the PANet structure of the original model is redesigned. Drawing on the idea of ​​the RFPN feature cycle, the DPAN network structure is proposed, which enhances the network's feature extraction ability and further enhances the model's ability to detect small pore defects. In order to ensure that the DPAN network is not overly redundant, the ECA attention mechanism is introduced and combined with the GhostConv module to improve the C3 module, and the GC3ECA module is proposed to reduce the model's parameters and enhance the network's representation ability.

[0032] In step 3, the designed MDD-YOLO neural network model is trained. During the formal training, the hyperparameters used are: 300 training cycles, 16 batch size, 0.001 initial learning rate, 0.937 momentum parameter, 0.0005 weight decay, stochastic gradient descent (SGD) optimizer, and linear decay learning rate adjustment strategy.

[0033] The trained model and the YOLO series model are analyzed and compared, as shown in Table 1: Table 1 Model parameter comparison table

[0034] As can be seen from Table 1, higher accuracy: The improved YOLOv5 has improved the accuracy of object detection compared to other YOLO algorithms, and can detect and locate target objects more accurately.

[0035] Faster speed: Despite the improvement in accuracy, the improved YOlOv5 can still maintain a faster detection speed while ensuring higher accuracy, which is very important for real-time applications.

[0036] Small model: Compared with other YOLOv5 series algorithms, the improved YOLOv5 has a smaller model and can be applied in many places.

[0037] The trained model is analyzed and compared with other detection models, as shown in Table 2: Table 2 Comparison of trained models and other detection models

[0038] As can be seen from Table 2, the improved MDD-YOLO maintains excellent levels of detection accuracy, detection speed and model parameter quantity in comparison with several current mainstream algorithms.

[0039] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting microstructure defects in arc additive manufacturing based on improved YOLOv5, characterized in that: The following steps are involved: Step 1: Create a test dataset: Collect arc additive manufacturing microstructure defect image data, and then perform data enhancement on different types of pore defect images through geometric transformation to obtain a test data set consisting of images to be tested; Step 2: Build a neural network model: Adjust and configure the corresponding hyperparameters, build the YOLOv5 neural network model, and improve the YOLOv5 neural network model; Step 3: Train to optimize the YOLOv5 neural network model: After data augmentation, the images of different types of pore defects are input into the YOLOv5 neural network model, and the YOLOv5 neural network model is trained to optimize the YOLOv5 neural network model and generate images; each time the YOLOv5 neural network model is trained, a corresponding weight file is generated; Step 4, output of test results: Save the trained weight file to the YOLOv5 network model. Then, based on the trained weight file, test the detection speed of YOLOv5 in the prediction program and output the test results.

2. According to claim 1, a method for detecting microstructure defects in arc additive manufacturing based on improved YOLOv5, characterized in that: In the step 2, the specific process of improving the neural network model of YOLOv5 is: replacing the original CSPDarkNet53 backbone network with a lightweight backbone network; replacing the original PANet with DPAN; and replacing the Conv module and C3 module in the neck network with the GhostConv module and the GC3ECA module.

3. According to claim 2, a method for detecting microstructure defects in arc additive manufacturing based on improved YOLOv5, characterized in that: The lightweight backbone network consists of GhostConv module, PConv module and ESPP module, which optimizes the number of parameters and GFLOPs value of the model; Conv in YOLOv5 is replaced by GhostConv, which reduces the number of parameters, FLOPs value and model size, and improves the inference speed; The C3 module in the backbone network was replaced with the GC3ECA module, and the network depth of the GC3ECA module in the model was increased to ensure the accuracy of the model; The SPPF module is replaced by the ESPP module to enhance the extraction of feature layer information.

4. According to claim 3, a method for detecting microstructure defects in arc additive manufacturing based on improved YOLOv5, characterized in that: The ESPP module retains the basic framework of SPPCSPC and introduces the PConv module at the front end of the spatial pyramid pooling input.

5. According to claim 4, a method for detecting microstructure defects in arc additive manufacturing based on improved YOLOv5, characterized in that: Change the SPP structure in the SPPSPC structure to an SPPF structure.

6. According to claim 2, a method for detecting microstructure defects in arc additive manufacturing based on improved YOLOv5, characterized in that: DPAN adds additional cross-layer connections and PAN networks on the basis of PANet to achieve extended-layer fusion of feature information.

7. The method for detecting microstructure defects in arc additive manufacturing based on improved YOLOv5 according to claim 6, characterized in that: The YOLOv5 neck network first receives information from the backbone network and performs up- and down-sampling operations through PANet to obtain three temporary feature layers. Through cross-layer connections, the feature information from the previous convolutional layer is fused with the temporary feature layer. The merged feature layer is then passed to the new PAN layer for implementing two cycles of PAN operations; through the above operations, DPAN is able to better aggregate more features and retain detailed information in the low-resolution feature layer.

8. According to claim 2, a method for detecting microstructure defects in arc additive manufacturing based on improved YOLOv5, characterized in that: The C3 module is replaced by the GC3ECA module as follows: By combining the ECA attention mechanism, the BottleNeck in the C3 module is improved to ECA_BottleNeck to enhance the module's perception of channel information; then, the Conv module at the input end of the C3 module is replaced by the GhostConv module.

Images