Electric arc additive manufacturing macroscopic surface defect detection method based on YOLOv8 improvement

Abstract

The invention provides an electric arc additive manufacturing macroscopic surface defect detection method based on YOLOv8 improvement, and the method comprises the steps: 1, obtaining a picture of a weld bead surface defect, and making a weld bead defect data set; 2, dividing the weld bead surface defect image data set into a training set, a verification set and a test set according to a set proportion; step 3, constructing a neural network model based on the improved YOLOv8; 4, training the weld bead surface defect detection model by using the training set and the verification set to obtain an optimal weld bead surface defect detection model; and 5, inputting the test set into the optimal weld bead surface defect detection model, and outputting a test result. The weld bead macroscopic surface defects are automatically detected based on the improved YOLOv8 model, and the accuracy and precision of weld bead macroscopic surface defect detection are greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of image processing and deep learning technology, and relates to a weld macro surface defect detection technology, and in particular to an arc additive manufacturing macro surface defect detection method based on improved YOLOv8. Background Art

[0002] Electric Arc Additive Manufacturing (WAAM) is a metal additive manufacturing technology based on arc welding. Compared with other metal additive manufacturing technologies, it has the advantages of high material utilization and low manufacturing cost, and has broad application prospects in aerospace, automobile manufacturing, shipbuilding, military industry and other fields.

[0003] Traditional arc additive manufacturing macro surface defect detection methods mainly include visual inspection and machine vision inspection. Since there is no significant regularity in the number, size and location distribution of arc additive manufacturing macro surface defects, and the characteristics of these defects are also significantly different in the image, the traditional methods have low detection efficiency and accuracy, and slow detection speed, which cannot meet the actual arc additive manufacturing macro surface defect detection requirements. With the rapid development of artificial intelligence and the increasing maturity of deep learning algorithms, convolutional neural networks have more excellent performance in face recognition, image classification, target detection, etc. After the network structures such as FasterRCNN and YOLO were proposed, the target detection method based on deep learning can automatically learn the characteristics of defects based on macroscopic surface defect images, simplifying the work of a large amount of manual preprocessing in the early stage of traditional methods. Deep neural networks have strong learning ability, good generalization ability and robustness. These characteristics improve the efficiency and accuracy of macroscopic surface defect detection in arc additive manufacturing.

[0004] Currently in the field of image processing and deep learning technology, there are detection methods based on RCNN, SSD, YOLO, etc. Among them, YOLO is the most widely used and can efficiently detect macro surface defects. Summary of the invention

[0005] To solve the above problems, the present invention provides an arc additive manufacturing macro surface defect detection method based on an improved YOLOv8, and optimizes and improves the YOLOv8 model. When used to detect defects on arc additive manufacturing macro surface defects, its detection accuracy can be improved; at the same time, compared with the original YOLOv8 model, the number of parameters and FLOPs value of the model are significantly reduced, and the accuracy of the model reaches 83.2%, which is suitable for deployment in embedded devices with limited resources; it can effectively solve the above problems.

[0006] In order to achieve the above technical features, the purpose of the present invention is to achieve: a method for detecting macroscopic surface defects in arc additive manufacturing based on improved YOLOv8, comprising the following steps: Step 1, obtaining images of weld surface defects and creating a weld defect dataset; Step 2, establishing a weld bead macro surface defect annotation dataset based on the weld bead defect dataset, and dividing the weld bead macro surface defect annotation dataset into a training set, a validation set, and a test set according to a set ratio; Step 3, build a neural network model based on improved YOLOv8; Step 4, training the improved YOLOv8 model, using the training set and the validation set to train the weld surface defect detection model to obtain the optimal weld surface defect detection model; Step 5: Input the test set into the optimal weld surface defect detection model and output the test results.

[0007] Preferably, in the step 3, the specific operation steps of improving the neural network model of YOLOv8 include: replacing the original CIoU loss function with the NWD loss function in the YOLOv8 network structure; introducing the C2f module in the DCNv3 convolution optimization backbone network; replacing the original Conv layer with the GSConv module in the Neck layer of the neck network, and obtaining the Slim-Neck neck network by integrating the GSbottleneck and VoV-GSCSP modules.

[0008] Preferably, in step 3, the specific process of improving the neural network model of YOLOv8 is: replacing the original CIoU loss function with the normalized Gaussian distance loss function; replacing the original C2f module of the backbone network with C2f-DCNv3; replacing the Conv module in the neck network with the GSConv module, and integrating the GSBottleneck and VoV-GSCSP modules.

[0009] Preferably, the backbone network is composed of a CBSModule module, a C2f module, a C2f-DCNv3 module and an SPPF module; The C2f in YOLOv8 is replaced by the C2f-DCNv3 module to achieve weight sharing between neurons.

[0010] Preferably, the NWD loss function has scale invariance, position deviation smoothness and the ability to measure the similarity between overlapping or mutually inclusive bounding boxes in terms of target detection; a two-dimensional Gaussian distribution is used for modeling; thereby converting the similarity of two bounding boxes into a distance distribution of two Gaussian distributions.

[0011] Preferably, the Slim-Neck network is an improvement on the original neck network, and Slim-Neck is composed of a VoV-GSCSP module, a GSConv module and a GSbottleneck module stacked separately.

[0012] Preferably, the GSConv module is composed of GC convolution, SC convolution, and Shuffle layer, and performs feature extraction on the input image.

[0013] Preferably, the specific operation steps of establishing the weld macro surface defect annotation data set in step 2 are: using laboratory arc additive equipment to make weld macro surface defect samples, using LabelImg tool software to manually annotate the weld macro defect pictures, and dividing the defect categories into weld, surface proes, spatter, and overlap; at the same time, indicating the category and position of the defect in each picture, and storing it in the YOLOv8 data format, and then randomly dividing the annotated weld macro surface defect data set into a training set, a validation set, and a test set in a ratio of 7:2:1; the training set is used as model training, the validation set is used as model verification, and the test set is used as model testing; and a mosaic data enhancement method is used: cropping, flipping, noise addition, and Gaussian blurring to reduce overfitting and improve model accuracy.

[0014] Preferably, step 4 trains the improved YOLOv8 model, and the specific steps include: dividing the weld macroscopic surface defect dataset into a training set, a test set, and a validation set in a ratio of 7:2:1; the experimental hardware configuration uses an Intel (R) Core (TM) i5-12490F 3.00GHz processor, an NVIDIA GeForce RTX 3060 video memory GPU, the operating system is Windows 10, the development platform uses Pycharm, the editing language is Python, the deep learning framework is Pytorch, and the GPU acceleration library is CUDA11.7; the integrated development environment is Anaconda2023, Python3.8, CUDA11.7, and CUDNN8.5.0; the training parameter settings of this experiment are: batch_size is set to 16, Epoch is set to 300, img size is set to 640 × 640, the initial learning rate is 0.01, SDG function optimization is adopted, the momentum factor is 0.937, and the weight attenuation coefficient is 0.0005.

[0015] Preferably, in step 5, an improved YOLOv8 model is loaded for detection processing, and the detection result is output.

[0016] The present invention has the following beneficial effects: 1. Based on YOLOv8, the present invention uses C2f-DCNv3 network to reconstruct the backbone network of YOLOv8, and uses NWD loss function to replace CIoU loss function. In order to reduce the amount of calculation and parameters of the model, the GSBottleneck module and VoV-GSCSP module are integrated into the Neck network to replace the Conv module. The improved GSConv module is used to reduce the calculation cost, optimize the calculation efficiency and structural complexity of the network, improve the feature extraction ability, and reduce the size of the model. The improved entire network is called NDS-YOLOv8. The model of the present invention has good performance in detection accuracy and speed.

[0017] 2. The present invention can complete the detection of macroscopic surface defects in a short time, basically meets the real-time requirements, and has high performance.

[0018] 3. The present invention can be applied to various occasions according to the requirements of arc additive manufacturing detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0020] Figure 1 This is a structural diagram of the improved YOLOv8 network in the present invention.

[0021] Figure 2 This is a structural diagram of C2f-DCNv3 in the present invention.

[0022] (a) C2f structure diagram; (b) C2f-DCNv3 structure diagram.

[0023] Figure 3 This is the GSConv structure diagram in the present invention.

[0024] Figure 4 This is a diagram of the neck network structure in the present invention.

[0025] Figure 5 This is a diagram of the detection results in the present invention. DETAILED DESCRIPTION

[0026] The following will be described in detail in conjunction with the technical solutions in the embodiments of the present invention, wherein the embodiments described are only a part of the embodiments of the present invention, rather than all of the embodiments. Without departing from the design concept of the present invention, various improvements made by ordinary persons in the art to the technical solutions of the present invention should all fall within the scope of protection of the present invention.

[0027] Embodiment 1: like Figure 1 As shown, a method for detecting macroscopic surface defects in arc additive manufacturing based on improved YOLOv8 includes the following steps: Step 1: Use laboratory arc additive manufacturing equipment to produce and use industrial cameras to take pictures of weld surface defects to create a weld defect dataset; Step 2, based on the original image data, manually annotate using the LabelImg tool to establish a weld bead macro surface defect annotation dataset, and divide the weld bead surface defect image dataset into a training set, a validation set, and a test set according to a set ratio; Step 3: Build a neural network model based on improved YOLOv8. The specific steps include: using NWD loss function to replace the original CIoU loss function in the YOLOv8 network structure; introducing the C2f module in the DCNv3 convolution optimization backbone network; replacing the original Conv layer with the GSConv module in the Neck layer of the neck network, and obtaining the Slim-Neck network by integrating the GSbottleneck and VoV-GSCSP modules; Among them, the NWD loss function is also called the normalized Gaussian Wasserstein distance loss function. It measures the similarity of bounding boxes through the Wasserstein distance, models the bounding boxes as two-dimensional Gaussian distributions, and calculates the similarity between them through these Gaussian distributions. This loss function is based on the two-dimensional Gaussian distribution. First, the predicted box and the true box are modeled as Gaussian distributions, and then the Wasserstein distance between them is calculated. It has better performance than CIoU. The original CIoU loss function is an optimization indicator for target detection tasks. This loss function comprehensively considers the consistency of the overlapping area (IoU), the center point distance, and the aspect ratio. By minimizing the geometric difference between the predicted bounding box and the true bounding box, the accuracy and robustness of the detection model are improved. The DCNv3 module consists of two parts. The first part is the deformable convolution layer, and the second part is the standard convolution layer. The deformable convolution layer is responsible for adjusting the sampling position, while the standard convolution layer extracts features from these adjusted sampling points. It introduces deformable convolution operations to enhance the network's adaptability to the geometric deformation of the input data; it can dynamically adjust the sampling position of the convolution kernel to better capture the complex geometric structure and deformation features in the image. In the model, the C2f module mainly consists of input convolution layer, Bottleneck layer, segmentation, residual connection and output convolution layer; and the Bottleneck layer is composed of convolution layer, batch normalization layer, etc.; this module converts the output feature map of the convolution layer into a feature representation suitable for the target detection task, and performs deep feature extraction on the input image or feature map through a series of convolution layers. These convolution layers can capture the local features and texture information in the image; in the process of feature extraction and fusion, the C2f module will also adjust the spatial dimension of the feature map; this not only improves the efficiency of feature extraction, but also enhances the model's ability to model complex data. The Neck part is the neck network of the model, including PPF module, convolution layer, upsampling, C2f module, etc. Conv layer refers to the convolution layer, which is the basic component used to extract features in the neural network. The convolution layer calculates the weighted sum of the local area by sliding the convolution kernel (or filter) through the input feature map, generating a new feature map, which helps to extract the local features of the input data. The GSConv module is mainly composed of convolution layer, DWConv convolution, Concat connection layer, and shuffle layer. It can achieve faster processing speed and reduce the model size while maintaining high accuracy; DWConv convolution is a deep convolution, which first performs spatial convolution on each input channel of the input tensor to generate an output channel, which can reduce the number of parameters and computational complexity of the model and can capture the spatial features of the input data; the shuffle layer enhances the information flow between channels and enhances the representation ability of the model by rearranging the channel order of the convolution layer output.The GSbottleneck module contains 2 GSConv convolutions, and the VoV-GSCSP module consists of one convolution, two GSConv convolutions, one Concat connection, and one output convolution. After 5 feature processing, the output features can be obtained. This is done to optimize the computational efficiency and structure of the model. Slim-Neck is an improved neck network of the original YOLO V8 model after the above optimization. The C2f-DCNv3 module replaces the standard convolution Conv module in the original C2f module with the DCNv3 module to enhance the image classification and recognition capabilities of the C2f module. The C2f modules in different parts of the YOLOV8 model backbone network are replaced with C2f-DCNv3 modules. CBSModule consists of three main parts: a two-dimensional convolutional layer (Conv), a two-dimensional batch normalization layer (BatchNorm), and a SiLU activation function (SiLU); this module can extract features and enhance the nonlinear expression ability of the network. It extracts features through the Conv convolution layer, performs normalization through the BatchNorm batch normalization layer, and introduces nonlinearity through the SiLU activation function, thereby enhancing the representation ability of the network and further improving the feature extraction and fusion capabilities of the model.

[0028] Step 4, training the improved YOLOv8 model, using the training set and the validation set to train the above weld surface defect detection model, and obtaining the optimal weld surface defect detection model; after using the above three methods to improve the model, not only the parameter quantity and calculation efficiency of the model are optimized, but also the information redundancy of the network neck is reduced, thereby improving the detection performance of the model; Step 5: Input the test set into the optimal weld surface defect detection model and output the test results.

[0029] As a preferred embodiment of the present invention, in step 3, the specific process of improving the neural network model of YOLOv8 is: replacing the original CIoU loss function with the NWD loss function; designing the C2f-DCNv3 module in the backbone network to replace the original C2f module; replacing the original Conv layer with the GSConv module in the neck network, and integrating the GSBottleneck and VoV-GSCSP modules to obtain the Slim-Neck neck network.

[0030] In step 3: The NDS-YOLOv8 network optimizes the computing performance and structure of the model through the C2f-DCNv3 module, GSBottleneck module, VoV-GSCSP module and GSConv module, and improves the feature extraction capability of the model.

[0031] It should be noted that the model uses the NWD loss function instead of the original CIoU loss function to improve the ability to identify and locate defects. This method first models the two-dimensional Gaussian distribution of the bounding box, thereby converting the similarity of the two bounding boxes into the distance distribution of two Gaussian distributions. NWD LOSS calculates the distance between distributions through the Wassertein distance in the optimal transfer theory, and converts the similarity of the two bounding boxes into the distance distribution of two Gaussian distributions. Without affecting the performance of the model, it has scale invariance and position deviation smoothness, can measure the compatibility between overlapping or mutually inclusive bounding boxes, and effectively improves the accuracy of macroscopic surface defect detection.

[0032] Furthermore, while retaining the basic structure of C2f, the C2f-DCNv3 module introduces the DCNv3 module at the front end of the SPPF layer of the backbone network, decomposing the traditional convolution weight Wk into two parts: depth and point convolution, thereby achieving weight sharing between neurons; at the same time, a grouped convolution mechanism is introduced to decompose spatial aggregation into multiple independent groups, each with its own sampling offset and modulation factor, providing richer feature representation for specific tasks. In addition, by changing the activation function of the modulation factor from sigmoid to softmax k factors for normalization, the training process becomes more stable, optimizing the performance and efficiency of the entire model.

[0033] Furthermore, the Slim-Neck network adds a GSConv module on the basis of the original Neck, and through the GSConv module, connects the adjacent layers in the Neck layer and transmits it to the neck network to realize the connection of channel information. Specifically, GSConv is used in the Neck part of the model, and the computational efficiency and structure of the network are optimized through the GSBottleneck and VoV-GSCSP modules. This process aims to achieve faster processing speed and smaller model size while maintaining high accuracy. Through the above operations, the spliced ​​feature map can not only effectively avoid information redundancy and adapt to complex computing requirements, but also improve the performance of the Neck network, achieve more effective feature extraction, and significantly improve the accuracy and speed of arc additive manufacturing macro surface defect model detection.

[0034] In summary, the introduction of the C2f-DCNv3 module, GSBottleneck module, VoV-GSCSP module, and GSConv module optimizes the computing performance and structure of the model, reduces the number of parameters and FLOPs value of the model, and improves the detection accuracy and speed of the model.

[0035] In step 4, the designed NDS-YOLOv8 neural network model is trained using the PyTorch architecture with the following hyperparameters: 300 training cycles, 16 batch size, 0.01 initial learning rate, 0.937 momentum parameter, 0.0005 weight decay, a stochastic gradient descent (SGD) optimizer, and a linear decay learning rate adjustment strategy.

[0036] The performance analysis and comparison of the Neck network of the present invention and other neck networks are shown in Table 1: Table 1 Performance comparison of different neck networks

[0037] As can be seen from Table 1, the Slim-Neck neck network structure of the present invention outperforms other structures with a mAP of 81.8% and a performance of 138.1FPS, and maintains a low model parameter amount (10.3M) and FLOPs value (25.3G). Therefore, the neck network of the present invention has better performance.

[0038] The trained model is analyzed and compared with other detection models, as shown in Table 2: Table 2 Comparison of different models

[0039] As can be seen from Table 2, the NDS-YOLOv8 model of the present invention has a model parameter amount of 10.3M and a processing speed of 25.3G FLOPS, while ensuring high detection accuracy. In the comparison of several mainstream algorithms, its detection accuracy, detection speed and model parameter amount are all good.

[0040] like Figure 5 As shown in the figure, the detection results of the present invention in the arc additive manufacturing macro surface defect data set are given. It can be seen from the figure that the model can accurately identify four types of defects: weld, surface proes, spatter, and overlap. The experimental results show that the detection results given by the present invention are accurately classified, the predicted value positioning is almost consistent with the true value, and it has high detection accuracy.

[0041] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the present invention. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that any modifications or substitutions made do not deviate from the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for detecting macroscopic surface defects in arc additive manufacturing based on improved YOLOv8, characterized in that: The following steps are involved: Step 1, obtaining images of weld surface defects and creating a weld defect dataset; Step 2, establishing a weld bead macro surface defect annotation dataset based on the weld bead defect dataset, and dividing the weld bead macro surface defect annotation dataset into a training set, a validation set, and a test set according to a set ratio; Step 3, build a neural network model based on improved YOLOv8; Step 4, training the improved YOLOv8 model, using the training set and the validation set to train the weld surface defect detection model to obtain the optimal weld surface defect detection model; Step 5: Input the test set into the optimal weld surface defect detection model and output the test results.

2. According to the method for detecting macroscopic surface defects in arc additive manufacturing based on improved YOLOv8 in claim 1, it is characterized in that: In the step 3, the specific operation steps of improving the neural network model of YOLOv8 include: using the NWD loss function to replace the original CIoU loss function in the YOLOv8 network structure; introducing the C2f module in the DCNv3 convolution optimization backbone network; replacing the original Conv layer with the GSConv module in the Neck layer of the neck network, and obtaining the Slim-Neck neck network by integrating the GSBottleneck and VoV-GSCSP modules.

3. According to claim 2, a method for detecting macroscopic surface defects in arc additive manufacturing based on improved YOLOv8, characterized in that: In step 3, the specific process of improving the neural network model of YOLOv8 is: replacing the original CIoU loss function with the normalized Gaussian distance loss function; replacing the C2f module of the original backbone network with C2f-DCNv3; replacing the Conv module in the neck network with the GSConv module, and integrating the GSBottleneck and VoV-GSCSP modules.

4. According to claim 3, the arc additive manufacturing macro surface defect detection method based on improved YOLOv8 is characterized in that: The backbone network consists of CBSModule, C2f, C2f-DCNv3 and SPPF modules; The C2f in YOLOv8 is replaced by the C2f-DCNv3 module to achieve weight sharing between neurons.

5. According to claim 4, the arc additive manufacturing macro surface defect detection method based on improved YOLOv8 is characterized in that: The NWD loss function has scale invariance, position deviation smoothness and the ability to measure the similarity between overlapping or mutually inclusive bounding boxes in target detection; a two-dimensional Gaussian distribution is used for modeling; thereby converting the similarity of two bounding boxes into a distance distribution of two Gaussian distributions.

6. According to claim 5, the arc additive manufacturing macro surface defect detection method based on improved YOLOv8 is characterized in that: The Slim-Neck network improves the Neck on the basis of the original Neck network. The Slim-Neck is composed of a VoV-GSCSP module, a GSConv module and a GSbottleneck module stacked separately.

7. The method for detecting macroscopic surface defects in arc additive manufacturing based on improved YOLOv8 according to claim 6, characterized in that: The GSConv module consists of GC convolution, SC convolution, and Shuffle layer to extract features from the input image.

8. The method for detecting macroscopic surface defects in arc additive manufacturing based on improved YOLOv8 according to claim 7, characterized in that: The specific operation steps for establishing the weld macro surface defect annotation dataset in step 2 are as follows: use laboratory arc additive equipment to make weld macro surface defect samples, use LabelImg tool software to manually annotate the weld macro defect pictures, and divide the defect categories into weld, surface proes, spatter, and overlap; at the same time, indicate the category and location of the defect in each picture, and store it in the YOLOv8 data format, and then randomly divide the annotated weld macro surface defect dataset into training set, validation set, and test set in a ratio of 7:2:1; the training set is used as model training, the validation set is used as model validation, and the test set is used as model testing; and the mosaic data enhancement method is used: cropping, flipping, noise addition, and Gaussian blurring to reduce overfitting and improve model accuracy.

9. The method for detecting macroscopic surface defects in arc additive manufacturing based on improved YOLOv8 according to claim 8, characterized in that: Step 4 trains the improved YOLOv8 model. The specific steps include: dividing the weld macroscopic surface defect dataset into training set, test set and validation set in a ratio of 7:2:

1. The experimental hardware configuration uses Intel (R) Core (TM) i5-12490F 3.00GHz processor, NVIDIA GeForce RTX 3060 video memory GPU, Windows 10 operating system, Pycharm development platform, Python editing language, Pytorch deep learning framework, and CUDA11.7 GPU acceleration library; the integrated development environment is Anaconda2023, Python3.8, CUDA11.7, CUDNN8.5.0; the training parameter settings of this experiment are: batch_size is set to 16, Epoch is set to 300, img size is set to 640 × 640, the initial learning rate is 0.01, SDG function optimization is adopted, the momentum factor is 0.937, and the weight decay coefficient is 0.0005.

10. The arc additive manufacturing macro surface defect detection method based on improved YOLOv8 according to claim 9, characterized in that: In step 5, the improved YOLOv8 model is loaded for detection processing, and the detection result is output.

Images