A method for small-scale damage detection on metal surface based on improved YOLOv8 network

Abstract

The present invention discloses a metal surface small-scale damage detection method based on an improved YOLOv8 network, S1: obtaining a metal surface defect data set, and preprocessing the metal surface defect data set; S2: establishing an improved YOLOv8 network model, including: an improved backbone network module for extracting defect features in a metal surface image; an improved neck network module for performing feature fusion operations on the defect features to obtain fused features; the detection head module for further extracting fused features, and outputting different categories of metal surface small-scale damage and corresponding damage probabilities; S3: training the improved YOLOv8 network model to obtain a trained YOLOv8 network model. The present invention can aggregate channel feature information through the improved YOLOv8 network model, avoid channel feature loss, and can capture narrow cracks and small-scale damage structures, thereby improving the detection accuracy of small-scale metal surface damage.

Description

Technical Field

[0001] The present invention relates to the technical field of metal damage detection, and in particular to a metal surface small-scale damage detection method based on an improved YOLOv8 network. Background Art

[0002] As my country's strategic goals gradually shift from infrastructure construction to infrastructure maintenance and inspection, the inspection and repair of building metals has also gradually emerged. The damage on the metal surface is mainly divided into cracks, pores, and dents, and the damage on the metal surface is usually small-scale. It usually takes a lot of time and effort to detect them one by one with the naked eye. With the development of machine learning, rule-based image processing technology has been applied, but rule-based image processing methods can only design special algorithms for specific types of damage, and it is difficult to deal with damage of different types and sizes. In addition, the algorithm is sensitive to noise and light changes, and is prone to misjudgment and missed detection.

[0003] With the continuous progress and development of computer vision and deep learning technology, intelligent damage detection methods have gradually replaced traditional manual detection and machine image processing methods, becoming one of the most widely used means of metal surface damage detection. In computer vision, both target detection and semantic segmentation are applied to metal surface damage detection. Through target detection, the damage position of the metal surface can be quickly and accurately located to help count the damage, while semantic segmentation divides the metal surface image into different areas, which can quickly segment the damaged area. At present, the most representative network for target detection is YOLO. The YOLO algorithm has been gradually upgraded from YOLOv1 to YOLOv10. Among them, the most widely used are YOLOv5 and YOLOv8. The overall structure of YOLOv5 and YOLOv8 is similar. YOLOv8 is an improved version of YOLOv5. It integrates advanced SOTA technology, introduces new functions and improvement strategies, and has excellent detection performance. Therefore, it is often used in metal surface damage detection. In existing detection methods, although the YOLOv8 algorithm is used to detect metal surface damage, it often fails to effectively retain the detailed information of small targets, and therefore there is a problem of missed detection of metal surface cracks and small-scale damage. Summary of the invention

[0004] The present invention provides a metal surface small-scale damage detection method based on an improved YOLOv8 network to overcome the technical problem that there are many missed detections of metal surface cracks and small-scale damages.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] A method for detecting small-scale damage on metal surfaces based on an improved YOLOv8 network, the specific steps of which include:

[0007] S1: Acquire a metal surface defect data set, and preprocess the metal surface defect data set to obtain a processed data set;

[0008] S2: establishing an improved YOLOv8 network model, wherein the improved YOLOv8 network model includes an improved backbone network module, an improved neck network module and a detection head module;

[0009] The improved backbone network module is used to extract defect features in the metal surface image and transmit them to the neck network module;

[0010] The improved neck network module is used to perform feature fusion operation on the defect features to obtain fused features and transmit them to the detection head module;

[0011] The detection head module is used to further extract the fused features and output different types of small-scale damage on the metal surface and the corresponding damage probability;

[0012] S3: Training the improved YOLOv8 network model based on the processed data set to obtain a trained YOLOv8 network model;

[0013] S4: Detect small-scale damage on metal surfaces based on the trained YOLOv8 network model.

[0014] Furthermore, the improved backbone network module is improved on the basis of the original YOLOv8 network model, and the improvement is as follows:

[0015] Replace each convolution module Conv in the backbone network module of the original YOLOv8 network model with a snake convolution module DSCConv;

[0016] Replace the Conv in each C2f module in the backbone network module of the original YOLOv8 network model with a convolution module DWGConv that combines depthwise separable convolution and ordinary convolution;

[0017] An MCA attention mechanism module is added between the fourth C2f module and the spatial pyramid pooling layer SPPF in the backbone network module of the original YOLOv8 network model.

[0018] Furthermore, the improved neck network module is improved on the basis of the original YOLOv8 network model, and the improvement is as follows:

[0019] A multi-scale feature fusion module USF is added between the second C2f module and the first convolution module Conv in the neck network module of the original YOLOv8 network model.

[0020] Further, the snake convolution module DSCConv includes a snake convolution layer DSC, a BN layer and a SiLU layer;

[0021] The serpentine convolution layer DSC is used to extract small-scale damage features in the input data and transmit them to the normalization layer. The normalization layer is used to normalize the input data and transmit them to the SiLU layer. The SiLU layer is used to smooth the input data.

[0022] Furthermore, a convolution module DWGConv combining depthwise separable convolution and common convolution includes a first convolution layer, a second normalization layer, a ReLU activation function layer, a second convolution layer, a depthwise separable convolution layer and a third convolution layer;

[0023] Among them, the process of processing data based on DWGConv is:

[0024] The defect features of the input data are extracted through the first convolution layer, and then the output of the first convolution layer is normalized by the second normalization layer, and then passed to the ReLu activation function layer for activation processing. The output feature maps are evenly grouped to obtain two sets of feature map sets, one of which is passed to the depthwise separable convolution layer for feature map channel feature extraction, and the other set of feature map sets is passed to the second convolution layer for feature map smoothing feature extraction. The feature maps extracted by the second convolution layer and the depthwise separable convolution layer are spliced ​​according to channels to form a new feature map. Finally, the new feature map is compressed through a 1×1 third convolution layer to complete the processing of the input data.

[0025] Furthermore, the MCA attention mechanism module includes a top branch, a middle branch and a bottom branch;

[0026] The process of data processing based on the added MCA attention mechanism module is:

[0027] 1) Extracting spatial feature information in the H direction of a first feature map of size C×H×W through the top branch, wherein the first feature map is a feature map output by the backbone network module and contains spatial feature information and channel feature information of small-scale damage on the metal surface, and the specific steps include:

[0028] First, the size of the first feature map is permuted by a first permutation function to obtain a second feature map with a size of H×C×W;

[0029] The feature information in the W and C directions of the second feature map is compressed by the first AvgPool and the first StdPool in sequence, and the spatial feature information in the H direction is retained to obtain a third feature map with a size of H×1×1, and the scale is transformed by the second permutation function to obtain a fourth feature map with a size of 1×1×H;

[0030] The spatial feature information in the H direction in the fourth feature map is extracted through the first two-dimensional convolution layer of 1×K to obtain a fifth feature map of size 1×1×H, which is then transformed in scale through the third permutation function and then passed through the first Sigmoid activation function layer to generate weights to obtain the first weighted feature map of size H×1×1;

[0031] Multiplying the first weighted feature map by the first feature map, and then performing channel restoration through a fourth permutation function, to finally obtain a top branch feature map containing H-direction damage space information, and having a size of C×H×W;

[0032] 2) extracting spatial feature information in the W direction from the first feature map of the input size of C×H×W through the middle branch, the specific steps include:

[0033] First, the size of the first feature map is permuted by a fifth permutation function to obtain a sixth feature map with a size of W×H×C;

[0034] The feature information of the H and C directions in the sixth feature map is compressed by the second AvgPool and the second StdPool in sequence, and the spatial feature information of the W direction is retained to obtain a seventh feature map of size W×1×1, and the scale is transformed by the sixth permutation function to obtain an eighth feature map of size 1×1×W;

[0035] The spatial information features in the W direction of the eighth feature map are extracted through the second two-dimensional convolution layer of 1×K to obtain a ninth feature map of size 1×1×W, which is then transformed in scale through the seventh permutation function and then passed through the second Sigmoid activation function layer to generate weights to obtain a second weighted feature map of size W×1×1.

[0036] The second weighted feature map is multiplied by the sixth feature map, and then the spatial feature information is restored through the eighth permutation function, so as to finally obtain an intermediate branch feature map containing the W-direction damage spatial information, and the size of the intermediate branch feature map is C×H×W;

[0037] 3) extracting channel feature information in the C direction from the first feature map of the input size of C×H×W through the bottom branch, the specific steps include:

[0038] First, the spatial feature information in the H and W directions is compressed by the third AvgPool and the third StdPool, and the channel feature information in the C direction is retained, thereby obtaining the tenth feature map with a size of C×1×1, and then the scale is transformed by the ninth permutation function to obtain the eleventh feature map with a size of 1×1×C;

[0039] The channel feature information in the C direction in the eleventh feature map is extracted through the third two-dimensional convolution layer of 1×K to obtain the twelfth feature map with a size of 1×1×C, and the scale is transformed by the tenth permutation function and then the weight is generated through the third Sigmoid activation function layer to obtain the third weighted feature map with a size of C×1×1;

[0040] Multiplying the third weighted feature map by the first feature map to obtain a bottom branch feature map containing channel feature information in the C direction, and having a size of C×H×W;

[0041] 4) Adding the top branch feature map, the middle branch feature map and the bottom branch feature map to obtain a fused feature map that is subsequently input into the spatial pyramid pooling layer SPPF for pyramid pooling.

[0042] Furthermore, the process of processing data based on the multi-scale feature fusion module USF is as follows:

[0043] The feature map output by the first upsampling layer in the improved neck network module is downsampled through the maximum pooling layer, and then channel-joined with the feature map output by the second upsampling layer in the improved neck network module to obtain a first fused feature map;

[0044] The number of channels of the first fused feature map is reduced through the fourth 3×3 convolutional layer, and then the channel is spliced ​​with the feature map output by the second C2f module to obtain the second fused feature map;

[0045] The number of channels of the second fused feature map is reduced through the fifth 3×3 convolutional layer, and then the second feature fused feature map with the reduced number of channels is upsampled and then channel-joined with the feature map output by the second C2f module to obtain the third fused feature map;

[0046] The number of channels of the third fused feature map is reduced through the sixth 3×3 convolutional layer, and then the third feature fused feature map with the reduced number of channels is upsampled, and then the channel is spliced ​​with the feature map output by the first upsampling layer to obtain the fourth fused feature map;

[0047] The number of channels of the fourth fused feature map is reduced through the seventh 3×3 convolution layer, and then the number of channels is changed through the eighth 1×1 convolution layer to obtain a fused feature map for input into the first convolution module in the improved neck network module for further processing.

[0048] Beneficial effect: The present invention designs an improved YOLOv8 network model based on the original YOLOv8 network model, that is, the backbone network module and the neck network module are improved respectively. The improved Y OLOv8 network model can aggregate spatial feature information and channel feature information, avoid channel feature loss, and capture narrow cracks and small-scale damage structures, thereby improving the detection accuracy of small-scale metal surface damage. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0050] Figure 1 It is a flow chart of a method for detecting small-scale damage on a metal surface based on an improved YOLOv8 network in the present invention;

[0051] Figure 2 It is a structural diagram of the original YOLOv8 network model in the present invention;

[0052] Figure 3 Schematic diagram of the structure of C2f and DWG_C2f in an embodiment of the present invention;

[0053] Figure 4 Schematic diagram of the structure of DWGConv in an embodiment of the present invention;

[0054] Figure 5 Schematic diagram of the structure of DSCConv and Conv in an embodiment of the present invention;

[0055] Figure 6 Schematic diagram of the structure of the spatial pyramid pooling layer SPPF in an embodiment of the present invention;

[0056] Figure 7 It is a structural diagram of the MCA attention mechanism module in an embodiment of the present invention;

[0057] Figure 8 Schematic diagram of the structure of the improved YOLOv8 network model in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] This embodiment provides a method for detecting small-scale damage on a metal surface based on an improved YOLOv8 network. Figure 1 As shown, the specific steps include:

[0060] S1: Acquire a metal surface defect dataset, and preprocess the metal surface defect dataset to obtain a processed dataset;

[0061] S2: establishing an improved YOLOv8 network model, wherein the improved YOLOv8 network model includes an improved backbone network module, an improved neck network module and a detection head module;

[0062] The improved backbone network module is used to extract defect features in the metal surface image and transmit them to the neck network module;

[0063] The improved neck network module is used to perform feature fusion operation on the defect features to obtain fused features and transmit them to the detection head module;

[0064] The detection head module Head is used to further extract the fused features and output different types of small-scale damage on the metal surface and the corresponding damage probability;

[0065] Specifically, in this embodiment, the detection head module in the existing YOLOv8 network model is not improved. The detection head module is divided into two parts, which can predict the target category and position of each image, and output a vector containing the probabilities of different damage categories, thereby converting the input feature map into different categories of metal surface damage and frame damage location information and predicted damage probability.

[0066] S3: Training the improved YOLOv8 network model based on the processed data set to obtain a trained YOLOv8 network model;

[0067] S4: Detect small-scale damage on metal surfaces based on the trained YOLOv8 network model.

[0068] In a specific embodiment, the improved backbone network module is improved on the basis of the original YOLOv8 network model, such as Figure 2 and Figure 8 As shown, the improvement is:

[0069] Replace each convolution module Conv in the backbone network module Backbone of the original YOLOv8 network model with a snake convolution module DSCConv;

[0070] like Figures 2 to 4 As shown in the figure, the Conv in each C2f module in the backbone network module Backbone of the original YOLOv8 network model is replaced with a convolution module DWGConv that combines depthwise separable convolution and ordinary convolution;

[0071] The fourth C2f module in the backbone network module Backbone of the original YOLOv8 network model is Figure 6 An MCA attention mechanism module is added between the spatial pyramid pooling layers SPPF shown.

[0072] Specifically, Figure 2 As shown, the backbone network module Backbone of the original YOLOv8 network model includes a first backbone convolution module, namely the first Conv, a second backbone convolution module, namely the second Conv, a first C2f module, a third backbone convolution module, a second C2f module, a fourth backbone convolution module, a third C2f module, a fifth backbone convolution module, a fourth C2f module and a spatial pyramid pooling layer SPPF. In this embodiment, an MCA attention mechanism module is added between the fourth C2f module and the spatial pyramid pooling layer SPPF.

[0073] In a specific embodiment, the improved neck network module is improved on the basis of the original YOLOv8 network model, and the improvement is as follows:

[0074] In the neck network module Neck of the original YOLOv8 network model, Figure 2 A multi-scale feature fusion module USF is added between the second C2f module and the first neck convolution module, i.e., the first Conv.

[0075] In a specific embodiment, Figure 5 As shown, the snake convolution module DSCConv includes a snake convolution layer DSC, a BN layer and a SiLU layer;

[0076] Because small-scale damage on the metal surface is usually tiny, band-shaped and has a narrow connected area, in this embodiment, the serpentine convolution layer DSC is used to extract small-scale damage features in the input data and transmit it to the normalization layer. The normalization layer is used to normalize the input data and transmit it to the SiLU layer. The SiLU layer is used to smooth the input data.

[0077] Among them, the snake convolution layer DSC improves the expressiveness of the neural network by introducing nonlinear activation functions and parameterization. The BN layer standardizes each batch of data to make the input distribution of each layer more stable, thereby accelerating the convergence speed of the model. The SiLU layer provides an effective activation mechanism for the neural network through its smooth nonlinear and adaptive characteristics, thereby improving the performance and stability of the model.

[0078] The working principle of the snake convolution layer DSC is as follows: Taking the X-axis direction as an example, the pixels of each image are divided and a pixel point is set as a grid. Therefore, the specific position of each grid is represented by the distance between the X-axis coordinates of different pixels and the center coordinates, that is:

[0079] K i+c =(x i+c ,y i+c ) (1)

[0080] Where c represents the distance from the pixel to the convolution center, c = 0, 1, 2, 3, 4; K i is the coordinate of the convolution center, and K i =(x i ,y i );

[0081] The different position parameters of the X-axis coordinates of different pixels are changed by a 3×3 convolution operation with a dilation of 1 as shown in formula (2), which is expressed as:

[0082] K={(x-1,y-1),(x-1,y),…(x+1,y+1)} (2)

[0083] The selection of each grid position in the convolution kernel is an accumulation process. The selection of its position depends on the position of the previous grid. Therefore, the different position parameters at the position covered by the convolution kernel in the X-axis direction are accumulated to obtain the value in the X-axis direction, which is expressed as:

[0084]

[0085] Similarly, the different position parameters at the positions covered by the convolution kernel in the Y-axis direction are accumulated to obtain the value in the Y-axis direction, which is expressed as:

[0086]

[0087] Specifically, by introducing dynamic snake convolution, the size and size of the convolution kernel can be adaptively changed, so as to capture narrow cracks and small-scale damaged structures.

[0088] In a specific embodiment, Figure 4As shown, the convolution module DWGConv combining depthwise separable convolution and ordinary convolution includes a first convolution layer, a second normalization layer, a ReLU activation function layer, a second convolution layer, a depthwise separable convolution layer and a third convolution layer;

[0089] The process of processing data based on DWGConv is:

[0090] The defect features of the input data are extracted through the first convolution layer, and then the output of the first convolution layer is normalized through the second normalization layer to accelerate the convergence speed of the model. It is then passed to the ReLu activation function layer for activation processing. The output feature maps are evenly grouped to obtain two sets of feature map sets, one of which is passed to the depthwise separable convolution layer for feature map channel feature extraction, and the other set of feature map sets is passed to the second convolution layer for feature map smoothing feature extraction. The feature maps extracted by the second convolution layer and the depthwise separable convolution layer are spliced ​​according to channels to form a new feature map. Finally, the new feature map is compressed through a 1×1 third convolution layer to complete the processing of the input data.

[0091] Specifically, in this embodiment, by performing an average grouping operation and then performing ordinary convolution and depth-wise separable convolution on the grouped feature maps, channel feature information can be aggregated, thereby avoiding channel feature loss.

[0092] In a specific embodiment, Figure 7 and Figure 8 As shown, the MCA attention mechanism module includes a top branch, a middle branch and a bottom branch; since the MCA attention mechanism module is located after the fourth DWG_C2f module in the backbone network module of the improved YOLOv8 network model, the input feature map of the MCA attention mechanism module is the output feature map extracted by a total of four DWG_C2f modules, and the feature map contains a large amount of spatial feature information in the H and W directions and channel feature information in the C direction of small-scale damage on the metal surface after passing through the DWG_C2f module;

[0093] The process of data processing based on the added MCA attention mechanism module is:

[0094] 1) Extracting spatial feature information in the H direction of a first feature map of size C×H×W through the top branch, wherein the first feature map is a feature map output by the backbone network module and contains spatial feature information and channel feature information of small-scale damage on the metal surface, and the specific steps include:

[0095] First, the size of the first feature map is permuted by a first permutation function to obtain a second feature map with a size of H×C×W;

[0096] The feature information in the W and C directions of the second feature map is compressed by the first AvgPool and the first StdPool in sequence, and the spatial feature information in the H direction is retained to obtain a third feature map with a size of H×1×1, and the scale is transformed by the second permutation function to obtain a fourth feature map with a size of 1×1×H;

[0097] The spatial feature information in the H direction in the fourth feature map is extracted through the first two-dimensional convolution layer of 1×K to obtain a fifth feature map of size 1×1×H, which is then transformed in scale through the third permutation function and then passed through the first Sigmoid activation function layer to generate weights to obtain the first weighted feature map of size H×1×1;

[0098] Multiplying the first weighted feature map by the first feature map, and then performing channel restoration through a fourth permutation function, to finally obtain a top branch feature map containing H-direction damage space information, and having a size of C×H×W;

[0099] 2) extracting spatial feature information in the W direction from the first feature map of the input size of C×H×W through the middle branch, the specific steps include:

[0100] First, the size of the first feature map is permuted by a fifth permutation function to obtain a sixth feature map with a size of W×H×C;

[0101] The feature information of the H and C directions in the sixth feature map is compressed by the second AvgPool and the second StdPool in sequence, and the spatial feature information of the W direction is retained to obtain a seventh feature map of size W×1×1, and the scale is transformed by the sixth permutation function to obtain an eighth feature map of size 1×1×W;

[0102] The spatial information features in the W direction of the eighth feature map are extracted through the second two-dimensional convolution layer of 1×K to obtain a ninth feature map of size 1×1×W, which is then transformed in scale through the seventh permutation function and then passed through the second Sigmoid activation function layer to generate weights to obtain a second weighted feature map of size W×1×1.

[0103] The second weighted feature map is multiplied by the sixth feature map, and then the spatial feature information is restored through the eighth permutation function, so as to finally obtain an intermediate branch feature map containing the W-direction damage spatial information, and the size of the intermediate branch feature map is C×H×W;

[0104] 3) extracting channel feature information in the C direction from the first feature map of the input size of C×H×W through the bottom branch, the specific steps include:

[0105] First, the spatial feature information in the H and W directions is compressed by the third AvgPool and the third StdPool, and the channel feature information in the C direction is retained, thereby obtaining the tenth feature map with a size of C×1×1, and then the scale is transformed by the ninth permutation function to obtain the eleventh feature map with a size of 1×1×C;

[0106] The channel feature information in the C direction in the eleventh feature map is extracted through the third two-dimensional convolution layer of 1×K to obtain the twelfth feature map with a size of 1×1×C, and the scale is transformed by the tenth permutation function and then the weight is generated through the third Sigmoid activation function layer to obtain the third weighted feature map with a size of C×1×1;

[0107] Multiplying the third weighted feature map by the first feature map to obtain a bottom branch feature map containing channel feature information in the C direction, and having a size of C×H×W;

[0108] 4) Adding the top branch feature map, the middle branch feature map and the bottom branch feature map to obtain a fused feature map that is subsequently input into the spatial pyramid pooling layer SPPF for pyramid pooling.

[0109] In a specific embodiment, the process of processing data based on the multi-scale feature fusion module USF is:

[0110] The feature map output by the first upsampling layer in the improved neck network module is downsampled through the maximum pooling layer, and then channel-joined with the feature map output by the second upsampling layer in the improved neck network module to obtain a first fused feature map;

[0111] Specifically, in this embodiment, Figure 8 As shown, the feature map output by the spatial pyramid pooling layer SPPF in the backbone network module is used as the input of the first upsampling layer in the improved neck network module, and after upsampling processing by the first upsampling layer, a feature map for input into the maximum pooling layer is obtained; the feature map processed by the first upsampling layer, the concatenation layer Concat and the first C2f module in the neck network module is used as the input of the second upsampling layer, and after upsampling processing by the second upsampling layer, a feature map for input into the USF is obtained.

[0112] The number of channels of the first fused feature map is reduced to half of the original through the fourth 3×3 convolutional layer, and then the channel is concatenated with the feature map output by the second C2f module to obtain the second fused feature map;

[0113] After the fifth 3×3 convolutional layer, the number of channels of the second fused feature map is reduced to half of the original number, and then the second feature fused feature map with reduced channel number is upsampled and combined with Figure 8 The feature map output by the second C2f module in the neck network module Neck shown is channel-joined to obtain a third fused feature map;

[0114] After the sixth 3×3 convolutional layer, the number of channels of the third fused feature map is reduced to half of the original number, and then the third feature fused feature map with the reduced number of channels is upsampled, and then the channel is spliced ​​with the feature map output by the first upsampling layer to obtain the fourth fused feature map;

[0115] The number of channels of the fourth fused feature map is reduced to half of the original through the seventh 3×3 convolution layer, and then the number of channels is changed through the eighth 1×1 convolution layer to obtain a fused feature map for input into the first convolution module in the improved neck network module for further processing.

[0116] Specifically, in this embodiment, the multi-scale feature fusion module uses the idea of ​​Unet to downsample the high-dimensional feature map, upsample the low-dimensional feature map, and then splice the feature maps in the two parts through jump connections, thereby realizing the fusion process of feature maps of different scales.

[0117] Specifically, the use of the MCA attention mechanism can make the backbone network pay more attention to small-scale damage. After the backbone network extracts the feature map, the original YOLOv8 network model only performs feature fusion through channel splicing, which will cause a large feature loss. Therefore, a multi-scale feature fusion module USF is designed in this embodiment. By downsampling the extracted feature maps of different scales and performing jump connections, the feature loss is reduced, thereby improving the accuracy of small-scale metal surface damage detection.

[0118] In a specific embodiment, the process of preprocessing the metal surface defect data set is:

[0119] Check and remove blurry, unclear or irrelevant metal surface defect images; randomly rotate and flip the inspected metal surface defect images to increase data diversity, and resize all metal surface defect images to the input size (640x640) required by the YOLOv8 model to ensure input consistency;

[0120] Use the labeling tool LabelImg to label the defects in the metal surface defect image, and obtain the YOLO format labeling files corresponding to each image, which contain the target category and bounding box information, and ensure that the format of all labeling files is consistent and the category labels are correct.

[0121] In order to verify the effectiveness of this embodiment, GC10-DET, NEU-DET and Northeastern University steel surface defect datasets were used, which were preprocessed and divided into test set, validation set and test set in a ratio of 8:1:1, and the defects were marked as punching, welding seam, crescent, water spot, oil spot, silk spot, foreign matter, indentation, crease and waist fold. The experimental hardware environment in this embodiment is configured as NVIDIA GeForce RTX 4090D, video memory 24G, compilation environment Python3.9.16+torch2.3.0+CUDA11.3, training is carried out in the PyTorch network framework, the training cycle is 550, the initial learning rate is 0.01, and the size of the input image is 640×640. The original YOLOv8 network model and the improved YOLOv8 network model were compared experimentally to verify the improved YOLOv8 network model's ability to detect metal surface defects. The following indicators were selected to measure the prediction accuracy of the improved YOLOv8 network model, including precision P, recall R, average accuracy mAP, and floating-point operations GFLOPs. Their calculation formulas are:

[0122]

[0123] GFLOPs = frequency × number of floating point operations (FLOPs) × clock cycles (Cycles) (9)

[0124] Among them, precision measures the accuracy of the model in predicting the positive category; recall measures the coverage of the model in the positive category samples; GFLOPs represents the billion floating-point operations performed by the model per second, which is related to the speed and performance of the model. To this end, an ablation experiment is designed in this embodiment, which includes the original YOLOv8 model, a model with only the convolution module DWGConv combining deep separable convolution and ordinary convolution added, a model with only the snake convolution module DSCConv added, a model with only the MCA attention mechanism module added, a model with only the multi-scale feature fusion module USF added, and the model proposed in this embodiment. The experimental results are shown in Table 1:

[0125] Table 1:

[0126]

[0127] As can be seen from Table 1, in the ablation test, the model proposed in this embodiment, i.e., the model with the convolution module DWGConv combining deep separable convolution and ordinary convolution, the snake convolution module DSCConv, the MCA attention mechanism module and the multi-scale feature fusion module USF, has achieved significant improvements in accuracy and recall. Specifically, the accuracy reached 76.6%, an increase of 13% compared with the baseline model; the recall rate reached 73.3%, an increase of 14% compared with the baseline model. In addition, the evaluation index GFLOPs has increased from the original 8.1G to the current 8.6G. Because the model complexity has increased, the improved YOLOv8 network model can capture more complex data features. This result verifies the stability and effectiveness of the model given in this embodiment. In summary, the improved YOLOv8 network model is suitable for the detection of small-scale damage on metal surfaces.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting small-scale damage on metal surfaces based on an improved YOLOv8 network, characterized in that: The specific steps include: S1: Acquire a metal surface defect data set, and preprocess the metal surface defect data set to obtain a processed data set; S2: establishing an improved YOLOv8 network model, wherein the improved YOLOv8 network model includes an improved backbone network module, an improved neck network module and a detection head module; The improved backbone network module is used to extract defect features in the metal surface image and transmit them to the neck network module; The improved neck network module is used to perform feature fusion operation on the defect features to obtain fused features and transmit them to the detection head module; The detection head module is used to further extract the fused features and output different types of small-scale damage on the metal surface and the corresponding damage probability; S3: Training the improved YOLOv8 network model based on the processed data set to obtain a trained YOLOv8 network model; S4: Detect small-scale damage on metal surfaces based on the trained YOLOv8 network model.

2. The metal surface small-scale damage detection method based on the improved YOLOv8 network according to claim 1 is characterized in that: The improved backbone network module is improved on the basis of the original YOLOv8 network model, and the improvement is as follows: Replace each convolution module Conv in the backbone network module of the original YOLOv8 network model with a snake convolution module DSCConv; Replace the Conv in each C2f module in the backbone network module of the original YOLOv8 network model with a convolution module DWGConv that combines depthwise separable convolution and ordinary convolution; An MCA attention mechanism module is added between the fourth C2f module and the spatial pyramid pooling layer SPPF in the backbone network module of the original YOLOv8 network model.

3. The metal surface small-scale damage detection method based on the improved YOLOv8 network according to claim 2 is characterized in that: The improved neck network module is improved on the basis of the original YOLOv8 network model, and the improvement is as follows: A multi-scale feature fusion module USF is added between the second C2f module and the first convolution module Conv in the neck network module of the original YOLOv8 network model.

4. The metal surface small-scale damage detection method based on the improved YOLOv8 network according to claim 3 is characterized in that: The snake convolution module DSCConv includes a snake convolution layer DSC, a BN layer and a SiLU layer; The serpentine convolution layer DSC is used to extract small-scale damage features in the input data and transmit them to the normalization layer. The normalization layer is used to normalize the input data and transmit them to the SiLU layer. The SiLU layer is used to smooth the input data.

5. The metal surface small-scale damage detection method based on the improved YOLOv8 network according to claim 4 is characterized in that: The convolution module DWGConv that combines depthwise separable convolution and ordinary convolution includes the first convolution layer, the second normalization layer, the ReLU activation function layer, the second convolution layer, the depthwise separable convolution layer and the third convolution layer; Among them, the process of processing data based on DWGConv is: The defect features of the input data are extracted through the first convolution layer, and then the output of the first convolution layer is normalized by the second normalization layer, and then passed to the ReLu activation function layer for activation processing. The output feature maps are evenly grouped to obtain two sets of feature map sets, one of which is passed to the depthwise separable convolution layer for feature map channel feature extraction, and the other set of feature map sets is passed to the second convolution layer for feature map smoothing feature extraction. The feature maps extracted by the second convolution layer and the depthwise separable convolution layer are spliced ​​according to channels to form a new feature map. Finally, the new feature map is compressed through a 1×1 third convolution layer to complete the processing of the input data.

6. The metal surface small-scale damage detection method based on the improved YOLOv8 network according to claim 5 is characterized in that: The MCA attention mechanism module includes a top branch, a middle branch and a bottom branch; The process of data processing based on the added MCA attention mechanism module is: 1) Extracting spatial feature information in the H direction of a first feature map of size C×H×W through the top branch, wherein the first feature map is a feature map output by the backbone network module and contains spatial feature information and channel feature information of small-scale damage on the metal surface, and the specific steps include: First, the size of the first feature map is permuted by a first permutation function to obtain a second feature map with a size of H×C×W; The feature information in the W and C directions of the second feature map is compressed by the first AvgPool and the first StdPool in sequence, and the spatial feature information in the H direction is retained to obtain a third feature map with a size of H×1×1, and the scale is transformed by the second permutation function to obtain a fourth feature map with a size of 1×1×H; The spatial feature information in the H direction in the fourth feature map is extracted through the first two-dimensional convolution layer of 1×K to obtain a fifth feature map of size 1×1×H, which is then transformed in scale through the third permutation function and then passed through the first Sigmoid activation function layer to generate weights to obtain the first weighted feature map of size H×1×1; Multiplying the first weighted feature map by the first feature map, and then performing channel restoration through a fourth permutation function, to finally obtain a top branch feature map containing H-direction damage space information, and having a size of C×H×W; 2) extracting spatial feature information in the W direction from the first feature map of the input size of C×H×W through the middle branch, the specific steps include: First, the size of the first feature map is permuted by a fifth permutation function to obtain a sixth feature map with a size of W×H×C; The feature information of the H and C directions in the sixth feature map is compressed by the second AvgPool and the second StdPool in sequence, and the spatial feature information of the W direction is retained to obtain a seventh feature map of size W×1×1, and the scale is transformed by the sixth permutation function to obtain an eighth feature map of size 1×1×W; The spatial information features in the W direction of the eighth feature map are extracted through the second two-dimensional convolution layer of 1×K to obtain a ninth feature map of size 1×1×W, which is then transformed in scale through the seventh permutation function and then passed through the second Sigmoid activation function layer to generate weights to obtain a second weighted feature map of size W×1×1. The second weighted feature map is multiplied by the sixth feature map, and then the spatial feature information is restored through the eighth permutation function, so as to finally obtain an intermediate branch feature map containing the W-direction damage spatial information, and the size of the intermediate branch feature map is C×H×W; 3) extracting channel feature information in the C direction from the first feature map of the input size of C×H×W through the bottom branch, the specific steps include: First, the spatial feature information in the H and W directions is compressed by the third AvgPool and the third StdPool, and the channel feature information in the C direction is retained, thereby obtaining the tenth feature map with a size of C×1×1, and then the scale is transformed by the ninth permutation function to obtain the eleventh feature map with a size of 1×1×C; The channel feature information in the C direction in the eleventh feature map is extracted through the third two-dimensional convolution layer of 1×K to obtain the twelfth feature map with a size of 1×1×C, and the scale is transformed by the tenth permutation function and then the weight is generated through the third Sigmoid activation function layer to obtain the third weighted feature map with a size of C×1×1; Multiplying the third weighted feature map by the first feature map to obtain a bottom branch feature map containing channel feature information in the C direction, and having a size of C×H×W; 4) Adding the top branch feature map, the middle branch feature map and the bottom branch feature map to obtain a fused feature map that is subsequently input into the spatial pyramid pooling layer SPPF for pyramid pooling.

7. The metal surface small-scale damage detection method based on the improved YOLOv8 network according to claim 6 is characterized in that: The process of processing data based on the multi-scale feature fusion module USF is as follows: The feature map output by the first upsampling layer in the improved neck network module is downsampled through the maximum pooling layer, and then channel-joined with the feature map output by the second upsampling layer in the improved neck network module to obtain a first fused feature map; The number of channels of the first fused feature map is reduced through the fourth 3×3 convolutional layer, and then the channel is spliced ​​with the feature map output by the second C2f module to obtain the second fused feature map; The number of channels of the second fused feature map is reduced through the fifth 3×3 convolutional layer, and then the second feature fused feature map with the reduced number of channels is upsampled and then channel-joined with the feature map output by the second C2f module to obtain the third fused feature map; The number of channels of the third fused feature map is reduced through the sixth 3×3 convolutional layer, and then the third feature fused feature map with the reduced number of channels is upsampled, and then the channel is spliced ​​with the feature map output by the first upsampling layer to obtain the fourth fused feature map; The number of channels of the fourth fused feature map is reduced through the seventh 3×3 convolution layer, and then the number of channels is changed through the eighth 1×1 convolution layer to obtain a fused feature map for input into the first convolution module in the improved neck network module for further processing.

Images