Methods for Improving YOLOv8 Networks and Their Application in Strip Surface Defect Detection

Abstract

This invention relates to the field of defect detection technology, specifically to a method for improving the YOLOv8 network and its application in strip steel surface defect detection. Based on YOLOv8, this invention first proposes an improved coordinate attention mechanism, the three-channel coordinate attention mechanism TCCA. Addressing the overly simplistic offset mask generation method in DCNv2, which leads to insufficient deformation modeling capabilities, this invention proposes deeply embedding TCCA into DCNv2. Furthermore, this invention introduces a global attention mechanism to improve the model's ability to extract global features. The invention also introduces a BiFPN structure and dynamic serpentine convolution to enhance the model's multi-scale feature fusion capabilities. MDPioU replaces the original loss function of YOLOv8, solving the problem of loss of effectiveness due to identical aspect ratios in predicted bounding boxes, while increasing convergence speed and localization ability. Extensive comparative and ablation experiments on the NEU-DET dataset demonstrate that the improved algorithm of this invention achieves higher defect detection accuracy and faster speed.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, specifically to a method for improving the YOLOv8 network and its application in the detection of defects on the surface of steel strip. Background Technology

[0002] Strip steel is a crucial industrial material in manufacturing. During industrial production, its large surface area, improper handling, and harsh industrial environments inevitably lead to surface defects (such as scratches, oil stains, cracks, and oxidation). These defects directly impact finished product quality, becoming a weak link in the quality control of downstream product lifespan and safety performance. Early methods for detecting surface defects in strip steel primarily relied on manual inspection. Manual inspection requires highly skilled personnel, is labor-intensive, inefficient, and lacks accuracy. Therefore, automating the inspection of strip steel surfaces to improve accuracy and efficiency is a pressing challenge for the manufacturing industry.

[0003] In recent years, deep learning combined with object detection technology has advanced rapidly, and many advanced object detection algorithms have been widely introduced into defect detection with remarkable results. In the field of object detection, this method can be roughly divided into two categories: one is one-stage object detection algorithms, represented by the YOLO series and SSD, and the other is two-stage object detection algorithms, represented by the R-CNN series.

[0004] Two-stage object detection algorithms refer to object detection that can be divided into two stages. First, a feature extractor is used to extract features from the input image, thereby generating a Region Proposal Network (RPN) and candidate anchor points. Then, these anchor points are further classified and filtered, and the content of the recommended regions is regressed. In the two-stage algorithm, the accuracy can be effectively improved by generating candidate anchor points first. Tang Yong et al. used Faster R-CNN for component identification and defect detection in power transmission lines; Li Hui et al. proposed a corrosion detection method that combines a deep learning-based Faster-RCNN object detection model with HSI color features of corrosion. However, by continuously spending time filtering anchor points, the detection speed cannot meet the real-time requirements. One-stage object detection algorithms, such as the YOLO series, abandon the use of feature extraction networks to generate candidate anchor points and use preset fixed anchor points to directly perform localization and classification tasks through neural networks, balancing speed and accuracy. Cheng Jingyi et al. proposed an improved YOLOv3 model for metal surface defect detection, which fuses shallow and deep features to improve detection accuracy, but ignores the shape and size of the defect target. Lu Junzhe et al. proposed combining the deformable convolutional network (DCN) with the original YOLOv5 to increase sensitivity to the shape and size of defect targets. However, they did not consider the similarity between different defects and the weak ability of DCN to extract features from small defect targets. Leng Hao et al. introduced Alpha-IoU to replace CIoU in the original YOLOv7 network model to optimize the loss function, increasing the target localization ability. However, they ignored the fact that when the predicted bounding box and the ground reality bounding box have the same aspect ratio but different width and height values, the above loss function used for bounding box regression will lose some effectiveness, which will limit the convergence speed and accuracy.

[0005] Chinese patent document 202310739468.1 discloses a method and system for detecting small target defects on the surface of strip steel based on super-resolution and YOLOv8. The method includes: acquiring original training data; preprocessing the original training data using the SRGAN algorithm; constructing a preprocessed image dataset based on the preprocessed image data and performing image data augmentation; constructing a small target defect detection model on the surface of strip steel based on the YOLOv8 algorithm, improving the model using normalized Wasserstein distance, and training the improved model using the augmented training dataset; acquiring test data and inputting it into the improved model for defect detection to obtain the defect detection result. This invention utilizes SRGAN to preprocess the original image data, improving image resolution; and then optimizes the YOLOv8-based small target defect detection model by introducing a new metric, NWD, thereby improving the detection accuracy of small target defects. However, the following problems still exist: 1. The above-mentioned existing technical solutions focus on changing the image SRGAN network to improve image resolution in order to improve image quality, but ignore the characteristics of the strip steel surface itself, such as small defects, complex texture structure, thin and diffuse branches. While improving the overall image resolution, the detection network cannot fully obtain the feature information of the above-mentioned strip steel defects, thus failing to accurately capture the defects on the strip steel surface.

[0006] 2. The existing technical solution mentioned above introduces a zoom loss function into the loss function, which can reduce the overall contribution of negative samples to the loss function. However, reducing its impact on the loss function cannot solve the problem or may lead to the loss of some feature information of negative samples.

[0007] The MPDioU used in this invention can enhance the regression ability of the bounding box of steel strip defects with large differences in shape and aspect ratio, reduce harmful gradients, improve fitting speed, and improve the generalization ability and accuracy of the model.

[0008] To address existing problems, this invention, based on YOLOv8, first proposes an improved coordinate attention mechanism, namely Three-Channel Coordinate Attention (TCCA). Recognizing the overly simplistic offset mask generation method in DCNv2, which leads to insufficient deformation modeling capabilities, this invention proposes deeply embedding TCCA into DCNv2. Furthermore, this invention introduces a global attention mechanism to enhance the model's ability to extract global features. A BiFPN structure and dynamic serpentine convolution are introduced to enhance the model's multi-scale feature fusion capabilities. MDPioU replaces the original loss function of YOLOv8, solving the problem of loss of effectiveness due to the same aspect ratio of predicted bounding boxes, while increasing convergence speed and localization ability. Extensive comparative and ablation experiments on the NEU-DET dataset demonstrate that the improved algorithm of this invention has higher defect detection accuracy and faster speed. Summary of the Invention

[0009] The purpose of this invention is to provide an improved method for YOLOv8 network and its application in strip steel surface defect detection. The improved YOLOv8 algorithm achieves a mean average accuracy (mAP) of 80.2%, which is 5.8% higher than the original YOLOv8n algorithm. The precision reaches 78.3%, which is 8.9% higher than the original model. The algorithm's detection speed reaches 61.73 FPS. The results show that compared with the original network model and other mainstream detection models, the improved algorithm has higher detection accuracy and faster detection speed, and can better meet the actual detection needs of strip steel.

[0010] To achieve the above-mentioned technical objectives and effects, the present invention is implemented through the following technical solution:

[0011] A method for improving YOLOv8 networks includes the following steps:

[0012] S1: The deformable convolution DCNv2 replaces the standard convolution in C2f of the original YOLOv8 model, enabling the capture of complex details on the strip with regular contour texture and scale. An offset is introduced in DCNv2 to avoid exceeding the range of the target defect to be identified. The three-channel coordinate attention mechanism TCCA is used to calculate the mask offset, which enhances the network's ability to express the features of irregular defects and complex targets.

[0013] S2: Introduce the Global Attention (GAM) mechanism to reduce the diffusion of local image information and amplify global interaction to improve the performance of neural network feature representation and avoid missing defects at the edge of the strip surface and defects with large spans.

[0014] S3: The Bidirectional Feature Pyramid Network (BiFPN) is introduced to enhance multi-scale feature fusion and avoid the loss of feature information as the number of feature extraction network layers is stacked.

[0015] S4: In the BiFPN structure, dynamic serpentine convolution DSC is combined to accurately capture thin and diffusely branched tree and tubular structures, adaptively focusing on slender and tortuous local structures, and dynamically adapting to the special texture structure of the strip surface, so as to better perceive the key features of small targets and improve the accuracy of detecting the location of small defects on the strip surface.

[0016] S5: The bounding box regression loss MPDioU is used instead of CIoU in the original YOLOv8 model to enhance the regression ability of the bounding box of steel defects with large differences in shape and aspect ratio, reduce harmful gradients, improve the fitting speed, and improve the generalization ability and accuracy of the model; the resulting improved structure based on YOLOv8 network.

[0017] Furthermore, step S1 specifically includes:

[0018] The calculation process of DCNv2 includes: Let K = 9,

[0019] P k ∈{(-1,-1), (-1,0), ..., (1,1)}(1)

[0020] Equation (1) defines a 3×3 convolution kernel with an expansion coefficient of 1.

[0021]

[0022] In Equation 2, x(p) and y(p) represent the features at position p in the input feature map x and the output feature map y, respectively, and Δp k and Δm k These are the learnable offset and modulation scalar at the k-th position, respectively. As shown in Equation 2, DCNv2 not only introduces an offset for each point to break the fixed geometric structure of traditional CNN convolutional modules, but also proposes a feature simulation scheme to guide the training of deformable convolutional networks. That is, a weight term is added to penalize the output features point-to-point, thereby avoiding the situation where DCNv1 exceeds the range of the target defect to obtain a larger receptive field, resulting in low correlation between defect features and extracted features.

[0023] The three-channel coordinate attention mechanism TCCA is used to calculate the mask offset. TCCA adds a global adaptive pooling operation on top of the two one-dimensional global pooling operations performed by CA on the input image, turning it into a three-channel system. In view of the fact that the localization information obtained by CA is not precise enough, the two one-dimensional coordinates are weighted in the fifth layer, which effectively enhances the network's ability to express the features of irregular, defective and complex targets.

[0024] Furthermore, step S2 specifically includes:

[0025] A Global Attention (GAM) mechanism is introduced to improve the performance of neural network feature representation by reducing local image information diffusion and amplifying global interactions. GAM employs a channel-spatial attention mechanism in its sequential module design. The channel attention submodule uses 3D permutation to preserve information in three dimensions, while cross-dimensional attention is amplified through a multilayer perceptron to enhance the spatial dependence of cross-dimensional channels. The spatial attention submodule uses a large 7×7 convolutional kernel for convolution, which facilitates the fusion of spatial information in large-size images and the extraction of global features. It optimizes the extraction of features from the edges of the strip surface and defects with large spans, increasing the accuracy of locating irregular defects on large-size strips and demonstrating superior generalization ability for defects of different sizes.

[0026] Furthermore, step S3 specifically includes:

[0027] The introduction of the Bidirectional Feature Pyramid Network (BiFPN) reduces feature information loss and enhances multi-scale feature fusion. BiFPN incorporates operations such as feature adjustment, feature selection, and bidirectional feature propagation. By continuously adjusting and optimizing the weight parameters of the representation in each iteration, a large number of relatively fitting defect features are obtained. Then, by utilizing the different priorities and confidence levels of these features, the best-fitting features are selected dynamically and quickly. The bidirectional structure of BiFPN allows feature information to flow between different levels from top to bottom and from bottom to top, enhancing the semantic and location information of the features.

[0028] Furthermore, step S4 specifically includes:

[0029] In the BiFPN structure, Dynamic Snake Convolution (DSC) is combined to accurately capture thin and diffusely branching tree-like and tubular structures, adaptively focusing on slender and tortuous local structures. The DSC uses a 9-size convolution kernel, straightening the standard convolution kernel along the X and Y axes. Taking the X direction as an example, the specific location in each grid K is represented as follows:

[0030] K i+c = (x i+c y i+c (3)

[0031] In Equation 3, K is a given standard two-dimensional convolution coordinate set, K i =(x i y i Using the center coordinates as an integer, a 3×3 convolution kernel with a dilation of 1 can be represented as: K = {(x-1, y-1), (x-1, y), ..., (x+1, y+1)}, where

[0032] c = {0, 1, 2, 3, 4} represents the distance to the central network. Each grid position K in the convolution kernel K... i+c The selection of each element is an accumulation process, and its position depends on the position of the previous network. The value in the X-axis direction becomes:

[0033]

[0034] The value in the Y-axis direction becomes:

[0035]

[0036] The offset is continuously adjusted to ensure that the convolution kernel presents a linear structure and covers a 9×9 pixel area. By referencing DSC, the special texture structure of the strip surface is dynamically adapted to better perceive the key features of small targets and improve the accuracy of detecting the location of minute defects on the strip surface.

[0037] Furthermore, step S5 specifically includes:

[0038] Bounding box regression is a crucial step in object detection, and it's a vital step in defect location. The original YOLOv8 model uses CIoU, which proposes incorporating the center point distance and aspect ratio into the loss calculation, as shown in Equation 3:

[0039]

[0040] in w gt w represents the width and height of the ground reality bounding box. prd and h prd This represents the predicted width and height of the bounding box. However, the aspect ratio in the CioU formula is a relative value, not an absolute value. This makes it less effective for targets with significant differences in aspect ratio. Furthermore, the defect structures on the strip surface are complex, with inconsistent aspect ratios, leading to lower positioning accuracy for defects with large shape differences, such as patches and scratches. Therefore, the bounding box regression loss MPDioU is used instead of CIoU, and its formula is shown in Equation 4:

[0041]

[0042] in

[0043]

[0044] As can be seen from the formula, MPDioU uses absolute values ​​of length and width, different from CIoU, fully exploring the geometric features of the bounding box rectangle. It is a novel metric for bounding box overlap based on minimum point distance. It not only considers many factors affecting bounding box regression calculations, such as center point distance and aspect ratio, but also simplifies the calculation process, greatly shortening the model convergence and training cycle; the final result is an improved structure based on the YOLOv8 network.

[0045] On the other hand, this invention proposes the application of the above-mentioned improved YOLOv8 network in the detection of surface defects in strip steel.

[0046] Furthermore, the improved YOLOv8 network method for detecting surface defects in strip steel includes the following steps:

[0047] Step 1: Take photos of the strip defect detection scenario using an industrial camera;

[0048] Step 2: Select and label the image data, and divide the labeled data into training set, test set and validation set in a ratio of 8:1:1;

[0049] Step 3: Perform data augmentation on the training set data, including but not limited to random flipping and cropping;

[0050] Step 4: Training and experimental exploration revealed that the offset mask generation method in DCNv2 was too simple. The coordinate attention mechanism was improved to a three-channel coordinate attention mechanism and embedded into the DCNv2 network.

[0051] Step 5: Incorporate the GAM global attention mechanism to enhance the overall network feature extraction global interaction capability;

[0052] Step 6: Introduce the BiFPN structure and fuse it with dynamic snake convolution to enhance feature fusion capabilities;

[0053] Step 7: Replace the MPDioU loss function;

[0054] Step 8: Train the network model using the improved YOLO network;

[0055] Step 9: Export the trained model file, test it, continuously adjust the parameters based on the test results, and select the parameters with the best effect to train the detection network.

[0056] The beneficial effects of this invention are:

[0057] This invention replaces the standard convolution in the original YOLOv8 model with deformable convolution DCNv2 to learn complex details and scale patterns on the strip surface. DCNv2 introduces offset and modulation scalars to flexibly adapt to the shape and location of different target defects. Furthermore, a three-channel coordinate attention mechanism (TCCA) is employed, which enhances the network's ability to learn irregular defects by performing weighted mask shifting on the input feature map. The improved model can more accurately capture small target defects on the strip surface.

[0058] This invention introduces a Global Attention (GAM) mechanism to improve the model's global feature perception capability. GAM, through the design of channel-spatial attention submodules, reduces the diffusion of local image information and amplifies global interactions, thereby improving the network's feature representation performance. This mechanism helps avoid missing defects at the edges of the strip surface and defects with large spans, making defect detection more comprehensive and accurate.

[0059] This invention employs a Bidirectional Feature Pyramid Network (BiFPN) to enhance multi-scale feature fusion capabilities. Replacing the original network with BiFPN facilitates the interaction, transmission, and fusion of feature information between the deep and shallow layers of the entire neural network, thereby improving the algorithm's performance in detecting small defects. Simultaneously, to capture elongated and tortuous local defects within the special texture structure of the strip steel surface, this scheme also introduces Dynamic Snake Convolution (DSC), which adaptively focuses on these local structures, improving the detection accuracy of minute defect locations.

[0060] This invention employs the MPDioU loss function to replace the CIoU loss function in the original model. Compared to CIoU, MPDioU considers more geometric features of the bounding box, especially the absolute values ​​of its length and width. Through multiple experimental comparisons, this invention demonstrates that the improved loss function, using MPDioU for regression calculation of the model's bounding box, focuses on enhancing the regression ability for steel strip defect bounding boxes with large differences in shape and aspect ratio. This reduces harmful gradients, improves fitting speed, and enhances the model's generalization ability and accuracy.

[0061] In summary, this invention improves the accuracy and efficiency of strip steel surface defect detection, better capturing complex details and irregular defects, enhancing the detection capability for small target defects, and strengthening the model's global perception and multi-scale feature fusion capabilities. Furthermore, the introduction of an adaptive bounding box regression loss function further improves the accuracy and generalization ability of defect location.

[0062] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0063] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a schematic diagram of the original YOLOv8 model structure;

[0065] Figure 2 A schematic diagram of the improved C2f model structure;

[0066] Figure 3 This is a schematic diagram of the TCCA module structure;

[0067] Figure 4 This is a schematic diagram of the GAM structure;

[0068] Figure 5 This is a schematic diagram of a bidirectional feature pyramid network structure;

[0069] Figure 6 This is a schematic diagram of a dynamic serpentine convolution structure;

[0070] Figure 7 A schematic diagram of the improved YOLOv8 network structure;

[0071] Figure 8 A schematic diagram illustrating the detection performance of the algorithm. Detailed Implementation

[0072] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0073] Example 1

[0074] The method for improving the YOLOv8 network described in this embodiment includes the following steps:

[0075] S1: The deformable convolution DCNv2 replaces the standard convolution in C2f of the original YOLOv8 model, enabling the capture of complex details on the strip with regular contour texture and scale. An offset is introduced in DCNv2 to avoid exceeding the range of the target defect to be identified. The three-channel coordinate attention mechanism TCCA is used to calculate the mask offset, which enhances the network's ability to express the features of irregular defects and complex targets.

[0076] S2: Introduce the Global Attention (GAM) mechanism to reduce the diffusion of local image information and amplify global interaction to improve the performance of neural network feature representation and avoid missing defects at the edge of the strip surface and defects with large spans.

[0077] S3: The Bidirectional Feature Pyramid Network (BiFPN) is introduced to enhance multi-scale feature fusion and avoid the loss of feature information as the number of feature extraction network layers is stacked.

[0078] S4: In the BiFPN structure, dynamic serpentine convolution DSC is combined to accurately capture thin and diffusely branched tree and tubular structures, adaptively focusing on slender and tortuous local structures, and dynamically adapting to the special texture structure of the strip surface, so as to better perceive the key features of small targets and improve the accuracy of detecting the location of small defects on the strip surface.

[0079] S5: The bounding box regression loss MPDioU is used instead of CIoU in the original YOLOv8 model to enhance the regression ability of the bounding box of steel defects with large differences in shape and aspect ratio, reduce harmful gradients, improve the fitting speed, and improve the generalization ability and accuracy of the model; the resulting improved structure based on YOLOv8 network.

[0080] On the other hand, this invention proposes the application of the above-mentioned improved YOLOv8 network in the detection of surface defects in strip steel.

[0081] In this embodiment, the improved YOLOv8 network method for detecting surface defects in strip steel includes the following steps:

[0082] Step 1: Take photos of the strip defect detection scenario using an industrial camera;

[0083] Step 2: Select and label the image data, and divide the labeled data into training set, test set and validation set in a ratio of 8:1:1;

[0084] Step 3: Perform data augmentation on the training set data, including but not limited to random flipping and cropping;

[0085] Step 4: Training and experimental exploration revealed that the offset mask generation method in DCNv2 was too simple. The coordinate attention mechanism was improved to a three-channel coordinate attention mechanism and embedded into the DCNv2 network.

[0086] Step 5: Incorporate the GAM global attention mechanism to enhance the overall network feature extraction global interaction capability;

[0087] Step 6: Introduce the BiFPN structure and fuse it with dynamic snake convolution to enhance feature fusion capabilities;

[0088] Step 7: Replace the MPDioU loss function;

[0089] Step 8: Train the network model using the improved YOLO network;

[0090] Step 9: Export the trained model file, test it, continuously adjust the parameters based on the test results, and select the parameters with the best effect to train the detection network.

[0091] Example 2

[0092] Existing YOLOv8 model networks such as Figure 1 As shown, the improved algorithm based on YOLOv8 includes:

[0093] Improvements to Deformable Convolutional Networks

[0094] In the Northeastern University's dataset on surface defect detection of steel strips, the surface area of ​​the steel strips is large. The traditional CNN module convolutional kernels in YOLOv8, limited by fixed geometric structures, struggle to capture complex details, irregular contours, and inconsistent scales of defects on the steel strips, resulting in inherent drawbacks in feature extraction. Therefore, this invention replaces the standard convolution in the eighth layer (C2f) of the original model's backbone network with Deformable Convolutional Networks version 2 (DCNv2). The replaced structure is as follows: Figure 2 As shown.

[0095] The calculation process for DCNv2 is similar to that for DCNv1. Let K = 9.

[0096] P k ∈{(-1,-1), (-1,0), ..., (1,1)}(1)

[0097] Equation (1) defines a 3×3 convolution kernel with an expansion coefficient of 1.

[0098]

[0099] In Equation 2, x(p) and y(p) represent the features at position p in the input feature map x and the output feature map y, respectively, and Δp k and Δm k These are the learnable offset and modulation scalar at the k-th position, respectively. As shown in Equation 2, DCNv2 not only introduces an offset for each point to break the fixed geometric structure of traditional CNN convolutional modules, but also proposes a feature simulation scheme to guide the training of deformable convolutional networks. That is, a weight term is added to penalize the output features point-to-point, thereby avoiding the situation where DCNv1 exceeds the range of the target defect to obtain a larger receptive field, resulting in low correlation between defect features and extracted features.

[0100] DCNv2 introduced offsets, but the offset calculation is generated by the input feature map and another standard convolution, which is relatively simple and has relatively weak feature extraction capabilities. Therefore, this invention uses a three-channel coordinate attention (TCCA) mechanism to calculate the mask offset, with the structure as follows: Figure 3 As shown.

[0101] TCCA is based on classic CA. However, considering the irregular contours and dimensions of surface defects in strip steel, simply using CA to aggregate features in two spatial directions and capture distance and position information in different directions is insufficiently precise. The improved TCCA structure is as follows: Figure 3 As shown, TCCA adds a global adaptive pooling operation to the input image after CA performs two one-dimensional global pooling operations, turning it into a three-channel system. To address the insufficient precision of the localization information obtained by CA, the two one-dimensional coordinates are weighted in the fifth layer, effectively enhancing the network's ability to express the features of irregular, defective, and complex targets.

[0102] Introducing a global attention mechanism

[0103] To address the challenges in strip steel defect inspection, common defects such as rolled oxide scale (RS) and patches (Pa) are characterized by irregular blocky shapes, uneven density across the strip surface, and varying lengths, widths, and depths. Figure 4 As shown, the original YOLOv8 model prioritizes all spatial, dimensional, and channel channels equally when extracting defect features. This results in an incomplete and imprecise representation of defect features, leading to missed detections of defects at the edges of the strip surface and defects with large spans, which is detrimental to improving model accuracy. To address this issue, this invention introduces a Global Attention Mechanism (GAM). The GAM structure diagram is shown below. Figure 4 As shown, the performance of neural network feature representation is improved by reducing the diffusion of local image information and amplifying global interactions.

[0104] GAM employs a channel-spatial attention mechanism in its sequential module design. The channel attention submodule uses 3D permutation to preserve information across three dimensions, while cross-dimensional attention is amplified through a multilayer perceptron to enhance the spatial dependence of cross-dimensional channels. The spatial attention submodule uses a large 7×7 convolutional kernel for convolution, which facilitates the fusion of spatial information in large-size images and the extraction of global features.

[0105] This attention mechanism optimizes the extraction of defect features at the edges of the strip surface and across large spans, increasing the accuracy of locating irregular defects on large-sized strips and making defects of different sizes more advantageous in generalization.

[0106] Introducing a bidirectional feature pyramid network

[0107] As the number of layers in the feature extraction network increases, especially for defect detection with complex textures, feature information is lost to some extent, significantly reducing detection accuracy. Therefore, this invention introduces a Bi-directional Feature Pyramid Network (BiFPN) to enhance multi-scale feature fusion and reduce feature information loss. The structure is as follows: Figure 5 As shown.

[0108] BiFPN introduces operations such as feature adjustment, feature selection, and bidirectional feature propagation. By continuously adjusting and optimizing the weight parameters of the representation in each iteration, a large number of relatively fitting defect features are obtained. Then, by utilizing the different priorities and confidence levels of these features, the features with better fit are selected dynamically and quickly. The bidirectional structure of BiFPN allows feature information to flow between different levels from top to bottom and from bottom to top, enhancing the semantic and location information of the features.

[0109] This invention improves the original feature fusion network by studying the results of multiple training iterations of the YOLOv8 feature fusion network and the head network. Replacing the original network with the aforementioned BiFPN facilitates the interaction, transmission, and fusion of feature information between the deep and shallow layers of the entire neural network, thereby improving the algorithm network's performance in detecting small defective targets.

[0110] Introducing dynamic snake convolution

[0111] Feature fusion networks introduce BiFPN to achieve multi-scale fusion, enhancing the interaction and transmission of complex and diverse feature information such as texture and structure of defects. However, it neglects the special characteristics of the defect structure itself. Common defects such as scratches (Sc) and cracks (Cr) exhibit varying depths on the strip surface, and cracks are characterized by diffuse, minute branches, thin local structures, and variable global morphology. Figure 7 As shown, this leads to the neglect of small target defect feature information, resulting in a decrease in the detection accuracy of defects with the aforementioned features and the occurrence of missed detections. Therefore, this invention combines Dynamic Snake Convolution (DSC) in the BiFPN structure to accurately capture thin and diffusely branched tree-like and tubular structures, adaptively focusing on slender and tortuous local structures, as shown in the figure. Figure 6As shown. DSC uses a convolutional kernel of size 9, straightening the standard convolutional kernel upwards along the X and Y axes. Taking the X direction as an example, the specific position in each grid K is represented as follows:

[0112] K i+c = (x i+c y i+c (3)

[0113] In Equation 3, K is a given standard two-dimensional convolution coordinate set, K i =(x i y i Using the center coordinates as an integer, a 3×3 convolution kernel with a dilation of 1 can be represented as: K = {(x-1, y-1), (x-1, y), ..., (x+1, y+1)}, where

[0114] c = {0, 1, 2, 3, 4} represents the distance to the central network. Each grid position K in the convolution kernel K... i+c The selection of each element is an accumulation process, and its position depends on the position of the previous network. The value in the X-axis direction becomes:

[0115]

[0116] The value in the Y-axis direction becomes:

[0117]

[0118] By continuously adjusting the offset, this invention ensures that the convolution kernel presents a linear structure and covers a 9×9 pixel area. This invention utilizes DSC to dynamically adapt to the special texture structure of the strip surface, thereby better perceiving the key features of small targets and improving the accuracy of detecting the location of minute defects on the strip surface.

[0119] Optimize the loss function

[0120] Bounding box regression is a crucial step in object detection, and it's a vital step in defect location. The original YOLOv8 model uses CIoU, which proposes incorporating the center point distance and aspect ratio into the loss calculation, as shown in Equation 3:

[0121]

[0122] in w gt and h gt w represents the width and height of the ground reality bounding box. prd and h prdThis represents the predicted bounding box width and height. However, the aspect ratio in the CioU formula is a relative value, not an absolute value. This makes it less effective for targets with significant differences in aspect ratio. Furthermore, the surface defects on strip steel have complex structures and inconsistent aspect ratios, leading to lower positioning accuracy for defects with large shape differences, such as patches and scratches. This invention employs an effective and accurate bounding box regression loss: MPDioU, whose formula is shown in Equation 4:

[0123]

[0124] in

[0125]

[0126] As can be seen from the formula, MPDioU uses absolute values ​​of length and width, unlike CIoU, to fully explore the geometric features of the bounding box rectangle. It is a novel measure of bounding box overlap based on minimum point distance. It not only considers many factors affecting bounding box regression calculations, such as center point distance and aspect ratio, but also simplifies the calculation process, significantly shortening the model convergence and training cycle. Through multiple experimental comparisons, this invention demonstrates that the improved loss function, using MPDIoU for model bounding box regression calculations, focuses on enhancing the regression ability for steel strip defect bounding boxes with large differences in shape and aspect ratio. This reduces harmful gradients, improves fitting speed, and enhances the model's generalization ability and accuracy.

[0127] Combining the above improvements, the final improved structure based on the YOLOv8 network is as follows: Figure 7 As shown.

[0128] Example 3

[0129] Experimental Dataset

[0130] This invention improves the YOLOv8 algorithm using the open-source dataset NEU-DET, created by Professor Song Kechen's team at Northeastern University. NEU-DET is a dataset of surface defects in steel strips. This dataset consists of 1800 grayscale images with a resolution of 200×200 pixels, collecting six common defects on steel surfaces: rolled-in scale (RS), patches (Pa), cracking (Cr), pitted surface (Ps), inclusions (In), and scratches (Sc), with 300 images for each defect. The 1800 images are divided into training, validation, and test sets in an 8:1:1 ratio, with 1440 images in the training set, 180 in the validation set, and 180 in the test set. To magnify small defects on the steel strip surface and accurately extract defect features, data augmentation techniques are used to adjust the resolution of the input dataset to 640×640 pixels.

[0131] Experimental setup and evaluation criteria

[0132] The improved experiments of this invention used an Intel(R) Xeon(R) Platinum 8255C CPU @ 2.50GHz processor, an NVIDIA GeForce RTX 3080 graphics card with 10GB of RAM, and Ubuntu 20.04 as the operating system. The entire network model was deployed in Python 3.8, using a deep learning framework based on PyTorch 1.11.0, with CUDA version 11.3. In the experiments of this invention, the number of training epochs was set to 300, the batch size to 32, the learning rate to 0.001, the weight decay coefficient to 0.0007, and the momentum to 0.937.

[0133] The main purpose of this invention in improving the YOLOv8 network is to enhance the detection accuracy of the network model. This invention uses AP, mAP, Precision, FPS, and Params as evaluation metrics for the improved algorithm. AP is the average accuracy for a single class, and mAP is the average accuracy for all classes; these are key indicators for measuring the overall performance of the object detection network and are also the main standards referenced in this invention. Their calculation formulas are shown in Equation 5.

[0134]

[0135] In this invention, c represents the type of defect, c = 6, i represents the number of detections, and AP is the average accuracy of identifying a single defect. Precision characterizes the detection accuracy of the network model, and the calculation formula is shown in Equation 6.

[0136]

[0137] TP represents the true positive instance, where the defect to be detected was correctly detected; FP represents the false positive instance, where the non-defect target was correctly detected. FPS is the number of images processed or detected per second, used to measure the speed at which the network model detects defects. Parms is the amount of memory required for the network model.

[0138] ablation experiment

[0139] The ablation experiments were conducted to verify whether the combined use of different modules described above could truly improve the performance of the detection network. Therefore, six sets of ablation experiments were performed in this invention. The experimental results are shown in the table. This invention uses YOLOv8 as the baseline model. As can be easily seen in Table 1, the improvement methods proposed above all improve the detection performance of the model to varying degrees.

[0140] The table compares the results of the first and second experimental groups. Replacing the original model's CioU with MDPioU resulted in a mAP of 74.9% for group 2, an improvement of 0.5%, and an accuracy improvement of 4.8% from 69.4% to 74.2%. These results demonstrate that the MDPioU module can effectively and accurately calculate the bounding box regression loss, reduce harmful gradients, and thus improve the network's generalization ability and accuracy.

[0141] The third set of experiments applied global attention radix in the feature extraction network, which improved mAP by 0.5% compared to the previous set. The accuracy was improved by 0.6%. The experimental results show that the GAM of the present invention can comprehensively extract the features of surface defects in strip steel, making the defect localization more accurate.

[0142] The fourth set of experiments innovatively incorporated dynamic serpentine convolution into the feature fusion network, achieving a 3.3% improvement in mAP compared to the previous set. The accuracy was improved by 2.7%. The experimental results show that applying DSC can dynamically acquire key features of small targets and optimize the algorithm for capturing small defect features on the strip surface.

[0143] The fifth group of experiments added a bidirectional feature pyramid network to the feature fusion network, which improved the accuracy by 1.7% and reduced the number of parameters by 1.87M compared to the previous group. The experimental results show that replacing the original feature fusion network with BiFPN not only reduced the loss of feature information and improved the detection accuracy, but also accelerated the defect detection speed.

[0144] The sixth set of experiments used the improved deformable convolutional network mentioned above in the feature extraction network, achieving an mAP of 80.2%. Experimental results show that the improved DCNv2, replacing the standard convolution of the original model, strengthens the connection between extracted features and defect features, effectively enhancing the network's representation ability of the learned target. Comparing the experimental results of the improved detection network proposed in this invention with the first set of original YOLOv8 models, the mAP was improved by 5.8%, the accuracy improved by 8.9%, and although the number of parameters increased slightly by 1.72M, the detection speed still outperforms current mainstream algorithms.

[0145] Overall, six sets of multiple ablation experiments demonstrate that the improved YOLOv8 algorithm of this invention effectively improves the accuracy of the entire model, indicating good coupling between the six different modules. Experimental results show that the improved YOLOv8 performs significantly better in detection performance than the original YOLOv8.

[0146] Table 1 Ablation Experiment Results

[0147]

[0148]

[0149] contrast

[0150] To demonstrate the advantages of the improved YOLOv8 network in detecting surface defects in strip steel, it was compared with several mainstream algorithms, such as YOLOv3, YOLOv5, YOLOv6, YOLOv7, and YOLOv8, on the same dataset. The experimental results are shown in Table 2.

[0151] Table 2. Algorithm Comparison Experiment Results

[0152]

[0153]

[0154] As shown in Table 2, the improved YOLOv8 algorithm in this invention achieves a 15.3% improvement in defect detection accuracy and a 14.6% improvement in precision compared to YOLOv3-tiny. Although YOLOv3-tiny has an advantage in detection speed (FPS), this is achieved at the cost of a large number of parameters and lower precision. The algorithm in this invention is 6.5% faster and 12.9% more accurate than YOLOv5, the most widely used defect detection algorithm currently available. Compared to the recently popular improved defect detection model YOLOv7, the algorithm in this invention not only improves detection speed by 39%, but also achieves improvements in accuracy and precision by 5.9% and 14.2%, respectively.

[0155] The detection results of the above algorithm for each defect category are listed in a table. Figure 8 To improve the detection effect of the algorithm, the chart shows that the improved algorithm of this invention has improved the detection accuracy by 8.8%, 10.2%, and 8% respectively compared with YOLOv8 for crack (Cr), rolling oxide scale (RS), and pitting surface (Ps) defects. This indicates that the algorithm of this invention has a very good improvement in the detection effect of defects with irregular geometric dimensions and complex structure on the surface of strip steel, and also verifies the effectiveness of the improved method of this invention.

[0156] In summary, this invention proposes a method for detecting surface defects in strip steel based on an improved YOLOv8 network. The improved coordinate attention mechanism is a three-channel coordinate attention mechanism, which is combined with variable convolution to replace the original C2f convolution module. A global attention mechanism is introduced, and BiFPN and dynamic serpentine convolution are incorporated into the feature fusion network. The loss function is replaced with MDPioU to improve the overall detection performance of the model. Extensive ablation experiments and comparative experiments demonstrate that this improved algorithm can effectively solve the problems of false positives and false negatives in strip steel defect detection, as well as slow detection speed. Furthermore, the improved modules of this invention are universal, and different modules are coupled together. Compared with current mainstream algorithms, it exhibits superior detection accuracy and defect recognition accuracy, and demonstrates excellent performance in terms of detection speed and parameter size.

[0157] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for improving YOLOv8 networks, characterized in that, Includes the following steps: S1: The deformable convolution DCNv2 replaces the standard convolution in C2f of the original YOLOv8 model, enabling the capture of complex details, regular contour textures, and scale regularities on the strip. An offset is introduced in DCNv2 to avoid exceeding the range of the target defects to be identified. The three-channel coordinate attention mechanism TCCA is used to calculate the mask offset, which enhances the network's ability to express the features of complex targets with irregular defects. S2: Introduce the Global Attention (GAM) mechanism to reduce the diffusion of local image information and amplify global interaction to improve the performance of neural network feature representation and avoid missing defects at the edge of the strip surface and defects with large spans. S3: The Bidirectional Feature Pyramid Network (BiFPN) is introduced to enhance multi-scale feature fusion and avoid the loss of feature information as the number of feature extraction network layers is stacked. S4: In the BiFPN structure, dynamic serpentine convolution DSC is combined to accurately capture thin and diffusely branched tree and tubular structures, adaptively focusing on slender and tortuous local structures, and dynamically adapting to the special texture structure of the strip surface, so as to better perceive the key features of small targets and improve the accuracy of detecting the location of small defects on the strip surface. S5: The bounding box regression loss MPDioU is used instead of CIoU in the original YOLOv8 model to enhance the regression ability of the bounding box of steel defects with large differences in shape and aspect ratio, reduce harmful gradients, improve the fitting speed, and improve the generalization ability and accuracy of the model; the resulting improved structure based on YOLOv8 network.

2. The method for improving the YOLOv8 network as described in claim 1, characterized in that: Step S1 specifically includes: The calculation process of DCNv2 includes: Let K=9, （1） Equation (1) defines a 3×3 convolution kernel with an inflation coefficient of 1; （2） In Equation 2, x(p) and y(p) represent the features at position p in the input feature map x and the output feature map y, respectively. and These are the learnable offset and modulation scalar at the k-th position, respectively. As can be seen from Equation 2, DCNv2 not only introduces an offset for each point to break the fixed geometric structure of the traditional CNN convolutional module, but also proposes a feature simulation scheme to guide the training of deformable convolutional networks. That is, a weight term is added to penalize the output features point-to-point, thereby avoiding the situation where DCNv1 exceeds the range of the target defect to obtain a larger receptive field, resulting in low correlation between the defect features and the extracted features. The three-channel coordinate attention mechanism TCCA is used to calculate the mask offset. TCCA adds a global adaptive pooling operation on top of the two one-dimensional global pooling operations performed by CA on the input image, turning it into a three-channel system. In view of the fact that the localization information obtained by CA is not precise enough, the two one-dimensional coordinates are weighted in the fifth layer, which effectively enhances the network's ability to express the features of irregular, defective and complex targets.

3. The method for improving the YOLOv8 network as described in claim 1, characterized in that: Step S2 specifically includes: A Global Attention (GAM) mechanism is introduced to improve the performance of neural network feature representation by reducing the diffusion of local image information and amplifying global interaction. GAM adopts a channel-spatial attention mechanism as the sequential structure of the modules. The channel attention submodule uses 3D permutation to retain information in three dimensions, and cross-dimensional channel spatial dependence is amplified through a multilayer perceptron. The spatial attention submodule uses a large 7×7 convolution kernel for convolution, which is beneficial for the fusion of spatial information in large images and the extraction of global features. The feature extraction of defects on the surface edge of the strip steel and defects with large spans is optimized, which increases the accuracy of locating irregular defects on large strip steel and makes defects of different sizes more advantageous in generalization ability.

4. The method for improving the YOLOv8 network as described in claim 1, characterized in that: Step S3 specifically includes: The Bidirectional Feature Pyramid Network (BiFPN) is introduced to reduce feature information loss and enhance multi-scale feature fusion. BiFPN incorporates feature adjustment, feature selection, and bidirectional feature propagation operations. By continuously adjusting and optimizing the weight parameters of the representation in each iteration, a large number of relatively fitting defect features are obtained. Then, by utilizing the different priorities and confidence levels of these features, the best-fitting features are selected dynamically and quickly. The bidirectional structure of BiFPN allows feature information to flow between different levels from top to bottom and from bottom to top, enhancing the semantic and location information of the features.

5. The method for improving the YOLOv8 network as described in claim 1, characterized in that: Step S4 specifically includes: In the BiFPN structure, dynamic serpentine convolution DSC is combined to accurately capture thin and diffusely branched tree-like and tubular structures, adaptively focusing on slender and tortuous local structures; DSC uses a convolution kernel of size 9, straightening the standard convolution kernel in the X and Y axes; taking the X direction as an example, the specific position in each grid K is represented as follows: （3） In Equation 3, K is a given standard two-dimensional convolution coordinate set. Using the center coordinates, a 3×3 convolution kernel with a dilation of 1 can be represented as: ,in Represents the distance to the central network; the position of each grid cell in the convolution kernel K. The selection of each element is an accumulation process, and its position depends on the position of the previous network. The value in the X-axis direction becomes: The value in the Y-axis direction becomes: The offset is continuously adjusted to ensure that the convolution kernel presents a linear structure and covers a range of 9×9 pixels; by referencing DSC, it dynamically adapts to the special texture structure of the strip surface to better perceive the key features of small targets and improve the accuracy of detecting the location of small defects on the strip surface.

6. The method for improving the YOLOv8 network as described in claim 1, characterized in that: Step S5 specifically includes: Bounding box regression is a crucial step in object detection and is essential for defect location. The original YOLOv8 model uses CIoU, which proposes to consider the center point distance and aspect ratio when calculating the loss, as shown in Equation 4. （4） in , , and This indicates the width and height of the ground reality bounding box. and The bounding box width and height are represented; the bounding box regression loss MPDioU is used instead of CIoU, and its formula is shown in Equation 5: （5） in , 。 7. The improved YOLOv8 network strip surface defect detection method as described in any one of claims 1-6, comprising the following steps: Step 1: Take photos of the strip defect detection scenario using an industrial camera; Step 2: Select and label the image data, and divide the labeled data into training set, test set and validation set in a ratio of 8:1:1; Step 3: Perform data augmentation on the training set data, including random flipping and cropping; Step 4: Train the network and improve the coordinate attention mechanism to a three-channel coordinate attention mechanism, which is then embedded into the DCNv2 network; Step 5: Incorporate the GAM global attention mechanism to enhance the overall network feature extraction global interaction capability; Step 6: Introduce the BiFPN structure and fuse it with dynamic snake convolution to enhance feature fusion capabilities; Step 7: Replace the MPDioU loss function; Step 8: Train the network model using the improved YOLO network; Step 9: Export the trained model file, test it, continuously adjust the parameters based on the test results, and select the parameters with the best effect to train the detection network.

Images