Improved YOLOv8-based steel surface defect detection method and system and medium

Abstract

The invention relates to a steel surface defect detection method and system based on improved YOLOv8 and a medium, and the method comprises the following steps: obtaining a surface image of a to-be-detected steel, and carrying out the image preprocessing of the surface image of the to-be-detected steel; inputting the surface image of the to-be-detected steel subjected to image preprocessing into a steel surface defect detection model to obtain a steel surface defect detection result; the steel surface defect detection model is an improved YOLOv8 and comprises a backbone network, a neck assembly and a head assembly which are connected in sequence, the backbone network adopts StarNet, a GICM module and a CARAFE up-sampling module are arranged in the neck assembly, and a rapid asymmetric detection head is adopted in the head assembly to replace an original detection head. Compared with the prior art, the capability of extracting and fusing complex defect features of the neck component is enhanced, the problem of asymmetry of classification and regression task resource requirements in defect detection is effectively solved, and meanwhile, the accuracy of defect detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of steel surface defect detection, and in particular to a steel surface defect detection method, system and medium based on improved YOLOv8. Background Art

[0002] Traditional methods for detecting steel surface defects typically rely on manual visual inspection. However, this method is highly subjective, inefficient, and susceptible to fatigue, resulting in unstable detection results. Subsequently, machine vision technology was introduced to the field of automated steel defect detection. However, these methods require specialized feature extraction algorithms designed for specific defect types and are sensitive to noise, resulting in poor robustness and generalization capabilities.

[0003] With the advancement of deep learning technology, deep learning-based algorithms have been widely used in surface defect detection. However, due to the diversity and complexity of steel surface defects, existing models may find it difficult to accurately identify all types of defects, resulting in reduced detection accuracy, including missed detections and false detections. In addition, factors such as the texture of the steel surface, lighting conditions, and background noise can also have a significant impact on the detection results. For example, tiny scratches or cracks may be ignored against a complex texture background, while surface reflections or shadows may lead to misjudgments. At the same time, inaccurate data annotation and uneven sample distribution can further exacerbate the difficulty of model generalization. For example, patent application CN118229655A discloses a steel surface defect detection method based on YOLOv8 multi-scale feature fusion. This method only improves the model feature extraction part without improving the detection head, resulting in low model classification accuracy and weak adaptability to complex scenarios. Therefore, how to improve the model's adaptability to complex scenarios and enhance its robustness and generalization performance is a key issue that needs to be urgently addressed in the current field of steel surface defect detection. Summary of the Invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a steel surface defect detection method, system and medium based on improved YOLOv8, which enhances the ability of the neck component to extract and fuse complex defect features, effectively solves the asymmetry problem in the resource requirements of classification and regression tasks in defect detection, and improves the accuracy of defect detection.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A steel surface defect detection method based on improved YOLOv8 includes the following steps:

[0007] Acquire a surface image of the steel to be inspected, and perform image preprocessing on the surface image of the steel to be inspected;

[0008] Inputting the surface image of the steel material to be inspected after image preprocessing into the steel material surface defect detection model to obtain the steel material surface defect detection result;

[0009] The steel surface defect detection model is an improved YOLOv8, which includes a backbone network, a neck component and a head component connected in sequence. The backbone network adopts StarNet, the neck component is provided with a GICM module and a CARAFE upsampling module, and the head component adopts a fast asymmetric detection head to replace the original detection head.

[0010] Furthermore, the image preprocessing includes grayscale processing, size adjustment and normalization processing.

[0011] Furthermore, the StarNet includes multiple star convolutions and star blocks. The star convolution uses ReLU6 activation function to replace the original GELU, and introduces depth-separable convolution at the beginning and end of each star block.

[0012] Furthermore, the unilateral branch of the GICM module consists of multiple stacked PLU blocks, which introduce partial convolution before the gated single-branch activation function of the GLU. The partial convolution specifically performs a partial convolution operation on 1 / 2 of the input channels, and the remaining 1 / 2 channels remain unchanged.

[0013] Furthermore, the number of floating-point operations performed per second by the GICM module is:

[0014]

[0015] Where FLOPS is the number of floating point operations per second, h is the height of the feature map, w is the width of the feature map, and c is the width of the feature map. partial is the number of channels in the partial convolution, and k is the size of the convolution kernel.

[0016] Furthermore, the CARAFE upsampling module includes a prediction unit and a content-aware reassembly unit, wherein the prediction unit is used to encode the input content, generate a rearranged convolution kernel, and process the rearranged convolution kernel using the content-aware reassembly unit.

[0017] Furthermore, the fast asymmetric detection head adopts a PConv module for feature extraction.

[0018] Furthermore, the steel surface defect detection results include multiple types of inclusion defects, pitting defects, the presence of pressed-in scaly metal skin, crack defects, scratch defects and plaque defects.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steel surface defect detection method based on the improved YOLOv8 can be implemented as described above.

[0020] According to another aspect of the present invention, a steel surface defect detection system based on an improved YOLOv8 is provided, comprising:

[0021] An image preprocessing module is used to obtain a surface image of the steel material to be detected and perform image preprocessing on the surface image of the steel material to be detected;

[0022] The steel surface defect detection module is used to input the surface image of the steel to be detected after image preprocessing into the steel surface defect detection model to obtain the steel surface defect detection result;

[0023] The steel surface defect detection model is an improved YOLOv8, which includes a backbone network, a neck component and a head component connected in sequence. The backbone network adopts StarNet, the neck component is provided with a GICM module and a CARAFE upsampling module, and the head component adopts a fast asymmetric detection head to replace the original detection head.

[0024] Compared with existing technologies, the present invention has the following beneficial effects:

[0025] 1. The present invention designs a GLU improved CSP module (GICM). The unilateral branch of the GICM module consists of multiple stacked PLU blocks. The PLU block introduces partial convolution before the gated single-branch activation function of the GLU, making full use of the redundancy and similarity of feature maps between different channels. Partial convolution is performed on 1 / 2 of the input channels, and the remaining 1 / 2 channels remain unchanged, reducing the computational complexity and memory usage, thereby improving the inference speed and enhancing the diversity of the GLU input, thereby enhancing the ability of the neck component to extract and fuse complex defect features.

[0026] 2. This paper designs a fast asymmetric detection head (FADH). By replacing the symmetric decoupling head with FADH in the YOLOv8 network and adopting the PConv module for feature extraction in FADH, the number of model parameters is significantly reduced. While maintaining high-speed inference, it effectively solves the asymmetry problem in the resource requirements of classification and regression tasks in defect detection, and at the same time improves the accuracy of defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flow chart of a steel surface defect detection method based on improved YOLOv8 proposed in the present invention;

[0028] Figure 2 This is a schematic diagram of the overall architecture of the steel surface defect detection model;

[0029] Figure 3 Schematic diagram of the structure of star convolution;

[0030] Figure 4 It is a structural diagram of the star block;

[0031] Figure 5 Schematic diagram of the basic convolution structure;

[0032] Figure 6 Improved schematic diagram of the CSP module for GLU;

[0033] Figure 7 It is a structural diagram of the PLU block;

[0034] Figure 8 Schematic diagram of the structure of the fast asymmetric detection head.

[0035] Legend: 1. Backbone network; 2. Neck component; 3. Head component; 4. Star convolution; 5. Star block; 6. Basic convolution. DETAILED DESCRIPTION

[0036] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0037] Abbreviations involved:

[0038] GLU improved CSP module: GLU improved CSP Module, GICM

[0039] Fast Asymmetric Decoupled Head: Fast Asymmetric Decoupled Head, FADH

[0040] Gaussian Error Linear Unit: Gaussian Error Linear Unit, GELU

[0041] Floating Point Operations Per Second (FLOPS)

[0042] Example 1

[0043] This embodiment provides a steel surface defect detection method based on improved YOLOv8, such as Figure 1 As shown, the following steps are included:

[0044] S1. Acquire a surface image of a steel material to be inspected, and perform image preprocessing on the surface image of the steel material to be inspected.

[0045] A high-resolution industrial camera captures images of the steel surface to be inspected, and then performs image preprocessing on the surface. Image preprocessing includes grayscale processing, resizing, and normalization. Grayscale processing simplifies image data, reducing computational effort while preserving image texture and defect information. Resizing adjusts the image resolution to an appropriate size for subsequent processing. Normalization adjusts the image pixel values ​​to a fixed range, eliminating lighting differences between images and enhancing image contrast, making subsequent processing more stable.

[0046] S2. Inputting the surface image of the steel material to be inspected after image preprocessing into the steel material surface defect detection model to obtain the steel material surface defect detection result.

[0047] The steel surface defect detection model is the improved YOLOv8, such as Figure 2 As shown, it includes a backbone network 1, a neck component 2 and a head component 3 connected in sequence. The backbone network 1 adopts StarNet, the neck component 2 is provided with a GICM module and a CARAFE upsampling module, and the head component 3 adopts a fast asymmetric detection head to replace the original detection head.

[0048] The most significant features are extracted through backbone network 1. Because the complexity and diversity of metal defects place higher demands on backbone network 1, StarNet is adopted as the backbone network. StarNet follows a traditional layered network structure, directly utilizing convolutional layers at each stage to reduce resolution and double the number of channels. Features are extracted by repeating multiple star blocks 5. The star operation in star block 5 maps the input into a high-dimensional nonlinear feature space. As a result, the network performs well without the need for complex structures and sophisticated hyperparameter tuning, while also achieving low latency and maintaining excellent performance with a compact network architecture and low computational power.

[0049] StarNet includes multiple star convolutions 4 and star blocks 5, such as Figure 3 As shown, the star convolution 4 uses the ReLU6 activation function to replace the original GELU, and introduces depth-separable convolution at the beginning and end of each star block 5, as shown in Figure 4 As shown in Figure 2. To improve the inference speed, batch normalization (BatchNorm2d) is placed after the depth-wise separable convolution.

[0050] The multi-scale features extracted from the backbone network 1 are optimized through the path aggregation network structure in the neck component 2, realizing the effective fusion and upsampling of feature maps of different scales. This significantly enhances the feature map's ability to characterize defect details and contours. In addition, by increasing the diversity and expressiveness of features, the accurate detection and positioning of defect targets of different scales are greatly improved. In order to enhance the network's ability to extract and fuse complex metal defect morphologies, low-scale defects, and occlusion defects, this embodiment proposes a lightweight GLU improved CSP module (GICM). At the same time, in order to ensure the rapidity and low loss of the upsampling process, the CARAFE upsampling operator is used to ensure the efficiency and accuracy of the model in processing details.

[0051] like Figure 5 As shown in FIG, the basic convolution 6 includes a two-dimensional convolution layer (Conv2d), a batch normalization (BatchNorm2d), and a SiLU activation function connected in sequence.

[0052] The structure of GLU improved CSP module (GICM) is as follows Figure 6 As shown in , the unilateral branch of the GICM module consists of multiple stacked PLU blocks, which aims to introduce significant correlation differences and thus quickly utilize feature maps. Figure 7 As shown in the figure, the PLU block introduces deep convolution and partial convolution before the gated single-branch activation function of the GLU. Partial convolution specifically performs a partial convolution operation on 1 / 2 of the input channels, and the remaining 1 / 2 channels remain unchanged. The number of floating-point operations performed per second by the GICM module is:

[0053]

[0054] Where FLOPS is the number of floating point operations per second, h is the height of the feature map, w is the width of the feature map, and c is the width of the feature map. partial is the number of channels in the partial convolution, and k is the size of the convolution kernel.

[0055] Partial convolution not only reduces the computational complexity and memory usage of the entire module, thereby improving inference speed, but also enhances the diversity of GLU inputs. This enables efficient feature selection and fusion at different levels, thereby improving overall module performance. It also facilitates the stacking of PLU blocks.

[0056] In YOLOv8, the default upsampling method is the nearest neighbor interpolation algorithm. However, this algorithm has a relatively small receptive field and is limited in capturing the rich semantic information required for complex tasks. This problem is particularly prominent when dealing with small defect targets, which may result in the loss of key details. Although the deconvolution (transposed convolution) upsampling method can learn more complex upsampling patterns from the data, alleviating this problem to some extent, it also introduces additional parameters and computational overhead. Furthermore, this method may produce checkerboard artifacts, affecting the overall quality of the upsampled feature maps. In view of this, this embodiment proposes the use of the CARAFE upsampling module as an alternative to address the aforementioned issues while achieving a better balance in computational efficiency. The CARAFE upsampling module consists of a prediction unit and a content-aware reconstruction unit. The prediction module's main function is to encode the input content and generate a reordered convolution kernel. In this process, the number of channels of the input feature map is first reduced to optimize computational efficiency. When processing the reordered convolution kernel, a content-aware reconstruction module is used. This module enhances the reordered convolution kernel based on a content-aware weighted summation algorithm to better preserve feature content. Unlike upsampling techniques that use nearest neighbor interpolation, the CARAFE module can perceive and process contextual information within a larger receptive field based on spatial and content information, thereby improving the ability to capture details. In addition, the CARAFE module achieves faster inference calculation speed through channel compression design.

[0057] This embodiment uses a fast asymmetric detection head (FADH) in the head assembly 3, such as Figure 8 As shown, the fast asymmetric detection head uses the PConv module for feature extraction. The bounding box loss (BboxLoss) is used to calculate the difference between the predicted bounding box and the ground-truth bounding box, and the classification loss (ClsLoss) is used to calculate the difference between the predicted class and the ground-truth class. The FADH consists of two parallel paths, each of which passes through the PConv, BaseConv, and Conv2d modules, and finally calculates the BboxLoss and ClsLoss, respectively. This design aims to improve the performance of the detection head through parallel processing while maintaining computational efficiency. The PConv module effectively expands the receptive field without introducing a large number of additional parameters, thus quickly completing the bounding box regression task. The PConv module generates key features by using a portion of the standard convolution operation. This approach significantly reduces the number of parameters and mitigates the impact of high-frequency noise on the detection task. By replacing the symmetric decoupling head with the FADH in the YOLOv8 network, the model parameter count is significantly reduced while improving defect detection accuracy. This effectively addresses the asymmetry in resource requirements between classification and regression tasks.

[0058] The results of steel surface defect detection include inclusion defects, pitting defects, the presence of indented scaly metal skin, crack defects, scratch defects and plaque defects.

[0059] All experiments in this embodiment were performed on an Nvidia RTX 4090D GPU (CUDA 11.8) equipped with 24GB of video memory, with a learning rate of 0.007, training for 250 cycles, and a batch size of 32. The weight decay rate was 0.0004. In addition, a mosaic data enhancement strategy was adopted. The NEU-DET dataset was used to train the steel surface defect detection model in this embodiment. The dataset contains six common defects on the steel surface: inclusions (In), pits (Ps), pressed scaly metal skin (Rs), cracks (Cr), scratches (Sc), and patches (Pa). Each category contains 300 images with a resolution of 200×200 pixels. Before training, all images were resized to 640×640 pixels. In this experiment, the training set and the validation set were randomly divided in a ratio of 8:2.

[0060] To verify the quality of the steel surface defect detection model proposed in this example for metal defect detection, an ablation experiment was conducted using the NEU-DET dataset. The performance changes after adding each component are shown in Table 1.

[0061] Table 1 Ablation study results of different modules

[0062]

[0063] It can be observed from Table 1 that the steel surface defect detection model proposed in this embodiment has significantly improved accuracy and reduced the number of parameters compared to the baseline model. First, after replacing the backbone network of YOLOv8 with the StarNet backbone network, the mAP increased from 76.6% to 77.0%, and the number of parameters (Params) was reduced, indicating that StarNet performs better with a smaller number of parameters. After adding the GICM module, the computational load (GFLOPs) of the network was significantly reduced, and the mAP increased by 2.7%. The results show that the GICM module enhances the selection and interaction of multi-scale features across different levels and can effectively detect complex defects. After the introduction of CARAFE, the performance was improved with only a slight increase in computational complexity. After adding FADH, the mAP reached 82.6%, and the GFLOPs was only 5.4, and the inference speed was significantly improved. This move greatly reduced the model parameters while improving the defect detection accuracy, successfully alleviating the problem of imbalance in resource requirements between classification and regression tasks.

[0064] To demonstrate the effectiveness of the steel surface defect detection method based on the improved YOLOv8 proposed in this embodiment, it is first compared with mainstream methods and known steel surface defect detection methods. The results are shown in Table 2.

[0065] Table 2 Comparison with different algorithms on NEU-DET

[0066]

[0067]

[0068] As can be seen from Table 2, the steel surface defect detection method based on the improved YOLOv8 proposed in this embodiment achieved an mAP of 82.6%, and achieved relatively good results for small target defects Cr and Rs. Compared with the baseline network YOLOv8, the method proposed in this embodiment showed varying degrees of improvement in the mAP of In, Ps, and Rs defects. In terms of FPS, the method proposed in this embodiment achieved 162.9, slightly lower than YOLOv5, effectively meeting the speed requirements of industrial deployment. Therefore, it can be concluded that the steel surface defect detection method based on the improved YOLOv8 proposed in this embodiment surpasses the widely used target detection model and performs well in both accuracy and FPS. In addition, its performance in steel surface defect detection is particularly outstanding, making it more suitable for deployment in actual industrial applications.

[0069] Example 2

[0070] This embodiment provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the steel surface defect detection method based on the improved YOLOv8 proposed in Example 1.

[0071] The rest is the same as Example 1.

[0072] Example 3

[0073] This embodiment provides a steel surface defect detection system based on an improved YOLOv8, including:

[0074] An image preprocessing module is used to obtain a surface image of the steel material to be inspected and perform image preprocessing on the surface image of the steel material to be inspected;

[0075] The steel surface defect detection module is used to input the surface image of the steel to be detected after image preprocessing into the steel surface defect detection model to obtain the steel surface defect detection result;

[0076] The steel surface defect detection model is an improved YOLOv8, which includes a backbone network 1, a neck component 2, and a head component 3 connected in sequence. The backbone network 1 uses StarNet, the neck component 2 is equipped with a GICM module and a CARAFE upsampling module, and the head component 3 uses a fast asymmetric detection head to replace the original detection head.

[0077] The rest is the same as Example 1.

[0078] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A steel surface defect detection method based on improved YOLOv8, characterized in that: The following steps are involved: Acquire a surface image of the steel to be inspected, and perform image preprocessing on the surface image of the steel to be inspected; Inputting the surface image of the steel material to be inspected after image preprocessing into the steel material surface defect detection model to obtain the steel material surface defect detection result; The steel surface defect detection model is an improved YOLOv8, comprising a backbone network (1), a neck component (2) and a head component (3) connected in sequence, wherein the backbone network (1) adopts StarNet, the neck component (2) is provided with a GICM module and a CARAFE upsampling module, and the head component (3) adopts a fast asymmetric detection head to replace the original detection head.

2. The steel surface defect detection method based on improved YOLOv8 according to claim 1 is characterized in that: The image preprocessing includes grayscale processing, size adjustment and normalization processing.

3. The steel surface defect detection method based on improved YOLOv8 according to claim 1 is characterized in that: The StarNet includes multiple star convolutions (4) and star blocks (5). The star convolution (4) uses the ReLU6 activation function to replace the original GELU, and introduces depth-separable convolution at the beginning and end of each star block (5).

4. The steel surface defect detection method based on improved YOLOv8 according to claim 1, characterized in that: The unilateral branch of the GICM module consists of multiple stacked PLU blocks, which introduce partial convolution before the gated single-branch activation function of the GLU. The partial convolution is specifically to perform a partial convolution operation on 1 / 2 of the input channels, and the remaining 1 / 2 channels remain unchanged.

5. The steel surface defect detection method based on improved YOLOv8 according to claim 1, characterized in that: The number of floating-point operations performed per second by the GICM module is: Where FLOPS is the number of floating point operations per second, h is the height of the feature map, w is the width of the feature map, and c is the width of the feature map. partial is the number of channels in the partial convolution, and k is the size of the convolution kernel.

6. The steel surface defect detection method based on improved YOLOv8 according to claim 1, characterized in that: The CARAFE upsampling module includes a prediction unit and a content E reorganization unit. The prediction unit is used to encode the input content, generate a rearranged convolution kernel, and use the content-aware reorganization unit to process the rearranged convolution kernel.

7. The steel surface defect detection method based on improved YOLOv8 according to claim 1, characterized in that: The fast asymmetric detection head adopts the PConv module for feature extraction.

8. The steel surface defect detection method based on improved YOLOv8 according to claim 1, characterized in that: The steel surface defect detection results include multiple types of inclusion defects, pitting defects, the presence of pressed-in scaly metal skin, crack defects, scratch defects and plaque defects.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, it can implement the steel surface defect detection method based on improved YOLOv8 as described in any one of claims 1 to 8.

10. A steel surface defect detection system based on improved YOLOv8, characterized in that: include: An image preprocessing module is used to obtain a surface image of the steel material to be detected and perform image preprocessing on the surface image of the steel material to be detected; The steel surface defect detection module is used to input the surface image of the steel to be detected after image preprocessing into the steel surface defect detection model to obtain the steel surface defect detection result; The steel surface defect detection model is an improved YOLOv8, comprising a backbone network (1), a neck component (2) and a head component (3) connected in sequence, wherein the backbone network (1) adopts StarNet, the neck component (2) is provided with a GICM module and a CARAFE upsampling module, and the head component (3) adopts a fast asymmetric detection head to replace the original detection head.

Images