Steel surface defect detection method and system based on improved YOLOv8 network

Abstract

The invention discloses a steel surface defect detection method and system based on an improved YOLOv8 network. The method comprises the following steps: firstly, preprocessing an input steel surface image; inputting to an improved YOLOv8 network, wherein the network sequentially comprises a backbone network, a neck network and a decoupling detection head; carrying out multi-scale feature extraction on an input image through a backbone network, and transmitting an output feature map containing high-frequency detail information to a neck network; the neck network receives the feature map output by the backbone network, performs cross-scale feature fusion on the feature map, and generates a fusion feature map with global context information; and the decoupling detection head receives the fusion feature map output by the neck network, carries out decoupling processing on the fusion feature map, and respectively outputs defect category classification information and bounding box regression information. According to the method, the surface defects of the steel product can be accurately and efficiently detected in real time in the manufacturing process, so that the quality, integrity and use safety of the product are ensured.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a method and system for detecting surface defects in steel based on an improved YOLOv8 network. Background Technology

[0002] Detection of surface defects in steel is a crucial step in ensuring the quality of industrial products. Traditional detection methods mainly rely on manual visual inspection or basic optical inspection systems. However, these methods generally have inherent limitations, such as high cost, susceptibility to subjective factors in the results, low efficiency, and difficulty in adapting to modern high-speed production lines.

[0003] With the rapid development of deep learning technology, automated visual inspection has ushered in a technological revolution. Deep learning-based object detection algorithms, especially convolutional neural networks (CNNs), have become the mainstream in this field. These algorithms are mainly divided into two categories: two-stage detectors and single-stage detectors. Two-stage detectors, represented by Faster R-CNN, achieve high detection accuracy by first generating candidate regions and then performing classification and regression. However, their detection speed is slow and their model size is large, making it difficult to meet the stringent real-time requirements of industrial scenarios. In contrast, single-stage detectors, represented by the YOLO (You Only Look Once) series, treat the detection task as a regression problem, directly predicting the target's location and category in a single forward propagation. Due to their excellent balance between speed and accuracy, they have become the preferred choice for industrial deployment. Among the many versions of YOLO, YOLOv8 is considered one of the most advanced benchmark models currently available and is widely used as the basis for improvements in related research.

[0004] While YOLOv8 performs well in general object detection tasks, its standard architecture reveals several limitations in the specific application scenario of steel surface defect detection. Existing research shows that directly applying or making single-direction improvements to the YOLOv8 model still struggles to fully meet the complex needs of industrial production.

[0005] First, the trade-off between accuracy and efficiency is a core challenge. To adapt to industrial edge devices with limited computing power, researchers often choose lightweight versions of YOLOv8 (such as YOLOv8n). However, this lightweight approach comes at the cost of sacrificing model complexity and feature extraction capabilities, resulting in decreased detection accuracy when dealing with defects that are complex in shape or have subtle textures. Conversely, while large models offer higher accuracy, their massive number of parameters and high computational cost (GFLOPS) make them difficult to implement inference in real time on resource-constrained edge devices, which constitutes a major obstacle to deployment.

[0006] Secondly, the loss of fine-grained feature information is a key factor leading to the missed detection of small defects. The morphology of defects on steel surfaces varies, especially for some minute defects (such as pitted surfaces) and linear defects (such as scratches and crazing), whose effective pixel ratio is extremely low. Standard CNN architectures such as YOLOv8 commonly use strided convolutions or pooling layers for rapid downsampling in the initial stages of the backbone network to reduce feature map size and computational cost. While this design is efficient, the process involves irreversible lossy compression, severely damaging the structural information and fine-grained features of small targets, preventing subsequent network layers from effectively learning and recognizing these defects.

[0007] Third, the model's ability to capture global contextual information is insufficient. For linear defects such as scratches and cracks, which have continuity and long-range spatial dependencies, the model's detection performance depends not only on local features but also on its understanding of the global context within a larger receptive field. Standard convolutional kernels have limited receptive fields, making it difficult to effectively model such long-range dependencies. This often results in multiple discontinuous and fragmented detection boxes for a complete scratch, or even complete missed detection.

[0008] Finally, the model's sensitivity to data quality and diversity limits its robustness in real-world industrial environments. Image data collected in industrial settings often exhibits inconsistent quality, including variations in lighting and uneven contrast. Furthermore, public datasets like NEU-DET have relatively limited sample size and diversity, making it difficult to cover all real-world scenarios. Additionally, manually annotated bounding boxes may contain biased or low-quality samples. Standard loss functions (such as CIoU) may excessively penalize these low-quality samples during training, interfering with the model's convergence direction and affecting its final generalization ability and localization accuracy.

[0009] These limitations indicate that the general-purpose YOLOv8 architecture cannot directly meet the specialized needs of steel defect detection. This has driven the research field towards more targeted architectural specialization. Different research teams have optimized different modules of YOLOv8 from their respective perspectives, resulting in multiple technical paths. For example, some studies focus on lightweighting and low-precision issues, achieving a 5.1% improvement in mAP@0.5 and a 12.9% improvement in recall by introducing Ghost Backbone and SE attention mechanisms into the backbone network and combining them with the WIoU loss function. Other studies are dedicated to solving global context awareness and data quality issues, significantly improving mAP@0.5 from 0.695 to 0.809 through histogram equalization preprocessing and combining it with the GnC3 backbone network and RCCA attention mechanism. To address the dual challenges of insufficient data diversity and lightweight models, some studies have employed LS-DCGAN for data augmentation and designed networks incorporating C2f_SCSA and HSFPN, achieving a 3.0% improvement in mAP@50 while reducing parameter count and computational cost by 26.7% and 12.3%, respectively. For the challenge of detecting high model complexity and small targets, researchers introduced SPD-Conv to preserve fine-grained features and constructed a slim-neck structure, resulting in a 2.2% improvement in mAP@0.5, while reducing parameter count and computational cost by 24.6% and 28.4%, respectively. Furthermore, to specifically optimize the detection of small and irregular defects, a scheme using C2f_MSBlock and CARAFE upsampling techniques improved mAP@50 by 2.7%. While these studies have made progress in specific areas, a unified solution that systematically and synergistically addresses all the aforementioned core challenges has yet to emerge. Summary of the Invention

[0010] In view of this, the purpose of the present invention is to provide a method and system for detecting surface defects in steel based on an improved YOLOv8 network. This method enables real-time, accurate and efficient detection of surface defects in steel products during the manufacturing process.

[0011] To achieve the above objectives, the present invention provides the following technical solution:

[0012] This invention provides a method for detecting surface defects in steel based on an improved YOLOv8 network, comprising the following steps:

[0013] Preprocess the input image of the steel surface;

[0014] The preprocessed image is fed into an improved YOLOv8 network, which consists of a backbone network, a neck network, and a decoupled detection head.

[0015] The backbone network uses the Ghost Backbone module as the core feature extraction unit to perform multi-scale feature extraction on the input image, and includes the HRDS module, which is used to preserve high-frequency detail information in the image through hierarchical residual downsampling, and transmits the output feature map containing high-frequency detail information to the neck network.

[0016] The neck network receives the feature map output by the backbone network, performs cross-scale feature fusion on it, and generates a fused feature map with global context information; its feature fusion path is constructed by the GSConv module and captures global context information through the RCCA module.

[0017] The decoupled detection head receives the fused feature map output by the neck network, performs decoupling processing on it, and outputs defect category classification information and bounding box regression information respectively.

[0018] Furthermore, the HRDS module implements hierarchical residual downsampling through a multi-path structure, specifically including:

[0019] The first path extracts multi-scale contextual features through multiple parallel convolutions with different dilation rates, and then downsamples them after weighting by the SE attention mechanism;

[0020] The second path extracts high-frequency detail features from the input image through depthwise separable convolution;

[0021] The third path involves skip connections that adjust the dimensions by performing identity mapping or 1×1 convolution based on the consistency of the input and output dimensions.

[0022] The three output features are fused and then output after high-frequency filtering and spatial attention weighting.

[0023] Furthermore, the neck network adopts a Slim-Neck architecture with an integrated RCCA module for global context-aware feature fusion, wherein a lightweight feature fusion path is constructed using the GSConv module;

[0024] Furthermore, the improved YOLOv8 network uses the WIoU loss function as the bounding box regression loss for model training; the WIoU loss function includes a distance attention term and a dynamic non-monotonic focusing mechanism, wherein the dynamic non-monotonic focusing mechanism adjusts the gradient contribution of samples of different quality through outlier β.

[0025] Furthermore, the Ghost Backbone module generates intrinsic features through the Ghost module, and then constructs the Backbone structure through splicing and residual connections.

[0026] Furthermore, the GSConv module simulates the performance of standard convolution by halving the input channels and processing them separately through standard convolution and depthwise separable convolution, then concatenating them and performing channel shuffling.

[0027] Furthermore, the preprocessing step includes: performing histogram equalization on the input image and performing at least one random data augmentation operation, including random scaling, cropping, and image stitching.

[0028] The present invention provides a lightweight, high-precision steel surface defect detection system based on a hybrid YOLOv8 architecture, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor. When the processor executes the program, it implements the above-mentioned method.

[0029] The present invention provides a lightweight, high-precision steel surface defect detection system based on a hybrid YOLOv8 architecture. The system includes an industrial camera, an edge computing device, and a YOLOv8-GHR model deployed in an industrial field.

[0030] Furthermore, it also includes a predictive information sending device, which is used to send out detection results, which are used to trigger alarms, mark non-conforming products, or drive a robot sorting system.

[0031] The beneficial effects of this invention are as follows:

[0032] This invention provides a method and system for detecting steel surface defects based on an improved YOLOv8 network. The method first preprocesses the input steel surface image; then, the image is input into a co-designed YOLOv8 network. The backbone of this network uses a Ghost Backbone module as the core unit for lightweighting and introduces an HRDS (Hierarchical Residual Downsampling) module to replace the traditional downsampling layer, aiming to preserve the fine-grained features of minute defects to the greatest extent. Subsequently, the neck network adopts a Slim-Neck structure built based on the GSConv module and integrates an RCCA (Recurrent Cross-Attention) module, efficiently performing multi-scale feature fusion and capturing global contextual information crucial for elongated defects while maintaining low computational cost. Finally, the decoupled detection head processes the fused features and independently outputs the defect category and location information. This invention can also use the WIoU loss function for model training to enhance robustness to complex industrial data.

[0033] This method involves its application in industrial automated quality control for real-time, accurate, and efficient detection of surface defects in steel products during the manufacturing process. It addresses the urgent need for reliable non-destructive testing technology in key industries such as automotive manufacturing, aerospace, and bridge construction to ensure product quality, integrity, and safety. Its core objective is to replace inefficient and error-prone traditional manual inspection methods, providing a robust and automated solution.

[0034] This method achieves a balance between accuracy and efficiency: its main advantage lies in achieving a better balance between mAP and GFLOPS. The YOLOv8-GHR model is expected to achieve higher mAP (especially in small-sized and linear defect categories) while maintaining computational complexity comparable to or even lower than YOLOv8n, surpassing the performance of existing single-improvement schemes. It significantly enhances the detection capability for challenging defects: through the targeted design of the HRDS and RCCA modules, this invention directly addresses the main bottlenecks of standard models in detecting specific types of defects. Quantifiable and significant improvements in mean accuracy (AP) are expected in categories such as crazing, scrapes, and pitted surfaces. Enhanced robustness and practical deployability: the lightweight architecture combined with a robust loss function makes the YOLOv8-GHR model not only more resistant to noisy data but also feasible for practical deployment on common industrial edge computing platforms (such as the NVIDIA Jetson series) without sacrificing key detection performance.

[0035] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0036] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the following drawings are provided for illustration.

[0037] Figure 1 The flowchart shows the method and system for detecting surface defects in steel based on the improved YOLOv8 network.

[0038] Figure 2 This is the overall framework diagram.

[0039] Figure 3 This is a network framework diagram of HRDS.

[0040] Figure 4Visualization of YOLOv8-GHR detection results in the NEU-DET portion of the dataset. Detailed Implementation

[0041] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0042] Example 1

[0043] like Figure 1 As shown, the steel surface defect detection method and system based on the improved YOLOv8 network provided in this embodiment includes the following steps:

[0044] The input steel surface image is preprocessed, including histogram equalization to standardize image contrast.

[0045] An improved YOLOv8 network is constructed, comprising a backbone network, a neck network, and a decoupled detection head;

[0046] The backbone network is used for multi-scale feature extraction of images. The HRDS module of the backbone network retains the high-frequency detail information of the image through hierarchical residual downsampling, and sends the high-frequency detail information feature map of the image output by the backbone network into the neck network.

[0047] The neck network is used to perform cross-scale feature fusion on the feature map fed into the backbone network to form a fused feature map that captures global context information; the GSConv module of the neck network performs cross-scale feature fusion on the feature map fed into the backbone network through a cyclic cross-attention mechanism.

[0048] The decoupled detection head receives the fused feature map from the neck network and decouples the fused feature map to obtain defect category classification information and bounding box regression information. In this embodiment, the decoupled detection head is used to predict the fused features and output the defect category and location information.

[0049] In this embodiment, the improved YOLOv8 network uses the WIoU loss function as the bounding box regression loss for model training.

[0050] The WIoU loss function includes a distance attention term and a dynamic non-monotonic focusing mechanism. The dynamic non-monotonic focusing mechanism adjusts the gradient contribution of samples of different quality through outlier β, prioritizes the optimization of medium-quality samples to improve the model's generalization ability, optimizes the bounding box regression process, and improves training robustness.

[0051] The model described in this embodiment is optimized for edge device deployment through pruning and quantization after training, while inference acceleration engines such as NVIDIA TensorRT are used to improve real-time inference speed.

[0052] In this embodiment, the backbone network adopts the Ghost-HRDS architecture for lightweight multi-scale feature extraction. The HRDS module is used to replace the standard downsampling layer to retain fine-grained features, and GhostBackbone, which integrates the SE attention mechanism, is used as the core feature extraction unit.

[0053] In this embodiment, the neck network adopts a Slim-Neck architecture with an integrated RCCA module for global context-aware feature fusion, in which GSConv is used to construct a lightweight feature fusion path;

[0054] The HRDS module in this embodiment implements hierarchical residual downsampling through a multi-path structure, specifically including:

[0055] The first path extracts multi-scale contextual features through multiple parallel convolutions with different dilation rates, and then downsamples them after weighting by the SE attention mechanism;

[0056] The second path extracts high-frequency detail features from the input image through depthwise separable convolution;

[0057] The third path involves skip connections that adjust the dimensions by performing identity mapping or 1×1 convolution based on the consistency of the input and output dimensions.

[0058] The three output features are fused and then output after high-frequency filtering and spatial attention weighting.

[0059] The HRDS module described in this embodiment includes three parallel paths: a multi-scale feature extraction and gating path, a high-frequency compensation path, and an adaptive skip connection path. The outputs of the three paths are further enhanced by fusing them through residual connections to enhance high-frequency detail features.

[0060] In this embodiment, the Ghost Backbone module generates intrinsic features and phantom features through the Ghost module, and then constructs the Backbone structure by splicing and residual connection.

[0061] The RCCA module described in this embodiment uses a cross-attention mechanism applied twice in a loop to efficiently aggregate contextual information across the entire image, thereby improving the detection completeness of elongated defects (such as scratches and cracks).

[0062] In this embodiment, GSConv simulates the performance of standard convolution by halving the input channels and processing them separately through standard convolution and depthwise separable convolution, then concatenating them and performing channel shuffling.

[0063] The preprocessing steps in this embodiment also include random data augmentation operations, such as random scaling, cropping, and image stitching, to enrich the diversity of training samples and improve the model's adaptability to complex scenes.

[0064] The method described in this embodiment can be deployed in industrial settings, capturing real-time images of steel surfaces using industrial cameras, and using the detection results to trigger alarms, mark non-conforming products, or drive a robotic sorting system.

[0065] Furthermore, a steel surface defect detection system can be obtained based on this method. The system includes an industrial camera, an edge computing device, and a YOLOv8-GHR model deployed in an industrial site. The system is used to detect defects such as oxide scale, inclusions, cracks, scratches, patches, and pitting on the steel surface in real time.

[0066] Example 2

[0067] This embodiment presents a steel surface defect detection method based on an improved YOLOv8 network, applied to steel surface defect detection. This method overcomes the inherent contradiction between detection accuracy and computational efficiency. It achieves high precision and recall across various defect types, particularly for small-sized defects (such as pitted surfaces), fine-grained defects, and long-range linear defects (such as scrapes), which are difficult for traditional models to handle. Simultaneously, the constructed detection model architecture must remain lightweight, i.e., have low parameter count and computational cost (GFLOPS), to meet the requirements for real-time deployment on computationally limited industrial edge hardware. This model is a novel, hybrid deep learning network architecture named YOLOv8-GHR (Ghost-HRDS-RCCA), or simply the improved YOLOv8 network. Through the collaborative design of the backbone network, neck network, and loss function, it simultaneously addresses four major challenges: lightweight design, fine-grained feature preservation, global context capture, and training robustness.

[0068] 1) Hybrid lightweight and feature-preserving backbone network

[0069] Lightweight Core: To achieve extreme computational efficiency, the core feature extraction unit of the backbone network is reconstructed using the GhostBackbone module. This module generates "intrinsic feature maps" through a small number of standard convolutions, and then uses computationally inexpensive linear transformations (such as depthwise separable convolutions) to generate a large number of "phantom feature maps," thereby significantly reducing the number of model parameters and computational cost without sacrificing feature representation capabilities.

[0070] Fine-grained feature preservation: To fundamentally address the problem of information loss due to minute defects, this invention replaces the traditional strided convolutional layers used for downsampling in the backbone network with an innovative HRDS (Hierarchical Residual Downsampling) module. HRDS, through a parallel multi-path design, collaboratively extracts multi-scale features, compensates for high-frequency details lost during downsampling, and preserves the original low-level features. This design ensures that the fine-grained information of minute defects is preserved and enhanced to the greatest extent during downsampling, laying the foundation for subsequent accurate identification.

[0071] 2) Global context-aware neck network

[0072] Global Information Capture: To efficiently capture long-range dependencies crucial for linear defect detection, this invention integrates an RCCA (Recurrent Criss-Cross Attention) module into the feature fusion neck network. The RCCA module, through recurrent attention computation on horizontal and vertical paths, can aggregate contextual information from the entire image at a computationally low cost compared to standard self-attention mechanisms, effectively improving the completeness of defect detection, such as scratches and cracks.

[0073] Efficient Network Structure: To maintain the lightweight nature of the neck network, its overall construction follows the Slim-neck design paradigm and uses GSConv as the basic convolutional unit. GSConv simulates the performance of standard convolution by mixing the outputs of standard convolutions and depthwise separable convolutions, while significantly reducing computational overhead, ensuring that the neck network enhances feature fusion capabilities without becoming a performance bottleneck.

[0074] 3) Robust Loss Function for Industrial Data

[0075] Enhancing Training Stability: To improve the training stability and localization accuracy of the model when dealing with noisy and low-quality labeled industrial datasets, this method employs WIoU (Wise-IoU) as the loss function for bounding box regression. WIoU introduces a dynamic, non-monotonic focusing mechanism that intelligently assigns gradient gains to anchor boxes of different qualities, prioritizing "normal quality" samples that are most beneficial to model generalization, while reducing the detrimental effects of high-quality (simple samples) and low-quality (abnormal or noisy samples), thereby guiding the model to learn more robustly and efficiently.

[0076] like Figure 2 As shown, Figure 2The overall framework diagram shows that the YOLOv8-GHR network follows the mainstream input-backbone-neck-head design paradigm. This network consists of a Backbone, a Neck, and a Head, and integrates various advanced lightweight and attention mechanisms. After preprocessing the input image, it is then processed by the Backbone and Neck networks, and finally output through the Head. This model is a lightweight, high-performance object detection network, and its data processing flow is as follows:

[0077] 1. Input and Pre-processing

[0078] Histogram Equalization:

[0079] The input image is first preprocessed using histogram equalization. This operation effectively normalizes the image contrast, reduces the negative impact of changes in ambient lighting conditions on model performance, and thus improves the model's environmental adaptability.

[0080] 2. Backbone (main network): Ghost-HRDS

[0081] The preprocessed image is fed into the hybrid Ghost-HRDS backbone network for multi-scale feature extraction. This backbone network is designed to synergistically achieve lightweight and lossless feature downsampling. This part is the core feature extraction module of the network, and its design goal is to achieve extreme computational efficiency and high-fidelity fine-grained feature preservation.

[0082] HRDS: In the early stages of the network, the innovative HRDS module is used to replace the standard strided convolutional layers for downsampling. Through its parallel multi-path design, HRDS is able to reduce the feature map size while preserving as much high-frequency and fine-grained information as possible, which is crucial for the detection of minute defects.

[0083] Ghost Backbone + SEAttn: The backbone network is primarily composed of stacked, efficient Ghost Bottleneck modules for main feature extraction. The Ghost modules significantly reduce computation and parameters by generating "intrinsic features" and "phantom features" generated through inexpensive linear transformations. In deeper stages of the network, some Ghost Bottleneck modules integrate Squeeze-and-Excitation (SE) attention modules, enabling them to adaptively learn the importance of each feature channel, enhancing key features and suppressing irrelevant noise.

[0084] SPPF: At the end of the backbone network, the SPPF module processes the feature map by cascading multiple small-sized max pooling layers, achieving receptive field fusion of different scales with minimal computational cost, thereby enhancing the model's ability to detect targets of different sizes.

[0085] 3. Neck network: Slim-Neck + RCCA:

[0086] The neck network is responsible for efficiently fusing feature maps output from different layers of the backbone network, combining deep semantic information with shallow detail information.

[0087] GSConv: The entire neck network uses GSConv as the basic convolutional unit. By mixing the outputs of standard convolutions and depthwise separable convolutions, GSConv simulates the performance of standard convolutions while significantly reducing computational overhead, thus constructing a lightweight slim-neck structure.

[0088] Upsample & Concat: These are standard operations in Feature Pyramid Network (FPN) and Path Aggregation Network (PANet). Upsampling enlarges the size of deep, low-resolution feature maps, while concatenation merges the upsampled feature maps with shallow feature maps of the corresponding size in the backbone network along the channel dimension, achieving cross-level fusion of information.

[0089] RCCA (Recurrent Criss-Cross Attention): Before the neck network passes features to the detection head, the RCCA module is used to process the top-level feature map with the richest semantic information. It captures contextual dependencies across the entire map at a computational cost far lower than standard self-attention through an efficient recurrent crescent-cross attention mechanism, which is crucial for the complete identification and localization of long-range linear defects such as scratches and cracks.

[0090] 4. Head (Detection Head): Finally, the multi-scale feature maps, fully fused by the neck network, are passed to the decoupled head to generate the final detection results. This head structure separates the classification and regression tasks into two independent branches, avoiding optimization conflicts between them and thus effectively improving detection performance.

[0091] Decoupled Head: This network employs an advanced decoupled head structure. "Decoupling" refers to separating the previously combined tasks of object classification and bounding box regression into two independent network branches for parallel processing. This avoids mutual interference and conflict between the two tasks during model optimization, allowing each branch to focus more on its own task, thereby effectively improving the final detection accuracy and performance.

[0092] Classify Branch: This branch is part of the decoupling head and is specifically responsible for the classification task. It receives the fused feature map, analyzes it, and ultimately predicts the defect category (e.g., "scratch", "crack", or "patch") for each detected target, and outputs the corresponding confidence score.

[0093] Regression Branch: This branch is another part of the decoupling head, specifically responsible for the bounding box regression task. It also receives the fused feature maps, but its goal is to accurately predict the location and size of each defect in the image, i.e., outputting information such as the center point coordinates, width, and height of the bounding box.

[0094] The data flow in this embodiment is as follows:

[0095] A standard-sized input image (e.g., 640x640x3) first enters a hybrid backbone network for multi-scale feature extraction; the feature map output by the backbone network is fed into a context-aware neck network for cross-scale feature fusion and refinement; finally, the fused feature map is passed to a decoupled detection head to perform bounding box regression and defect category classification tasks in parallel.

[0096] like Figure 3 As shown, Figure 3 This is a network framework diagram for HRDS (Hierarchical Residual Downsampling), which illustrates a deep convolutional neural network (CNN) model, specifically including the following components:

[0097] like Figure 3 As shown, the HRDS module processes the input feature map X through a parallel multi-path structure, maximizing the preservation of high-frequency detail information while achieving downsampling. Its data processing procedure is as follows:

[0098] Input feature X splits into three routes:

[0099] The first path (multi-scale feature extraction and downsampling path): The input X first passes through three parallel 3x3 convolutions with different dilation rates (d = 1, 3, 5) to extract multi-scale contextual features, and its output F ms Concat (in the diagram) (Symbol). F ms Then, the SE attention module performs channel weighting to obtain F~ ms Then, downsampling is performed using 1x1 convolution and 3x3 convolution to finally output F. down .

[0100] The second path (high-frequency compensation path): The input X is passed through a 1x1 convolution and a 3x3 depthwise separable convolution (DWConv) to directly extract high-frequency detail features F. hf Used to compensate for information loss during the downsampling process.

[0101] The third path (adaptive skip connection path): This path determines the relationship between the input X and the downsampled output F. down Does the dimension match? If it matches (Yes), then X is directly used as the output.

[0102] F skip (Identity mapping); if there is no match (No), adjust the number of channels through a 1x1 convolution, and then output F after passing through the SE attention module. s kip. Finally, the three outputs F down ,F hf , and F skip F is obtained by fusing elements one by one through addition. fusion F fusion After passing through a high-frequency detail enhancement module (implemented via 3x3 DWConv), the final output F is obtained. out .

[0103] Conv1x1: indicates that a 1x1 kernel is used;

[0104] Conv3x3 d=1 indicates a convolution with a kernel size of 3x3 and a dilation rate of 1;

[0105] Conv3x3 d=3 indicates a convolution with a kernel size of 3x3 and an inflation rate of 3;

[0106] Conv3x3 d=5 indicates a convolution with a kernel size of 3x3 and an inflation rate of 5;

[0107] The `dim mach` statement represents a conditional statement that checks whether the dimensions of the input and output are consistent.

[0108] 3x3 DWConv represents a depthwise convolution with a kernel size of 3x3.

[0109] SE attn represents the Squeeze-and-Excitation attention module;

[0110] F skip This represents the feature map output by the adaptive skip connection path;

[0111] F down This represents the feature map output by the multi-scale feature extraction and downsampling path;

[0112] F fusion This represents the feature map after the features of the three paths are fused;

[0113] F hf This represents the feature map output by the high-frequency compensation path;

[0114] F ms This represents the original multi-scale features of the output of parallel multi-scale convolutions;

[0115] This represents the multi-scale features after SE attention weighting;

[0116] The backbone network in this method aims to collaboratively achieve lightweight and lossless feature downsampling. Its specific structure and working principle are as follows:

[0117] 1. Initial downsampling using the HRDS module: In the standard YOLOv8 backbone network, the initial downsampling stages are performed using convolutional layers with a stride of 2. These layers are replaced by the HRDS (Hierarchical Residual Downsampling) module. The HRDS module is designed to maximize the retention of high-frequency and fine-grained information that is crucial for the detection of minute defects through a multi-path design.

[0118] Its structure contains three parallel paths:

[0119] First approach: Multi-scale feature extraction and gating: This approach first uses multiple parallel convolutions with different dilation rates (e.g., 1, 3, 5) to extract features from the input feature map X∈R. C×H×W Extracting multi-scale contextual information F ms Subsequently, the SE attention mechanism was introduced to address F. ms Channel recalibration is performed to enhance the response of key defect features. Finally, depthwise separable convolution is used for downsampling to obtain feature F. down This process preserves details better than traditional stride convolution.

[0120] The second approach: High-frequency compensation mechanism: To compensate for high-frequency information that may be lost during downsampling, this approach directly extracts high-frequency detail features F from the original input X through depthwise separable convolution. hf It is specifically designed to detect minute defects on the surface of steel.

[0121] The third path: Adaptive skip connection: To preserve low-level features and effectively fuse them with high-level features, this path uses adaptive skip connections based on the dimensions of the input and the downsampled features. If the dimensions are the same, the input X is directly passed; otherwise, the dimensions are adjusted through a 1×1 convolution to obtain F. skip .

[0122] Path fusion and high-frequency detail enhancement: The outputs of the three paths are fused through residual connections.

[0123] F fusion =F down +F hf +F skip ;

[0124] Subsequently, enhanced high-frequency detail features F are generated through high-frequency filtering (subtracting the average pooling result from the fused features). f .

[0125] Finally, in F f The spatial attention mechanism is applied to generate a mask and apply it to F. fusion Perform dynamic weighting.

[0126] Final output F out This further enhances the response to defective areas.

[0127] The following are the relevant mathematical expressions and the network framework diagram of HRDS:

[0128]

[0129] H f =F fusion —AvgPool 3×3 (F fusion )

[0130] F out =F fusion ⊙(1+σ(DWConv 3×3 (H f )))

[0131] Where Identity(X) represents the identity mapping, that is, the input X is directly output without any change;

[0132] dim(X) represents the dimension of tensor X;

[0133] dim(F down ) represents the dimension of the tensor Fdown;

[0134] SE(Conv 1×1(X)) indicates that the input X is first subjected to a 1x1 convolution, and then SE attention operation is performed;

[0135] H f This indicates the enhanced high-frequency detail features generated through high-frequency filtering;

[0136] AvgPool 3×3 This represents an average pooling operation with a kernel size of 3x3;

[0137] σ() represents the Sigmoid activation function, whose output value is between 0 and 1, and can be used as attention weights;

[0138] F out This represents the final output feature map of the HRDS module;

[0139] 2. Use Ghost Backbone for feature extraction:

[0140] The core building block of the backbone network is the Ghost Backbone module.

[0141] This module is inspired by standard residual blocks, but its internal convolutions are replaced by efficient Ghost Modules. A Ghost Module first uses a 1×1 convolution to generate a small number of "intrinsic feature maps";

[0142] Then, a series of computationally inexpensive linear operations (such as depthwise separable convolutions) are applied to these intrinsic feature maps to generate multiple "phantom feature maps".

[0143] Finally, the intrinsic feature map and the phantom feature map are concatenated along the channel dimension to form the final output.

[0144] Suppose the goal is to generate a feature map with 256 channels. The traditional approach is to use a standard convolutional layer with 256 kernels, which is computationally expensive. The Ghost Module takes a different approach: First, it generates 128 "intrinsic feature maps" using a less computationally expensive 1x1 convolution (e.g., with only 128 kernels). Then, for each of these 128 intrinsic feature maps, it applies an extremely inexpensive linear operation (such as a 3x3 depthwise separable convolution) to generate another 128 "phantom feature maps." These phantom feature maps are generated based on the intrinsic feature maps and therefore contain similar but slightly modified feature representations. Finally, the 128 intrinsic and 128 phantom feature maps are concatenated along the channel dimension to obtain the final 256-channel output. In this way, the same number of feature maps are obtained with far less computation than traditional convolutions.

[0145] The Ghost Backbone consists of two stacked Ghost Modules and a residual connection. The first Ghost Module is used to increase the number of channels (expansion layer), and the second is used to decrease the number of channels to match the dimension of the residual connection.

[0146] 3. Integrated SE attention mechanism:

[0147] To enable the network to adaptively focus on more important feature channels, Squeeze-and-Excitation (SE) attention modules are embedded in the residual branches of some Ghost Backbone modules at deeper stages of the backbone network (e.g., when processing low-resolution, highly semantically informative feature maps). The SE module captures global information for each channel through a global average pooling (Squeeze) operation, and then learns the non-linear dependencies between channels through two fully connected layers (Excitation), generating a set of channel weights. These weights are used to reweight the original feature maps channel by channel, thereby enhancing useful features and suppressing irrelevant features. Table 1 shows the detailed layer specifications of the YOLOv8-GHR backbone network.

[0148] Table 1

[0149] Stage Operation Output channel number Kernel size / stride Number of repetitions Description Stem HRDS Module 64 N / A 1 Replace the first stride convolution, keep and enhance fine-grained features. Stage 1 Ghost Backbone 128 s＝2 1 Downsampling module. Stage 2 Ghost Backbone 128 s＝1 2 Feature extraction module. Stage 3 Ghost Backbone 256 s＝2 1 Downsampling module. Stage 4 Ghost Backbone(+SE) 256 s＝1 3 Feature extraction module with channel attention. Stage 5 Ghost Backbone 512 s＝2 1 Downsampling module. Stage 6 Ghost Backbone(+SE) 512 s＝1 2 Final feature extraction module with channel attention.

[0150] Feature fusion neck network design (Slim-Neck with RCCA)

[0151] The neck network of this method has made two key improvements on the PANet (Path Aggregation Network) topology of YOLOv8 to achieve efficient global contextual feature fusion.

[0152] 1. Slim-neck structure based on GSConv:

[0153] All standard convolutional layers in the neck network are replaced with GSConv modules to construct a lightweight Slim-neck.

[0154] The GSConv module is a novel convolutional method that halves the number of channels in the input feature map, processing one half through standard convolution and the other half through depthwise separable convolution, which has a lower computational cost.

[0155] The results from the two parts are then concatenated and channel shuffled.

[0156] This design allows the computational results of depthwise separable convolutions to be fully integrated into the results of standard convolutions, simulating the performance of standard convolutions at a lower computational cost, thereby significantly reducing the computational burden on the neck network while maintaining feature fusion capabilities.

[0157] Suppose a GSConv module takes a 128-channel input feature map. The module first splits these 128 channels into two halves, each with 64 channels. The first half's 64 channels are fed into a standard convolutional layer. The second half's 64 channels are fed into a less computationally expensive depthwise separable convolutional layer. After processing, the 64-channel feature maps from each branch are concatenated to restore the 128-channel configuration. Finally, a channel shuffle operation thoroughly mixes the information from the two branches, allowing the features extracted by the depthwise separable convolution to permeate the features of the standard convolution. This significantly reduces computation while simulating performance close to that of a standard convolution.

[0158] 2. Integrated RCCA module:

[0159] At the output of PANet's highest-level feature map (i.e., the smallest feature map with the richest semantic information), before it is fed into the detection head, an RCCA (Recurrent Criss-Cross Attention) module is inserted.

[0160] The RCCA works as follows:

[0161] 1) Criss-Cross Attention:

[0162] For each pixel in the feature map, this module only calculates the attention relationship between it and all other pixels in the same row and column. This "cross-shaped" attention calculation method has a computational complexity of O(N(H+W)), which is much lower than the O(V) of the standard self-attention mechanism. 2 ), where N is the total number of pixels, and H and W are the height and width of the feature map.

[0163] 2) Recurrent Operation:

[0164] By applying the RCCA module twice (R=2), efficient aggregation of global information can be achieved. After the first calculation, each pixel has already incorporated the contextual information of its row and column. During the second calculation, when a pixel again aggregates information along its cross-shaped path, it indirectly collects information from the entire image. This cyclical mechanism achieves highly efficient contextual modeling across the entire image, which is crucial for identifying long, narrow scratches and cracks.

[0165] Taking a single pixel P on a feature map as an example, the RCCA module works as follows:

[0166] First cross-attention: The module calculates the attention relationships between point P and all other pixels in its row, as well as all other pixels in its column. After calculation, point P's new feature value incorporates all the information from its "cross" path.

[0167] Second application loop: When the module is applied a second time, point P again aggregates information along its cross-shaped path. This time, however, each other pixel on the path (e.g., pixel Q in the same row as P) has already incorporated information from its own cross-shaped path (i.e., the row and column of Q) during the first calculation. Therefore, when P aggregates information from Q, it indirectly obtains information from the column where Q is located. Through this loop, information is propagated across the entire feature map. After two loop operations, point P can efficiently aggregate contextual information from the entire map, achieving a global receptive field, while the computational complexity is far lower than calculating attention directly between point P and all other points in the map.

[0168] Detection head and loss function

[0169] 1) Decoupled Detection Head:

[0170] This method retains YOLOv8's advanced decoupled head structure. This structure sets up independent network branches for classification and regression tasks, avoiding optimization conflicts between the two, and has been proven to effectively improve detection performance.

[0171] 2) WIoU loss function:

[0172] This method uses WIoU as the bounding box regression loss function. The WIoU loss consists of two parts: distance attention and dynamic non-monotonic focusing mechanism.

[0173] Distance attention requires defining an attention term based on Euclidean distance. When the predicted bounding box is far from the center point of the ground truth bounding box, this term will have a smaller weight, thus reducing the competitiveness of high-quality samples. Its calculation formula is:

[0174]

[0175] Where (x,y) and (x gt ,y gt These are the center coordinates of the predicted bounding box and the ground truth bounding box, respectively.

[0176] (W g H g ) is the width and height of the smallest rectangle that encloses both.

[0177] The superscript * indicates that this part is not involved in gradient calculation to prevent the generation of gradients that would hinder convergence.

[0178] R WIoU Indicates dynamic weights;

[0179] The dynamic non-monotonic focusing mechanism introduces a dynamic weight based on "outlier." Outlier β is defined as the ratio of the IoU loss of the current sample to the dynamic average of the IoU loss, and is used to measure the quality of an anchor box.

[0180]

[0181] Where α and δ are hyperparameters;

[0182] r represents the weighting factor or gradient gain of the dynamic non-monotonic focusing mechanism. It is adjusted according to the outlier β of the anchor box, preferentially allocating larger gradient contributions to medium-quality samples, while suppressing the gradient contributions of high-quality (simple) and low-quality (abnormal) samples, thereby improving the generalization ability of the model;

[0183] This represents the dynamic moving average of the IoU (Intersection over Union) loss. It serves as a dynamic benchmark to measure the outlier nature of the current sample's IoU loss.

[0184] L IoU This represents the standard IoU (Intersection over Union) loss, used to measure the degree of overlap between the predicted bounding box and the ground truth bounding box in the current sample;

[0185] This focusing mechanism results in smaller values ​​for r when the anchor frame quality is good (small β) or poor (large β), thus reducing their gradient contributions. However, when the anchor frame quality is at a moderate level, the value of r is larger, and the model will focus on optimizing these "average quality" samples because they are the most critical to improving the model's generalization ability.

[0186] The final WIoU loss is the product of the two parts mentioned above and the standard IoU loss:

[0187] L WIoU =r·R WIoU ·L IoU

[0188] Among them, L WIoU Indicates the final loss;

[0189] r represents the weighting factor or gradient gain generated by the dynamic non-monotonic focusing mechanism. It is adjusted by the outlier β, which assigns higher weights to medium-quality samples while reducing the contribution of high-quality and low-quality samples to the loss calculation, thereby guiding the model to learn more robustly and efficiently.

[0190] In this embodiment, histogram equalization is applied to all input images before training begins. This operation effectively standardizes image contrast, reduces the negative impact of changes in ambient lighting conditions on model performance, and improves the model's environmental adaptability. During training, this strategy randomly selects four images, randomly scales, crops, and rearranges them, then stitches them together to form a new training image. This greatly enriches the background and target combinations of the training samples, forcing the model to learn to detect targets in more complex scenes (such as partially occluded targets or targets with drastic size changes), effectively improving the model's robustness and ability to detect small targets. In cases of extremely scarce data, LS-DCGAN can be selectively used to generate synthetic defect images to further expand the training set.

[0191] Training Strategy: The model was trained and evaluated on the publicly available NEU-DET steel surface defect dataset. This dataset contains six typical defects: rolled-in scale, inclusions, crazing, scratches, patches, and pitted surface. The training process employed a stochastic gradient descent (SGD) optimizer with a batch size of 16, an initial learning rate of 0.01, and a total of 300 training epochs.

[0192] Inference and Industrial Deployment: Model Optimization. After training, a series of optimization operations can be performed on the model to adapt it for edge device deployment. For example, model pruning techniques can be used to remove redundant weights and channels, and INT8 quantization can be applied to convert model weights from 32-bit floating-point numbers to 8-bit integers. These operations can significantly reduce the model size and speed up inference with almost no impact on accuracy.

[0193] Hardware integration: The optimized YOLOv8-GHR model can be efficiently deployed on edge computing hardware commonly used in industrial settings, such as NVIDIA Jetson Xavier NX or Orin series, with the help of inference acceleration engines such as NVIDIA TensorRT.

[0194] System Integration: In a real-world quality control production line, the model proposed in this method will serve as the core visual perception module. The deployment process is as follows: industrial cameras on the production line capture real-time images of the steel surface. These images are then fed into an edge computing device deployed with the YOLOv8-GHR model for real-time inference. The model's output detection results (including defect category, location coordinates, and confidence level) can be used by downstream systems to trigger alarms, mark non-conforming products, or drive a robotic sorting system, thereby achieving a fully automated quality control closed loop.

[0195] Simulated ablation experiments: To verify the independent contributions of each innovative component in this method and their synergistic effects, a set of simulated ablation experiments was designed. Starting with the baseline model YOLOv8n, the core modules of this method were gradually added, and the performance metrics changes on the NEU-DET dataset were recorded. The experimental results strongly demonstrate the rationality and systematic advantages of the proposed method, as shown in Table 2, which presents the module ablation experiments.

[0196] Table 2

[0197] Model configuration Key added modules mAP@0.5(%) Parameter amount(M) GFLOPS Baseline model(YOLOv8n) - 79.1 3 8.1 Model A +Ghost Bottleneck 80.2 2.4 6.9 Model B +Ghost B.+HRDS Module 81.7 2.5 7 Model C +Ghost B.+HRDS+WloU 82.5 2.5 7 YOLOv8-GHR(Complete model) +Ghost B.+HRDS+WloU+RCCA 84.2 2.6 7.2

[0198] The ablation experiments clearly demonstrate that each component of this method contributes positively to the final performance. From model A to the complete YOLOv8-GHR, mAP@0.5 improved by more than 5 percentage points, while the number of parameters and GFLOPS decreased compared to the original YOLOv8n. This proves that the YOLOv8-GHR architecture is not a simple stacking of modules, but a carefully designed, highly efficient system where all components work together.

[0199] The visualization effect is shown below:

[0200] like Figure 4 As shown, this figure displays the actual detection results of the YOLOv8-GHR model on the NEU-DET dataset for some steel surface defects. The figure includes detection samples of four different defects:

[0201] The top left image detected "crazing" with a confidence level of 0.52, and the detection box accurately covered the dense network of cracks.

[0202] The top right image detected "inclusion" with a confidence level of 0.79. A slender bounding box precisely located the vertically distributed strip defects.

[0203] The lower left image shows the detection of "patches" with a confidence level of 0.82. The detection box tightly surrounds the irregularly shaped dark patch defect.

[0204] The lower right image shows two instances of "rolled-in scale" (oxidized scale), indicating that the model can simultaneously identify multiple defect instances within the field of view.

[0205] These visualization results intuitively demonstrate that the model proposed in this invention has high-precision positioning and accurate classification capabilities, especially when dealing with defects of different shapes (such as linear, blocky, and mesh-like).

[0206] The above-described embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. A method for detecting surface defects in steel based on an improved YOLOv8 network, characterized in that: Includes the following steps: Preprocess the input image of the steel surface; The preprocessed image is fed into an improved YOLOv8 network, which consists of a backbone network, a neck network, and a decoupled detection head. The backbone network uses the Ghost Backbone module as the core feature extraction unit to perform multi-scale feature extraction on the input image, and includes the HRDS module, which is used to preserve high-frequency detail information in the image through hierarchical residual downsampling, and transmits the output feature map containing high-frequency detail information to the neck network. The neck network receives the feature map output by the backbone network, performs cross-scale feature fusion on it, and generates a fused feature map with global context information; its feature fusion path is constructed by the GSConv module and captures global context information through the RCCA module. The decoupled detection head receives the fused feature map output by the neck network, performs decoupling processing on it, and outputs defect category classification information and bounding box regression information respectively.

2. The method for detecting steel surface defects based on an improved YOLOv8 network as described in claim 1, characterized in that: The HRDS module implements hierarchical residual downsampling through a multi-path structure, specifically including: The first path extracts multi-scale contextual features through multiple parallel convolutions with different dilation rates, and then downsamples them after weighting by the SE attention mechanism; The second path extracts high-frequency detail features from the input image through depthwise separable convolution; The third path involves skip connections that adjust the dimensions by performing identity mapping or 1×1 convolution based on the consistency of the input and output dimensions. The three output features are fused and then output after high-frequency filtering and spatial attention weighting.

3. The method for detecting steel surface defects based on an improved YOLOv8 network as described in claim 1, characterized in that: The neck network adopts a Slim-Neck architecture with an integrated RCCA module for global context-aware feature fusion, in which a lightweight feature fusion path is constructed using the GSConv module.

4. The method for detecting steel surface defects based on an improved YOLOv8 network as described in claim 1, characterized in that: The improved YOLOv8 network uses the WIoU loss function as the bounding box regression loss for model training; the WIoU loss function includes a distance attention term and a dynamic non-monotonic focusing mechanism, wherein the dynamic non-monotonic focusing mechanism adjusts the gradient contribution of samples of different quality through outlier β.

5. The method for detecting steel surface defects based on an improved YOLOv8 network as described in claim 1, characterized in that: The Ghost Backbone module generates intrinsic features through the Ghost module, and then constructs the Backbone structure by splicing and residual connections.

6. The method for detecting steel surface defects based on an improved YOLOv8 network as described in claim 1, characterized in that: The GSConv module simulates the performance of standard convolution by halving the input channels and processing them separately through standard convolution and depthwise separable convolution, then concatenating them and performing channel shuffling.

7. The method for detecting surface defects in steel based on an improved YOLOv8 network as described in claim 1, characterized in that, The preprocessing steps include: performing histogram equalization on the input image and performing at least one random data augmentation operation, including random scaling, cropping, and image stitching.

8. A lightweight, high-precision steel surface defect detection system based on a hybrid YOLOv8 architecture, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 7.

9. A lightweight, high-precision steel surface defect detection system based on a hybrid YOLOv8 architecture, characterized in that, The system includes industrial cameras, edge computing devices, and YOLOv8-GHR models deployed in industrial sites.

10. The steel surface defect detection system based on the improved YOLOv8 network as described in claim 9, characterized in that: It also includes a predictive information sending device, which is used to send out detection results, which are used to trigger alarms, mark non-conforming products, or drive a robot sorting system.

Images