Method for detecting small defects on surface of lightweight steel

Abstract

The invention discloses a method for detecting small defects on the surface of lightweight steel, and belongs to the technical field of industrial machine vision. The method is based on an improved YOLOv8n architecture and cooperatively works through three core technical means: firstly, a Swin Transform module is introduced into a backbone network to model long-distance spatial dependence and suppress complex background interference; secondly, a high-resolution P2 detection branch is constructed, shallow details and up-sampling semantic features are fused, and microdefect characterization is enhanced through bidirectional refining circulation; finally, P2 exclusive adaptive threshold focus loss ATFL is adopted, threshold update is only limited to P2 detection branch samples, and precise optimization of difficult and tiny defects is achieved in cooperation with a gradient directional return mechanism. According to the scheme, the detection recall rate and the positioning precision of the tiny defects are effectively improved while the model parameter quantity is remarkably reduced, and high-precision and light-weight industrial deployment is realized.

Description

Technical Field

[0001] This invention relates to the field of industrial machine vision technology, and in particular to a method for detecting minute defects on the surface of lightweight steel. Background Technology

[0002] In modern industrial automated production, product quality control is a crucial link in ensuring product safety and reliability. Taking industries such as steel, semiconductors, and textiles as examples, even minute defects on the product surface (such as inclusions, dents, and scratches), though small in size, can seriously affect its mechanical properties, appearance quality, and even the safety of the final product. Traditional quality inspection methods based on manual visual inspection or simple image processing are not only inefficient and highly subjective, but also difficult to meet the real-time requirements of high-speed production lines. In recent years, deep learning-based target detection technology, especially the YOLO series algorithms, has become a research hotspot in the field of industrial vision inspection due to its good balance between detection accuracy and inference speed. However, unlike general object detection scenarios, industrial imaging environments have their unique characteristics: First, the inspected surfaces often have strong specular reflections, oxide scale, and repetitive rolling textures, leading to high visual confusion between real defects and background artifacts; second, critical defects are often extremely small (<0.1% of image area), making them prone to detail loss during downsampling by deep networks; most challengingly, real-world defect datasets exhibit an extreme long-tail distribution, with a scarcity of rare but highly harmful micro-defect samples, which are easily overwhelmed by massive amounts of simple negative samples during training, resulting in low model recall. These factors collectively constitute the "triple dilemma" of industrial micro-defect detection, urgently requiring a dedicated solution that can collaboratively handle strong interference suppression, micro-detail preservation, and difficult sample learning.

[0003] Chinese invention patent application CN202410626863.3 proposes a target detection method for road scenes based on an improved YOLOv8n architecture. It adds a 160×160 detector head to the P2 stage of the YOLOv8n backbone network to capture small distant targets (such as pedestrians and vehicles). However, the design intent and technical approach of this scheme differ fundamentally from industrial quality inspection scenarios, making direct application difficult. Specifically, firstly, this scheme focuses on open and dynamic road environments with relatively simple backgrounds and clear target semantics. Therefore, it only optimizes localization accuracy by replacing the bounding box regression loss (CIoU→EIoU), failing to address the severe feature confusion problem caused by complex textures in industrial scenarios. Secondly, and most importantly, this scheme lacks a dynamic perception mechanism for the difficulty of samples in classification tasks. While the added P2 detector head improves spatial resolution, its loss function still applies to all global samples, failing to effectively cope with the interference of massive, easily classified negative samples in industrial datasets on the learning of minor defects. This scheme solves the problem of seeing more clearly, but fails to address the core challenge of learning more accurately. Therefore, when faced with highly interfering and unbalanced industrial surfaces such as steel, its ability to recall critical micro-defects remains insufficient. In view of this, there is an urgent need in this field for a novel, scenario-customized detection framework. This framework should not only possess high-resolution perception capabilities but also be equipped with an intelligent optimization mechanism that can precisely guide the model's learning focus to micro-defects and adapt to the learning difficulty, in order to achieve the high precision and high reliability required by industrial quality inspection. Summary of the Invention

[0004] In view of this, the present invention proposes a lightweight method for detecting minute defects on the surface of steel. This scheme employs a Swing Transformer backbone to enhance global context modeling capabilities, constructs a high-resolution P2 detection branch to preserve micro-defect details, and uses P2-specific adaptive threshold focus loss (ATFL) for directional optimization, significantly improving the recall and detection accuracy of minute defects while simultaneously achieving a lightweight model.

[0005] This invention provides a method for detecting minute defects on the surface of lightweight steel, comprising the following steps: S1 uses the Swin Transformer backbone network to extract features from the input steel image and outputs a multi-level feature map. S2 constructs a multi-scale feature fusion network and a P2 detection branch specifically for detecting minute defects. The multi-scale feature fusion network fuses the shallow feature map of the Swin Transformer backbone network with the upsampled mid-level feature map to generate a fused high-resolution feature map. The fused high-resolution feature map is then provided to the P2 detection branch to generate defect category and localization prediction for candidate defect regions. S3 uses Adaptive Threshold Focus Loss (ATFL) to calculate the classification loss. The ATFL dynamically maintains an adaptive threshold for each defect category, reflecting the real-time difficulty level of the current minor defect detection task. The update calculation of the adaptive threshold is limited to the sample set that the P2 detection branch is responsible for predicting. S4 optimizes the multi-scale feature fusion network based on the classification loss calculated by the ATFL, and mainly backpropagates the gradient of the classification loss to the feature fusion module and its detection head parameters in the P2 detection branch, thereby enhancing the ability of the P2 detection branch to distinguish between real small defects and background artifacts.

[0006] Furthermore, the Swin Transformer backbone network models the long-distance spatial dependence of the steel surface through its window self-attention mechanism.

[0007] Furthermore, the resolution of the P2 detection branch is 160×160, and the shallow feature map is further downsampled to 80×80 resolution and fused with the middle feature map for a second time, thereby injecting fine-grained information of minute defects into higher-level semantic features in reverse, forming a bidirectional feature refinement loop.

[0008] Furthermore, the multi-scale feature fusion network is a customized network based on the PAN-FPN structure, in which the PAN path contains a feature enhancement module specifically designed for the P2 detection branch. The feature enhancement module receives shallow and mid-level feature maps from the backbone network, upsamples the mid-level feature map by 2x, and then performs element-wise addition and fusion with the shallow feature map. Finally, it outputs the fused high-resolution feature map through a feature refinement unit containing multiple convolutional layers and activation functions.

[0009] Furthermore, the update formula for the adaptive threshold is: , in, The adaptive threshold for the current training batch, where t is the training batch size. The adaptive threshold of the previous training batch. The momentum coefficient, This represents the number of positive samples in the current batch predicted by the P2 detection branch, with the true label being category c. Let be the prediction confidence level of the i-th positive sample.

[0010] Furthermore, the gradient backpropagation of the classification loss mainly refers to the fact that, during the backpropagation process, the classification loss gradient weight assigned to the P2 detection branch is higher than the classification loss gradient weight assigned to other scale detection branches P3, P4, and P5, so that the optimized signal is preferentially used to improve the ability of the P2 detection branch to distinguish small defects.

[0011] Furthermore, the Swin Transformer backbone network is an improvement on the YOLOv8n model architecture. By introducing at least two Swin Transformer modules into this architecture, the model's ability to learn complex patterns is enhanced, while maintaining the lightweight nature of the overall network. The Swin Transformer modules adopt a self-attention mechanism with a window size of 3x3 to improve computational efficiency and model performance.

[0012] Furthermore, the present invention also provides a device for detecting minute defects on the surface of lightweight steel, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the above-described method for detecting minute defects on the surface of lightweight steel.

[0013] Furthermore, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method for detecting minute defects on the surface of lightweight steel.

[0014] The present invention has the following advantages over the prior art: By integrating the Swin Transformer module into the backbone network, its window self-attention mechanism is used to effectively model the long-distance spatial dependence of the steel surface. While maintaining computational efficiency, the model can perform global context-aware feature extraction, which significantly enhances the ability to distinguish real small defects under strong reflection, oxide scale and other large-area background noise.

[0015] By constructing a high-resolution P2 detection branch and fusing shallow detail features with upsampled semantic features, the feature representation of tiny defect regions is greatly enriched, effectively restoring the fine spatial details lost due to deep downsampling, thereby significantly improving the localization accuracy and detection rate of subpixel-level tiny defects.

[0016] By introducing a bidirectional feature refinement loop and constructing a high-resolution P2 detection branch, the detection grid density is increased by four times compared to the traditional P3 head, greatly enhancing the ability to finely locate and detect sub-pixel-level micro-defects.

[0017] By employing the Adaptive Threshold Focus Loss (ATFL) and strictly limiting threshold updates to the sample set of the P2 detection branch, dynamic perception and accurate response to the difficulty of the minute defect detection task are achieved. This mechanism automatically assigns higher learning weights to difficult and rare defect samples that are hard to classify, thereby effectively guiding the optimization process, effectively alleviating the problem of extreme class imbalance, and significantly improving the recall rate of key minute defects.

[0018] By primarily backpropagating the classification loss gradient calculated by ATFL to the P2 detection branch, targeted optimization of the high-resolution detection path is achieved. This strategy ensures that valuable optimization signals are prioritized for improving the core discriminative power of the P2 detection branch, avoiding resource dispersion caused by global optimization. Ultimately, this reduces the overall number of model parameters while achieving a synergistic improvement in detection accuracy.

[0019] By leveraging the synergistic mechanism of ATFL, Swin Transformer, and YOLO, the model achieves high precision and high recall for detecting minute defects in industrial scenarios while maintaining a lightweight design. Its overall performance significantly outperforms the results achieved by simply combining individual technical features. Experiments on the NEU-DET and GC10-DET benchmarks demonstrate that, with a 16.0% reduction in parameters (3.157M → 2.650M), this approach achieves a +0.9% improvement in mAP@50 and a +1.5% improvement in mAP@50–95 on NEU-DET; on the more complex GC10-DET, the improvement reaches +1.7% in mAP@50–95. Ablation studies and cross-dataset evaluations further confirm the complementarity of the components and the model's strong generalization ability to unknown defects and imaging conditions, validating its significant value as a high-performance, practical industrial inspection solution. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention; Figure 2 This is a diagram showing the overall architecture of YOLOv8 according to an embodiment of the present invention; Figure 3 This is a diagram of the improved YOLOv8n model architecture according to an embodiment of the present invention; Figure 4 This is a structural diagram of the high-resolution P2 detector head according to an embodiment of the present invention; Figure 5 These are visual example images of the six defect categories in the NEU-DET dataset from an embodiment of the present invention. Figure 6 A visual example of a defect category in the GC10-DET dataset of this invention; Figure 7 This is a comparison of the detection results of the original YOLOv8n model and the improved algorithm on the NEU-DET dataset in this embodiment of the invention; Figure 8 This is a qualitative comparison chart of the detection results of the GC10-DET dataset in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, the present invention provides a method for detecting minute defects on the surface of lightweight steel, comprising the following steps: S1 extracts features from the input steel image based on the Swin Transformer backbone network and outputs a multi-level feature map; the Swin Transformer backbone network models the long-distance spatial dependence of the steel surface through its window self-attention mechanism.

[0024] The Swin Transformer backbone network is an improvement on the YOLOv8n model architecture. By introducing at least two Swin Transformer modules into this architecture, the model's ability to learn complex patterns is enhanced, while maintaining the lightweight nature of the overall network. The Swin Transformer modules adopt a self-attention mechanism with a window size of 3x3 to improve computational efficiency and model performance.

[0025] YOLOv8 is a versatile object detection framework that supports a variety of computer vision tasks, including object detection, instance segmentation, pose estimation, and image classification, while maintaining a good balance between inference speed and detection accuracy.

[0026] like Figure 2As shown, its architecture follows a classic three-stage design: a backbone network, a neck network, and a head network. The backbone network is derived from CSPDarknet (as used in YOLOv5), but a C2f module is introduced to replace the original C3 block. This improvement enhances feature reuse and gradient flow through dense shortcut connections and feature segmentation, thereby improving representation capabilities without significantly increasing computational cost. The backbone network processes the input image through a series of convolutional layers with 2-step downsampling, generating multi-scale feature maps at three resolutions: P3 (high resolution, low-level semantics), P4, and P5 (low resolution, high-level semantics). At the end of the backbone network, an SPPF (Spatial Pyramid Pooling-Fast) module is used to enhance robustness to multi-scale objects.

[0027] The neck region integrates these multi-level features through an optimized PAN-FPN (Path Aggregation Network and Feature Pyramid Network) structure. It achieves bidirectional feature fusion: the top-down path propagation uplink, through upsampling and feature concatenation, transfers high-level semantic information from layer P5 to layer P3; the downlink path, through downsampling and feature fusion, extracts spatial details from lower to higher levels. This dual-path mechanism effectively integrates fine-grained localization cues with strong semantic context, thereby improving cross-scale detection performance.

[0028] The head network performs the final prediction based on the feature maps fused from the neck region. YOLOv8 employs an anchor-free design, directly regressing the center coordinates, width, and height of the bounding box without relying on predefined anchor templates. This design simplifies the training process and enhances flexibility in handling objects of arbitrary shapes. Furthermore, the head network uses a decoupled architecture, separating the classification and localization tasks into different branches. Each branch independently outputs the class probability and the object presence probability.

[0029] By utilizing the confidence scores and bounding box parameters on three feature maps (corresponding to P3, P4, and P5), more professional and accurate predictions can be achieved. YOLOv8 offers five model sizes: N, S, M, L, and X, with YOLOv8n having the fewest parameters. Due to its small parameter count, low computational overhead, and competitive accuracy, YOLOv8n is particularly suitable for industrial applications where real-time performance and edge device deployment are critical. Therefore, this invention selects YOLOv8n as the baseline model.

[0030] like Figure 3 As shown, in the original YOLOv8n model's backbone network, this invention replaces some C2f modules with Swing Transformer modules. Figure 3 (Two yellow boxes in the middle) to solve the problem that traditional CNNs have limited receptive fields and are difficult to model long-distance spatial dependencies of steel surfaces.

[0031] Swin Transformer, through its window-based self-attention mechanism, can effectively capture global contextual relationships across regions in an image, such as large-area rolling textures or continuous scratches. This allows the model to distinguish between real defects and background interference such as strong reflections and oxide scale during the feature extraction stage, significantly improving its noise resistance.

[0032] In one specific embodiment, the Swin Transformer module uses a fixed 3×3 window size and is embedded in the intermediate stage of the backbone network, directly replacing the original C2f module. This design does not require refactoring the overall architecture, and by decomposing global self-attention into local window computations, the complexity is reduced from O(N²) to O(N), controlling computational overhead while maintaining high modeling capabilities, thus balancing performance and efficiency.

[0033] This step enhances the backbone network's global perception and anti-interference capabilities by strategically integrating the Swing Transformer module, providing high-quality, robust multi-level feature map outputs for subsequent high-precision small defect detection.

[0034] S2 constructs a multi-scale feature fusion network and a P2 detection branch specifically for detecting minute defects. The multi-scale feature fusion network fuses the shallow feature map of the Swin Transformer backbone network with the upsampled mid-level feature map to generate a fused high-resolution feature map. The fused high-resolution feature map is then provided to the P2 detection branch to generate defect category and localization prediction for candidate defect regions. The P2 detection branch has a resolution of 160×160, and the shallow feature map is further downsampled to 80×80 resolution and fused with the middle feature map for a second time, thereby injecting fine-grained information of minute defects into higher-level semantic features, forming a bidirectional feature refinement loop.

[0035] The multi-scale feature fusion network is a customized network based on the PAN-FPN structure. Its PAN path contains a feature enhancement module specifically designed for the P2 detection branch. The feature enhancement module receives shallow and mid-level feature maps from the backbone network, upsamples the mid-level feature map by 2x, and then performs element-wise addition and fusion with the shallow feature map. The fused high-resolution feature map is then output through a feature refinement unit containing multiple convolutional layers and activation functions.

[0036] The standard YOLOv8 model employs three detection scales (P3: 80×80, P4: 40×40, P5: 20×20), but its ability to detect minute defects (such as small scratches or pitting) occupying only a few pixels is insufficient. To improve spatial resolution, this invention develops the P2 detection module, which has a working resolution of up to 160×160 pixels.

[0037] like Figure 4 As shown, this invention constructs a high-resolution P2 detection branch specifically for detecting minute defects within a multi-scale feature fusion network (Neck network). This branch has a resolution of 160×160 and aims to capture sub-pixel-level minute defects with higher spatial accuracy. The construction of the P2 detection branch is as follows: (1) Retain the high-resolution feature map (160×160, 256 channels) of the second stage of the backbone as a source of spatial detail; (2) Upsample the P3 feature map (80×80) to 160×160 and stitch it with the second-stage feature map; (3) The fused features are processed by the C2f module to reduce the channel dimension to 128; (4) The obtained P2 features are input into a dedicated detection head for micro-defect prediction.

[0038] Furthermore, the fused high-resolution P2 feature map is downsampled to 80×80 resolution and then fused with the feature map of the original P3 path, thus forming a reverse information injection from shallow to deep layers. This establishes a bidirectional refinement-memory loop that enriches the representations of both shallow and deep layers. Compared to P3, this design increases the effective detection grid density by 4 times, significantly improving the model's ability to locate and classify micro-defects.

[0039] Specifically, the multi-scale feature fusion network is custom-designed based on the PAN-FPN structure. Its core is a feature enhancement module specifically designed for the P2 detection branch. This module is located on the PAN path and receives shallow and mid-level feature maps from the SwinTransformer backbone network.

[0040] The implementation process of this feature enhancement module is as follows: Upsampling and fusion: The mid-level feature map is upsampled by 2x bilinear interpolation to improve its resolution to match that of the shallow feature map. Figure 1 To.

[0041] Element-wise addition: The upsampled mid-level feature map is fused with the original shallow feature map element-wise to generate a high-resolution feature map that incorporates rich detail and semantic information.

[0042] Feature refinement: The fused feature map is further refined and optimized through a feature refinement unit containing multiple convolutional layers and activation functions. The final output is used as the input to the P2 detection branch.

[0043] The P2 detection branch receives this high-resolution feature map and uses modules such as C2f to generate category and location predictions for candidate defect regions.

[0044] The P2 detection branch has a resolution of 160×160, while the P3 branch has a resolution of 80×80. This means that, for the same input image size, the P2 detection branch can provide four times the grid density of the P3 branch, significantly improving the detection capability for small targets. By combining shallow feature maps (rich in detail information) with upsampled mid-level feature maps (containing rich semantic information), the P2 detection branch can enhance its contextual understanding ability while preserving fine-grained features of small defects. The P2 detection branch not only utilizes the aforementioned feature fusion mechanism but also introduces a bidirectional feature refinement loop, further downsampling the fused high-resolution feature map to 80×80 resolution and then fusing it a second time with the original P3 path feature map. This process injects fine-grained information of small defects back into higher-level semantic features, forming a closed loop of positive propagation and negative feedback from low to high levels, significantly enhancing the model's ability to recognize small defects.

[0045] To verify the effectiveness of the P2 detection branch, tests were conducted on two benchmark datasets, NEU-DET and GC10-DET, and its performance was compared with that of the original YOLOv8n model.

[0046] On the NEU-DET dataset, after adopting the P2 detection branch, the mAP@50 accuracy improved from 76.7% to 77.6%, and the mAP@50-95 accuracy improved from 41.0% to 42.5%. In particular, the detection recall rate for small defect categories such as inclusions (In) and pitted surfaces (PS) was significantly improved.

[0047] On the GC10-DET dataset, the improved model also demonstrated superior performance, with mAP@50 reaching 77.6%, an improvement of 1.4 percentage points compared to the baseline model, and mAP@50-95 also improving from 41.0% to 42.7%.

[0048] Furthermore, ablation studies have confirmed that using the P2 detection branch alone or in combination with other components can bring varying degrees of performance improvement, especially demonstrating stronger capabilities when dealing with minute defects.

[0049] S3 uses Adaptive Threshold Focus Loss (ATFL) to calculate the classification loss. The ATFL dynamically maintains an adaptive threshold for each defect category, reflecting the real-time difficulty level of the current minor defect detection task. The update calculation of the adaptive threshold is limited to the sample set that the P2 detection branch is responsible for predicting. Understandably, industrial defect datasets naturally exhibit a long tail: common defects (such as scratches) far outnumber rare but critical defects (such as lateral cracks). Standard cross-entropy loss functions tend to overfit to the majority of classes, resulting in poor recall for the minority of samples. To address this, focus loss improves by reducing the weight of correctly classified samples, while variants such as SloU and internal crossover ratio (IoU) optimize by enhancing location confidence. However, these loss functions rely on fixed focus parameters and static classification thresholds (e.g., 0.5), failing to account for the dynamic distribution of difficulty across defect categories and training epochs. Therefore, these methods remain insufficient in scenarios where rare defects consistently exhibit low prediction confidence throughout training, requiring adaptive learning focus.

[0050] To address the widespread problem of extreme class imbalance in industrial micro-defect detection—that is, the massive number of easily classified negative samples severely interferes with the model's learning process for rare but critical micro-defects, causing the optimization direction to deviate and ultimately affecting the recall rate—this solution proposes a novel classification loss function—Adaptive Threshold Focus Loss (ATFL)—and deeply integrates it with the entire detection framework.

[0051] Specifically, the core idea of ​​ATFL is to dynamically adjust the classification threshold for each defect category, enabling it to reflect the actual difficulty level of the current minor defect detection task in real time. However, a crucial design decision is that the calculation of this dynamic threshold update is not applied to all samples globally, but is strictly limited to the sample set predicted by the high-resolution P2 detection branch. This "P2-specific" strategy is the fundamental difference between this invention and existing technologies. It means that the optimization signal of ATFL is entirely driven by the prediction performance of the P2 detection branch, ensuring that the loss function accurately captures the changes in the difficulty of the specific task of minor defect detection, rather than being overwhelmed by a large number of simple samples at other scales (such as P3, P4, P5).

[0052] To implement this mechanism, ATFL introduces an adaptive threshold for each defect category c. This threshold is iteratively updated during training based on the prediction results of the current batch. Let... ∈(0,1] represents the predicted probability of a true positive sample. Unlike traditional Focal Loss, which uses a fixed threshold (e.g., 0.5) to distinguish between easy and hard samples, ATFL uses a class-specific adaptive threshold, which is updated according to the training batch as follows: , in, The adaptive threshold is the threshold value for the current training batch, where t is the training batch size. initial value It can be set to 0.5, which represents a neutral classification confidence benchmark. During training, this threshold will be dynamically adjusted based on the actual prediction performance of the P2 detection branch according to the update formula described above. The adaptive threshold of the previous training batch. This is the momentum coefficient, used to ensure smooth changes in the threshold, and is typically set to 0.05. This represents the number of positive samples predicted by the P2 detection branch in the current batch that have the true label of category c. Let be the prediction confidence of the i-th positive sample, satisfying ∈(0,1). This formula is used when a minor defect category is generally difficult to classify correctly in the current batch, i.e. When the level is low, its average prediction confidence will be low, thus leading to This decreases the difficulty of training. As a result, samples of this category are more likely to be considered "difficult samples" and receive higher learning weights, thus guiding the model to focus more on these challenging aspects.

[0053] Then, the formula for ATFL is: , in, The classification loss for a single sample; This is the balancing coefficient, used to control the weight of the entire loss term. It is usually set to 1 or determined experimentally. To predict confidence levels; The adaptive threshold is a dynamically maintained threshold for defect category c, reflecting the real-time difficulty level of the current task for that category; γ is a focusing parameter used to control the difference in how much attention the model pays to difficult and easy samples. ATFL aims to make the model focus more on samples whose predictions are close to the difficulty level of the current task.

[0054] In the Adaptive Threshold Focus Loss (ATFL) of this invention, in order to more accurately penalize difficult samples, an adaptive weight β and a focusing parameter γ are introduced. They work together on the loss function to achieve dynamic optimization for the task of detecting minute defects.

[0055] The adaptive weight β is designed as follows: , in, λ is a base scaling factor used to control the baseline level and overall magnitude of the entire adaptive weights. It is usually set to a small positive number, such as 0.5 or 1.0, to avoid excessively large weights that could lead to training instability. λ is a sensitivity coefficient used to control... The steepness of the function. It determines the prediction confidence. With dynamic threshold Deviation between For the final weight The strength of the influence of λ. A larger λ value makes the function response more sensitive, meaning that a small deviation can cause a significant change in the weights; a smaller λ value makes the response smoother. Its typical range is [0.1, 1.0], and the optimal value needs to be determined experimentally.

[0056] This weight will be applied to the prediction confidence level. With dynamic threshold Samples with significant deviations (i.e., hard-classified or misclassified samples) are penalized more severely. Specifically, when A larger value indicates either a very confident or very incorrect prediction. When β approaches 1, it causes β to approach 2a, thus amplifying the loss; conversely, when β approaches 1, it causes β to approach 2a. When the value is small (indicating that the prediction result is close to the threshold and in a "boundary" state), β is close to a, and the penalty is relatively mild. This allows the model to pay more attention to those difficult samples where the prediction is unstable.

[0057] The focusing parameter γ further adjusts the gradient through input dependence, and its calculation formula is as follows: , in, It is a very small numerically stable term, usually set to This design ensures that the loss function remains numerically stable even when the prediction confidence is extremely low (common in cases of minor defects or undetected defects), avoiding the problem of infinite gradients.

[0058] In practice, the ATFL loss function is integrated into the model's loss calculation module. During the forward propagation phase, the model first uses the P2 detection branch to generate prediction results. Subsequently, for each positive sample output by the P2 detection branch, the system calculates or updates the corresponding prediction based on its class and prediction confidence using the formula described above. Then, based on these dynamically adjusted thresholds... The final classification loss is calculated. This loss signal not only includes the traditional focus loss weighted by the difficulty of the samples, but also incorporates additional factors from... A determined, adaptive response to the current task difficulty.

[0059] S4 optimizes the multi-scale feature fusion network based on the classification loss calculated by the ATFL, and mainly backpropagates the gradient of the classification loss to the feature fusion module and its detection head parameters in the P2 detection branch, thereby enhancing the ability of the P2 detection branch to distinguish between real small defects and background artifacts.

[0060] The gradient backpropagation of the classification loss refers to the fact that, during the backpropagation process, the classification loss gradient weight assigned to the P2 detection branch is higher than the classification loss gradient weight assigned to other scale detection branches P3, P4, and P5, so that the optimized signal is given priority to improve the ability of the P2 detection branch to distinguish small defects.

[0061] In standard deep learning training, the gradient of the loss function is backpropagated to all trainable parameters of the network, either evenly or in a fixed proportion, via backpropagation. However, for the purposes of this invention, this fair optimization method is inefficient and even harmful. This is because the P2 detection branch is primarily responsible for detecting small defects, while other branches (such as P3, P4, and P5) mainly handle medium and large-sized targets. If all branches have the same gradient weights, the gradients from a massive number of simple samples will overwhelm the learning signal of the P2 detection branch for difficult small defects, causing the model optimization direction to deviate.

[0062] To address this issue, this scheme introduces a gradient weight allocation strategy during backpropagation. A higher weight coefficient, such as 1.5, is assigned to the classification loss of the P2 detection branch, while standard weights, such as 1.0, are assigned to the classification losses of other branches (P3, P4, P5). When the gradient of the ATFL loss begins backpropagation, the system scales the gradient flowing through each branch according to the aforementioned preset weight coefficients. The loss gradient output from the P2 detection branch is first multiplied by 1.5 before being used to update the parameters of the feature fusion module (C2f), the detector head, and the paths connecting to P2 in the backbone network within the P2 detection branch. The loss gradients output from the P3, P4, and P5 branches are directly multiplied by 1.0 for updating.

[0063] By increasing the gradient weights of the P2 detection branch, its loss signal is artificially amplified, thus enhancing its optimization priority. This ensures that in each iteration, the parameters adjusted by the optimizer Adam are more significantly influenced by the performance of the P2 detection branch. This allows the model to learn more quickly how to distinguish between minute defects and background artifacts.

[0064] Furthermore, since the gradient of the P2 detection branch is amplified while the gradients of other branches remain unchanged, this effectively prioritizes the allocation of limited optimization resources (gradient information) to the most critical detection task, avoiding over-optimization on a large number of simple samples.

[0065] This strategy forms a perfect closed loop with the ATFL in step S3. ATFL uses a dynamic threshold. The process accurately identifies difficult samples in the P2 detection branch and calculates the corresponding high loss; while the S4 step ensures that these high loss signals are processed first through gradient weight allocation, thus truly realizing an intelligent optimization process.

[0066] In summary, this step, through conscious gradient weight allocation, strategically focuses the optimization of the entire network on the P2 detection branch. This is the key difference between this invention and existing technologies, and it significantly improves the accuracy of detecting minute defects. The loss signal of ATFL is first applied to the P2 detection branch, and then, through the specific gradient weights of this branch, it guides parameter updates, thereby achieving a targeted enhancement of the ability to discriminate minute defects.

[0067] By employing a collaborative design of a dedicated ATFL for the P2 detection branch and directional gradient backpropagation, the entire optimization process forms a closed loop: the prediction performance of the P2 detection branch, dynamic updates to the ATFL threshold, calculation of a more accurate classification loss, gradient-prioritized backpropagation to the P2 detection branch, improvement of P2 detection branch performance, and enhanced prediction performance. This cycle continues, effectively addressing the problem of "hard samples being ignored" caused by global optimization in traditional methods.

[0068] To verify the effectiveness of the ATFL-Swin-YOLO method of this invention, comprehensive experiments were designed and evaluated on two widely used industrial surface defect detection benchmark datasets: NEU-DET and GC10-DET. All models were trained on an NVIDIA RTX 3060 GPU (6GB VRAM) using PyTorch 2.1 and CUDA 12.1. Input images were resized to 640×640 pixels, and the Adam optimizer was used with an initial learning rate of 0.001, gradually decaying over 300 epochs using cosine annealing. The batch size was set to 16, and other hyperparameters followed the default configuration of YOLOv8n. To ensure the reproducibility of the results, a random seed was fixed, and deterministic training mode was enabled.

[0069] (1) Data set and evaluation metrics NEU-DET: This dataset contains 1800 grayscale images of size 200×200, covering six typical steel defect types—cracks (Cr), inclusions (In), patches (Pa), pitted surfaces (PS), involute defects (RS), and scratches (Se), totaling 3000 samples. Figure 5 As shown, this dataset performs well in terms of background complexity, making it an ideal platform for testing the model's robustness to interference.

[0070] GC10-DET: This dataset contains 2294 labeled images (total sample size 3570), with an image size of 2048×1000 pixels, covering various defect morphologies such as weld type, oil stains, and crescent-shaped gaps. Figure 6 As shown, its background noise is much higher than that of NEU-DET, so it can be used as a test platform for cross-dataset generalization ability.

[0071] According to standard operating procedures, this invention primarily reports average accuracy (mAP@50) (IoU threshold = 0.5) and mAP@50-95 (IoU threshold ∈ [0.5:0.05:0.95]) as the main evaluation metrics. In addition, precision, recall, and F1 score are also used to comprehensively analyze model performance.

[0072] (2) Experimental results and analysis The proposed ATFL-Swin-YOLO was evaluated on the NEU-DET and GC10-DET datasets and compared with the baseline YOLOv8n and several state-of-the-art methods. As shown in Table 1, the method achieves a continuous improvement in detection accuracy while reducing model size. On the NEU-DET dataset, ATFL-Swin-YOLO improves mAP@50 from 76.7% to 77.6% and mAP@50-95 from 41.0% to 42.5%, achieving gains of 0.9 and 1.5 percentage points, respectively. Notably, these improvements only require a 16.0% reduction in the number of model parameters (from 3.157 million to 2.65 million), indicating that the introduced components—the Swing Transformer module, the adaptive threshold focus loss, and the P2 detection head—enhance representation capabilities without increasing model complexity.

[0073] Table 1: Comparison of experimental results on the NEU-DET dataset.

[0074]

[0075] Table 2: Comparison of class performance of YOLOv8n and ATFL-Swin-YOLO on the NEU-DET dataset.

[0076]

[0077] As shown in Table 2, the proposed model achieves performance improvements across most key defect categories. Specifically, mAP@50 for inclusions increases from 81.4% to 85.7% (+4.3pp), for pitted surfaces from 76.7% to 78.3% (+1.6pp), for involute defects from 67.9% to 69.1% (+1.2pp), and for scratches from 83.7% to 84.6% (+0.9pp). Even for patches, where the baseline performance is already high (93.9%), a small improvement to 94.1% (+0.2pp) was observed. These sustained improvements validate the effectiveness of three key components: (1) the Adaptive Threshold Focus Loss (ATFL), which adaptively reweights hard-to-classify samples; (2) the Swin Transformer backbone network, which enhances discriminative feature learning under strong background interference; and (3) the high-resolution detector head (P2), which preserves fine spatial details crucial for micro-defect localization.

[0078] However, the performance of the crack degraded, with the mean accuracy mAP@50 dropping from 57.3% to 53.9% (-3.4pp). Visual inspection revealed that these crack defects typically exhibited elongated, low-contrast, and spatially discontinuous linear structures. While the window-based self-attention mechanism in SwinTransformer effectively handles local regions, its limited cross-window interaction capability in shallow networks makes it difficult to model long-range spatial continuity between non-adjacent segments. Therefore, compared to the original convolutional backbone, this model is less sensitive to the overall crack morphology. This limitation highlights the key challenge in detecting anisotropic and sparse defect patterns, suggesting that future research could explore axial attention mechanisms, path-aware feature aggregation, or directional convolutional modules to better capture the structural consistency of elongated defects.

[0079] like Figure 7 As shown, the model proposed in this invention exhibits stronger robustness in detecting defects in complex backgrounds and at microscale. For pitted surfaces and involute defect samples—samples that often cause the baseline YOLOv8n model to miss true defects or make false alarms due to interference from oxide layers, rolling textures, or surface noise—the improved model consistently achieves more accurate localization. Notably, the model successfully reproduced several defect regions that YOLOv8n had previously missed or misjudged. In inclusion defect detection, the new method can identify multiple micrometer-scale defect instances (typically smaller than 20 pixels) that the baseline model failed to capture, fully validating its high sensitivity to fine-grained anomalies.

[0080] To evaluate the generalization ability of the proposed method under different steel surface conditions, experiments were further conducted on the GC10-DET dataset—a benchmark dataset that differs significantly from NEU-DET in image resolution, defect morphology, and background complexity. As shown in Table 3, the improved YOLOv8n model consistently outperforms the original model on key evaluation metrics, demonstrating strong cross-dataset adaptability. Notably, the model's parameter count was reduced by 16.0% (from 3.157M to 2.650M), and the proposed method achieved higher detection accuracy: mAP@50 increased from 76.2% to 77.6% (+1.4pp), and mAP@50-95 increased from 41.0% to 42.7% (+1.7pp). Although precision slightly decreased from 69.0% to 66.8%, the overall improvement in mAP—especially the increase in recall—remained significant. In industrial surface inspection, missed defects (low recall) typically pose higher safety and economic risks than false positives (low precision). Therefore, the observed trade-offs are consistent with actual deployment priorities, namely, maximizing defect coverage is the top priority.

[0081] Table 3: Comparison of experimental results on the GC10-DET dataset.

[0082]

[0083] like Figure 8 As shown, the qualitative analysis results on the GC10-DET dataset further validate the significantly improved robustness of the proposed model in handling complex backgrounds and small-scale defects. The improved algorithm successfully detects subtle defect instances that baseline methods easily overlook—especially those hidden in highly textured or cluttered areas—and accurately locates spatially discontinuous defects in strongly disturbed environments. These scenes are extremely challenging due to low contrast, local occlusion, or similarity to normal surface patterns. These results confirm that the proposed enhancements—Adaptive Thresholding Focus Loss (ATFL), SwinTransformer-based feature extraction, and a high-resolution small-object detection head—combined improve the model's robustness to neighborhood shifts, its ability to capture discriminative features under different imaging conditions, and its sensitivity to multi-scale defects. Therefore, the model maintains high detection performance on previously unseen and more challenging datasets, highlighting its potential for diverse deployments in the real world.

[0084] Comparison with other YOLO variants: To further evaluate the competitiveness of the proposed ATFL-Swin-YOLO, this invention compares its performance on the NEUDET dataset with three widely used nanoscale YOLO detectors—YOLOv5n, YOLOv8n, and YOLOv11n. All models were trained from scratch under the same experimental conditions (including input resolution, optimizer, learning rate scheduling, and data augmentation) to ensure a fair comparison.

[0085] As shown in Table 4, the YOLOv8n baseline model, thanks to its improved anchorless design and loss function, achieved 76.7% mAP@50 and 41.0% mAP@50-59, outperforming YOLOv5n's 75.9% and 41.7%. The newly released YOLOv11n made incremental improvements in detection head and label allocation strategies, achieving 76.8% mAP@50 and 42.0% mAP@50-95, representing only a slight improvement over YOLOv8n.

[0086] Table 4: Performance comparison of different models on the NEU-DET dataset.

[0087]

[0088] In comparison, ATFL-Swin-YOLO, based on the YOLOv8n architecture, performed best among all comparison methods, achieving 77.6% mAP@50 and 42.5% mAP@50-95. This means that compared to the strongest base model, mAP@50 was improved by 0.8 percentage points and mAP@50-95 by 0.5 percentage points.

[0089] In addition, the model contains 2.65 million parameters, which is 16.0% less than YOLOv8n (3.157 million) and similar to YOLOv5n (2.504 million) and YOLOv11n (2.0624 million).

[0090] Ablation studies: To systematically evaluate the contribution of each proposed component—namely, the Swing Transformer backbone, the Adaptive Threshold Focus Loss (ATFL), and the High-Resolution P2 Detector—a series of ablation experiments were conducted on the NEU-DET dataset. The results are summarized in Table 5, where Experiment 1 and Experiment 4 correspond to the original YOLOv8n and ATFL-Swin-YOLO, respectively.

[0091] Table 5: NEU-DET ablation study.

[0092]

[0093] First, this invention examines the impact of integrating the Swin Transformer and P2 detector head while preserving the original loss function (Experiment 2). Although the number of model parameters decreased from 3.157 million to 2.921 million, this configuration resulted in a drop in mAP@50 from 76.7% to 75.5%, and mAP@50-95 from 41.0% to 40.2%. This performance degradation indicates that while the Swin Transformer enhances global context modeling capabilities and the P2 detector head preserves fine spatial details, the lack of a rebalanced learning objective leads to an increased false positive rate—particularly noticeable in blurred or low-contrast defect areas.

[0094] Next, this invention replaces the original loss function with ATFL, while retaining the Swin Transformer but removing the P2 head (Experiment 3). This variant restores performance to baseline levels with fewer parameters (2.84 million) (mAP@50=76.7%, mAP@50-95=41.0%). This result confirms that ATFL effectively mitigates the negative impacts of class imbalance and hard sample dominance, enabling the model to maintain detection accuracy even under architectural changes that could disrupt training stability.

[0095] Finally, when all three components were integrated (Experiment 4), the model achieved optimal overall performance: mAP@50 = 77.6% and mAP@50-95 = 42.5%, with the fewest parameters (2.65 million) across all configurations. The consistent improvement in both mAP metrics—particularly the 1.5 percentage point improvement in mAP@50-95 compared to Experiment 2—indicates that the three modules work synergistically: the Swin Transformer provides rich, context-aware features; ATFL ensures stable and balanced optimization; and the P2 head enables precise localization of small-scale defects. Together, they address complementary challenges in industrial defect detection—feature representation, learning bias, and spatial resolution—resulting in a lightweight yet high-performance detector. This ablation study not only validates the independent utility of each component but also underscores the importance of their co-design in achieving robust and accurate defect detection on NEU-DET.

[0096] In this work, the present invention addresses three key challenges in industrial steel surface defect detection—complex background interference, high false negative rates for micro-defects, and performance degradation under extreme class imbalances—by enhancing the lightweight YOLOv8n framework. Specifically, this is achieved by task-aware integration of existing advanced components. Specifically, the invention incorporates a Swing Transformer module into the backbone to capture long-range contextual dependencies and suppress large-scale background artifacts; employs Adaptive Threshold Focus Loss (ATFL) to dynamically rebalance the learning priorities of different defect categories; and proposes a novel high-resolution P2 detection head to recover fine spatial details of micro-defects. Extensive experiments on the NEU-DET and GC10-DET benchmarks demonstrate that the proposed ATFL-Swin-YOLO achieves a better balance between accuracy and efficiency: model parameters are reduced by 16.0% (from 3.157 million to 2.65 million), and mAP@50-95 is improved by 1.5-1.7 percentage points.

[0097] Notably, the detection recall for key micro-defects such as inclusions and pitted surfaces is significantly improved, validating the effectiveness of the co-adaptive design. However, this method has limitations when detecting slender and discontinuous defects such as cracks, where window-based attention mechanisms may fail to maintain structural continuity. This suggests that future research could explore direction-sensitive attention mechanisms or multi-scale linear feature aggregators to better model anisotropic defect patterns.

[0098] Overall, ATFL-Swin-YOLO provides a practical and deployable solution for high-precision steel defect detection for resource-constrained edge devices, demonstrating that strategic integration and domain-specific improvements to existing technologies can achieve significant performance gains without increasing architectural complexity.

[0099] Experimental results fully validate the effectiveness of this strategy. On the NEU-DET dataset, compared to the baseline model using only standard Focal Loss, the use of the P2-specific ATFL improved mAP@50 by +0.9% and mAP@50–95 by +1.5%. Particularly in the categories of minor defects such as "inclusions" and "dented surfaces," the recall rate was significantly improved. This indicates that ATFL not only improves overall accuracy but also directly enhances the detection capability of key minor defects, perfectly meeting the core requirement of high recall in industrial quality inspection. As a loss calculation unit, the input of ATFL is directly related to the output of the P2 detection branch, intuitively demonstrating its "P2-specific" characteristic.

[0100] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting minute defects on the surface of lightweight steel, characterized in that, Includes the following steps: S1 uses the Swin Transformer backbone network to extract features from the input steel image and outputs a multi-level feature map. S2 constructs a multi-scale feature fusion network and a P2 detection branch specifically for detecting minute defects. The multi-scale feature fusion network fuses the shallow feature map of the Swin Transformer backbone network with the upsampled mid-level feature map to generate a fused high-resolution feature map. The fused high-resolution feature map is then provided to the P2 detection branch to generate defect category and localization prediction for candidate defect regions. S3 uses Adaptive Threshold Focus Loss (ATFL) to calculate the classification loss. The ATFL dynamically maintains an adaptive threshold for each defect category, reflecting the real-time difficulty level of the current minor defect detection task. The update calculation of the adaptive threshold is limited to the sample set that the P2 detection branch is responsible for predicting. S4 optimizes the multi-scale feature fusion network based on the classification loss calculated by the ATFL, and mainly backpropagates the gradient of the classification loss to the feature fusion module and its detection head parameters in the P2 detection branch, thereby enhancing the ability of the P2 detection branch to distinguish between real small defects and background artifacts.

2. The method according to claim 1, characterized in that, The Swin Transformer backbone network models the long-distance spatial dependence of the steel surface through its window self-attention mechanism.

3. The method according to claim 1, characterized in that, The P2 detection branch has a resolution of 160×160, and the shallow feature map is further downsampled to 80×80 resolution and fused with the middle feature map for a second time, thereby injecting fine-grained information of minute defects into higher-level semantic features, forming a bidirectional feature refinement loop.

4. The method according to claim 1, characterized in that, The multi-scale feature fusion network is a customized network based on the PAN-FPN structure, and its PAN path contains a feature enhancement module specifically designed for the P2 detection branch; The feature enhancement module receives shallow and mid-level feature maps from the backbone network. After upsampling the mid-level feature map by 2x, it performs element-wise addition and fusion with the shallow feature map, and outputs the fused high-resolution feature map through a feature refinement unit containing multiple convolutional layers and activation functions.

5. The method according to claim 1, characterized in that, The update formula for the adaptive threshold is: ， in, The adaptive threshold for the current training batch, where t is the training batch size. The adaptive threshold of the previous training batch. The momentum coefficient, This represents the number of positive samples in the current batch predicted by the P2 detection branch, with the true label being category c. Let be the prediction confidence level of the i-th positive sample.

6. The method according to claim 1, characterized in that, The gradient backpropagation of the classification loss refers to the fact that, during the backpropagation process, the classification loss gradient weight assigned to the P2 detection branch is higher than the classification loss gradient weight assigned to other scale detection branches P3, P4, and P5, so that the optimized signal is given priority to improve the ability of the P2 detection branch to distinguish small defects.

7. The method according to claim 1, characterized in that, The Swin Transformer backbone network is an improvement on the YOLOv8n model architecture. By introducing at least two Swin Transformer modules into this architecture, the model's ability to learn complex patterns is enhanced, while maintaining the lightweight nature of the overall network. The Swin Transformer modules adopt a self-attention mechanism with a window size of 3x3 to improve computational efficiency and model performance.

8. A device for detecting minute defects on the surface of lightweight steel, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method for detecting minute defects on the surface of lightweight steel as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the method for detecting minute defects on the surface of lightweight steel as described in any one of claims 1 to 7.

Images