A lightweight method for steel surface defect detection based on adversarial distillation

Abstract

The present application relates to steel surface defect detection, and in particular to a lightweight method for steel surface defect detection based on adversarial distillation, comprising the following steps executed in sequence: S1: constructing or obtaining a teacher model and a student model, inputting a steel surface image into the teacher model and the student model respectively for feature extraction, obtaining a teacher model output image and a student model output image; S2: calculating a distillation loss between the teacher model output image and the student model output image based on adversarial distillation; S3: optimizing the student model using the distillation loss calculated in step S2; S4: using the optimized student model to detect steel surface defects; the improved adversarial distillation technology is used to improve the performance of the lightweight model, meeting the demand of steel surface defect detection in industry.

Description

Technical Field

[0001] This invention relates to the detection of surface defects in steel, and more specifically to a lightweight method for detecting surface defects in steel based on antagonistic distillation. Background Technology

[0002] The diversity and irregularity of defect shapes and sizes pose significant challenges to accurate detection. Existing technologies employ multi-scale feature fusion techniques to improve the detection accuracy of defects at different scales. However, traditional fusion methods such as element-wise addition or concatenation often fail to allow the network to effectively focus on the defect region. Furthermore, the subtle characteristics of many defects are intertwined with complex background textures, further increasing the difficulty of accurate localization. To address these challenges, recent researchers have improved detection accuracy by introducing dedicated convolutional modules into the backbone network. Nevertheless, convolution and pooling operations during feature extraction still lead to the loss of fine-grained information and semantic details, which are crucial for the identification of small defects and limit localization performance.

[0003] The application of YOLO-based surface defect detection technology in industries such as metals, semiconductors, and ceramics is continuously expanding. Its ongoing improvements provide strong support for efficient and accurate surface defect detection, and YOLOv11 has now emerged as the latest technology in the YOLO series. RT-DETR is an optimized target detection framework designed to improve the real-time performance and accuracy of Transformer-based detection methods. This model significantly reduces computational overhead through an improved multi-head attention mechanism and an efficient multi-scale feature fusion network, while enhancing the detection capabilities for small and multi-scale targets. Both have advantages in speed and accuracy, but still face challenges in generalization and lightweight design.

[0004] In steel surface defect detection, real-time performance and accuracy are two critical requirements that must be balanced. However, while existing lightweight models have improved computational efficiency, they often compromise on accuracy and detection capabilities. Summary of the Invention

[0005] The purpose of this invention is to provide a lightweight method for detecting surface defects in steel based on adversarial distillation, which improves detection accuracy.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A lightweight method for detecting surface defects in steel based on antagonistic distillation includes the following steps performed sequentially:

[0008] S1: Construct or obtain teacher and student models, input the steel surface image into the teacher model and student model respectively for feature extraction, and obtain the output image of the teacher model and the output image of the student model;

[0009] S2: Calculate the distillation loss between the teacher model's output image and the student model's output image based on adversarial distillation. The specific calculation steps are as follows:

[0010] S2-1: Assume two random variables are input. and ,calculate and Spearman correlation coefficient between :

[0011] ;

[0012] in, , express The Middle Rank of each data point express The Middle Rank of each data point For the number of categories, Indicates the difference in variable ranking;

[0013] S2-2: Calculated using the following formula Soft ranking:

[0014] ;

[0015] ;

[0016] in, Represents element soft ranking, This represents the Sigmoid function. This is the scaling factor. This represents the loss of the saliency plot. Spearma correlation coefficient and These represent the outputs of the student model and the teacher model, respectively.

[0017] S2-3: Use CAM to calculate the attribution regions of the output images of the teacher model and the student model respectively:

[0018] ;

[0019] ;

[0020] in, Output the attribution region of the image for the teacher model. Output the attribution region of the image for the student model. and The first and second models are respectively the teacher model and the student model. The weights of each feature map, and Let represent the activation values ​​of the k-th feature maps in the teacher model and the student model, respectively. Indicates the input image;

[0021] Attribution region alignment is achieved by minimizing the mean squared error between the two models:

[0022] ;

[0023] in, and These are the width and height of the attribution region, respectively. Indicates bilinear interpolation. and These represent the locations of the attribution regions. ;

[0024] S2-4: Calculate the overall output loss of antagonistic distillation using the following formula. :

[0025] ;

[0026] ;

[0027] in, This represents the loss of the saliency plot. Indicates CAM alignment loss. To balance the weighting parameters of the two loss terms, For constant terms, and Representing the original sample and the adversarial sample, For the joint loss function; This represents the teacher model's confidence in the adversarial example on y. This represents the confidence modulation index. Indicates the weighting coefficient. This represents the difference in output between the student model and the adversarial examples. This represents the cross-entropy loss of the student model on the original samples.

[0028] S3: Optimize the student model using the distillation loss calculated in step S2;

[0029] S4: Use the optimized student model to detect defects on the surface of steel.

[0030] Preferably, a teacher model and a student model are constructed for detecting surface defects in steel. Both the teacher model and the student model include a backbone network, an attention calculation module, a feature fusion network, a stitching module, and a detection head.

[0031] The image of steel surface defects is input into the backbone network for multi-scale feature extraction. The output feature map is filtered by saliency supervision and query refinement. Four feature maps S2, S3, S4 and S5 of different scales are selected. S5 is input into the attention calculation module for feature extraction and output feature map F5.

[0032] The feature fusion network includes a first convolutional module, a first fusion network and a second fusion network connected in sequence. The first fusion network includes three MCEA modules and three Dual-GG modules, which are arranged alternately in sequence.

[0033] The second converged network includes four Dual-GG modules and three MCEA modules, with the MCEA modules and Dual-GG modules arranged alternately in sequence.

[0034] The feature map F5 is input into the feature fusion network. The first convolutional module performs preliminary feature extraction on the feature map F5. The first MCEA module and the first Dual-GG module of the first fusion network perform feature fusion on the extracted preliminary features in sequence to obtain the first fused features.

[0035] The first fused feature is processed by the second MCEA module of the first fusion network and then input into the second Dual-GG module for feature fusion with feature map S4 to obtain the second fused feature.

[0036] The second fusion feature is processed by the third MCEA module of the first fusion network and then input into the third Dual-GG module for feature fusion with feature map S3 and the second fusion feature to obtain the third fusion feature.

[0037] The third fusion feature and feature map S2 are input into the first Dual-GG module of the second fusion network for feature fusion to obtain the fourth fusion feature;

[0038] The fourth fusion feature is processed by the first MCEA module of the second fusion network and then fused with the second and fourth fusion features in the second Dual-GG module to obtain the fifth fusion feature.

[0039] The fifth fusion feature is processed by the second MCEA module of the second fusion network, and then fused with the first and fifth fusion features in the third Dual-GG module to obtain the sixth fusion feature.

[0040] The sixth fusion feature is processed by the third MCEA module of the second fusion network and then fused with the sixth fusion feature in the fourth Dual-GG module to obtain the seventh fusion feature;

[0041] The stitching module is used to stitch together the output feature maps of each Dual-GG module of the second fusion network to obtain a stitched feature map, and input the stitched feature map into the detection head to output a predicted image.

[0042] Preferably, the specific operation of saliency supervision is: assuming the position in the first feature map is... Each query at the location The coordinates corresponding to the original image are Calculate the true bounding box arrive distance Calculate the confidence score of the query. The feature map output by the saliency supervision is refined by query, which includes background embedding and redundancy removal.

[0043] Preferably, the backbone network of the teacher model is a ResNet-50 network, and the backbone network of the student model is a PPLCNet network.

[0044] Preferably, the Dual-GG module includes two branches, one of which uses a second convolutional module to extract features from the input image;

[0045] Another branch uses a third convolutional module to extract features from the input image, and then inputs the extracted features into GGBlock for processing;

[0046] The GGBlock consists of three branches. The first branch is processed by the GhostConv module twice, and the second branch is processed by the GSConv module twice. The feature maps processed by the two branches are each subjected to a Softmax operation, and then the resulting feature maps are residually concatenated. At the same time, the feature maps of the two branches that have not undergone the Softmax operation are also residually concatenated. The two residually concatenated feature maps are multiplied by a dot product, and finally concatenated with the input feature map of the third branch to obtain the output.

[0047] The feature maps output from the two branches are concatenated and then output.

[0048] Preferably, the MCEA module includes three branches:

[0049] The feature map of the first branch is pooled in the X direction;

[0050] The feature map of the second branch is subjected to pooling in the Y direction;

[0051] The feature maps output from the first and second branches are concatenated and subjected to a 1×1 convolution operation to output the first pooling feature map. The weights of this first pooling feature map in the X and Y directions are dynamically generated using the Sigmoid function. This first pooling feature map is then subjected to Re-weighting, 1×1 convolution, GroupNorm, and 1×1 convolution operations in sequence, and then superimposed with the feature map from the third branch after 3×3 convolution and Softmax operations. Finally, it is superimposed with the feature map from the fourth branch and subjected to a Re-weighting operation to obtain the output.

[0052] By adopting the aforementioned design scheme, the beneficial effects of this invention are as follows: This application can effectively transfer the knowledge of the teacher model to the student model by maximizing the Spearman correlation coefficient. During the training phase, attribution region alignment is achieved by minimizing the mean square error (MSE) between the two models, thereby promoting the student model to imitate the behavior of the teacher model. The improved adversarial distillation technique is used to enhance the performance of the lightweight model, meeting the industrial demand for steel surface defect detection. Attached Figure Description

[0053] Figure 1 This is a diagram of the overall network structure of the present invention;

[0054] Figure 2 This is a schematic diagram of the structure of the teacher model and student model of the present invention;

[0055] Figure 3 This is a schematic diagram of the structure of the Dual-GG module of the present invention;

[0056] Figure 4 This is a schematic diagram of the structure of the MCEA of the present invention;

[0057] Figure 5 A comparative chart showing the number of different model parameters, accuracy, and FPS of the present invention;

[0058] Figure 6 This is a visual comparison of the detection method of the present invention with other models. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0060] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0061] A lightweight method for detecting surface defects in steel based on adversarial distillation, such as Figure 1 As shown, the steps are executed sequentially as follows:

[0062] S1: Construct or obtain teacher and student models, input the steel surface image into the teacher model and student model respectively for feature extraction, and obtain the output image of the teacher model and the output image of the student model;

[0063] S2: Calculate the distillation loss between the teacher model's output image and the student model's output image based on adversarial distillation. The specific calculation steps are as follows:

[0064] To maintain consistency in the predictive relationship between the teacher and student models, particularly in preserving relative predictive rankings during knowledge transfer, this application discards the traditional temperature coefficient T and replaces the KL divergence with the Spearman correlation coefficient. This choice allows for a more flexible matching relationship between the teacher and student models, facilitating the assessment of the correlation between their predictions. The Spearman correlation coefficient measures the correlation between two variables; when one variable changes, the value of the other tends to change in the same direction, rather than emphasizing the consistency of their absolute values. This characteristic mitigates the problem of over-approximation of the teacher model during training, reduces training interference, and facilitates the student model's learning from the teacher model. By maximizing the Spearman correlation coefficient, we are able to effectively transfer knowledge from the teacher model to the student model.

[0065] S2-1: Assume two random variables are input. and ,calculate and Spearman correlation coefficient between :

[0066] ;

[0067] in, , express The Middle Rank of each data point express The Middle Rank of each data point For the number of categories, This indicates the difference in the ranking of variables, that is, the difference in the ranking of two variables at each data point.

[0068] S2-2: Calculated using the following formula Soft ranking:

[0069] ;

[0070] ;

[0071] in, Represents element soft ranking, This represents the Sigmoid function, which compresses the input to the interval (0,1). This is the scaling factor. This represents the loss of the saliency plot. Spearma correlation coefficient and These represent the outputs of the student model and the teacher model, respectively.

[0072] because Using hard ranking, which sorts data values ​​and assigns them exact rankings, may affect the model's convergence process. Therefore, a continuous function is used to approximate the ranking function, making the entire calculation process differentiable. Here, differentiability means that the soft ranking function can be differentiated, supporting backpropagation, thus enabling the neural network to be trained through gradient descent.

[0073] S2-3: To enable the student model to understand the behavior of the teacher model, CAM is used to calculate the attribution regions of the output images of the teacher model and the student model, respectively.

[0074] ;

[0075] ;

[0076] in, Output the attribution region of the image for the teacher model. Output the attribution region of the image for the student model. and The first and second models are respectively the teacher model and the student model. The weights of each feature map, which come from the classification layer, represent the importance of the model to each feature map. and Let represent the activation values ​​of the k-th feature maps in the teacher model and the student model, respectively. Indicates the input image; This is an interpretable technique for image classification tasks. It visualizes the regions of interest of a neural network in an image to understand which regions the network focuses on when making predictions. For each input image, Generate a heatmap to represent the importance of different regions.

[0077] Attribution region alignment is achieved by minimizing the mean squared error (MSE) between the two models, thereby encouraging the student model to mimic the behavior of the teacher model.

[0078] ;

[0079] in, and These are the width and height of the attribution region, respectively. Indicates bilinear interpolation. and These represent the locations of the attribution regions. Since the structures of the student and teacher models may differ, the width and height of the attribution region may also differ. Therefore, to address this difference, this embodiment uses bilinear interpolation as the transformation method. Messenger and Its width and height are equal.

[0080] S2-4: Calculate the overall output loss of antagonistic distillation using the following formula. :

[0081] ;

[0082] ;

[0083] in, This represents the loss of the saliency plot. Indicates CAM alignment loss. To balance the weighting parameters of the two loss terms, For constant terms, and Representing the original sample and the adversarial sample, For the joint loss function; This represents the teacher model's confidence in the adversarial example on y. This represents the confidence modulation index. Indicates the weighting coefficient. This represents the difference in output between the student model and the adversarial examples. This represents the cross-entropy loss of the student model on the original samples.

[0084] S3: Optimize the student model using the distillation loss calculated in step S2;

[0085] S4: Use the optimized student model to detect defects on the surface of steel.

[0086] In this embodiment, the following is constructed: Figure 2 The teacher model and student model shown are used for detecting surface defects in steel. Both the teacher model and the student model include a backbone network, an attention calculation module, a feature fusion network, a stitching module, and a detection head.

[0087] The image of steel surface defects is input into the backbone network for multi-scale feature extraction. The output feature map is filtered by saliency supervision and query refinement. Four feature maps S2, S3, S4 and S5 of different scales are selected. S5 is input into the attention calculation module for feature extraction and output feature map F5.

[0088] In this embodiment, the specific operation of saliency supervision is as follows: assuming that the first feature map represents the importance position. Each query at the location The coordinates corresponding to the original image are Calculate the true bounding box arrive distance Calculate the confidence score of the query. According to confidence level A saliency feature map is output, and the feature map output by saliency supervision is used for query refinement, which includes background embedding and redundancy removal. In this embodiment, the saliency supervision and query refinement are conventional operations in the art, but other supervision and refinement methods can also be used.

[0089] The feature fusion network includes a first convolutional module, a first fusion network and a second fusion network connected in sequence. The first fusion network includes three MCEA modules and three Dual-GG modules, which are arranged alternately in sequence.

[0090] The second converged network includes four Dual-GG modules and three MCEA modules, with the MCEA modules and Dual-GG modules arranged alternately in sequence.

[0091] The feature map F5 is input into the feature fusion network. The first convolutional module performs preliminary feature extraction on the feature map F5. The first MCEA module and the first Dual-GG module of the first fusion network perform feature fusion on the extracted preliminary features in sequence to obtain the first fused features.

[0092] The first fused feature is processed by the second MCEA module of the first fusion network and then input into the second Dual-GG module for feature fusion with feature map S4 to obtain the second fused feature.

[0093] The second fusion feature is processed by the third MCEA module of the first fusion network and then input into the third Dual-GG module for feature fusion with feature map S3 and the second fusion feature to obtain the third fusion feature.

[0094] The third fusion feature and feature map S2 are input into the first Dual-GG module of the second fusion network for feature fusion to obtain the fourth fusion feature;

[0095] The fourth fusion feature is processed by the first MCEA module of the second fusion network and then fused with the second and fourth fusion features in the second Dual-GG module to obtain the fifth fusion feature.

[0096] The fifth fusion feature is processed by the second MCEA module of the second fusion network, and then fused with the first and fifth fusion features in the third Dual-GG module to obtain the sixth fusion feature.

[0097] The sixth fusion feature is processed by the third MCEA module of the second fusion network and then fused with the sixth fusion feature in the fourth Dual-GG module to obtain the seventh fusion feature;

[0098] The stitching module is used to stitch together the output feature maps of each Dual-GG module of the second fusion network to obtain a stitched feature map, and input the stitched feature map into the detection head to output a predicted image.

[0099] This feature fusion network employs a multi-level, multi-path design to enhance the representation of steel surface defect features and improve detection accuracy. The network uses a hierarchical processing strategy, dividing features into different levels. Each layer of features, after optimization by the Dual-GG module, extracts richer semantic information. Simultaneously, the network fully integrates high-level semantic information with low-level detailed features through cross-layer connections. High-level features provide global context support, aiding in target localization and category discrimination, while low-level features retain detailed information, making them particularly suitable for detecting small targets and complex backgrounds. Furthermore, a multi-path fusion strategy runs throughout the entire network, allowing features to interact and complement each other at different levels, forming a complete and highly discriminative feature representation. This multi-level, multi-path feature fusion design not only improves detection accuracy but also demonstrates excellent operational efficiency.

[0100] In this embodiment, the backbone network of the teacher model is a ResNet-50 network, and the backbone network of the student model is a PPLCNet network.

[0101] In this embodiment, as Figure 3 As shown, the Dual-GG module includes two branches, one of which uses a second convolutional module to extract features from the input image;

[0102] Another branch uses a third convolutional module to extract features from the input image, and then inputs the extracted features into GGBlock for processing;

[0103] The GGBlock consists of three branches. The first branch is processed by the GhostConv module twice, and the second branch is processed by the GSConv module twice. The feature maps processed by the two branches are each subjected to a Softmax operation, and then the resulting feature maps are residually concatenated. At the same time, the feature maps of the two branches that have not undergone the Softmax operation are also residually concatenated. The two residually concatenated feature maps are multiplied by a dot product, and finally concatenated with the input feature map of the third branch to obtain the output.

[0104] The feature maps output from the two branches are concatenated and then output.

[0105] This Dual-GG module combines two lightweight convolutional operations, GhostConv and GSConv, to design an efficient feature extraction and dynamic weighting mechanism, aiming to improve the model's performance in multi-scale feature fusion. The module introduces a Softmax operation to dynamically adjust the weights of features in each branch, enabling adaptive fusion of different features based on contextual information, thereby further optimizing feature representation. The introduction of the Softmax operation allows for adaptive adjustment of weights during feature fusion based on the current context, overcoming the limitations of static weight design. The dynamic weighting mechanism makes the model more flexible in feature fusion, focusing on important features and suppressing redundant information. The parallel design of the two convolutional operations allows the module to capture diverse feature representations. Residual connections and dot product operations enhance feature transfer and interaction capabilities, particularly for small object detection and complex background processing. The combination of GhostConv and GSConv significantly reduces computational cost and increases computational power while improving feature representation, making it suitable for lightweight applications. Residual connections and dot product operations further enhance the performance of small object defect localization and detection in complex industrial backgrounds.

[0106] In this embodiment, as Figure 4 As shown, the MCEA module includes three branches:

[0107] The feature map of the first branch is pooled in the X direction; the feature map of the second branch is pooled in the Y direction; max pooling and average pooling are performed on the feature maps in the X and Y directions respectively to capture global contextual information in the feature maps from different angles and achieve directional feature aggregation.

[0108] The feature maps output from the first and second branches are concatenated and subjected to a 1×1 convolution operation to output the first pooling feature map. The weights of this first pooling feature map in the X and Y directions are dynamically generated using the Sigmoid function, allowing the model to adaptively adjust the importance distribution of the feature maps based on the input context. This first pooling feature map is then processed sequentially through re-weighting, 1×1 convolution, GroupNorm, and another 1×1 convolution operation. This output is then superimposed on the feature map from the third branch, which has undergone a 3×3 convolution and Softmax operation. Finally, it is superimposed on the feature map from the fourth branch and subjected to another re-weighting operation to obtain the final output. Using GroupNorm instead of the conventional BatchNorm operation eliminates the dependency on batch size, making the module more robust in small-batch training or high-resolution feature map processing.

[0109] Assume the feature map of the input MCEA module is The corresponding processing procedure is represented by the following formula:

[0110] ;

[0111] ;

[0112] ;

[0113] ;

[0114] ;

[0115] ;

[0116] ;

[0117] in, This represents the max pooling operation in the X direction. This represents the average pooling operation in the X direction. This indicates a splicing operation. This represents the result of the max pooling operation in the X direction. This represents the result of the average pooling operation in the X direction. This represents the result of the max pooling operation in the Y direction. This represents the result of the average pooling operation in the Y direction. Represents a 1×1 convolution. This represents a 3×3 convolution. express operate, This represents the Sigmoid function, which compresses the input to the interval (0,1). and Indicates the weights in the X and Y directions. This indicates the multi-directional features after splicing. This represents the first pooling feature map after 1×1 convolution to compress the channels. This represents the attention weights in the X direction. This represents the features after coordinate attention reweighting. Represents the spatial attention weight map. This indicates the final output characteristics of the MCEA module.

[0118] The MCEA module's multi-path feature enhancement Softmax path adds an independent global attention path to feature fusion, effectively balancing the interrelationship between local and global features. Utilizing multi-directional pooling and dynamic weighting mechanisms significantly enhances feature discriminative power, making it suitable for steel surface defect detection in complex scenarios. The module drastically reduces computational complexity through 1×1 convolution and grouping operations, improving efficiency while maintaining high performance. It balances fine-grained and global feature representation.

[0119] To effectively verify the method proposed in this application, this embodiment uses the NEU-DET dataset, GC10-DET dataset, and DAGM dataset collected and generated by Northeastern University for experiments. The above datasets are divided into three parts: training set, validation set, and test set in a ratio of 7:2:1.

[0120] The NEU-DET dataset contains six common defect types in the hot-rolled steel strip production process: rolled-in scales (Rs), patches (Pa), cracks (Cr), inclusions (In), surface depressions (Ps), and scratches (Sc). The dataset contains a total of 1800 images, each 200×200 pixels, with 300 samples for each defect type.

[0121] The GC10-DET dataset was acquired using line-scan digital cameras in actual industrial settings and includes ten types of steel surface defects: stamping (Pu), weld (Wl), crescent gap (Cg), watermarks (Ws), oilmarks (Os), filamentous spots (Ss), inclusions (In), rolling pits (Rp), creases (Cr), and waist creases (Wf). The dataset contains a total of 2294 images, with a different number of samples for each defect type.

[0122] The DAGM2007 dataset is an artificially generated dataset consisting of 10 classes of statistical texture defects (labeled as class 10). We selected 2100 defect images of size 512×512 for our experiment.

[0123] Table 1 Ablation experiments of the teacher model:

[0124]

[0125] Table 2 Ablation experiments of the student model:

[0126]

[0127] Table 3. Experiments on fusion methods:

[0128]

[0129] Table 4 Comparison experiments of different models on the NEU-DET dataset:

[0130]

[0131] Table 5 Comparison of classification accuracy of different models

[0132]

[0133] As shown in Table 1, in the teacher model backbone network using ResNet-50, the complete module combination achieves an mAP of 83.1%. Dual-GG and MCEA play key roles in feature fusion and multi-scale feature enhancement, respectively. The saliency supervision and query refinement modules further improve the efficiency of saliency information utilization without increasing the overall parameter count. As shown in Table 2, in the student model backbone network using PPLCNet, the complete module achieves an mAP of 81.6%, significantly reducing the parameter count to 12.9M, while still significantly improving model performance. The experimental results of the teacher and student models validate the effectiveness of the feature fusion and saliency refinement methods.

[0134] As shown in Table 3, the mAP of the concatenation fusion method is 82.4% and 80.7%, while the mAP of the weighted addition fusion method is improved to 83.1% and 81.6%. The weighted addition fusion method, by assigning weights to different features, can more effectively fuse features and improve the effectiveness of feature representation, thus slightly outperforming the concatenation fusion method in detection accuracy. In contrast, the concatenation fusion method may increase feature redundancy in some cases, thereby affecting accuracy. Experimental results show that the weighted addition fusion method performs better in terms of adaptability and accuracy in feature extraction and fusion.

[0135] Tables 4, 5 and Figure 5 This represents a comparison of the accuracy, number of parameters, and FPS of different models on the NEU-DET dataset.

[0136] Compared with conventional object detection models, MINI-DETR performs exceptionally well in steel surface defect detection, demonstrating significant performance improvements and lightweight advantages compared to traditional object detection models such as YOLOv5-s, DetectoRS, and PCGAN. As shown in Tables 4 and 5, MINI-DETR achieves an mAP of 82.3%, representing improvements of 8.2% and 6.7% respectively compared to YOLOv5-s' 74.1% and DetectoRS' 75.4%. Furthermore, MINI-DETR has only 12.9M parameters, significantly lower than ESNet's 148.2M and PCGAN's 80.4M, while achieving an inference speed of 83 FPS, demonstrating superior real-time performance and resource utilization efficiency. In terms of classification accuracy, MINI-DETR excels particularly in detecting Cr (58.5%), Pa (94.3%), and Rs (74.2%) defects, especially performing exceptionally well in detecting small targets (such as Cr) and complex background scenarios. This demonstrates that the MINI-DETR can achieve efficient inspection in resource-constrained industrial inspection scenarios, while meeting the requirements of high precision and high speed.

[0137] Comparison with DETR-like models: Compared to other DETR-like models (such as Focus-DETR and DINO), MINI-DETR achieves the best balance between performance and efficiency through further optimization of feature extraction and model architecture. MINI-DETR's mAP is 82.3%, slightly higher than DINO (81.2%) and Focus-DETR (80.6%), but its parameter count is only 12.9M, significantly lower than DINO (47.7M) and Focus-DETR (44.4M). Its inference speed reaches 87 FPS, far superior to Focus-DETR's 56 FPS and DINO's 40 FPS. This lightweight advantage makes MINI-DETR more practically valuable in environments with limited hardware resources. Furthermore, MINI-DETR maintains a leading position in classification accuracy, achieving 94.3% for Pa defects, and 74.2% and 91.7% for Rs and Sc defects, respectively, demonstrating strong multi-scale feature modeling capabilities and effective integration of contextual information.

[0138] Figure 6 This section presents a visual comparison of different improved models on the NEU-DET dataset. Each column shows the output of different detection methods, including the original label, TriS-YOLO, READ-DETR, and MINI-DETR.

[0139] The TriS-YOLO model demonstrated some capability in detecting surface defects in steel, but it still had some errors, with some small defects not being accurately located. Compared to TriS-YOLO, READ-DETR performed more accurately in handling more complex defects, with more precise localization of some detection boxes. The MINI-DETR model performed better than the other two models, handling most defects well and producing more accurate bounding boxes, especially excelling in detecting small and complex defects.

[0140] In summary, this application improves the adversarial distillation method to effectively reduce the number of model parameters while maintaining detection performance. A saliency supervision mechanism is introduced after the backbone network to reduce the computational burden of the DETR model, enhance the ability to capture details, and suppress background interference.

[0141] For defect detection of different shapes and sizes, a multi-scale spatial fusion module is proposed to achieve efficient feature fusion and enhance the output feature representation capability. At the same time, a context fusion feature pyramid for small target detection is designed.

[0142] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lightweight method for detecting surface defects in steel based on antagonistic distillation, characterized in that: The steps are as follows, performed sequentially: S1: Construct or obtain teacher and student models, input the steel surface image into the teacher model and student model respectively for feature extraction, and obtain the output image of the teacher model and the output image of the student model; S2: Calculate the distillation loss between the teacher model's output image and the student model's output image based on adversarial distillation. The specific calculation steps are as follows: S2-1: Assume two random variables are input. and ,calculate and Spearman correlation coefficient between : ； in, , express The Middle Rank of each data point express The Middle Rank of each data point For the number of categories, Indicates the difference in variable ranking; S2-2: Calculated using the following formula Soft ranking: ； ； in, Represents element soft ranking, This represents the Sigmoid function. This is the scaling factor. This represents the loss of the saliency plot. Spearma correlation coefficient and These represent the outputs of the student model and the teacher model, respectively. S2-3: Use CAM to calculate the attribution regions of the output images of the teacher model and the student model respectively: ； ； in, Output the attribution region of the image for the teacher model. Output the attribution region of the image for the student model. and The first and second models are respectively the teacher model and the student model. The weights of each feature map, and Let represent the activation values ​​of the k-th feature maps in the teacher model and the student model, respectively. Indicates the input image; Attribution region alignment is achieved by minimizing the mean squared error between the two models: ； in, and These are the width and height of the attribution region, respectively. Indicates bilinear interpolation. and These represent the locations of the attribution regions. ; S2-4: Calculate the overall output loss of antagonistic distillation using the following formula. : ； ； in, This represents the loss of the saliency plot. Indicates CAM alignment loss. To balance the weighting parameters of the two loss terms, For constant terms, and Representing the original sample and the adversarial sample, For the joint loss function; This represents the teacher model's confidence in the adversarial example on y. This represents the confidence modulation index. Indicates the weighting coefficient. This represents the difference in output between the student model and the adversarial examples. This represents the cross-entropy loss of the student model on the original samples; S3: Optimize the student model using the distillation loss calculated in step S2; S4: Use the optimized student model to detect defects on the surface of steel.

2. The lightweight method for detecting steel surface defects based on antagonistic distillation as described in claim 1, characterized in that: A teacher model and a student model are constructed for detecting surface defects in steel. Both the teacher model and the student model include a backbone network, an attention calculation module, a feature fusion network, a stitching module, and a detection head. The image of steel surface defects is input into the backbone network for multi-scale feature extraction. The output feature map is filtered by saliency supervision and query refinement. Four feature maps S2, S3, S4 and S5 of different scales are selected. S5 is input into the attention calculation module for feature extraction and output feature map F5. The feature fusion network includes a first convolutional module, a first fusion network and a second fusion network connected in sequence. The first fusion network includes three MCEA modules and three Dual-GG modules, which are arranged alternately in sequence. The second converged network includes four Dual-GG modules and three MCEA modules, with the MCEA modules and Dual-GG modules arranged alternately in sequence. The feature map F5 is input into the feature fusion network. The first convolutional module performs preliminary feature extraction on the feature map F5. The first MCEA module and the first Dual-GG module of the first fusion network perform feature fusion on the extracted preliminary features in sequence to obtain the first fused features. The first fused feature is processed by the second MCEA module of the first fusion network and then input into the second Dual-GG module for feature fusion with feature map S4 to obtain the second fused feature. The second fusion feature is processed by the third MCEA module of the first fusion network and then input into the third Dual-GG module for feature fusion with feature map S3 and the second fusion feature to obtain the third fusion feature. The third fusion feature and feature map S2 are input into the first Dual-GG module of the second fusion network for feature fusion to obtain the fourth fusion feature; The fourth fusion feature is processed by the first MCEA module of the second fusion network and then fused with the second and fourth fusion features in the second Dual-GG module to obtain the fifth fusion feature. The fifth fusion feature is processed by the second MCEA module of the second fusion network, and then fused with the first and fifth fusion features in the third Dual-GG module to obtain the sixth fusion feature. The sixth fusion feature is processed by the third MCEA module of the second fusion network and then fused with the sixth fusion feature in the fourth Dual-GG module to obtain the seventh fusion feature; The stitching module is used to stitch together the output feature maps of each Dual-GG module of the second fusion network to obtain a stitched feature map, and input the stitched feature map into the detection head to output a predicted image.

3. The lightweight method for detecting steel surface defects based on antagonistic distillation as described in claim 2, characterized in that: The specific operation of saliency supervision is as follows: assuming the position in the first feature map... Each query at the location The coordinates corresponding to the original image are Calculate the true bounding box arrive distance Calculate the confidence score of the query. The feature map output by the saliency supervision is refined by query, which includes background embedding and redundancy removal.

4. The lightweight method for detecting steel surface defects based on antagonistic distillation as described in claim 3, characterized in that: The backbone network of the teacher model is a ResNet-50 network, and the backbone network of the student model is a PPLCNet network.

5. The lightweight method for detecting steel surface defects based on antagonistic distillation as described in claim 4, characterized in that: The Dual-GG module includes two branches, one of which uses a second convolutional module to extract features from the input image; Another branch uses a third convolutional module to extract features from the input image, and then inputs the extracted features into GGBlock for processing; The GGBlock consists of three branches. The first branch is processed by the GhostConv module twice, and the second branch is processed by the GSConv module twice. The feature maps processed by the two branches are each subjected to a Softmax operation, and then the resulting feature maps are residually concatenated. At the same time, the feature maps of the two branches that have not undergone the Softmax operation are also residually concatenated. The two residually concatenated feature maps are multiplied by a dot product, and finally concatenated with the input feature map of the third branch to obtain the output. The feature maps output from the two branches are concatenated and then output.

6. The lightweight method for detecting steel surface defects based on antagonistic distillation as described in claim 5, characterized in that: The MCEA module includes three branches: The feature map of the first branch is pooled in the X direction; The feature map of the second branch is subjected to pooling in the Y direction; The feature maps output from the first and second branches are concatenated and subjected to a 1×1 convolution operation to output the first pooling feature map. The weights of this first pooling feature map in the X and Y directions are dynamically generated using the Sigmoid function. This first pooling feature map is then subjected to Re-weighting, 1×1 convolution, GroupNorm, and 1×1 convolution operations in sequence, and then superimposed with the feature map from the third branch after 3×3 convolution and Softmax operations. Finally, it is superimposed with the feature map from the fourth branch and subjected to a Re-weighting operation to obtain the output.

Images