A method and system for detecting surface defects in steel based on improved RT-DETR

By introducing the SCS-Downsample module and DTA-Attention module into the RT-DETR model, the problems of insufficient multi-scale feature extraction and insufficient feature fusion adaptability in steel surface defect detection are solved, thereby improving detection accuracy and inference efficiency, and making it suitable for real-time detection of industrial edge devices.

CN123118595APending Publication Date: 2026-11-17HUAIYIN INSTITUTE OF TECHNOLOGY
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611466272.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-20
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing RT-DETR models have problems in steel surface defect detection, such as insufficient multi-scale feature extraction capability, easy omission of slender and small defects, lack of adaptability in feature fusion, and large amount of attention computation, making it difficult to achieve real-time detection on industrial edge devices.

Method used

By introducing the SCS-Downsample module and the DTA-Attention module, we can improve the multi-scale feature extraction capability and the adaptability of feature fusion by dynamically selecting effective features and reducing computational overhead, thereby enhancing the ability to identify minute defects and slender cracks.

Benefits of technology

It achieves simultaneous improvement in detection accuracy and inference efficiency, adaptability to defects of different scales and adaptiveness of feature fusion, reduces attention computation overhead, and is suitable for real-time detection on edge devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118595A_ABST
    Figure CN123118595A_ABST
Patent Text Reader

Abstract

This invention discloses a method and system for detecting surface defects in steel based on an improved RT-DETR, within the field of defect detection technology. The method includes: acquiring images of the steel surface, labeling defects in the images, and performing preprocessing, data augmentation, and dataset partitioning; constructing an improved RT-DETR model, introducing a DTA-Attention module in the encoder for dynamic threshold binarization and global context modeling, and introducing an SCS-Downsample module in the bottom-up downsampling path of the feature fusion network for multi-scale spatial-channel joint selection; training the model using a training set and adjusting parameters and weights using a validation set; inputting the image of the steel surface to be detected into the model, and outputting the category and location of the defects. This method solves the problems of insufficient multi-scale feature extraction capability, missed detection of slender and small defects, lack of adaptability in feature fusion, and high computational cost of attention, achieving a simultaneous improvement in detection accuracy and inference efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a method and system for detecting defects on the surface of steel based on an improved RT-DETR. Background Technology

[0002] During the production, processing and use of steel, defects such as scratches, cracks, pitting, rust, peeling, stains and dents may appear on its surface. The results of defect detection directly affect product quality assessment, production process adjustment and equipment operation safety.

[0003] Currently, steel surface defect detection mainly employs manual inspection, traditional image processing methods, and deep learning-based target detection methods. Manual inspection relies on the experience of inspectors, resulting in low efficiency, high subjectivity, fatigue from prolonged work, and difficulty in continuous online detection. Traditional image processing methods typically depend on manually designed color, texture, edge, or shape features, leading to poor detection stability when the contrast between the defect and background is low, or when affected by lighting changes, reflections, noise, or complex textures. In recent years, deep learning-based target detection methods have been increasingly applied to steel surface defect detection. Among them, RT-DETR, as an end-to-end target detection model, utilizes a Transformer structure to establish global dependencies between features and omits traditional non-maximum suppression post-processing, offering advantages such as a simple detection process and high inference efficiency.

[0004] However, existing RT-DETR models still have the following problems when applied to steel surface defect detection: First, steel surface defects usually have significant scale differences. Targets such as tiny point defects, slender cracks, and large-area spalling vary greatly in spatial morphology, making it difficult for fixed convolution kernels or fixed-scale feature extraction methods to simultaneously take into account the feature representation of different types of defects. Second, existing feature fusion structures usually use fixed addition, splicing, or convolution operations, which cannot dynamically select based on the actual contribution of features from different spatial locations and branches, easily causing key defect information to be submerged by background features. Third, the multi-head self-attention in the Transformer encoder requires large-scale matrix operations, resulting in high computational and storage overhead on high-resolution feature maps, which is not conducive to real-time deployment on industrial edge devices. Fourth, traditional binary attention usually uses a fixed zero value as the binarization threshold, which is difficult to adapt to changes in feature distribution across different images, channels, and defect regions, easily leading to loss of detail information after binarization. Summary of the Invention

[0005] This application provides a steel surface defect detection method and system based on an improved RT-DETR. By introducing the SCS-Downsample module and DTA-Attention module into the RT-DETR model, the model can dynamically select effective features according to the response of different spatial locations and feature branches. Furthermore, the dynamic binary attention mechanism reduces the computational overhead of data representation and matrix multiplication. This solves the problems of insufficient multi-scale feature extraction capability, easy omission of slender and small defects, lack of adaptability of feature fusion, and large attention computation in existing steel surface defect detection methods, thus achieving a simultaneous improvement in detection accuracy and inference efficiency.

[0006] This application provides a method for detecting surface defects in steel based on an improved RT-DETR, including the following steps: S1. Obtain a steel surface image, label the defects in the steel surface image to obtain a dataset containing defect categories and defect bounding boxes; preprocess and augment the dataset, and divide the dataset to obtain a training set and a validation set. S2. Construct an improved RT-DETR target detection model, which includes a backbone network, an encoder, a feature fusion network, and a detection decoder. The encoder introduces a DTA-Attention module, which generates query features, key features, and value features from the deep features output by the backbone network. The query features and key features are dynamically binarized by generating dynamic offsets that adapt to batches and channels. Channel-level scaling compensation is performed on the binarized features, and then global context features are obtained through binary attention calculation. The global context features are fused with local deep convolutional features to obtain enhanced deep semantic features. An SCS-Downsample module is introduced into the bottom-up downsampling path of the feature fusion network. The SCS-Downsample module divides the input features into multiple sub-features and sends them to multiple multi-scale feature extraction branches respectively. After dynamic weighted fusion of the features of each branch through spatial-channel joint selection gating, downsampling is completed to obtain fused features enhanced by multi-scale spatial enhancement. S3. The improved RT-DETR target detection model is trained using the training set. The detection loss is calculated based on the difference between the predicted category, the predicted bounding box and the real annotation. The model parameters are updated through backpropagation. The model parameters and model weights are adjusted using the validation set to obtain the trained steel surface defect detection model. S4. Obtain the surface image of the steel to be detected, preprocess it, and input it into the trained steel surface defect detection model to output the category information and location coordinates of the defect target.

[0007] The beneficial effects of the above embodiments are as follows: By introducing the SCS-Downsample module and the DTA-Attention module, this application addresses the problems in existing steel surface defect detection, such as insufficient multi-scale feature extraction capability, easy omission of slender and small defects, lack of adaptability in feature fusion, and large attention computation cost, thereby achieving a simultaneous improvement in detection accuracy and inference efficiency. Specifically, the SCS-Downsample module enhances the model's adaptability to defects of different scales and strengthens the adaptability of feature fusion through multi-scale feature extraction branches and a spatial-channel joint selection gating mechanism; the DTA-Attention module, through dynamic binarization thresholding and channel-level scaling compensation, enhances the ability to identify small defects, slender cracks, and low-contrast defects while reducing attention computation cost.

[0008] Based on the above embodiments, this application can be further improved as follows: In one embodiment of this application, the DTA-Attention module in step S2 includes a QKV linear mapping unit, a dynamic threshold generation unit, a dynamic threshold binarization unit, a channel-level scaling compensation unit, a global binary attention calculation unit, a local depthwise convolution branch, and an output fusion unit. The dynamic threshold generation branch includes adaptive average pooling, linear transformation, SiLU activation function, and Sigmoid function to generate dynamic offsets, which are broadcast to each spatial location along the feature sequence dimension. After subtracting the corresponding dynamic offsets from the query features and the key features, they are binarized using a sign function to obtain binary query features and binary key features.

[0009] Beneficial effects: By generating branches with dynamic thresholds, the binarization threshold can be adaptively changed with batches and channels, thereby adapting to the differences in feature distribution of different images, channels and defect regions, reducing information loss caused by fixed zero-threshold binarization, and improving the adaptability of binary attention to complex industrial scenarios.

[0010] In one embodiment of this application, the DTA-Attention module in step S2 uses a pass-through estimation method to transmit the back gradient during the binarization process of the sign function, so that the sign function can participate in model training.

[0011] Beneficial effects: By using the direct estimation method, the non-differentiable sign function can properly transmit gradients during backpropagation, thereby ensuring that the binary attention structure can participate in model training end-to-end, maintaining the trainability of binary attention while reducing computational overhead.

[0012] In one embodiment of this application, the channel-level scaling compensation in step S2 is: to perform amplitude compensation on the binary query feature and the binary key feature based on the statistical mean of the query feature and the key feature and the learnable channel-level scaling parameter.

[0013] Beneficial effects: By restoring the feature amplitude information lost during binarization through the channel-level scaling compensation mechanism, the damage of quantization to feature representation is reduced, thereby enhancing the ability of binary attention to preserve defect details and low-contrast features.

[0014] In one embodiment of this application, the multiple multi-scale feature extraction branches of the SCS-Downsample module in step S2 include a strip convolution branch, a low dilation rate dilated convolution branch, and a normal convolution branch. The strip convolution branch consists of horizontal convolution and vertical convolution, which is used to enhance the feature expression of directional defects such as scratches and cracks. The low dilation rate dilated convolution branch is used to expand the receptive field, and the normal convolution branch is used to extract local texture and edge information.

[0015] Beneficial effects: Through multi-scale collaboration of strip convolution, dilated convolution and ordinary convolution, it is possible to simultaneously capture local texture, directional edge and regional structure information, significantly improving the adaptability to defects of different scales and shapes (especially fine scratches, pinhole cracks and low contrast defects).

[0016] In one embodiment of this application, the SCS-Downsample module in step S2 further sets up a global context branch and a salient spatial attention branch; the global context branch obtains global semantic information through adaptive average pooling and restores it to the spatial size of the input features through interpolation to obtain global context features; the salient spatial attention branch calculates the spatial attention response based on the mean, maximum, minimum and variance information of the input features, then obtains salient spatial attention through the Sigmoid function, adjusts the contribution by combining learnable adjustable parameters, and then adds it to the global context features to obtain salient spatial attention features.

[0017] Beneficial effects: By supplementing the global semantic relationship between the defect region and the overall steel surface texture through global contextual branches, and by enhancing the feature response of the defect region and the structural edge region through significant spatial attention branches, the model's ability to identify low-contrast defects is improved.

[0018] In one embodiment of this application, the spatial-channel joint selection gating in step S2 for dynamic weighted fusion of features of each branch includes: stacking the features of the multiple multi-scale feature extraction branches, the global context branch, and the salient spatial attention branch and summing them element-wise to obtain a summary feature; performing 1×1 convolution, batch normalization, SiLU activation function, and 3×3 convolution on the summary feature S in sequence to generate spatial branch scores corresponding to the three feature branches; normalizing the spatial branch scores in the branch dimension using the Softmax function to obtain spatial branch weights; broadcasting the spatial branch weights along the channel dimension and multiplying them pixel-wise with each branch feature, and summing them along the branch dimension to obtain multi-branch adaptive fusion features.

[0019] Beneficial effects: By dynamically weighting different feature branches through a spatial-channel joint selection gating mechanism, the model can adaptively select effective branches based on the defect feature response at different spatial locations, reducing feature redundancy caused by fixed fusion methods, preventing key defect information from being submerged by background features, and improving the adaptability and accuracy of feature fusion.

[0020] In one embodiment of this application, the SCS-Downsample module in step S2 completes downsampling by: adding the multi-scale adaptive fusion features after convolutional projection and downsampling to the input features after corresponding downsampling and channel projection via residual branching, to obtain the output features of the SCS-Downsample module.

[0021] Beneficial effects: By adding the input features to the fused features after corresponding downsampling and channel projection through residual branching, the original information before downsampling is preserved, ensuring the stability of feature transfer and reducing the risk of information decay and gradient vanishing during deep network training.

[0022] In one embodiment of this application, the improved RT-DETR target detection model adopts an end-to-end detection structure.

[0023] Beneficial effects: By directly regressing the category and bounding box of the defect target through the end-to-end detection structure, the post-processing steps such as region proposal generation and non-maximum suppression in traditional methods are omitted, simplifying the detection process and improving the efficiency of detection inference.

[0024] This application also provides a steel surface defect detection system based on an improved RT-DETR, including: The image acquisition module is used to acquire images of the steel surface, annotate the defects in the steel surface images to obtain a dataset containing defect categories and defect bounding boxes, and preprocess, augment, and partition the dataset to obtain a training set and a validation set. The model building module is used to construct an improved RT-DETR object detection model. This improved RT-DETR object detection model includes a backbone network, an encoder, a feature fusion network, and a detection decoder. The encoder incorporates a DTA-Attention module, which generates query features, key features, and value features from the deep features output by the backbone network. It dynamically binarizes the query and key features by generating dynamic offsets that adapt to batch and channel variations, and performs channel-level scaling compensation on the binarized features. Then, it calculates global context features through binary attention calculation. These global context features are fused with local deep convolutional features to obtain enhanced deep semantic features. The feature fusion network incorporates an SCS-Downsample module in its bottom-up downsampling path. This SCS-Downsample module divides the input features into multiple sub-features and feeds them into multiple multi-scale feature extraction branches. It performs dynamic weighted fusion of the features from each branch through spatial-channel joint selection gating to complete downsampling, resulting in fused features enhanced by multi-scale spatial enhancement. The training module is used to train the improved RT-DETR target detection model using the training set, calculate the detection loss based on the difference between the predicted category, the predicted bounding box and the real annotation, update the model parameters through backpropagation, and adjust the model parameters and model weights using the validation set to obtain the trained steel surface defect detection model. The detection module is used to acquire images of the steel surface to be detected, preprocess them, and then input them into the trained steel surface defect detection model, outputting the category information, location coordinates, and confidence information of the defect target.

[0025] The system in this application achieves the above-mentioned steel surface defect detection method based on improved RT-DETR by working together through an image acquisition module, a model building module, a training module, and a detection module. This method can improve defect detection accuracy and inference efficiency, and adapt to the real-time detection needs of edge devices. Attached Figure Description

[0026] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale.

[0027] Figure 1 This is a flowchart illustrating the steps of the steel surface defect detection method based on the improved RT-DETR in the embodiments of this application; Figure 2This is a schematic diagram of the overall structure of the improved RT-DETR target detection model in the embodiments of this application; Figure 3 This is a schematic diagram of the DTA-Attention module in an embodiment of this application; Figure 4 This is a schematic diagram of the dynamic threshold binarization and channel-level scaling compensation process in the DTA-Attention module of this application embodiment; Figure 5 This is a schematic diagram illustrating the feature fusion of the global binary attention branch and the local depthwise convolution branch in the DTA-Attention module of this application embodiment; Figure 6 This is a schematic diagram of the overall structure of the SCS-Downsample module in an embodiment of this application; Figure 7 This is a schematic diagram of the multi-scale feature extraction branches of strip convolution, dilated convolution, and ordinary convolution in the SCS-Downsample module of this application embodiment; Figure 8 This is a schematic diagram of the spatial-channel joint selection gating and multi-branch adaptive fusion process in the SCS-Downsample module of this application embodiment; Figure 9 This is a schematic diagram showing the module embedding position of the improved RT-DETR model in the feature fusion network in the embodiments of this application. Detailed Implementation

[0028] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0029] Example 1: like Figure 1 As shown, a method for detecting surface defects in steel based on an improved RT-DETR includes the following steps: S1. Data acquisition and preprocessing; Acquiring images of steel surfaces requires including defect categories such as pinholes, inclusions, spots, pitted surfaces, entrained oxide layers, and scratches, covering common point-like, block-like, linear, and irregularly shaped defects on steel surfaces. This embodiment uses the NEU-DET steel surface defect dataset as an example. This dataset contains 1800 images, with 300 images each for pinholes, inclusions, spots, pitted surfaces, entrained oxide layers, and scratches.

[0030] After acquiring images of the steel surface, defects in the images are first labeled to obtain a labeled dataset containing defect categories and defect bounding boxes, which serves as the ground truth annotations for supervised training of the model. The dataset is then preprocessed as follows: the original images are uniformly resized to a specific input size (e.g., 640×640), and the image pixel values ​​are normalized. Finally, the dataset is divided into training, validation, and test sets in an 8:1:1 ratio, with the training set containing 1440 images, and the validation and test sets each containing 180 images.

[0031] To improve the model's ability to generalize to complex industrial scenarios, this embodiment also employs data augmentation strategies during training, including random flipping, random scaling, random cropping, brightness and contrast perturbation, random rotation, and noise perturbation, to simulate the differences in steel surface imaging under different acquisition conditions.

[0032] In this embodiment, the training set is used for training and updating model parameters; the validation set is used to adjust model hyperparameters and model weights during training to suppress overfitting and obtain a better model; the test set does not participate in model training and parameter adjustment, but is only used to evaluate the final detection performance of the model after the model training is completed.

[0033] S2. Construct an improved RT-DETR target detection model; An improved steel surface defect detection model is constructed using the RT-DETR model as the basic detection framework. For example... Figure 2 As shown, the model includes a backbone network, an encoder, a feature fusion network, and a detection decoder. The backbone network extracts multi-level features from the input image; the encoder performs global context modeling of deep semantic features; the feature fusion network fuses features at different scales in a top-down and bottom-up manner; and the detection decoder outputs the defect category, location, and confidence level based on the fused multi-scale features. Specifically, it includes: S2.1, Introduce the DTA-Attention module into the encoder; like Figure 2 As shown, in this embodiment, the three scale feature maps output by the backbone network are denoted as P3 / 8, P4 / 16, and P5 / 32, respectively. P3 / 8 represents a shallow high-resolution feature map downsampled by 8 times relative to the input image, P4 / 16 represents a mid-level feature map downsampled by 16 times relative to the input image, and P5 / 32 represents a deep semantic feature map downsampled by 32 times relative to the input image. The terms P3 / 8, P4 / 16, and P5 / 32 are used to identify the feature scale and network connection relationships in this embodiment, and do not imply that unique, fixed layer names must be used.

[0034] The deep feature P5 / 32 is input into the DTA-Attention module for global contextual feature modeling, resulting in the enhanced deep feature Y5. Y5 is upsampled and concatenated with the mid-level feature P4 / 16, then fused using the RepC3 module to obtain the mid-level fused feature Y4. Y4 is then upsampled and concatenated with the shallow feature P3 / 8, then fused using the RepC3 module to obtain the high-resolution fused feature X3.

[0035] In the bottom-up feature fusion path, X3 is downsampled by the SCS-Downsample module and concatenated with Y4, then fused by the RepC3 module to obtain feature F4. F4 is further downsampled by the SCS-Downsample module and concatenated with Y5, then fused by the RepC3 module to obtain feature F5. Finally, X3, F4, and F5 are input together into the RT-DETR detection decoder to achieve the detection of defects at different scales.

[0036] like Figure 3 and Figure 4 As shown, the DTA-Attention module includes a QKV linear mapping unit, a dynamic threshold generation unit, a dynamic threshold binarization unit, a channel-level scaling compensation unit, a global binary attention calculation unit, a local depthwise convolution branch, and an output fusion unit. The input to this module is a deep feature map X of shape B×C×H×W, where B represents the batch size, C represents the number of channels, and H and W represent the feature map height and width, respectively; the output is an enhanced feature map that matches the input spatial dimensions.

[0037] First, the DTA-Attention module flattens and transposes the input feature map X into a feature sequence of shape N×C, where N=H×W; then, it generates query feature Q, key feature K, and value feature V through QKV linear mapping, and rearranges Q, K, and V into a multi-head attention form.

[0038] like Figure 4As shown, the DTA-Attention module constructs dynamic threshold generation branches for the query feature Q and the key feature K, respectively. For ease of description, the features to be quantized can be uniformly denoted as Z. When Z is Q, it corresponds to the query dynamic threshold generation branch; when Z is K, it corresponds to the key dynamic threshold generation branch. The dynamic threshold generation branch includes adaptive average pooling, a first linear transformation, a SiLU activation function, a second linear transformation, and a Sigmoid function, used to generate dynamic offsets. For Q and K, query dynamic offsets and key dynamic offsets are obtained, respectively. The dynamic offsets are generated according to the dimensions of input samples, attention heads, and channels, and broadcast to various spatial locations along the feature sequence dimension, enabling different input samples, different attention heads, and different channels to use different binarization thresholds. Unlike traditional methods that directly use a fixed zero value as the binarization threshold, this embodiment achieves adaptation to changes in feature distribution through dynamic offsets. After obtaining the dynamic offsets, the query feature Q and the key feature K are subtracted by the corresponding dynamic offsets (τ_Q and τ_K), respectively, and then binarized using a sign function. The binarization process can be represented as B_Q=Sign(Q-τ_Q), B_K=Sign(K-τ_K). Here, Sign represents the sign function, and its output value is -1 or +1. During training, the sign function uses a pass-through estimation method to propagate the backpropagation gradient, enabling the binarization operation to participate in end-to-end training.

[0039] To reduce the loss of amplitude information during binarization, this embodiment further introduces channel-level scaling compensation. For example... Figure 4 As shown, for the feature Z to be quantized, its mean amplitude Mean_N(|Z|) is calculated along the feature sequence dimension N, and combined with the learnable channel-level scaling parameter α_Z to form the scaling compensation coefficient γ_Z, i.e., γ_Z = Mean_N(|Z|) × α_Z; for the query feature Q and the key feature K, the learnable channel-level scaling parameters α_Q and α_K are set respectively, and the corresponding scaling compensation coefficients γ_Q and γ_K are obtained respectively, thus obtaining the compensated binary query feature and binary key feature. After dynamic threshold binarization and channel-level scaling compensation, the compensated binary query feature Q_hat and the compensated binary key feature K_hat are obtained, and the process can be expressed as: Q_hat=Sign(Q-τ_Q)×Mean_N(|Q|)×α_Q; K_hat=Sign(K-τ_K)×Mean_N(|K|)×α_K; The aforementioned Q_hat and K_hat are used for subsequent global binary attention correlation calculations.

[0040] like Figure 5As shown, the DTA-Attention module performs matrix multiplication on the transpose of the compensated binary query features and binary key features (Q_hat and K_hat), and then performs scaling and Softmax normalization to obtain the attention weight A. Subsequently, the attention weight A is used to weight the value features V to obtain the global context enhancement feature F_g. This global branch preserves the long-distance dependency modeling capability of the Transformer encoder, while reducing the data representation and matrix multiplication overhead in the attention relevance calculation process through the binarized query and key features (Q_hat and K_hat).

[0041] At the same time, such as Figure 3 and Figure 5 As shown, the DTA-Attention module also includes a local depthwise convolution branch. This branch can consist of a 3×3 depthwise convolution, batch normalization, and the SiLU activation function. The 3×3 depthwise convolution performs local spatial modeling on the input feature map X according to channel grouping, which is used to extract minute defects, fine scratches, edge textures, and local high-frequency details on the steel surface. Finally, the global context enhancement feature F_g is restored to a spatial feature map form after linear projection and Dropout processing, and then added element-wise with the local feature F_l obtained from the local depthwise convolution branch to obtain the output feature Y of the DTA-Attention module.

[0042] With the above structure, the DTA-Attention module has at least the following improvements compared to the original model: First, it achieves dynamic threshold binarization of query features and key features through dynamic offsets, avoiding the problem that a fixed zero threshold is difficult to adapt to changes in the distribution of steel surface defect features; Second, it reduces the expression loss caused by binary quantization by using amplitude statistics and learnable channel-level scaling parameters for amplitude compensation; Third, it supplements the local spatial details required for small defects, fine scratches, and edge textures through local depth convolution branches.

[0043] S2.2, Introduce the SCS-Downsample module into the feature fusion network; like Figure 6 As shown, the SCS-Downsample module is used for bottom-up downsampling of the feature fusion network. Its input is the fused feature map X from the previous layer, and its output is the downsampled enhanced feature map Y. If the shape of the input feature X is B×C×H×W, then the spatial size of the output feature Y is H / 2×W / 2, and the number of output channels is the preset Cout. This module includes a multi-scale spatial feature extraction branch, a global context branch, a salient spatial attention branch, a spatial-channel joint selection gating unit, a projection downsampling unit, and a residual connection unit.

[0044] like Figure 6As shown, the SCS-Downsample module constructs a global context branch and a salient spatial attention branch. The global context branch obtains global features through adaptive average pooling and 1×1 convolution, and recovers the spatial size of the input feature X through bilinear interpolation to obtain the global context feature x_g. This branch is used to supplement the global semantic relationship between the defect region and the overall steel surface texture. The salient spatial attention branch calculates the spatial attention response based on the mean, maximum, minimum, and variance information of the input feature X. In one implementation, the mean, maximum, minimum, and variance var of the input feature in the spatial dimension are first calculated, and then the spatial response is obtained according to q_ij=((X-mean)^2 / (max-min+ε))×var, where ε is a minimal constant to prevent the denominator from being zero; then, the salient spatial attention is obtained through the Sigmoid function, and its contribution is adjusted by combining a learnable adjustable parameter β, and then added to the global context feature x_g to obtain the salient spatial attention feature x_s. Multi-scale spatial features x_m, global context features x_g, and salient spatial attention features x_s are input into a spatial-channel joint selection gating unit to generate selection weights for each branch. The features of each branch are then weighted and fused according to these selection weights to obtain a multi-branch adaptive fusion feature. Subsequently, this multi-branch adaptive fusion feature is input into a projection downsampling unit, which uses a 3×3 convolution with a stride of 2 to perform spatial downsampling and channel projection on the fusion feature, resulting in a projection downsampled feature. Simultaneously, a residual connection unit downsamples and performs channel matching on the input feature X to obtain a residual feature. Finally, the projection downsampled feature and the residual feature are added element-wise to obtain the output enhanced feature Y, where the spatial size of Y is H / 2×W / 2 and the number of channels is Cout.

[0045] like Figure 7 As shown, the SCS-Downsample module constructs a multi-scale feature extraction branch. First, it divides the input feature X into three sub-features according to a preset channel ratio. In one implementation, the channel division ratio is 2:1:1, that is, the input feature is divided into a strip convolution branch input X_strip with C / 2 channels, a low-dilation dilated convolution branch input X_3 with C / 4 channels, and a regular convolution branch input X_1 with C / 4 channels. The above ratio is used to illustrate a specific implementation method; in actual implementation, it can be adjusted equivalently according to the number of channels and computing resources.

[0046] For the strip convolution branch, 1×7 horizontal convolution and 7×1 vertical convolution are used to extract slender structural information in the horizontal and vertical directions, respectively. The outputs of the two are then added and processed by the SiLU activation function to obtain the strip feature f_strip. This branch is used to enhance the slender defect features with obvious directionality, such as scratches and pinhole cracks on the steel surface. For the low dilation rate dilated convolution branch, a 3×3 dilated convolution with a dilation rate of 3 is used to obtain the dilated feature f_3, in order to expand the receptive field while avoiding the dilution of local details that may be caused by a large dilation rate dilated convolution. For the ordinary convolution branch, a 3×3 ordinary convolution with a dilation rate of 1 is used to obtain the local feature f_1, which is used to extract defect texture, edge, and contour information. After concatenating the features of the above three branches (f_strip, f_3, and f_1) in the channel dimension, a 1×1 convolution is used for channel fusion to obtain the multi-scale spatial feature x_m.

[0047] like Figure 8 As shown, the SCS-Downsample module stacks multi-scale spatial features x_m, global context features x_g, and salient spatial attention features x_s into a branch feature set F=[x_m,x_g,x_s], whose shape can be represented as B×3×C×H×W. Simultaneously, the summation of the three features x_m, x_g, and x_s element-wise yields the summary feature S. The spatial-channel joint selection gating unit sequentially performs 1×1 convolution, batch normalization, SiLU activation function, and 3×3 convolution on the summary feature S, generating spatial branch scores corresponding to the three feature branches. The spatial branch scores are normalized in the branch dimension using the Softmax function to obtain spatial branch weights of shape B×3×1×H×W. Since the input to the gating unit is the summary feature S containing channel information, the channel information participates in weight generation; and since the output weights retain the spatial dimension, different branch responses can be selected for different spatial locations. Then, the spatial branch weight W is broadcast along the channel dimension and multiplied pixel-wise with the three branch features x_m, x_g and x_s respectively, and summed along the branch dimension to obtain the multi-branch adaptive fusion feature fused, that is, fused=Σ_i W_i⊙F_i, where ⊙ represents element-wise multiplication.

[0048] With the above structure, the SCS-Downsample module has at least the following improvements compared to the original module: First, it uses strip convolution branches to enhance orientation perception for fine scratches and cracks; second, it replaces the combination of convolutions with larger dilation rates with a combination of low-dilation rate dilated convolutions and ordinary convolutions, taking into account both the receptive field and local details; third, it replaces fixed fusion with spatial-channel joint selection gating, allowing effective branches to be dynamically selected at different spatial locations; and fourth, it maintains feature transfer stability through residual downsampling connections, reducing the risk of information decay during deep training.

[0049] like Figure 9 As shown, the deep features P5 / 32 are first projected into the input through a 1×1 convolution to adjust the number of channels to the dimension required by the subsequent encoder; then the projected deep features are input into the DTA-Attention module. In this embodiment, the DTA-Attention module is used to replace the standard AIFI encoding module or standard attention calculation structure in the original RT-DETR, while retaining the residual connections, normalization layers, and feedforward network structure in the Transformer encoding layer, thereby enhancing the expression of deep semantic features while maintaining the end-to-end detection framework. In the feature fusion network, the deep features processed by the DTA-Attention module are laterally projected to obtain Y5; Y5 is upsampled and concatenated with P4 / 16, and then fused through RepC3 to obtain Y4; Y4 is upsampled and concatenated with P3 / 8, and then fused through RepC3 to obtain the high-resolution fused feature X3. In the bottom-up path, X3 is downsampled by the SCS-Downsample module and concatenated with Y4, then fused with RepC3 to obtain F4; F4 is then downsampled by the SCS-Downsample module and concatenated with Y5, then fused with RepC3 to obtain F5. Finally, X3, F4, and F5 are input together into the RT-DETR detection decoder to complete multi-scale defect detection.

[0050] In this embodiment, DTA-Attention is a dynamic threshold adaptive attention module improved to address the problems of insufficient binarization and local detail compensation in the original module; SCS-Downsample is a spatial-channel adaptive selection downsampling module designed for the task of detecting defects on steel surfaces.

[0051] S2.3, Detector; During the forward propagation process of training or inference, X3, F4, and F5, processed by the feature fusion network, are input into the RT-DETR detection decoder. The detection decoder directly outputs the predicted category, predicted bounding box, and confidence score of the steel surface defect target through the interaction between the target query and multi-scale image features. Since this embodiment is based on the RT-DETR end-to-end detection framework, no additional complex non-maximum suppression post-processing is required during the inference stage.

[0052] S3. Train the model and adjust the parameters and weights; The preprocessed training set is input into the improved RT-DETR model for training. During training, the model outputs predicted bounding boxes, class probabilities, and confidence scores through forward propagation, and calculates the loss function based on the difference between the prediction results and the ground truth labels. Subsequently, the learnable parameters in the backbone network, DTA-Attention module, SCS-Downsample module, feature fusion network, and detector decoder are updated through backpropagation until the model converges or reaches the preset number of training epochs.

[0053] During training, the model hyperparameters and model weights are adjusted using the validation set. After training, the final detection performance of the model is evaluated using the test set.

[0054] S4. Model detection, outputting the type and location of defects; During the detection phase, the model weights that perform best on the validation or test set are loaded and used for inference on the surface image of the steel to be detected. After resizing and normalization, the input image enters the backbone network to extract multi-scale features such as P3 / 8, P4 / 16, and P5 / 32; deep features are enhanced by dynamic threshold binary attention and local detail compensation through the DTA-Attention module; the bottom-up downsampling path in the feature fusion network is enhanced by spatial-channel adaptive downsampling through the SCS-Downsample module; finally, the RT-DETR detection decoder outputs the category, bounding box coordinates, and confidence score of the defect target.

[0055] Experimental Results and Analysis: To verify the effectiveness of the method of the present invention, this experiment was conducted on a server equipped with an NVIDIA GeForce RTX 3090 GPU. The hyperparameter settings of the training core are shown in Table 1.

[0056]

[0057] This embodiment compares the improved RT-DETR model with the original RT-DETR model and other comparative models on the NEU-DET dataset. Evaluation metrics include Precision, Recall, F1-Score, mAP@0.5, mAP@0.5:0.95, and number of parameters. The experimental results for each model are shown in Table 2.

[0058]

[0059] Experimental results show that the improved RT-DETR model proposed in this invention has a precision of 72.4%, a recall of 67.7%, an F1-Score of 70.0%, a mAP@0.5 of 71.6%, a mAP@0.5:0.95 of 43.1%, and a parameter count of 20,185,256 on the NEU-DET dataset.

[0060] Compared to the original RT-DETR, the method of this invention improves mAP@0.5 by 2.2% and recall by 4.3%, and exhibits better stability in complex backgrounds and scenarios involving the detection of minute defects. Especially for small or elongated defects such as pinholes, scratches, and dents, the SCS-Downsample module enhances local structural and directional features, while the DTA-Attention module maintains global context modeling capabilities while reducing the computational overhead of binary attention data representation and matrix multiplication, resulting in superior overall detection performance.

[0061] Example 2: A steel surface defect detection system based on an improved RT-DETR is provided to implement the detection method shown in Example 1. The system includes the following modules: The image acquisition module is used to acquire images of the steel surface, annotate defects in the steel surface images, obtain a dataset containing defect categories and defect bounding boxes, and perform preprocessing, data augmentation, and dataset partitioning to obtain training and validation sets. The model building module is used to build an improved RT-DETR object detection model, which includes a backbone network, an encoder, a feature fusion network, and a detection decoder. The encoder introduces a DTA-Attention module, and the feature fusion network introduces an SCS-Downsample module in its bottom-up downsampling path. The training module is used to train the improved RT-DETR object detection model using the training set, calculate the detection loss based on the difference between the predicted category, the predicted bounding box and the ground truth annotation, update the model parameters through backpropagation, and adjust the model parameters and model weights using the validation set to obtain the trained steel surface defect detection model. The detection module is used to acquire images of the steel surface to be detected, preprocess them, and then input them into the trained steel surface defect detection model. It outputs the category information, location coordinates, and confidence information of the defect target.

[0062] The functions of each module mentioned above correspond to the content of their respective method embodiments, and will not be repeated here.

[0063] The technical solutions described in the embodiments of this application have at least the following technical effects or advantages: 1. By constructing a multi-scale feature extraction structure through the SCS-Downsample module, including strip convolution, low dilation rate dilated convolution, ordinary convolution, global context branch and significant spatial attention branch, it can simultaneously capture local texture, directional edges, regional structure and global semantic information, and improve the model's adaptability to defects of different scales. 2. By using a spatial-channel joint selection gating mechanism, different feature branches are dynamically weighted, enabling the model to adaptively select effective features based on the location of the defect and the feature response, thereby reducing feature redundancy and loss of key information caused by fixed fusion methods. 3. By using dynamic offsets instead of fixed zero thresholds through the DTA-Attention module, query features and key features can adapt to the distribution changes of different channels and different input features, thereby improving the feature representation capability of binarized attention. 4. By restoring the amplitude information lost during binarization through a channel-level scaling compensation mechanism and supplementing spatial details by combining local depth convolution branches, the model's ability to identify minute defects, slender cracks and low-contrast defects is enhanced. 5. By using binary query features and key features, the data representation complexity in the attention calculation process is reduced, and the computational overhead when binary features participate in matrix multiplication is reduced. While retaining the global context modeling capability, this is beneficial for the deployment of the model on edge devices. 6. This invention is based on the RT-DETR end-to-end detection framework, which eliminates the need for additional complex non-maximum suppression post-processing procedures, thereby simplifying the detection process and improving the stability and real-time performance of the detection system.

[0064] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for detecting surface defects in steel based on an improved RT-DETR, characterized in that, Includes the following steps: S1. Obtain a steel surface image, annotate the defects in the steel surface image, and obtain a dataset containing defect categories and defect bounding boxes; The dataset is preprocessed and augmented, and then divided into training and validation sets. S2. Construct an improved RT-DETR target detection model, which includes a backbone network, an encoder, a feature fusion network, and a detection decoder. The encoder introduces a DTA-Attention module, which generates query features, key features, and value features from the deep features output by the backbone network. The query features and key features are dynamically binarized by generating dynamic offsets that adapt to batches and channels. Channel-level scaling compensation is performed on the binarized features, and then global context features are obtained through binary attention calculation. The global context features are fused with local deep convolutional features to obtain enhanced deep semantic features. An SCS-Downsample module is introduced into the bottom-up downsampling path of the feature fusion network. The SCS-Downsample module divides the input features into multiple sub-features and sends them to multiple multi-scale feature extraction branches respectively. After dynamic weighted fusion of the features of each branch through spatial-channel joint selection gating, downsampling is completed to obtain fused features enhanced by multi-scale spatial enhancement. S3. The improved RT-DETR target detection model is trained using the training set. The detection loss is calculated based on the difference between the predicted category, the predicted bounding box and the real annotation. The model parameters are updated through backpropagation. The model parameters and model weights are adjusted using the validation set to obtain the trained steel surface defect detection model. S4. Obtain the surface image of the steel to be detected, preprocess it, and input it into the trained steel surface defect detection model to output the category information and location coordinates of the defect target.

2. The method for detecting surface defects in steel according to claim 1, characterized in that: The DTA-Attention module in step S2 includes a QKV linear mapping unit, a dynamic threshold generation unit, a dynamic threshold binarization unit, a channel-level scaling compensation unit, a global binary attention calculation unit, a local depthwise convolution branch, and an output fusion unit. The dynamic threshold generation branch includes adaptive average pooling, linear transformation, SiLU activation function, and Sigmoid function to generate dynamic offsets, which are broadcast to each spatial location along the feature sequence dimension. After subtracting the corresponding dynamic offsets from the query features and the key features, they are binarized using a sign function to obtain binary query features and binary key features.

3. The method for detecting surface defects in steel according to claim 2, characterized in that: In step S2, the DTA-Attention module uses a pass-through estimation method to transmit the back gradient during the binarization process of the sign function, enabling the sign function to participate in model training.

4. The method for detecting surface defects in steel according to claim 2, characterized in that: The channel-level scaling compensation in step S2 is as follows: based on the statistical mean amplitude of the query feature and the key feature, and the learnable channel-level scaling parameter, amplitude compensation is performed on the binary query feature and the binary key feature.

5. The method for detecting surface defects in steel according to claim 1, characterized in that: The SCS-Downsample module in step S2 includes multiple multi-scale feature extraction branches such as a strip convolution branch, a low dilation rate dilated convolution branch, and a normal convolution branch. The strip convolution branch consists of horizontal and vertical convolutions and is used to enhance the feature representation with directional defects. The low dilation rate dilated convolution branch is used to expand the receptive field. The normal convolution branch is used to extract local texture and edge information.

6. The method for detecting surface defects in steel according to claim 5, characterized in that: The SCS-Downsample module described in step S2 also sets up a global context branch and a salient spatial attention branch; The global context branch obtains global semantic information through adaptive average pooling and restores it to the spatial size of the input features through interpolation to obtain global context features; The salient spatial attention branch calculates the spatial attention response based on the mean, maximum, minimum and variance information of the input features. Then, it obtains the salient spatial attention through the Sigmoid function, adjusts the contribution by combining learnable adjustable parameters, and adds it to the global context features to obtain the salient spatial attention features.

7. The method for detecting surface defects in steel according to claim 6, characterized in that: Step S2, which involves the spatial-channel joint selection gating to dynamically weight and fuse the features of each branch, includes: stacking the features of the multiple multi-scale feature extraction branches, the global context branch, and the salient spatial attention branch, and summing them element-wise to obtain a summary feature; sequentially performing 1×1 convolution, batch normalization, SiLU activation function, and 3×3 convolution on the summary feature S to generate spatial branch scores corresponding to the three feature branches; normalizing the spatial branch scores along the branch dimension using the Softmax function to obtain spatial branch weights; broadcasting the spatial branch weights along the channel dimension, multiplying them pixel-wise with each branch feature, and summing them along the branch dimension to obtain multi-branch adaptive fusion features.

8. The method for detecting surface defects in steel according to claim 7, characterized in that: The SCS-Downsample module in step S2 completes downsampling by: processing the multi-scale adaptive fusion features through convolutional projection and downsampling, and then adding them to the input features after corresponding downsampling and channel projection through residual branching, to obtain the output features of the SCS-Downsample module.

9. The method for detecting surface defects in steel according to claim 1, characterized in that: The improved RT-DETR target detection model adopts an end-to-end detection structure.

10. A steel surface defect detection system based on an improved RT-DETR, characterized in that, A method for detecting surface defects in steel as described in any one of claims 1-9, comprising: The image acquisition module is used to acquire images of the steel surface, annotate the defects in the steel surface images to obtain a dataset containing defect categories and defect bounding boxes, and preprocess, augment, and partition the dataset to obtain a training set and a validation set. The model building module is used to construct an improved RT-DETR object detection model. This improved RT-DETR object detection model includes a backbone network, an encoder, a feature fusion network, and a detection decoder. The encoder incorporates a DTA-Attention module, which generates query features, key features, and value features from the deep features output by the backbone network. It dynamically binarizes the query and key features by generating dynamic offsets that adapt to batch and channel variations, and performs channel-level scaling compensation on the binarized features. Then, it calculates global context features through binary attention calculation. These global context features are fused with local deep convolutional features to obtain enhanced deep semantic features. The feature fusion network incorporates an SCS-Downsample module in its bottom-up downsampling path. This SCS-Downsample module divides the input features into multiple sub-features and feeds them into multiple multi-scale feature extraction branches. It performs dynamic weighted fusion of the features from each branch through spatial-channel joint selection gating to complete downsampling, resulting in fused features enhanced by multi-scale spatial enhancement. The training module is used to train the improved RT-DETR target detection model using the training set, calculate the detection loss based on the difference between the predicted category, the predicted bounding box and the real annotation, update the model parameters through backpropagation, and adjust the model parameters and model weights using the validation set to obtain the trained steel surface defect detection model. The detection module is used to acquire images of the steel surface to be detected, preprocess them, and then input them into the trained steel surface defect detection model, outputting the category information, location coordinates, and confidence information of the defect target.