A multi-scale steel surface defect detection method based on RT-DETR improvement

Abstract

This application discloses a multi-scale steel surface defect detection method based on an improved RT-DETR model, belonging to the field of target detection and recognition technology. The method includes the following steps: acquiring an image of the steel surface to be detected; inputting the image into an improved RT-DETR model for defect detection; wherein the improvement lies in: replacing the feature fusion module of the RT-DETR model with a multi-scale fusion and reconstruction module; deploying dynamic sampling modules at the upsampling nodes located at the output ends of the second and third level feature maps in the upsampling module; and using a scale-adaptive Gaussian localization loss function as a supervision signal for bounding box regression during the training process of the improved model. This application can improve the accuracy of steel surface defect detection.

Description

Technical Field

[0001] This application belongs to the field of target detection and recognition technology, specifically relating to a multi-scale steel surface defect detection method based on RT-DETR improvement. Background Technology

[0002] In the task of detecting defects on the surface of steel, a large number of defects exist in the form of tiny targets, and the size differences between various types of defects are huge.

[0003] Therefore, the model faces the following major challenges during detection: (1) The defect samples have a significant scale imbalance problem, which makes it difficult for the model to accurately identify multi-scale and small defects in complex backgrounds. At the same time, their feature information is weak and easily masked by background noise, resulting in low detection accuracy.

[0004] (2) Industrial production lines have high requirements for real-time detection, but existing high-performance models are often complex in structure, have high computational costs, and their inference speed is difficult to meet the real-time detection needs of continuous operation.

[0005] (3) The strategy of directly aggregating multi-level features adopted by some models can lead to problems such as feature redundancy, which hinders the model's feature learning. Summary of the Invention

[0006] The purpose of this application is to provide a multi-scale steel surface defect detection method based on RT-DETR improvement, which can improve the accuracy of steel surface defect detection.

[0007] To solve the above-mentioned technical problems, this application is implemented as follows: In a first aspect, embodiments of this application provide a multi-scale steel surface defect detection method based on RT-DETR improvement, the method comprising: Acquire images of the steel surface to be inspected; The image is input into an improved RT-DETR model for defect detection; In this improved RT-DETR model, the feature fusion module is replaced by a multi-scale fusion and reconstruction module. The multi-scale fusion and reconstruction module includes sequentially cascaded SCConv and PKIS sub-modules, with the output of the SCConv sub-module connected to the input of the PKIS sub-module. In the upsampling module of the improved RT-DETR model, dynamic sampling modules are deployed at the upsampling nodes located at the output ends of the second-level feature map and the third-level feature map, while the nearest neighbor interpolation upsampling structure is retained at the remaining upsampling nodes. The improved RT-DETR model uses a scale-adaptive Gaussian localization loss function as a supervision signal for bounding box regression during training. The scale-adaptive Gaussian localization loss function models the predicted bounding box and the real bounding box as a two-dimensional Gaussian distribution, and introduces a scale-aware weighting factor based on the defect target area to weight the Gaussian distance. The weighting factor has an exponential decay relationship with the defect target area.

[0008] In a second aspect, embodiments of this application provide a computer device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0009] Thirdly, embodiments of this application provide a computer-readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0010] Fourthly, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0011] This application proposes an MDG-DETR model (i.e., an RT-DETR model based on a Multi-Scale Fusion & Reconstruction (MSFR) module, a Dynamic Sampling (DySample) module, and a Scale-Adaptive Gaussian Localization (SAGL) loss). Starting from the actual imaging characteristics of steel surface defect detection, it unifies background redundancy suppression, multi-scale defect representation, preservation of small defect boundaries, and scale imbalance supervision into a single detection process. The MSFR module, DySample module, and SAGL loss are comprehensively designed to adapt to steel inspection scenarios in industrial applications. Specifically: (1) To address the issues of drastic changes in multi-scale defects and feature redundancy, this application proposes a multi-scale fusion module called MSFR and specifically deploys it into the feature fusion stage of the model. MSFR combines two complementary sub-modules, Spatial and Channel Reconstruction Convolution (SCConv) and Poly Kernel Inception with SimAM (PKIS), in an order of action, forming a causal chain from redundancy filtering to feature enhancement. Embedding MSFR into the RT-DETR hybrid encoder enables efficient feature rearrangement and redundant feature filtering, thereby enhancing the model's multi-scale feature fusion capability.

[0012] (2) To address the problem of blurred edge details of minor defects, the DySample dynamic sampling mechanism is introduced to reconstruct the upsampling process into a process of bilinear interpolation combined with content-aware point sampling, and it is limited to a specific level in the feature fusion path, so that the model can more accurately retain the semantic information in the feature map and improve the quality of upsampling.

[0013] (3) To address the difficulty in locating minute defects and irregular edges of defects, this application proposes an SAGL loss. This SAGL loss introduces a scale-aware dynamic weighting mechanism on the basis of the traditional Gaussian distribution distance metric, and adaptively adjusts the gradient gain according to the actual area of ​​the defect, thus solving the problem of insufficient gradient for small targets under fixed weights. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a multi-scale steel surface defect detection method based on RT-DETR improvement provided in some embodiments of this application; Figure 2 These are structural diagrams of the RT-DETR model involved in some embodiments of this application; Figure 3 These are structural diagrams of the MDG-DETR model provided in some embodiments of this application; Figure 4 These are some MSFR module structure diagrams provided in embodiments of this application; Figure 5 This is a structural diagram of the DySample module provided in some embodiments of this application; Figure 6 These are structural diagrams of sampling blocks based on static range factors (i.e., offsets) and sampling blocks based on dynamic range factors provided in some embodiments of this application; Figure 7These are example images of six types of strip surface defects from the NEU-DET dataset involved in some embodiments of this application; Figure 8 These are example images of ten types of steel surface defects from the GC10-DET dataset involved in some embodiments of this application; Figure 9 These are internal structural diagrams of a computer device provided in some embodiments of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0017] In one exemplary embodiment, this application proposes a multi-scale steel surface defect detection method based on RT-DETR improvement. The following, in conjunction with the accompanying drawings, provides a detailed description of the RT-DETR-based multi-scale steel surface defect detection method provided by this application through specific embodiments and application scenarios.

[0018] like Figure 1 , Figure 2 and Figure 3 As shown, the method includes steps 102-104. Wherein: Step 102: Obtain an image of the steel surface to be inspected.

[0019] Specifically, an image of the steel surface to be inspected can be acquired using an industrial camera or image acquisition device. This steel surface image can be an RGB image or a grayscale image; the image size is not limited in this embodiment and can be uniformly scaled to the target size in subsequent processing.

[0020] Step 104: Input the image into the improved RT-DETR model for defect detection.

[0021] In this improved RT-DETR model, the feature fusion module is replaced by a multi-scale fusion and reconstruction module. The multi-scale fusion and reconstruction module includes sequentially cascaded SCConv and PKIS sub-modules, with the output of the SCConv sub-module connected to the input of the PKIS sub-module.

[0022] In the upsampling module of the improved RT-DETR model, dynamic sampling modules are deployed at the upsampling nodes located at the output ends of the second-level feature map and the third-level feature map, while the nearest neighbor interpolation upsampling structure is retained at the remaining upsampling nodes.

[0023] The decoder of the improved RT-DETR model uses a scale-adaptive Gaussian localization loss function as a supervision signal for bounding box regression. The scale-adaptive Gaussian localization loss function models the predicted bounding box and the real bounding box as a two-dimensional Gaussian distribution, and introduces a scale-aware weighting factor based on the defect target area to weight the Gaussian distance. The weighting factor has an exponential decay relationship with the defect target area.

[0024] Specifically, in response to the challenges of uneven scale distribution of defect samples and easy loss of information during the feature fusion stage in current steel inspection, this embodiment proposes an efficient MSFR module to achieve improvement.

[0025] like Figure 4 As shown, the MSFR module is composed of two complementary and sequentially combined sub-modules (i.e., cascaded sequentially). First, the SCConv sub-module is placed at the beginning. This SCConv sub-module performs redundancy suppression in both the channel and spatial dimensions, and combines the suppression results from the channel and spatial dimensions to perform feature rearrangement. In other words, this SCConv sub-module achieves feature filtering and feature rearrangement operations through collaborative redundancy suppression in both spatial and channel dimensions, providing high-quality input features for subsequent multi-scale feature fusion and further enhancing defect edge information.

[0026] Building upon this foundation, secondary processing is then performed by the PKIS submodule. The core design of PKIS is a combination of multi-scale depthwise separable convolution (DWConv) and an asymmetrically deployed attention enhancement module. Compared to other convolution methods, in steel surface defect detection tasks, using multi-scale DWConv not only significantly reduces the number of model parameters and computational cost, but also further enhances the model's representational capabilities through the expanded receptive field effect resulting from the combination of multi-scale receptive field and pointwise convolution.

[0027] In some embodiments, the PKIS submodule includes multiple parallel depthwise separable convolutional branches, each of which uses a convolutional kernel of a different size, and the outputs of each branch are concatenated in the channel dimension and then fused by a 1×1 convolutional layer.

[0028] For example, there are five branches, with convolutional kernel sizes of 3×3, 5×5, 7×7, 9×9, and 11×11, respectively. The 3×3 branch focuses on preserving edge details and local textures of defects, the 5×5 and 7×7 branches balance local and contextual information, and the 9×9 and 11×11 branches focus on capturing global contextual information of large defects. Differential attention deployment strategies can be used on these five parallel, depthwise separable convolutional branches for receptive field expansion and feature fusion.

[0029] Furthermore, the plurality of parallel depthwise separable convolutional branches include at least one branch employing a large-size convolutional kernel, and each of the branches employing a large-size convolutional kernel is internally provided with a parameterless attention module SimAM, which is used to perform attention weighting on the feature maps within its branch.

[0030] Specifically, SimAM can assign high weights to defect neurons and low weights to background neurons, achieving parameter-free suppression of background noise in a large receptive field, and realizing secondary noise filtering in the large-size convolutional kernel of PKIS. The motivation for the design of PKIS stems from the extreme scale differences of defects on the steel surface. Small defects rely on high-resolution local features, while large defects require broader contextual semantic information and are more prone to introducing background noise effects.

[0031] The above embodiments input the redundancy suppression output of SCConv into PKIS to achieve multi-scale fusion, providing higher quality feature input. The core of this design lies in the deployment of redundancy suppression (SCConv) at cross-scale feature alignment nodes, ensuring that redundancy filtering occurs on the critical path of information fusion. Multi-scale fusion (PKIS) follows closely, utilizing the filtered features for efficient receptive field expansion, thereby achieving a comprehensive improvement in the model's feature representation and generalization performance.

[0032] In some embodiments, the SCConv submodule and the PKIS submodule are cascaded in a fixed order, and the output feature map of the SCConv submodule serves as the input feature map of the PKIS submodule. Specifically, the order of the two submodules in MSFR is not interchangeable; SCConv must be placed before the cross-scale feature alignment node to perform pre-redundancy filtering, and PKIS must be placed after to expand the receptive field in the filtered feature space.

[0033] If the order is reversed, PKIS will expand in a feature space containing redundancy, reducing the effectiveness of the expansion; if the pre-filtering of SCConv and the built-in filtering mechanism of SimAM are missing, PKIS will amplify redundant noise simultaneously. Therefore, the deployment position and order of the two have a non-random structural dependency. Embedding the proposed MSFR into the feature fusion stage can overcome the shortcomings of the original RepC3 module in RT-DETR, which has a single receptive field scale, is difficult to adapt to the multi-scale distribution of steel surface defects, and lacks a redundant feature filtering mechanism.

[0034] In some embodiments, the dynamic sampling module is used for dynamic sampling, and the dynamic sampling process includes: Obtaining the input feature map and upsampling scaling factor And generate a sample set through a generator. .

[0035] The input feature map is combined with a grid sampling function. and the sampling set The coordinate information in the map is used to generate the output feature map. .

[0036] Specifically, in the RT-DETR baseline, the upsampling block using nearest-neighbor interpolation logic is prone to blurring of small defect edges and loss of defect details during the upsampling process. To address this issue, this embodiment limits the deployment of the DySample module to the upsampling nodes of the second and third level feature maps in the CCFM feature fusion path, allowing it to directly act on the locations where small defect edges are most likely to be lost during multi-scale feature alignment. The remaining upsampling nodes retain their original structure to avoid introducing unnecessary sampling offsets and computational overhead into the semantic stabilization layer. This dynamic sampling process complements the pre-feature filtering of MSFR, while reducing the edge jaggedness, detail breakage, and small defect response dilution problems caused by nearest-neighbor interpolation. DySample reconstructs the upsampling process into a process of bilinear interpolation combined with content-aware dynamic point sampling, adaptively adjusting the sampling coordinates based on the features of the defect image, which not only improves the model's detail reconstruction capability but also further optimizes the model's detection accuracy.

[0037] like Figure 5 As shown, the input feature map The initial size is Output feature map The size is The specific generation process can be represented by formula (1): in, This is the grid sampling function.

[0038] Depend on Figure 6 It can be seen that the generator generates the sample set. There are two ways to generate it: static generation and dynamic generation.

[0039] Offset in static generation method Sampling set Output feature map The implementation process is shown in formulas (2) to (4): in, For linear projection; This is the original sampling grid.

[0040] In the dynamic generation method, a dynamic range factor is introduced before pixel rearrangement to improve the offset. The flexibility of the dynamic generation method is shown in formulas (5) and (6): in, For activation functions; and These are the linear projections of the dynamic range factor generation branch and the original sampling offset generation branch, respectively.

[0041] In this embodiment, DySample is used as a key component in the feature fusion and reconstruction mechanism of the MDG-DETR model. Based on the multi-scale and numerous small defects in steel surface defect detection, it is combined and adapted with the corresponding level of the feature fusion encoder. The roles of DySample and MSFR complement each other to further improve the overall detection rate of small target defects in complex backgrounds, thereby optimizing the model's overall detail reconstruction capability and comprehensive performance.

[0042] Furthermore, in steel defect detection tasks, there are defects of various scales and sparsely distributed micro-defects. The GIoU loss in the RT-DETR model can cause a sharp drop in IoU value even with a tiny positional deviation during localization, leading to missed detections. To address this issue, this embodiment designs an SAGL loss to replace the original GIoU loss. The SAGL loss first models the bounding box using a Gaussian distribution, assigning a higher weight to the center pixel of the bounding box and decreasing it radially towards the boundary. Based on this, a scale-aware weighting factor based on the target area is further introduced, so that the loss function applies a gradient intensity inversely proportional to the size of the defect to defects of different scales during optimization. This ensures the accuracy of large-scale defect localization while enhancing the supervision signal for micro-defects. (Gaussian distribution mean vector) Covariance Matrix As shown in formula (7): SAGL loss dynamically adjusts the gradient for defects at different scales during model training using a scale-aware weighting factor. That is, for defects with small areas, the weighting factor is greater than 1, which enhances the gradient gain of targets with small width and height while optimizing the center distance gradient, thereby strengthening the focus of the loss on small defects; for defects with large areas, the weighting factor approaches 1, degenerating into a standard Gaussian distance metric. Formula (8) is the definition of the scale-aware weighting factor.

[0043] in, The area of ​​the actual defect target. This is the maximum weighted amplitude control parameter. For scale control threshold, It is an exponential function. It should be noted that when... much smaller hour, Approaching The loss gradient is significantly amplified; when Much larger hour, Approaching 1, it degenerates into a standard Gaussian distance metric. The weighted SAGL distance can be expressed as... .

[0044] The gradient calculation formulas (9)-(11) for the scale-adaptive Gaussian localization loss function on the predicted bounding box parameters during training are as follows: in, , , and These are the center coordinates, width, and height of the predicted bounding box, respectively. , , and These are the center coordinates, width, and height of the actual bounding box, respectively. This represents the squared Gaussian combination distance between the Gaussian distributions corresponding to the two bounding boxes; and These are the mean vector and covariance matrix of the Gaussian distribution, respectively.

[0045] In the above embodiments, the SAGL loss achieves differentiated supervision of multi-scale defects through a scale-aware weighting factor, enabling small defects to obtain stronger gradient signals while maintaining stable constraint strength for large-scale defects. This design directly maps the defect scale to the gradient adjustment factor of the localization loss, allowing defects of different areas to obtain differentiated supervision strength during the same training process. Compared with fixed-weight IoU-type losses, the SAGL loss can improve the matching sensitivity of small-area defects without changing the detector head structure, while maintaining stable localization constraints for large-scale defects, thus compensating for the training bias caused by the uneven distribution of steel defect scales at the loss function level.

[0046] The following is a comparative experiment: like Figure 7 and Figure 8 As shown, the datasets used in this experiment are the GC10-DET dataset, which is based on real-world scene data and consists of 10 types of steel surface defects, containing 2294 high-resolution images of steel surface defects at 2048×1000 pixels. The NEU-DET dataset, provided by Northeastern University, contains 1800 grayscale defect images at 200×200 pixels, covering six types of strip steel surface defects. Both datasets were randomly divided into training, validation, and test sets in an 8:1:1 ratio.

[0047] Experimental Setup: The experimental server was equipped with an NVIDIA RTX 4070 GPU with 12GB of VRAM, an i5-12400F CPU, and 16GB of DDR5 RAM. The software used was Python 3.8, CUDA 12.8, and PyTorch 2.0.1. Images were resized to 640×640 pixels during model input. During training, a batch size of 16 was set on the GC10-DET dataset for 200 epochs; a batch size of 8 was set on the NEU-DET dataset for 300 epochs.

[0048] Experimental Procedure: Performance comparisons of the baseline model and the improved method were conducted on NEU-DET and GC10-DET, using ablation experiments to demonstrate the effectiveness of the method. Subsequently, comparative experiments were designed to comprehensively compare the MDG-DETR model with mainstream object detection models such as Faster R-CNN, and YOLO series models such as YOLOv5m, YOLOv8m, YOLOv10m, YOLOv11m, and YOLO26m to evaluate the overall performance advantage.

[0049] Evaluation metrics: To comprehensively reflect the performance of this model in steel surface defects, this experiment uses mean precision (mAP), precision (P), recall (R), floating-point operations (FLOPS), frame rate (FPS), and model size (M) as evaluation metrics. mAP is calculated using an intersection-union ratio (IUU) threshold of 0.5, referred to as mAP50. The formulas for the relevant metrics are shown in formulas (12)-(15): Experimental results show that the MDG-DETR model outperforms the baseline method RT-DETR in both precision (P) and recall (R). Simultaneously, mAP50 and mAP50:95 are improved, demonstrating superior detection performance across multiple dimensions. Param and GFLOPs also show that the MDG-DETR model offers significant improvements in parameters and computational cost compared to the original baseline model. In comparative experiments across different categories, the MDG-DETR model also demonstrates superior accuracy compared to other comparable models such as Faster R-CNN, YOLOv5m, YOLOv8m, YOLOv10m, YOLOv11m, and YOLO26m, achieving optimal detection results in some categories. Therefore, the MDG-DETR model achieves high-precision detection of multi-scale steel defects in complex defect scenarios.

[0050] Some experimental results show that the mAP50 of the MDG-DETR model reached 71.8% (+5.7%) and 76.2% (+4.2%), which is a significant improvement compared to the baseline RT-DETR. At the same time, the number of parameters and the computational cost also decreased by 6.4% and 11.6%, respectively.

[0051] In summary, this application proposes an MDG-DETR model based on RT-DETR, aiming to balance detection accuracy, model lightweighting, and real-time inference. First, an MSFR module integrating feature filtering and multi-scale feature fusion is designed, enhancing the model's ability to detect multi-scale defects while reducing computational cost. Then, a DySample module is introduced, replacing nearest-neighbor interpolation with content-aware dynamic sampling and selectively deploying it at the upsampling nodes most prone to losing details in the feature fusion path, achieving accurate reconstruction of small defect boundaries and further enhancing the model's detection capability. Finally, a scale-adaptive Gaussian localization loss (SAGL) is proposed to optimize the model's ability to identify small target defects. In the synergy of these three modules, the MSFR module plays a fundamental supporting role, DySample refines the feature representation in the upsampling stage, and SAGL strengthens the supervision of small defects at the loss function level, collectively achieving high-precision detection of multi-scale steel defects.

[0052] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0053] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a multi-scale steel surface defect detection method based on RT-DETR improvement.

[0054] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0055] In one embodiment, a computer-readable storage medium is provided, on which a program or instructions are stored, which, when executed by a processor, implement the steps in the above-described method embodiments.

[0056] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0057] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0058] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0059] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A multi-scale steel surface defect detection method based on RT-DETR improvement, characterized in that, The improved multi-scale steel surface defect detection method based on RT-DETR includes: Acquire images of the steel surface to be inspected; The image is input into an improved RT-DETR model for defect detection; In this improved RT-DETR model, the feature fusion module is replaced by a multi-scale fusion and reconstruction module. The multi-scale fusion and reconstruction module includes sequentially cascaded SCConv and PKIS sub-modules, with the output of the SCConv sub-module connected to the input of the PKIS sub-module. In the upsampling module of the improved RT-DETR model, dynamic sampling modules are deployed at the upsampling nodes located at the output ends of the second-level feature map and the third-level feature map, while the nearest neighbor interpolation upsampling structure is retained at the remaining upsampling nodes. During the training process of the improved RT-DETR model, a scale-adaptive Gaussian localization loss function is used as the supervision signal for bounding box regression. The scale-adaptive Gaussian localization loss function models the predicted bounding box and the real bounding box as a two-dimensional Gaussian distribution, and introduces a scale-aware weighting factor based on the area of ​​the defective target to weight the Gaussian distance. The weighting factor has an exponential decay relationship with the area of ​​the defective target.

2. The multi-scale steel surface defect detection method based on RT-DETR improvement according to claim 1, characterized in that, The SCConv submodule is used to perform redundancy suppression in the channel dimension and the spatial dimension respectively, and to combine the suppression results in the channel dimension with the suppression results in the spatial dimension to perform feature rearrangement. The PKIS submodule is used to perform multi-scale depthwise separable convolution operations in the feature space after redundancy suppression to expand the receptive field.

3. The multi-scale steel surface defect detection method based on RT-DETR improvement according to claim 1, characterized in that, The SCConv submodule and the PKIS submodule are cascaded in a fixed order, and the output feature map of the SCConv submodule is used as the input feature map of the PKIS submodule.

4. The multi-scale steel surface defect detection method based on RT-DETR improvement according to claim 1, characterized in that, The PKIS submodule includes multiple parallel depthwise separable convolutional branches, each of which uses a convolutional kernel of a different size. The outputs of each branch are concatenated in the channel dimension and then fused by a 1×1 convolutional layer.

5. The multi-scale steel surface defect detection method based on RT-DETR improvement according to claim 4, characterized in that, The plurality of parallel depthwise separable convolutional branches include at least one branch with a large-size convolutional kernel. Each branch with a large-size convolutional kernel has a parameterless attention module SimAM inside, which is used to perform attention weighting on the feature maps within its branch.

6. The multi-scale steel surface defect detection method based on RT-DETR improvement according to claim 1, characterized in that, The dynamic sampling module is used for dynamic sampling, and the dynamic sampling process includes: Obtaining the input feature map and upsampling scaling factor And generate a sample set through a generator. ; The input feature map is combined with a grid sampling function. and the sampling set The coordinate information in the map is used to generate the output feature map. .

7. The multi-scale steel surface defect detection method based on RT-DETR improvement according to claim 1, characterized in that, The scale-aware weighting factor Represented as: in, The area of ​​the actual defect target. This is the maximum weighted amplitude control parameter. For scale control threshold, It is an exponential function.

8. The multi-scale steel surface defect detection method based on RT-DETR improvement according to claim 1, characterized in that, The gradient calculation formula for the predicted bounding box parameters during the training process of the scale-adaptive Gaussian localization loss function is as follows: in, , , and These are the center coordinates, width, and height of the predicted bounding box, respectively. , , and These are the center coordinates, width, and height of the actual bounding box, respectively. This represents the squared Gaussian combination distance between the Gaussian distributions corresponding to the two bounding boxes; and These are the mean vector and covariance matrix of the Gaussian distribution, respectively.

9. A computer device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the multi-scale steel surface defect detection method based on RT-DETR as described in any one of claims 1-8.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the multi-scale steel surface defect detection method based on RT-DETR as described in any one of claims 1-8.

Images