Methods, apparatus, computer equipment and storage media for detecting defects on steel surfaces

Abstract

This application relates to methods, devices, computer equipment, and storage media for detecting defects on steel surfaces. The proposed method uses a lightweight RT-DETR model, overcoming performance limitations through a bidirectional optimization strategy: First, ghost convolutions replace the original convolutional layers, generating phantom feature maps through linear transformation of feature map channels, achieving simultaneous reduction in computational load and parameter quantity. Simultaneously, a CBAM reconstructed feature fusion network is embedded, significantly enhancing the separability of micro-scale defects such as rust and scratches in complex background noise through adaptive channel weight calibration and spatial pixel focusing dual modulation, ultimately achieving optimized detection efficiency and accuracy. This method addresses the triple challenges of parameter redundancy, missed edge defects, and insufficient real-time performance in complex steel texture scenarios using the traditional Transformer architecture, providing a next-generation solution for high-precision industrial quality inspection systems.

Description

Technical Field

[0001] This invention belongs to the field of steel testing technology, and relates to a method, device, computer equipment and storage medium for detecting defects on the surface of steel. Background Technology

[0002] With the increasing demands for steel quality in the industrial sector, the detection of surface defects in steel has become a core element in ensuring product quality and reducing production costs. However, steel production often generates diverse and complex types of defects due to equipment and on-site environmental limitations, posing a significant challenge to achieving efficient and accurate defect identification. Traditional defect identification methods, such as manual inspection and early machine vision techniques, have significant shortcomings in terms of detection speed, cost control, and identification accuracy, making it difficult to meet the stringent requirements of modern large-scale mechanized production.

[0003] With the widespread adoption and advancement of CCD imaging equipment, the field of steel surface defect detection has undergone a systematic transformation towards intelligent machine vision methods. Key research achievements include: local binary feature modeling, noise-resistant hypersphere support vector mechanisms, singular value deconstruction frameworks, and multi-kernel correlation vector machine architectures. These innovative solutions have significantly improved the efficiency and accuracy of defect detection, systematically replacing outdated technologies such as traditional manual inspection. Despite substantial progress, existing methods still have room for improvement in terms of detection accuracy.

[0004] Revolutionary breakthroughs in deep learning have driven a paradigm shift in the application of Convolutional Neural Networks (CNNs) in industrial defect detection. This technology relies on a multi-level convolutional-pooling architecture to extract local image features, which are then fused through fully connected layers to generate high-order semantic representations. This achieves end-to-end loss mapping from feature maps to target detection, comprehensively optimizing recognition accuracy and timeliness. Typical technological evolutions include: constructing a residual feature pyramid based on YOLOv5 and integrating a multi-head attention mechanism to achieve sub-second localization of steel plate defects; embedding heterogeneous convolutional kernel groups into the YOLOv7 backbone, coupling coordinate attention (CA) and spatial pyramid decomposition (SPD) modules at the output, significantly enhancing the ability to capture micro-defects in complex textures; and modifying the YOLOv8 architecture, introducing a parameter-free attention feature weighting mechanism and reconstructing the spatial pyramid pooling (SPPF) module, overcoming the challenge of missing small targets by expanding the receptive field.

[0005] Currently, the Transformer architecture, with its superior modeling capabilities in natural language processing and computer vision, has given rise to a new generation of object detection paradigms. Typical examples include DETR (Detection Transformer) and its real-time detection variant RT-DETR, which establish global dependency modeling through a self-attention mechanism, significantly improving object recognition performance in complex scenes.

[0006] The DETR model is limited by its large number of parameters and complex computational schema, which significantly restricts the real-time performance of inference, especially in resource-constrained scenarios such as edge computing. Its Transformer module requires extremely long training cycles and high-end hardware support, making it difficult to adapt to conventional industrial-grade deployment environments. In addition, under conditions of diverse defect features and complex backgrounds, this architecture has inherent limitations such as high parameter sensitivity, insufficient model robustness, and weak cross-scenario generalization ability.

[0007] To overcome the aforementioned technical bottlenecks, researchers have developed several enhanced RT-DETR architectures. A typical example is Zhou's innovative approach: First, a lightweight FasterNet-T1 is used to reconstruct the detector backbone network, significantly reducing computational complexity; then, an attention embedding extended residual fusion (AERF) module is designed to improve the capture capability of deep microscopic targets; simultaneously, a cross-scale information selection feature pyramid is constructed to solve the feature degradation problem in hierarchical transmission; finally, a composite loss strategy that fuses normalized Wasserstein distance and improved Inner-MPDIoU is proposed, which significantly improves the accuracy of microscopic defect identification and cross-scene generalization performance while accelerating model convergence.

[0008] Despite the progress made by RT-DETR in the field of steel surface defect detection, it is still limited by three core constraints: high computational complexity, insufficient lightweight design, and insufficient ability to represent features in complex texture scenes. Summary of the Invention

[0009] To address the problems existing in the above-mentioned traditional methods, this invention proposes a method, apparatus, computer equipment, and storage medium for detecting defects on steel surfaces.

[0010] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0011] On the one hand, a method for detecting defects on the surface of steel is provided, the method comprising:

[0012] A backbone network is used to extract multi-level features from the acquired grayscale images of the steel surface, resulting in high-resolution low-semantic features, mid-level features, and high-level semantic features.

[0013] The high-resolution low-semantic features, the intermediate features, and the high-level semantic features are input into the Neck module to obtain three intermediate features. The Neck module includes three branches: the first and second branches are used to process the high-resolution low-semantic features and the intermediate features through convolutional layers, respectively; and the third branch is used to process the high-level semantic features using convolution and AIFI modules.

[0014] Three intermediate features are input into the feature encoding module to obtain three encoded features at different scales; the feature encoding module is used to encode the features at three different scales using the GhostConv module, the CBAM module, and the RepC3 feature reuse structure.

[0015] The encoded features at three different scales are processed using an IoU-aware query mechanism to obtain the initial target query features.

[0016] The initial target query features are decoded using a Transformer decoder with an auxiliary prediction head to obtain the detection results of steel surface defects.

[0017] On the other hand, a steel surface defect detection device is also provided, the device comprising:

[0018] The feature extraction unit is used to perform multi-level feature extraction on the acquired grayscale image of the steel surface using a backbone network, so as to obtain high-resolution low semantic features, mid-level features and high-level semantic features.

[0019] An intermediate feature extraction unit is used to input the high-resolution low-semantic features, the mid-level features, and the high-level semantic features into the Neck module to obtain three intermediate features. The Neck module includes three branches: the first and second branches are used to process the high-resolution low-semantic features and the mid-level features through convolutional layers, respectively; and the third branch is used to process the high-level semantic features using convolution and AIFI modules.

[0020] The encoding unit is used to input three intermediate features into the feature encoding module to obtain three encoded features at different scales; the feature encoding module is used to encode the features at three different scales using the GhostConv module, the CBAM module, and the RepC3 feature reuse structure.

[0021] The initial query vector determination unit is used to process the encoded features of three different scales using an IoU-aware query mechanism to obtain the initial target query features;

[0022] The steel surface defect identification unit is used to decode the initial target query features using a Transformer decoder with an auxiliary prediction head to obtain the detection results of steel surface defects.

[0023] In another aspect, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the above-described steel surface defect detection methods.

[0024] Furthermore, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of any of the above-described steel surface defect detection methods.

[0025] One of the above technical solutions has the following advantages and beneficial effects:

[0026] The aforementioned steel surface defect detection methods, devices, computer equipment, and storage media are all discussed. This method proposes a lightweight RT-DETR model, overcoming performance limitations through a bidirectional optimization strategy: First, ghost convolution replaces the original convolutional layers, generating phantom feature maps through linear transformation of feature map channels, achieving simultaneous reduction in computational load and parameter quantity. Simultaneously, a channel-spatial attention mechanism (CBAM) is implanted to reconstruct the feature fusion network. Through adaptive channel weight calibration and spatial pixel focusing, the separability of micro-scale defects such as rust and scratches in complex background noise is significantly enhanced, ultimately achieving optimized detection efficiency and accuracy. This method addresses the triple challenges of parameter redundancy, missed edge defects, and insufficient real-time performance in complex steel texture scenarios using the traditional Transformer architecture, providing a next-generation solution for high-precision industrial quality inspection systems. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating a method for detecting defects on the surface of steel in one embodiment;

[0029] Figure 2 This is a general framework diagram of a steel surface defect detection model in one embodiment;

[0030] Figure 3 This is a schematic diagram of the CBAM module framework in one embodiment;

[0031] Figure 4 This is a schematic diagram of the GhostConv module framework in one embodiment. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0034] It should be noted that, in this document, the reference to "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The presentation of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will understand that the embodiments described herein can be combined with other embodiments. The term "and / or" as used herein refers to any combination of one or more of the associated listed items, and all possible combinations, including such combinations.

[0035] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0036] In one embodiment, such as Figure 1 As shown, a method for detecting defects on the surface of steel is provided, which may include the following processing steps 100 to 104:

[0037] Step 100: Use a backbone network to extract multi-level features from the acquired grayscale image of the steel surface, and obtain high-resolution low semantic features, mid-level features and high-level semantic features.

[0038] Step 101: Input the high-resolution low-semantic features, mid-level features, and high-level semantic features into the Neck module to obtain three intermediate features; the Neck module includes three branches. The first and second branches are used to process the high-resolution low-semantic features and mid-level features through convolutional layers, respectively, and the third branch is used to process the high-level semantic features using convolution and AIFI modules.

[0039] Step 102: Input the three intermediate features into the feature encoding module to obtain three encoded features at different scales; the feature encoding module is used to encode the features at three different scales using the GhostConv module, CBAM module and RepC3 feature reuse structure.

[0040] Specifically, this application innovatively proposes a lightweight RT-DETR model to address the industrial-grade needs of surface defect detection in the steel metallurgy field. By integrating Ghost convolutional compression computational load, CBAM two-dimensional attention-enhanced defect feature response, and a multi-feature fusion architecture, a detection paradigm that combines high efficiency and accuracy is constructed.

[0041] Ghost convolution is used to replace the original convolutional layers. Phantom feature maps are generated through linear transformation of feature map channels, achieving simultaneous reduction in computational load and parameter quantity. At the same time, channel-space attention mechanism (CBAM) is implanted to reconstruct the feature fusion network. Through dual modulation of adaptive channel weight calibration and spatial pixel focusing, the separability of micro-scale defects such as rust and scratches in complex background noise is significantly enhanced, ultimately achieving optimization of detection efficiency and accuracy.

[0042] Step 103: Process the encoded features at three different scales using an IoU-aware query mechanism to obtain the initial target query features.

[0043] Step 104: Decode the initial target query features using a Transformer decoder with an auxiliary prediction head to obtain the detection results of steel surface defects.

[0044] The aforementioned backbone network, Neck module, feature encoding module, IoU-aware query mechanism, and Transformer decoder with auxiliary prediction head constitute the steel surface defect detection model, employing an end-to-end deep learning solution. The overall framework of the steel surface defect detection model is as follows: Figure 2 As shown.

[0045] The process begins with a grayscale image of the steel surface input on the left. Multi-level feature extraction is performed via the ResNet-50 backbone network (yellow area), generating a three-level feature pyramid: high-resolution low-semantic features S5, mid-level features S4, and high-level semantic features S3. The core innovation layer employs a modular coding design: embedding Ghost lightweight convolutions to reduce the number of parameters, and deploying RepC3 feature reuse structures to enhance the continuity of defect extraction. A CBAM module is embedded, using a two-dimensional attention mechanism to automatically enhance the response intensity of crack regions through channel weighting and spatial focusing. Multi-scale features are fused across layers via the topological path indicated by the black arrow. After bilinear upsampling to restore detail resolution, crack location is resolved by decoding on the right. An IoU-aware query mechanism is introduced to dynamically correct bounding box offsets, ultimately outputting crack localization results with accurate detection boxes. The entire architecture achieves industrial-grade accuracy for identifying cracks on steel surfaces while maintaining real-time processing capabilities.

[0046] The aforementioned steel surface defect detection methods propose a lightweight RT-DETR model, which overcomes performance limitations through a bidirectional optimization strategy: First, ghost convolutions replace the original convolutional layers, generating phantom feature maps through linear transformation of feature map channels, achieving simultaneous reduction in computational load and parameter quantity; simultaneously, a CBAM reconstructed feature fusion network is implanted, significantly enhancing the separability of micro-scale defects such as rust and scratches in complex background noise through dual modulation of adaptive channel weight calibration and spatial pixel focusing, ultimately achieving optimized detection efficiency and accuracy. This method addresses the triple challenges of parameter redundancy, missed edge defects, and insufficient real-time performance in complex steel texture scenarios using the traditional Transformer architecture, providing a new generation solution for high-precision industrial quality inspection systems.

[0047] In one embodiment, the backbone network in step 100 is a ResNet-50 backbone network.

[0048] In one embodiment, the first and second branches of the Neck module each include one convolutional layer; the third branch of the Neck module includes a convolutional layer and an AIFI module; step 101 includes: inputting high-resolution low-semantic features into the first branch of the Neck module to obtain the first intermediate feature; inputting mid-level features into the second branch of the Neck module to obtain the second intermediate feature; and inputting high-level semantic features into the third branch of the Neck module to obtain the third intermediate feature.

[0049] In one embodiment, the convolutional layers in the three branches of the Neck module are all convolutional layers with a kernel of 1×1.

[0050] In one embodiment, the feature encoding module includes: four GhostConv modules, four RepC3 feature reuse structures, and four CBAM modules; step 102 includes: processing the third intermediate feature through the first GhostConv module to obtain the first convolutional feature; upsampling the first convolutional feature and concatenating it with the second intermediate feature, then processing it through the first CBAM module, the first RepC3 feature reuse structure, and the second GhostConv module, followed by upsampling; concatenating the obtained upsampling result with the first intermediate feature, then processing it through the second CBAM module and the second RepC3 feature reuse structure to obtain the first encoded feature; processing the first encoded feature through the third GhostConv module and concatenating it with the first convolutional feature, then processing the concatenated result through the third CBAM module and inputting it into the third RepC3 feature reuse structure to obtain the second encoded feature; processing the second encoded feature through the fourth GhostConv module, the concatenation operation, the fourth CBAM module, and the fourth RepC3 feature reuse structure to obtain the third encoded feature.

[0051] Specifically, Figure 3 This clearly demonstrates the core working mechanism of the Convolutional Block Attention Module (CBAM). The entire process begins with the input features on the left, which represent the raw image information extracted by the current layer of the network. Next, the feature maps enter a Channel Attention Module. The core task of this module is to calculate the feature attention points of each channel in the input feature map. Attention analysis is performed on the information content carried by each channel, generating a one-dimensional channel attention weight vector, where higher values ​​indicate that the channel carries more critical information. Subsequently, these calculated weights are multiplied channel-by-channel with the original input feature map. This effectively enhances the prominence of important information channels and reduces the detail extraction of secondary or irrelevant channels, thus performing the first adaptive refinement of the input features in the channel dimension, highlighting the role of key feature channels.

[0052] After channel-level optimization, the results then flow into the Spatial Attention Module. This module focuses on the spatial dimension (Width x Height) of the feature map, aiming to determine the importance of each spatial location (pixel region) on the feature map. It integrates information from all channels to generate a two-dimensional spatial attention weight map. This set of spatial weights is then multiplied positionally with the result from the previous step. This multiplication operation spatially weights the feature map, significantly strengthening the responses of regions containing key objects or defects while suppressing the influence of background or irrelevant noise regions. Through this attention approach, CBAM allows the network to continuously and self-learningly concentrate computational resources on the most informative feature channels and image regions during feature extraction, thereby significantly improving the model's ability to capture key information (such as minute defects on the steel surface) and its final discrimination accuracy.

[0053] In one embodiment, in the GhostConv module: a mapping operation is performed on each channel feature of the input GhostConv module to obtain the corresponding Ghost feature map; wherein the first mapping operation is an identity mapping, and the remaining mappings are lightweight transformations based on depthwise separable convolution; the features of the input GhostConv module and the Ghost feature map are concatenated along the channel dimension to obtain the output features of the GhostConv module.

[0054] Specifically, such as Figure 4 As shown, the entire feature extraction process of the GhostConv module begins with the Input layer (labeled on the left), which is a multidimensional data tensor input to the model, typically representing the original image or its initial feature representation. The input data immediately flows into the adjacent Conv layer for basic feature extraction. This convolutional layer acts like a filter, transforming and extracting the spatial dimensions of the input data, implying that a more semantically meaningful primary feature map is formed after the convolution operation.

[0055] Subsequently, the core of the model is that the features output from the convolutional layers do not flow in a single direction, but are instead distributed in parallel to multiple feature processing paths. These paths are arranged horizontally in the diagram, with their number varying, and each path is labeled with a unique symbol. .These The symbols represent independent, customizable feature transformation parameters for each branch. This multi-path design gives the model great flexibility, meaning that each branch can use different computational strategies to process the feature flow. These differentiated processing paths enable the model to simultaneously learn and fuse rich feature information from different receptive fields and different levels of abstraction, while reducing a large number of convolution parameters. This reduction in parameters greatly enhances its ability to represent complex patterns.

[0056] It is worth noting that, in Figure 4 At the top, there exists an Identity path spanning multiple processing units. This means that the original input features are directly bypassed to deeper layers without any transformation, a key mechanism for achieving residual connections. During model training, this connection effectively alleviates the vanishing and exploding gradient problems common in deep networks, ensuring that gradient information can be stably backpropagated even in very deep networks, greatly improving the trainability and stability of deep networks. Ultimately, all gradients passing through their respective branches ( arrive The transformed feature stream, along with the untransformed Identity feature stream, converges into the Output layer on the right. This convergence can be achieved through simple channel concatenation. The output layer is displayed as a stacked block structure with a red-orange gradient, visually representing the composite form of the fused feature data in terms of spatial dimensions (width and height) and feature depth (number of channels), forming the final input for subsequent network layers and providing a robust feature foundation. The entire structure is concise, efficient, and requires very few parameters, profoundly embodying the essence of multi-path fusion and residual learning in modern CNN models.

[0057] In some implementations, experimental examples are also provided, using the NEU-DET dataset to verify the effectiveness of the proposed method. The NEU-DET dataset, released by Northeastern University, covers six typical surface defects in steel production, containing 1800 grayscale sample images. This dataset accurately labels the morphologies of defects such as cracks (Cr), inclusions (In), patches (Pa), pitted surfaces (Ps), entanglement scales (Rs), and scratches (Sc). Each image provides pixel-level bounding box coordinates and category labels. All data was collected at a uniform resolution of 200×200 and divided into a training set (1440 samples), a validation set (180 samples), and a test set (180 samples) in an 8:1:1 ratio. Table 1 shows the accuracy of each category on the NEU-DET dataset, and Table 2 shows the overall accuracy of the model on the NEU-DET dataset.

[0058] Table 1. Accuracy of each category in the NEU-DET dataset.

[0059]

[0060] Table 2 shows the overall accuracy of the model on the NEU-DET dataset.

[0061]

[0062] This experiment fully presents the model's training results on a steel surface defect dataset. The number of parameters reaches 42.37M, and the computational cost is 128.3 GFLOPs. Performance evaluation covers six typical industrial defects (cracking, inclusions, patches, pitted surfaces, entangled rulers, and scratches). The detection model exhibits polarized performance characteristics in identifying these six typical steel surface defects: it demonstrates excellent detection accuracy (overall precision P=0.915) and good localization ability (mAP50=0.81) at the global level, especially achieving superior performance in pitted surfaces (Ps) and scratches (Sc). Ps achieves a high level of recognition with 0.98 precision, 0.875 recall, and 0.971 mAP50, while Sc closely follows with 0.98 precision, 0.857 recall, and 0.953 mAP50. While Pa has a slightly weaker recall (0.805), it still maintains an exceptionally high mAP50 value of 0.945. Inclusions (In) exhibit a balanced performance with a precision of 0.779 and a recall of 0.830. However, the model suffers from significant bottlenecks: Rs becomes the weakest link (P=0.75, R=0.556, mAP50=0.675), and Cr exposes even more severe defects. The contradiction between its high recall (0.8) and extremely low localization precision (mAP50=0.579) reveals that although the model can detect the presence of Cr, it struggles to accurately define its complex spread patterns. The insufficient fine-grained detection capability (mAP50-95 is only 0.44) further highlights the problem, indicating that the model's adaptability to boundary-sensitive defects (such as crack textures and fine lines) urgently needs improvement. Overall, this solution demonstrates industrial-grade maturity for detecting regular defects such as pits and scratches, but further optimization using morphological learning mechanisms is needed for irregular, low-contrast microscopic defects. In terms of inference speed, single-image processing takes 46ms, meeting the requirements for industrial real-time detection. Overall results show that the model maintains high detection accuracy while possessing industrial-grade real-time processing capabilities.

[0063] In the comprehensive performance comparison of six types of defects on steel surfaces, our method (Ours) leads the benchmark model with a global mAP of 0.81, which is 2.3 percentage points higher than the closest model, YOLOv10n, with an mAP of 0.792. Specifically, in pitted surface detection, it achieves an mAP of 0.97150, which is better than YOLOv7's 0.965. The scrapes recognition performance reaches 0.953, which is a significant improvement over Fast R-CNN's 0.950. The patches detection achieves a record-breaking 0.945, surpassing YOLOv7's 0.940. For the detection bottleneck crazing, although it surpasses DETR's recognition accuracy (mAP=0.268) and YOLOv7's (mAP=0.318) with an accuracy of 0.579, it is still slightly inferior to SSD (mAP=0.627). In challenging tasks such as rolled-in scale, this method achieves the highest mAP50 for this category with 0.675 (YoLOv8n is 0.668). Notably, this scheme ranks first in inclusion detection, tied with FastRCNN (mAP=0.78), and overcomes the inherent limitation of DETR's performance drop-off on complex texture defects (such as Cr). Overall, this scheme maintains high generalization (balanced improvement across all six defect categories) while exhibiting a significant advantage in recognizing defects with obvious surface geometric features. The experimental results of advanced models are compared in Table 3.

[0064] Table 3 Comparison of Experimental Results of Advanced Models

[0065]

[0066] In the NEU-DET benchmark dataset validation, the model achieved a real-time performance of 46 FPS with 42.37M parameters, and its global detection accuracy (mAP50=0.81) significantly outperformed mainstream models (such as YOLOv10n's 0.792). Key technological breakthroughs are reflected in: high computational complexity, insufficient lightweight design, and inadequate feature representation capabilities for complex texture scenes. Experiments demonstrate that this solution successfully addresses the triple challenges of parameter redundancy, missed edge defects, and insufficient real-time performance in complex steel texture scenes using the traditional Transformer architecture, providing a next-generation solution for high-precision industrial quality inspection systems.

[0067] It should be understood that, although the above process Figure 1 The steps in the diagram are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed; they can be performed in other orders. Furthermore, the above process... Figure 1At least some of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0068] In one embodiment, a steel surface defect detection device is also provided, the device comprising:

[0069] The feature extraction unit is used to perform multi-level feature extraction on the acquired grayscale image of the steel surface using a backbone network, so as to obtain high-resolution low semantic features, mid-level features and high-level semantic features.

[0070] The intermediate feature extraction unit is used to input high-resolution low-semantic features, mid-level features, and high-level semantic features into the Neck module to obtain three intermediate features. The Neck module includes three branches. The first and second branches are used to process the high-resolution low-semantic features and mid-level features through convolutional layers, respectively. The third branch is used to process the high-level semantic features using convolution and AIFI modules.

[0071] The encoding unit is used to input three intermediate features into the feature encoding module to obtain three encoded features at different scales; the feature encoding module is used to encode the features at three different scales using the GhostConv module, the CBAM module, and the RepC3 feature reuse structure.

[0072] The initial query vector determination unit is used to process the encoded features of three different scales using an IoU-aware query mechanism to obtain the initial target query features;

[0073] The steel surface defect identification unit is used to decode the initial target query features using a Transformer decoder with an auxiliary prediction head to obtain the detection results of steel surface defects.

[0074] In one embodiment, the backbone network in the feature extraction unit is a ResNet-50 backbone network.

[0075] In one embodiment, the first and second branches of the Neck module each include one convolutional layer; the third branch of the Neck module includes a convolutional layer and an AIFI module; the intermediate feature extraction unit is further used to input high-resolution low semantic features into the first branch of the Neck module to obtain the first intermediate feature; input mid-level features into the second branch of the Neck module to obtain the second intermediate feature; and input high-level semantic features into the third branch of the Neck module to obtain the third intermediate feature.

[0076] In one embodiment, the convolutional layers in the three branches of the Neck module are all convolutional layers with a kernel of 1×1.

[0077] In one embodiment, the feature encoding module includes: four GhostConv modules, four RepC3 feature reuse structures, and four CBAM modules; the encoding unit is further configured to: process the third intermediate feature through the first GhostConv module to obtain a first convolutional feature; upsample the first convolutional feature and concatenate it with the second intermediate feature, then process it through the first CBAM module, the first RepC3 feature reuse structure, and the second GhostConv module, and then perform upsampling; concatenate the obtained upsampled result with the first intermediate feature, then process it through the second CBAM module and the second RepC3 feature reuse structure to obtain a first encoded feature; process the first encoded feature through the third GhostConv module and then concatenate it with the first convolutional feature; process the concatenated result through the third CBAM module and then input it into the third RepC3 feature reuse structure to obtain a second encoded feature; process the second encoded feature through the fourth GhostConv module, the concatenation operation, the fourth CBAM module, and the fourth RepC3 feature reuse structure to obtain a third encoded feature.

[0078] In one embodiment, in the GhostConv module: a mapping operation is performed on each channel feature of the input GhostConv module to obtain the corresponding Ghost feature map; wherein the first mapping operation is an identity mapping, and the remaining mappings are lightweight transformations based on depthwise separable convolution; the features of the input GhostConv module and the Ghost feature map are concatenated along the channel dimension to obtain the output features of the GhostConv module.

[0079] It is understood that for detailed explanations of the steel surface defect detection device, please refer to the corresponding explanations of the various embodiments of the steel surface defect detection method above, and will not be repeated here. Each module in the above-mentioned steel surface defect detection device can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in hardware or independently of a device with data processing capabilities, or stored in software in the memory of the aforementioned device, so that the processor can call and execute the operations corresponding to each module. The aforementioned device can be, but is not limited to, various types of data processing computer devices already existing in the art.

[0080] In one embodiment, a computer device is also provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described embodiment of the steel surface defect detection method.

[0081] It is understood that, in addition to the memory and processor mentioned above, the computer equipment described above also includes other hardware and software components not listed in this specification. The specific components can be determined according to the model of the image processing computer in different application scenarios, and will not be listed and described in detail in this specification.

[0082] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps in the above-described embodiment of the steel surface defect detection method.

[0083] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus DRAM (RDRAM), and interface DRAM (DRDRAM), etc.

[0084] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0085] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and all such modifications and improvements fall within the scope of protection of this application.

Claims

1. A method for detecting defects on the surface of steel, characterized in that, Including the following steps: A backbone network is used to extract multi-level features from the acquired grayscale images of steel surfaces, resulting in high-resolution low-semantic features, mid-level features, and high-level semantic features. The high-resolution low-semantic features, the intermediate features, and the high-level semantic features are input into the Neck module to obtain three intermediate features. The Neck module includes three branches: the first and second branches are used to process the high-resolution low-semantic features and the intermediate features through convolutional layers, respectively; and the third branch is used to process the high-level semantic features using convolution and AIFI modules. Three intermediate features are input into the feature encoding module to obtain three encoded features at different scales; the feature encoding module is used to encode the features at three different scales using the GhostConv module, the CBAM module, and the RepC3 feature reuse structure; The encoded features at three different scales are processed using an IoU-aware query mechanism to obtain the initial target query features; The initial target query features are decoded using a Transformer decoder with an auxiliary prediction head to obtain the detection results of steel surface defects.

2. The method for detecting defects on steel surfaces according to claim 1, characterized in that, The backbone network is the ResNet-50 backbone network.

3. The method for detecting defects on steel surfaces according to claim 1, characterized in that, The first and second branches of the Neck module each include one convolutional layer; the third branch of the Neck module includes a convolutional layer and an AIFI module. The high-resolution low-semantic features, the mid-level features, and the high-level semantic features are input into the Neck module to obtain three intermediate features, including: The high-resolution low-semantic features are input into the first branch of the Neck module to obtain the first intermediate feature; The intermediate feature is input into the second branch of the Neck module to obtain the second intermediate feature; The high-level semantic features are input into the third branch of the Neck module to obtain the third intermediate feature.

4. The method for detecting defects on steel surfaces according to claim 3, characterized in that, The convolutional layers in the three branches of the Neck module are all convolutional layers with a kernel of 1×1.

5. The method for detecting defects on steel surfaces according to claim 1, characterized in that, The feature encoding module includes: four GhostConv modules, four RepC3 feature reuse structures, and four CBAM modules; The three intermediate features are input into the feature encoding module to obtain three encoded features at different scales, including: The third intermediate feature is processed by the first GhostConv module to obtain the first convolutional feature; After upsampling the first convolutional feature and concatenating it with the second intermediate feature, the result is processed by the first CBAM module, the first RepC3 feature reuse structure, and the second GhostConv module, and then upsampled. The resulting upsampled feature is then concatenated with the first intermediate feature and processed by the second CBAM module and the second RepC3 feature reuse structure to obtain the first encoded feature. The first encoded feature is processed by the third GhostConv module and then concatenated with the first convolutional feature. The concatenation result is processed by the third CBAM module and then input into the third RepC3 feature reuse structure to obtain the second encoded feature. The second encoded feature is processed through the fourth GhostConv module, concatenation operation, fourth CBAM module, and fourth RepC3 feature reuse structure to obtain the third encoded feature.

6. The method for detecting defects on steel surfaces according to claim 1, characterized in that, In the GhostConv module: each channel feature of the input GhostConv module is mapped to obtain the corresponding Ghost feature map; the first mapping operation is an identity mapping, and the remaining mappings are lightweight transformations based on depthwise separable convolution; The features of the input GhostConv module are concatenated with the Ghost feature map along the channel dimension to obtain the output features of the GhostConv module.

7. A steel surface defect detection device, characterized in that, The device includes: The feature extraction unit is used to perform multi-level feature extraction on the acquired grayscale image of the steel surface using a backbone network, so as to obtain high-resolution low semantic features, mid-level features and high-level semantic features. An intermediate feature extraction unit is used to input the high-resolution low-semantic features, the mid-order features, and the high-level semantic features into the Neck module to obtain three intermediate features. The Neck module includes three branches: the first and second branches are used to process the high-resolution low-semantic features and the mid-order features through convolutional layers, respectively; and the third branch is used to process the high-level semantic features using convolution and AIFI modules. The encoding unit is used to input three intermediate features into the feature encoding module to obtain three encoded features at different scales; the feature encoding module is used to encode the features at three different scales using the GhostConv module, the CBAM module, and the RepC3 feature reuse structure. The initial query vector determination unit is used to process the encoded features of three different scales using an IoU-aware query mechanism to obtain the initial target query features; The steel surface defect identification unit is used to decode the initial target query features using a Transformer decoder with an auxiliary prediction head to obtain the detection results of steel surface defects.

8. A computer device, comprising a memory and a processor, characterized in that, The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the steel surface defect detection method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the steel surface defect detection method according to any one of claims 1 to 6.

Images