Steel surface defect detection method based on multi-scale edge enhancement

Abstract

The present application relates to the field of steel surface detection, and particularly relates to a steel surface defect detection method based on multi-scale edge enhancement, which carries out steel surface defect detection through a constructed defect detection model, wherein the defect detection model is established through the following steps: constructing an MEF-NET model, the MEF-NET model is an improved YOLOv 11 network model, the MEF-NET model comprises a backbone network for performing feature extraction on an input image, a neck network for performing multi-scale feature extraction and fusion on a feature map, and a head network for performing task-specific output on fusion features provided by the neck network, replacing a C3k2 module of the original backbone network with an AHED module, replacing an SPPF module with an MD-FPN module, replacing a C2PSA module with a C2BRT module, and replacing a detection head of the original YOLOv 11 network model with an Ehead, and the MEF-NET model proposed in the present application significantly improves detection performance.

Description

Technical Field

[0001] This invention relates to the field of steel surface inspection, and more specifically to a method for detecting steel surface defects based on multi-scale edge enhancement. Background Technology

[0002] Single-stage models, such as the YOLO series, offer advantages like simplified structure and fast inference speed, maintaining high accuracy while meeting the real-time requirements of various application scenarios. However, in the highly challenging scenario of steel surface defect detection, the YOLO series networks use fixed-size feature maps for prediction, such as P3, P4, and P5. This inherently suffers from insufficient receptive field for small defects, such as pinholes and cracks. Since steel surface defects often exhibit low contrast, fine granularity, and irregularity, YOLO-like models tend to overlook these local details, leading to missed detections. Summary of the Invention

[0003] The purpose of this invention is to provide a steel surface defect detection method based on multi-scale edge enhancement that can improve the detection accuracy of steel surface defects.

[0004] To achieve the above objectives, the present invention adopts the following technical solution:

[0005] A multi-scale edge-enhanced steel surface defect detection method is used to detect steel surface defects through a constructed defect detection model. The defect detection model is established through the following steps:

[0006] The MEF-NET model is constructed. The MEF-NET model is an improved YOLOv11 network model. The MEF-NET model includes a backbone network for feature extraction of input images, a neck network for multi-scale feature extraction and fusion of feature maps, and a head network for task-specific output of the fused features provided by the neck network.

[0007] Replace the original C3k2 module in the backbone network with the AHED module, replace the SPPF module with the MD-FPN module, and replace the C2PSA module with the C2BRT module.

[0008] The AHED module extracts global features of different resolutions from the input image through multi-scale adaptive average pooling, enhances the edges of each global feature using the edge enhancement mechanism of the lightweight edge enhancement module, and adaptively fuses the enhanced global features using a dual-domain selection mechanism (DDSM) composed of SSM and FSM.

[0009] Preferably, the detection head of the original YOLOv11 network model is replaced by an Ehead, which includes two branches, one of which uses an EMA module to model the global dependence of the input image, two layer-by-layer convolution modules to extract the boundary position features of the modeled feature map in turn, and a third convolution module to output four directional distributed regression of the extracted feature map, which is optimized by using CIOU as the regression loss function.

[0010] The other branch uses an EMA module to model the global dependence of the input image, and successively uses a depth separable convolution module, a fourth convolution module, a feature enhancement module and a fifth convolution module to extract features from the modeled feature map, generates a class prediction output through a sixth convolution module, and uses a Sigmoid activation function to cooperate with a cross-entropy loss for target classification training.

[0011] Preferably, the AHED module includes two branches, one of which performs convolution operation on the input feature to obtain a first feature;

[0012] The other branch parallelly inputs the input feature into multiple scale context branches, each of which compresses the input feature to different spatial scales through adaptive average pooling, and successively performs feature extraction on the compressed features through channel dimension reduction and depth separable convolution, and performs upsampling operation on each extracted feature, inputs each upsampled feature into the lightweight edge enhancement module to enhance the edges of each global feature, and splices the enhancement results of each scale with the first feature in the channel dimension to obtain a fused feature map, which is reweighted and optimally selected by a dual-domain selection mechanism (DDSM), and the optimized feature is compressed back to the original channel number through convolution operation.

[0013] Preferably, the lightweight edge enhancement module includes two branches, one of which extracts the spatial domain edge information of the input feature by low-pass smoothing and input difference, and performs convolution operation on the extracted spatial domain edge information;

[0014] The other branch simulates high-frequency edge response through dilated depth convolution and point-by-point convolution, and generates channel attention weights through global adaptive pooling;

[0015] The adaptive gate fusion is realized in the form of softmax on the two features, and the edge enhancement result is output in the form of residual.

[0016] Preferably, the MD-FPN module maps the channel number of the input feature to the bottleneck channel through a 1x1 compression sub-block to obtain a compressed feature, and performs dilated operation on the compressed feature in turn based on the weight of a single 3x3 convolution kernel to obtain a multi-scale context feature map.

[0017] The compressed features and their respective orders of expansion responses are spliced in the channel dimension to obtain a multi-scale feature splicing feature, and a 1x1 projection block is used to sequentially perform linear mixing, normalization and nonlinear activation on the multi-scale feature splicing feature.

[0018] Preferably, the C2BRT module is based on C2PSA, and the stack of C2PSA is replaced by a plurality of cascaded B RABlock modules.

[0019] The B RABlock module extracts features from the input features through a depth separable convolution module, adds the extracted features and the input features in residual, obtains initial features, inputs the initial features into a Bi-Level Routing Attention module after normalization processing for feature extraction, adds the extracted features and the initial features in residual, obtains local enhanced features, inputs the local enhanced features into a multi-layer perceptron after normalization processing for feature extraction, and adds the extracted features and the local enhanced features in residual to obtain the final output.

[0020] By adopting the foregoing design scheme, the application has the advantages that: the AHED module designed in the application adopts a multi-angle edge feature extraction strategy, which significantly improves the sensitivity to irregular defect contours; the MD-FDN module uses a shared multi-scale dilation convolution scheme to expand the network receptive field and strengthen the context information fusion between defects of different sizes; a double-layer routing attention mechanism is also introduced, and a C2BRT module is designed to recalibrate the feature representation, thereby improving the detection accuracy of small defects and low-contrast defects. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 Fig. 1 is a structural schematic diagram of the MEF-NET model of the application;

[0022] Figure 2 Fig. 2 is a structural schematic diagram of the AHED module of the application;

[0023] Figure 3 Fig. 3 is a structural schematic diagram of the MD-FDN module of the application;

[0024] Figure 4 Fig. 4 is a structural schematic diagram of the C2BRT module of the application;

[0025] Figure 5 Fig. 5 is a structural schematic diagram of the Ehead module of the application;

[0026] Figure 6 Fig. 6 is a comparison effect of the detection method and the baseline model on defect detection in the NEU-DET dataset of the application;

[0027] Figure 7 The comparison effect of the detection method of the present application and the baseline model on defect detection in the GC10-DET data set. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be described in further detail below with reference to the drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0029] The terms "first", "second", "third" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish different objects, and are not used to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device.

[0030] The steel surface defect detection method based on multi-scale edge enhancement is used for steel surface defect detection through a constructed defect detection model, and the defect detection model is established through the following steps: constructing a MEF-NET model as shown in Figure 1 The MEF-NET model is an improved YOLOv11 network model, and the MEF-NET model includes a backbone network for feature extraction of an input steel surface image, a neck network for multi-scale feature extraction and fusion of a steel surface feature map, and a head network for task-specific output of fusion features provided by the neck network.

[0031] The C3k2 module of the original backbone network is replaced by an AHED module, the SPPF module is replaced by an MD-FPN module, and the C2PSA module is replaced by a C2BRT module.

[0032] As shown in Figure 2 The AHED module extracts global features of different resolutions of the input steel surface image through multi-scale adaptive average pooling, enhances the edges of each global feature using the edge enhancement mechanism of a lightweight edge enhancement (HF-EdgeRefiner) module, and adaptively fuses each enhanced global feature using a dual-domain selection mechanism DDSM composed of SSM and FSM.

[0033] The AHED module includes two branches, one of which performs convolution operation on the input feature to obtain a first feature; in this embodiment, the branch performs convolution operation on the input feature by using local convolution to retain fine-grained detail information of the original scale;

[0034] The other branch sends the input feature into multiple scale context branches in parallel, each of which compresses the input feature to different spatial scales by adaptive average pooling, and each of the compressed features sequentially passes through 1x1 convolution for channel dimension reduction and depth separable convolution for feature extraction, and each of the extracted features is up-sampled and input into a lightweight edge enhancement module for edge enhancement of each global feature. In this embodiment, the multiple scale context branches are four convolution modules of 3*3, 6*6, 9*9 and 12*12.

[0035] In this embodiment, the lightweight edge enhancement (HF-EdgeRefiner) module includes two branches, one of which extracts spatial edge information of the input feature by using low-pass smoothing and input difference, and performs convolution operation on the extracted spatial edge information;

[0036] The other branch simulates high-frequency edge response by using dilated depth convolution and point-by-point convolution, and generates channel attention weights by using global adaptive pooling;

[0037] Adaptive gating fusion is realized on the two features in the form of softmax, and the edge enhancement result is output in a residual manner to complete adaptive enhancement of the edge features within the scale.

[0038] The enhancement results of each scale are concatenated with the first feature in the channel dimension to form a fusion feature map containing double the number of channels, and the fusion feature map is input into a dual domain selection mechanism (DualDomainSelectionMechanism) to reweight and optimally select the feature channels at the global and local levels, thereby realizing joint feature extraction across domains and scales. Finally, the feature map is compressed back to the original number of channels by one layer of 3x3 convolution to ensure compatibility with the backbone network structure.

[0039] In summary, the AHED module is sequentially executed by "local detail preservation - multi-scale context modeling - intra-scale edge enhancement - cross-domain selection optimization", effectively fusing the context information and significant edge features of different scales, and providing more discriminative feature representation for defect detection in complex scenes.

[0040] As Figure 3As shown, the MD-FPN module maps the channel number of the input feature to the bottleneck channel through a 1x1 compression sub-block to obtain compressed features that both preserve spatial resolution and reduce computational burden, and sequentially dilates the compressed features based on the weight of a single 3x3 convolution kernel with multiple preset dilation rates to obtain multi-scale context feature maps; in this embodiment, the preset dilation rates are 1, 3 and 5, and the 3x3 convolution kernel is used as a shared kernel to perform dilated convolution on the compressed features to generate a first scale response, and then a convolution with a larger dilation rate is applied to the first scale response as input to form a set of multi-scale context feature maps generated in sequence.

[0041] The compressed features and their respective dilated responses are spliced in the channel dimension to obtain multi-scale feature splicing features, and a 1x1 projection block is used to sequentially perform linear mixing, normalization and nonlinear activation on the multi-scale feature splicing features.

[0042] As shown, Figure 4 The C2BRT module is based on the C2PSA, and replaces the stack of the C2PSA with a plurality of cascaded BRA Block modules; during the feature extraction process of the overall network, the input feature first passes through a convolution layer for preliminary channel compression and feature transformation to enhance local representation capability and lay a foundation for subsequent deep modeling. Subsequently, the feature is sent to a deep structure stacked by a plurality of BRA Block modules, wherein each BRA Block is sequentially composed of three core units: a deep separable convolution (DWConv 3x3), a Bi-Level Routing Attention module and a multi-layer perceptron (MLP), and the feature is accumulated and stably transmitted level by level through a residual connection.

[0043] The BRA Block module extracts features from the input feature through the deep separable convolution module, adds the extracted features and the input feature in residual, obtains initial features to capture local spatial patterns and texture details while maintaining efficient computational complexity; the initial features are normalized to ensure stable feature distribution, and the normalized features are input into the Bi-Level Routing Attention module for feature extraction and added to the initial features in residual to obtain local enhanced features. The local enhanced features are normalized and input into the multi-layer perceptron for feature extraction and added to the local enhanced features in residual to obtain the final output.

[0044] In this embodiment, the Bi-Level Routing Attention module of the C2BRT module divides the input feature map into a plurality of regions, calculates the query Q, key K and value V of each region, and adopts a region-level affinity matrix The relationship between the regions is calculated, and the top-k operation is used to select the k most relevant regions for each region, to obtain a routing index wherein the affinity matrix is as follows:

[0045] ;

[0046] wherein, and represent the aggregated key-value pairs, respectively;

[0047] According to the routing index The keys and values of the selected regions are collected, and token-to-token attention calculation is performed on the collected keys and values to output a feature map.

[0048] As a preferred mode of the present embodiment, the detection head of the original YOLOv11 network model is replaced by an Ehead, as shown in Figure 5 The Ehead includes two branches, one of which uses an EMA module to model the global dependence of the input image, two layer-by-layer convolution modules to sequentially extract the features of the boundary position of the modeled feature map, and a third convolution module to perform distributed regression output in four directions on the extracted feature map, and the CIOU is used as the regression loss function for optimization.

[0049] The other branch uses an EMA module to model the global dependence of the input image, and sequentially uses a depth separable convolution module, a fourth convolution module, a feature enhancement module and a fifth convolution module to extract features from the modeled feature map, generates a class prediction output through a sixth convolution module, and uses a Sigmoid activation function in conjunction with a cross-entropy loss (CLSLoss) for target classification training.

[0050] As shown in Figure 6 and as shown in Figure 7 The present application also provides a comparison of the effects of the detection method and the baseline model on defect detection on the NEU-DET dataset and the GC10-DET dataset.

[0051] Table 1 Ablation experiment of the detection method of the present application (NEU-DET dataset)

[0052]

[0053] As shown in Table 1, the baseline YOLOv 11n architecture achieved an average precision (mAP) of 76.38% on the NEU-DET dataset. The MEF-NET model proposed in this application produced a significant performance gain: the AHED increased the mAP by 1.26%, the MD-FPN contributed an additional 1.26%, and the C2BRT module further improved the performance by 1.05%. These improvements highlight the efficacy of the components proposed in this application in extracting discriminative edge features and facilitating robust multi-scale feature fusion, enabling the model to capture more subtle representations, reduce the interference of defect-induced noise, and preserve key morphological details. When the aforementioned three modules are integrated into a unified architecture, the resulting model achieves an mAP of 79.34% on the NEU-DET dataset, a significant improvement of 2.96 percentage points over the baseline YOLOv 11n model. This substantial enhancement validates our redesigned backbone network's significant improvement in defect recognition capability and generalization performance, even when applied to datasets with specific domain defect characteristics, through improved multi-angle edge feature selection, extraction, and fusion mechanisms. Our architectural innovations are realized in the final MEF-NET model, which integrates the EMA module into the detection head. This complete framework achieves an mAP of 81.48% on the NEU-DET dataset. These compelling results not only validate the effectiveness of the proposed architectural enhancements in significantly improving detection performance but also establish MEF-NET as an efficient and accurate framework for real-time surface defect detection in industrial steel production environments. Furthermore, the detailed ablation study systematically quantifies the individual and synergistic contributions of the proposed modules to overall performance improvement, providing valuable insights for architectural design considerations in industrial defect detection systems.

[0054] As always, the AHED module designed in this application employs a multi-angle edge feature extraction strategy, significantly improving sensitivity to irregular defect contours; the MD-FDN module utilizes a shared multi-scale dilated convolution scheme to expand the network's receptive field and enhance context information fusion between defects of different sizes; a dual-layer routing attention mechanism is also introduced, and the C2BRT module is designed to recalibrate feature representations, thereby improving the detection accuracy of small defects and low-contrast defects; the use of the MEF-NET model for steel surface defect detection can effectively improve detection accuracy.

[0055] The specific embodiments described above further illustrate the objectives, technical solutions, and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and does not limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for detecting surface defects in steel based on multi-scale edge reinforcement, characterized in that: Surface defects in steel are detected using a constructed defect detection model, which is established through the following steps: The MEF-NET model is constructed. The MEF-NET model is an improved YOLOv11 network model. The MEF-NET model includes a backbone network for feature extraction of input images, a neck network for multi-scale feature extraction and fusion of feature maps, and a head network for task-specific output of the fused features provided by the neck network. Replace the original C3k2 module in the backbone network with the AHED module, replace the SPPF module with the MD-FPN module, and replace the C2PSA module with the C2BRT module. The AHED module extracts global features of different resolutions from the input image through multi-scale adaptive average pooling, enhances the edges of each global feature using the edge enhancement mechanism of the lightweight edge enhancement module, and adaptively fuses the enhanced global features using the dual-domain selection mechanism DDSM composed of SSM and FSM. It also includes replacing the detection head of the original YOLOv11 network model with Ehead. Ehead includes two branches. One branch uses the EMA module to perform global dependency modeling on the input image, uses two layer-by-layer convolutional modules to extract features at the boundary positions of the modeled feature map in sequence, and uses a third convolutional module to perform distributed regression output in four directions on the extracted feature map. CIOU is used as the regression loss function for optimization. Another branch uses the EMA module to model the global dependencies of the input image, and then uses the depthwise separable convolution module, the fourth convolution module, the feature enhancement module and the fifth convolution module to extract features from the modeled feature map in sequence. The sixth convolution module generates the class prediction output, and the Sigmoid activation function is used in conjunction with cross-entropy loss for target classification training. The AHED module includes two branches, one of which performs a convolution operation on the input features to obtain the first feature; Another branch feeds the input features in parallel into multiple scale context branches. Each scale context branch compresses the input features to different spatial scales through adaptive average pooling. The compressed features are then extracted by channel reduction and depthwise separable convolution. The extracted features are upsampled and then input into the lightweight edge enhancement module for edge enhancement of each global feature. The enhancement results at each scale are concatenated with the first feature by channel dimension to obtain a fused feature map. The dual-domain selection mechanism DDSM reweights and optimizes the fused feature map. The optimized features are then compressed back to the original number of channels by convolution. The MD-FPN module maps the number of channels of the input feature to the bottleneck channel through 1×1 compressed sub-blocks to obtain compressed features. Based on the weight of a single 3×3 convolution kernel, the compressed feature is sequentially expanded with multiple preset expansion rates to obtain a multi-scale context feature map. The compressed features and their dilated responses at each order are concatenated along the channel dimension to obtain multi-scale feature concatenation features. A 1×1 projection block is then used to sequentially perform linear mixing, normalization, and nonlinear activation on the multi-scale feature concatenation features.

2. The method for detecting steel surface defects based on multi-scale edge reinforcement as described in claim 1, characterized in that: The lightweight edge enhancement module includes two branches. One branch uses low-pass smoothing and input difference to extract the spatial edge information of the input features and performs a convolution operation on the extracted spatial edge information. Another branch simulates high-frequency edge responses through dilated depthwise convolution and pointwise convolution, and uses global adaptive pooling to generate channel attention weights; Adaptive gating fusion is achieved using the softmax method on the two-way features, and the edge enhancement result is output in a residual manner.

3. The method for detecting steel surface defects based on multi-scale edge reinforcement as described in claim 1 or 2, characterized in that: The C2BRT module is based on C2PSA, replacing the C2PSA stack with several cascaded BRABlock modules; The BRABlock module extracts features from the input features using a depthwise separable convolution module. The extracted features are then added to the input features using residuals to obtain initial features. These initial features are normalized and then input to a Bi-Level Routing Attention module for further feature extraction. The extracted features are then added to the initial features using residuals to obtain local enhancement features. These local enhancement features are then normalized and input to a multilayer perceptron for further feature extraction. Finally, the extracted features are added to the local enhancement features using residuals to obtain the final output.

Images