Steel surface defect detection method based on multi-scale feature collaborative optimization

Abstract

The application discloses a steel surface defect detection method based on multi-scale feature cooperative optimization, and is based on a YOLOv11 model architecture. First, a multi-core reparameterization convolution MKRConv is fused in a C3K2 module, multi-scale and structured defect features are synchronously captured through multiple types of lightweight convolution branches, and representation capability and feature stability are considered. Second, a frequency domain guided space adaptive module FSAM is introduced before a detection head, feature weights are dynamically allocated, and the feature response of small and boundary fuzzy defects is strengthened. Finally, a bidirectional guided fusion module BGFM is designed, high and low layer features are interactively calibrated in a bidirectional mode, the semantic misplacement problem in traditional fusion is solved, and then the accurate capture of multi-scale defect features, the adaptive allocation of attention and the bidirectional accurate fusion of high and low layer features are realized.

Description

Technical Field

[0001] This invention relates to the field of industrial steel surface defect detection technology, and in particular to a steel surface defect detection method based on multi-scale feature collaborative optimization. Background Technology

[0002] As a basic industrial material, steel surface defect detection is a core part of product quality control. Manual visual inspection is affected by subjective factors, resulting in a high rate of missed and false detections, and it cannot meet the real-time detection needs of high-speed production lines. Traditional manual feature extraction methods rely on manually designed operators, which have poor generalization ability and insufficient robustness to changes in lighting and background interference.

[0003] With the development of deep learning technology and computer hardware, automatic feature extraction methods based on convolutional neural networks have been widely used in the field of industrial defect detection due to their powerful end-to-end learning capabilities. Based on differences in technical approaches, mainstream deep learning object detection algorithms can be divided into two main categories: two-stage detection algorithms and single-stage detection algorithms.

[0004] The core idea of ​​the two-stage detection algorithm is to decompose the detection task into two independent stages: candidate region generation and target classification and regression. High-precision detection is achieved through staged optimization, and it has excellent detection accuracy. However, the complex candidate region generation and iterative optimization process leads to slow inference speed, which makes it difficult to meet the needs of real-time detection in steel production lines.

[0005] Single-stage detection algorithms break away from the step-by-step logic of two-stage algorithms. They eliminate the need for a separate candidate region generation stage, directly using convolutional neural networks to predict images at the pixel level, achieving simultaneous output of defect category and location. This results in significant advantages such as high detection speed and low deployment cost, making them more suitable for real-time detection scenarios in modern industrial production lines. In recent years, single-stage algorithms, represented by the YOLO series, SSD, and RetinaNet, have been continuously iterated and optimized, with their detection accuracy gradually approaching or even surpassing that of two-stage algorithms. However, existing single-stage steel surface defect detection methods still face two major technical challenges in practical applications:

[0006] 1. Insufficient ability to characterize multi-scale defects: Steel surface defects range from micron-level microcracks to macro-level wide cracks, with a large scale span. The fixed receptive field of existing models is difficult to adapt to small, medium and large-scale defects at the same time. Small-scale defect features are easily submerged by background noise or large-scale defects, resulting in missed detection. The positioning accuracy of large-scale irregular defects is low, and the model lacks scale adaptation ability, which cannot adapt to the dynamic changes in defect scale of steel plates of different specifications in the production line.

[0007] 2. Insufficient effectiveness of feature fusion: Existing models mostly use Feature Pyramid (FPN) and its variants for feature fusion, which integrates shallow detail features and deep semantic features by simply splicing or adding elements one by one. This does not solve the semantic gap and feature misalignment problem of cross-layer features. The precise spatial location of shallow details and the abstract features of deep semantics cannot be effectively aligned. The discriminative ability of fused features is limited, and it is difficult to establish a precise association between details and semantics. Summary of the Invention

[0008] Technical Problem: To address the technical problems of insufficient multi-scale feature representation and feature fusion misalignment in the existing YOLOv11 model for multi-scale defect detection on steel surfaces, this invention proposes a steel surface defect detection method based on multi-scale feature collaborative optimization. This method improves and deeply integrates YOLOv11, achieving accurate capture of multi-scale defect features, adaptive selection of receptive field, and accurate bidirectional fusion of high and low layer features.

[0009] Technical solution: A method for detecting surface defects in steel based on multi-scale feature collaborative optimization, comprising the following steps:

[0010] Step 1: Construct a steel surface defect detection model based on YOLOv11, optimize the key feature extraction module C3K2 of YOLOv11, and replace the standard convolution that constitutes the bottleneck module inside C3K2 with multi-kernel reparameterized convolution MKRConv to enhance the feature representation capability.

[0011] Step 2: Embed the Frequency Domain Guided Spatial Adaptive Module (FSAM) in front of the YOLOv11 detector head to improve the ability to adaptively select different receptive field features;

[0012] Step 3: In the YOLOv11 neck network, a bidirectional guided fusion module (BGFM) is constructed to replace the original feature fusion unit. The bidirectional guided fusion module (BGFM) includes high-level guided gating and low-level guided gating, which are used to realize the fusion of shallow detail information and deep semantic information, and to perform feature complementarity.

[0013] Step 4: Use publicly available steel surface defect detection datasets to iteratively train the constructed steel surface defect detection model;

[0014] Step 5: Evaluate the performance of the trained steel surface defect detection model to obtain the model's detection capability for defects of different sizes, which can be used for real-time detection in steel production lines.

[0015] Furthermore, step 1 specifically involves:

[0016] We construct a multi-kernel reparameterized convolution MKRConv, which consists of multiple types of parallel deep convolution branches. Through lightweight design, it significantly enhances the model's ability to represent multi-scale defects while ensuring computational efficiency. These branches include small kernel convolution, dilated convolution, large kernel convolution, and asymmetric convolution, which can simultaneously capture multi-scale defect features and structured defect features.

[0017] To balance the originality and stability of the feature distribution, each convolutional branch is equipped with an independent batch normalization layer, so that the output branch contains both the detailed features of the original convolution and the stable features after the batch normalization layer regularization. The final fused features contain both rich defect details and good generalization ability. Finally, after summing the elements of each branch, the multi-scale features are integrated using a standard 1×1 convolution.

[0018] During the inference phase, the multi-branch structure of the multi-kernel reparameterized convolution MKRConv is reparameterized into a single depthwise convolution kernel, reducing the number of parameters while maintaining performance. Zero-value padding is used to transform small-kernel, dilated, and asymmetric convolution kernels into large-kernel convolutions. Next, the parameters of the batch normalization layer are incorporated into the weights and biases of the convolution kernel, resulting in multiple convolution kernels with identical configurations. Finally, based on the linear additivity of convolution kernels, the kernels are summed to obtain a single depthwise convolution kernel used for inference.

[0019] The constructed MKRConv is integrated into the core feature extraction module C3K2 of YOLOv11. The multi-core reparameterized bottleneck module MKR-BottleNeck is constructed using MKRConv and replaces the original bottleneck module BottleNeck in C3K2. This is used to construct the MKR-C3K2 module, which is used to replace the deep C3K2 module of the network.

[0020] In particular, the MKR-C3K2 module, built on multi-kernel reparameterized convolution, also inherits the flexible structural adaptation capability of the C3K2 module. It can convert the module according to the value of parameter C3K: when parameter C3K is set to False, the module will be converted into the MKR-C2F module. At the same time, MKRConv will introduce an additional 1×1 convolution branch and the corresponding batch normalization layer branch to meet the differentiated needs of feature expression and computational efficiency at different feature extraction stages.

[0021] Furthermore, step 2 specifically involves:

[0022] Input feature map Based on this, multi-scale features are extracted using multiple sets of decomposed strip-shaped depthwise convolution kernels (1×k / k×1) to alleviate the problem of secondary parameter growth. At the same time, strip-shaped convolution is more likely to capture the texture and shape features of slender defects (cracks, scratches) on the steel surface, thus obtaining the features of multi-scale convolution branches. ;

[0023] Features of multi-scale convolution branches Element-wise addition is performed, and scale-aware features are generated through 1×1 convolution. Then, a frequency domain representation is obtained by two-dimensional Fourier transform. Low-frequency components mainly reflect the overall brightness distribution and large-scale contour information, while high-frequency components mainly respond to edge abrupt changes, fine-grained textures, and local abnormal areas.

[0024] Subsequently, the amplitude spectrum of the frequency domain representation is taken and averaged along the channel dimension to obtain a single-channel spectral statistics chart. This operation compresses the frequency responses on different channels into a unified global spectral description, highlighting the overall activity of the frequency components corresponding to that location, thereby providing a stable prior for subsequent multi-scale branch selection;

[0025] Then, a learnable mapping is used to convert the frequency domain statistical graph into multi-channel spatial attention weights with corresponding branch numbers. For each channel's spatial attention graph, a sigmoid function needs to be applied for activation to obtain the attention weights. ;

[0026] Attention weight Features of multi-scale convolution branches Element-wise weighted summaries are then fused using 1×1 convolutions to generate attention feature maps. Finally, it is compared with the original input features. Element-wise multiplication generates scale-adaptive enhanced feature maps. .

[0027] Furthermore, step 3 specifically involves:

[0028] Assume the dimension of the low-level feature map is The dimension of the high-level feature map is The processing of the bidirectional guided fusion module BGFM includes the following sub-steps:

[0029] Step 3.1, High-rise guidance gate control:

[0030] High-level features themselves contain rich semantic information, from high-level features Extract semantically relevant channel weights to guide low-level features in preserving key details;

[0031] The number of channels in the high-level features is adjusted to match that of the low-level features using standard 1×1 convolution and 3×3 depthwise convolution, and the channel-aligned feature map is denoted as . ;

[0032] Global average pooling is used to aggregate spatial information into channel-dimensional global semantic information. This global semantic information is then sent to a bottleneck module, allowing the model to focus on learning the non-linear relationships between channels. This module consists of two PWConv layers, with a ReLU activation function in between to introduce non-linear transformations, and a dimensionality reduction coefficient r is set to control the number of intermediate channels. Finally, a sigmoid activation function is used to generate semantically relevant channel weights. .

[0033] Step 3.2, Low-level guidance gating:

[0034] As low-level features, a lot of original detailed information is retained. Spatial location weights are extracted from low-level features to guide high-level features to focus on the target defect area.

[0035] Convolutional layers are used to focus on the contextual information of low-level features, and their multi-channel information is weighted and aggregated into single-channel features. Then, attention weights are generated in the spatial dimension using the Sigmoid activation function. .

[0036] Step 3.3, Feature Complementary Aggregation:

[0037] The obtained channel weights and spatial weights are applied to low-level features and high-level features respectively. The channel weights of high-level semantics are used to filter low-level details and retain semantically related spatial information. The spatial weights of low-level details are used to filter high-level semantics, so that semantic information is focused on the target location.

[0038] The enhanced high- and low-level features are concatenated along the channel dimension to obtain the final fused feature. .

[0039] Furthermore, step 4 specifically involves:

[0040] A publicly available steel surface defect detection dataset was selected, and the dataset was preprocessed and divided into training set, validation set and test set in a ratio of 7:1.5:1.5.

[0041] Set the hyperparameters required for model training: a total of 300 training epochs, a batch size of 16, an input image size of 640×640, an optimizer of SGD, an initial learning rate of 1e-2, a final learning rate of 1e-4, and a weight decay of 5e-4.

[0042] Furthermore, step 5 specifically involves:

[0043] Experiments are conducted on the best-performing model obtained through training using a test set, and the model's performance is analyzed based on core evaluation metrics.

[0044] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0045] 1. This invention is based on the YOLOv11 model architecture. It integrates multi-kernel heavily parameterized convolution MKRConv into the C3K2 module, simultaneously capturing multi-scale and structured defect features through multiple types of lightweight convolutional branches, balancing representational capability and feature stability. Secondly, a frequency-domain guided spatial adaptive module FSAM is introduced before the detection head to dynamically allocate feature weights and enhance the feature response of minute and blurred-boundary defects. Finally, a bidirectional guided fusion module BGFM is designed to solve the semantic misalignment problem in traditional fusion through bidirectional interactive calibration of high and low-level features, achieving accurate collaboration of multi-scale features and providing strong discriminative feature support for efficient detection.

[0046] 2. This invention achieves a significant improvement in detection accuracy without introducing substantial computational overhead, balancing the model's detection accuracy and inference efficiency, and can meet the real-time detection needs of continuous steel production lines. Attached Figure Description

[0047] Figure 1 This is an overall flowchart of the steel surface defect detection method based on multi-scale feature collaborative optimization of the present invention;

[0048] Figure 2 This is a diagram of the multi-kernel reparameterized convolution structure in an embodiment of the present invention;

[0049] Figure 3 This is a structural diagram of the frequency domain guided space adaptive module in an embodiment of the present invention;

[0050] Figure 4 This is a structural diagram of the bidirectional guidance fusion module in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the application will be further described in detail below with reference to the accompanying drawings. The described embodiments are only a part of the embodiments involved in this invention. All non-innovative embodiments based on these embodiments by other researchers in the art are within the protection scope of this invention. Furthermore, the step numbers in the embodiments of this invention are only set for ease of explanation and do not limit the order of the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0052] In one embodiment of the present invention, a method for detecting steel surface defects based on multi-scale feature collaborative optimization is provided. A steel surface defect detection model is constructed based on YOLOv11, and the model is trained and optimized, then applied to real-time defect detection in steel production lines. The overall process is as follows: Figure 1 As shown, it includes the following steps:

[0053] Step 1: Optimize the key feature extraction module C3K2 of YOLOv11. Replace the standard convolution, which is the bottleneck module, with multi-kernel reparameterized convolution MKRConv to enhance the feature representation capability. The overall structure is as follows: Figure 2 As shown.

[0054] Multi-kernel reparameterized convolution MKRConv consists of multiple types of parallel depthwise convolution branches. Through lightweight design, it significantly enhances the model's ability to represent multi-scale defects while ensuring computational efficiency.

[0055] In this embodiment, the multi-kernel reparameterized convolution MKRConv includes: 3×3 standard convolution, 3×3 dilated convolution (dilation=2), 5×5 standard convolution, and 1×5 / 5×1 asymmetric convolution, which can simultaneously capture multi-scale defect features and structured defect features.

[0056] Furthermore, to balance the originality and stability of the feature distribution, each convolutional branch is equipped with an independent batch normalization layer, so that the output branch contains both the detailed features of the original convolution and the stable features after regularization by the batch normalization layer. The final fused features contain both rich defect details and good generalization ability.

[0057] Finally, after element-wise summation of each branch, multi-scale features are integrated using a standard 1×1 convolution. The multi-branch aggregation process can be represented as:

[0058] ;

[0059] in, For the output of the j-th branch, This is the batch normalization operation corresponding to the j-th branch.

[0060] During the inference phase, MKRConv's multi-branch structure is reparameterized into a single depthwise convolution kernel, reducing the number of parameters while maintaining performance. Zero-value padding is used to transform small kernel convolution kernels, dilated kernel convolution kernels, and asymmetric kernel convolution kernels into large kernel convolution kernels.

[0061] Next, the parameters of the batch normalization layer are incorporated into the weights and biases of the convolutional kernels, resulting in multiple convolutional kernels with the same configuration. For the j-th branch, its fused weights and biases can be expressed as:

[0062] ;

[0063] in, and Let these represent the weights and biases of the j-th convolutional branch, respectively. , , and These represent the scaling factor, translation parameter, mean, and variance of the corresponding Batch Normalization (BN) layer, respectively. To prevent numerically unstable minimal constants.

[0064] Finally, based on the linear additivity of convolution kernels, the convolution kernels are summed to obtain a single depthwise convolution kernel for inference, which can be specifically represented as:

[0065] ;

[0066] in, and These are the equivalent convolution kernel weights and biases, respectively.

[0067] The constructed MKRConv is integrated into the core feature extraction module C3K2 of YOLOv11. Specifically, MKRConv is used to construct a multi-core reparameterized bottleneck module MKR-BottleNeck and replace the original bottleneck module BottleNeck in C3K2, thereby constructing the MKR-C3K2 module.

[0068] Using small kernel convolutions in the early stages of the network can preserve low-level details. Therefore, in this embodiment, the first two C3K2 modules are retained, and the remaining C3K2 modules are replaced with MKR-C3K2 modules.

[0069] The MKR-C3K2 module, based on multi-kernel reparameterized convolution design, also inherits the flexible structural adaptation capability of the C3K2 module. It can convert the module according to the value of the parameter C3K. When C3K is set to False, the module will be converted into the MKR-C2F module. At the same time, MKRConv will introduce an additional 1×1 convolution branch and a corresponding batch normalization layer branch to meet the differentiated needs of feature expression and computational efficiency at different feature extraction stages.

[0070] Step 2: Embed a frequency-domain guided spatial adaptive module in front of the YOLOv11 detector head to effectively improve the ability to adaptively select different receptive field features. Detailed structure is as follows: Figure 3 As shown:

[0071] Input feature map Based on this, multi-scale features are extracted using multiple sets of decomposition-based strip-shaped depthwise convolution kernels (1×k / k×1), alleviating the problem of secondary parameter growth. Simultaneously, strip-shaped convolution more readily captures the texture and shape features of slender defects (cracks, scratches) on steel surfaces, resulting in the features of multi-scale convolution branches. :

[0072] ;

[0073] in, These are batch size, number of channels, height, and width, respectively. For the number of branches, Let be the size of the convolution kernel in the i-th branch. The function represents depthwise convolution.

[0074] Features of multi-scale convolution branches Element-wise addition is performed, and scale-aware features are generated through 1×1 convolution. Then, a frequency domain representation is obtained by two-dimensional Fourier transform. Low-frequency components mainly reflect the overall brightness distribution and large-scale contour information, while high-frequency components mainly respond to edge abrupt changes, fine-grained textures, and local abnormal areas.

[0075] Subsequently, the amplitude spectrum of the frequency domain representation is taken, and the average is calculated along the channel dimension to obtain the single-channel spectral statistics. :

[0076] As a preferred embodiment, in this example, the spectrum statistics diagram The calculation process is as follows:

[0077] ;

[0078] ;

[0079] Then, the spectral statistics are converted into multi-channel spatial attention weights with corresponding branch numbers using a learnable mapping. For each channel's spatial attention map, a sigmoid function needs to be applied for activation to obtain the attention weights. :

[0080] Attention weight With corresponding multi-scale branches The features are weighted element-wise and fused through a 1×1 convolution to generate an attention feature map. Finally, it is compared with the original input features. Element-wise multiplication generates scale-adaptive enhanced feature maps. :

[0081] ;

[0082] ;

[0083] ;

[0084] Wherein, the P function represents the standard convolution operation. This is element-wise multiplication.

[0085] Step 3: In the YOLOv11 neck network, a bidirectional guided fusion module (BGFM) is designed to replace the original feature fusion unit. The BGFM module mainly includes high-level guided gating and low-level guided gating to achieve accurate fusion of shallow detail information and deep semantic information. The detailed structure is as follows: Figure 4 As shown.

[0086] In this embodiment, it is assumed that the dimension of the low-level feature map is . The dimension of the high-level feature map is The specific processing of the bidirectional guided fusion module BGFM is as follows:

[0087] 1. High-rise access control, the specific procedures are as follows:

[0088] As high-level features themselves contain rich semantic information, semantically related channel weights are extracted from high-level features to guide low-level features to retain key details.

[0089] The number of channels in the high-level features is adjusted to match that of the low-level features using standard 1×1 convolution and 3×3 depthwise convolution, and the channel-aligned feature map is denoted as . The calculation process is expressed as follows:

[0090] ;

[0091] Global average pooling is used to aggregate spatial information into global semantic information of the channel dimension. Then, the global semantic information is sent to a bottleneck module to allow the model to focus on learning the nonlinear correlation between channels. It consists of two PWConv layers, with a ReLU activation function in between to introduce nonlinear transformation, and a dimensionality reduction coefficient r is set to control the number of intermediate channels.

[0092] Finally, semantically relevant channel weights are generated using the Sigmoid activation function. The calculation process is expressed as follows:

[0093] ;

[0094] in, , The ReLU activation function is used. It is the Sigmoid activation function. This is global average pooling.

[0095] 2. Low-level guidance gating, the specific processing is as follows:

[0096] As low-level features, a lot of original detailed information is retained. Spatial location weights are extracted from low-level features to guide high-level features to focus on the target defect area.

[0097] Convolutional layers are used to focus on the contextual information of low-level features, and their multi-channel information is weighted and aggregated into single-channel features. Then, attention weights are generated in the spatial dimension using the Sigmoid activation function. The calculation process can be expressed as follows:

[0098] ;

[0099] in, .

[0100] 3. Feature complement aggregation, the specific processing is as follows:

[0101] The obtained channel weights and spatial weights are applied to low-level features and high-level features respectively. The channel weights of high-level semantics are used to filter low-level details and retain semantically related spatial information. The spatial weights of low-level details are used to filter high-level semantics, so that semantic information is focused on the target location.

[0102] The enhanced high- and low-level features are then concatenated along the channel dimension to obtain the final fused feature. The expression is:

[0103] ;

[0104] in, .

[0105] Step 4: Iteratively train the constructed MSCE-NET using publicly available steel surface defect detection datasets.

[0106] This embodiment uses the steel surface defect detection datasets published by NEU-DET and GC10-DET for training and evaluation, and randomly divides them into training set, validation set and test set in a ratio of 7:1.5:1.5.

[0107] The NEU-DET dataset focuses on surface defect detection in rolled steel strips, containing six typical defect classes: rolled-in scale, patches, cracking, pitted surface, inclusion, and scratches. The dataset comprises 1800 grayscale images, all with an original resolution of 200×200, and a balanced number of samples for each defect class, containing 300 labeled samples for each class.

[0108] The GC10-DET dataset contains samples collected from real industrial production environments, making it more relevant to actual steel surface defect detection applications. The dataset contains a total of 2292 images, with an original size of 2048×1000, covering 10 types of surface defects: punching holes (Pu), welds (Wl), crescent-shaped cracks (Cg), water spots (Ws), oil spots (Os), thread spots (Ss), inclusions (In), rolling craters (Rp), creases (Cr), and waist creases (Wf). Unlike the NEU-Det dataset, this dataset exhibits a significant difference in the number of samples for each category, displaying an unbalanced distribution.

[0109] The hardware environment used in this embodiment consists of an Intel Xeon Gold 6240 CPU and eight NVIDIA Tesla V100S graphics processing units with a total video memory of 32GB. The software environment is based on Python 3.8.10, PyTorch 2.0.1, and CUDA 11.7.

[0110] Furthermore, the total number of training epochs was set to 300, the batch size to 16, the input image size to 640×640, the stochastic gradient descent (SGD) optimizer was selected, the momentum was set to 0.937, the initial learning rate was set to 1e-2, the final learning rate was decayed to 1e-4, and the weight decay was set to 5e-4.

[0111] Step 5: Evaluate the performance of the trained model.

[0112] In this embodiment, precision (P), average precision (mAP), and recall (R) are primarily used as evaluation metrics for the network model's performance. Frames per second (FPS) are also used to measure the model's inference speed. The detailed meanings and calculation formulas of these evaluation metrics are as follows:

[0113] ;

[0114] Accuracy This metric measures the proportion of samples that are actually classified as "positive" by the model, reflecting the accuracy of the prediction. TP (True Positive) represents the number of samples correctly predicted as defective by the model, while FP (False Positive) represents the number of samples incorrectly predicted as defective by the model but which are actually normal.

[0115] ;

[0116] Recall rate This measure represents the proportion of samples that are actually classified as "positive" but are successfully predicted as "positive" by the model, reflecting the degree to which the model misses defects. FN (False Negative) indicates the number of samples that the model incorrectly predicted as normal but were actually defective.

[0117] ;

[0118] ;

[0119] Where C represents the total number of defect categories, AP represents the average precision of a single defect category, which is the area under the precision-recall curve for each category. mAP represents the average AP across all defect categories, comprehensively reflecting the model's overall detection performance for multi-category defects. mAP50 requires setting the IOU threshold to 50%, meaning that a predicted bounding box is considered a True Positive only when the overlap between the predicted and actual bounding boxes is greater than or equal to 50%.

[0120] To further validate the proposed model's application potential across multiple scales, three evaluation metrics, mAPs, mAPm, and mAPl, were introduced. Defects were categorized into small, medium, and large based on their target size, and the mAP for each size was calculated to evaluate the model's ability to detect defects of different sizes.

[0121] This embodiment uses the NEU-DET dataset to conduct detailed ablation experiments, verifying the technical effect of the synergistic effect of the three modules. The results are shown in Table 1 below.

[0122] Table 1: Ablation test results of the embodiments of the present invention on the NEU-DET dataset

[0123]

[0124] After gradually introducing various modules into the YOLOv11n baseline model, the overall detection performance of the model continued to improve. After introducing the bidirectional guided fusion module (BGFM), mAP50 increased from 73.2% to 74.4%, and mAP50:95 increased from 40.4% to 41.1%, indicating that BGFM can improve the information complementarity and position alignment issues between high- and low-level features during the fusion process. Further introducing the frequency-domain guided spatial adaptive module (FSAM), mAP50 and mAP50:95 increased to 75.4% and 41.5%, respectively, indicating that the frequency-domain statistical guided branch selection mechanism before the detection head can enhance the model's adaptability to multi-scale defects. Continuing to introduce the multi-kernel reparameterized convolution (MKRConv), mAP50 reached 76.0%, mAP50:95 reached 42.7%, and the recall rate significantly improved to 75.0%, indicating that MKRConv enhances multi-scale feature extraction capabilities while helping to reduce missed defect detections. It is worth noting that after introducing all the improved modules, the final number of parameters and computational cost of the model changed little compared with the benchmark model, and remained at a similar level overall. This indicates that the method in this paper significantly improves the detection performance without bringing significant additional model complexity overhead.

[0125] This embodiment conducted detailed comparative experiments on the NEU-DET and GC10-DET datasets, verifying the superior overall performance of the constructed MSCE-Net model in the multi-scale defect detection task on steel surfaces. Comparative experiments were conducted between MSCE-Net and current mainstream target detection algorithms, comprehensively comparing the detection accuracy, multi-scale defect adaptation capability, lightweight nature, and computational efficiency of each model. The results are shown in Table 2 below.

[0126] Table 2: Comparative Test Results of Embodiments of the Invention on the NEU-DET Dataset

[0127]

[0128] As can be seen, MSCE-Net of this invention achieves the best overall detection performance on the NEU-DET dataset, with mAP50 and mAP50:95 reaching 76.0% and 42.7% respectively, both higher than other comparative methods. From the perspective of scale detection results, the performance improvement of MSCE-Net on targets of different scales is more representative. First, in small target defect detection, MSCE-Net's mAPs reach 52.1%, significantly higher than all comparative methods, indicating that the multi-scale feature enhancement and dynamic selection mechanism designed in this invention can effectively alleviate the problem of small defects being easily submerged and missed in complex backgrounds. Second, in medium-scale defect detection, MSCE-Net's mAPs reach 33.3%, slightly higher than YOLOv12n's 33.2%, indicating that the model also has good adaptability in medium-scale defect representation. Finally, in terms of large target detection, MSCE-Net achieved an mAP1 of 50.8%, which is higher than YOLOv6n's 50.4% and YOLOv10n's 50.3%, indicating that the present invention also maintains effective modeling of large-scale defect contours and overall structures, and achieves a comprehensive improvement in multi-scale defect detection performance.

[0129] In summary, this invention, through the design of the MKR-C3K2 module, the embedded frequency-domain guided spatial adaptive module FSAM, and the bidirectional guided fusion module BGFM, and deep integration with YOLOv11, can achieve accurate capture of multi-scale defect features, adaptive allocation of dynamic attention, and bidirectional accurate fusion of high and low-level features. While maintaining the model's lightweight nature and real-time inference efficiency, it significantly improves the detection accuracy of small, medium, and large-scale defects on steel surfaces, solving the problems of high false negative rate, low positioning accuracy, and weak generalization ability of existing models, and adapting to the high-speed and complex actual detection scenarios on steel production lines.

[0130] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects in steel based on multi-scale feature collaborative optimization, characterized in that, Includes the following steps: Step 1: Construct a steel surface defect detection model based on YOLOv11, optimize the key feature extraction module C3K2 of YOLOv11, and replace the standard convolution that constitutes the bottleneck module inside C3K2 with multi-kernel reparameterized convolution MKRConv to enhance the feature representation capability. Step 2: Embed the Frequency Domain Guided Spatial Adaptive Module (FSAM) in front of the YOLOv11 detector head to improve the ability to adaptively select different receptive field features; Step 3: In the YOLOv11 neck network, a bidirectional guided fusion module (BGFM) is constructed to replace the original feature fusion unit. The bidirectional guided fusion module (BGFM) includes high-level guided gating and low-level guided gating, which are used to realize the fusion of shallow detail information and deep semantic information, and to perform feature complementarity. Step 4: Use publicly available steel surface defect detection datasets to iteratively train the constructed steel surface defect detection model; Step 5: Evaluate the performance of the trained steel surface defect detection model to obtain the model's detection capability for defects of different sizes, which can be used for real-time detection in steel production lines.

2. The method for detecting defects on steel surfaces according to claim 1, characterized in that, The multi-kernel reparameterized convolution MKRConv consists of multiple types of parallel deep convolution branches, including small kernel convolution, dilated convolution, large kernel convolution, and asymmetric convolution, which simultaneously capture multi-scale defect features and structured defect features. Each depthwise convolutional branch is equipped with an independent batch normalization layer, so that the output branch simultaneously contains the detailed features of the original convolution and the stable features after regularization by the batch normalization layer. After summing the elements of each branch, a standard 1×1 convolution is used to obtain multi-scale features. The multi-branch aggregation process is represented as follows: ； in, For the output of the j-th branch, This is the batch normalization operation corresponding to the j-th branch; During the inference phase, the multi-branch structure of the multi-kernel reparameterized convolution MKRConv is reparameterized into a single depth convolution kernel. Zero-value padding is used to transform the small kernel convolution kernel, dilated kernel convolution kernel, and asymmetric kernel convolution kernel into the size of the large kernel convolution kernel. The parameters of the batch normalization layer are incorporated into the weights and biases of the convolution kernel to obtain multiple convolution kernels with the same configuration. Based on the linear additivity of the convolution kernels, the convolution kernels are summed to obtain a single depth convolution kernel used for inference.

3. The method for detecting defects on steel surfaces according to claim 2, characterized in that, Constructing the MKR-C3K2 module includes: The multi-kernel reparameterized convolution MKRConv is integrated into the core feature extraction module C3K2 of YOLOv11. The multi-kernel reparameterized bottleneck module MKR-BottleNeck is constructed using MKRConv and replaces the original bottleneck module BottleNeck in C3K2. The module conversion is performed based on the value of parameter C3K. When C3K is set to False, it is converted to the MKR-C2F module. At the same time, MKRConv introduces an additional 1×1 convolution branch and a corresponding batch normalization layer branch to meet the differentiated requirements of feature expression and computational efficiency at different feature extraction stages.

4. The method for detecting defects on steel surfaces according to claim 1, characterized in that, The frequency-domain guided spatial adaptive module FSAM includes: Step 2.1: Input feature map Based on this, multiple sets of decomposed strip-shaped depthwise convolution kernels are used to extract multi-scale features, capturing the texture and shape features of slender defects on the steel surface, thus obtaining the features of the multi-scale convolution branches. : ； in, These are batch size, number of channels, height, and width, respectively. For the number of branches, Let be the size of the convolution kernel in the i-th branch. The function represents depthwise convolution; Step 2.2, Features Element-wise addition is performed, and scale-aware features are generated through 1×1 convolution. Then, a frequency domain representation is obtained by two-dimensional Fourier transform. Low-frequency components mainly reflect the overall brightness distribution and large-scale contour information, while high-frequency components mainly respond to edge abrupt changes, fine-grained textures, and local abnormal areas. Step 2.3: Take the amplitude spectrum of the frequency domain representation and average it along the channel dimension to obtain the single-channel spectrum statistics. The frequency domain statistical graph is converted into multi-channel spatial attention weights with corresponding branch numbers using a learnable mapping. For each channel's spatial attention graph, the attention weights are obtained by activation using the Sigmoid function. ; Step 2.4: Weight attention Features corresponding to multi-scale convolution branches Element-wise weighted summaries are then fused using 1×1 convolutions to generate attention feature maps. element-wise comparison with the original input features Multiply to generate scale-adaptive enhanced feature maps. .

5. The method for detecting defects on steel surfaces according to claim 1, characterized in that, In the Bidirectional Guidance Fusion Module (BGFM), the higher-level guidance gating process includes: From high-level feature maps In the middle, semantically relevant channel weights are extracted to guide low-level features in preserving key details. This represents the number of channels in the high-level feature map. The number of channels in the high-level features is adjusted to match that of the low-level features using standard 1×1 convolution and 3×3 depthwise convolution, and the channel-aligned feature map is denoted as... The calculation process is as follows: ； Global average pooling is used to aggregate spatial information into channel-dimensional global semantic information. This global semantic information is then sent to a bottleneck module that focuses on learning the nonlinear correlations between channels. Semantically relevant channel weights are generated using the Sigmoid activation function. , This represents the number of channels in the low-level feature map.

6. The method for detecting defects on steel surfaces according to claim 5, characterized in that, The bottleneck module consists of two PWConv layers, with a ReLU activation function in between to introduce nonlinear transformation, and a dimensionality reduction coefficient r is set to control the number of intermediate channels; The channel weight The calculation process is as follows: ； in, The ReLU activation function is used. It is the Sigmoid activation function. This is global average pooling.

7. The method for detecting defects on steel surfaces according to claim 6, characterized in that, In the Bidirectional Guided fusion module (BGFM), the low-level guided gating process includes: From low-level feature maps In the process, spatial location weights are extracted to guide high-level features to focus on target defect areas; Convolutional layers are used to focus on the contextual information of low-level features, and their multi-channel information is weighted and aggregated into single-channel features. Spatial weights are generated in the spatial dimension using the Sigmoid activation function. ; The channel weight The calculation process is as follows:

8. The method for detecting defects on the surface of steel according to claim 7, characterized in that, In the bidirectional guided fusion module BGFM, the feature complementarity aggregation method is as follows: The obtained channel weights and spatial weights are applied to low-level features and high-level features respectively. The channel weights of high-level semantics are used to filter low-level details and retain semantically related spatial information. The spatial weights of low-level details are used to filter high-level semantics, so that semantic information is focused on the target location. The enhanced high- and low-level features are concatenated along the channel dimension to obtain the final fused feature. The expression is: ； in, This is a concatenation function.

9. The method for detecting defects on the surface of steel according to claim 1, characterized in that, Based on the publicly available steel surface defect detection dataset, it is divided into a training set, a validation set, and a test set in a ratio of 7:1.5:1.

5. The training hyperparameters were set as follows: total number of training epochs: 300; batch size: 16; input image size: 640×640; optimizer: SGD; initial learning rate: 1e-2; final learning rate: 1e-4; weight decay: 5e-4.

10. The method for detecting defects on the surface of steel according to claim 1, characterized in that, The performance evaluation indicators include: average precision, multi-scale average precision, recall rate, number of parameters and computational cost; steel surface defects are divided into three categories—small, medium and large—according to the target size, and the performance evaluation indicators for each category of defects are calculated to obtain the model's detection capability for defects of different sizes.

Images