Lightweight multi-scale dynamic convolution steel surface defect detection method

Abstract

The invention discloses a lightweight multi-scale dynamic convolution steel surface defect detection method, which comprises the following steps: firstly, constructing a multi-level feature pyramid backbone network based on a dynamic multi-scale perception C3k2PKI module, reserving detail textures of an image through shallow high-resolution features, balancing semantics and positioning precision through middle-level features, and constructing a multi-level feature pyramid backbone network; a global context is captured by deep low-resolution features, and multi-scale convolution kernel and attention weighting are combined; then the feature map is transmitted to a neck network, and the neck network further performs multi-scale feature fusion and adaptive weighting on the feature map; and finally, the fused feature map is transmitted to a head network, and an LSCD detection head of the head network is responsible for final positioning and classification of defects and outputting a detection result through channel compression, cross-resolution feature alignment splicing and grouping convolution. According to the method, the recall rate is improved, and the key positioning precision is effectively maintained and improved while the parameter quantity is reduced.

Description

Invention Field

[0001] This invention belongs to the field of material surface inspection technology, specifically relating to a lightweight, multi-scale dynamic convolutional steel surface defect detection method. Background Technology

[0002] In fields such as automobile manufacturing, shipbuilding, aerospace, and precision machinery, defects such as pits, inclusions, indentations, or pitting on the surface of steel plates not only reduce the material's corrosion resistance, wear resistance, and fatigue strength, but may also lead to structural failure, resulting in safety hazards. Current detection methods include manual inspection, machine learning, and deep learning. Manual inspection methods rely on visual inspection, which is difficult to meet the real-time online inspection requirements of high-speed rolling production lines. Machine learning-based detection methods, such as support vector machines and decision trees, are often limited by manual feature design and have weak generalization ability. Deep learning-based object detection methods can be mainly divided into two-stage and one-stage end-to-end methods. The core idea of ​​two-stage is to decompose the detection process into two sequentially executed stages: first, candidate target regions are generated, and then these regions are classified and bounding box regressed. Representative models include R-CNN (Region-CNN), Fast R-CNN, Faster R-CNN, Mask R-CNN, and Cascade R-CNN. Girshick et al. pioneered the introduction of CNNs into object detection, generating candidate regions through Selective Search and independently extracting CNN features, laying the foundation for deep learning methods, but with low efficiency. Choi et al. used the Fast R-CNN deep learning model, which can classify rail surface conditions and predict internal rail defects. Fast R-CNN proposes sharing convolutional feature maps and RoI pooling layers to achieve unified feature extraction of candidate regions and integrates classification and regression into a single network for end-to-end training; however, its candidate region generation remains an independent and slow module, resulting in a speed bottleneck in the detection process. Xu et al. proposed a steel surface defect detection model based on an improved Faster R-CNN. This method replaces the ResNet50 network with a RegNet network, employs a transformer spatial attention mechanism to enable the network to more accurately focus on the target region, and improves detection accuracy through transfer learning, multi-scale training, and cosine annealing learning rate adjustment strategies. Wang H et al. proposed a novel track surface defect detection network based on Mask R-CNN. This network introduces a dual fusion path to achieve deep fusion of feature maps at each stage and adopts a novel evaluation metric, CIOU, which achieves a certain balance between accuracy and speed, but is still inferior to other single-stage object detection models. Wang Y et al. developed a method based on an improved Cascade R-CNN. This method uses ResNet and PAFPN as feature extraction networks, replaces ROI pooling with ROI Align, and uses bicubic interpolation instead of bilinear interpolation, improving the challenge of detecting metal surface defects with blurred backgrounds.Overall, the two-stage detection method has good detection accuracy, but its complex cascaded structure and computational redundancy lead to low deployment efficiency.

[0003] End-to-end One-Stage object detection merges object localization and classification into a single-stage detection architecture. The core idea is to eliminate the time-consuming candidate region generation step and directly predict the class probability and bounding box coordinates of all objects in the input image at once through dense sampling or preset anchor points. Typical methods include SSD, the YOLO series, RetinaNet, MirageNet, and MobileNet. SSD uses knowledge distillation to optimize the ResNet50 backbone network, employs feature fusion technology to identify small-sized defects, and utilizes channel attention mechanisms to reduce computational costs. Xie et al. designed a lightweight multi-scale feature fusion module, C2f_LMSMC, which combines EGAM attention and stacked convolutions for cross-dimensional interaction, improving the detection accuracy of small defects. Huang et al. proposed the SSA-YOLO model, which integrates the Swing Transformer and Adaptive Spatial Feature Fusion (ASFF) technology into YOLOv5, significantly improving the detection accuracy of the most difficult-to-detect crack defects on the NEU-DET dataset. To address the insufficient robustness of existing methods for complex defect morphologies, Lu et al. proposed an improved YOLOv8 network, WSS-YOLO. This method introduces the WIoU loss function and the C2f-DSC dynamic serpentine convolution module, enabling the model to adaptively adjust its receptive field and providing a new technical approach for real-time defect detection in industrial settings. Ma et al. proposed ELA-YOLO, which introduces a linear attention mechanism into the network, enhancing the model's representational capabilities while controlling computational complexity, enabling accurate identification of multi-scale surface defects with limited computational resources. To address the issue of steel surface defects being susceptible to interference from lighting, texture, and noise in complex industrial environments, making real-time and accurate detection difficult, Zhao et al. proposed a lightweight network based on YOLOv8n, MSAF-YOLOv8n. This method improves the model's feature fusion capability by introducing the multi-scale adaptive fusion module MS-AFB and the DCA-GSPPFCSPC structure enhanced by dynamic coordinate attention. To address the issue of small-scale surface defects in steel bridge welds, Luo et al. proposed a lightweight network, SFW-YOLO, based on YOLOv8s. This method enhances the recognition of small-sized defects while optimizing detection accuracy for multi-scale defects and complex backgrounds by introducing a high-resolution P2 layer to capture fine-grained features and constructing a DyHead dynamic detection head to strengthen multi-scale attention. Xie et al. contributed to solving the trade-off between lightweight design and multi-scale feature fusion in existing steel surface defect detection networks by introducing a lightweight multi-scale module LighterMSMC, a heavily parameterized feature fusion pyramid DE-FPN, and a grouped convolutional Efficient Head.To address the issues of minute defects and class imbalance on steel surfaces, Akhyaret et al. proposed an upgraded RetinaNet defect detection module. Its feature pyramid network and anchor point strategy improved accuracy, but at a higher cost. Fu et al. achieved high accuracy while significantly reducing computational cost (FLOPs) through dynamic sparse convolution techniques and a hierarchical feature reuse mechanism. This method employs hardware-friendly sparse operators and edge-device-oriented neural architecture search (NAS) to achieve a substantial reduction in FLOPs. Subsequently, Fu et al. proposed an improved method. Their proposed Frequency Recalibration Module (FREM) was the first to combine frequency domain analysis with channel attention; their designed Cascaded Dilated Convolutional Pyramid (CDCP) compressed FLOPs while maintaining receptive field coverage of multi-scale lesions. Although one-stage detection algorithms simplify the training process and are fast and easy to deploy, making them a common solution for steel defect detection, they still suffer from problems such as ignoring small targets and relatively low localization accuracy.

[0004] Overall, steel surface defect detection still faces two main challenges: 1) Static computational models are difficult to adapt to scenarios of varying complexity, exhibiting computational redundancy in simple scenarios and insufficient detection capabilities in complex scenarios. 2) The contradiction between the rich detail and weak semantic abstraction of shallow features and the rich semantic content but low localization accuracy of deep features restricts the detection accuracy of minute defects on steel surfaces. Summary of the Invention

[0005] The purpose of this invention is to overcome the above-mentioned problems by proposing a lightweight multi-scale dynamic convolution method for detecting defects on the surface of steel that improves recall, reduces the number of parameters, and effectively maintains and improves key positioning accuracy.

[0006] The present invention provides a lightweight, multi-scale, dynamic convolutional steel surface defect detection method, comprising the following steps:

[0007] Step 1: Construct a multi-level feature pyramid backbone network based on the C3k2_PKI module with dynamic multi-scale perception. The image retains details and textures through shallow high-resolution features at the 1 / 8 scale, balances semantics and localization accuracy through medium-level features at the 1 / 16 scale, and captures global context through deep low-resolution features at the 1 / 32 scale. Combined with 3×3 to 11×11 multi-scale convolutional kernels and attention weighting.

[0008] Step 2: The feature map is passed to the neck network, which further performs multi-scale feature fusion and adaptive weighting on the feature map;

[0009] Step 3: The fused feature map is passed to the head network. The LSCD detection head of the head network uses channel compression, cross-resolution feature alignment and splicing, and group convolution to be responsible for the final localization and classification of defects and output the detection results.

[0010] The aforementioned lightweight multi-scale dynamic convolutional steel surface defect detection method, wherein step 1 involves constructing a multi-level feature pyramid backbone network based on the C3k2_PKI module of dynamic multi-scale perception:

[0011] 1) Fine-grained feature fusion of multi-scale dynamic convolution:

[0012] For the input feature map (s {P3, P4, P5}) perform multi-scale convolution operations in parallel, as shown in Equation (1):

[0013] (1)

[0014] in express Convolutional kernels, each convolutional branch is responsible for capturing local structural and global semantic information at its corresponding scale;

[0015] First, the output of each scale branch is statistically analyzed using global average pooling (GAP), as shown in equation (2):

[0016] (2)

[0017] Where, in the formula and These represent the height and width of the feature maps at different feature levels, respectively.

[0018] Then, after a series of fully connected layers and non-linear activation functions, the corresponding weight vector is generated. The ReLU activation function is shown in equation (3):

[0019] (3)

[0020] Next, the weights of each branch are normalized using the softmax function to ensure that the contribution ratio of each scale feature can be learned and dynamically adjusted, as shown in equation (4):

[0021] (4)

[0022] Where, in the formula , For the weights of the fully connected layer of the SE module, The intermediate layer is biased for channel attention. Bias for the channel attention output layer. satisfy ;

[0023] Finally, the features of each branch are weighted and summed according to their respective weights to form the refined features after fusion. The calculation formula is shown in equation (5):

[0024] (5);

[0025] 2) Dynamic parameter control mechanism:

[0026] First, the input feature analysis performs global average pooling on the multi-scale feature maps P3, P4, and P5 output by the backbone network to generate channel description vectors as shown in equation (6), where Number of channels;

[0027] (6)

[0028] The description vectors of the three scales are then concatenated into a global description vector, as shown in equation (7):

[0029] , (7)

[0030] Then input a two-layer fully connected MLP network (hidden layer dimension) ), output rating vector S = [ ], corresponding to the salience scores of small, medium, and large objectives respectively, as shown in equation (8):

[0031] (8)

[0032] Where, in the formula Here is the hidden layer weight matrix of the MLP. This is the weight matrix of the MLP output layer. This is used to bias the hidden layer of global features. Bias for the importance score output layer;

[0033] Finally, the scores are normalized, and the score distribution is constrained using the Softmax function as shown in equation (9):

[0034] = , k∈{1,2,3} (9)

[0035] in, , , These represent the importance weights of small, medium, and large objectives, respectively, satisfying... ;

[0036] Dynamic parameter adjustment is based on importance weights. To handle real-time computational load, the kernel size and computational intensity are adaptively selected. If the inference latency exceeds the threshold, the 11×11 kernel is automatically replaced with a 7×7 kernel, as shown in Equation (10):

[0037] (10);

[0038] The above-mentioned lightweight multi-scale dynamic convolutional steel surface defect detection method includes step 2, multi-scale feature fusion and adaptive weighting of the neck network:

[0039] 1) Bidirectional feature pyramid construction: adopting the FPN+PAN structure, firstly, the deep semantic features P5 and P4 are upsampled and fused with the shallow high-resolution feature P3 layer by layer through the top-down path to preserve the details and textures, and then the shallow features are downsampled and aggregated with the deep features through the bottom-up path to strengthen the global context information and form a bidirectional closed loop of multi-scale feature interaction.

[0040] 2) Cross-stage dynamic weight fusion: A lightweight channel attention module is introduced during the feature fusion process to adaptively assign weights to features from different levels, as shown in equation (11):

[0041] = , i∈{P3,P4,P5} (11)

[0042] Where σ is the Sigmoid activation function. This represents the convolutional layer after global average pooling. The weighted features... = Suppress redundant features while preserving important channel information;

[0043] 3) Multi-scale feature alignment and output: The feature resolution of each level is unified to 1 / 8 scale through bilinear interpolation, and the number of channels is aligned using 1×1 convolution. Finally, a set of feature maps P3, P4 and P5 with consistent scale and enhanced semantics are output.

[0044] The aforementioned lightweight multi-scale dynamic convolutional steel surface defect detection method, wherein step 3 describes the LSCD (lightweight shared convolutional detection head) of the head network:

[0045] 1) Cross-stage feature fusion: First, feature maps P3 (1 / 8 scale), P4 (1 / 16 scale), and P5 (1 / 32 scale) are extracted from the backbone network, with the number of channels being C3=256, C4=512, and C5=1024, respectively.

[0046] Then, bilinear interpolation is performed on P4 and P5 to unify the resolution to 1 / 8 of the scale of P3. A 1×1 convolution is used to compress the number of channels of P4 and P5 to 256, which is consistent with P3, as shown in Equation (12):

[0047] ( , (12)

[0048] Finally, output the aligned feature map. , , ;

[0049] The bidirectional feature interaction is divided into bottom-up and top-down paths, which model cross-stage dependencies through a lightweight attention mechanism. The bottom-up path concatenates the P3 feature with the compressed P4 and P5 features to generate a global context vector, as shown in Equation (13).

[0050] = ⊕ ⊕ (13)

[0051] Long-range dependencies are modeled using a lightweight Transformer encoder with 4 attention heads and a hidden layer dimension of 128, as shown in Equation (14):

[0052] = Transformer( (14)

[0053] The top-down path will F trans The data is divided into three parts, which are added to the original features P3, P4, and P5 respectively to achieve information feedback, as shown in equation (15):

[0054] = + ( ),s∈{3,4,5} (15)

[0055] Channel compression and enhancement strategies retain key information by reducing redundant channels. Dimensionality reduction through grouped convolution is used to enhance the concatenated features. ∈ The number of channels is compressed to 128 using grouped convolution with 8 groups, as shown in equation (16):

[0056] F fused= GroupConv 3×3 ( ,Group = 8 ) (16)

[0057] Then, through dynamic channel recalibration, a lightweight SE module (Squeeze-and-Excitation) is introduced to dynamically adjust the channel weights as shown in equation (17):

[0058] (17)

[0059] 2) Parameter optimization: For each output channel of the detection head, the mean absolute value of its weight gradient is calculated as shown in equation (18):

[0060] (18)

[0061] Where C is the number of channels at the current level. Let be the weight gradient of the Cth channel; remove channels with gradient sensitivity lower than the mean μ, and retain important channels;

[0062] Finally, the channels with gradient magnitudes below the mean of the LSCD detector head are sparsified, and the retention rate r is calculated as shown in equation (19):

[0063] r=1 - (19);

[0064] Compared with existing technologies, this invention has significant advantages. As can be seen from the above technical solution: First, this invention constructs a multi-level feature pyramid backbone network based on the C3k2_PKI module for dynamic multi-scale perception. The image retains detailed texture through shallow high-resolution features (1 / 8 scale), medium-level features (1 / 16 scale) balance semantics and localization accuracy, and deep low-resolution features (1 / 32 scale) capture global context. Combined with multi-scale convolutional kernels (3×3 to 11×11) and attention weighting, it not only enhances the feature representation ability of small and multi-scale defects, increasing the recall rate to 72.70%, but also achieves adaptive matching of cross-scale receptive fields, thereby enhancing the generalization ability of this invention for defects of multiple sizes. Next, the feature map is passed to the neck network, which further processes the features. Through the fusion and weighting of multi-scale features, the robustness of this invention to defects of different sizes is improved. Finally, the fused feature map is passed to the head network. The LSCD detection head of the head network uses channel compression, cross-resolution feature alignment and stitching, and grouped convolution to be responsible for the final localization and classification of defects, and outputs the detection results. This reduces the number of parameters while enhancing localization accuracy, adapting to the high robustness requirements of industrial quality inspection. C3k2_PKI and LSCD build an end-to-end detection architecture on the basis of the YOLOv11 framework, achieving an organic balance between multi-granularity feature representation and computational efficiency. This invention co-optimizes multi-scale dynamic perception and lightweight cross-stage fusion. This collaborative approach significantly reduces the number of model parameters and computational complexity while maintaining high detection accuracy, meeting the requirements for real-time steel surface defect detection in industrial scenarios. This invention achieves a superior balance between detection accuracy, recall, and model lightweighting. Attached Figure Description

[0065] Figure 1 : Network structure diagram of the present invention;

[0066] Figure 2 C3k2_PKI module structure diagram;

[0067] Figure 3 LSCD detector head structure diagram;

[0068] Figure 4 Loss curve comparison chart;

[0069] Figure 5 mAP curve comparison chart;

[0070] Figure 6 : Visualizations of various ablation techniques using Gradcam. Detailed Implementation

[0071] The following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effectiveness of the lightweight multi-scale dynamic convolution steel surface defect detection method proposed in this invention:

[0072] Example 1:

[0073] See Figure 1 A lightweight, multi-scale dynamic convolutional steel surface defect detection method includes the following steps:

[0074] Step 1: Construct a multi-level feature pyramid backbone network based on the C3k2_PKI module with dynamic multi-scale perception. The C3k2_PKI module is one of the core components of this invention. This module adopts a parallel multi-scale convolution design to perform fine-grained feature extraction on different input features, effectively enhancing the model's ability to perceive defects on the steel surface. The module integrates dynamic convolution weight adjustment and residual connections, significantly improving feature representation ability and robustness of cross-scale defect detection while maintaining a lightweight structure.

[0075] 1) Fine-grained feature fusion of multi-scale dynamic convolution: The C3k2_PKI module introduces the idea of ​​multi-scale convolution kernels (see...). Figure 2 By performing fine-grained convolution operations on different scales of the input image, rich feature representations can be extracted.

[0076] For the input feature map (s {P3, P4, P5}) perform multi-scale convolution operations in parallel, as shown in Equation (1):

[0077] (1)

[0078] in express Convolutional kernels, each convolutional branch is responsible for capturing local structural and global semantic information at its corresponding scale;

[0079] After obtaining the features at each scale, to avoid information redundancy that may be introduced by simple splicing or summation operations, a dynamic weight calculation mechanism was designed, and a lightweight attention module was introduced to generate adaptive weights; first, the output of each scale branch was statistically analyzed by global average pooling (GAP), as shown in Equation (2):

[0080] (2)

[0081] Where, in the formula and These represent the height and width of the feature maps at different feature levels, respectively.

[0082] Then, after a series of fully connected layers and non-linear activation functions, the corresponding weight vector is generated. The ReLU activation function is shown in equation (3):

[0083] (3)

[0084] Next, the weights of each branch are normalized using the softmax function to ensure that the contribution ratio of each scale feature can be learned and dynamically adjusted, as shown in equation (4):

[0085] (4)

[0086] Where, in the formula , For the weights of the fully connected layer of the SE module, The intermediate layer is biased for channel attention. Bias for the channel attention output layer. satisfy ;

[0087] Finally, the features of each branch are weighted and summed according to their respective weights to form the refined features after fusion. The calculation formula is shown in equation (5):

[0088] (5);

[0089] In this way, cross-resolution feature interaction is supported, sensitivity to small targets is improved, and the model can effectively learn about objects of various scales.

[0090] 2) Dynamic parameter control mechanism: First, the input feature analysis performs global average pooling on the multi-scale feature maps (P3, P4, P5) output by the backbone network to generate channel description vectors as shown in equation (6), where Number of channels;

[0091] (6)

[0092] The description vectors of the three scales are then concatenated into a global description vector, as shown in equation (7):

[0093] , (7)

[0094] Then input a two-layer fully connected MLP network (hidden layer dimension) ), output rating vector S = [ ], corresponding to the salience scores of small, medium, and large objectives respectively, as shown in equation (8):

[0095] (8)

[0096] Where, in the formula Here is the hidden layer weight matrix of the MLP. This is the weight matrix of the MLP output layer. This is used to bias the hidden layer of global features. Bias for the importance score output layer;

[0097] Finally, the scores are normalized, and the score distribution is constrained using the Softmax function as shown in equation (9):

[0098] = , k∈{1,2,3} (9)

[0099] in, , , These represent the importance weights of small, medium, and large objectives, respectively, satisfying... ;

[0100] Dynamic parameter adjustment is based on importance weights. To handle real-time computational load, the kernel size and computational intensity are adaptively selected. If the inference latency exceeds the threshold, the 11×11 kernel is automatically replaced with a 7×7 kernel, as shown in Equation (10):

[0101] (10);

[0102] Step 2, Multi-scale feature fusion and adaptive weighting of the neck network: The neck network receives multi-level features P3, P4 and P5 output by the backbone network. Through multi-level feature fusion and dynamic weighting mechanism, the robustness of the model to defects of different scales is enhanced, providing feature maps with better structure and richer information for the accurate detection of the subsequent head network.

[0103] 1) Bidirectional feature pyramid construction: adopting the FPN+PAN structure, firstly, the deep semantic features P5 and P4 are upsampled and fused with the shallow high-resolution feature P3 layer by layer through the top-down path to preserve the details and textures, and then the shallow features are downsampled and aggregated with the deep features through the bottom-up path to strengthen the global context information and form a bidirectional closed loop of multi-scale feature interaction.

[0104] 2) Cross-stage dynamic weight fusion: A lightweight channel attention module is introduced during the feature fusion process to adaptively assign weights to features from different levels, as shown in equation (11):

[0105] = , i∈{P3,P4,P5} (11)

[0106] Where σ is the Sigmoid activation function. This represents the convolutional layer after global average pooling. The weighted features... = Suppress redundant features while preserving important channel information;

[0107] 3) Multi-scale feature alignment and output: The feature resolution of each level is unified to 1 / 8 scale through bilinear interpolation, and the number of channels is aligned using 1×1 convolution. Finally, a set of feature maps P3, P4 and P5 with consistent scale and enhanced semantics are output, providing the head network with fused features that take into account both details and semantics.

[0108] This step significantly improves the model's adaptability to defects of various sizes on steel surfaces, enabling highly robust feature representation in complex industrial scenarios.

[0109] Step 3, the LSCD (lightweight shared convolutional detection head) detection head of the head network (see...) Figure 3 This detection head is responsible for the final defect localization and classification. It adopts a multi-branch design to process feature maps of different scales. Each branch contains a lightweight convolution module and a dynamic receptive field enhancement module. It adaptively focuses on the defect region through deformable convolution and attention mechanism. The outputs of each branch are fused after scale alignment. Finally, the defect category and bounding box are output through classification and regression layers. This detection head can maintain high detection accuracy in complex backgrounds. It is especially suitable for multi-scale and multi-morphological defect detection tasks on steel surfaces.

[0110] 1) Cross-stage feature fusion: First, feature maps P3 (1 / 8 scale), P4 (1 / 16 scale), and P5 (1 / 32 scale) are extracted from the backbone network, with the number of channels being C3=256, C4=512, and C5=1024, respectively.

[0111] Then, bilinear interpolation is performed on P4 and P5 to unify the resolution to 1 / 8 of the scale of P3. A 1×1 convolution is used to compress the number of channels of P4 and P5 to 256, which is consistent with P3, as shown in Equation (12):

[0112] ( , (12)

[0113] Finally, output the aligned feature map. , , ;

[0114] The bidirectional feature interaction is divided into bottom-up and top-down paths, which model cross-stage dependencies through a lightweight attention mechanism. The bottom-up path concatenates the P3 feature with the compressed P4 and P5 features to generate a global context vector, as shown in Equation (13).

[0115] = ⊕ ⊕ (13)

[0116] Long-range dependencies are modeled using a lightweight Transformer encoder with 4 attention heads and a hidden layer dimension of 128, as shown in Equation (14):

[0117] = Transformer( (14)

[0118] The top-down path will F trans The data is divided into three parts, which are added to the original features P3, P4, and P5 respectively to achieve information feedback, as shown in equation (15):

[0119] = + ( ),s∈{3,4,5} (15)

[0120] Channel compression and enhancement strategies retain key information by reducing redundant channels. Dimensionality reduction through grouped convolution is used to enhance the concatenated features. ∈ The number of channels is compressed to 128 using grouped convolution with 8 groups, as shown in equation (16):

[0121] F fused = GroupConv 3×3 ( ,Group = 8 ) (16)

[0122] Then, through dynamic channel recalibration, a lightweight SE module (Squeeze-and-Excitation) is introduced to dynamically adjust the channel weights as shown in equation (17):

[0123] (17)

[0124] 2) Parameter optimization: For each output channel of the detection head, the mean absolute value of its weight gradient is calculated as shown in equation (18):

[0125] (18)

[0126] Where C is the number of channels at the current level. Let be the weight gradient of the Cth channel; remove channels with gradient sensitivity lower than the mean μ, and retain important channels;

[0127] Finally, the channels with gradient magnitudes below the mean of the LSCD detector head are sparsified, and the retention rate r is calculated as shown in equation (19):

[0128] r=1 - (19);

[0129] This invention adopts an end-to-end detection architecture integration and balanced training, systematically integrating C3k2_PKI and LSCD into the YOLOv11 framework to build a complete end-to-end defect detection system, and through joint training and optimization strategies, achieves an organic balance between multi-granularity feature representation and computational efficiency.

[0130] 1) Construction of an end-to-end steel surface defect detection architecture with C3k2_PKI as the core of feature extraction and fusion and LSCD as the core of detection inference: The C3k2_PKI module is used in the backbone network of the YOLOv11 framework. Its function is to enable the network to capture multi-granular features from fine texture to macro structure in the early and middle stages of feature extraction through parallel multi-scale convolution kernels and dynamic weight mechanism, so as to provide rich and discriminative feature representations for subsequent detection.

[0131] In the feature pyramid network part, the C3k2_PKI module is used for feature fusion. When fusing features from different levels of the backbone network, the multi-scale and dynamic characteristics of C3k2_PKI are further utilized to enhance the representation ability of the feature pyramid and ensure that defect information at different scales can be effectively transmitted and enhanced.

[0132] The YOLOv11 framework's detection head is replaced with an LSCD detection head. Utilizing LSCD's cross-stage feature fusion, bidirectional feature interaction, and channel optimization strategies, it efficiently decodes multi-scale features from the neck network, integrates feature information from P3, P4, and P5, and strengthens key features and suppresses redundancy through lightweight Transformer and SE modules, ultimately achieving accurate defect classification and localization.

[0133] At this point, an end-to-end steel surface defect detection architecture with C3k2_PKI as the core for feature extraction and fusion and LSCD as the core for detection inference has been completed.

[0134] 2) Balanced training and optimization strategies

[0135] To achieve a balance between "multi-granularity feature representation" and "computational efficiency," the following strategies are implemented during training:

[0136] An end-to-end training approach is adopted, using a composite loss function that includes bounding box regression, classification, and target confidence. In the early stages of training, the focus is on building the model's expressive power, allowing the C3k2_PKI module and LSCD detection head to fully learn complex multi-scale features.

[0137] In training, computational load monitoring is introduced. According to the dynamic parameter control mechanism defined in formula (10) in step 1, the inference latency of the model is monitored in real time. When the average inference latency during training or validation exceeds the preset threshold, the kernel size is automatically downgraded (the maximum kernel is switched from 11×11 to 7×7). This is used as an online regularization method to constrain the computational complexity of the model and guide the model to become lightweight while ensuring performance.

[0138] After the model training converges, the gradient sensitivity channel pruning described in formulas (18) and (19) in step 3 is applied to remove redundant channels with low gradient contribution in the LSCD detector head and the entire network, which significantly reduces the number of parameters and computation.

[0139] Through systematic architectural integration and targeted balanced training strategies, the C3k2_PKI module and LSCD detection head are deeply integrated into the YOLOv11 framework, forming a powerful end-to-end detection system. This system not only enhances feature representation capabilities through multi-scale dynamic convolution and refined feature fusion mechanisms, but also ensures computational efficiency through dynamic adjustment, structural optimization, and post-quantization techniques, ultimately achieving a balance between detection performance and running speed.

[0140] Experiment 1: Performance Testing

[0141] (1) Experimental setup, dataset and evaluation metrics

[0142] On the Windows 11 operating system, Python 3.10.16 and PyTorch 2.5.1 were used as the deep learning framework, PyCharm was used as the development environment, the system CPU was an Intel Core i5-13500H, the GPU was an RTX 4050, the graphics card had 6 GB of RAM, and CUDA 12.4 was used to accelerate the GPU. The hyperparameters of this invention are shown in Table 1:

[0143] Table 1 Hyperparameters of the present invention

[0144] Hyperparameters Hyperparameter information Input resolution (Image Size) 640×640 Training epochs 300 Batch Size 8 Mosaic 1.0 Mosaic data augmentation disabling timing (Close Mosaic) 0 Number of parallel processes (Workers) 4 Optimizer SGD Learning rate (Initial Learning Rate) 0.01 Momentum factor 0.937 Weight Decay Factor 0.0005

[0145] NEU-DET and GC10-DET were used as test datasets. The NEU-DET steel defect dataset contains 6 defect categories: Cracking, Inclusion, Patches, Pitted Surface, Rolled-in Scale, and Scratches, with 300 images for each category, totaling 1800 images. The GC10-DET steel plate surface defect dataset contains 10 defect categories: Punching, Welding line, Crescent gap, Water spot, Oil spot, Silk spot, Inclusion, Rolled pit, Crease, and Waist folding. NEU-DET can be downloaded at https: / / drive.google.com / open?id=1qrdZlaDi272eA79b0uCwwqPrm2Q_WI3k. GC10-DET can be downloaded at https: / / github.com / lvxiaoming2019 / GC10-DET-Metallic-Surface-Defect-Matasets. During the experiments, to ensure the rigor and objectivity of the validation results, five-fold cross-validation was used to establish training, testing, and validation sets for both the NEU-DET and GC10-DET datasets.

[0146] Precision (P), recall (R), mean precision (mAP), number of parameters, and floating-point operations (FLOPs) are used as metrics to evaluate model performance. mAP50 represents the mean precision calculated with an intersection-over-union (IoU) threshold of 0.5; mAP50~95 is the average of multiple mAP values ​​within the IoU threshold range of 0.5 to 0.95. Number of parameters refers to the total number of learnable parameters that need to be optimized during model training, while FLOPs quantify the computational complexity of the model, representing the total number of floating-point operations required to complete the forward propagation.

[0147] (2) Ablation test

[0148] To observe the performance of different modules in this invention, ablation experiments were conducted on the NEU-DET dataset according to the settings in Table 1, using YOLOv11 as the baseline (denoted as S0). The performance statistics of the ablation experiments are shown in Table 2.

[0149] As can be seen from Table 2:

[0150] 1) Since the C3k2_PKI module can adaptively fuse features at multiple scales, it enhances the model's ability to perceive small defects and complex backgrounds. The mAP50 of S1 is improved by 0.29%, but due to the increased computational cost of large convolutional kernels, the mAP50~95 decreases slightly by 0.35%.

[0151] 2) The number of parameters in S2 is reduced by 6.20%, which indicates that the lightweight cross-stage detection head LSCD reduces redundant parameters through channel compression and group convolution, while retaining detailed information of cross-resolution features, thereby improving detection accuracy.

[0152] 3) This invention achieves more robust feature representation through collaborative optimization while maintaining the number of parameters at 2.42M. Although mAP50 decreases slightly due to the reallocation of computing resources (78.00%→76.81%), its mAP50~95 reaches a maximum of 44.86% (an improvement of 0.16% over the baseline S0), especially in the critical defect category.

[0153] Table 2 Ablation Experiment

[0154] method C3k2_PKI LSCD mAP50(%) mAP 50~95% Number of parameters (M) S0 × × 78.00 ± 1.00 44.70 ± 1.10 2.58 S1 √ × 78.29 ± 0.98 44.35 ± 1.69 2.58 S2 × √ 78.62 ± 1.40 45.97 ± 1.10 2.42 S3 (This invention) √ √ 76.81 ± 1.25 44.86 ± 0.33 2.42

[0155] In addition, loss curves were created to visually observe the performance changes and status of the model during training and validation. Figure 4 ) and mAP curve ( Figure 5 ).exist Figure 4 In this study, the loss of the present invention (S3) decreases significantly faster than that of S0, indicating that the present invention converges more rapidly. The loss curve of the present invention also exhibits smoother fluctuations, suggesting a more stable model and reduced risk of overfitting. (Observation) Figure 5 As can be seen, the mAP curve shows that the index of S3 improves faster than that of S0 and the mAP curve is smoother overall, which indicates that the model stability is enhanced.

[0156] Observe the Gradcam heatmaps of six defects in the NEU-DET dataset ( Figure 6 ).observe Figure 6 It can be seen that the present invention is more comprehensive and accurate in identifying key defect features.

[0157] In summary, both C3k2_PKI and LSCD play positive roles. The feature fusion strategy of this invention, which incorporates C3k2_PKI and LSCD, has a synergistic effect and provides comprehensive detection capabilities for targets at different scales.

[0158] (3) Performance comparison experiment

[0159] Seven object detection models—YOLOv3-tiny, YOLOv5n, YOLOv6n, YOLOv8n, YOLOv9t, YOLOv10n, and YOLOv11—were selected as comparison methods. A five-fold crossover experiment was used to verify the advantages of the improved model presented in this paper. The test results on the NEU-DET and GC10-DET datasets are shown in Table 4.

[0160] Table 3. Performance statistics of comparative experiments on NEU-DET and GC10-DET datasets

[0161]

[0162] As shown in Table 3, the present invention achieves an mAP50-95 of 44.86% on the NEU-DET dataset, the highest among the compared models. Its parameter count (2.42M) is 41.8% lower than YOLOv6n (4.16M), which has the closest detection accuracy, and its computational cost (FLOPs=6.90G) is 40.0% lower than YOLOv6n (11.50G), indicating that its lightweight design significantly optimizes resource consumption while maintaining accuracy. Compared to YOLOv11, the present invention improves mAP50-95 by 0.16% (44.70%→44.86%), while mAP50 decreases by 1.19% (78.00%→76.81%). This phenomenon suggests that the model is more robust under stricter IoU thresholds, but under looser thresholds, lightweight compression may sacrifice some classification confidence.

[0163] Meanwhile, this invention demonstrates comprehensive performance advantages on the GC10-DET dataset: achieving the highest mAP50 (64.92%) and mAP50~95 (33.39%), while also boasting the best recall (63.58%). Only the precision (66.89%) is slightly lower than the best-performing YOLOv11 (68.38%), outperforming other comparative algorithms. More importantly, this invention exhibits the lowest standard deviation in both mAP50~95 and recall, demonstrating its excellent stability and robustness. Furthermore, in terms of model lightweighting, this invention maintains the lowest number of parameters (2.42M) while also keeping computational cost (6.90G) relatively low, further confirming its high efficiency in terms of computational complexity. In summary, this invention achieves lightweight characteristics while maintaining optimal overall accuracy and highest stability, demonstrating a balance between accuracy and efficiency in steel surface defect detection tasks.

[0164] The above are merely preferred embodiments of the present invention and do not impose any limitations on the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from any technical solution of the present invention shall still fall within the scope of the technical solution of the present invention.

Claims

1. A method for detecting surface defects in lightweight, multi-scale dynamic convolutional steel, comprising the following steps: Step 1: Construct a multi-level feature pyramid backbone network based on the C3k2_PKI module with dynamic multi-scale perception. The image retains details and textures through shallow high-resolution features at the 1 / 8 scale, balances semantics and localization accuracy through medium-level features at the 1 / 16 scale, and captures global context through deep low-resolution features at the 1 / 32 scale. Combined with 3×3 to 11×11 multi-scale convolutional kernels and attention weighting. Step 2: The feature map is passed to the neck network, which further performs multi-scale feature fusion and adaptive weighting on the feature map; Step 3: The fused feature map is passed to the head network. The LSCD detection head of the head network uses channel compression, cross-resolution feature alignment and splicing, and group convolution to be responsible for the final localization and classification of defects and output the detection results.

2. The lightweight multi-scale dynamic convolutional steel surface defect detection method as described in claim 1, wherein step 1 involves constructing a multi-level feature pyramid backbone network based on the C3k2_PKI module of dynamic multi-scale perception: 1) Fine-grained feature fusion of multi-scale dynamic convolution: For the input feature map (s {P3, P4, P5}) perform multi-scale convolution operations in parallel, as shown in Equation (1): (1)； in express Convolutional kernels, each convolutional branch is responsible for capturing local structural and global semantic information at its corresponding scale; First, the output of each scale branch is statistically analyzed using global average pooling (GAP), as shown in equation (2): (2)； in, In the formula and These represent the height and width of the feature maps at different feature levels, respectively. Then, after a series of fully connected layers and non-linear activation functions, the corresponding weight vector is generated. The ReLU activation function is shown in equation (3): (3)； Next, the weights of each branch are normalized using the softmax function to ensure that the contribution ratio of each scale feature can be learned and dynamically adjusted, as shown in equation (4): (4)； Where, in the formula , For the weights of the fully connected layer of the SE module, The intermediate layer is biased for channel attention. Bias for the channel attention output layer. satisfy ; Finally, the features of each branch are weighted and summed according to their respective weights to form the refined features after fusion. The calculation formula is shown in equation (5): (5)； 2) Dynamic parameter control mechanism: First, the input feature analysis performs global average pooling on the multi-scale feature maps P3, P4, and P5 output by the backbone network to generate channel description vectors as shown in equation (6), where Number of channels; (6)； The description vectors of the three scales are then concatenated into a global description vector, as shown in equation (7): , (7) Then input a two-layer fully connected MLP network (hidden layer dimension) ), output rating vector S = [ ], corresponding to the salience scores of small, medium, and large objectives respectively, as shown in equation (8): (8)； Where, in the formula Here is the hidden layer weight matrix of the MLP. This is the weight matrix of the MLP output layer. This is used to bias the hidden layer of global features. Bias for the importance score output layer; Finally, the scores are normalized, and the score distribution is constrained using the Softmax function as shown in equation (9): = , k∈{1,2,3} (9)； in, , , These represent the importance weights of small, medium, and large objectives, respectively, satisfying... ; Dynamic parameter adjustment is based on importance weights. To handle real-time computational load, the kernel size and computational intensity are adaptively selected. If the inference latency exceeds the threshold, the 11×11 kernel is automatically replaced with a 7×7 kernel, as shown in equation (10). (10)。 3. The lightweight multi-scale dynamic convolutional steel surface defect detection method as described in claim 1, wherein step 2, multi-scale feature fusion and adaptive weighting of the neck network: 1) Bidirectional feature pyramid construction: adopting the FPN+PAN structure, firstly, the deep semantic features P5 and P4 are upsampled and fused with the shallow high-resolution feature P3 layer by layer through the top-down path to preserve the details and textures, and then the shallow features are downsampled and aggregated with the deep features through the bottom-up path to strengthen the global context information and form a bidirectional closed loop of multi-scale feature interaction. 2) Cross-stage dynamic weight fusion: A lightweight channel attention module is introduced during the feature fusion process to adaptively assign weights to features from different levels, as shown in equation (11): = ,i∈{P3,P4,P5} (11); Where σ is the Sigmoid activation function. This represents the convolutional layer after global average pooling. The weighted features... = Suppress redundant features while preserving important channel information; 3) Multi-scale feature alignment and output: The feature resolution of each level is unified to 1 / 8 scale through bilinear interpolation, and the number of channels is aligned using 1×1 convolution. Finally, a set of feature maps P3, P4 and P5 with consistent scale and enhanced semantics are output.

4. The lightweight multi-scale dynamic convolutional steel surface defect detection method as described in claim 1, wherein the LSCD (lightweight shared convolutional detection head) of the head network in step 3: 1) Cross-stage feature fusion: First, feature maps P3 (1 / 8 scale), P4 (1 / 16 scale), and P5 (1 / 32 scale) are extracted from the backbone network, with the number of channels being C3=256, C4=512, and C5=1024, respectively. Then, bilinear interpolation is performed on P4 and P5 to unify the resolution to 1 / 8 of the scale of P3. A 1×1 convolution is used to compress the number of channels of P4 and P5 to 256, which is consistent with P3, as shown in Equation (12): ( , ) (12)； Finally, output the aligned feature map. , , ; The bidirectional feature interaction is divided into bottom-up and top-down paths, which model cross-stage dependencies through a lightweight attention mechanism. The bottom-up path concatenates the P3 feature with the compressed P4 and P5 features to generate a global context vector, as shown in Equation (13). = ⊕ ⊕ (13)； Long-range dependencies are modeled using a lightweight Transformer encoder with 4 attention heads and a hidden layer dimension of 128, as shown in Equation (14): = Transformer( ) (14); The top-down path will F trans The data is divided into three parts, which are added to the original features P3, P4, and P5 respectively to achieve information feedback, as shown in equation (15): = + ( ),s∈{3,4,5} (15)； Channel compression and enhancement strategies retain key information by reducing redundant channels. Dimensionality reduction through grouped convolution is used to enhance the concatenated features. ∈ The number of channels is compressed to 128 using grouped convolution with 8 groups, as shown in equation (16): F fused = GroupConv 3×3 ( ,Group = 8 ) (16); Then, through dynamic channel recalibration, a lightweight SE module (Squeeze-and-Excitation) is introduced to dynamically adjust the channel weights as shown in equation (17): (17)； 2) Parameter optimization: For each output channel of the detection head, the mean absolute value of its weight gradient is calculated as shown in equation (18): (18)； in, C represents the number of channels at the current level. Let be the weight gradient of the Cth channel; remove channels with gradient sensitivity lower than the mean μ, and retain important channels; Finally, the channels with gradient magnitudes below the mean of the LSCD detector head are sparsified, and the retention rate r is calculated as shown in equation (19): r=1 - (19)。

Images