A steel surface defect high-precision lightweight detection method and system based on improved YOLOv11

Abstract

The application provides a steel surface defect high-precision lightweight detection method and system based on an improved YOLOv11, and belongs to the technical field of industrial visual detection and intelligent manufacturing. The method collects surface image data of steel in a complex production environment, and performs pretreatment operations such as noise suppression and contrast enhancement. A training data set is constructed through multi-scale data enhancement and defect sample balancing strategy to improve the representation ability of the model for small targets and weak texture defects. Based on the improved target detection network framework, the main feature extraction structure is optimized and designed, and a multi-scale feature fusion and semantic enhancement mechanism is introduced to strengthen the discrimination ability of defects of different scales such as cracks, inclusions and pits. An efficient feature reconstruction path is constructed in the neck network.

Description

Technical Field

[0001] This application relates to the field of industrial machine vision and deep learning target detection technology, and in particular to a high-precision and lightweight detection method and system for steel surface defects based on improved YOLOv11 for complex industrial scenarios. Background Technology

[0002] Steel, with its excellent hardness, ductility, and corrosion resistance, is widely used in construction, transportation, and daily necessities manufacturing, and plays a crucial role in cutting-edge industries such as aerospace and defense. As the core basic material of the steel industry, steel production quality is constrained by multiple factors, including raw material purity, element ratios, temperature control precision, and mechanical wear during continuous rolling, making it prone to surface defects such as scratches, inclusions, and cracks. These defects not only damage the product's appearance and mechanical properties but also cause stress concentration and fatigue fracture, significantly reducing the material's service safety. Especially for high-speed rolled steel products, the real-time evolution of defects directly degrades the reliability of the final product (e.g., broken automotive leaf springs, cracked container walls). Therefore, online, real-time, high-precision detection of steel surface defects has become a core technological link in ensuring steel production quality and preventing major accidents. Traditional image processing methods rely on manually designed feature engineering, which suffers from low equipment versatility, high requirements for acquisition equipment, and high operating and maintenance costs when facing complex and changing industrial environments. Furthermore, their detection performance is highly dependent on prior knowledge of defect morphology by domain experts. When the detection scene contains strong light reflection, shadow occlusion, or multiple light sources, traditional methods often lead to missed detections and false detections. With the development of deep learning technology, mainstream methods have begun to rely on object detection technology (such as the YOLO series), automatically learning the deep semantic information of images through deep neural networks, thus improving adaptability. However, in real-world industrial applications, current deep learning still faces many challenges in steel surface defect detection: First, the dataset of steel surface defects is relatively small, resulting in insufficient generalization ability of the model; second, small-to-medium scale defects, complex morphological defects, and structurally ambiguous defects lead to low accuracy, high false detection and missed detection rates in existing algorithms; finally, high-performance models often have high computational complexity and a large number of parameters, making them inconvenient to deploy on resource-constrained edge computing devices such as smartphones and embedded platforms. Summary of the Invention

[0003] This invention provides a high-precision, lightweight detection method, system, and device for steel surface defects based on an improved YOLOv11, aiming to solve the technical problems of existing deep learning models in steel defect detection, such as low recognition rate of small targets, susceptibility to interference from complex backgrounds, and high computational complexity that makes them difficult to deploy on edge devices.

[0004] In a first aspect, this application provides a high-precision, lightweight detection method for steel surface defects based on an improved YOLOv11, including: Step 1: Perform data augmentation and preprocessing based on a pre-constructed steel surface defect dataset, which covers various typical steel surface defect images and corresponding annotation information, and establish training and validation sets.

[0005] Step 2: Construct a basic YOLOv11 target detection model. Based on the pre-identified steel surface defect target features, introduce a novel convolutional module Vimconv with global perception and long-distance dependency modeling capabilities to replace some of the standard convolutional structures in the original network, construct a multi-scale context association structure, and enhance the model's ability to capture fine-grained defects and global information.

[0006] Step 3: Introduce the Dynamic Hyperbolic Tangent (DyT) mechanism into the feature extraction network of the model to dynamically adjust the normalization parameters, replace the normalization layer of the traditional Transformer architecture, and improve the stability and convergence speed of the model's feature representation in small sample and noisy industrial environments.

[0007] Step 4: During the sampling stage on the model's neck network, the DySample dynamic upsampling module is used to generate content-aware sampling points to resample the feature map, improving the restoration accuracy and localization accuracy of minor defect features, while reducing computational overhead.

[0008] Step 5: Lightweight reconstruction of the model neck network. The core module StarBlock of the StarNet lightweight network is introduced to replace the Bottleneck unit in the original C3k2 structure, and the C3k2-star lightweight module is constructed. While maintaining the ability to map high-dimensional nonlinear space, the number of model parameters and the overall volume are significantly compressed.

[0009] Step 6: Use the enhanced training set to train the improved YOLOv11 model end-to-end. By optimizing the loss function, extract the depth features of the target object from different perspectives to obtain the trained steel surface defect detection model. Deploy it on industrial edge devices to output defect classification and location prediction results.

[0010] Optionally, in step 1, the dataset primarily uses the NEU-DET dataset, supplemented by open-source datasets. Data augmentation techniques used include random translation, rotation, scaling, flipping, and color perturbation to simulate real, complex industrial scenarios and address the issue of low model generalization ability caused by training with the original data.

[0011] Optionally, in step 2, the Vimconv module connects the input and output with depthwise separable convolutions (DWConv), splits the features through a split operation, extracts fine-grained features through multiple ViM Blocks, and then fuses them using Concat. Specifically, the ViM Block internally employs two 3×3 DWConv operations to capture local details and introduces an implicit hybrid global hidden state (HSM-SSD) module, reducing computational complexity from... Reduce to This enables efficient long-distance dependency modeling.

[0012] Optionally, in step 3, the DyT mechanism is applied to the input tensor. Using formula Perform element-wise operations. Among them, Learnable scalar parameters are used for adaptive scaling. and These are learnable channel-based vector parameters used for range mapping of the output, thus replacing the overhead of additional normalization operations.

[0013] Optionally, in step 4, the DySample dynamic upsampling module does not need to use a dynamic convolution kernel. Instead, given a feature map and an upsampling scale factor, it uses a linear layer to generate an offset, reshapes it through pixel shuffling, adds the offset to the original sampling grid to obtain a sampling set, and finally generates a high-quality content-aware feature map after upsampling through a grid sampling function.

[0014] Optionally, in step 5, the StarBlock forms a star-shaped topology through direct connections between the central node and the peripheral nodes. It uses element-wise multiplication interactions (star operations) between feature maps to fuse multi-scale and multi-channel information, and combines depthwise separable convolution to reduce computational load and parameter scale, thus resolving the contradiction between capturing multi-scale features such as fine cracks and deploying in resource-constrained environments.

[0015] Secondly, this application provides a high-precision, lightweight inspection system for steel surface defects based on an improved YOLOv11, comprising: The data acquisition and enhancement module is used to construct and enhance a dataset of images of steel surface defects. The feature extraction enhancement module is used to extract multi-level defect features with long-distance dependence and stable expression from the input image by embedding the Vimconv module and the DyT mechanism in the backbone network. The lightweight feature fusion module is used to efficiently aggregate and parse multi-scale features by introducing DySample dynamic upsampling and the neck network of the C3k2-star lightweight module; The target detection and output module is used to output multi-scale defect category confidence and bounding box regression results based on the decoupled detection head.

[0016] Thirdly, this application provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, communication interface, and memory communicate with each other through the communication bus; the memory is used to store computer programs; and the processor, when executing the program stored in the memory, implements the steps of any of the detection methods described in the first aspect. The advantages of this application are: by introducing the Vimconv module and DySample upsampler into the basic YOLOv11 model, the feature perception and localization accuracy of the model under blurred edges, small-scale targets, and complex lighting interference is significantly improved; by introducing the DyT mechanism and the StarNet lightweight architecture, while effectively suppressing background noise interference, the parameter redundancy and computational complexity of the model are greatly reduced, achieving synergistic optimization of detection accuracy and inference speed, thereby meeting the low-latency, high-reliability real-time deployment requirements of edge computing platforms in industrial production lines. Attached Figure Description

[0017] Figure 1 This application provides a schematic diagram of data augmentation and preprocessing for a steel surface defect dataset, as illustrated in an embodiment of the present application. Figure 2 A schematic diagram of the basic YOLOv11 network structure and its improved locations provided in the embodiments of this application; Figure 3 The Vimconv module and ViM Block internal structure block diagram provided in the embodiments of this application; Figure 4 This is a block diagram of the DySample dynamic upsampling module provided in an embodiment of this application; Figure 5 The diagram shows the C3k2-star lightweight module and StarBlock structure provided in the embodiments of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application.

[0019] Combination Figures 1 to 5This application provides a high-precision, lightweight method for detecting surface defects in steel based on an improved YOLOv11. This method treats target detection as a regression problem, directly predicting the class probabilities and bounding box positions of all targets from the original image, achieving fully differentiable integrated training. The specific implementation steps are as follows: Step 1: Dataset Construction and Augmented Preprocessing The imaging environment in industrial settings is complex and variable; factors such as light intensity, surface reflection, and noise interference can lead to significant differences in defect images. This embodiment primarily uses the NEU-DET dataset, which contains 1800 200×200 resolution images of steel surface defects collected in industrial scenarios. These images cover six defect categories: cracks, inclusions, pits, patches, rolled-in oxide scale, and scratches, with 300 images for each category. To address the limitations of the dataset, such as the overall darkness of the images and the small grayscale range which can easily lead to confusion between defects and background, and to simulate complex and variable real-world scenarios and suppress model overfitting, various data augmentation techniques were applied to the collected samples, including random image translation, rotation, scaling, flipping, and color perturbation. The augmented data, supplemented with regularization techniques (such as Dropout and L2 weight decay), participated in model training, significantly improving the model's robustness and generalization ability.

[0020] Step 2: Construct an improved model to enhance feature extraction This embodiment uses the latest single-stage end-to-end object detector YOLOv11 as the base network model. The backbone of YOLOv11 extracts semantic information at different levels through modules such as CBS, C3X, and E-Rep. The neck uses a BiFPN structure for multi-scale feature fusion, and the head uses decoupled detection heads to handle classification and regression tasks respectively.

[0021] To address the issues of blurred features of small targets and insufficient fusion of long-range semantic information in steel defect detection, this application designs and embeds a Vimconv convolutional module into the feature extraction network. For example... Figure 3 As shown, the Vimconv module connects the input and output with depthwise separable convolutions (DWConv). After splitting the features through a split operation, fine-grained features are extracted through multiple ViM Blocks and then fused by Concat. Inside the ViM Blocks, an implicit hybrid global hidden state (HSM-SSD) module is introduced. HSM-SSD transforms the originally massive channel dimension calculation into processing the number of states by sharing the global hidden state, estimating the importance of a single-head token, and using a multi-stage hidden state fusion (MSF) mechanism, thus reducing the computational complexity from... Significantly reduced to This hierarchical, top-down data flow design balances the precise capture of local defect details with the modeling of global long-distance dependencies with extremely low computational overhead.

[0022] Furthermore, in the C2PSA module of the model, to avoid gradient instability and additional computational overhead caused by the traditional Transformer normalization layer in small-sample, high-noise environments, this embodiment introduces a Dynamic Hyperbolic Tangent (DyT) mechanism as an alternative to the normalization layer. This is done for the input features... Through learnable scalar parameters and by channel vector parameters and Directly perform nonlinear mapping: This mechanism enables stable training without hyperparameter tuning and significantly improves the convergence speed of feature representation.

[0023] Step 3: Multi-scale feature fusion and dynamic upsampling of the neck network Since minute cracks and pitting defects on the steel surface are prone to feature loss after multiple downsampling, this application replaces traditional bilinear or nearest-neighbor interpolation with the DySample dynamic upsampling module in the Neck feature fusion network. Unlike computationally expensive dynamic convolutional kernel upsampling (such as CARAFE), DySample uses a built-in PyTorch function to learn the offset in the input feature map using a linear layer. After being reshaped by pixel shuffling, it is added to the original sampling grid to generate a content-aware set of sampling points. This enables the network to efficiently reconstruct continuous high-dimensional feature maps, significantly reducing model inference latency, memory usage, and floating-point operations (FLOPs), and improving the ability to finely locate blurred boundaries and small-scale defects.

[0024] To achieve efficient deployment on resource-constrained edge computing devices, this application further optimizes the model for lightweight design. Addressing the parameter redundancy issue in the C3k2 module of the Neck network, this application modifies the model using the StarNet architecture. The highly redundant standard Bottleneck units within C3k2 are replaced with StarBlock modules, forming the improved C3k2-star module. Figure 5 As shown, StarBlock constructs a star-shaped topology connecting the central node and peripheral nodes, utilizing element-wise multiplication interactions (star operations) between feature maps, combined with depthwise separable convolutions, to reduce information attenuation caused by network depth. This design achieves efficient mapping of input features to a high-dimensional nonlinear space while strictly maintaining constant computational complexity, compressing channel redundancy and providing a highly optimized multi-scale feature representation for the final detection head.

[0025] Step 4: Model Training and Loss Optimization The improved YOLOv11 model was trained using an enhanced training set of steel surface defects. Regarding the loss function design, the model comprehensively employs VFL Loss (Varifocal Loss) to address the imbalance between positive and negative samples in the classification branch, introduces SIoU Loss (SCYLLA-IoU) to fuse angle, distance, and shape factors for fine-grained optimization of bounding box regression, and combines DFL Loss (Distribution Focal Loss) for distributed boundary modeling to enhance localization accuracy. On the multi-scale detection head, the model automatically adjusts the weights between channels using Dynamic Convolution, ultimately outputting the prediction results. After post-processing techniques such as non-maximum suppression, the training parameters are iteratively updated until the network converges, resulting in a high-precision model for identifying defects in the pond (referring to steel in an industrial environment).

[0026] Step 5: Industrial Deployment and Real-time Monitoring The trained lightweight model exhibits significantly reduced parameter size and volume, demonstrating strong potential for industrial application. Deployed on an edge computing platform within a production line, it receives real-time images of steel from high-speed industrial cameras. Through the lightweight detection network constructed in this application, which possesses strong global perception and the ability to capture minute local features, the model can accurately locate defects such as scratches, inclusions, and cracks within tens of milliseconds, and determine their type and severity, providing real-time and reliable data support for subsequent process optimization and defective product removal.

[0027] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0028] The embodiments of this application have been described above with reference to the accompanying drawings. This application is not limited to the specific embodiments described above, which are merely illustrative and not restrictive. Those skilled in the art, under the guidance of this application, can make many variations without departing from the spirit and scope of the claims, all of which fall within the scope of protection of this application. The above descriptions are merely embodiments of the present invention and are not intended to limit the patent protection scope of this invention; any equivalent structural or procedural transformations made based on the content of this specification and drawings, or direct or indirect applications to other related technical fields, are similarly included within the patent protection scope of this invention.

Claims

1. A high-precision, lightweight detection method for steel surface defects based on an improved YOLOv11, characterized in that, Includes the following steps: Step 1: Acquire image data of steel surface defects in industrial scenarios and construct an original image sample library; to address the issues of limited grayscale value range and uneven distribution, perform adaptive image preprocessing and data augmentation to generate an enhanced three-dimensional image data matrix of steel surface defects containing multi-scale and multi-morphological features, and construct a joint dataset for model training and validation. Step 2: Construct a steel surface defect target detection network based on the improved YOLOv11 architecture; design and embed the Vimconv convolution module in the feature extraction backbone network to replace the original standard convolution unit; use the HSM-SSD layer in the Vimconv module to capture local details and model long-distance dependencies of the input features; Step 3: Dynamically optimize the feature normalization and upsampling mechanisms in the network architecture; replace the traditional normalization layer in the network with a dynamic hyperbolic tangent mechanism, and dynamically adjust the normalization scaling parameters according to the feature distribution; at the same time, introduce the DySample module in the feature reconstruction stage of the neck network, and perform nonlinear remapping of the grid based on the feature content to adaptively generate dynamic sampling points. Step 4: Perform a lightweight architecture refactoring on the improved YOLOv11-V model described above; The C3k2 structure of the neck network feature fusion stage is extracted, and its highly redundant Bottleneck units are replaced with the StarBlock core module based on the StarNet architecture. By using channel element-level star operation multiplication interaction, the efficient mapping of features to a high-dimensional nonlinear space and channel compression are achieved while strictly maintaining the basic unchanged number of floating-point operations. Step 5: Input the image of the industrial steel surface to be detected into the trained and lightweight YOLOv11-V model. By decoupling the detection head, output the scale-corrected set of defect category confidence and bounding box regression coordinates, and finally output the steel surface defect recognition result with high accuracy and low latency.

2. The high-precision, lightweight detection method for steel surface defects based on improved YOLOv11 according to claim 1, characterized in that, Step 1 involves processing the original image sequence, specifically as follows: Step 1-1: Acquire surface images of steel containing defects such as cracks, inclusions, pits, patches, rolled-in oxide scale, and scratches. Integrate the NEU-DET dataset with Kaggle open-source data to construct an initial 3D image data matrix. ,in Indicates the pixel index of the image height. Indicates the width in pixels. Indicates the number of channels; Steps 1-2: Targeting To address the issues of overall low illumination and low local contrast in the image, we extracted the grayscale histograms of each channel and performed contrast-limited adaptive histogram equalization to widen the grayscale difference between the background and the defective target, generating a contrast-enhanced, interference-free data matrix. ; Steps 1-3: Based on Spatial transformation is performed by introducing geometric and photometric data enhancement strategies; Random translation, rotation, scaling mapping, and random perturbation calculations of color channels are performed on the image to simulate complex industrial lighting and physical stain interference. Steps 1-4: Introduce regularization parameters such as Dropout and L2 weight decay into the expanded dataset to suppress overfitting of the model to a single feature; divide the processed data into training, validation and test sets according to the set ratio, and generate standardized PASCAL VOC format labeled tensors as the input benchmark for subsequent network training.

3. The high-precision, lightweight detection method for steel surface defects based on improved YOLOv11 according to claim 1, characterized in that, Step 2 achieves global and local feature fusion extraction, specifically as follows: Step 2-1: In the Vimconv module, the input feature map is first subjected to depthwise separable convolution for preliminary spatial feature extraction, and then the features are split along the channel dimension using the Split operation and fed into multiple sets of parallel ViM Blocks. Step 2-2: Within the ViM Block, utilize two lightweighting processes. DWConv precisely captures local details of scratches and patches; then the local features are sent to the HSM-SSD module; Steps 2-3: The HSM-SSD module utilizes the shared global hidden state Perform channel mixing and calculate the input hidden state. And linearly project it onto a low-dimensional latent array, reducing the computational complexity from Reduce to ,in For the number of states, For sequence length, The number of feature channels; Steps 2-4: Employ a single-head design by setting the weight matrix. and Estimate the importance of each feature state, combine it with a multi-stage hidden state fusion mechanism, integrate multi-scale semantic information, and output the enhanced global context features.

4. The high-precision, lightweight detection method for steel surface defects based on improved YOLOv11 according to claim 1, characterized in that, Step 3 implements parameter normalization and dynamic upsampling, specifically as follows: Step 3-1: Extract the input feature tensor to be normalized at the feature flow node of the network. ; Replace the original LayerNorm layer with a Dynamic Hyperbolic Tangent (DyT) layer; For the input tensor Calculate its element-wise nonlinear dynamic scaling mapping: in, To be based on tensor Learnable scalar parameters for adaptive optimization of numerical distribution range and A learnable affine compensation vector that is strictly aligned with the channel dimension; Step 3-2: In the feature resolution enhancement stage of the neck network, the scale factor is extracted as follows: Low-resolution input feature map ;Will After being fitted into a continuous feature map using bilinear interpolation, a spatial offset matrix is ​​calculated and generated using the built-in linear sensing layer. ; Step 3-3: Perform subpixel rearrangement operation using Pixel Shuffling to adjust the offset matrix. Dimensional reshaping The reshaping matrix is ​​then compared with the preset original grid coordinate system. Perform element-wise addition to generate a set of irregular sampling point coordinates with content-adaptive properties. ; Steps 3-4: Based on the coordinate set The method employs a grid sampling function to directly extract and reconstruct spatial points from continuous feature maps, outputting high-quality upsampled feature maps that eliminate blur artifacts and accurately preserve the edges of minute defects. .

5. The high-precision, lightweight detection method for steel surface defects based on improved YOLOv11 according to claim 1, characterized in that, Step 4 achieves minimal compression and topology optimization of the network computation flow, specifically as follows: Step 4-1: Traverse the multi-scale feature fusion path in the YOLOv11-V neck network, locate and strip the traditional Bottleneck standard convolution stacked units within the C3k2 module which contains a large amount of redundant and dense computation. Step 4-2: Introduce the lightweight network StarNet architecture, and implant the StarBlock module with star topology connection at the removal position to build a brand-new lightweight aggregation module C3k2-star; this topology establishes direct radial connection between the central node and the peripheral nodes; Step 4-3: In the forward propagation stage of StarBlock, low-dimensional input features are received and decoupled and mapped through depthwise separable convolution; then, tensor element-level multiplication interaction (star operation) is used to guide and activate the information flow direction of features in multiple channels, and nonlinearly map the input features to a high-dimensional latent space. Step 4-4: Apply this mapping mechanism to the multi-scale feature network layer, deeply compress feature redundancy at the channel level and enhance cross-layer feature reuse, ultimately generating a lightweight target detection model with significantly reduced weight file size, greatly reduced inference latency, and high adaptability to industrial mobile terminals and edge computing node deployments.

6. A high-precision, lightweight inspection system for steel surface defects based on an improved YOLOv11, characterized in that, It can implement the method described in any one of claims 1-5.

Images