Steel Surface Defect Detection Method and Device Improved Based on YOLO11n

Abstract

The present invention discloses a method and device for detecting steel surface defects improved based on YOLO11n, which relates to the technical field of defect detection. The method includes: acquiring a steel surface image to be detected collected by an image acquisition device; inputting the steel surface image to be detected into a defect detection model trained by an improved YOLO11n model architecture to obtain surface defect detection information of the steel surface image to be detected. Among them, the backbone network of the improved YOLO11n model architecture includes a C3k2_NAM module formed by a C3k2 module integrated with a NAM dynamic attention mechanism, and the Neck network introduces an NCB convolutional block, an MSDA attention mechanism, and a CoT Attention module. The present invention solves the problem that the existing steel surface defect detection cannot accurately identify multi-scale targets.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and particularly to a steel surface defect detection method and device improved based on YOLO11n. Background Art

[0002] Under the background of the integration of technology and the upgrading of industry, steel, as a core material, its quality directly affects engineering safety and product reliability. Although steel is widely used in fields such as construction, bridges, aerospace, and precision manufacturing, during the production process, due to factors such as raw materials, temperature control, and stress distribution, defects such as cracks and inclusions may occur, affecting mechanical properties and even leading to safety accidents. Therefore, accurately detecting steel defects is the key to ensuring product quality and industrial safety.

[0003] Traditional steel defect detection methods, such as manual visual inspection, ultrasonic testing, magnetic particle flaw detection, and radiographic testing, have long played an important role in the industrial field. These methods rely on manual labor or specific equipment, and the detection results are affected by operation experience and are difficult to standardize. The introduction of machine learning technology has improved the automation level of steel defect detection. For example, methods such as support vector machine (SVM), random forest (RF), and K-nearest neighbor (KNN) perform well in the identification of common defects. In the field of steel defect detection, the YOLO model has become the mainstream choice due to its advantages of speed, accuracy, and automation, and can accurately detect and locate various defects, thus being widely used to improve product quality and safety.

[0004] However, the current industrial defect detection methods based on the YOLO model still have deficiencies in the identification of steel surface defects. For example, due to their small size and unclear features, small defects often have poor detection effects. When detecting medium-scale targets (such as large cracks and medium spalling), the feature expression is insufficient, resulting in blurred boundaries and inaccurate positioning. When detecting large targets (such as severe spalling and rail fractures), it will be affected by the background complexity, resulting in false detections or blurred boundaries. Therefore, the existing steel surface defect detection cannot achieve accurate identification of multi-scale targets. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a steel surface defect detection method and device improved based on YOLO11n, aiming to solve the problem that the existing steel surface defect detection cannot achieve accurate identification of multi-scale targets.

[0006] On the one hand, the present invention proposes a steel surface defect detection method improved based on YOLO11n, and the method includes:

[0007] Obtain the steel surface image to be detected collected by the image acquisition device;

[0008] Input the surface image of the steel to be detected into the defect detection model trained by the improved YOLO11n model architecture to obtain the surface defect detection information of the surface image of the steel to be detected;

[0009] Among them, the improved YOLO11n model architecture includes a backbone network, a detection head, and a Neck network arranged between the backbone network and the detection head. The backbone network includes a C3k2_NAM module formed by integrating the C3k2 module with the NAM dynamic attention mechanism to adaptively adjust the channel and spatial weights according to the features of the input image. The Neck network introduces an NCB convolutional block, an MSDA attention mechanism, and a CoT Attention module. The NCB convolutional block is used to enhance the global correlation of small target features through non-local operations. The MSDA attention mechanism is used to combine multi-dilation rate convolutions to enrich the context information and improve the discrimination ability between the target and the background. The CoT Attention module is used to model the relationship between the target and the background through context awareness.

[0010] Furthermore, in the above steel surface defect detection method improved based on YOLO11n, the neck module in the C3k2 module and the neck module of the C3k module in the C3k2 module are replaced with the NAM neck module to form a C3k2_NAM module containing C3k_NAM;

[0011] When C3k_NAM is True, the input features received by the C3k2_NAM module pass through 1×1 convolution and 3×3 convolution to extract local features and perform preliminary dimensionality reduction or channel transformation;

[0012] The input features after convolution processing are divided into multiple branches and each branch is input into the C3k_NAM module. In the C3k_NAM module, the divided features pass through 1×1 convolution and 3×3 convolution to extract local features and perform preliminary dimensionality reduction or channel transformation;

[0013] The input features after convolution processing pass through multiple NAM neck module recursive processing respectively and directly perform 1×1 convolution and 3×3 convolution;

[0014] The input features after recursive processing and re-convolution are feature concatenated and then pass through 1×1 convolution and 3×3 convolution to obtain the features processed by the C3k_NAM module;

[0015] The features processed by all C3k_NAM modules are feature concatenated and then pass through 1×1 convolution and 3×3 convolution to obtain the finally processed features of the C3k2_NAM module;

[0016] When C3k_NAM is False, the input features received by the C3k2_NAM module undergo 1×1 convolution and 3×3 convolution to extract local features and perform preliminary dimensionality reduction or channel transformation;

[0017] The input features after convolution processing are divided into multiple branches, and each branch is input into the NAM neck module. After feature concatenation of all the features processed by the NAM neck module, they undergo 1×1 convolution and 3×3 convolution to obtain the finally processed features of the C3k2_NAM module.

[0018] Furthermore, in the above steel surface defect detection method improved based on YOLO11n, the NCB convolution block is set in the P3 branch of the Neck network, and the NCB convolution block introduces a multi-head convolution attention mechanism and a multi-layer perceptron;

[0019] When the NCB convolution block receives the corresponding multi-scale feature map from the backbone network, it first performs normalization processing;

[0020] Then, group convolution in the multi-head convolution attention mechanism is used to extract local features, and pointwise convolution is used for information fusion in the channel dimension;

[0021] The two fully connected layers included in the multi-layer perceptron are used for non-linear transformation;

[0022] Batch Normalization is used for normalization, and combined with the ReLU activation function for non-linear activation;

[0023] Finally, the corresponding feature map is output and transmitted to the detection head for target recognition.

[0024] Furthermore, in the above steel surface defect detection method improved based on YOLO11n, in the P4 branch of the Neck network, the MSDA attention mechanism is introduced;

[0025] The MSDA attention mechanism is used for sparse self-attention calculation within the sliding window. The self-attention calculation formula is:

[0026]

[0027] Among them, respectively represent the feature matrices of the query vector, key vector, and value vector of the i-th attention head, is the dilation rate, which controls the receptive field size;

[0028] Multi-head aggregation is performed through the linear layer:

[0029]

[0030] Among them,h 1 ，h 2 ，...，h n respectively represent the feature vectors output by the 1st to the nth attention heads. Concat represents the concatenation operation, and Linear represents fusing the information of multiple attention heads through a linear layer.

[0031] Furthermore, in the above steel surface defect detection method improved based on YOLO11n, in the P4 branch of the Neck network, a CoT Attention module is introduced;

[0032] The CoT Attention module uses a 3×3 convolution to capture the local relationship between keys in the received input features:

[0033]

[0034] Among them, K1 represents the key in the static context, and K is the original key input. represents a 3×3 convolution. The key represents the context structure information extracted from the input features through local convolution;

[0035] The K1 of the static context is concatenated with the input Q, and the attention matrix is calculated through two 1×1 convolutions:

[0036]

[0037] Among them, [K1,Q] represents the concatenation of the key and query of the static context. is a 1×1 convolution kernel used to generate the attention matrix A. Q represents the information encoding of each position in the input;

[0038] Through the attention matrix A, the CoT Attention module performs weighted aggregation on the input value 𝑉 to obtain a dynamic context representation:

[0039]

[0040] The CoT Attention module fuses the static context K1 and the dynamic context K2, and generates the final output feature Y through an addition operation.

[0041] Furthermore, in the above steel surface defect detection method improved based on YOLO11n, the training process of the defect detection model includes:

[0042] Select a public dataset, and after data cleaning, image enhancement, and annotation processing in sequence, obtain the corresponding training dataset;

[0043] Divide the training data set into a training set, a validation set, and a test set according to a preset ratio;

[0044] Define a loss function and train the improved YOLO11n model architecture on the training set until the training stop condition is met to obtain a defect detection model.

[0045] Furthermore, in the above steel surface defect detection method improved based on YOLO11n, the loss function adopts one of cross-entropy loss combined with Inner-IoU loss and regression loss combined with Inner-IoU loss.

[0046] Another object of the present invention is to provide a steel surface defect detection device improved based on YOLO11n, and the device includes:

[0047] An acquisition module for acquiring the steel surface image to be detected collected by an image acquisition device;

[0048] A detection module for inputting the steel surface image to be detected into the defect detection model trained by the improved YOLO11n model architecture to obtain the surface defect detection information of the steel surface image to be detected;

[0049] Among them, the improved YOLO11n model architecture includes a backbone network, a detection head, and a Neck network arranged between the backbone network and the detection head. The backbone network includes a C3k2_NAM module formed by integrating a C3k2 module with a NAM dynamic attention mechanism to adaptively adjust the channel and spatial weights according to the features of the input image. The Neck network introduces an NCB convolution block, an MSDA attention mechanism, and a CoT Attention module. The NCB convolution block is used to enhance the global correlation of small target features through non-local operations. The MSDA attention mechanism is used to combine multi-dilation rate convolutions to enrich the context information and improve the discrimination ability between the target and the background. The CoT Attention module is used to model the relationship between the target and the background through context awareness.

[0050] Another object of the present invention is to provide a readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented.

[0051] Another object of the present invention is to provide an electronic device, including a memory, a processor, and a computer program stored on the memory and running on the processor. When the processor executes the program, the steps of the above method are implemented.

[0052] The present invention improves the current YOLO11n model architecture. In the backbone network, the C3k2_NAM module is adopted, and the NAM dynamic attention mechanism is integrated into the C3k2 module, enabling the model to adaptively adjust the channel and spatial weights according to the features of the input image, thereby enhancing the ability to extract key features and improving the object recognition accuracy of the model in complex backgrounds. In the Neck network, in order to improve the detection ability of multi-scale objects, the feature layers of different scales are optimized. For small object detection, the NCB convolution block is introduced, and the global correlation of small object features is enhanced through non-local operations, effectively aggregating context information at a long distance, thereby reducing the missed detection phenomenon caused by the loss of details. For medium object detection, the MSDA attention mechanism is adopted, combined with multi-dilation rate convolution to enrich the context information and improve the discrimination ability between the object and the background. Especially in the case of complex backgrounds or blurred object boundaries, the detection accuracy is enhanced. For large object detection, the CoT module is introduced, and the relationship between the object and the background is modeled through context awareness, enabling the model to more accurately locate large objects while effectively reducing the false detection and missed detection risks caused by background interference. It solves the problem that the existing steel surface defect detection technology cannot accurately identify multi-scale objects.

[0053] In addition, the embodiments of the present invention at least further have the following beneficial effects:

[0054] 1. Although some high-precision object detection algorithms (such as Transformer-based methods) have high precision, they have a large amount of computation and are not suitable for real-time industrial detection. By optimizing the network structure, YOLO11n can still achieve real-time inference in embedded devices or industrial detection systems while ensuring high precision, meeting the requirements of efficient detection.

[0055] 2. In complex environments such as light changes, occlusion, and stain interference, the traditional YOLO detection model has poor robustness. By combining multiple attention mechanisms, the adaptability of the model in complex environments is enhanced, enabling it to maintain high detection accuracy under different lighting and interference conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a flowchart of the steel surface defect detection method based on the improved YOLO11n in the first embodiment of the present invention;

[0057] Figure 2 It is a schematic diagram of the improved YOLO11n model architecture in the steel surface defect detection method based on the improved YOLO11n in the first embodiment of the present invention;

[0058] Figure 3Schematic diagram of the C3k2_NAM module in the improved YOLO11n model architecture of the steel surface defect detection method improved based on YOLO11n in the first embodiment of the present invention;

[0059] Figure 4 Schematic diagram of the C3k_NAM module in the improved YOLO11n model architecture of the steel surface defect detection method improved based on YOLO11n in the first embodiment of the present invention;

[0060] Figure 5 Schematic diagram of the position of the NCB convolutional block in the Neck network of the improved YOLO11n model architecture of the steel surface defect detection method improved based on YOLO11n in the first embodiment of the present invention;

[0061] Figure 6 Schematic diagram of the structure of the NCB convolutional block in the improved YOLO11n model architecture of the steel surface defect detection method improved based on YOLO11n in the first embodiment of the present invention;

[0062] Figure 7 Schematic diagram of the position of the MSDA attention mechanism in the Neck network of the improved YOLO11n model architecture of the steel surface defect detection method improved based on YOLO11n in the first embodiment of the present invention;

[0063] Figure 8 Schematic diagram of the structure of the module corresponding to the MSDA attention mechanism in the improved YOLO11n model architecture of the steel surface defect detection method improved based on YOLO11n in the first embodiment of the present invention;

[0064] Figure 9 Schematic diagram of the position of the CoT Attention module in the Neck network of the improved YOLO11n model architecture of the steel surface defect detection method improved based on YOLO11n in the third embodiment of the present invention;

[0065] Figure 10 Schematic diagram of the structure of the CoT Attention module in the improved YOLO11n model architecture of the steel surface defect detection method improved based on YOLO11n in the third embodiment of the present invention;

[0066] Figure 11 Structure block diagram of the steel surface defect detection device improved based on YOLO11n in the third embodiment of the present invention.

[0067] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. Specific Embodiments

[0068] To facilitate the understanding of the present invention, the present invention will be described more comprehensively below with reference to the relevant accompanying drawings. Several embodiments of the present invention are shown in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, these embodiments are provided to make the disclosure of the present invention more thorough and comprehensive.

[0069] It should be noted that when an element is referred to as being "fixed to" another element, it can be directly on the other element or there can also be an intermediate element. When an element is considered to be "connected" to another element, it can be directly connected to the other element or there may be an intermediate element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0071] Embodiment 1

[0072] Please refer to Figure 1 , which shows the steel surface defect detection method improved based on YOLO11n in the first embodiment of the present invention. The method includes step S10 to step S11.

[0073] Step S10, obtaining the steel surface image to be detected collected by the image acquisition device.

[0074] Among them, an image acquisition device such as a camera will collect images of the steel surface to be detected, obtain high-resolution detection data, and obtain the corresponding steel surface image to be detected. Specifically, in actual detection applications, in order to further improve the detection accuracy of the model, after the steel surface image to be detected is collected, the collected image is preprocessed, including denoising, contrast enhancement, and normalization, to improve the quality of the input data and ensure that the model can operate stably under various environmental conditions.

[0075] Step S11, inputting the steel surface image to be detected into the defect detection model trained by the improved YOLO11n model architecture to obtain the surface defect detection information of the steel surface image to be detected;

[0076] Among them, the defect detection model has mastered the internal logic of identifying steel surface defects based on steel surface images. Therefore, by inputting the steel surface image to be detected into the defect detection model, the surface defect detection information of the steel surface image to be detected can be obtained, which may include the position and type information of the defects, such as peeling, pits, etc.

[0077] Specifically, in the embodiments of the present invention, the defect detection model is trained by an improved YOLO11n model architecture. YOLO11n is selected as the basic model architecture, and some of its internal architectures are improved to obtain a customized model architecture for accurate identification of steel surface defects.

[0078] More specifically, the improved YOLO11n model architecture includes a backbone network, a detection head, and a Neck network arranged between the backbone network and the detection head. Among them, the main task of the backbone network is to extract high-quality feature representations from the input image. It is usually composed of a series of convolutional layers, normalization layers, and activation functions, and is used to gradually extract feature information at different levels, from low-level texture features to high-level semantic features. For example, extract features: convert the input image into a multi-scale feature map and retain key structural information; enhance discriminative ability: capture discriminative features of different category targets through a deep network structure; reduce computational cost: adopt efficient convolutional structures (such as depthwise separable convolution, SPP, FPN, etc.) to reduce the amount of calculation and improve the inference speed; the Neck network is used to further process the features extracted by the backbone network and enhance the feature fusion ability to meet the detection requirements of different scale targets. It usually adopts a multi-scale feature fusion structure, enabling the model to have good performance when detecting small, medium, and large targets; for example, multi-scale feature fusion: effectively detect small, medium, and large targets by fusing features at different levels; improve target distinctiveness: enhance the discrimination ability between targets and the background through a feature enhancement module (such as an attention mechanism, context modeling); optimize detection accuracy: enhance feature expression at different resolutions and reduce missed detections and false detections; the detection head is the final output part of the model, responsible for processing the features transmitted by the Neck network and generating the category, position (bounding box), and confidence of the target; for example, regression bounding box: predict the position coordinates of the target based on the feature map; classify the target category: predict the category to which the target belongs and calculate the classification confidence; generate the final detection result: combine post-processing methods such as NMS (non-maximum suppression) to remove overlapping boxes and output the final detection result.

[0079] In this embodiment, aiming at the deficiencies of YOLO11n in target detection performance, the optimization focuses on two parts: the backbone network and the Neck network. Specifically, as Figure 2As shown in the figure, in the backbone network, the C3k2_NAM module is adopted, which integrates the NAM dynamic attention mechanism into the C3k2 module, enabling the model to adaptively adjust the channel and spatial weights according to the features of the input image, thereby enhancing the ability to extract key features and improving the object recognition accuracy of the model in complex backgrounds. In the Neck network, in order to improve the detection ability of multi-scale objects, the feature layers of different scales are optimized. For small object detection, the NCB convolution block is introduced, and the global correlation of small object features is enhanced through non-local operations, effectively aggregating long-distance context information, thereby reducing the missed detection phenomenon caused by detail loss. For medium object detection, the MSDA attention mechanism is adopted, combined with multi-dilation rate convolution to enrich the context information and improve the discrimination ability between the object and the background. Especially in the case of complex backgrounds or blurred object boundaries, the detection accuracy is enhanced. For large object detection, the CoT Attention module is introduced, and the relationship between the object and the background is modeled through context awareness, enabling the model to more accurately locate large objects while effectively reducing the false detection and missed detection risks caused by background interference.

[0080] In actual applications, during the detection stage, after receiving the steel surface image to be detected, the defect detection model first dynamically adjusts the feature weights through the C3k2_NAM module to enhance the feature extraction ability of small objects (such as cracks and slight spalling). Then, the NCB convolution block further enhances the detail capture of small objects through non-local operations to ensure that fine defects are not overlooked. For medium-scale defects (such as larger cracks or medium-degree spalling), the MSDA attention mechanism enriches the context information of the object through multi-dilation rate convolution, enabling the model to more accurately identify and distinguish these defects. During the detection of large objects (such as severe spalling or rail fracture), the CoT Attention module introduces context awareness, enabling the model to more accurately locate large-area defects and reduce the occurrence of false detection and missed detection. Finally, the detection results are output, including the detection image with defect annotations, and the analysis results can be visualized.

[0081] It can be understood that through the above improvements, the problems of difficult small object detection, easy false detection and missed detection in complex scenarios, and insufficient adaptability to multi-scale objects are fundamentally alleviated. While improving the detection performance of the model, the module design is optimized to maintain a high detection efficiency.

[0082] Furthermore, the training process of the defect detection model includes:

[0083] Select a public dataset, and after data cleaning, image enhancement, and annotation processing in sequence, obtain the corresponding training dataset;

[0084] Divide the training dataset into a training set, a validation set, and a test set according to a preset ratio;

[0085] Define the loss function and train the improved YOLO11n model architecture on the training set until the training stop condition is met to obtain the defect detection model.

[0086] Exemplarily, perform dataset preparation and preprocessing, and select the public dataset NEU-DET; use the public dataset NEU-DET widely used in the field of steel defect detection. The dataset contains various defect types (such as cracks, dents, corrosion, etc.) and has already marked the positions and categories of the defects; then perform data cleaning to ensure the accuracy of the annotation information in the dataset and remove noisy data; then use data augmentation techniques (such as rotation, cropping, flipping, scaling, color transformation, etc.) to increase the diversity of the dataset and enhance the generalization ability of the model; finally, perform annotation processing to ensure that the defect areas in each image are correctly marked to meet the detection requirements.

[0087] Divide the training dataset composed of the NEU-DET dataset into a training set and a validation set. Specifically, divide the NEU-DET dataset into a training set, a validation set, and a test set, with a division ratio of 60% for the training set, 20% for the validation set, and 20% for the test set. Standardize the images (such as normalizing the pixel values to the [0, 1] interval) to meet the network input requirements.

[0088] Select a loss function suitable for object detection (such as cross-entropy loss, regression loss, etc.), and combine the Inner-IoU loss to enhance the regression accuracy. Train the improved YOLO11n model architecture on the training set until the loss function tends to be stable or the performance of the model tends to be stable to obtain the defect detection model. Additionally, at the beginning of training, set the basic values of the training hyperparameters. For example, set the batch size to 16, adjust the resolution of the input image to 640×640 pixels, set the number of parallel threads for data loading to 8, set the initial learning rate to 0.01, and set the weight decay of the optimizer to 0.0005 to reduce overfitting. In the post-processing stage, adopt non-maximum suppression (NMS) with an intersection over union (IoU) threshold set to 0.7. In addition, set the training cycle of all models to 300 epochs to ensure sufficient training and achieve stable convergence.

[0089] In addition, after the model training is completed, experiments can be used to evaluate the performance of the model. For example, tests can be conducted on multiple different steel defect datasets (e.g., in addition to NEU-DET, other publicly available datasets are also used). Compare the evaluation metrics such as detection accuracy, recall rate, and F1 value of the model on different datasets to test the performance of the model in different scenarios. By introducing different environmental conditions (such as light changes, noise, etc.) to simulate complex situations in actual applications, further test the robustness of the model; or, select several common steel defect detection algorithms (such as YOLOv3, YOLOv5, YOLOv8, etc.) and other mainstream deep learning object detection algorithms as comparison objects. Use the same NEU-DET dataset for training and testing to ensure that each model is evaluated under the same conditions. Compare the performance of each model in terms of detection accuracy (mAP), recall rate, precision, F1 value, training time, inference speed, etc.; or, perform step-by-step ablation on the basic YOLO11n model, removing or replacing different improved modules, such as the C3k2_NAM module, NCB convolutional block, MSDA attention mechanism, etc. After each modification, retrain and evaluate the model, and compare the detection accuracy and performance before and after removing specific modules. Record the evaluation metrics (such as mAP, precision, recall rate, etc.) under different experimental settings and analyze the specific impact of each improved module on the model performance.

[0090] In summary, the steel surface defect detection method based on the improvement of YOLO11n in the above embodiments of the present invention improves the current YOLO11n model architecture. In the backbone network, the C3k2_NAM module is adopted, and the NAM dynamic attention mechanism is integrated into the C3k2 module, enabling the model to adaptively adjust the channel and spatial weights according to the features of the input image, thereby enhancing the key feature extraction ability and improving the target recognition accuracy of the model in complex backgrounds; in the Neck network, to improve the detection ability of multi-scale targets, the feature layers of different scales are optimized. For small target detection, the NCB convolutional block is introduced, and the global correlation of small target features is enhanced through non-local operations, effectively aggregating long-distance context information, thereby reducing the missed detection phenomenon caused by detail loss. For medium target detection, the MSDA attention mechanism is adopted, combined with multi-dilation rate convolution to enrich the context information and improve the discrimination ability between the target and the background. Especially in the case of complex backgrounds or blurred target boundaries, the detection accuracy is enhanced. For large target detection, the CoT module is introduced, and the relationship between the target and the background is modeled through context awareness, enabling the model to more accurately locate large targets and effectively reducing the false detection and missed detection risks caused by background interference. It solves the problem that the existing steel surface defect detection technology cannot accurately identify multi-scale targets.

[0091] Embodiment 2

[0092] This embodiment also proposes a steel surface defect detection method improved based on YOLO11n. The difference between the steel surface defect detection method improved based on YOLO11n in this embodiment and that in Embodiment 1 is as follows:

[0093] The neck module in the C3k2 module and the neck module of the C3k module in the C3k2 module are replaced with a NAM neck module to form a C3k2_NAM module containing C3k_NAM; as Figure 3 、 Figure 4 shown, on the basis of retaining the original convolution operation, the NAM neck module incorporates the NAM dynamic attention mechanism, enabling the module to have the ability of adaptive optimization in both the channel and spatial dimensions.

[0094] The C3k2_NAM module flexibly adjusts the feature extraction strategy through the setting of the parameter C3k_NAM. When C3k_NAM is False, its structure is the same as that of C2f, and a traditional neck module is used for feature extraction, which is suitable for scenarios with limited computing power. However, this structure has limitations in capturing complex features. When C3k_NAM is True, the neck module is replaced by an optimized module, and the hierarchical design of CSPNet is used to enhance the feature modeling ability and improve the performance of complex background and multi-object detection.

[0095] Furthermore, replacing the traditional neck module with a NAM neck module can dynamically adjust the channel weights and optimize the spatial features, making the C3k2_NAM module more accurate and robust in multi-scale detection. The parameterized design of this module ensures flexibility, enabling it to dynamically adjust the structure under different task requirements, taking into account both computational efficiency and feature expression ability, thereby improving the overall detection performance and adaptability.

[0096] Specifically, when the C3k2_NAM module receives the input features, when C3k_NAM is True, the neck module adopts the C3k_NAM module. First, through 1×1 convolution and 3×3 convolution, local features are extracted and preliminary dimensionality reduction or channel transformation is performed; the input features after convolution processing are divided into multiple branches (Split) and each branch is input into a C3k_NAM module. In the C3k_NAM module, the divided features go through 1×1 convolution and 3×3 convolution to extract local features and perform preliminary dimensionality reduction or channel transformation; the input features after convolution processing are respectively recursively processed by multiple NAM neck modules and directly go through 1×1 convolution and 3×3 convolution; the input features after recursive processing and re-convolution are feature concatenated and then go through 1×1 convolution and 3×3 convolution to obtain the features processed by the C3k_NAM module; the features processed by all C3k_NAM modules are feature concatenated and then go through 1×1 convolution and 3×3 convolution to obtain the finally processed features of the C3k2_NAM module; when C3k_NAM is False, the neck module adopts the NAM neck module, and the input features go through 1×1 convolution and 3×3 convolution to extract local features and perform preliminary dimensionality reduction or channel transformation; the input features after convolution processing are divided into multiple branches (Split) and each branch is input into a NAM neck module, and the features processed by all NAM neck modules are feature concatenated and then go through 1×1 convolution and 3×3 convolution to obtain the finally processed features of the C3k2_NAM module.

[0097] Among them, through the NAM neck module, the selectivity of channel and spatial features is enhanced, and the feature expression ability of the model is improved. Multiple NAM neck modules are recursively processed, and features are gradually refined layer by layer through multiple NAM neck modules. Another path directly performs 1×1 convolution and 3×3 convolution, which can be used to maintain the original information or assist in feature fusion. The features of the two paths are finally subjected to a feature concatenation operation, combining multi-level information to enhance the feature expression ability, and finally through 1×1 convolution and 3×3 convolution to further integrate information, and finally output the optimized feature map.

[0098] Furthermore, the NCB convolution block is set in the P3 branch of the Neck network, as Figure 5 , Figure 6As shown, when the NCB convolution block receives the corresponding multi-scale feature maps from the backbone network, it first performs normalization, then uses grouped convolution in the multi-head convolution attention mechanism to extract local features, and conducts information fusion in the channel dimension through pointwise convolution, thereby enhancing the expressive ability of the attention mechanism. The two fully connected layers included in the multi-layer perceptron are used for non-linear transformation to further enhance the feature representation ability. Batch Normalization (BN) is used for normalization, and combined with the ReLU activation function for non-linear activation to improve gradient stability. Finally, the optimized feature maps are output and transmitted to the subsequent detection head for object recognition.

[0099] It can be understood that through this design, the NCB module significantly enhances the feature representation ability of low-resolution small targets. Especially in the P3 branch, the extraction of low-level semantic features is particularly prominent. On the premise of keeping the number of parameters relatively stable, NCB has made significant improvements in aspects such as detailed feature capture, anti-background interference ability, and optimization of feature distribution uniformity.

[0100] Furthermore, as Figure 7 , Figure 8 shown, in the P4 branch of the Neck network, the MSDA attention mechanism is introduced. MSDA enhances the model's ability to model local and global features by performing sparse self-attention calculations within the sliding window through the Dilated Attention mechanism. Its basic formula is:

[0101]

[0102] Among them, respectively represent the feature matrices of the query vector, key vector, and value vector of the i-th attention head. is the dilation rate, which controls the receptive field size. i represents a position (token / pixel / patch, etc.) on the feature map where the attention calculation is currently being performed. The meaning of the entire formula is that for this position i, using its combined with r i , the final output representation h i is calculated through the SWDA module, and finally, multi-head aggregation is performed through the linear layer:

[0103]

[0104] Among them, h 1 ，h 2 ，...，h n respectively represent the feature vectors output by the 1st to the nth attention heads. Each h iis the result calculated by the corresponding attention head, usually with a shape of ( L , d h ), where L is the sequence length (such as the number of tokens in an image), d h is the dimension of each attention head. Concat means concatenating the outputs of these heads in the feature dimension, and after concatenation, the dimension becomes ( L , n· d h ). Linear means passing through a linear transformation layer (fully connected layer) to fuse the multi-head information, and usually the dimension will be changed to the final output dimension.

[0105] Furthermore, as shown in Figure 9 , Figure 10 , in the P4 branch of the Neck network, a CoT Attention module is introduced. The first step of the CoT Attention module is static context modeling, which uses a 3×3 convolution to capture the local relationship between keys in the input features. The static context K1 represents the correlation between adjacent features within a fixed receptive field (3×3 convolution), which helps to retain the local spatial structure of the features and thus more effectively extract key information in visual tasks.

[0106]

[0107] Among them, K1 represents the key in the context, K is the original key input. Through the 3×3 convolution, the model can capture the static relationship between neighborhood features. The key represents the context structure information extracted from the input features through local convolution.

[0108] The second step of the CoT Attention module is dynamic self-attention calculation. First, the static context K1 is concatenated with the input query Q, and then two 1×1 convolutions are used to calculate the attention matrix. This not only focuses on the query-key relationship but also combines local context information, making up for the shortcoming of the traditional self-attention mechanism that ignores neighborhood information.

[0109]

[0110] Among them, [K1,Q] represents the concatenation of the context key and the query, is the 1×1 convolution kernel used to generate the attention matrix A. Q represents the information encoding of each position in the input, that is, the feature representation of the "current target position". This attention matrix is used to weight the input value V to form a dynamic context representation.

[0111] Subsequently, through the attention matrix CoT performs weighted aggregation on the input value V to obtain a dynamic context representation. This operation can capture long-range dependencies between features, especially for those features that are spatially separated but semantically related.

[0112]

[0113] Finally, the CoT Attention module fuses the static context K1 and the dynamic context K2, and generates the final output feature Y through an addition operation. The formula is as follows:

[0114]

[0115] Through this fusion, the CoT Attention module can enhance the perception of global dependencies while retaining local information.

[0116] In summary, the steel surface defect detection method based on the improvement of YOLO11n in the above embodiments of the present invention improves the current YOLO11n model architecture. In the backbone network, the C3k2_NAM module is adopted, and the NAM dynamic attention mechanism is integrated into the C3k2 module, enabling the model to adaptively adjust the channel and spatial weights according to the features of the input image, thereby enhancing the key feature extraction ability and improving the target recognition accuracy of the model in complex backgrounds; in the Neck network, in order to improve the detection ability of multi-scale targets, the feature layers of different scales are optimized. For small target detection, the NCB convolution block is introduced, and the global correlation of small target features is enhanced through non-local operations, effectively aggregating context information at long distances, thereby reducing the missed detection phenomenon caused by detail loss. For medium target detection, the MSDA attention mechanism is adopted, combined with multi-dilation rate convolution to enrich the context information and improve the discrimination ability between the target and the background, especially in the case of complex backgrounds or blurred target boundaries, enhancing the detection accuracy. For large target detection, the CoT module is introduced, and the relationship between the target and the background is modeled through context awareness, enabling the model to more accurately locate large targets and effectively reducing the false detection and missed detection risks caused by background interference. It solves the problem that the existing steel surface defect detection technology cannot accurately identify multi-scale targets.

[0117] Embodiment 3

[0118] Please refer to Figure 11 , which shows the steel surface defect detection device based on the improvement of YOLO11n proposed in the third embodiment of the present invention. The device includes:

[0119] An acquisition module 100, configured to acquire the steel surface image to be detected collected by the image acquisition device;

[0120] The detection module 200 is configured to input the surface image of the steel to be detected into the defect detection model trained by the improved YOLO11n model architecture, so as to obtain the surface defect detection information of the surface image of the steel to be detected;

[0121] Among them, the improved YOLO11n model architecture includes a backbone network, a detection head, and a Neck network arranged between the backbone network and the detection head. The backbone network includes a C3k2_NAM module formed by integrating the C3k2 module with the NAM dynamic attention mechanism to adaptively adjust the channel and spatial weights according to the features of the input image. The Neck network introduces the NCB convolution block, the MSDA attention mechanism, and the CoT Attention module. The NCB convolution block is used to enhance the global correlation of small target features through non-local operations. The MSDA attention mechanism is used to combine multi-dilation rate convolutions to enrich the context information and improve the discrimination ability between the target and the background. The CoT Attention module is used to model the relationship between the target and the background through context awareness.

[0122] Furthermore, for the above-mentioned steel surface defect detection device improved based on YOLO11n, among them, the neck module in the C3k2 module and the neck module of the C3k module in the C3k2 module are replaced with the NAM neck module to form a C3k2_NAM module including C3k_NAM;

[0123] When C3k_NAM is True, the input features received by the C3k2_NAM module undergo 1×1 convolution and 3×3 convolution to extract local features and perform preliminary dimensionality reduction or channel transformation;

[0124] The input features after convolution processing are divided into multiple branches and each branch is input into the C3k_NAM module. In the C3k_NAM module, the divided features undergo 1×1 convolution and 3×3 convolution to extract local features and perform preliminary dimensionality reduction or channel transformation;

[0125] The input features after convolution processing respectively undergo recursive processing by multiple NAM neck modules and directly undergo 1×1 convolution and 3×3 convolution;

[0126] The input features after recursive processing and re-convolution are feature concatenated and then undergo 1×1 convolution and 3×3 convolution to obtain the features processed by the C3k_NAM module;

[0127] The features processed by all C3k_NAM modules are feature concatenated and then undergo 1×1 convolution and 3×3 convolution to obtain the finally processed features of the C3k2_NAM module;

[0128] When C3k_NAM is False, the input features received by the C3k2_NAM module go through 1×1 convolution and 3×3 convolution to extract local features and perform preliminary dimensionality reduction or channel transformation;

[0129] The input features processed by convolution are divided into multiple branches and each branch is input into the NAM neck module. After the features processed by all NAM neck modules are feature concatenated, they go through 1×1 convolution and 3×3 convolution to obtain the finally processed features of the C3k2_NAM module.

[0130] Furthermore, in the above steel surface defect detection device improved based on YOLO11n, the NCB convolution block is set in the P3 branch of the Neck network, and the NCB convolution block introduces a multi-head convolution attention mechanism and a multi-layer perceptron;

[0131] When the NCB convolution block receives the corresponding multi-scale feature map from the backbone network, it first performs normalization processing;

[0132] Then, group convolution in the multi-head convolution attention mechanism is used to extract local features, and pointwise convolution is used for information fusion in the channel dimension;

[0133] The two fully connected layers included in the multi-layer perceptron are used for non-linear transformation;

[0134] Batch Normalization is adopted for normalization, and combined with the ReLU activation function for non-linear activation;

[0135] Finally, the corresponding feature map is output and transmitted to the detection head for target recognition.

[0136] Furthermore, in the above steel surface defect detection device improved based on YOLO11n, in the P4 branch of the Neck network, the MSDA attention mechanism is introduced;

[0137] The MSDA attention mechanism is used for sparse self-attention calculation within the sliding window, where the self-attention calculation formula is:

[0138]

[0139] Where respectively represent the feature matrices of the query vector, key vector, and value vector of the i-th attention head, is the dilation rate, which controls the receptive field size;

[0140] Multi-head aggregation is performed through a linear layer:

[0141]

[0142] Whereh 1 ，h 2 ，...，h n respectively represent the feature vectors output by the 1st to the nth attention heads, Concat represents the concatenation operation, and Linear represents fusing the information of multiple attention heads through a linear layer.

[0143] Furthermore, in the above steel surface defect detection device improved based on YOLO11n, in the P4 branch of the Neck network, a CoT Attention module is introduced;

[0144] The CoT Attention module uses a 3×3 convolution to capture the local relationship between keys in the received input features:

[0145]

[0146] Among them, K1 represents the key in the static context, K is the original key input, represents a 3×3 convolution, and key represents the context structure information extracted from the input features through local convolution;

[0147] The K1 of the static context is concatenated with the input Q, and the attention matrix is calculated through two 1×1 convolutions:

[0148]

[0149] Among them, [K1,Q] represents the concatenation of the key and query of the static context, is a 1×1 convolution kernel used to generate the attention matrix A, and Q represents the information encoding of each position in the input;

[0150] Through the attention matrix A, the CoT Attention module performs weighted aggregation on the input value 𝑉 to obtain a dynamic context representation:

[0151]

[0152] The CoT Attention module fuses the static context K1 and the dynamic context K2, and generates the final output feature Y through an addition operation.

[0153] Furthermore, in the above steel surface defect detection device improved based on YOLO11n, the training process of the defect detection model includes:

[0154] Select a public dataset, and after data cleaning, image enhancement, and annotation processing in sequence, obtain the corresponding training dataset;

[0155] Divide the training dataset into a training set, a validation set, and a test set according to a preset ratio;

[0156] Define a loss function and train the improved YOLO11n model architecture on the training set until the training stop condition is met to obtain a defect detection model.

[0157] Further, in the steel surface defect detection device improved based on YOLO11n, the loss function adopts one of cross-entropy loss combined with Inner-IoU loss, and regression loss combined with Inner-IoU loss.

[0158] The functions or operation steps implemented when the above modules are executed are substantially the same as those in the above method embodiments, and will not be described in detail here.

[0159] Example 4

[0160] On the other hand, the present invention also provides a readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method described in any one of the above Embodiment 1 to Embodiment 2 are implemented.

[0161] Example 5

[0162] On the other hand, the present invention also provides an electronic device, the electronic device includes a memory, a processor, and a computer program stored on the memory and running on the processor, and when the processor executes the program, the steps of the method described in any one of the above Embodiment 1 to Embodiment 2 are implemented.

[0163] The technical features of each of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0164] Those skilled in the art can understand that the logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a definite sequence list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable storage medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch instructions from the instruction execution system, apparatus, or device and execute the instructions), or in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable storage medium" can be any device that can contain, store, communicate, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0165] More specific examples (nonexhaustive list) of computer-readable storage media include the following: electrical connection parts (electronic devices) with one or more wirings, portable computer disk cartridges (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber devices, and portable compact disc read-only memory (CDROM). Additionally, the computer-readable storage media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other suitable processing when necessary, and then stored in a computer memory.

[0166] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits with suitable combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0167] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0168] The above-described embodiments merely represent several implementation manners of the present invention, and their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

Claims

1. A steel surface defect detection method improved based on YOLO11n, characterized in that, The method includes: Obtaining a surface image of steel to be detected collected by an image acquisition device; Inputting the surface image of the steel to be detected into a defect detection model trained by an improved YOLO11n model architecture to obtain surface defect detection information of the surface image of the steel to be detected; Among them, the improved YOLO11n model architecture includes a backbone network, a detection head, and a Neck network arranged between the backbone network and the detection head. The backbone network includes a C3k2_NAM module formed by a C3k2 module incorporated with a NAM dynamic attention mechanism to adaptively adjust channel and spatial weights according to the features of the input image. The Neck network introduces an NCB convolutional block, an MSDA attention mechanism, and a CoT Attention module. The NCB convolutional block is used to enhance the global correlation of small target features through non-local operations. The MSDA attention mechanism is used to combine multi-dilation rate convolutions to enrich context information and improve the discrimination ability between the target and the background. The CoT Attention module is used to model the relationship between the target and the background through context awareness.

2. The steel surface defect detection method improved based on YOLO11n according to claim 1, characterized in that, The neck module in the C3k2 module and the neck module of the C3k module in the C3k2 module are replaced with a NAM neck module to form a C3k2_NAM module containing C3k_NAM; When C3k_NAM is True, the input features received by the C3k2_NAM module undergo 1×1 convolution and 3×3 convolution to extract local features and perform preliminary dimensionality reduction or channel transformation; The input features after convolution processing are divided into multiple branches and each branch is input into the C3k_NAM module. In the C3k_NAM module, the divided features undergo 1×1 convolution and 3×3 convolution to extract local features and perform preliminary dimensionality reduction or channel transformation; The input features after convolution processing respectively undergo recursive processing by multiple NAM neck modules and directly undergo 1×1 convolution and 3×3 convolution; The input features after recursive processing and re-convolution are feature concatenated and then undergo 1×1 convolution and 3×3 convolution to obtain the features processed by the C3k_NAM module; The features processed by all C3k_NAM modules are feature concatenated and then undergo 1×1 convolution and 3×3 convolution to obtain the finally processed features of the C3k2_NAM module; When C3k_NAM is False, the input features received by the C3k2_NAM module undergo 1×1 convolution and 3×3 convolution to extract local features and perform preliminary dimensionality reduction or channel transformation; The input features after convolution processing are divided into multiple branches and each branch is input into the NAM neck module. The features processed by all NAM neck modules are feature concatenated and then undergo 1×1 convolution and 3×3 convolution to obtain the finally processed features of the C3k2_NAM module.

3. The steel surface defect detection method improved based on YOLO11n according to claim 1, characterized in that, The NCB convolutional block is arranged in the P3 branch of the Neck network, and the NCB convolutional block introduces a multi-head convolutional attention mechanism and a multi-layer perceptron; When the NCB convolutional block receives the corresponding multi-scale feature map from the backbone network, it first performs normalization processing; Subsequently, group convolution in the multi-head convolutional attention mechanism is used to extract local features, and pointwise convolution is used for information fusion in the channel dimension; The two fully connected layers included in the multi-layer perceptron are used for non-linear transformation; Batch Normalization is adopted for normalization, and combined with the ReLU activation function for non-linear activation; Finally, the corresponding feature map is output and passed to the detection head for target recognition.

4. The steel surface defect detection method improved based on YOLO11n according to claim 3, characterized in that, In the P4 branch of the Neck network, the MSDA attention mechanism is introduced; The MSDA attention mechanism is used for sparse self-attention calculation within the sliding window, where the self-attention calculation formula is: Among them, respectively represent the feature matrices of the query vector, key vector, and value vector of the i-th attention head, is the dilation rate, which controls the receptive field size; Multi-head aggregation is performed through a linear layer: Among them, h 1 ，h 2 ，...，h n respectively represent the feature vectors output by the 1st to the nth attention heads, Concat represents the concatenation operation, and Linear represents fusing the information of multiple attention heads through a linear layer.

5. The steel surface defect detection method improved based on YOLO11n according to claim 4, wherein, In the P4 branch of the Neck network, the CoT Attention module is introduced; The CoT Attention module uses a 3×3 convolution to capture the local relationship between keys in the received input features: Among them, K1 represents the key in the static context, and K is the original key input. represents a 3×3 convolution, and the key represents the context structure information extracted from the input features through local convolution. The K1 of the static context is concatenated with the input Q, and the attention matrix is calculated through two 1×1 convolutions: Among them, [K1, Q] represents the concatenation of the key and query of the static context, is a 1×1 convolutional kernel used to generate the attention matrix A, and Q represents the information encoding of each position in the input itself; Through the attention matrix A, the CoT Attention module performs weighted aggregation on the input value 𝑉 to obtain the dynamic context representation: The CoT Attention module fuses the static context K1 and the dynamic context K2, and generates the final output feature Y through an addition operation.

6. The steel surface defect detection method improved based on YOLO11n according to claim 1, characterized in that The training process of the defect detection model includes: Select a public dataset, and sequentially perform data cleaning, image enhancement, and annotation processing to obtain the corresponding training dataset; The training dataset is divided into a training set, a validation set, and a test set according to a preset ratio; Define a loss function, and train the improved YOLO11n model architecture on the training set until the training stop condition is met to obtain the defect detection model.

7. The steel surface defect detection method improved based on YOLO11n according to claim 6, characterized in that, The loss function adopts one of cross-entropy loss combined with Inner-IoU loss, and regression loss combined with Inner-IoU loss.

8. A steel surface defect detection device improved based on YOLO11n, characterized in that, The device includes: An acquisition module, which is used to acquire the surface image of the steel to be detected collected by the image acquisition device; A detection module, which is used to input the surface image of the steel to be detected into the defect detection model trained by the improved YOLO11n model architecture to obtain the surface defect detection information of the surface image of the steel to be detected; Among them, the improved YOLO11n model architecture includes a backbone network, a detection head, and a Neck network arranged between the backbone network and the detection head. The backbone network includes a C3k2_NAM module formed by integrating the NAM dynamic attention mechanism into the C3k2 module to adaptively adjust the channel and spatial weights according to the features of the input image. The Neck network introduces an NCB convolutional block, an MSDA attention mechanism, and a CoT Attention module. The NCB convolutional block is used to enhance the global correlation of small target features through non-local operations. The MSDA attention mechanism is used to combine multi-dilation rate convolutions to enrich the context information and improve the discrimination ability between the target and the background. The CoT Attention module is used to model the relationship between the target and the background through context awareness.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the described program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and running on the processor. When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

Images