A steel defect detection method based on edge enhancement extraction

Abstract

The application discloses a steel defect detection method based on edge enhancement extraction and relates to the technical field of image recognition, and comprises the following steps: constructing and preprocessing a steel surface defect dataset, and dividing a training set, a verification set and a test set; taking YOLOv11 as a benchmark, improving a YOLOv11 network structure; improving the original YOLOv11 network to set training parameters, verifying a steel defect detection model obtained through training; and comparing the improved model to be used for real-time steel defect detection. The application overcomes the problem of fuzzy edge contour positioning caused by poor robustness and similar defects and background, and has the advantages of high detection accuracy and faster detection speed.

Description

Technical Field

[0001] This invention designs a steel defect detection method based on edge enhancement extraction, belonging to the field of image recognition technology. Background Technology

[0002] Steel, as the cornerstone of modern industry, is indispensable in construction, automobile manufacturing, and numerous other fields. However, its production, transportation, and storage processes are susceptible to complex technologies and conditions, often leading to surface defects such as cracks and dents. These defects not only affect appearance but also reduce mechanical properties, create safety hazards, and even cause major accidents. Therefore, detecting surface defects in steel to ensure its quality is of paramount importance. This is significant not only for accelerating the entire production and inspection process but also for guaranteeing the steel's inherent quality and long-term safety during construction and use.

[0003] With the development of technology, deep learning-based object detection technology has gradually become the mainstream detection technology, and this method has also been applied to the detection of defects in steel. The YOLO series of algorithms, as one of the representatives of single-stage object detection algorithms, is highly favored for its excellent detection speed. However, in steel surface defect detection, defects are diverse in type and size, with tiny defects occupying only a small number of pixels in the image and possessing complex textures. This makes it difficult to extract defect edge features, distinguish defects from the background, and accurately locate tiny defects, increasing the detection difficulty. Most current models fail to focus on the edge contour information of defects. Therefore, in steel defect detection tasks, it is necessary to propose a method that can effectively focus on the edge contour features of defects and improve the defect detection effect. Summary of the Invention

[0004] Objective: This invention aims to provide a steel defect detection method based on edge enhancement extraction. Based on YOLOv11, an EEMSE module is designed to improve the Bottleneck part of the C3k2 module in the original network where the C3k parameter is False. Simultaneously, a self-designed EEMSE module is used to improve the detection head structure, thereby enhancing and recognizing the defect edge contours and achieving defect localization with faster detection speed. Compared to conventional detection models, this method focuses more on the edge contour information of defects, thus achieving effective defect location and accurate defect detection. Furthermore, this method has a significant advantage in detection speed, better meeting the practical requirements for a balance between detection accuracy and speed.

[0005] Technical solution: This invention provides a method for detecting defects in steel based on edge enhancement extraction, the method comprising the following steps:

[0006] Step 1: Construct and process the steel surface defect dataset, and divide it into training set, validation set, and test set;

[0007] Step 2: Improve the network model based on YOLOv11.

[0008] Step 2.1: Construct an edge enhancement module (EEM) to suppress low-frequency information and enhance high-frequency information of edge contours during feature extraction.

[0009] Step 2.2: Design the multi-scale edge enhancement extraction module MSEEM. This module combines feature maps that have undergone edge enhancement at different scales, and then uses a dual-domain selection mechanism to process the enhanced feature maps, giving them clearer edge contour information. Replace the C3k parameter in the C3k2 module with the Bottleneck part under the Flase branch using the MSEEM module.

[0010] Step 2.3: Combine EEM and SE attention mechanisms to form the EEMSE module. Use EEMSE to improve the detection head part, thereby improving the model's accurate positioning of defect edges while reducing the number of model parameters and increasing detection speed.

[0011] Step 3: Improve the original model, set training parameters, and validate the trained model.

[0012] Step 4: Compare the improved model with the actual model and use it for real-time detection of defects in steel.

[0013] Furthermore, the steel surface defect dataset mentioned in step 1 contains 1800 defect images across six types of defects: cracks, inclusions, spots, pitting corrosion, oxide scale, and scratches. The preprocessing step primarily involves dividing the image data into training, validation, and test sets in an 8:1:1 ratio. The training set contains 1440 images, the validation set contains 180 images, and the test set contains 180 images.

[0014] Furthermore, the specific operating steps of the edge enhancement module (EEM) described in step 2.1 are as follows:

[0015] For the input feature image, the high-frequency edge information is first blurred by average pooling, thus preserving the low-frequency information of the entire image. Then, the pooled feature map information is subtracted from the original feature map data to obtain the high-frequency information, i.e., the edge information. The features enhanced by the edge enhancement operation are added to the original features to form a residual connection, thus ensuring that the semantic information of the original image is preserved while enhancing the edge features.

[0016] Furthermore, the multi-scale edge enhancement extraction module MSEEM described in step 2.2 mainly consists of two parts: multi-scale edge enhancement extraction and a dual-domain selection mechanism. Its specific structure is as follows:

[0017] In the multi-scale edge enhancement extraction part, for the input feature image, the task is first divided into len(bins) + 1 branches based on the length of the multi-scale feature extraction list bins. These branches mainly consist of multiple multi-scale feature extraction branches and one local feature extraction branch. The values ​​stored in the bins list represent the pooling target size applied to the feature map. Based on the number of target sizes defined in the list, a corresponding branch is assigned to each scale for processing, thereby pooling the feature map to the specified scale for feature extraction.

[0018] Furthermore, in the multi-scale feature extraction branch, adaptive average pooling is used to adjust the feature image to different target sizes. Information extraction and edge enhancement are then achieved through convolution operations and EEM module processing. Finally, upsampling restores the image to its original size, resulting in a multi-scale enhanced feature image. In the local feature extraction branch, only simple feature extraction operations are performed. Ultimately, all branch feature maps are fused along the channel dimension. The enhanced feature image not only highlights target edges but also possesses stronger expressive power, making it suitable for detection tasks.

[0019] For the acquired feature image enhanced with multi-scale edge enhancement, the dual-domain selection mechanism further processes the image by combining image differences in the spatial and spectral domains. By locating key regions of the input image and then enhancing them, noise is suppressed, and the image's feature representation capability is further improved.

[0020] Furthermore, the EEMSE module described in step 2.3 comprises two parts: the edge feature enhancement module EEM and the SE attention mechanism. The specific structure of this module is as follows:

[0021] In the EEM module processing section, high-frequency edge information is obtained by subtracting pooled features from the original features, and then added to the original features to form a residual structure, which finally yields more obvious edge feature information in the feature image.

[0022] In the SE attention mechanism, the spatial information of each channel of the input feature map of size H×W×C is compressed into a single value through a Squeeze operation, forming a 1×1×C vector, thus preserving the global information of each channel. Subsequently, in the Excitation operation, the vector is processed through a fully connected layer to generate channel weights, thereby adjusting the feature intensity of each channel in the original feature map and obtaining a weighted feature image.

[0023] Furthermore, in step 3, the training configuration used was CUDA 11.8, and an RTX 4090 GPU was used for accelerated computation. The training iterations were all 300 rounds, and the batch size was set to 16.

[0024] Furthermore, the improved model obtained in step 4 achieved improvements of 4.1% and 2.2% in mAP@0.5 and mAP@0.5:0.95 in YOLOv11n, respectively, and the Gflops value decreased from the original 6.4 to 6.2, achieving the requirement of improving detection accuracy while reducing computation time. After verification, the model can be loaded into the visualization interface to realize real-time defect detection.

[0025] Beneficial effects:

[0026] This invention replaces the Bottleneck portion of the Flase branch in the C3k2 module of the original model with the MSEEM module designed in this invention. This module fuses and processes feature images enhanced with edge contours at different scales, enhancing defect edge contour features while preserving the original image information. By combining the SE attention mechanism with the EEM module, an EEMSE module is proposed to improve the detection head structure. This module dynamically adjusts the importance of each channel while enhancing the defect feature edge contours, thereby enhancing the model's focus on channels with important information and improving feature representation capabilities. In addition, this module has fewer parameters, effectively contributing to the improvement of model detection speed. By improving the YOLOv11 network using a self-designed module, the final trained model can pay more attention to the edge contour information of the defect target, achieving accurate defect detection with faster computation speed. Attached Figure Description

[0027] Figure 1 The detection flowchart designed for this invention;

[0028] Figure 2 This is a diagram of the improved network model structure.

[0029] Figure 3 This is a structural diagram of the EEM module designed for this invention;

[0030] Figure 4 The structural diagram of the MSEEM module designed for this invention;

[0031] Figure 5 A comparison diagram of the C3k2 and the improved C3k2_MutilScaleEdgeInformationSelect structures;

[0032] Figure 6 The structural diagram of the EEMSE module designed for this invention;

[0033] Figure 7 Diagram of the improved detection head structure;

[0034] Figure 8 The figure shows the experimental results of the model in the design example of this invention; Detailed Implementation

[0035] The present invention will now be described in further clarity and completeness with reference to the accompanying drawings. The specific embodiments described below are for illustrative purposes only and are not intended to limit the scope of the invention.

[0036] like Figure 1 As shown, a steel defect detection method based on edge enhancement extraction is implemented through the following steps:

[0037] Step 1: Use the NEU-DET steel surface defect dataset released by Northeastern University as the base dataset;

[0038] Step 2: The dataset contains 1800 defect images across six defect categories: cracks, inclusions, spots, pitting corrosion, oxide scale, and scratches, with 300 samples per category. The defect sample dataset is divided into training, validation, and test sets in an 8:1:1 ratio.

[0039] Step 3: Improve the steel surface defect detection model based on YOLOvl1. The improved network structure is as follows: Figure 2 As shown, the specific improvement measures are as follows:

[0040] Step 3.1: Design an edge enhancement module (EEM) to suppress low-frequency information and highlight high-frequency information at the defect edges.

[0041] Typically, the background of a feature map is a low-frequency component, while the defect edges exhibit rapid changes and are therefore high-frequency components. The design philosophy of the EEM module is to obtain low-frequency information from the feature image through average pooling, and then subtract the pooled features from the original features; the result is the high-frequency edge information. By combining the original feature image to form residual connections, EEM can enhance the target edge contour features while preserving the original semantic information of the image. The EEM module structure diagram is shown below. Figure 3 As shown.

[0042] Step 3.2: Design a multi-scale edge feature extraction module MSEEM based on EEM, and use it to replace the Bottleneck part in the C3k2 module where the C3k parameter is False.

[0043] Based on the length of the multi-scale feature extraction list `bins`, the task is divided into `len(bins) + 1` branches for processing. For the input feature map, the values ​​stored in the `bins` list represent the pooling target size applied to the feature map. The branches include multiple multi-scale feature extraction branches and one local feature extraction branch. In the multi-scale feature extraction branch, adaptive average pooling is used to dynamically adjust the feature image to a specified size according to the output target size. Then, convolution operations and EEM are used to extract information and enhance edge information in the feature image, and upsampling is used to restore the feature image size. This yields feature images enhanced with edge information at different scales. In the local feature extraction branch, only simple feature extraction is performed on the feature image. Finally, the processing results of all branches are fused along the channel dimension, giving the feature image better expressive power while highlighting target edge features. Further processing is then performed using a dual-domain selection mechanism. The MSEEM module structure is as follows: Figure 4 As shown.

[0044] By combining image differences in the spatial and spectral domains, the dual-domain selection mechanism enables the model to focus more on key regions in the image. The dual-domain selection mechanism mainly consists of a spatial selection module and a frequency selection module. The spatial selection module helps the network focus on key regions such as degenerate regions in the spatial domain and provides the initial locations of these regions to the frequency selection module. For the input feature map F∈R... H×W×C The spatial selection module will combine the results of max pooling and average pooling in a concatenated manner to generate a general feature map F′∈R. H×W×1 This feature contains information about the location of degradation. This process can be expressed as:

[0045] F′=Conv3([AvgPool(F),MaxPool(F)]) (1)

[0046] Here, Conv3 represents a convolution operation with a kernel size of 3×3, and AvgPool(F) and MaxPool(F) represent average pooling and max pooling operations on the feature map, respectively. Secondly, by applying depthwise convolution to the feature map, channel-level representations can be obtained, which can be expressed as follows:

[0047]

[0048] Among them, DConvs 5，7 This indicates that depthwise convolution operations are performed with kernel sizes of 5×5 and 7×7, similar to DConv3. T(F′, C) represents copying the general feature map C times along the channel dimension. This represents element-wise multiplication. F sThis is the final spatial selection feature map.

[0049] Subsequently, the frequency selection module further extracts high-frequency features from the feature map by removing low-frequency signals, making key information, including blurred regions and edge information, more clearly presented. The final feature map output by the MSEEM module will maintain clarity while possessing more prominent edge information, thus effectively meeting subsequent detection requirements. The C3k2 module is improved by using the MSESE module, with the C3k parameter set to the Bottleneck part under the Flase branch, forming C3k2_MutilScaleEdgeInformationSelect. A comparison of the original C3k2 structure and the improved C3k2_MutilScaleEdgeInformationSelect structure is shown below. Figure 5 As shown.

[0050] Step 3.3: Based on the EEM module, the SE attention mechanism is integrated to design the EEMSE module, and the detection head structure is improved using it.

[0051] The SE attention mechanism enhances the model's expressive power through two operations: compression and excitation of the input feature image. The Squeeze operation uses global average pooling to compress the spatial information of each channel in the H×W×C feature map into a single numerical value, forming a 1×1×C vector, thus preserving the global information of each channel. Subsequently, the Excitation operation further processes the resulting vector through two fully connected layers to obtain channel weights. These weights are used to adjust the feature intensity of each channel in the original feature map, resulting in a weighted feature image. By introducing the SE attention mechanism into the EEM module, the EEMSE module is constructed, with the structure as follows: Figure 6 As shown. The enhanced feature image is input into the SE module to dynamically adjust the importance of each channel, thus making the model focus more on channels with important information and improving the effect of feature representation. By introducing the EEMSE module into the original toilet head structure, a new detection head structure Detect_EdgeEnhancerSE is constructed, the structure of which is shown below. Figure 7 As shown in the diagram. Ultimately, this approach can improve the ability to represent edge features while reducing the amount of parameter computation and increasing detection speed.

[0052] Step 4: Set training parameters and train the model on the steel surface defect dataset using the improved network, while using GPU acceleration for training. The experimental environment was set to Python 3.8, CUDA 11.8, and RTX 4090 GPU for accelerated training. The total number of training epochs was 300, and the batch size was 16.

[0053] Step 5: Model validation. Set validation parameters and validate the model trained in Step 4.

[0054] Validation results show that applying the improved scheme to the YOLOv11n / s / m networks respectively resulted in improvements of 4.1%, 2.2%, and 2.7% in mAP@0.5. The improved YOLOv11 achieved a 2.2% improvement at mAP@0.5:0.95, while the improved YOLOv11m achieved a 1.9% improvement. Furthermore, the improved Gflops values ​​decreased by 0.2, 0.6, and 3.5 respectively.

[0055] Step 6: Use the trained model to detect defects on the steel surface.

[0056] The comparative experiments in this embodiment are shown in Table 1. The reliability of the improvement is verified by comparing the improved model with some current mainstream models. The evaluation metrics include Precision, Recall, mAP, Parameters, and Gflops. Among them, Precision and Recall are calculated as shown in formulas (3) and (4). mAP can measure the average detection performance of the model for all categories at different confidence thresholds. According to different IoU threshold ranges, mAP can be further subdivided into two cases: mAP@0.5 and mAP@0.5:0.95. The former only represents the mAP when the IoU threshold is 0.5, while the latter reflects the average mAP calculated between the IoU thresholds of 0.5 and 0.95, which is relatively more comprehensive.

[0057]

[0058] Wherein, TP is the number of samples correctly predicted as positive by the model, FP is the number of samples incorrectly predicted as positive by the model, and FN is the number of samples incorrectly predicted as negative by the model.

[0059] Table 1

[0060]

[0061] The training effects of the model before and after the improvement are as follows: Figure 8 As shown in the figure, the light-colored lines represent the training trend of the original model, while the dark-colored lines correspond to the training process of the improved model. It can be seen that in the early stages, the mAP values ​​of each model fluctuated upwards with the increase of training iterations, while the improved model was more stable than the original. When training reached 200 epochs, the fluctuations in the model's mAP value began to stabilize, at which point the improved network model was significantly better than the original model. In a stable state, the mAP value of the improved model was significantly higher than that of the original, thus confirming that the improvement had a good effect on improving the model's detection performance.

[0062] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting steel defects based on edge enhancement extraction, characterized in that, Includes the following steps: Step 1: Construct and preprocess the steel surface defect dataset, and divide it into training set, validation set, and test set; Step 2: Based on YOLOv11, improve the YOLOv11 network structure; Step 2.1: Construct an edge enhancement module (EEM) to suppress low-frequency information and enhance high-frequency information of edge contours during feature extraction; Step 2.2: Design a multi-scale edge enhancement extraction module MSEEM. By combining feature maps that have undergone edge enhancement at different scales, and then using a dual-domain selection mechanism to process the enhanced feature maps, the feature maps will have clearer edge contour information. Replace the C3k parameter in the C3k2 module with the Bottleneck part under the Flase branch using the MSEEM module. Step 2.3: Combine the edge enhancement module (EEM) and the SE attention mechanism to form the EEMSE module. Use the EEMSE module to improve the detection head, thereby improving the model's accurate localization of defect edges while reducing the number of model parameters and increasing the detection speed. Step 3: Improve the training parameters of the original YOLOv11 network and verify the trained steel defect detection model; Step 4: Compare the improved model with the actual model and use it for real-time detection of defects in steel. The multi-scale edge enhancement extraction module MSEEM described in step 2.2 consists of two parts: multi-scale edge enhancement extraction and a dual-domain selection mechanism. Its specific structure is as follows: In the multi-scale edge enhancement extraction part, for the input feature image, the task is first divided into len(bins) + 1 branches according to the length of the multi-scale feature extraction list bins. The branches are divided into multiple multi-scale feature extraction branches and one local feature extraction branch. The size of the value stored in the bins list represents the pooling target size applied to the feature map. According to the number of target sizes defined in the list, a corresponding branch is assigned to each scale for processing, thereby pooling the feature map to the specified scale for feature extraction. In the multi-scale feature extraction branch, the feature image is adjusted to different target sizes through adaptive average pooling. Information extraction and edge enhancement are achieved through convolution operation and EEM module processing. Then, it is restored to the original size through upsampling to obtain the multi-scale enhanced feature image. In the local feature extraction branch, only simple feature extraction operation is performed. Finally, all branch feature maps are fused in the channel dimension. For the acquired feature image after multi-scale edge enhancement, the dual-domain selection mechanism will further process it by combining the image differences in the spatial and spectral domains. By locating the key regions of the input image and then performing enhancement processing, noise is suppressed, and the feature representation capability of the image is further improved.

2. The steel defect detection method based on edge enhancement extraction according to claim 1, characterized in that, The steel surface defect dataset mentioned in step 1 contains 1800 defect images of 6 types of defects, including cracks, inclusions, spots, pitting corrosion, oxide scale, and scratches. The preprocessing step is to divide the image data into training set, validation set, and test set in a ratio of 8:1:

1. The training set contains 1440 images, the validation set contains 180 images, and the test set contains 180 images.

3. The steel defect detection method based on edge enhancement extraction according to claim 1, characterized in that, The specific operation steps of the edge enhancement module (EEM) mentioned in step 2.1 are as follows: For the input steel defect feature image, the high-frequency information of the edges in the image is first blurred by average pooling, thereby preserving the low-frequency information of the whole image. Then, the feature map information after pooling is subtracted from the original steel defect feature map data to obtain the high-frequency information part of the image, that is, the edge information part. The features after edge enhancement are added to the original features to form a residual connection, thereby ensuring that the semantic information of the original image is preserved while enhancing the edge features.

4. The steel defect detection method based on edge enhancement extraction according to claim 1, characterized in that, The EEMSE module described in step 2.3 consists of two parts: the edge feature enhancement module EEM and the SE attention mechanism. The specific structure of this module is as follows: In the EEM module processing section, high-frequency edge information is obtained by subtracting pooled features from the original features, and then added to the original features to form a residual structure, ultimately obtaining more obvious edge feature information in the feature image; In the SE attention mechanism, the input size is... The spatial information of each channel of the feature map will be compressed into a single value through the Squeeze operation, forming The vector is generated to preserve the global information of each channel. Then, in the excitation operation, the vector is processed by a fully connected layer to generate channel weights, thereby adjusting the feature intensity of each channel in the original feature map and obtaining the weighted feature image.

5. The steel defect detection method based on edge enhancement extraction according to claim 1, characterized in that, In step 3, the training configuration used was CUDA 11.8, and an RTX 4090 GPU was used for accelerated computation. The number of training iterations was 300 rounds, and the batch size was set to 16.

Images