Steel defect detection method based on edge enhancement extraction

Abstract

The invention discloses a steel defect detection method based on edge enhancement extraction, and relates to the technical field of image recognition, and the method comprises the following steps: constructing and preprocessing a steel surface defect data set, and dividing a training set, a verification set and a test set; the method comprises the following steps of: improving a YOLOv11 network structure by taking YOLOv11 as a reference; an original YOLOv11 network is improved, training parameters are set, and the steel defect detection model obtained through training is verified; and comparing the improved model for real-time detection of steel defects. The method overcomes the problem of fuzzy edge contour positioning caused by poor robustness and similar defects and backgrounds, and has the advantages of high detection accuracy and higher detection speed.

Description

Technical Field

[0001] The invention designs a steel defect detection method based on edge enhancement extraction, belonging to the technical field of image recognition. Background Art

[0002] Steel, a cornerstone of modern industry, is indispensable in construction, automotive manufacturing, and numerous other fields. However, its production, transportation, and storage processes are subject to complex processes and conditions, often leading to surface defects such as cracks and dents. These defects not only affect appearance but also reduce mechanical properties, pose safety risks, and even cause major accidents. Therefore, detecting steel surface defects and ensuring surface quality is crucial. This not only speeds up the entire production and testing process but also ensures the quality of the steel itself and its long-term safety during construction and use.

[0003] With the development of technology, target detection technology based on deep learning has gradually become the mainstream detection technology, and this method has also been applied to the research of steel defect detection. As one of the representatives of single-stage target detection algorithms, the YOLO series of algorithms has been favored for its excellent detection speed. However, in the detection of steel surface defects, the types of defects are diverse and the sizes vary. Small defects only occupy a small number of pixels in the image, and their own textures are complex. This makes it difficult to extract the edge features of the defects, and it is difficult to distinguish between defects and backgrounds. Small defects are difficult to accurately locate, which increases the difficulty of detection. Most current models fail to focus on the edge contour information of the defects. Therefore, in the task of steel defect detection, it is necessary to propose a method that can effectively focus on the edge contour features of the defects and improve the defect detection effect. Summary of the Invention

[0004] Purpose of the Invention: The present invention aims to provide a steel defect detection method based on edge enhancement extraction. Based on YOLOv11, the EEMSE module is designed to improve the Bottleneck portion of the C3k2 module in the original network under the branch where the C3k parameter is False. At the same time, the self-designed module EEMSE is used to improve the detection head structure, thereby enhancing and identifying the edge contours of the defect and achieving defect location with faster detection speed. Compared with conventional detection models, this method can pay more attention to the edge contour information of the defect, thereby effectively locating the defect position and accurately detecting the defect. In addition, this method has an advantage in detection speed and can better meet the demand for a balance between detection accuracy and detection speed in actual detection.

[0005] Technical solution: The present invention provides a steel defect detection method based on edge enhancement extraction, the method comprising the following steps:

[0006] Step 1: Construct and process the steel surface defect dataset and divide it into training set, validation set, and test set;

[0007] Step 2: Improve the network model based on YOLOv11

[0008] Step 2.1: Construct an edge enhancement module (EEM) to suppress low-frequency information and enhance high-frequency information of edge contours during feature extraction.

[0009] Step 2.2: Design a multi-scale edge enhancement extraction module (MSEEM). This module combines edge-enhanced feature maps at different scales and processes them using a dual-domain selection mechanism to provide clearer edge contour information. Use the MSEEM module to replace the Bottleneck component of the False branch in the C3k2 module with the C3k parameter.

[0010] Step 2.3: Combine EEM and SE attention mechanisms to form the EEMSE module, and use EEMSE to improve the detection head part to improve the model's accurate positioning of defect edges while reducing the number of model parameters and improving detection speed.

[0011] Step 3: Improve the original model, set training parameters, and verify the trained model.

[0012] Step 4: Compare and improve the obtained model and use it for real-time detection of steel defects.

[0013] Furthermore, the steel surface defect dataset described in step 1 contains 1,800 defect images representing six defect categories: cracks, inclusions, spots, pitting, scale, and scratches. The preprocessing step primarily involves dividing the image data into training, validation, and test sets in an 8:1:1 ratio. The training set contains 1,440 images, the validation set contains 180 images, and the test set contains 180 images.

[0014] Furthermore, the edge enhancement module EEM described in step 2.1 specifically operates as follows:

[0015] For the input feature image, the high-frequency edge information is first blurred through an average pooling operation, thereby preserving the low-frequency information of the entire image. The feature map after the pooling operation is then subtracted from the original feature map data to obtain the high-frequency information portion of the image, namely the edge information portion. The edge-enhanced features are added to the original features to form a residual connection, ensuring that the semantic information of the original image is preserved while enhancing the edge features.

[0016] Furthermore, the multi-scale edge enhancement extraction module MSEEM described in step 2.2 mainly consists of two parts: multi-scale edge enhancement extraction and dual-domain selection mechanism. Its specific structure is as follows:

[0017] In the multi-scale edge enhancement extraction part, for the input feature image, the task is first divided into len(bins)+1 branches based on the length of the multi-scale feature extraction list bins. The branches are mainly divided into multiple multi-scale feature extraction branches and one local feature extraction branch. The size of the values stored in the bins list represents the pooling target size applied to the feature map. Based on the number of target sizes defined in the list, the corresponding branch is assigned to each scale for processing, thereby pooling the feature map to the specified scale for feature extraction.

[0018] Furthermore, in the multi-scale feature extraction branch, adaptive average pooling is used to resize the feature image to different target sizes. Information is extracted and edge enhanced through convolution and processing with the EEM module. Upsampling restores the image to its original size, yielding a multi-scale enhanced feature image. In the local feature extraction branch, only simple feature extraction is performed. Finally, the feature maps from all branches are fused in the channel dimension. The resulting enhanced feature image emphasizes target edges while offering enhanced expressiveness, making it suitable for detection tasks.

[0019] The dual-domain selection mechanism further processes the acquired feature image after multi-scale edge enhancement by combining image differences in the spatial and spectral domains. By locating key areas of the input image for enhancement and suppressing noise, the image's feature expression capabilities are further enhanced.

[0020] Furthermore, the EEMSE module described in step 2.3 consists of two parts, namely the edge feature enhancement module EEM and the SE attention mechanism. The specific structure of the module is as follows:

[0021] In the EEM module processing part, the high-frequency edge information is obtained by subtracting the pooled features from the original features, and then added to the original features to form a residual structure, and finally more obvious edge feature information in the feature image can be obtained.

[0022] In the SE attention mechanism, the spatial information of each channel in the input feature map of size H×W×C is compressed into a single value through the Squeeze operation, forming a 1×1×C vector, thereby preserving the global information of each channel. Subsequently, in the Excitation operation, the vector is processed through a fully connected layer to generate channel weights, which adjust the feature strength of each channel in the original feature map to obtain a weighted feature image.

[0023] Furthermore, in step 3, the training configuration selected was CUDA 11.8, and the RTX 4090 GPU was used for accelerated computing. The number of training iterations used was 300, and the batch size was set to 16.

[0024] Furthermore, the improved model in step 4 achieved 4.1% and 2.2% improvements in YOLOv11n's mAP@0.5 and mAP@0.5:0.95, respectively, and the Gflops value dropped from the original 6.4 to 6.2, achieving the goal of improving detection accuracy while reducing calculation time. After verification, the model can be loaded into the visualization interface for real-time defect detection.

[0025] Beneficial effects:

[0026] The present invention replaces the Bottleneck part under the Flase branch in the C3k2 module of the original model with the MSEEM module designed by the present invention. This module fuses and processes the feature images after edge contour enhancement at different scales, and enhances the edge contour features of the defect while retaining the original information of the image. By combining the SE attention mechanism with the EEM module, an EEMSE module that can be used to improve the detection head structure is proposed. This module dynamically adjusts the importance of each channel while enhancing the edge contour of the defect feature, thereby enhancing the model's attention to channels with important information and improving feature expression capabilities. In addition, the module has fewer parameters, which effectively helps improve the detection speed of the model. By using the self-designed module to improve the YOLOv11 network, the model finally trained can pay more attention to the edge contour information of the defect target and achieve accurate detection of defects with faster computing speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Detection flow chart designed for the present invention;

[0028] Figure 2 This is the structure diagram of the improved network model;

[0029] Figure 3 The EEM module structure diagram designed for the present invention;

[0030] Figure 4 The MSEEM module structure diagram designed for the present invention;

[0031] Figure 5 Comparison diagram of C3k2 and improved C3k2_MutilScaleEdgeInformationSelect structures;

[0032] Figure 6 The EEMSE module structure diagram designed for the present invention;

[0033] Figure 7 This is the structural diagram of the improved detection head;

[0034] Figure 8 This is a diagram of the model experiment results in the design example of the present invention; DETAILED DESCRIPTION

[0035] The present invention will be described in detail below with reference to the accompanying drawings. The specific embodiments of the present invention described below are only used to further explain the present invention and are not intended to limit the present invention.

[0036] like Figure 1 As shown in the figure, a steel defect detection method based on edge enhancement extraction is implemented in the following steps:

[0037] Step 1: Use the NEU-DET steel surface defect dataset released by Northeastern University as the basic dataset;

[0038] Step 2: The dataset contains 1,800 defect images of six types of defects: cracks, inclusions, spots, pitting, scale, and scratches. The number of samples for each defect type is 300. The defect sample dataset is divided into training, validation, and test sets in a ratio of 8:1:1.

[0039] Step 3: Improve the steel surface defect detection model based on YOLOvl1. The improved network structure is as follows: Figure 2 The specific improvement measures are as follows:

[0040] Step 3.1: Design the edge enhancement module EEM to suppress low-frequency information and highlight the high-frequency information of the defect edge.

[0041] Usually, the background part of the feature map is the low-frequency part, while the defect edge shows rapid changes and is therefore the high-frequency part. The design idea of the EEM module is to obtain the low-frequency information of the feature image through the average pooling operation, and then subtract the features after the pooling operation from the original features. The result is the high-frequency information part of the edge. By combining the original feature image to form a residual connection, EEM can enhance the target edge contour features while retaining the original semantic information of the image. The EEM module structure diagram is shown in the figure below. Figure 3 shown.

[0042] Step 3.2: Design a multi-scale edge feature extraction module MSEEM based on EEM, and use it to replace the Bottleneck part under the branch where the C3k parameter is False in the C3k2 module.

[0043] According to the length of the multi-scale feature extraction list bins, the task will be divided into len(bins)+1 branches for processing. For the input feature map, the size of the value stored in the bins list represents the pooling target size applied to the feature map. The branch contains multiple multi-scale feature extraction branches and 1 local feature extraction branch. In the multi-scale feature extraction branch, according to the different output target sizes, the adaptive average pooling operation will be used to dynamically adjust the feature image to the specified size, and then the convolution operation and EEM will be used to extract information and enhance the edge information of the feature image, and the upsampling operation will be used to restore the feature image size. In this way, feature images with edge information enhancement at different scales can be obtained. In the local feature extraction branch, only simple feature extraction operations are performed on the feature image. Finally, the processing results of all branches are fused in the channel dimension, so that the feature image has better expression ability while highlighting the target edge features, and is then further processed by the dual-domain selection mechanism. The MSEEM module structure is as follows Figure 4 shown.

[0044] By combining image differences in the spatial domain and the spectral domain, the dual-domain selection mechanism enables the model to focus more on key areas in the image. The dual-domain selection mechanism mainly consists of a spatial selection module and a frequency selection module. Among them, the spatial selection module helps the network focus on key areas such as degraded areas in the spatial domain and provides the initial positions of these areas for the frequency selection module. For the input feature map F∈R H×W×C , the spatial selection module will combine the results of the maximum pooling and average pooling in series to generate a universal feature map F′∈R H×W×1 , which contains the information of the degenerate position. This process is expressed as:

[0045] F′=Conv3([AvgPool(F),MaxPool(F)]) (1)

[0046] Among them, Conv3 represents the convolution operation with a convolution kernel size of 3×3, AvgPool(F) and MaxPool(F) represent the average pooling and maximum pooling operations on the feature map respectively. Secondly, by using depthwise convolution on the feature map, channel-level representation can be obtained. This process is expressed as:

[0047]

[0048] Among them, DConvs 5，7 Indicates that the convolution operation is performed with kernel sizes of 5×5 and 7×7. The same is true for DCnv3. T(F′, C) represents replicating the general feature map C times along the channel dimension. F stands for element-wise multiplication. sThis is the final spatial selection feature map.

[0049] Subsequently, the frequency selection module will further obtain the high-frequency features of the feature map by removing the low-frequency signals, so that the key information in the feature map, including the fuzzy area and edge information, can be presented more clearly. The final feature map output by the MSEEM module will be able to have more prominent edge information while maintaining clarity, thereby effectively responding to subsequent detection needs. By using the MSESE module to improve the C3k2 module C3k parameter to the Bottleneck part under the Flase branch, C3k2_MutilScaleEdgeInformationSelect is formed. The original C3k2 structure is compared with the improved C3k2_MutilScaleEdgeInformationSelect structure. Figure 5 shown.

[0050] Step 3.3: Based on the EEM module, the SE attention mechanism is integrated to obtain the EEMSE module, and it is used to improve the detection head structure.

[0051] The SE attention mechanism improves the model's expressive power by performing two operations, compression and excitation, on the input feature image. The Squeeze operation compresses the spatial information of each channel of the input feature map of size H×W×C into a single value through global average pooling, forming a 1×1×C vector, thereby retaining the global information of each channel. Subsequently, the obtained vector is further processed by two fully connected layers in the Excitation operation to obtain the channel weight. The obtained weight coefficient will be used to adjust the feature intensity of each channel in the original feature map to obtain a weighted feature image. The EEMSE module is formed by introducing the SE attention mechanism into the EEM module, and its structure is as follows: Figure 6 As shown. The enhanced feature image will be input into the SE module to dynamically adjust the importance of each channel, so that the model pays more attention to the channels with important information and improves the effect of feature representation. By introducing the EEMSE module into the original toilet cleaning head structure, a new detection head structure Detect_EdgeEnhancerSE is constructed. Its structure is as follows Figure 7 As shown in the figure, it can ultimately improve the edge feature expression capability while reducing the amount of parameter calculation and improving the detection speed.

[0052] Step 4: Set training parameters and use the improved network to train the model on the steel surface defect dataset, using GPU-accelerated training. The experimental environment was set to Python 3.8 and CUDA 11.8, and the RTX 4090 GPU was used for training acceleration. A total of 300 training epochs were performed, with a batch size of 16.

[0053] Step 5: Model verification, set verification parameters, and verify the model trained in step 4.

[0054] Verification results show that applying the improved scheme to the YOLOv11n / s / m networks, respectively, resulted in improvements of 4.1%, 2.2%, and 2.7% in mAP@0.5. At mAP@0.5:0.95, the improved YOLOv11 achieved a 2.2% improvement, while the improved YOLOv11m achieved a 1.9% improvement. Furthermore, the improved Gflops decreased by 0.2%, 0.6%, and 3.5%, respectively.

[0055] Step 6: Use the trained model to detect steel surface defects.

[0056] The comparative experiment of this embodiment is shown in Table 1. The reliability of the improvement is verified by comparing the improved model with some current mainstream models. The evaluation indicators include Precision, Recall, mAP, Parameters and Gflops. Among them, Precision and Recall are calculated as shown in formulas (3) and (4). mAP can measure the average detection performance of the model for all categories under different confidence thresholds. According to different IoU threshold ranges, mAP can be further subdivided into mAP@0.5 and mAP@0.5:0.95. The former only represents the mAP when the IoU threshold is 0.5, while the latter reflects the average mAP calculated between the IoU thresholds of 0.5 and 0.95, which is relatively more comprehensive.

[0057]

[0058] Among them, TP is the number of samples correctly predicted by the model as positive, FP is the number of samples incorrectly predicted by the model as positive, and FN is the number of samples incorrectly predicted by the model as negative.

[0059] Table 1

[0060]

[0061] The model training effects before and after improvement are as follows Figure 8 As shown in the figure, the light-colored lines represent the training trends of the original model, while the dark-colored lines correspond to the training process of the improved model. As can be seen from the figure, the mAP values of each model fluctuated upward with the number of training iterations, while the improved model was more stable than before. After 200 training rounds, the fluctuations in the model's mAP value began to stabilize, indicating that the improved network model was significantly superior to the original model. Under steady-state conditions, the mAP value of the improved model was significantly higher than that of the original model, confirming that the improvements have significantly improved the model's detection performance.

[0062] The above embodiments are intended only to illustrate the technical concepts and features of the present invention. Their purpose is to enable those skilled in the art to understand the contents of the present invention and implement them accordingly. They are not intended to limit the scope of protection of the present invention. Any equivalent changes or modifications made in accordance with the spirit of the present invention are intended to be covered by the scope of protection of the present invention.

Claims

1. A steel defect detection method based on edge enhancement extraction, characterized in that: The steps include: Step 1: Construct and preprocess the steel surface defect dataset and divide it into training set, validation set, and test set; Step 2: Using YOLOv11 as a benchmark, improve the YOLOv11 network structure; Step 2.1: Construct an edge enhancement module (EEM) to suppress low-frequency information and enhance high-frequency information of edge contours during feature extraction. Step 2.2: Design a multi-scale edge enhancement extraction module (MSEEM). By combining feature maps processed with edge enhancement at different scales and then processing the enhanced feature maps using a dual-domain selection mechanism, the feature maps are given clearer edge contour information. The MSEEM module is used to replace the Bottleneck part of the False branch in the C3k2 module with the C3k parameter. Step 2.3: Combine the edge enhancement module EEM and the SE attention mechanism to form the EEMSE module. Use the EEMSE module to improve the detection head part, thereby improving the model's accurate positioning of defect edges while reducing the number of model parameters and improving detection speed. Step 3: Improve the original YOLOv11 network to set training parameters and verify the trained steel defect detection model; Step 4: Compare and improve the obtained model and use it for real-time detection of steel defects.

2. A steel defect detection method based on edge enhancement extraction according to claim 1, characterized in that: The steel surface defect dataset described in step 1 contains 1,800 defect images of six types of defects, including cracks, inclusions, spots, pitting, scale, and scratches. The preprocessing step is to divide the image data into training set, validation set, and test set in a ratio of 8:1:

1. The training set contains 1,440 images, the validation set contains 180 images, and the test set contains 180 images.

3. The steel defect detection method based on edge enhancement extraction according to claim 1 is characterized in that: The edge enhancement module EEM described in step 2.1 operates as follows: For the input steel defect feature image, the high-frequency edge information part of the image is first blurred through the average pooling operation, thereby retaining the low-frequency information of the entire image. Then, the original steel defect feature map data is subtracted from the feature map information after the pooling operation to obtain the high-frequency information part of the image, that is, the edge information part. The features after the edge enhancement operation will be added to the original features to form a residual connection, thereby ensuring that the semantic information of the original image is retained while enhancing the edge features.

4. The steel defect detection method based on edge enhancement extraction according to claim 1 is characterized in that: The multi-scale edge enhancement extraction module MSEEM described in step 2.2 consists of two parts: multi-scale edge enhancement extraction and dual-domain selection mechanism. Its specific structure is as follows: In the multi-scale edge enhancement extraction part, for the input feature image, the task will first be divided into len(bins)+1 branches for processing according to the length of the multi-scale feature extraction list bins. The branches are divided into multiple multi-scale feature extraction branches and one local feature extraction branch. The size of the value stored in the bins list represents the pooling target size applied to the feature map. According to the number of target sizes defined in the list, the corresponding branch is assigned to each scale for processing, so as to pool the feature map to the specified scale for feature extraction; In the multi-scale feature extraction branch, the feature image is adjusted to different target sizes through adaptive average pooling, and information extraction and edge enhancement are achieved through convolution operation and EEM module processing. Then, it is restored to the original size through upsampling to obtain a multi-scale enhanced feature image. In the local feature extraction branch, only simple feature extraction operations are performed. Finally, all branch feature maps are fused in the channel dimension. For the feature image obtained after multi-scale edge enhancement, the dual-domain selection mechanism will further process it by combining the image differences in the spatial domain and the spectral domain. By locating the key areas of the input image and then performing enhancement processing, noise is suppressed, and the feature expression ability of the image is further improved.

5. The steel defect detection method based on edge enhancement extraction according to claim 1, characterized in that: The EEMSE module described in step 2.3 consists of two parts: the edge feature enhancement module EEM and the SE attention mechanism. The specific structure of the module is as follows: In the EEM module processing part, the high-frequency edge information is obtained by subtracting the pooled features from the original features, and then added to the original features to form a residual structure, ultimately obtaining more obvious edge feature information in the feature image; In the SE attention mechanism, the spatial information of each channel of the feature map with an input size of H×W×C will be compressed into a single value through the Squeeze operation to form a 1×1×C vector, thereby retaining the global information of each channel; then in the Excitation operation, the vector is processed through the fully connected layer to generate channel weights, thereby adjusting the feature strength of each channel in the original feature map to obtain a weighted feature image.

6. The steel defect detection method based on edge enhancement extraction according to claim 1, characterized in that: In step 3, the training configuration selected is CUDA 11.8, and the RTX 4090 GPU is used for accelerated computing. The number of training iterations used is 300 rounds, and the batch size is set to 16.

Images