Steel surface defect detection method

Abstract

The invention discloses a steel surface defect detection method which comprises the following steps: S1, acquiring a steel surface defect data set, and preprocessing image data in the data set; s2, constructing a steel surface defect detection model based on YOLOv12, and improving the performance of the model in a steel surface defect detection task; s3, inputting the data set obtained in the step 1 into the YOLOv12 steel surface defect detection model in the step 2 for training, dividing the data set into a training set and a test set, training the model and storing model parameters; step S4, a detection stage: loading the trained model, and inputting a steel surface defect picture to carry out steel surface defect detection; and S5, outputting a final result of the to-be-detected steel surface defect image. According to the system and the method, the LDConv module, the SCSA module and the MPDIOU module are effectively combined with the YOLOv12, the advantages of the modules are fully utilized, and the steel surface defect detection effect is well improved.

Description

Technical Field

[0001] The present invention relates to a detection method, in particular to a steel surface defect detection method. Background Art

[0002] In the steel manufacturing industry, surface defect detection plays a vital role in ensuring product quality and production efficiency. Surface defects such as cracks, scratches, dents, and rust not only affect the aesthetics of steel but also its performance and safety. If these defects go undetected and untreated, they can lead to premature product failure, increased safety risks, and significant economic losses. Traditional inspection methods, which rely heavily on human experience and subjective judgment, are not only inefficient but also lack the accuracy and repeatability required for consistent quality control.

[0003] With the advancement of automation technology, many steel producers have adopted automated surface defect detection systems. These systems are typically based on traditional image processing techniques such as edge detection and texture analysis, which have improved inspection speed and accuracy to a certain extent. However, the rapid development of computer vision and machine learning has made automated defect detection an effective solution for improving inspection accuracy and efficiency. Deep learning techniques, particularly convolutional neural networks (CNNs) and generative adversarial networks (GANs), are increasingly being used to identify and classify surface defects such as cracks, inclusions, scratches, and rust. These systems can quickly and automatically identify and classify a variety of defects, significantly improving inspection accuracy and consistency while reducing labor requirements and operating costs. In recent years, the application of deep learning in image processing has made significant progress, particularly in object detection tasks. The YOLO (You Only Look Once) family of models, renowned for their excellent real-time performance and efficiency, has attracted widespread attention.

[0004] An innovative approach for steel surface defect recognition using an improved YOLOv8 algorithm has been introduced. Their approach combines the SimAM attention mechanism with an enhanced SPFF module to expand the receptive field. A YOLOv9-based CK-Net steel surface defect detection network has also been proposed, which includes the CK-GELAN feature extraction module and the CK-FFM feature fusion module. By integrating the self-attention mechanism and an improved CBAM, CK-Net provides enhanced multi-scale feature fusion, effectively improving the model's mAP on NEU-DET. An enhanced DCN_C2f module and SC_SA attention mechanism have also been introduced to optimize the YOLOv8 model for better performance in real-time object detection in complex backgrounds. These improvements have been found to effectively enhance feature representation and detection accuracy, enabling the model to meet the requirements of real-time inference.

[0005] Despite these advances, data annotation in defect detection remains an expensive challenge due to the need to provide accurate bounding boxes and class labels for each defect. The progress in this field is mainly attributed to two main factors: (1) pre-trained ImageNet models, and (2) large-scale base convolutional neural networks, which have driven significant progress in deep learning-based defect classification. However, limitations in data availability and high annotation costs still restrict further development of defect detection. The YOLO family, with its solid foundation of large-scale pre-trained datasets, provides an ideal framework for such tasks. The model is fine-tuned using the NEU-DET defect detection dataset, leveraging the advantages of the YOLOv12 framework to enhance surface defect detection. Current steel surface defect detection technology still faces many challenges: at the data level, the existing data sets are small in scale and the annotations are highly subjective, resulting in insufficient model generalization ability and difficulty in adapting to the diverse defect types in complex industrial scenarios; in terms of detection capabilities, traditional methods have limited recognition accuracy for micron-level defects, composite defects (such as superposition of cracks and scratches) or special material defects, and are easily affected by surface finish, lighting and noise; in practical applications, high-speed online detection needs to balance high precision and real-time performance, but existing algorithms are prone to losing detailed features at processing speeds of thousands of frames per second. Summary of the Invention

[0006] In order to solve the shortcomings of the above technologies, the present invention provides a method for detecting steel surface defects.

[0007] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for detecting steel surface defects, comprising the following steps:

[0008] Step S1: obtaining a steel surface defect dataset and performing preprocessing operations on the image data in the dataset;

[0009] Step S2: Build a steel surface defect detection model based on YOLOv12 to improve the performance of the model in the steel surface defect detection task;

[0010] Step S3: input the data set obtained in step 1 into the YOLOv12 steel surface defect detection model in step 2 for training, divide the data set into a training set and a test set, train the model and save the model parameters;

[0011] Step S4: Detection phase: Load the trained model and input the steel surface defect image to perform steel surface defect detection;

[0012] Step S5: output the final result of the steel surface defect image to be tested.

[0013] Preferably, in step S1, sample augmentation is performed on the image data in the dataset using illumination enhancement and occlusion enhancement operations, specifically:

[0014] Lighting enhancement simulates different lighting environments through HSV space adjustment (hue / saturation / lightness perturbation) and color dithering (random changes in brightness / contrast), improving the model's adaptability to lighting changes;

[0015] Occlusion enhancement uses random erasing (local noise filling), CutMix / Mosaic (multi-image stitching) and affine transformation to enhance the model's feature extraction capabilities in scenes where objects are occluded.

[0016] Preferably, step S2 includes the following process:

[0017] S21. Replace the C3k2 layer in the Backbone part of the original YOLOv12 with the LDConv layer to achieve adaptive adjustment of the receptive field;

[0018] S22: The Neck network in the original YOLOv12 is added with an SCSA layer after A2C2f and C3k2 and before the Detect layer. This improves the performance of various downstream visual tasks by combining spatial attention and channel attention.

[0019] S23. Replace the loss function of the last Detect layer in the original YOLOv12 with MPDIoU to more accurately capture the metric of the difference between the predicted box and the true box.

[0020] Preferably, the collaborative attention module SCSA consists of two parts: shared multi-semantic spatial attention SMSA and progressive channel self-attention PCSA, which integrates multi-semantic information and effectively guides channel recalibration;

[0021] Among them, the SMSA module: first uses multi-scale, depth-shared 1D convolution to extract spatial information of different semantic levels from four independent sub-features, and accelerates model convergence through group normalization, and then inputs the feature map modulated by SMSA into PCSA;

[0022] PCSA module: Combining progressive compression and channel-specific single-head self-attention mechanism, it uses input-aware single-head self-attention mechanism to effectively explore channel similarity, thereby alleviating the semantic differences between different sub-features in SMSA and promoting information fusion.

[0023] Preferably, MPDIoU calculates the minimum point distance based on the overlap area, center point distance, and width and height changes. The MPDIoU bounding box loss function is defined as:

[0024]

[0025] Among them, Bgt is the true bounding box, Bprd is the predicted bounding box, and Θ is the parameter of the deep regression model.

[0026] Preferably, in step S3, the steel surface defect dataset obtained in step 1 is used to train the YOLOv12 steel surface defect detection model, and the training parameters are set: batch-size, initial learning rate, number of training rounds and optimizer SGD (stochastic gradient descent), and the best performing model weights are saved as a file best.pt.

[0027] Preferably, step S4 includes the following sub-steps:

[0028] S41. Input the image to be detected into the YOLOv12 steel surface defect detection model, load the pre-trained optimal weight file for detection, and mark the detected object categories (loose, included, spots, granular surface, strip cracks, scratches) and their corresponding confidence levels on the output image;

[0029] S42. Verify the accuracy of the YOLOv12 steel surface defect detection model, using mAP50 and mAP50-75 accuracy as key indicators to evaluate model performance.

[0030] The present invention develops an enhanced YOLOv12 algorithm to meet the challenges in steel surface defect detection tasks. The linear deformable convolution LDConv module is integrated into the backbone network. In addition, it integrates a new collaborative attention module (SCSA) in the spatial and channel attention mechanism. MPDIoU improves the detection accuracy and the ability to process details. This model helps to address the limitations of existing networks, and the generalization and detection capabilities of the model are improved. By effectively combining the three modules LDConv, SCSA and MPDIoU with YOLOv12, the present invention fully utilizes the advantages of each module and greatly improves the effect of steel surface defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is the overall flow chart of the present invention.

[0032] Figure 2 This is the overall network structure diagram of the present invention.

[0033] Figure 3 This is an example diagram of the test comparison of the present invention.

[0034] Figure 4 Schematic diagram of the minimum point distance calculated by MPDIoU in the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] like Figure 1 A method for detecting steel surface defects is shown, comprising the following steps:

[0037] S1. Obtain steel surface defect dataset;

[0038] S2, use illumination enhancement and occlusion enhancement operations to perform sample augmentation on the image data in the dataset;

[0039] S3. Build a steel surface defect detection method model based on YOLOv12 to improve the model's target detection challenges in steel surface defect detection tasks;

[0040] Use the steel surface defect dataset obtained in step 1 to train the YOLOv12 steel surface defect detection model. Set the training parameters: batch size 16, initial learning rate 0.01, number of training rounds 300, and optimizer SGD (stochastic gradient descent). Save the best-performing model weights as the file best.pt.

[0041] S4. Input the data set obtained in step 2 into the YOLOv12 steel surface defect detection model in step 3 for training, divide the data set into a training set and a test set, train the model and save the model parameters; including the following sub-steps:

[0042] S41. Input the image to be detected into the YOLOv12 steel surface defect detection model, load the pre-trained optimal weight file for detection, and mark the detected object categories (loose, included, spots, granular surface, strip cracks, scratches) and their corresponding confidence levels on the output image;

[0043] S42. Verify the accuracy of the YOLOv12 steel surface defect detection model, using mAP50 and mAP50-75 accuracy as key indicators to evaluate model performance.

[0044] S5, detection stage, load the trained model, input the steel surface defect picture to perform steel surface defect detection;

[0045] S6. Output the final result of the steel surface defect image to be tested.

[0046] The application of deep learning-based defect detection models has significantly reduced the workload of workers and improved inspection efficiency. This paper proposes an enhanced YOLOv12 model (LDConv+SCSA+MPDIoU+YOLOv12) to address the target detection challenges in steel surface defect detection tasks. This enhanced model integrates the linear deformable convolution (LDConv) module into the backbone network to achieve adaptive adjustment of the receptive field. In addition, it integrates the collaborative attention module (SCSA) in the spatial and channel attention mechanism. MPDIoU improves detection accuracy and the ability to handle details. This model helps to address the limitations of existing networks. Experimental results show that compared with existing models, the proposed model has improved in terms of precision (P), recall (R) and average precision (mAP).

[0047] This invention is an enhanced model based on YOLOv12, providing multiple scales (N / S / M / L / X) to meet the needs of different scenarios. The network architecture of this invention is as follows Figure 2 As shown in the figure, in the Backbone part, the C3k2 layer is replaced by the LDConv layer, which is an efficient and universal linear deformable convolution layer suitable for a wide range of visual applications. LDConv provides greater flexibility than deformable convolution, allowing the number of convolution kernel parameters to grow linearly, thus overcoming the problem of the quadratic growth of the number of deformable convolution parameters. This method can play a lightweight role.

[0048] LDConv aims to address the limitations of standard convolution operations, which sample local windows of fixed shape and size and struggle to dynamically adapt to varying object shapes. LDConv offers greater flexibility than deformable convolution, allowing the number of convolution kernel parameters to grow linearly, thus overcoming the quadratic growth problem of deformable convolution. This approach can also be lightweight. The Spatial and Channel Co-Attention (SCSA) module aims to improve performance on various downstream vision tasks by combining spatial and channel attention. SCSA, consisting of Shared Multi-Semantic Spatial Attention (SMSA) and Progressive Channel Self-Attention (PCSA), achieves improved performance by integrating multi-semantic information and effectively guiding channel recalibration. In object detection and instance segmentation, the traditional bounding box regression (BBR) loss function often fails to optimize when the predicted and ground-truth boxes have the same aspect ratio but different sizes. To address this issue, the original loss function is replaced with MPDIoU, a metric designed to more accurately capture the difference between the predicted and ground-truth boxes. MPDIoU calculates the minimum point distance based on the overlap area, center point distance, and changes in width and height.

[0049] B gt is the ground-truth bounding box, B prdis the predicted bounding box, C is the box containing B gt and B prd The minimum bounding box of |C| is the area of the bounding box C, ρ 2 (B gt ,B prd ) is the Euclidean distance between the center point of the predicted bounding box and the center point of the true bounding box, C 2 Represents the diagonal length of the minimum containing rectangle, such as Figure 4 shown.

[0050] The MPDIoU bounding box loss function is defined as:

[0051]

[0052] Among them, B gt is the set of ground-truth bounding boxes, Θ is the parameter of the deep regression model. The typical form of L is the n-norm.

[0053] The Spatial and Channel Co-Attention module (SCSA) aims to improve the performance of various downstream vision tasks by combining spatial and channel attention. Composed of two components, Shared Multi-Semantic Spatial Attention (SMSA) and Progressive Channel Self-Attention (PCSA), SCSA achieves this performance improvement by integrating multi-semantic information and effectively guiding channel recalibration.

[0054] The SMSA module first uses multi-scale, depth-shared 1D convolution to extract spatial information at different semantic levels from four independent sub-features, and accelerates model convergence through group normalization. The feature map modulated by SMSA is then input into PCSA.

[0055] PCSA module: Combining progressive compression and channel-specific single-head self-attention mechanism, it uses input-aware single-head self-attention mechanism to effectively explore channel similarity, thereby alleviating the semantic differences between different sub-features in SMSA and promoting information fusion.

[0056] The SCSA module improves the model's feature extraction capabilities by integrating multi-semantic spatial information and channel-wise attention, enabling the model to learn higher-quality representations from multiple perspectives. Experimental results on multiple benchmark datasets demonstrate that the SCSA module not only outperforms current state-of-the-art plug-and-play attention methods, but also demonstrates enhanced generalization across a variety of task scenarios. The SCSA module, incorporated into the Neck of YOLOv12, improves feature extraction capabilities by integrating multi-semantic information and coordinating spatial and channel-wise attention. In object detection tasks, the improved YOLOv12 model using the SCSA module achieves significant improvements in key metrics such as precision and recall. Furthermore, due to the lightweight and efficient design of the SCSA module, the improved model also achieves improvements in inference speed and memory usage. The improved YOLOv12 model performs exceptionally well in object detection tasks in complex scenarios, demonstrating enhanced robustness and accuracy, providing strong support for practical applications.

[0057] MPDIoU aims to provide an easy-to-implement solution for computing the MPDIoU between two axis-aligned rectangles by directly minimizing the keypoint distance between the predicted and ground-truth boxes. MPDIoU is calculated based on the minimum point distance of the horizontal rectangles and comprehensively accounts for overlap, center point distance, and width and height deviations. By directly computing the keypoint distance between the predicted and ground-truth boxes, MPDIoU provides a more accurate loss metric when distinguishing bounding boxes with the same aspect ratio but different sizes or positions.

[0058] Based on the YOLO framework, the introduction of three modules—LDConv (Linear Deformable Convolution), SCSA (Spatial and Channel-wise Co-Attention), and MPDIoU (Minimum Point Distance over Union (IoU) loss—significantly improves the accuracy and robustness of steel surface defect detection. LDConv dynamically adjusts the shape and number of convolution kernel parameters to enhance the geometric feature capture of irregular defects (such as cracks and scratches). Its linear parameter growth balances computational efficiency and feature diversity. SCSA integrates spatial and channel-wise attention mechanisms. Through multi-scale feature fusion and adaptive weight allocation, it strengthens the model's focus on small defect areas, suppresses complex background interference, and improves localization accuracy. MPDIoU optimizes bounding box regression by minimizing the point distance between the predicted and ground-truth bounds, addressing the blind spots of traditional IoU optimization under aspect ratio variations and enabling more accurate defect localization. The three modules work together: LDConv enhances feature extraction, SCSA optimizes feature selection, and MPDIoU improves regression accuracy, ultimately significantly improving the performance of steel surface defect detection.

[0059] like Figure 3As shown, the first row of labels represents the original dataset annotations, and the boxed areas indicate the locations of manually annotated defects. The second row demonstrates the defect detection capabilities of the original YOLOv12 model. The third row demonstrates the defect detection capabilities of the YOLOv12 enhanced model proposed in this paper. Compared to YOLOv12, this paper reduces the probability of missed detections and duplicate detections, thereby improving detection performance.

[0060] This paper integrates the linear deformable convolution (LDConv) module into the backbone network to achieve adaptive adjustment of the receptive field. A new collaborative attention module (SCSA) is integrated into the spatial and channel attention mechanisms, effectively combining the advantages of channel and spatial attention, fully utilizing multi-semantic information to improve the performance of visual tasks. MPDIoU improves detection accuracy and the ability to handle details.

[0061] The above embodiments are not limitations of the present invention, and the present invention is not limited to the above examples. Any changes, modifications, additions or substitutions made by technicians in this technical field within the scope of the technical solution of the present invention also fall within the scope of protection of the present invention.

Claims

1. A method for detecting steel surface defects, characterized by: The following steps are involved: Step S1: obtaining a steel surface defect dataset and performing preprocessing operations on the image data in the dataset; Step S2: Build a steel surface defect detection model based on YOLOv12 to improve the performance of the model in the steel surface defect detection task; Step S3: input the data set obtained in step 1 into the YOLOv12 steel surface defect detection model in step 2 for training, divide the data set into a training set and a test set, train the model and save the model parameters; Step S4: Detection phase: Load the trained model and input the steel surface defect image to perform steel surface defect detection; Step S5: output the final result of the steel surface defect image to be tested.

2. The method for detecting steel surface defects according to claim 1, wherein: In step S1, sample augmentation is performed on the image data in the dataset using illumination enhancement and occlusion enhancement operations. Specifically: Lighting enhancement simulates different lighting environments through HSV space adjustment and color dithering to improve the model's adaptability to lighting changes; Occlusion enhancement uses random erasing, CutMix / Mosaic, and affine transformation to enhance the model's feature extraction capabilities in scenes where objects are occluded.

3. The method for detecting steel surface defects according to claim 1, wherein: The step S2 includes the following process: S21. Replace the C3k2 layer in the Backbone part of the original YOLOv12 with the LDConv layer to achieve adaptive adjustment of the receptive field; S22: The Neck part of the original YOLOv12 is added with the SCSA layer after the A2C2f and C3k2 layers and before the Detect layer. By combining spatial attention and channel attention, the performance of various downstream visual tasks is improved. S23. Replace the loss function of the last Detect layer in the original YOLOv12 with MPDIoU to more accurately capture the metric of the difference between the predicted box and the true box.

4. The method for detecting steel surface defects according to claim 3, wherein: The collaborative attention module (SCSA) consists of two parts: shared multi-semantic spatial attention (SMSA) and progressive channel self-attention (PCSA). It integrates multi-semantic information and effectively guides channel recalibration. Among them, the SMSA module: first uses multi-scale, depth-shared 1D convolution to extract spatial information of different semantic levels from four independent sub-features, and accelerates model convergence through group normalization, and then inputs the feature map modulated by SMSA into PCSA; PCSA module: Combining progressive compression and channel-specific single-head self-attention mechanism, it uses input-aware single-head self-attention mechanism to effectively explore channel similarity, thereby alleviating the semantic differences between different sub-features in SMSA and promoting information fusion.

5. The method for detecting steel surface defects according to claim 3, wherein: The MPDIoU calculates the minimum point distance based on the overlap area, center point distance, and changes in width and height. The MPDIoU bounding box loss function is defined as: Among them, B gt is the ground-truth bounding box, B prd is the predicted bounding box, and Θ is the parameter of the deep regression model.

6. The method for detecting steel surface defects according to claim 1, wherein: In step S3, the steel surface defect dataset obtained in step 1 is used to train the YOLOv12 steel surface defect detection model, and the training parameters are set: batch size, initial learning rate, number of training rounds, and optimizer SGD. The weight of the best performing model is saved as a file best.pt.

7. The method for detecting steel surface defects according to claim 1, wherein: The step S4 includes the following sub-steps: S41. Input the image to be detected into the YOLOv12 steel surface defect detection model, load the pre-trained optimal weight file for detection, and mark the detected object categories (loose, included, spots, granular surface, strip cracks, scratches) and their corresponding confidence levels on the output image; S42. Verify the accuracy of the YOLOv12 steel surface defect detection model, using mAP50 and mAP50-75 accuracy as key indicators to evaluate model performance.

Images