A steel surface defect detection method based on RDD-YOLO algorithm

Abstract

The application discloses a steel surface defect detection method based on an RDD-YOLO algorithm, and comprises the following steps: using a Res2Net module to replace a module CSP-n in an original backbone network of YOLOv5 to enhance the feature extraction capability; designing a new feature fusion module, a double feature pyramid feature fusion module DFPN, in a Neck part of YOLOv5 to fuse feature maps of different scales and further extract information in the feature maps; and using a structure of a decoupled head to replace an original YOLOv5 detection head in a head part. The double feature pyramid feature fusion module DFPN is constructed to fully utilize all information in the network, so that the features are more finely fused, and the problem that the sizes of steel surface defects are different is relieved. The use of the decoupled head can separate the classification task from the regression task of the positioning frame, so that the prediction is more accurate. The application can be used for the steel surface defect detection task in a complex scene, and the detection precision is greatly improved compared with traditional models.

Description

Technical Field

[0001] This invention belongs to the field of image detection and recognition technology, specifically relating to a method for detecting defects on steel surfaces based on the RDD-YOLO algorithm. Background Technology

[0002] Steel is one of the most common raw materials, widely used in industrial production and manufacturing. In actual industrial production processes, steel surfaces are easily damaged by corrosion or deformation. Various defects exist on steel surfaces, such as scratches, cracks, and patches. These defects can cause abnormal vibrations and noise in processing machinery, thereby damaging the machine or leading to fatal accidents. Therefore, the surface quality inspection of steel is of paramount importance.

[0003] Currently, surface defect detection can be broadly categorized into two types: traditional methods and deep learning methods. Traditional defect detection based on machine vision is used in many industrial fields to replace manual visual inspection. The machine vision detection process includes feature extraction and classification. However, feature extraction relies excessively on manually designed algorithms, which can lead to poor robustness and generalization ability of defect detection methods.

[0004] Currently, deep learning-based methods for detecting surface defects are mainly divided into two categories: single-stage detection and two-stage detection. Two-stage detection offers higher accuracy, but its slow detection speed and large model parameter count make it difficult to meet the practical needs of industrial production. Single-stage detection, on the other hand, balances accuracy and speed, making it widely used in industrial scenarios. This invention is based on the single-stage YOLO algorithm, focusing on improving detection accuracy and enhancing the ability to detect small targets. Summary of the Invention

[0005] Purpose of the invention: This invention provides a steel surface defect detection method based on the RDD-YOLO algorithm, which is used for steel surface defect detection in complex scenarios, and the detection accuracy is significantly improved compared with traditional models.

[0006] Technical solution: The present invention provides a method for detecting defects on steel surfaces based on the RDD-YOLO algorithm, comprising the following steps:

[0007] (1) Obtain data samples from the strip surface defect image dataset, divide the data samples into training set and test set according to a specific ratio; and preprocess the images in the training set.

[0008] (2) Construct a steel surface defect detection model based on the RDD-YOLO algorithm. The steel surface defect detection model includes a backbone part, a neck part, and a head part. The backbone part uses a Res2Net network structure module to replace the CSP-n module in the original YOLOv5 backbone network. The neck part adopts the dual feature pyramid feature fusion module DFPN. The head part uses the decoupled head to replace the original detection head in YOLOv5.

[0009] (3) Use the training set to train the steel surface defect detection model and obtain the optimal detection model;

[0010] (4) Use the optimal detection model obtained from training to detect images in the test set;

[0011] (5) Evaluate the detection accuracy and speed of the test set detection results; the evaluation indicators include the average accuracy mAP of all classes and the number of frames per second (FPS).

[0012] Furthermore, the preprocessing of images in the training set described in step (1) includes mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling.

[0013] Furthermore, the specific implementation process of replacing the CSP-n module in the original YOLOv5 backbone network with the Res2Net network structure module in step (2) is as follows:

[0014] In a Res2Net module, the input is first passed through a 1×1 convolutional kernel to adjust the dimension of the feature map. Then, the feature map of input channel C is divided into S equal parts, namely x1, x2, x3, and x4. x1 is directly mapped to y1 without any operation. x2 is passed through a 3×3 convolutional filter and mapped to y2, and then merged with x3. This merged result is then passed through the 3×3 convolutional kernel corresponding to x3, and the output is mapped to y3. This merged result is then passed through the 3×3 convolutional kernel corresponding to x4, and the output is mapped to y4. Finally, y1, y2, y3, and y4 are concatenated, and then passed through a 1×1 convolutional kernel to restore the dimension of the feature map to match the input dimension.

[0015] Further, the Dual Feature Pyramid Feature Fusion Module (DFPN) described in step (2) consists of two parts. With the SPP module as the boundary, the left part is the original neck structure of YOLOv5, and the SPP module and the part to its right are additional parts. First, the feature map passes through the original neck part to obtain three feature maps of different sizes. Then, the three feature maps of these sizes are subjected to max pooling through the SPP module. Then, the feature map with the smallest size is selected and subjected to a series of upsampling and downsampling operations to obtain the three feature maps of the same scale as the left half. At the same time, in the process of sending to the prediction layer, the output of the original SPP module and the feature maps of these three scales are connected by a residual connection.

[0016] Furthermore, the head part described in step (2) uses a decoupled head to replace the original detection head of YOLOv5. The implementation process is as follows:

[0017] In the Decoupled Head, the classification task and the localization task are separated. First, the three feature maps of different scales are passed through a 1×1 convolution kernel to adjust their dimensions. Then, the classification and localization tasks are separated by two sub-branches. After convolution, the localization task's acquisition of the bounding box position and confidence score are separated again. Finally, the classification result, the bounding box position, and the confidence score are output separately.

[0018] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention uses a Res2Net module to replace the original backbone network to enhance the feature extraction capability of the network; it designs a DFPN structure to fully utilize the information in the network and integrate more features; it uses a decoupled head structure to replace the original detection head, making the model classification and detection more accurate; it can be used to detect steel surface defects in complex scenarios, and the detection accuracy is significantly improved compared with traditional models. Attached Figure Description

[0019] Figure 1 A schematic diagram of the existing YOLOv5 network structure;

[0020] Figure 2 This is a schematic diagram of the overall network structure of RDD-YOLO proposed in this invention;

[0021] Figure 3 This is a schematic diagram of the Res2Net network module structure proposed in this invention;

[0022] Figure 4 This is a schematic diagram of the dual-feature pyramid feature fusion module DFPN proposed in this invention;

[0023] Figure 5This is a schematic diagram of the structure of the DecoupledHead proposed in this invention.

[0024] Figure 6 The image shows the detection results obtained using this invention. Detailed Implementation

[0025] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] Since its release in 2020, YOLOv5 has incorporated the latest research from various fields, making it more lightweight while achieving better detection performance. It has become one of the mainstream object detection algorithms. YOLOv5 consists of a backbone network, a neck structure, and a prediction head. The backbone network comprises structures such as CSP-n, CBS, and SPP, as shown in the diagram below. Figure 1 As shown.

[0027] This invention proposes a method for detecting surface defects in steel based on the RDD-YOLO algorithm, such as... Figure 2 As shown, based on YOLOv5, the original backbone network module CSP-n is replaced with a Res2Net network module in the backbone part, increasing the network's feature extraction and learning capabilities. In the neck part, a Dual Feature Pyramid (DFPN) feature fusion module is designed to fuse information at different scales, further extracting information from the feature maps and enabling more refined feature fusion, thus alleviating the problem of varying defect sizes on steel surfaces. In the head part, a decoupled head structure is used to replace the original YOLOv5 detection head, making predictions more accurate; this algorithm is named RDD (Res2Net-DFPN-Decoupled Head)-YOLO. The specific implementation steps are as follows:

[0028] S1: Obtain data samples from the strip surface defect image dataset, divide the data samples into training set and test set according to a specific ratio, and preprocess the images in the training set.

[0029] The dataset images are input to the input layer for preprocessing. The input layer includes mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling. Mosaic data augmentation stitches together four images using random scaling, cropping, and arrangement, essentially feeding four images at once for training, greatly enriching the background of detected objects. Adaptive anchor box calculation uses a genetic algorithm and k-means iteration to calculate the optimal anchor boxes, improving network accuracy. Adaptive image scaling scales the original input image to a fixed size, adding minimal black borders to the scaled image, reducing computational load. In this implementation, the input image size is 416×416.

[0030] S2: Construct a steel surface defect detection model based on the RDD-YOLO algorithm.

[0031] The feature map obtained from the preprocessing in step S1 is input into the backbone layer to extract category features. The backbone layer includes a standard convolutional (CBS) module, a Res2Net network module, and a spatial pyramid pooling (SPP) module.

[0032] The CBS module contains a 1×1 convolutional kernel, a batch normalization (BN) layer, and a Leaky ReLU activation function layer.

[0033] Res2Net network modules, such as Figure 3 As shown, in a Res2Net module, the input is first passed through a 1×1 convolutional kernel to adjust the dimension of the feature map. Then, the feature map of input channel C is divided into S equal parts, x1, x2, x3, and x4. x1 is directly mapped to y1 without any operation. x2 is filtered by a 3×3 convolutional kernel and mapped to y2, then merged with x3. This merged result is then convolved with the corresponding 3×3 kernel, mapping the output to y3. This merged result is then combined with x4, and convolved with the corresponding 3×3 kernel, mapping the output to y4. Finally, y1, y2, y3, and y4 are concatenated, and then passed through a 1×1 convolutional kernel to restore the feature map dimension to match the input dimension. The calculation formula for the Res2Net network module output is as follows:

[0034]

[0035] Where S represents the number of regions (S = 4 in this invention), oi represents the output of this network module, and fi represents the convolution operation.

[0036] Input the defect feature maps of steel plates of different sizes obtained after processing the main layer into the neck layer, such as... Figure 4 As shown, its *CSP-1 module is divided into two branches that perform convolution operations separately and remove the Res residual connections. These feature maps first pass through the original neck layer of YOLOv5 to obtain feature maps of three scales: 13×13, 26×26, and 52×52. Then, these three feature maps of different sizes are max-pooled through an SPP module. After max-pooling, the 13×13 feature map is selected and subjected to a series of upsampling and downsampling operations to obtain feature maps of the same scale as the three in the left half. Simultaneously, during the process of feeding the prediction layer, the output of the original SPP module is residually connected to these three feature maps. This fully utilizes the information in the network, increases the feature map fusion operation, and enables the network to extract more useful information.

[0037] Three sizes of tensor input detection heads were obtained after neck layer processing. The detection heads are decoupled heads, and their specific structures are as follows: Figure 5 As shown, in the Decoupled Head, the prediction layer needs to output the target classification and bounding box position. Since classification and localization focus on different aspects—classification focuses on the target's texture content, while localization focuses on the target's edge information—a decoupled head structure is used to separate the classification and localization tasks to achieve higher prediction accuracy. First, three feature maps of different scales are passed through a 1×1 convolutional kernel to adjust their dimensions. Then, they are separated into classification and localization tasks through two sub-branches. The first branch predicts the result cls (defect category), and the second branch handles the localization task. Next, after convolution, the localization task's bounding box position and confidence score are separated again, outputting reg (bounding box position) and obj (confidence score) respectively. Finally, the classification result, bounding box position, and confidence score are output separately. These sub-branches are then output separately, and Non-Maximum Suppression (NMS) is used for filtering multiple bounding boxes. NMS effectively removes redundant boxes.

[0038] S3: Train the steel surface defect detection model using the training set to obtain the optimal detection model; use the trained optimal detection model to detect images in the test set; evaluate the detection accuracy and speed of the detection results on the test set; the evaluation metrics include the average accuracy (mAP) across all classes and the number of frames per second (FPS).

[0039] This implementation uses a dataset of 1800 defective images of steel strip surfaces, comprising 6 types of defects, with 300 images for each type. The defect categories include cracks (cr), impurities (in), patches (pa), scratches (sc), indented oxidation (rs), and pitting (ps). These images are randomly divided into training and testing sets in an 8:2 ratio for input training. Furthermore, Mosaic data augmentation is used during image input to the model, where four images are randomly cropped and scaled to create a more abundant and larger training sample. The input image size is 416×416.

[0040] To evaluate the performance of the model, this invention uses multiple evaluation metrics for comparison. Among them, mean average precision (mAP) and frames per second (FPS) are the most commonly used evaluation metrics in the field of object detection. mAP is calculated from precision (P) and recall (R):

[0041]

[0042]

[0043]

[0044] Where TP, FP, and FN are the number of actual positive samples predicted as positive samples, the number of actual negative samples predicted as positive samples, and the number of actual positive samples predicted as negative samples, respectively, and c is the number of defect types.

[0045] The algorithm of this invention was run on Ubuntu 20.04, with an AMD 5900X CPU, an Nvidia GeForce RTX 3090 GPU, a batch size of 4, and a training duration of 300 epochs. Finally, the corresponding important evaluation parameters were generated, and their specific parameters are shown in Table 1. Some detection results are shown below. Figure 6 As shown, mAP0.5 represents the mAP value when the threshold is set to 0.5. By comparing with other methods, it can be verified that the method proposed in this invention has better performance in detecting various defects compared to other methods, while also maintaining a relatively fast detection speed.

[0046] Table 1 compares the invention with other methods.

[0047]

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting surface defects in steel based on the RDD-YOLO algorithm, characterized in that, Includes the following steps: (1) Obtain data samples from the strip surface defect image dataset, divide the data samples into training set and test set according to a specific ratio; and preprocess the images in the training set. (2) Construct a steel surface defect detection model based on the RDD-YOLO algorithm. The steel surface defect detection model includes a backbone part, a neck part, and a head part. The backbone part uses a Res2Net network structure module to replace the CSP-n module in the original YOLOv5 backbone network. The neck part adopts the dual feature pyramid feature fusion module DFPN. The head part uses the decoupled head to replace the original detection head in YOLOv5. (3) Use the training set to train the steel surface defect detection model and obtain the optimal detection model; (4) Use the optimal detection model obtained from training to detect images in the test set; (5) Evaluate the detection accuracy and speed of the test set detection results; the evaluation metrics include the average accuracy across all classes mAP and the number of frames per second (FPS); The specific implementation process of replacing the CSP-n module in the original YOLOv5 backbone network with the Res2Net network structure module in step (2) is as follows: In a Res2Net module, the input is first passed through a 1×1 convolutional kernel to adjust the dimension of the feature map. Then, the feature map of input channel C is divided into S equal parts, x1, x2, x3, and x4. x1 is directly mapped to y1 without any operation. x2 is filtered by a 3×3 convolutional kernel and mapped to y2, then merged with x3. This merged result is then convolved with the corresponding 3×3 kernel, mapping the output to y3. This merged result is then merged with x4, and again convolved with the corresponding 3×3 kernel, mapping the output to y4. Finally, y1, y2, y3, and y4 are concatenated, and then passed through a 1×1 convolutional kernel to restore the feature map dimension to match the input dimension. The Dual Feature Pyramid (DFPN) feature fusion module described in step (2) consists of two parts. With the SPP module as the boundary, the left part is the original neck structure of YOLOv5, and the SPP module and the part to its right are additional parts. First, the feature map passes through the original neck part to obtain three feature maps of different sizes. Then, these three feature maps of different sizes are max pooled through the SPP module. Then, the feature map with the smallest size is selected and subjected to a series of upsampling and downsampling operations to obtain feature maps of the same scale as the three in the left half. At the same time, during the process of sending to the prediction layer, the output of the original SPP module and the feature maps of these three scales are joined by a residual connection.

2. The method for detecting steel surface defects based on the RDD-YOLO algorithm according to claim 1, characterized in that, Step (1) involves preprocessing the images in the training set, including mosaic data augmentation, adaptive anchor box calculation, and adaptive image scaling.

3. The method for detecting steel surface defects based on the RDD-YOLO algorithm according to claim 1, characterized in that, The process of replacing the original detection head of YOLOv5 with a decoupled head in step (2) is as follows: In the DecoupledHead, the classification task and the localization task are separated. First, the three feature maps of different scales are adjusted in dimension by passing them through a 1×1 convolution kernel. Then, the classification and localization tasks are separated by two sub-branches. After convolution, the localization task's acquisition of the bounding box position and confidence score are separated again. Finally, the classification result, bounding box position, and confidence score are output separately.

Images