A method for detecting surface defects in strip steel based on an improved YOLOv7 algorithm

Abstract

This invention discloses a strip steel surface defect detection method based on an improved YOLOv7 algorithm. The network improvements include: calculating target box parameters more suitable for the model using k-means++ clustering, replacing some convolutions in the backbone network with PConv, changing the original model's loss function CIOU to SIOU, and adding an SE attention mechanism at the output of each feature layer. Through these improvements, the model's detection accuracy increased from 0.626 to 0.792, recall increased from 0.611 to 0.653, and mAP increased from 0.65 to 0.709. This improved YOLOv7 strip steel surface defect detection method, compared to the original network, achieves higher detection accuracy while maintaining detection speed. The weight file size is reduced from 74.9Mb to 66.3Mb. It can play a significant role in strip steel production, enabling automated detection, rapid identification of defect areas, and improved production efficiency and product quality. In the later stages of strip steel production, using the improved YOLOv7 algorithm to detect surface defects in finished strip steel can ensure product quality, prevent defective products from entering the market, and avoid affecting the company's reputation.

Description

Technical Field

[0001] This invention belongs to the field of target detection and relates to a method for detecting surface defects in strip steel based on an improved YOLOv7 algorithm. Background Technology

[0002] Early methods for detecting surface defects in strip steel mainly included manual inspection, eddy current testing, infrared detection, magnetic flux leakage detection, and laser scanning. In actual industrial production, the significant differences in shape, size, texture, color, background, layout, and imaging illumination of the objects being inspected make defect detection and classification in complex environments a challenging task. In the context of deep learning, convolutional neural networks, with their powerful end-to-end automatic feature extraction capabilities, have been widely used for surface defect detection. Depending on the detection task, they can be categorized into classification networks, object detection networks, and instance segmentation networks. With the development of object detection technology, deep learning-based detection methods can be used to detect surface defects in strip steel, offering far superior reliability, practicality, and efficiency compared to earlier methods for identifying surface defects.

[0003] Currently, deep learning-based object detection algorithms mainly fall into two categories: The first category is two-stage detection algorithms, also known as region-based object detection algorithms. These algorithms use methods such as sliding windows to extract multiple candidate regions from an image, then classify and regress bounding boxes for each candidate region to obtain the final object detection result. Examples of this category include R-CNN, FastR-CNN, FasterR-CNN, and MaskR-CNN. The second category is single-stage object detection algorithms. These algorithms do not require explicit candidate region extraction but directly perform classification and bounding box regression on the image to achieve object detection. Examples of this category include YOLO, SSD, and RetinaNet. Each category has its advantages and disadvantages. Region-based algorithms have higher accuracy but are slower, while single-stage object detection algorithms are faster but have relatively lower accuracy. Summary of the Invention

[0004] To overcome the shortcomings of the existing technology, this invention proposes a strip steel surface defect detection method based on the improved YOLOv7 algorithm, which can improve the detection accuracy while ensuring a high detection speed.

[0005] A method for detecting surface defects in strip steel based on an improved YOLOv7 algorithm includes the following steps:

[0006] (1) Prepare the dataset;

[0007] (2) Process the dataset;

[0008] (3) Improve the network;

[0009] (4) Train the network;

[0010] (5) Test the network.

[0011] The further step (1) includes the following steps:

[0012] Download the dataset of surface defects in hot-rolled strip steel from Northeastern University. Select 1800 images from the downloaded dataset, categorizing them into six types of surface defects: inclusions, scratches, rolled-in scale, cracks, pitted surfaces, and patches. Preprocess these images.

[0013] The further step (2) includes the following steps:

[0014] The dataset is processed by dividing it into training, testing, and validation sets in a ratio of 8:1:1. The XML files of the labeled dataset are stored in the Annotations folder. The XML files of the labeled results are converted into TXT files using a script and stored in the Labels folder. The names of the images contained in the training, testing, and validation sets of the dataset are stored in the ImageSets folder using a script, and the image paths are placed in the data directory.

[0015] The further step (3) includes the following steps:

[0016] (3-1) Calculate the size of the anchorbox suitable for the dataset using the k-means++ clustering method, and replace the parameters of the original anchorbox with the parameters of the newly generated anchorbox in the yaml file.

[0017] (3-2) The partial convolution is replaced with the lighter and faster PConv convolution module. The principle is that only pixels outside the occluded region are convolved during the convolution operation, while pixels within the occluded region are treated as missing values. The core of PConv is the partial convolution operation, which only performs convolution operations on pixels in the non-occluded region of the input feature map, while pixels in the occluded region are ignored or treated as 0. Then, the partial convolution operation combines the pixels in the non-occluded region with the mask of the occluded region to calculate the corresponding pixel value in the output feature map. This method can effectively utilize the contextual information of surrounding pixels, thus better recovering the missing parts of the image.

[0018] Replace some convolutions in the ELAN module of the network with PConv, and rename the CBS module after the convolutions are replaced to PBS module. The new ELAN module is named ELANP.

[0019] (3-3) Add an SE attention mechanism to the feature layer output in the network. To facilitate this addition, the attention mechanism is added to the last convolutional layer of the ELAN module. The last CBS module is named CBS-Att, and the new ELAN module is named ELANP-Att. To better extract feature map information, all ELAN modules before concatenation are replaced with the ELANP-Att module with the added attention mechanism. Add the attention mechanism to the SPCSPC module, naming it SPCSPC-Att, and replace the only SPCSPC module in the network with the SPCSPC-Att module.

[0020] (3-4) Replace the original CIOU loss function in the network with the SIOU loss function. SIOU introduces the vector angle between the ground truth box and the predicted box, which speeds up the convergence.

[0021] (1) Angle loss:

[0022] Where c h σ is the height difference between the center points of the ground truth bounding box and the predicted bounding box, and σ is the distance between the center points of the ground truth bounding box and the predicted bounding box.

[0023] (2) Distance loss:

[0024]

[0025] Where c w and c h Let the width and height be the minimum bounding rectangle of the ground truth bounding box and the predicted bounding box. The coordinates of the center of the true bounding box. These are the coordinates of the center of the prediction box.

[0026] (3) Shape loss: Ω=(1-e -Ww ) θ +(1-e -Wh ) θ

[0027]

[0028] Where (w,h) and (w) gt ,h gt) represent the width and height of the predicted bounding box and the ground truth bounding box, respectively. θ controls the degree of attention paid to the shape loss, with a value range of [2,6].

[0029]

[0030] The further step (4) includes the following steps:

[0031] Create a new YAML file named YOLOv7-improve.yaml. Replace the original ELAN and SPPCSPC modules with the improved modules. Change the path to the cfg file in the train.py file to the path to the YOLOv7-improve.yaml file. Run the train.py file to train the model. After training, record the results, including accuracy, recall, and mAP values, and save the generated weight file.

[0032] The further step (5) includes the following steps:

[0033] Change the weight file path in the `detect.py` file to the path of the weight file generated after training the improved model. Import the dataset of images of steel strips to be detected into the model, run the `detect.py` file, and save the detection results. To compare the object detection performance before and after the model improvement, change the weight file path in the `detect.py` file to the weight file generated after training the original network, and then perform recognition on the same batch of image datasets, saving the detection results. Then compare the results of these two detections. The comparison image is shown below. Figure 4-6 As shown. Attached Figure Description

[0034] Figure 1 This is a flowchart of a method for detecting surface defects in strip steel based on an improved YOLOv7 algorithm.

[0035] Figure 2 A simplified network framework diagram for a strip surface defect detection method based on the improved YOLOv7 algorithm.

[0036] Figure 3 The effect of strip defect identification by the original network Figure 1

[0037] Figure 4 To improve the effectiveness of the network in identifying strip defects Figure 1

[0038] Figure 5 The effect of strip defect identification by the original network Figure 2

[0039] Figure 6To improve the effectiveness of the network in identifying strip defects Figure 2 Detailed Implementation

[0040] To make the technical solution of the present invention clearer, the present invention will be further described below with reference to the accompanying drawings.

[0041] refer to Figure 1-6 The present invention is implemented according to the following specific steps:

[0042] The first step is to download the dataset of hot-rolled strip steel from Northeastern University.

[0043] Download the strip steel defect dataset from the website, and select 1800 images from the downloaded dataset. These images are divided into six categories of strip steel surface defects, including inclusions, scratches, rolled-in scale, cracks, pitted surfaces, and patches. These images are then preprocessed.

[0044] The second step is to process the dataset prepared in the first step.

[0045] The dataset is processed by dividing it into training, testing, and validation sets in a ratio of 8:1:1. The XML files of the labeled dataset are stored in the Annotations folder. The XML files of the labeled results are converted into TXT files using a script and stored in the Labels folder. The names of the images contained in the training, testing, and validation sets of the dataset are stored in the ImageSets folder using a script, and the image paths are placed in the data directory.

[0046] The third step is to improve the original network model.

[0047] (3-1) The size of the anchor box suitable for the dataset is calculated by using the k-means++ clustering method, and the parameters of the original anchor box are replaced with the parameters of the newly generated anchor box in the yaml file.

[0048] The k-means++ algorithm is an improved k-means clustering algorithm. This algorithm is simple, fast, and particularly suitable for use on regular datasets. It avoids the weak clustering that sometimes occurs when using the k-means algorithm. It improves the accuracy of clustering results through an intelligent initialization method. The steps of the k-means++ algorithm are as follows:

[0049] (1) Randomly select a sample point from the input dataset as the first cluster center.

[0050] (2) For each remaining sample point, calculate its shortest distance to each of the selected cluster centers.

[0051] (3) Select a new cluster center such that the probability of its selection is proportional to the square of the shortest distance to the already selected cluster center.

[0052] (4) Repeat steps 2 and 3 until k cluster centers are selected.

[0053] (5) Run the standard k-means clustering algorithm with the selected k cluster centers.

[0054] This method makes it easier for the algorithm to avoid local optima, thereby improving the accuracy of clustering results.

[0055] (3-2) The partial convolution is replaced with the lighter and faster PConv convolution module. The principle is that only pixels outside the occluded region are convolved during the convolution operation, while pixels within the occluded region are treated as missing values. The core of PConv is the partial convolution operation, which only performs convolution operations on pixels in the non-occluded region of the input feature map, while pixels in the occluded region are ignored or treated as 0. Then, the partial convolution operation combines the pixels in the non-occluded region with the mask of the occluded region to calculate the corresponding pixel value in the output feature map. This method can effectively utilize the contextual information of surrounding pixels, thus better recovering the missing parts of the image.

[0056] Replace some convolutions in the ELAN module of the network with PConv, and rename the CBS module after the convolutions are replaced to PBS module. The new ELAN module is named ELANP.

[0057] (3-3) Add an SE attention mechanism to the output of the feature layer in the network. To facilitate adding the attention mechanism, it is added to the last convolutional layer of the ELAN module. The last CBS module is named CBS-Att, and the new ELAN module is named ELANP-Att. To better extract information from the feature maps, all ELAN modules before concatenation are replaced with ELANP-Att modules with the added attention mechanism. Add the attention mechanism to the SPCSPC module, name it SPCSPC-Att, and replace the only SPCSPC module in the network with the SPCSPC-Att module. The SE attention mechanism mainly includes three steps:

[0058] (1) Squeeze: Performs global average pooling on the feature map, transforming the feature map size from h*w*c to 1*1*c.

[0059]

[0060] (2) Excitation: Generate a weight value for each feature channel and construct the correlation between channels through two fully connected layers.

[0061] s=σ(W2δ(W1z))

[0062] W1 and W2 are two fully connected layers that process the vector z obtained in the previous step to obtain the channel weights s. The activation function of the first fully connected layer is ReLU, and the activation function of the second fully connected layer is Sigmoid.

[0063] Scale: Assign weights to the feature map U using the weight file s generated in (2) to obtain the required feature map.

[0064]

[0065] (3-4) Replace the original CIOU loss function in the network with the SIOU loss function. SIOU introduces the vector angle between the ground truth box and the predicted box, which speeds up the convergence.

[0066] (1) Angle loss:

[0067] Where c h σ is the height difference between the center points of the ground truth bounding box and the predicted bounding box, and σ is the distance between the center points of the ground truth bounding box and the predicted bounding box.

[0068] (2) Distance loss:

[0069]

[0070] Where c w and c h Let the width and height be the minimum bounding rectangle of the ground truth bounding box and the predicted bounding box. The coordinates of the center of the true bounding box. These are the coordinates of the center of the prediction box.

[0071] (3) Shape loss: Ω=(1-e -Ww ) θ +(1-e -Wh ) θ

[0072]

[0073] Where (w,h) and (w) gt ,h gt) represent the width and height of the predicted bounding box and the ground truth bounding box, respectively. θ controls the degree of attention paid to the shape loss, with a value range of [2,6].

[0074]

[0075] The fourth step is to train the improved network model.

[0076] Create a new YAML file named YOLOv7-improve.yaml. Replace the original ELAN and SPPCSPC modules with the improved modules. Change the path to the cfg file in the train.py file to the path to the YOLOv7-improve.yaml file. Run the train.py file to train the model. After training, record the results, including precision, recall, and mAP values, and save the generated weight file. Compare the training results with the original model. The model's detection precision improved from 0.626 to 0.792, recall improved from 0.611 to 0.653, and mAP improved from 0.65 to 0.709.

[0077] The fifth step is to test the improved model.

[0078] Change the weight file path in the detect.py file to the path of the weight file generated after the improved model is trained. Import the image dataset of the steel strip to be detected into the model, run the detect.py file, and save the detection results of the images. In order to compare the difference in object detection performance between the model before and after the improvement, change the weight file path in the detect.py file to the weight file generated after the original network is trained, and then perform recognition on the same batch of image datasets, save the detection results of the images, and then compare the performance of these two detections.

[0079] Step 6, Application in strip steel production:

[0080] In the strip steel production process, detecting defects in the strip steel is crucial. Defect detection ensures that the quality of the strip steel products meets standards and customer requirements, and it also allows for the identification and elimination of any defects in the product, thereby improving product quality and reliability. Defective products can malfunction and cause accidents during use, posing dangers to users and the surrounding environment. Therefore, detecting defects in strip steel can also eliminate many safety hazards.

[0081] When inspecting the quality of strip steel products, a fixed camera is used to film the strip steel that needs to be inspected on the conveyor belt. The filmed video is then imported into a computer that has been configured to run the improved YOLOv7 model, and the improved model is used to inspect the filmed video.

[0082] After using the original and improved models to identify strip steel in the video footage of production, it was found that the improved YOLOv7 model has higher accuracy and better anti-interference capabilities in strip steel identification. A comparison of the original and improved networks for strip steel identification is shown in the following figure. Figure 3-6 As shown. When using it, the improved YOLOv7 model is retained, and the video data of the strip steel captured by the machine is transferred to the computer. On the computer where the improved model has been deployed, the defects of the strip steel are identified, and the model will output the detection results of the strip steel defects.

[0083] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting surface defects in strip steel based on an improved YOLOv7 algorithm, characterized in that, Includes the following steps: Step 1. Prepare the dataset by downloading a dataset on strip defects from the internet; Step 2. Process the dataset to obtain a dataset suitable for training; Step 3. Improve the network: (a) The dimensions of the anchorboxes suitable for this dataset are calculated using the k-means++ clustering method, and the parameters of the original anchorboxes are replaced with the parameters of the newly generated anchorboxes in the yaml file. (b) Replace some convolutions in the ELAN module of the network with PConv, and rename the CBS module after the convolutions are replaced to PBS module. The new ELAN module is named ELANP. (c) Add an attention mechanism to the last convolutional layer of the ELAN module, name the last CBS module CBS-Att, name the new ELAN module ELANP-Att, replace all ELAN modules before concatenation with the ELANP-Att module with the added attention mechanism, add the attention mechanism to the SPCSPC module, name it SPCSPC-Att, and replace the only SPCSPC module in the network with the SPCSPC-Att module. (d) Replace the original CIOU loss function in the network with the SIOU loss function; Step 4. Train the network. Create a new YAML file named YOLOv7-improve.yaml. Replace the original ELAN and SPPCSPC modules with the improved modules. Change the path of the cfg file in the train.py file to the path of the YOLOv7-improve.yaml file. Run the train.py file to train the model. After training, record the training results, including the values ​​of precision, recall, and mAP, and save the generated weight file. Deploy the trained model to a computer to identify defects in the strip steel during the production process.

2. The method for detecting surface defects in strip steel based on the improved YOLOv7 algorithm according to claim 1, characterized in that, In step 1, a dataset of strip steel defects is downloaded from the website. 1800 images are selected from the downloaded dataset and divided into six categories of strip steel surface defects, including inclusions, scratches, rolled-in scale, cracks, pitted surfaces, and patches. These images are then preprocessed.

3. The method for detecting surface defects in strip steel based on the improved YOLOv7 algorithm according to claim 1, characterized in that, In step 2, the dataset is processed by dividing it into a training set, a test set, and a validation set in a ratio of 8:1:

1. The XML files of the labeled dataset are stored in the Annotations folder. The XML files of the labeled results are converted into TXT files using a script and stored in the Labels folder. The names of the images contained in the training set, test set, and validation set of the dataset are stored in the ImageSets folder using a script, and the image paths are placed in the data directory.

4. The method for detecting surface defects in strip steel based on the improved YOLOv7 algorithm according to claim 1, characterized in that, In the strip steel production workshop, fixed cameras are used to film the strip steel that needs to be inspected on the conveyor belt. The filmed video is then imported into a computer that has been configured to run the improved YOLOv7 model. The improved model is used to inspect the filmed video, and the model will output the detection results of the defects in the strip steel.

Images