A strip steel surface defect detection method based on improved YOLOv5 algorithm

Abstract

The present application relates to the technical field of surface defect detection, and particularly relates to a strip steel surface defect detection method based on an improved YOLOv5 algorithm, aiming at the defect detection problems such as cracks, patches and scratches generated in the production process of strip steel, a strip steel surface defect detection model is constructed based on the improved YOLOv5 algorithm for strip steel defect detection, specifically, a multi-scale detection layer is added at the output end, a CA attention mechanism is introduced before the pooling layer in the Backbone network structure, BIFPN_Add is used in the Neck network structure to perform multi-scale feature fusion, the step of changing the C3 module into the CNeB module is improved, the parameter quantity and the calculation quantity are reduced, and meanwhile the feature extraction capability of the network can be improved, compared with the current strip steel defect detection method, the detection accuracy is improved on the basis of maintaining low parameter quantity and fast detection speed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of surface defect detection, and in particular to a strip steel surface defect detection method based on an improved YOLOv5 algorithm. BACKGROUND

[0002] Strip steel is one of the important raw materials of steel, and is widely used in the important products of China's national economy and steel industry, and plays an important role in various production and life. The main basis for measuring material efficiency is the performance, geometric size and surface quality of the material. At present, the strip steel product can basically meet the requirements in terms of material and size, but the surface quality problem often becomes the main influencing factor. Due to the limitation of production process, cracks, patches, scratches and other defects will occur on the surface of the strip steel. These defects have a great influence on the corrosion resistance and fatigue strength of the strip steel product, and may reduce the service life and use of the strip steel, or even affect the construction safety of buildings and cause building accidents.

[0003] The existing defect detection means mainly uses artificial naked eye detection, which relies too much on human experience, has low detection efficiency, high labor intensity and high production cost, and cannot meet the needs of strip steel surface defect detection. In addition, some enterprises use template matching and other methods to improve the efficiency to a certain extent and save labor costs, but such methods have certain limitations and still cannot meet the needs of industrial defect detection.

[0004] However, the deep learning convolutional neural network has strong autonomous image feature extraction capability, and with the rapid development of big data and the improvement of hardware computing power, the computer vision-based detection scheme gradually replaces the traditional detection method. Due to the complex background of industrial strip steel surface defects, the multiple defect categories, the different representation forms of various defects, and the differences in the representation form of the same defect category, the deep learning model cannot well extract the features, resulting in general detection effect of individual categories. SUMMARY

[0005] The purpose of the present application is to provide a strip steel surface defect detection method based on an improved YOLOv5 algorithm, which aims to improve the detection accuracy of the existing deep learning model strip steel defect detection method based on maintaining low parameter quantity and fast detection speed.

[0006] To achieve the above purpose, the present application provides a strip steel surface defect detection method based on an improved YOLOv5 algorithm, comprising the following steps:

[0007] Collecting strip steel surface defect images and performing screening and preprocessing;

[0008] Performing defect labeling on the preprocessed strip steel surface defect images to obtain a data set, and then dividing the data set into a training set, a validation set and a test set;

[0009] construct a strip steel surface defect detection model based on an improved YOLOv5 algorithm;

[0010] input the training set into the strip steel surface defect detection model for training;

[0011] use the trained strip steel surface defect detection model to detect the test set.

[0012] The preprocessing process of the strip steel surface defect image includes size normalization, random flipping, brightness adjustment, and grayscale processing, etc., wherein the size normalization specification is 640*640.

[0013] The Labelimg software is used to mark the defect type and the defect position in the defect marking process, and the training set, the verification set and the test set are divided according to the ratio of 8:1:1.

[0014] The improved process of constructing the strip steel surface defect detection model based on the improved YOLOv5 algorithm includes the following steps:

[0015] a new detection head is added at the output end;

[0016] a CA attention mechanism is introduced before the pooling layer in the Backbone network structure;

[0017] feature stacking and deep-shallow feature fusion are performed through BIFPN_Add in the Neck network structure;

[0018] the C3 module corresponding to the four detection layers in the Neck network structure is changed to the CNeB module.

[0019] The new detection head is a detection head with a downsampling of 64 times, and the other three detection heads are respectively a detection head with a downsampling of 32 times, a detection head with a downsampling of 16 times, and a detection head with a downsampling of 8 times.

[0020] The CA attention mechanism decomposes the channel attention into two one-dimensional feature encoding processes, respectively along two different spatial directions to aggregate the defect features.

[0021] The BIFPN_Add replaces the Concat in the original BIFPN, and each input layer of the BIFPN_Add requires that the image size and the image channel number remain consistent.

[0022] The CNeB module is constructed by changing the Bottleneck in C3 to ConvNextBlock, wherein the ConvNextBlock structure is similar to the residual module and has two branches.

[0023] Wherein, in the training process, epoch is set to 250, batch size is set to 16, and image size is set to 640*640.

[0024] The application provides a strip steel surface defect detection method based on an improved YOLOv5 algorithm. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description.

[0026] Figure 1 is an improved YOLOv5 network structure diagram of a strip steel surface defect detection method based on the improved YOLOv5 algorithm of the application.

[0027] Figure 2 is a CA attention mechanism principle diagram of the application.

[0028] Figure 3 is a ConvNextBlock structure principle diagram of the application.

[0029] Figure 4 is a structure principle diagram of the CNeB module of the application.

[0030] Figure 5 is an mAP diagram of a YOLOv5s original model of the application.

[0031] Figure 6 is an mAP diagram of an improved YOLOv5s model of the application. DETAILED DESCRIPTION

[0032] Embodiments of the present application are described below in detail, examples of which are shown in the drawings, wherein the same or similar reference numbers represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0033] The present application provides a strip steel surface defect detection method based on improved YOLOv5 algorithm, comprising the following steps:

[0034] S1: collecting strip steel surface defect images and performing screening and preprocessing;

[0035] S2: performing defect labeling on the preprocessed strip steel surface defect images to obtain a data set, and then dividing the data set into a training set, a validation set and a test set;

[0036] S3: constructing a strip steel surface defect detection model based on an improved YOLOv5 algorithm;

[0037] S4: inputting the training set into the strip steel surface defect detection model for training;

[0038] S5: using the trained strip steel surface defect detection model to detect the test set.

[0039] The following is further described in combination with specific implementation steps:

[0040] In step S1, an industrial camera is used to collect strip steel surface defect images, and the collected images are screened and preprocessed, wherein the preprocessing operations include size normalization, random flipping, brightness adjustment and grayscale processing operations, and the size normalization specification is 640x640.

[0041] Step S2, specifically, the defects of the strip steel surface defect images are labeled and divided into a training set, a validation set and a test set.

[0042] The Labelimg software is used to label the obtained strip steel surface defect images for defect type and defect position, and generate a data set; the data set is randomly divided into a training set, a validation set and a test set in a ratio of 8:1:1 by a program.

[0043] Step S3, the improved steps based on the improved YOLOv5 algorithm are as follows:

[0044] As shown in Figure 1 , it is the improved YOLOv5 network structure diagram in the present application.

[0045] 1) adding a new detection layer;

[0046] The original YOLOv5 model has three detection layers. When the input image size is 640×640, the output sizes of the feature maps of the detection layers are 80×80, 40×40, and 20×20, respectively, used for detecting small, medium, and large targets. Due to the characteristics of industrial strip steel surface defects—such as blurred backgrounds, numerous defect categories, different representations of various defects, and even differences in the representation of the same type of defect—and considering the size of the defect targets in the dataset, in order to improve the accuracy of defect detection, such as... Figure 1 As shown in the Head network structure, an output layer with a 64x downsampling was added, resulting in a 10×10 output size for the detection layer feature map, meeting the requirement for accurate identification of defects at various scales. During training, adaptive anchor boxes are used to calculate the anchor box size based on the training data. Each output layer has 3 prior boxes, which increases to 12 after the addition of the new detection layer. The new detection layer has a larger receptive field, making it more sensitive to large target features and meeting the requirement for accurate identification of defects at various scales. The multi-scale detection layer helps improve the network's generalization ability, enhances the model's detection performance, and reduces the occurrence of missed and false detections.

[0047] 2) Deep and shallow multi-scale fusion feature network;

[0048] like Figure 1 As shown in the Neck network structure, multi-scale feature fusion of deep and shallow layers is performed in a manner similar to BIFPN, but BIFPN_Add replaces the Concat function in the original BIFPN. First, the C3 module layer of the backbone feature network is connected to the upsampling layer in the FPN network structure via Concat to achieve feature fusion. Then, it skips to connect to the network layers in the PAN network structure, thereby achieving multi-scale feature fusion between deep and shallow networks. BIFPN_Add essentially performs an add operation, not a Concat operation; therefore, the image size and number of image channels must be consistent across the input layers of BIFPN_Add.

[0049] 3) The backbone network incorporates the Coordinate Attention (CA) mechanism;

[0050] like Figure 1 As shown in the Backbone network structure, a Coordinate Attention (CA) mechanism is introduced before the pooling layer. The principle diagram of the CA attention mechanism is as follows. Figure 2As shown, the channel attention is decomposed into two one-dimensional feature encoding processes, which respectively aggregate the defect features along two different spatial directions. In this way, long-range dependencies can be captured along one spatial direction for long-range spatial interaction, while precise location information can be preserved along the other spatial direction. A pair of direction-aware and location-sensitive feature maps are formed, which can be applied to the input feature map complementarily to enhance the feature representation of the target.

[0051] The formula for decomposing the channel attention into two one-dimensional feature encoding processes described above can be expressed as:

[0052]

[0053] The output of the c-th channel with a height of h can be expressed as:

[0054]

[0055] The output of the c-th channel with a width of w can be written as:

[0056]

[0057] 4) Change the C3 module to the CNeB module.

[0058] As shown in the Neck network structure in the Figure 1 , the CNeB module is changed to the CNeB module corresponding to the four detection layers in the Neck network structure, which improves the Bottleneck of the CSP module to the ConvNextBlock, as shown in Figure 3 , the ConvNextBlock structure is similar to the residual module, which has two branches. One branch first passes through a convolution with a kernel size of 7, then a convolution with a kernel size of 1 to increase the number of channels to four times the original, and finally a convolution with a kernel size of 1 to change the number of channels to the initial input; the other branch directly adds the above branch after skipping the above three convolution operations and then outputs. As shown in Figure 4 , the Bottleneck in C3 is changed to the ConvNextBlock to build a new module CNeB. The CNeB module further enhances the receptive field of the model and improves the feature extraction capability of the model for defects such as background blur and difficult-to-distinguish features.

[0059] Step S4, input the training set in step S2 into the strip surface defect detection model based on the improved YOLOv5 algorithm constructed in step 3 for training to obtain the trained strip surface defect detection model.

[0060] The basic parameters of the training are as follows: the epoch is set to 250, the batch size is set to 16, and the image size is set to 640*640.

[0061] The hyperparameters of the training are set as default in the official file: hyp.scratch.yaml

[0062] In step S5, the strip surface defect detection model obtained by training in step 4 is used to detect the test set in step S2.

[0063] Further, the performance of the present application is evaluated by using the public NEU-DET strip surface defect dataset published by Song et al. The NEU-DET dataset is a relatively classic strip defect dataset, which contains a total of six defects, i.e. scratch (Sc), spot (Pa), pitting (Ps), inclusion (in), crack (Cr) and rolled-in scale (Rs), and there are 300 pictures for each type of defect, a total of 1800 pictures.

[0064] Specifically, the experiment takes the YOLOv5s original model as the baseline, uses the dataset and the improved YOLOv5s network structure model in step 3 to train, obtains the weight file of the strip surface defect detection model and the mAP value and parameter quantity of the improved YOLOv5s network structure model, and performs an ablation experiment on the four improved models in step 3 and the YOLOv5s original model.

[0065] The ablation experiment results are shown in Table 1 as follows:

[0066] Table 1 Comparison of ablation experiment results

[0067] Method First group Second group Third group Fourth group Fifth group Yolov5s √ √ √ √ √ Add detection layer √ √ √ √ BIFPN_Add √ √ √ CA attention mechanism √ √ CNeB √ Parameter quantity / 10 6 ]] 7.03 11.64 11.65 11.68 11.5 mAP / % 77.2 78.7 79.8 80.5 80.9

[0068] Specifically, the background of industrial strip surface defects is complex, and the defect categories are various, and the representation forms of various defects are different, and the representation forms of the same defect category are different, etc. The deep learning model cannot well extract features, resulting in general detection effect of individual categories. Using the public NEU-DET strip surface defect dataset and the YOLOv5s original model for training, when Iou=0.5, the mAP value is only 77.2%, and the mAP graph of the YOLOv5s original model is as shown in Figure 5 .

[0069] The accuracy of various strip defects is shown in Table 2:

[0070] Table 2 Strip defect accuracy table of YOLOv5s original model

[0071] Crack Inclusion Spot Pit Rolled-in scale Scratch mAP / % 45.8 83.9 93.3 86.1 61.2 93.2

[0072] Among them, the defect detection accuracy of the crack is only 45.8%, which obviously does not meet the demand of industrial production for accuracy, and the accuracy of penetration into the oxide skin is low.

[0073] Further, the disclosed NEU-DET strip steel surface defect dataset is used for training the improved YOLOv5s model, and when Iou=0.5, the mAP value can reach 80.9% without increasing too many parameters, and the mAP graph of the improved YOLOv5s model is as shown in Figure 6

[0074] The improved accuracy of various strip steel defects is shown in Table 3:

[0075] Table 3 Improved accuracy of strip steel defects

[0076] Crack Inclusion Spot Pit Rolled-in scale Scratch mAP / % 60.7 89.8 91.7 84.1 66.7 92.4

[0077] Among them, the defect detection accuracy of the crack, inclusion and penetration into the oxide skin has been improved significantly, and the experiment shows that the method proposed in the present application further improves the precision of the YOLOv5s network model in the industrial strip steel defect detection under the condition of ensuring that the parameter amount is not increased too much, can effectively detect the position and classification of six common defects, and further alleviates the problem of general detection effect of individual categories for the industrial strip steel dataset.

[0078] The present application is based on the YOLOv5 model, and a multi-scale detection layer is added at the output end, the new detection layer has a larger receptive field and is more sensitive to large target features, meeting the demand for accurate identification of defects of various scales, the multi-scale detection layer is beneficial to improve the generalization ability of the network, improve the detection performance of the model, and reduce the occurrence of missed detection and false detection; the present application introduces CoordinateAttention before the pooling layer in the Backbone network structure, CoordinateAttention decomposes the channel attention into two one-dimensional feature encoding processes of aggregating features along different directions, enhancing the expression ability of the strip steel defect features; the present application performs multi-scale feature fusion through BIFPN_Add in the Neck network structure, superimposes the features of the deep and shallow layers of the network, which significantly reduces the parameter amount and calculation amount compared with the existing Concat operation in BIFPN, and at the same time can improve the feature extraction capability of the network; the present application changes the C3 module corresponding to the four detection layers in the Neck network structure to CNeB module, further enhances the receptive field of the model, and improves the feature extraction capability of the model for defects such as background blur and difficult-to-distinguish features.

[0079] In summary, the present application has the following advantages compared with the existing strip steel surface defect detection method:

[0080] ​1. The application provides a strip steel surface defect detection method based on an improved YOLOv5 algorithm, aiming at the defect detection problems of cracks, patches, scratches and other defects generated in the production process of strip steel, an improved YOLOv5s target detection algorithm is used to detect common strip steel defects in the industrial production process, compared with the current strip steel defect detection method, the detection accuracy is improved on the basis of maintaining low parameter quantity and fast detection speed.

[0081] 2. The application adds a multi-scale detection layer at the output end, the new detection layer has a larger receptive field and is more sensitive to large target features, meeting the demand for accurately identifying defect targets of various scales. The multi-scale detection layer helps to improve the generalization ability of the network, improve the detection performance of the model, and reduce the occurrence of missed detection and false detection.

[0082] 3. The application introduces CoordinateAttention before the pooling layer in the Backbone network structure, Coordinate Attention decomposes the channel attention into two one-dimensional feature encoding processes that aggregate features along different directions, enhancing the expression ability of strip steel defect features.

[0083] 4. The application performs multi-scale feature fusion through BIFPN_Add in the Neck network structure, superimposes the features of the deep and shallow layers of the network, and compared with the Concat operation in the existing BIFPN, significantly reduces the parameter quantity and calculation amount, and at the same time improves the feature extraction capability of the network.

[0084] 5. The application changes the C3 module corresponding to the four detection layers in the Neck network structure to the CNeB module, further enhances the receptive field of the model, and improves the feature extraction capability of the model for defects such as background blur and difficult-to-distinguish features.

[0085] The above disclosure is only a preferred embodiment of the application, and of course cannot limit the scope of the application, and those skilled in the art can understand that the above-mentioned embodiment can be implemented in whole or in part, and the equivalent changes made according to the claims of the application still belong to the scope covered by the application.

Claims

1. A strip steel surface defect detection method based on an improved YOLOv5 algorithm, characterized in that, The method comprises the following steps: Collecting strip steel surface defect images and performing screening and pretreatment; Pretreated strip steel surface defect images are labeled to obtain a data set, and then divided into a training set, a validation set and a test set; Constructing a strip steel surface defect detection model based on an improved YOLOv5 algorithm; The process of constructing a strip steel surface defect detection model based on an improved YOLOv5 algorithm comprises the following steps: Adding a new detection head to the output end; The new detection head is a detection head with a downsampling factor of 64, and the other three detection heads are detection heads with downsampling factors of 32, 16 and 8, respectively; Introducing a CA attention mechanism before the pooling layer in the Backbone network structure; In the Neck network structure, BIFPN_Add is used to perform feature stacking and fusion of deep and shallow layer features; The C3 module corresponding to the four detection layers in the Neck network structure is changed to a CNeB module; The BIFPN_Add replaces the Concat in the original BIFPN, and each input layer of the BIFPN_Add requires the image size and the image channel number to be consistent; The CNeB module is constructed by changing the Bottleneck in C3 to a ConvNextBlock, wherein the ConvNextBlock structure has two branches. The training set is input into the strip steel surface defect detection model for training; The trained strip steel surface defect detection model is used to detect the test set.

2. The strip steel surface defect detection method based on the improved YOLOv5 algorithm according to claim 1, wherein The strip steel surface defect image is pretreated, and the pretreatment operation includes size normalization, random flipping, brightness adjustment and grayscale processing, wherein the size normalization specification is 640x640.

3. The strip steel surface defect detection method based on the improved YOLOv5 algorithm according to claim 1, wherein In the defect labeling process, Labelimg software is used to label the defect type and defect position, and the training set, the validation set and the test set are divided according to the ratio of 8:1:

1.

4. The strip steel surface defect detection method based on the improved YOLOv5 algorithm according to claim 1, wherein The CA attention mechanism decomposes the channel attention into two one-dimensional feature encoding processes, which respectively aggregate the defect features along two different spatial directions.

5. The strip steel surface defect detection method based on the improved YOLOv5 algorithm according to claim 1, wherein In the training process, the epoch is set to 250, the batch size is set to 16, and the image size is set to 640x640.

Images