A YOLO v5 method for detecting surface damage on steel by introducing attention mechanism and ConvNext network

Abstract

This invention discloses a YOLO v5 method for detecting surface damage on steel by incorporating an attention mechanism and a ConvNext network, comprising the following steps: S1: By acquiring image data, data augmentation is performed on images of six types of defects: cracks, patches, inclusions, scratches, pitting, and indented iron oxide scale, thereby obtaining 8100 images to be detected and establishing a test dataset; S2: A neural network model for YOLO v5 is established and improved, including the incorporation of an attention mechanism and a ConvNext network model; S3: Images of steel from the training dataset are input into the YOLO v5 network model to train and deepen the YOLO v5 network model; S4: The trained weight file is saved to the YOLO network model, and this weight file is used to detect the steel surface to determine whether there is damage on the steel surface; This invention combines deep learning networks and image processing to achieve rapid and accurate detection of surface damage on steel.

Description

Technical Field

[0001] This invention belongs to the field of mechanical surface fault diagnosis and intelligent image recognition. More specifically, it designs a YOLO v5 method for detecting steel surface damage by introducing an attention mechanism and a ConvNext network. Background Technology

[0002] Steel is an extremely important material in the machinery manufacturing industry. Steel possesses excellent surface quality and mechanical properties, and is widely used in industries such as vehicle production, aviation, and shipbuilding. Currently, my country is in the late stage of industrialization, with a huge demand for steel, presenting the steel industry with significant market potential. As the demand for steel increases, the requirements for steel quality will also become increasingly stringent. However, during the steel production process, various surface defects inevitably occur due to various reasons, such as cracks, inclusions, patches, pitting, indented iron oxide scale, and scratches. Surface damage has a significant impact on the quality of steel, not only deteriorating its appearance but also reducing its mechanical properties and corrosion resistance. Therefore, how to quickly and efficiently detect surface damage in steel plates is a current research focus.

[0003] Traditional methods for detecting surface damage on steel mainly include manual inspection and traditional photoelectric inspection. Manual inspection, also known as visual inspection, relies primarily on the worker's eye, lacks fixed standards, and has very low efficiency, making it unsuitable for rapid and accurate detection requirements; therefore, it has gradually been abandoned. Traditional photoelectric inspection mainly includes eddy current testing and magnetic flux leakage testing. Compared to manual inspection methods, these photoelectric technology systems offer improvements in speed, accuracy, and efficiency. However, they all have limitations. For example, eddy current testing has advantages such as high sensitivity, rapid response, and ease of operation. However, it also wastes resources and is difficult to detect small damaged targets, making it less suitable for surface damage detection on steel.

[0004] YOLO v5, as one of the best-performing object detection algorithms currently available, can detect objects in real time and achieve good detection results, and has been widely used in academia. However, YOLO v5 has the following drawbacks: 1) It is not accurate enough when detecting small objects; 2) The bounding box position is not precise enough; 3) It consumes a lot of memory when training on large datasets. Summary of the Invention

[0005] The purpose of this invention is to provide a YOLO v5 method for detecting surface damage on steel by incorporating an attention mechanism and a ConvNext network. This method can efficiently and accurately detect and extract damage features under complex background conditions, and the model exhibits good transferability and generalization. It also lowers the hardware requirements for model training and detection, making it widely applicable to industrial inspection.

[0006] This invention is achieved through the following technical solution:

[0007] A YOLO v5 method for detecting surface damage on steel, incorporating an attention mechanism and a ConvNext network, includes the following steps:

[0008] S1: Create a test dataset:

[0009] Image data was acquired, and then physical transformations were used to enhance the images of six types of defects, namely inclusions, cracks, pits, patches, scratches, and indented iron oxide scale, to obtain a test dataset consisting of 8,100 images to be tested.

[0010] S2: Building and improving the neural network model of YOLO v5:

[0011] Build a neural network model for YOLO v5: adjust and configure the corresponding parameters;

[0012] Improved YOLO v5 neural network model: replaced the original CSPdarknet with ConvNext; added an attention mechanism to the enhanced network layer for feature extraction;

[0013] S3: Training to optimize the YOLO v5 network model:

[0014] The augmented test dataset from S1 is input into the improved YOLO v5 network model for training, with 300 training generations and a batch size of 6. Each training iteration of the improved YOLO v5 network model generates a corresponding weight file.

[0015] S4: Save the trained weight file to the YOLO network model, and then predict the steel based on the weight file to detect whether there is damage on the steel surface.

[0016] Preferably, S1 specifically includes:

[0017] S11: Select the target structure and use a camera to acquire images of the steel surface;

[0018] The images include both damaged and undamaged steel.

[0019] The data collection should include various conditions, including changes in light intensity and shadow.

[0020] S12: Segment the steel surface image obtained in S11 into smaller images to increase the number of images in the training library;

[0021] S13: The segmented small images are labeled as images with damage and images without damage, forming an image database;

[0022] S14: Perform data augmentation on the image data:

[0023] Data augmentation methods include horizontal image flipping, vertical image flipping, adding Gaussian noise, and color temperature transformation.

[0024] Preferably, S2 specifically includes:

[0025] S21: The backbone network for feature extraction of the network model is ConvNext, which is constructed from the ConvNet module;

[0026] ConvNext first uses 7×7 layer-by-layer convolution, and then performs normalization to keep the parameters of image size and pixel size of steel consistent.

[0027] In this method, layer normalization replaces batch normalization to accelerate network convergence and reduce overfitting.

[0028] S22: Use the new predicted box size obtained by K-Means re-clustering to obtain a new prior box;

[0029] S23: An FPN feature pyramid was constructed in the enhanced network layer for enhanced feature extraction;

[0030] An attention mechanism (ECA) module is added to the enhanced network layer for feature extraction;

[0031] S24: Set frozen training: The learning rate uses cosine annealing decay. The learning rate will first rise and then fall. When it rises, it uses linear rise, and when it falls, it simulates the cosine function decline.

[0032] S25: The GELU activation function is used on the backbone network, where the GELU activation function is expressed as:

[0033] x*sigmoid(1.703x).

[0034] Preferably, S21 specifically includes:

[0035] S211: The ConvNext network model uses a 4×4 convolutional kernel with a stride of 4 to ensure that the windows do not intersect and only one patch of features is processed at a time. At the same time, LayerNorm is added after the convolutional layer to replace the original BatchNorma.

[0036] S212: The ConvNext network model uses fewer activation layers: activation layers are used between the two 1×1 convolutional kernels in the bottleneck, and not in other locations.

[0037] Preferably, S23 specifically includes:

[0038] ECA attention mechanism is added when the backbone network of feature extraction enters the feature fusion layer of the feature enhancement network and when it enters the upsampling layer after convolution, in order to enhance the feature extraction of the target to be detected in the image and reduce the feature extraction of background targets.

[0039] Preferably, S24 specifically includes:

[0040] S241: In frozen training, the training of the first 50 generations is frozen, and the batch_size is 12;

[0041] S242: Select the SGD optimizer in the training model, with the value of the momentum parameter set to 0.937, and use the cosine learning rate descent method.

[0042] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0043] 1) To address the issue of insufficient accuracy in detecting small targets in YOLO v5, this invention incorporates an ECANet attention mechanism module into the feature layer module's upsampling module. This allows the network to focus on the target to be detected, thereby highlighting important features and improving the accuracy of small target detection.

[0044] 2) Regarding the problem of insufficient accuracy of bounding box position, this invention improves the K-Means algorithm for obtaining prediction boxes. It abandons the original prediction box size of YOLO v5 and uses the new prediction box size obtained by K-Means re-clustering for small target detection such as steel damage to obtain a more suitable prior box, thereby improving the detection accuracy.

[0045] 3) To address the issue of high memory consumption in YOLO v5 when training on large datasets, this invention applies ConvNext to the YOLO v5 backbone network. ConvNext can capture global information and rich contextual information, and it performs better on high-density objects, improving YOLOv5's performance in detecting small targets and thus increasing detection accuracy. Simultaneously, the ConvNext network model consumes less memory and exhibits good robustness when processing large datasets.

[0046] 4) This invention proposes a YOLO v5 method for detecting surface damage on steel by introducing an attention mechanism and a ConvNext network model. The damage detection can be performed in real time, ensuring the high efficiency of damage detection. Attached Figure Description

[0047] Figure 1 The diagram shows a basic flow chart of an embodiment of the present invention.

[0048] Figure 2 Here is an example of a horizontally flipped image after data augmentation;

[0049] Figure 3 Example of a vertically flipped image after data augmentation;

[0050] Figure 4 Example of an image with Gaussian noise added after data augmentation;

[0051] Figure 5 Example of an image with added color temperature transformation after data augmentation;

[0052] Figure 6 This is a flowchart illustrating the backbone network of ConvNext.

[0053] Figure 7 This is a flowchart of the improved YOLO v5.

[0054] Figure 8 The mAP plot of the improved YOLO v5 for steel damage detection;

[0055] Figure 9 Example image of the detected damage portion of the model output. Detailed Implementation

[0056] This invention aims to improve the accuracy of detecting small targets by optimizing and improving YOLO v5.

[0057] This invention proposes a YOLO v5 method for detecting surface damage on steel by incorporating an attention mechanism and a ConvNext network. The damage detection can be performed in real time, ensuring high efficiency.

[0058] This invention replaces the original CSPdarknet backbone network with ConvNext. ConvNext is a pure convolutional neural network built from standard ConvNet modules, offering good accuracy and scalability. ConvNext can capture global information and rich contextual information, and performs better on high-density objects. Therefore, leveraging these characteristics, this invention uses it in the YOLOv5 backbone network, improving YOLOv5's performance in detecting small targets and thus increasing detection accuracy. ConvNext allows the network model to consume less memory and exhibits good robustness when processing large datasets. The ConvNet network also provides better parallelism for the algorithm, giving the improved YOLOv5 model greater research potential.

[0059] This invention also incorporates an ECANet attention mechanism module into the feature layer upsampling module, using one-dimensional convolution to efficiently achieve local cross-channel interaction and extract inter-channel dependencies. This allows the network to focus on the target to be detected, thereby highlighting important features and improving the accuracy of the network model in target detection.

[0060] To obtain more accurate bounding boxes, this invention improves the K-Means algorithm for acquiring bounding boxes, abandoning the original bounding box size of YOLO v5. For small target detection such as steel defects, the new bounding box size obtained by K-Means re-clustering yields more suitable prior boxes, resulting in better detection performance.

[0061] This invention uses the GELU activation function on the backbone network, which can avoid the gradient vanishing problem.

[0062] This invention also proposes a step-by-step training method for the model to accelerate training speed. This model has low requirements for the operating environment and is easy to deploy on computers with moderate performance, or in the Internet of Things (IoT) field, embedded systems, and mobile device systems, offering higher portability.

[0063] Example 1:

[0064] This embodiment presents a YOLO v5 method for detecting surface damage on steel by incorporating an attention mechanism and a ConvNext network, such as... Figure 1 As shown, it includes the following steps:

[0065] S1: Acquire image data. For images of six types of defects, namely inclusions, cracks, pits, patches, scratches, and indented iron oxide scale, perform data enhancement on the images through physical transformations such as horizontal flipping, adding Gaussian noise, and vertical flipping, thereby obtaining a test dataset consisting of 8100 images to be tested.

[0066] S2: Build and improve the neural network model of YOLO v5.

[0067] Adjust and configure the corresponding parameters, such as the parameters of the prior boxes, the learning rate descent method, the number of network layers, and the activation function; in order to make the network model have higher detection accuracy and better robustness in processing large datasets, this invention replaces the original CSPdarknet with ConvNext in the backbone network model; in order to improve the accuracy of detecting small targets, this invention also adds an attention mechanism to the feature extraction enhancement network layer.

[0068] S3: Training to optimize the YOLO v5 network model.

[0069] The augmented test dataset from S1 is input into the improved YOLO v5 network model for training, with 300 training generations and a batch size of 6. Each training iteration of the improved YOLO v5 network model generates a corresponding weight file.

[0070] S4: Save the trained weight file to the YOLO network model, and then use this weight file to predict steel and detect whether there is damage on the steel surface.

[0071] Example 2:

[0072] This embodiment is a further optimization based on Embodiment 1, and S1 specifically includes:

[0073] S11: Select the target structure and acquire images of the steel surface using a camera. The images should include both damaged and undamaged steel, and the acquisition should cover various conditions, such as light intensity, shadow variations, and other factors that may be present in the actual conditions.

[0074] S12: Segment the images obtained in S11 into smaller images to increase the number of images in the training library. Scanning smaller images makes it easier to transfer to larger images, while the reverse process is more prone to errors.

[0075] S13: The segmented images are labeled as images with damage and images without damage, forming an image database. During the labeling process, images with suspected edge damage are removed because it is difficult to distinguish the specific cause and analyze it, which is not conducive to the convolutional neural network's recognition of its features.

[0076] S14: Data augmentation of image data. To reduce data usage and improve efficiency, data augmentation techniques are applied after the test dataset is established. Data augmentation includes horizontal image flipping, etc. Figure 2 As shown, vertically flipped as Figure 3 As shown, Gaussian noise is added as follows: Figure 4 As shown, color temperature changes are as follows Figure 5 As shown, data augmentation can increase the size of the dataset and reduce overfitting.

[0077] Example 3:

[0078] This embodiment is a further optimization based on embodiment 2, and S2 specifically includes:

[0079] S21: The backbone network for feature extraction in the network model is ConvNext. ConvNext is a pure convolutional neural network built from the standard ConvNet module, offering good accuracy and scalability. ConvNext can capture global information and rich contextual information, and performs better on high-density objects.

[0080] Therefore, taking advantage of this characteristic, this invention uses it in the backbone network of YOLO v5, improving the performance of YOLO v5 in detecting small targets and thus increasing detection accuracy. With the same number of FLOPs, ConvNext has faster inference speed and higher accuracy, showing superior performance when handling large datasets, and this network also possesses strong domain generalization capabilities. Therefore, in this invention, ConvNext is used as the backbone network.

[0081] It first utilizes 7×7 layer-by-layer convolutions, followed by normalization to ensure consistency in the parameters of the steel image size and pixel dimensions. During training, the network model excludes irrelevant features, training only on key features, thus improving training efficiency. Layer normalization is used instead of batch normalization, which accelerates network convergence, reduces overfitting, and improves accuracy.

[0082] S22: The original YOLO v5 prediction box size was obtained by selecting 9 clusters using the COCO dataset. This invention, targeting small targets such as steel defects, uses K-Means re-clustering to obtain a new prediction box size, resulting in a more suitable prior box and thus improving detection accuracy.

[0083] S23: An FPN feature pyramid was constructed in the enhanced feature extraction network layer to improve feature extraction. An attention mechanism (ECA) module was also added to the enhanced feature extraction network layer to further improve the network's feature extraction capabilities.

[0084] The ECA module improves network performance by adding parameters. This module generates channels through fast one-dimensional convolutions, and its kernel size is adaptively determined through a non-linear mapping of channel dimensions. ECA is a lightweight, plug-and-play module that allows the network to focus on the target being detected, thereby highlighting important features and improving the accuracy of target detection.

[0085] S24: To achieve higher training efficiency, this model also employs a frozen training method. The learning rate of this model uses cosine annealing decay, which first increases and then decreases. The increase is linear, and the decrease is simulated by the cosine function.

[0086] S25: This invention uses the GELU activation function on the backbone network, which can avoid the gradient vanishing problem. GELU has the characteristics of upper and lower bounds, smoothness, and non-monotonicity. It introduces the idea of ​​random regularization in the activation process, which makes the model have better generalization ability.

[0087] The GELU activation function is expressed as:

[0088] x*sigmoid(1.703x).

[0089] The sigmoid function is a basic activation function with the formula 1 / (1+ex); where x refers to the input rate.

[0090] Example 4:

[0091] This embodiment is a further optimization based on embodiment 3. S21 specifically includes:

[0092] S211: The ConvNext network model uses a 4×4 convolutional kernel with a stride of 4 to ensure that the windows do not intersect and only one patch of features is processed at a time. At the same time, LayerNorm is added after the convolutional layer to replace the original BatchNorma.

[0093] S212: The ConvNext network model uses fewer activation layers. Activation layers are only used between the two 1×1 convolutional kernels in the bottleneck, and not at other locations. It eliminates the need to add activation layers after every convolutional layer; frequent non-linear mappings are actually detrimental to the learning of network features.

[0094] Example 5:

[0095] This embodiment further optimizes upon embodiment 4. Specifically, S23 includes: In this invention, when the backbone network for feature extraction enters the feature fusion layer of the feature enhancement network and when it enters the upsampling layer after convolution, an ECA attention mechanism is added to enhance the feature extraction of the target to be detected in the image and reduce the feature extraction of background targets, thereby enhancing the representativeness of important features and improving the accuracy of target detection.

[0096] Example 6:

[0097] This embodiment is a further optimization based on embodiment 5, and step S24 specifically includes:

[0098] S241: In frozen training, this invention freezes the training for the first 50 generations, with a batch size of 12. During the frozen training of the first 50 generations, the training set loss value shows a continuous downward trend overall. During the 50th generation training, the training set loss value suddenly drops due to the unfreezing of the training. In subsequent training processes, the loss value slowly decreases until it hardly changes, and the model converges.

[0099] S242: In training the model, the SGD optimizer is selected, with the value of the momentum parameter set to 0.937, and the cosine learning rate descent method is used to control the parameter update speed.

[0100] Example 7:

[0101] This embodiment further optimizes the previous embodiment 6. Specifically, S3 includes: inputting the enhanced test dataset from S1 into the improved YOLO v5 network model for training; then unifying the trained sample data to form a feature extraction matrix with consistent dimensions; enhancing the extracted feature matrix; and training the convolutional neural network model based on a large number of collected image samples to obtain the trained training weights.

[0102] The improved YOLO v5 network structure was applied to a dataset of 8100 images created through data augmentation of six types of defects: cracks, inclusions, patches, pitting, indented iron oxide scale, and scratches. Experimental results verified that the improved YOLO v5 achieved an mAP value of 81.64% in steel surface inspection. Figure 8 As shown, compared with existing methods, it achieves better results and can meet practical needs.

[0103] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications or equivalent changes made to the above embodiments based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for detecting surface damage on steel using YOLO v5 with an attention mechanism and a ConvNext network, characterized in that, Includes the following steps: S1: Create a test dataset: Image data was acquired, and then physical transformations were used to enhance the images of six types of defects, namely inclusions, cracks, pits, patches, scratches, and indented iron oxide scale, to obtain a test dataset consisting of 8,100 images to be tested. S2: Building and improving the neural network model of YOLO v5: Build a neural network model for YOLO v5: adjust and configure the corresponding parameters; Improved YOLO v5 neural network model: replaced the original CSPdarknet with ConvNext; added an attention mechanism to the enhanced network layer for feature extraction; S2 specifically includes: S21: ConvNext is the backbone network for feature extraction in the YOLO v5 network model. ConvNext is constructed from the ConvNet module; ConvNext first uses 7×7 layer-by-layer convolution, and then performs normalization to keep the parameters of image size and pixel size of steel consistent. In this method, layer normalization replaces batch normalization to accelerate network convergence and reduce overfitting. S22: Use the new predicted box size obtained by K-Means re-clustering to obtain a new prior box; S23: An FPN feature pyramid was constructed in the enhanced network layer for enhanced feature extraction; An attention mechanism (ECA) module is added to the enhanced network layer for feature extraction; S24: Set frozen training: The learning rate uses cosine annealing decay. The learning rate will first rise and then fall. When it rises, it uses linear rise, and when it falls, it simulates the cosine function decline. S25: The GELU activation function is used on the backbone network, where the GELU activation function is expressed as: x*sigmoid(1.703x); S3: Training to optimize the YOLO v5 network model: The augmented test dataset from S1 is input into the improved YOLO v5 network model for training, with 300 training generations and a batch size of 6. Each training iteration of the improved YOLO v5 network model generates a corresponding weight file. S4: Save the trained weight file to the YOLO network model, and then predict the steel based on the weight file to detect whether there is damage on the steel surface.

2. The method for detecting surface damage on steel using YOLO v5 with an attention mechanism and ConvNext network as described in claim 1, characterized in that, S1 specifically includes: S11: Select the target structure and use a camera to acquire images of the steel surface; The images include both damaged and undamaged steel. The data collection should include various conditions, including changes in light intensity and shadow. S12: Segment the steel surface image obtained in S11 into smaller images to increase the number of images in the training library; S13: The segmented small images are labeled as images with damage and images without damage, forming an image database; S14: Perform data augmentation on the image data: Data augmentation methods include horizontal image flipping, vertical image flipping, adding Gaussian noise, and color temperature transformation.

3. The method for detecting surface damage on steel using YOLO v5 with an attention mechanism and ConvNext network as described in claim 1, characterized in that, S21 specifically includes: S211: The ConvNext network model uses a 4×4 convolutional kernel with a stride of 4 to ensure that the windows do not intersect and only one patch of features is processed at a time. At the same time, LayerNorm is added after the convolutional layer to replace the original BatchNorma. S212: The ConvNext network model uses fewer activation layers: activation layers are used between the two 1×1 convolutional kernels in the bottleneck, and not in other locations.

4. The method for detecting surface damage on steel using YOLO v5 with an attention mechanism and ConvNext network as described in claim 3, characterized in that, S23 specifically includes: ECA attention mechanism is added when the backbone network of feature extraction enters the feature fusion layer of the feature enhancement network and when it enters the upsampling layer after convolution, in order to enhance the feature extraction of the target to be detected in the image and reduce the feature extraction of background targets.

5. The method for detecting surface damage on steel using YOLO v5 with an attention mechanism and ConvNext network as described in claim 4, characterized in that, S24 specifically includes: S241: In frozen training, the training of the first 50 generations is frozen, and the batch_size is 12; S242: Select the SGD optimizer in the training model, with the value of the momentum parameter set to 0.937, and use the cosine learning rate descent method.

Images