High-precision lightweight steel surface defect detection method based on YOLOv5 improvement

Abstract

The invention provides a high-precision lightweight steel surface defect detection method based on YOLOv5 improvement, and the method comprises the steps: firstly adding ASF-YOLO design to a Neck part of an original model YOLOv5, and adding an SSFF module, a TFE module and a CPAM attention mechanism module; the SSFF module combines global or high-level semantic information from a plurality of image scales and is used for combined harvesting of combined steel surface multi-scale features; the TFE module is used for capturing local detail features of small objects in the steel surface detection process; the CPAM module is used for integrating information processed by the SSFF module and the TFE module; secondly, a C3 module in an original model YOLOv5 is replaced with a RepNCSPELAN4 module, so that the reasoning speed of the model is increased, the capacity of aggregating multi-scale features is improved, and the model has the high parameter utilization rate; and finally, replacing an original loss function with a WIoU loss function to improve the detection capability of the model, reduce the probability of less detection and missing detection, and improve the overall precision of the model. The precision of the model in a steel defect detection task can be improved, and the real-time detection speed can be guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of steel surface detection, and in particular to a high-precision lightweight steel surface defect detection method based on an improved YOLOv5. Background Art

[0002] The research on steel defect detection is not only of great significance for safety assurance, but also of far-reaching significance at the economic and social levels. The earliest method for steel defect detection is traditional manual visual inspection. However, traditional manual visual inspection methods are highly subjective and susceptible to empirical bias. This may limit the reliability of test results. In addition, manual inspection is inefficient and costly, which limits the further development of traditional manual visual inspection methods.

[0003] With the development of computer vision and image processing technology, traditional image processing methods have begun to be introduced into the field of defect detection. These methods include edge detection, contour extraction, shape analysis, etc., which process and analyze images by computers in an attempt to achieve automated defect detection. However, these methods are often limited by factors such as image quality, lighting changes, and the complexity of defect morphology, and their accuracy and robustness have certain limitations.

[0004] In recent years, with the continuous development of artificial intelligence technology, deep learning has been widely used in steel surface defect detection. In the field of target detection, the two mainstream algorithm architectures are single-stage detection model and two-stage detection model. Among them, the single-stage detection model directly predicts the result from the picture without cropping the picture. This type of model usually uses convolutional neural network for feature extraction and target classification and positioning. The two-stage model divides the target detection task into two stages for processing: the first stage generates candidate frames, and the second stage classifies and locates these candidate frames. Since the single-stage detection model has low computational complexity and memory usage, it is suitable for scenarios with high real-time requirements. At the same time, the steel surface defect detection task requires certain real-time requirements and a model that is easier to deploy. The YOLO series algorithm is the most popular algorithm model in the current single-stage detection model. Therefore, the YOLO series algorithm model has gradually been widely used by scholars in the direction of steel defect detection.

[0005] In view of the above problems, in order to improve the accuracy of the model in the steel defect detection task and ensure the real-time detection speed, this paper proposes a new steel defect detection model ASFRW-YOLO based on YOLOv5. The algorithm uses YOLOv5 as the benchmark model and adds the design of ASF-YOLO

[16] to the neck part, so that the model can integrate multi-scale features and ensure the extraction of small target features and global features. At the same time, the C3 module in YOLOv5 is replaced by the RepNCSPELAN4 module. Compared with the C3 module, this module has the characteristics of high parameter utilization, fast inference speed, better aggregation of multi-scale features, and more flexible forward propagation. Finally, the WIoU loss function is introduced, and its dynamic non-monotonic focusing mechanism is used to further increase the detection accuracy and improve the model training efficiency. Summary of the invention

[0006] The present invention proposes a high-precision lightweight steel surface defect detection method based on the improvement of YOLOv5, which can improve the accuracy of the model in steel defect detection tasks and ensure real-time detection speed.

[0007] The present invention adopts the following technical solutions.

[0008] A high-precision lightweight steel surface defect detection method based on the improvement of YOLOv5 can improve the small target detection capability, real-time detection capability and accuracy of the steel surface detection process. The method first adds ASF-YOLO design to the Neck part on the basis of the original model YOLOv5, and adds SSFF module, TFE module and CPAM attention mechanism module; the SSFF module combines global or high-level semantic information from multiple image scales to jointly harvest the combined multi-scale features of the steel surface; the TFE module is used to capture the local detail features of small objects in the steel surface detection process and enhance the model's detection capability for small targets; the CPAM module is used to integrate the information processed by the SSFF module and the TFE module;

[0009] Secondly, the C3 module in the original model YOLOv5 is replaced with the RepNCSPELAN4 module to enhance the model's reasoning speed and the ability to aggregate multi-scale features, and make the model have a higher parameter utilization rate; finally, the WIoU loss function is used to replace the original loss function to improve the model's detection ability and reduce the probability of under-detection and missed detection, thereby improving the overall accuracy of the model.

[0010] The ASF-YOLO is designed to improve the model's detection capability and accuracy for small targets in steel defect data, and includes a scale sequence feature fusion module SSFF, a triple feature encoding module TFE, and a channel position attention mechanism module CPAM.

[0011] In steel surface defect detection, the SSFF module combines the global semantic information of images at different scales by normalizing, upsampling, and connecting multi-scale features into 3D convolutions to effectively process objects of different sizes, directions, and aspect ratios in the scale space representation of the detection data. Suppose the multi-scale feature maps generated from the model backbone are P3, P4, and P5, which capture the image content in different details or proportions. The feature maps P3, P4, and P5 are first convolved with a series of Gaussian kernels with increasing standard deviations, as shown in the following formula:

[0012] F σ (i, j) = ∑ u ∑ v f(iu,iv)×G σ (u,v) (1)

[0013]

[0014] Where f represents a two-dimensional 2D feature map, and F σ It is generated by smoothing with a series of convolutions using a 2D Gaussian filter with increasing standard deviation σ; the features of P4 and P5 are then resized to the size of P3 using the nearest neighbor interpolation method, and these feature maps of different scales are stacked horizontally; finally, 3D convolution, 3D batch normalization, and SiLU activation function are used to complete the scale sequence feature extraction. The SSFF module structure is as follows Figure 4 shown.

[0015] When detecting surface defects on steel, the TFE module fuses small, medium, and large-scale feature maps to capture local detail information of small objects in images of different scales. Figure 5 The structure of the TFE module is explained; before feature encoding, the TFE module first adjusts the number of feature channels to make it consistent with the main scale feature. After the large-size feature map Large is processed by the convolution module, its number of channels is adjusted to 1C. Then, a hybrid structure of maximum pooling and average pooling is used for downsampling to reduce the spatial dimension of the feature and achieve translation invariance, enhancing the network's robustness to spatial changes and translations of the input image; for the small-size feature map Small, a convolution module is used to adjust the number of channels, and then the nearest neighbor interpolation method is used for upsampling to preserve local features and prevent the loss of small object feature information. Finally, the three feature maps of large, medium, and small sizes with the same dimension are convolved once and then spliced ​​in the channel dimension, as expressed in the following formula:

[0016] F TFE =Concat(F l , F m , F s ) (3)

[0017] where FTFE Represents the feature map output by the TFE module; F l 、F m and F s Represents large, medium and small size feature maps respectively; F TFE By F l 、F m and F s Cascade generation; F TFE With F m Same resolution and three times the number of channels.

[0018] In steel surface defect detection, the CPAM module is used to integrate feature information from the SSFF and TFE modules to allow the model to adaptively adjust its focus on channels and spatial locations related to small objects at different scales;

[0019] The architecture of CPAM is as follows: Figure 6 As shown in the figure, the architecture of CPAM consists of a channel attention network and a position attention network; it includes an input 1 for receiving input from TFE and an input 2 for receiving superimposed input from the channel attention network and SSFF; an attention mechanism without dimensionality reduction is introduced to capture cross-channel interactions in an effective way, and local cross-channel interactions are captured by considering each channel and its k nearest neighbors after performing channel global average pooling without dimensionality reduction.

[0020] When detecting steel surface defects, the RepNCSPELAN4 module is used to reduce the computational complexity and improve the feature extraction effect. Its structure is as follows: Figure 7 As shown in the figure, in the module structure, RepNCSP is a submodule in RepNCSPELAN4, RepNBottleneck is a submodule in RepNCSP, and RepConvN is a submodule in RepNCSP; RepNCSPELAN4 combines RepNCSP, RepNBottleneck and RepConvN together to form an efficient feature extraction unit, which enables the model to extract multi-scale features while maintaining low computational complexity, and ensures the high performance of the model in real-time detection tasks while taking into account computational efficiency and detection accuracy. The module first processes the input X through convolution and divides the result into two parts. The second part is processed through the RepNCSP module and convolution, and finally all the results are merged and output.

[0021] The structures of RepNCSP, RepNBottleneck and RepConvN are as follows Figure 8As shown. In steel surface defect detection, the RepNCSP module is used to enhance the model's detection capability for targets of different sizes through multi-scale feature extraction. It extracts features at different spatial scales by introducing multiple convolutional layers and different strides, so that the model can better capture the features of small and large targets; RepNCSP uses a non-convex convolution kernel configuration to more flexibly adapt to targets of different shapes and sizes, improving the expressiveness of features; RepNCSP reduces the number of model parameters and improves computational efficiency by reusing the same convolution structure. The module processes the input X through convolution and RepNBottleneck respectively, and merges the results, as shown in Figure 8 As shown in (a);

[0022] RepNBottleneck is used to receive multi-scale feature maps from RepNCSP. Its structure is close to the residual structure. When it uses residual connection and the number of input and output channels is the same, the input X is added to the result of X after passing through RepConvN and Conv for output. Otherwise, only the result of X after passing through RepConvN and Conv is returned, such as Figure 8 as shown in (b);

[0023] RepConvN is a basic convolution module that supports training and deployment states and is used to optimize convolution operations, such as Figure 8 as shown in (c).

[0024] The WIoU loss function is the Wise-IoU loss function, which uses a dynamic non-monotonic focusing mechanism (Focusing Mechanism, FM) to solve the adverse effects of high-quality and low-quality samples on model training. When used for the steel surface defect detection model, the following method is used:

[0025] First, the outlier is defined to describe the quality of the anchor box of the labeled sample, which is defined as:

[0026]

[0027] A small outlier means that the anchor box has high quality. A small gradient gain is assigned to it so that the bounding box regression focuses on the anchor box of normal quality. A smaller gradient gain is assigned to the anchor box with a large outlier to prevent low-quality examples from generating large harmful gradients. A non-monotonic focusing coefficient is constructed using β and applied to WIoU:

[0028]

[0029] Among them, when β = δ, δ makes r = 1. When the outlier degree of the anchor box satisfies β = C, and C is a constant, the anchor box will obtain the highest gradient gain;

[0030] because It is dynamic, and the quality division standard of the anchor box is also dynamic, so that WIoU v3 can make the gradient gain allocation strategy that best suits the current situation at every moment.

[0031] The ASF-YOLO design is part of the ASFRW-YOLO model. The ASFRW-YOLO model is trained with the NEU-DET dataset. The surface defects of steel detected by the model include: cracks, patches, inclusions, pitting, pressed-in scaly metal skin, and scratches.

[0032] When the parameter amount of the ASFRW-YOLO model is 1.61M, the detection method runs the ASFRW-YOLO model using an embedded device as a computing unit, and the embedded device includes a smart phone, an industrial camera, or an intelligent monitoring device.

[0033] The deployment of the detection method includes the following steps:

[0034] Step S1, hardware connection: connect the camera and display to the computing unit (such as NVIDIA JetsonNano or Raspberry Pi) to view the detection results in real time; connect the device to the power supply and network at the same time; if the alarm function when a defect is detected is required, connect the LED indicator or buzzer to the interface of the computing unit; Step S2, software environment configuration: install the operating system on the embedded device, install the required deep learning framework PyTorch, ensure that the operation of the ASFRW-YOLO model is supported, and install image processing libraries such as CV for image preprocessing and result visualization;

[0035] Step S3, model deployment: Convert the ASFRW-YOLO model to a format that is compatible with the target hardware and optimize it using TensorRT (for NVIDIA Jetson); then load the ASFRW-YOLO model in the application and initialize it;

[0036] Step S4, data acquisition and processing: setting a camera to collect image data of the steel surface in real time; writing code in the application to perform image preprocessing, including adjusting image size, normalization, etc.;

[0037] Step S5, real-time reasoning: input the preprocessed image into the ASFRW-YOLO model for reasoning to obtain the detection result; draw the detected defect boundary box and category label on the image;

[0038] Step S6, result analysis and feedback: Visualize the test results and display them on the screen, for example Fig.11As shown, an alarm system (such as an LED indicator light or a buzzer) is triggered according to the detection results to notify the operator, and the detection data is recorded for subsequent analysis and improvement.

[0039] The present invention discloses a small sample steel surface defect detection method based on a lightweight network, which relates to the technical field of steel surface defect detection, and includes the following contents: obtaining an image data set of small sample steel surface defects, which data set only contains 1800 images; constructing an ASFRW-YOLO steel surface defect detection lightweight model based on the YOLOv5 improvement, which model has only 1.61M parameters; using the image data set to train the steel surface defect detection model to obtain the best detection model. Compared with the traditional steel surface detection model, the ASFRW-YOLO model constructed by the present invention has higher detection accuracy and lighter model parameters, which can effectively improve the accuracy and reliability of steel surface defect detection, and meets the current needs of steel defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0041] Attached Figure 1 It is a schematic diagram of the steel surface defect categories in the dataset;

[0042] Attached Figure 2 It is the original model. Schematic diagram of the YOLOv5 algorithm network structure;

[0043] Attached Figure 3 This is a schematic diagram of the ASFRW-YOLO algorithm network structure;

[0044] Attached Figure 4 is a schematic diagram of the structure of the scale sequence feature fusion (SSFF) module;

[0045] Attached Figure 5 It is a schematic diagram of the triple feature encoding (TFE) module structure;

[0046] Attached Figure 6 is a schematic diagram of the channel and position attention mechanism module (CPAM) structure;

[0047] Attached Figure 7 It is a schematic diagram of the RepNCSPELAN4 module structure;

[0048] Attached Figure 8 It is a schematic diagram of the structure of RepNCSP, RepNBottleneck and RepConvN;

[0049] Attached Fig. 9 This is a schematic diagram of the PR curve of the YOLOv5s algorithm model;

[0050] Attached Fig.10 This is a schematic diagram of the PR curve of the ASFRW-YOLO algorithm model;

[0051] Attached Fig.11 It is an example schematic diagram of the present invention in actual detection application. DETAILED DESCRIPTION

[0052] As shown in the figure, a high-precision lightweight steel surface defect detection method based on the improvement of YOLOv5 can improve the small target detection capability, real-time detection capability and accuracy of the steel surface detection process, and is characterized in that: the method first adds ASF-YOLO design to the Neck part on the basis of the original model YOLOv5, and adds SSFF module, TFE module and CPAM attention mechanism module; the SSFF module combines global or high-level semantic information from multiple image scales to jointly harvest the combined multi-scale features of the steel surface; the TFE module is used to capture the local detail features of small objects in the steel surface detection process, and enhance the model's detection capability for small targets; the CPAM module is used to integrate the information processed by the SSFF module and the TFE module;

[0053] Secondly, the C3 module in the original model YOLOv5 is replaced with the RepNCSPELAN4 module to enhance the model's reasoning speed and the ability to aggregate multi-scale features, and make the model have a higher parameter utilization rate; finally, the WIoU loss function is used to replace the original loss function to improve the model's detection ability and reduce the probability of under-detection and missed detection, thereby improving the overall accuracy of the model.

[0054] The ASF-YOLO is designed to improve the model's detection capability and accuracy for small targets in steel defect data, and includes a scale sequence feature fusion module SSFF, a triple feature encoding module TFE, and a channel position attention mechanism module CPAM.

[0055] In steel surface defect detection, the SSFF module combines the global semantic information of images at different scales by normalizing, upsampling, and connecting multi-scale features into 3D convolutions to effectively process objects of different sizes, directions, and aspect ratios in the scale space representation of the detection data. Suppose the multi-scale feature maps generated from the model backbone are P3, P4, and P5, which capture the image content in different details or proportions. The feature maps P3, P4, and P5 are first convolved with a series of Gaussian kernels with increasing standard deviations, as shown in the following formula:

[0056] F σ (i, j) = ∑ u ∑ v f(iu,iv)×G σ (u,v) (1)

[0057]

[0058] Where f represents a two-dimensional 2D feature map, and F σ It is generated by smoothing with a series of convolutions using a 2D Gaussian filter with increasing standard deviation σ; the features of P4 and P5 are then resized to the size of P3 using the nearest neighbor interpolation method, and these feature maps of different scales are stacked horizontally; finally, 3D convolution, 3D batch normalization, and SiLU activation function are used to complete the scale sequence feature extraction. The SSFF module structure is as follows Figure 4 shown.

[0059] When detecting surface defects on steel, the TFE module fuses small, medium, and large-scale feature maps to capture local detail information of small objects in images of different scales. Figure 5 The structure of the TFE module is explained; before feature encoding, the TFE module first adjusts the number of feature channels to make it consistent with the main scale feature. After the large-size feature map Large is processed by the convolution module, its number of channels is adjusted to 1C. Then, a hybrid structure of maximum pooling and average pooling is used for downsampling to reduce the spatial dimension of the feature and achieve translation invariance, enhancing the network's robustness to spatial changes and translations of the input image; for the small-size feature map Small, a convolution module is used to adjust the number of channels, and then the nearest neighbor interpolation method is used for upsampling to preserve local features and prevent the loss of small object feature information. Finally, the three feature maps of large, medium, and small sizes with the same dimension are convolved once and then spliced ​​in the channel dimension, as expressed in the following formula:

[0060] F TFE =Concat(F l , F m , F s ) (3)

[0061] where F TFE Represents the feature map output by the TFE module; F l 、F m and F s Represents large, medium and small size feature maps respectively; F TFE By F l 、F m and F s Cascade generation; F TFE With F m Same resolution and three times the number of channels.

[0062] In steel surface defect detection, the CPAM module is used to integrate feature information from the SSFF and TFE modules to allow the model to adaptively adjust its focus on channels and spatial locations related to small objects at different scales;

[0063] The architecture of CPAM is as follows: Figure 6 As shown in the figure, the architecture of CPAM consists of a channel attention network and a position attention network; it includes an input 1 for receiving input from TFE and an input 2 for receiving superimposed input from the channel attention network and SSFF; an attention mechanism without dimensionality reduction is introduced to capture cross-channel interactions in an effective way, and local cross-channel interactions are captured by considering each channel and its k nearest neighbors after performing channel global average pooling without dimensionality reduction.

[0064] When detecting steel surface defects, the RepNCSPELAN4 module is used to reduce the computational complexity and improve the feature extraction effect. Its structure is as follows: Figure 7 As shown in the figure, in the module structure, RepNCSP is a submodule in RepNCSPELAN4, RepNBottleneck is a submodule in RepNCSP, and RepConvN is a submodule in RepNCSP; RepNCSPELAN4 combines RepNCSP, RepNBottleneck and RepConvN together to form an efficient feature extraction unit, which enables the model to extract multi-scale features while maintaining low computational complexity, and ensures the high performance of the model in real-time detection tasks while taking into account computational efficiency and detection accuracy. The module first processes the input X through convolution and divides the result into two parts. The second part is processed through the RepNCSP module and convolution, and finally all the results are merged and output.

[0065] The structures of RepNCSP, RepNBottleneck and RepConvN are as follows Figure 8 As shown. In steel surface defect detection, the RepNCSP module is used to enhance the model's detection capability for targets of different sizes through multi-scale feature extraction. It extracts features at different spatial scales by introducing multiple convolutional layers and different strides, so that the model can better capture the features of small and large targets; RepNCSP uses a non-convex convolution kernel configuration to more flexibly adapt to targets of different shapes and sizes, improving the expressiveness of features; RepNCSP reduces the number of model parameters and improves computational efficiency by reusing the same convolution structure. The module processes the input X through convolution and RepNBottleneck respectively, and merges the results, as shown in Figure 8 As shown in (a);

[0066] RepNBottleneck is used to receive multi-scale feature maps from RepNCSP. Its structure is close to the residual structure. When it uses residual connection and the number of input and output channels is the same, the input X is added to the result of X after passing through RepConvN and Conv for output. Otherwise, only the result of X after passing through RepConvN and Conv is returned, such as Figure 8 as shown in (b);

[0067] RepConvN is a basic convolution module that supports training and deployment states and is used to optimize convolution operations, such as Figure 8 as shown in (c).

[0068] The WIoU loss function is the Wise-IoU loss function, which uses a dynamic non-monotonic focusing mechanism (Focusing Mechanism, FM) to solve the adverse effects of high-quality and low-quality samples on model training. When used for the steel surface defect detection model, the following method is used:

[0069] First, the outlier is defined to describe the quality of the anchor box of the labeled sample, which is defined as:

[0070]

[0071] A small outlier means that the anchor box has high quality. A small gradient gain is assigned to it so that the bounding box regression focuses on the anchor box of normal quality. A smaller gradient gain is assigned to the anchor box with a large outlier to prevent low-quality examples from generating large harmful gradients. A non-monotonic focusing coefficient is constructed using β and applied to WIoU:

[0072]

[0073] Among them, when β = δ, δ makes r = 1. When the outlier degree of the anchor box satisfies β = C, and C is a constant, the anchor box will obtain the highest gradient gain;

[0074] because It is dynamic, and the quality division standard of the anchor box is also dynamic, so that WIoU v3 can make the gradient gain allocation strategy that best suits the current situation at every moment.

[0075] The ASF-YOLO design is part of the ASFRW-YOLO model. The ASFRW-YOLO model is trained with the NEU-DET dataset. The surface defects of steel detected by the model include: cracks, patches, inclusions, pitting, pressed-in scaly metal skin, and scratches.

[0076] When the number of parameters of the ASFRW-YOLO model is not greater than 1.61M, the detection method uses an embedded device as a computing unit to run the ASFRW-YOLO model, and the embedded device includes a smart phone, an industrial camera, or an intelligent monitoring device.

[0077] The deployment of the detection method includes the following steps:

[0078] Step S1, hardware connection: connect the camera and display to the computing unit (such as NVIDIA JetsonNano or Raspberry Pi) to view the detection results in real time; connect the device to the power supply and network at the same time; if the alarm function when a defect is detected is required, connect the LED indicator or buzzer to the interface of the computing unit; Step S2, software environment configuration: install the operating system on the embedded device, install the required deep learning framework PyTorch, ensure that the operation of the ASFRW-YOLO model is supported, and install image processing libraries such as CV for image preprocessing and result visualization;

[0079] Step S3, model deployment: Convert the ASFRW-YOLO model to a format that is compatible with the target hardware and optimize it using TensorRT (for NVIDIA Jetson); then load the ASFRW-YOLO model in the application and initialize it;

[0080] Step S4, data acquisition and processing: setting a camera to collect image data of the steel surface in real time; writing code in the application to perform image preprocessing, including adjusting image size, normalization, etc.;

[0081] Step S5, real-time reasoning: input the preprocessed image into the ASFRW-YOLO model for reasoning to obtain the detection result; draw the detected defect boundary box and category label on the image;

[0082] Step S6, result analysis and feedback: Visualize the test results and display them on the screen, for example Fig.11 As shown, an alarm system (such as an LED indicator light or a buzzer) is triggered according to the detection results to notify the operator, and the detection data is recorded for subsequent analysis and improvement.

[0083] This example uses the NEU-DET dataset from Northeastern University, which includes six common steel surface defects: Cracking (Cr), Patches (Pa), Inclusion (In), Pitted Surface (PS), Rolled-in Scale (RS), and Scratches (Sc). There are 300 images in each category, a total of 1,800 images, and the image size is 200×200. The dataset is provided with annotations that mark the location and category information of the defect images. Since the dataset is relatively small, in order to better train the model, 1,770 images are used as the training set and 30 images are used as the test set. The categories of steel surface defects in the NEU-DET dataset are as follows: Figure 1 shown.

[0084] The original YOLOv5 model in this example is improved based on the YOLOv3 model, and there are four models: YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x. Among them, YOLOv5s is the least complex model, with only 7.23M parameters. Due to the limited number of samples in the NEU-DET dataset, the model may be prone to overfitting during training. Compared with other models, YOLOv5 is relatively lightweight and can converge faster on small datasets, reducing the risk of overfitting. Therefore, the YOLOv5 algorithm network model is used to adapt to this example. The YOLOv5 network structure is as follows: Figure 2 shown.

[0085] In view of the fact that the current steel defect detection model needs to be improved in terms of small target detection capability, real-time detection capability and accuracy, this example improves the YOLOv5 model and proposes a new steel surface defect detection model ASFRW-YOLO. Its network structure is as follows: Figure 3As shown in the figure. First, the design of ASF-YOLO is added to the Neck part, and the SSFF module, TFE module and CPAM attention mechanism module are added. Among them, the SSFF module combines the global or high-level semantic information from multiple image scales to jointly harvest the combined multi-scale features; the TFE module can capture the local detail features of small objects and enhance the model's detection ability for small targets; the CPAM module is used to integrate the information processed by the SSFF module and the TFE module. Secondly, the C3 module in the original model (YOLOv5) is replaced with the RepNCSPELAN4 module to enhance the model's reasoning speed and the ability to aggregate multi-scale features, while making the model have a higher parameter utilization rate. Finally, the WIOU loss function is used to replace the original loss function to effectively improve the model's detection ability, reduce the probability of under-detection and missed detection, and improve the overall accuracy of the model. By making the above changes to the YOLOv5 model, the overall performance of the model in the steel defect detection task is improved.

[0086] Example:

[0087] This example is conducted in an experimental environment. The experimental environment uses the Windows 11 operating system, the CPU is Intel(R) Core(TM) i9-13900HX 2.20GHz, the graphics card is NVIDIA GeForce RTX4060 Laptop with 8GB memory, the deep learning framework is PyTorch 2.2.2, the Python version is 3.8.0, and the CUDA version is 12.7. The specific experimental environment configuration is shown in Table 1.

[0088] Table 1 Experimental environment configuration

[0089]

[0090]

[0091] In addition, in the experimental parameters, the number of epochs is set to 300, the batch size is set to 16, the initial learning rate is set to 0.01, the momentum is set to 0.937, and the optimizer is set to SGD, as shown in Table 2.

[0092] Table 2 Experimental parameter configuration

[0093]

[0094] In order to evaluate the overall performance of the model, the evaluation indicators used in this experiment are average precision (AP), mean average precision (mAP), and parameters. The IOU threshold is set to 0.5. When the intersection of the detection box and the ground truth box is greater than 0.5, the prediction is considered correct. The calculation formula is as follows:

[0095]

[0096]

[0097]

[0098]

[0099] Where P represents precision, R represents recall, n represents the number of predicted categories, TP, FP, and FN represent true positive examples, false positive examples, and false negative examples, respectively.

[0100] In order to verify the feasibility and effectiveness of the proposed improvement strategy, a set of ablation experiments were conducted. The experiment uses YOLOv5s as the baseline model, and the results are shown in Table 3. In the table, ASF, RepNCSPELAN4 and WIoU represent the three improvements proposed in this study. The symbol √ indicates that the improvement point is adopted.

[0101] Table 3 Ablation experiment results

[0102]

[0103] As shown in Table 3, No. 2 only adopts the design of ASF. Compared with the original model YOLOv5s, it improves the mAP value and FPS at the same time, and greatly reduces the number of parameters. This shows that the design of adding ASF can ensure lightweight while improving the model detection accuracy, enhancing the real-time detection capability, and improving the model training efficiency. In No. 3, only the RepNCSPELAN4 module is introduced on the basis of the original model, which also improves the mAP value and reduces the number of parameters, but the FPS value does not increase. This shows that although the RepNCSPELAN4 module is not very helpful for real-time detection, it can improve the model accuracy while lightweighting the model. No. 4 introduces the WIoU loss function alone. Although the addition of the WIoU loss function alone does not show obvious performance changes in the model, it can be seen from the comparison between No. 6 and No. 7 and No. 2 and No. 3 that the model after adding the WIoU loss function has achieved a certain improvement in detection accuracy and has no negative impact on the FPS value. At the same time, it can be seen from the comparison between No. 5 and No. 2 and No. 3 that after the introduction of ASF and RepNCSPELAN4 modules, the detection accuracy of the model can still be improved, and the number of parameters has been greatly reduced, indicating that these two improvements play a very important role in the lightweighting of the model. And the comparison between No. 8 and No. 5 shows that the introduction of WIoU has greatly improved the average detection accuracy of the model, reaching 86.4%, which is about 8 percentage points higher than the original model of No. 1. While being lightweight to the maximum, the FPS value can also meet the requirements of real-time detection, reaching more than 100. In summary, the ASFRW-YOLO model proposed in this example shows the best performance in the ablation experiment. The model greatly improves the average detection accuracy of the model while ensuring the lightweight of the model, and its real-time detection capability can also meet the basic requirements. This shows that the introduction of ASF design, RepNCSPELAN4 module and WIoU loss function can effectively improve the comprehensive performance of the algorithm model, providing performance improvement effects and reference value for the practical application of steel defect detection tasks.

[0104] In order to more clearly demonstrate the performance improvement of the ASFRW-YOLO algorithm model compared with the original model (YOLOv5s), the average detection accuracy of the two models for six types of defects is compared. The results are shown in Table 4. The PR curves of the two models are also shown. Fig. 9 , 10 shown.

[0105] Table 4 Comparison of the detection accuracy of different defects of the benchmark model

[0106]

[0107] It can be seen from Table 4 that the ASFRW-YOLO model proposed in this example has improved detection accuracy for the six defects, among which the defects Cr and In are improved by about 10 percentage points, and the defect Rs is improved by about 16 percentage points.

[0108] The above ablation experiment shows the feasibility of the algorithm proposed in this example. In order to further verify the performance of the improved model, the improved model is compared with the results of 6 mainstream models including Faster R-CNN and YOLO series under the same data set division conditions. The results of detection accuracy and model parameter quantity are shown in Table 5.

[0109] Table 5 Experimental results of different algorithm models

[0110]

[0111]

[0112] From the results in Table 5, it can be seen that the ASFRW-YOLO algorithm model proposed in this example has a significantly higher mAP value than the other five mainstream target detection models, and is more than 10 percentage points higher than the YOLOv7 algorithm model, and is about 4 percentage points higher than the current mainstream YOLOv8. From the perspective of the detection accuracy of the six defects, although the detection accuracy of the previous algorithm models is slightly higher than that of the algorithm model in this example in some steel defects, overall, the detection accuracy of other models on these defects is no more than 5 percentage points higher than that of the model in this example, and the algorithm model in this example has the highest accuracy in the detection of three of the defects, so the algorithm model in this example still has the best comprehensive performance.

[0113] From the perspective of model parameter quantity, although YOLOX, YOLOv5, YOLOv8 and ASFRW-YOLO are all lightweight models, the ASFRW-YOLO model has the least number of parameters, accounting for only 1.61M, and compared with other lightweight models, its model complexity is also greatly reduced, and its mean average precision is also better than other algorithms.

[24] As a two-stage algorithm, it does have some improvement in detection accuracy compared to YOLOX, YOLOv4 and other models, but its algorithm complexity is high, the number of parameters is high and the speed is slow, which makes it difficult to meet the current needs of real-time detection. YOLOX is a high-performance detector formed by using YOLOv3 as the benchmark model and a variety of improved technologies. It performs well in many detection tasks, but compared to the model in this example, it still has a certain gap in accuracy and number of parameters.

[0114] For the three algorithm models of YOLOv4, YOLOv5, and YOLOv7, since the data set in this example is relatively small, the lightweight model is more suitable for the training task in this example. Among them, YOLOv5 is the lightest model, and it can be seen that its detection accuracy is also higher than that of the other two models. YOLOv8 is the most popular and commonly used target detection model, and its model is also a lightweight model, with only 3.15M parameters. Its detection accuracy is also higher than other models, reaching 82.1%. However, the ASFRW-YOLO algorithm model proposed in this example can be more lightweight in terms of parameters, with only 1.61M, and its accuracy is also higher, reaching 86.4%. At the same time, it can be seen from Table 2 that the FPS value of the ASFRW-YOLO algorithm model can still be maintained above 100, which is enough to prove that it still has certain real-time detection capabilities. In summary, the ASFRW-YOLO algorithm proposed in this example performs better than other algorithms in the steel defect detection task. It can improve the detection accuracy while ensuring the lightweight of the model and meet the needs of real-time detection.

Claims

1. A high-precision lightweight steel surface defect detection method based on YOLOv5 can improve the small target detection capability, real-time detection capability and accuracy of the steel surface detection process, and is characterized by: The method first adds ASF-YOLO design to the Neck part based on the original model YOLOv5, and adds SSFF module, TFE module and CPAM attention mechanism module; the SSFF module combines global or high-level semantic information from multiple image scales to jointly harvest the combined multi-scale features of the steel surface; the TFE module is used to capture the local detail features of small objects in the steel surface detection process and enhance the model's detection ability for small targets; the CPAM module is used to integrate the information processed by the SSFF module and the TFE module; Secondly, the C3 module in the original YOLOv5 model is replaced with the RepNCSPELAN4 module to enhance the model’s reasoning speed and ability to aggregate multi-scale features, and to make the model have a higher parameter utilization rate; Finally, the WIoU loss function is used to replace the original loss function to improve the detection ability of the model and reduce the probability of under-detection and missed detection, thereby improving the overall accuracy of the model.

2. According to claim 1, a high-precision lightweight steel surface defect detection method based on the improvement of YOLOv5 is characterized in that: The ASF-YOLO is designed to improve the model's detection capability and accuracy for small targets in steel defect data, and includes a scale sequence feature fusion module SSFF, a triple feature encoding module TFE, and a channel position attention mechanism module CPAM.

3. According to claim 2, a high-precision lightweight steel surface defect detection method based on YOLOv5 improvement is characterized in that: In steel surface defect detection, the SSFF module combines the global semantic information of images at different scales by normalizing, upsampling, and connecting multi-scale features into 3D convolutions to effectively process objects of different sizes, directions, and aspect ratios in the scale space representation of the detection data. Suppose the multi-scale feature maps generated from the model backbone are P3, P4, and P5, which capture the image content in different details or proportions. The feature maps P3, P4, and P5 are first convolved with a series of Gaussian kernels with increasing standard deviations, as shown in the following formula: F σ (i,j)=∑ u ∑ v f(i-u,i-v)×G σ (u,v) (1) Where f represents a two-dimensional 2D feature map, and It is generated by smoothing with a series of convolutions using a 2D Gaussian filter with increasing standard deviation σ; the features of P4 and P5 are then adjusted to the size of P3 level using the nearest neighbor interpolation method, and these feature maps of different scales are stacked horizontally; finally, 3D convolution, 3D batch normalization and SiLU activation function are used to complete the scale sequence feature extraction. The SSFF module structure is shown in Figure 4.

4. According to claim 2, a high-precision lightweight steel surface defect detection method based on YOLOv5 improvement is characterized in that: When detecting surface defects in steel, the TFE module fuses small, medium and large feature maps to capture local detail information of small objects in images of different scales. Before feature encoding, the TFE module first adjusts the number of feature channels to make it consistent with the main scale feature. After the large-size feature map Large is processed by the convolution module, its number of channels is adjusted to 1C. Then, a hybrid structure of maximum pooling and average pooling is used for downsampling to reduce the spatial dimension of the feature and achieve translation invariance, thereby enhancing the network's robustness to spatial changes and translations of the input image. For the small-size feature map Small, a convolution module is used to adjust the number of channels, and then the nearest neighbor interpolation method is used for upsampling to retain local features and prevent the loss of small object feature information. Finally, the three feature maps of large, medium and small sizes with the same dimension are convolved once and then spliced ​​in the channel dimension, as expressed in the following formula: F TFE =Concat(F l ,F m ,F s ) (3) where F TFE Represents the feature map output by the TFE module; F l 、F m and F s Represents large, medium and small size feature maps respectively; F TFE By F l 、F m and F s Cascade generation; F TFE With F m Same resolution and three times the number of channels.

5. According to claim 2, a high-precision lightweight steel surface defect detection method based on YOLOv5 improvement is characterized in that: In steel surface defect detection, the CPAM module is used to integrate feature information from the SSFF and TFE modules to allow the model to adaptively adjust its focus on channels and spatial locations related to small objects at different scales; The architecture of CPAM consists of a channel attention network and a position attention network; it includes input 1 for receiving input from TFE and input 2 for receiving superimposed input from the channel attention network and SSFF; an attention mechanism without dimensionality reduction is introduced to capture cross-channel interactions in an effective way, and local cross-channel interactions are captured by considering each channel and its k nearest neighbors after performing channel global average pooling without dimensionality reduction.

6. The high-precision lightweight steel surface defect detection method based on the improved YOLOv5 according to claim 1 is characterized in that: In the detection of steel surface defects, the RepNCSPELAN4 module is used to reduce the computational complexity and improve the feature extraction effect. In the module structure, RepNCSP is a submodule of RepNCSPELAN4, RepNBottleneck is a submodule of RepNCSP, and RepConvN is a submodule of RepNCSP. RepNCSPELAN4 combines RepNCSP, RepNBottleneck and RepConvN to form an efficient feature extraction unit, which enables the model to extract multi-scale features while maintaining low computational complexity, ensuring the high performance of the model in real-time detection tasks while taking into account computational efficiency and detection accuracy. The module first processes the input X through convolution and divides the result into two parts; the second part is processed through the RepNCSP module and convolution, and finally all the results are merged and output.

7. The high-precision lightweight steel surface defect detection method based on the improved YOLOv5 according to claim 6 is characterized in that: When detecting surface defects in steel, the RepNCSP module is used to enhance the model's detection capability for targets of different sizes through multi-scale feature extraction. It extracts features at different spatial scales by introducing multiple convolutional layers and different strides, so that the model can better capture the features of small and large targets. RepNCSP uses a non-convex convolution kernel configuration to more flexibly adapt to targets of different shapes and sizes and improve the expressiveness of features. RepNCSP reduces the number of model parameters and improves computational efficiency by reusing the same convolution structure. The module processes the input X through convolution and RepNBottleneck respectively, and merges the results. RepNBottleneck is used to receive multi-scale feature maps from RepNCSP. Its structure is close to the residual structure. When it uses residual connection and the number of input and output channels is the same, it adds the input X and the result of X passing through RepConvN and Conv for output. Otherwise, it only returns the result of X passing through RepConvN and Conv. RepConvN is a basic convolution module that supports training and deployment states and is used to optimize convolution operations.

8. The high-precision lightweight steel surface defect detection method based on the improved YOLOv5 according to claim 1 is characterized in that: The WIoU loss function is the Wise-IoU loss function, which uses a dynamic non-monotonic focusing mechanism FM to solve the adverse effects of high-quality and low-quality samples on model training. When used for the steel surface defect detection model, the following method is used: First, the outlier is defined to describe the quality of the anchor box of the labeled sample, which is defined as: A small outlier means that the anchor box has high quality. A small gradient gain is assigned to it so that the bounding box regression focuses on the anchor box of normal quality. A smaller gradient gain is assigned to the anchor box with a large outlier to prevent low-quality examples from generating large harmful gradients. A non-monotonic focusing coefficient is constructed using β and applied to WIoU: Among them, when β = δ, δ makes r = 1. When the outlier degree of the anchor box satisfies β = C, and C is a constant, the anchor box will obtain the highest gradient gain; because It is dynamic, and the quality division standard of the anchor box is also dynamic, so that WIoU v3 can make the gradient gain allocation strategy that best suits the current situation at every moment.

9. The high-precision lightweight steel surface defect detection method based on the improved YOLOv5 according to claim 1 is characterized in that: The ASF-YOLO design is part of the ASFRW-YOLO model. The ASFRW-YOLO model is trained with the NEU-DET dataset. The surface defects of steel detected by the model include: cracks, patches, inclusions, pitting, pressed-in scaly metal skin, and scratches. When the number of parameters of the ASFRW-YOLO model is not greater than 1.61M, the detection method uses an embedded device as a computing unit to run the ASFRW-YOLO model, and the embedded device includes a smart phone, an industrial camera, or an intelligent monitoring device.

10. The high-precision lightweight steel surface defect detection method based on the improved YOLOv5 according to claim 9 is characterized in that: The deployment of the detection method includes the following steps: Step S1, hardware connection: connect the camera and display to the computing unit so as to view the detection results in real time; connect the device to the power supply and network at the same time; if an alarm function is required when a defect is detected, connect the LED indicator or buzzer to the interface of the computing unit; Step S2, software environment configuration: install the operating system on the embedded device, install the required deep learning framework PyTorch, ensure support for the operation of the ASFRW-YOLO model, and install the CV image processing library for image preprocessing and result visualization; Step S3, model deployment: convert the ASFRW-YOLO model into a format suitable for the target hardware and optimize it; Then load the ASFRW-YOLO model in the application and initialize it; Step S4, data acquisition and processing: setting a camera to collect image data of the steel surface in real time; writing code in the application to perform image preprocessing, including adjusting image size and normalization; Step S5, real-time reasoning: input the preprocessed image into the ASFRW-YOLO model for reasoning to obtain the detection result; draw the detected defect boundary box and category label on the image; Step S6, result analysis and feedback: visualize and display the test results on the screen, trigger an alarm system based on the test results to notify the operator, and record the test data for subsequent analysis and improvement.

Images