Steel surface defect detection method based on lightweight DYOLO-Steel network framework

Abstract

The invention discloses a steel surface defect detection method based on a lightweight DYOLO-Steel network framework, and the method comprises the following steps: 1) collecting a metal surface defect real sample, and making a training data set; 2) using a lightweight network Faster Net as a backbone network to extract feature maps of different levels; 3) designing a detection head as a lightweight GDetect module, and compressing the volume of the model; 4), a lightweight feature extraction module C2fEMSC is used in the neck network; 5), designing an MDown module in the down-sampling module, and improving the transmission efficiency of the feature information; 6) enhancing the anti-interference capability of the model to a complex background by using a contextual information enhancement module CEM; and 7) guiding the self-distillation technology to continue training the self-distillation technology. On the basis of DYOLO, through the improved steps, high-quality and diversified metal surface defect samples can be stably and efficiently generated.

Description

Technical field

[0001] The present invention discloses a lightweight steel surface defect automatic detection method, specifically a steel surface defect detection method based on a lightweight DYOLO-Steel network framework, belonging to the field of computer vision. [Background Technology]

[0002] Strip steel, a core product of the steel industry, is subject to a variety of surface defects, such as cracks, pores, and pits, due to factors such as raw materials, processes, and the environment. These defects can affect steel quality, making surface defect detection crucial.

[0003] Current deep learning-based computer vision methods offer powerful capabilities for steel surface defect detection. However, in practical applications, accurate identification of specific products with unique appearances and detection environments requires significant storage and computing resources. For many small and medium-sized enterprises, these requirements are difficult to meet due to economic constraints, limiting the practical application of current detection methods. Therefore, the use of lightweight detection methods presents a promising alternative. While lightweight detection methods offer the advantage of lowering costs, ensuring accurate monitoring remains a challenge for lightweight detection models.

[0004] The YOLOv8n network model is currently a common method for industrial object detection in general scenarios. However, the task of detecting steel surface defects is complex, and the model's lightweightness needs to be improved. Current detection models based on this model often suffer from the excessive number of parameters that the backbone network must learn. While some existing small networks offer advantages in terms of resource efficiency, their detection accuracy often falls short of practical application requirements.

[0005] Therefore, in order to solve the above problems, some lightweight methods of small networks are used to optimize and improve the lightweight aspects of the YOLOv8n network model, which can effectively overcome the above difficulties. [Summary of the invention]

[0006] The purpose of the present invention is to disclose a steel surface defect detection method based on a lightweight DYOLO-Steel network framework. Based on the optimized baseline model YOLOv8n, the method introduces a lightweight backbone network, lightweight detection, lightweight feature extraction modules, lightweight downsampling operators, and context information enhancement modules, and uses a self-distillation method for continuous training to obtain a lightweight model that consumes less computing and storage resources. To achieve the above objectives, the technical solution adopted by the present invention is: a steel surface defect detection method based on a lightweight DYOLO-Steel network framework, the method comprising the following steps:

[0007] 1) Collect real samples of strip surface defects and create a training data set;

[0008] 2) Using the lightweight network FasterNet as the backbone network to extract feature maps at different levels;

[0009] 3) Design the detection head as a lightweight GDetect module to compress the model size;

[0010] 4) Using the lightweight feature extraction module C2f_EMSC in the neck network;

[0011] 5) Design the MDown module in the downsampling module to improve the efficiency of feature information transmission;

[0012] 6) Use the context information enhancement module CEM to enhance the model's anti-interference ability against complex backgrounds

[0013] 7) Using self-distillation technology to guide the continued training of the model. Based on DYOLO, the present invention can stably and efficiently generate high-quality and diverse metal surface defect samples through the above improvement steps.

[0014] The steel surface defect detection method based on the lightweight DYOLO-Steel network framework of the present invention is further as follows: in the step 1), the real samples of strip surface defects collected include defect images of cracks, inclusions, plaques, pitting surfaces, rolling scales and scratches, which are scale-normalized to obtain a real defect sample data set with the same resolution.

[0015] The steel surface defect detection method based on the lightweight DYOLO-Steel network framework of the present invention is further as follows: in the step 2), the input RGB image is embedded encoded, and then feature extraction is performed through four FasterBlocks as partial convolution operation units to obtain a feature image.

[0016] The steel surface defect detection method based on the lightweight DYOLO-Steel network framework of the present invention is further as follows: in the step 3), the input channels are evenly divided into several groups through GConv, and convolution operations are performed only on the channels within each group, and finally the outputs of each group are merged to form the final output.

[0017] The steel surface defect detection method based on the lightweight DYOLO-Steel network framework of the present invention further comprises: in step 4), half of the channels are first inactive using an EMSC with a depthwise separable convolution mechanism. The remaining channels are then convolved with different convolution kernels to obtain features at different scales.

[0018] The steel surface defect detection method based on the lightweight DYOLO-Steel network framework of the present invention further comprises the following steps:

[0019] 5-1), first divide the number of channels of the feature map into two groups evenly;

[0020] 5-2), the first set of feature map channels are processed by maximum pooling, and then the features are fused by stationary convolution;

[0021] 5-3), the second set of feature map channels directly use 3×3 convolution kernel for strided convolution;

[0022] 5-4), and finally merge the two sets of processed feature maps to complete downsampling while reducing the amount of computation.

[0023] The steel surface defect detection method based on the lightweight DYOLO-Steel network framework of the present invention further comprises the following steps:

[0024] 6-1), use dilated convolution with a kernel size of 3×3 at dilation rates of 1, 3, and 5;

[0025] 6-2), extract features through point-by-point convolution;

[0026] 6-3), the three processed feature maps are spliced ​​and fused.

[0027] The steel surface defect detection method based on the lightweight DYOLO-Steel network framework of the present invention is further as follows: the specific operations of step 7) are:

[0028] 7-1), the type probability distribution is obtained by predicting the teacher model of the classification branch, which is recorded as The type probability distribution is obtained by predicting the student model of the classification branch, denoted as

[0029] 7-2), the type probability distribution is obtained by predicting the teacher model of the positioning branch, which is recorded as The type probability distribution is obtained by predicting the student model of the positioning branch, denoted as

[0030] 7-3), the distillation loss can be expressed as:

[0031]

[0032] Among them, KL represents the process of realizing knowledge transfer by minimizing the Kullback-Leibler divergence between the teacher model and the student model.

[0033] 7-4), the overall loss is obtained by using the distillation loss and the loss of model prediction, data and labels, and the expression is:

[0034] L total =L det +αL KD

[0035] Among them, L det is the detection loss of data annotation and model prediction;

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] 1. The steel surface defect detection method based on the lightweight DYOLO-Steel network framework disclosed in this paper can use fewer computationally intensive modules in scenarios requiring less storage and computing resources. This significantly reduces computational burden and memory consumption, lightweighting the backbone network and improving model inference speed. It is suitable for environments with limited computing and memory resources and has important practical implications for real-time defect detection tasks.

[0038] 2. The steel surface defect detection method based on the lightweight DYOLO-Steel network framework of the present invention introduces a self-distillation training framework and utilizes knowledge transfer and dynamic weight adjustment to further improve the detection accuracy and robustness of the model.

Brief Description of the Drawings

[0039] In order to more clearly illustrate the purpose and technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments, wherein:

[0040] Figure 1 This is a schematic diagram of the overall network framework flow of an embodiment of a steel surface defect detection method based on a lightweight DYOLO-Steel network framework disclosed in the present invention;

[0041] Figure 2 This is a diagram showing the internal structure of FasterNet Block, an embodiment of a steel surface defect detection method based on a lightweight DYOLO-Steel network framework disclosed in the present invention;

[0042] Figure 3 This is a schematic diagram of the EMSC operation process of an embodiment of the steel surface defect detection method based on the lightweight DYOLO-Steel network framework disclosed in the present invention.

Specific implementation method

[0043] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0044] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] Example 1

[0046] like Figure 1 As shown, the present invention provides a steel surface defect detection method based on a lightweight DYOLO-Steel network framework, comprising the following steps:

[0047] 1) The collected real samples of strip surface defects include defect images of cracks, inclusions, plaques, pitting surfaces, rolling scales, and scratches. They are scale-normalized to obtain a real defect sample dataset with the same resolution.

[0048] 2) Use FasterNet as the backbone network for feature extraction to reduce computational complexity. The input RGB image is embedded and encoded, and then four FasterBlocks are used as partial convolution units for feature extraction to obtain a feature image. Figure 2 As shown in the figure, FasterBlock uses PConv internally, selecting the first or last continuous channels as samples for convolution operation. The computational cost of PConv is calculated as follows:

[0049]

[0050] Where h, w, c are the height, width and number of channels of the input and output feature maps. k is the size of the convolution kernel.

[0051] 3) After the last FasterBlock process is completed, context information enhancement processing (CEM) is performed, which specifically includes the following steps:

[0052] 3-1), use dilated convolution with a kernel size of 3×3 at dilation rates of 1, 3, and 5;

[0053] 3-2), extract features through point-by-point convolution;

[0054] 3-3), the three processed feature maps are spliced ​​and fused.

[0055] 4) Through the neck network Neck, the backbone network feature map is downsampled (MDown), spliced, and multi-scale feature fused (C2f_EMSC).

[0056] 5) Using MDown to downsample the feature map extracted by FasterNet includes the following steps:

[0057] 5-1), first divide the number of channels of the feature map into two groups evenly;

[0058] 5-2), the first set of feature map channels are processed by maximum pooling, and then the features are fused by stationary convolution;

[0059] 5-3), the second set of feature map channels directly use 3×3 convolution kernel for strided convolution;

[0060] 5-4), and finally merge the two sets of processed feature maps to complete downsampling while reducing the amount of computation.

[0061] 6) The feature map extracted by FasterNet is processed using EMSC multi-scale feature fusion. Figure 3 As shown in Figure 1, half of the channels are initially inactive using the EMSC with a depthwise separable convolution mechanism. The remaining channels are then convolved with different convolution kernels to obtain features at different scales. The computational effort is expressed as follows:

[0062]

[0063] Where h, w, c are the height, width and number of channels of the input and output feature maps. k is the size of the convolution kernel.

[0064] 7) Use the GDetect module to detect the fused feature map. GConv evenly divides the input channels into several groups, performs convolution operations only on the channels within each group, and finally merges the outputs of each group to form the final output.

[0065] 8) Using the self-distillation training framework, define the loss function for the model. The specific operations are as follows: 8-1) The teacher model of the classification branch predicts the type probability distribution, which is recorded as The type probability distribution is obtained by predicting the student model of the classification branch, denoted as

[0066] 8-2), the type probability distribution is obtained by predicting the teacher model of the positioning branch, which is recorded as The type probability distribution is obtained by predicting the student model of the positioning branch, denoted as

[0067] 8-3), the distillation loss can be expressed as:

[0068]

[0069] Among them, KL represents the process of realizing knowledge transfer by minimizing the Kullback-Leibler divergence between the teacher model and the student model.

[0070] 8-4), the overall loss is obtained by using the distillation loss and the loss of model prediction, data and labels, and the expression is:

[0071] L total =L det +αL KD

[0072] Among them, L det is the detection loss of data annotation and model prediction.

Claims

1. A steel surface defect detection method based on the lightweight DYOLO-Steel network framework, characterized by: The steps include: 1) Collect real samples of metal surface defects and create a training data set; 2) Using the lightweight network FasterNet as the backbone network to extract feature maps at different levels; 3) Design the detection head as a lightweight GDetect module to compress the model size; 4) Using the lightweight feature extraction module C2f_EMSC in the neck network; 5) Design the MDown module in the downsampling module to improve the efficiency of feature information transmission; 6) Use the context information enhancement module CEM to enhance the model's anti-interference ability against complex backgrounds 7) Using self-distillation technology to guide the continued training of the model. Based on DYOLO, the present invention can stably and efficiently generate high-quality and diverse metal surface defect samples through the above improvement steps.

2. The steel surface defect detection method based on the lightweight DYOLO-Steel network framework according to claim 1 further comprises: in the step 1), the collected real samples of strip surface defects include defect images of cracks, inclusions, plaques, pitting surfaces, rolling scale, and scratches, which are scale-normalized to obtain a real defect sample dataset with the same resolution.

3. The steel surface defect detection method based on the lightweight DYOLO-Steel network framework according to claim 1 further comprises: in the step 2), the input RGB image is embedded encoded, and then feature extraction is performed using four FasterBlocks as partial convolution operation units to obtain a feature image.

4. The steel surface defect detection method based on the lightweight DYOLO-Steel network framework according to claim 1 further comprises: in step 3), the input channels are evenly divided into several groups through GConv, convolution operations are performed only on the channels within each group, and finally the outputs of each group are merged to form a final output.

5. The steel surface defect detection method based on the lightweight DYOLO-Steel network framework according to claim 1 further comprises: in step 4), half of the channels are first placed in an inactive state using an EMSC with a depthwise separable convolution mechanism. The remaining channels are then convolved with different convolution kernels to obtain features at different scales.

6. The steel surface defect detection method based on the lightweight DYOLO-Steel network framework according to claim 1 further comprises: said step 5) specifically comprising the following steps: 5-1), first divide the number of channels of the feature map into two groups evenly; 5-2), the first set of feature map channels are processed by maximum pooling, and then the features are fused by stationary convolution; 5-3), the second set of feature map channels directly use 3×3 convolution kernel for strided convolution; 5-4), and finally merge the two sets of processed feature maps to complete downsampling while reducing the amount of computation.

7. The steel surface defect detection method based on the lightweight DYOLO-Steel network framework according to claim 1 further comprises: said step 6) specifically comprising the following steps: 6-1), use dilated convolution with a kernel size of 3×3 at dilation rates of 1, 3, and 5; 6-2), extract features through point-by-point convolution; 6-3), the three processed feature maps are spliced ​​and fused.

8. The steel surface defect detection method based on the lightweight DYOLO-Steel network framework according to claim 1 further comprises: the specific operations of step 7) are: 7-1), the type probability distribution is obtained by predicting the teacher model of the classification branch, which is recorded as The type probability distribution is obtained by predicting the student model of the classification branch, denoted as 7-2), the type probability distribution is obtained by predicting the teacher model of the positioning branch, which is recorded as The type probability distribution is obtained by predicting the student model of the positioning branch, denoted as 7-3), the distillation loss can be expressed as: in, KL represents the process of achieving knowledge transfer by minimizing the Kullback-Leibler divergence between the teacher model and the student model. 7-4), the overall loss is obtained by using the distillation loss and the loss of model prediction, data and labels, and the expression is: L total =L det +αL KD Among them, L det is the detection loss of data annotation and model prediction.

Images