Strip steel surface defect detection method based on deep learning

Through deep learning methods, the YOLOv8 network structure and multi-scale fusion network are improved, combined with the idea of ​​task alignment, the existing strip surface defect detection technology is solved, and efficient and accurate defect detection is achieved, suitable for high-speed Production line reduces operating costs.

CN119048432APending Publication Date: 2024-11-29HAIYAN ZHONGDA METAL ELECTRONIC MATERIAL CO LTD
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411006527.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The existing strip surface defect detection technology has problems such as low efficiency, poor accuracy and consistency, high labor intensity, strong subjectivity, difficulty in detecting subtle defects and strong environmental dependence. When traditional machine learning methods deal with large-scale and high-dimensional data, they are used to deal with large-scale and high-dimensional data. Poor performance and limited application of deep learning models in industrial environments.

Method used

Deep learning-based strip surface defect detection method, through data acquisition and preprocessing, the improved YOLOv8 network structure is used for feature extraction and detection, combined with the idea of ​​multi-scale fusion network and task alignment, and positioning loss, classification loss and confidence are adopted. Losses are trained and post-processing operations such as threshold filtering, non-maximum suppression and bounding box adjustments are performed to improve detection accuracy and efficiency.

Benefits of technology

It realizes high detection accuracy and high efficiency of strip surface defect detection, reduces calculation costs, adapts to complex production environments and diversified defect types, reduces operation and maintenance costs, and is suitable for real-time inspection of high-speed production lines.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048432A_ABST
    Figure CN119048432A_ABST
Patent Text Reader

Abstract

The invention discloses a strip steel surface defect detection method based on deep learning. S1, data acquisition and preprocessing; defect images are collected and labeled through data acquisition device design and data labeling; s2, a deep learning model; training data by using a deep neural network, and outputting a defect detection result; s3, obtaining a loss function; the invention relates to the technical field of strip steel surface defect detection. According to the strip steel surface defect detection method based on deep learning, high detection precision is achieved, the improved YOLOv8 network can accurately identify and classify various strip steel surface defects, the detection precision is remarkably improved, the missing detection rate and the false detection rate are reduced, the product quality is ensured, and high detection efficiency is achieved; the detection speed is high, the real-time detection requirement of a high-speed production line can be met, and production stagnation and bottlenecks are reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of strip steel surface defect detection, and specifically to a strip steel surface defect detection method based on deep learning. Background Art

[0002] Steel is one of the most common metal materials in our daily life. It is widely used and is a perfect material in many fields. Steel is widely used in civil engineering infrastructure, aerospace, shipbuilding, automobiles, machinery manufacturing and the manufacture of various household tools. However, the production process of strip steel involves many factors such as raw materials, production processes, equipment conditions, environmental conditions, etc., which will cause surface defects such as scratches, peeling, pits and notches on the surface of the strip steel. These defects directly affect the quality of industrial products and infrastructure construction. Therefore, it is particularly important to control the quality of the produced steel, which is the first guarantee of product qualification.

[0003] Early strip surface defect detection mainly used manual visual inspection, which relied on experienced workers to identify defects by visually observing the strip surface. It was usually carried out under good lighting conditions, and sometimes the surface needed to be cleaned to improve visibility. Workers would refer to standard samples or defect comparison charts to judge and classify defects. Its advantages are high flexibility, instant feedback and low cost. It is suitable for various types and shapes of strips, and is especially effective in small batch production and quality control.

[0004] In addition to manual visual inspection, traditional surface inspection technologies also include magnetic particle inspection, liquid penetrant inspection and eddy current inspection. Magnetic particle inspection uses the principle of electromagnetic induction to magnetize ferromagnetic materials and then sprinkle fine magnetic powder on their surface. These magnetic powders will gather at cracks or other defects on the surface of the material in the magnetic field. Because the magnetic field at the defective part will be distorted, the magnetic lines of force will leak out, causing the magnetic powder to gather at the defect to form visible signs. The inspection process includes four steps: magnetization, powder sprinkling, observation and demagnetization. Liquid penetrant inspection is to use capillary action to transfer penetrant liquid (usually colored or Fluorescent liquid) penetrates into the defects on the surface of the material, then wipes off the excess penetrant on the surface, applies a developer, and allows the penetrant to seep out of the defect to reveal the location and shape of the defect. The detection steps include pre-cleaning, applying penetrant, removing excess penetrant, developing and observing. Eddy current detection generates eddy currents (circulating currents) by applying an alternating electromagnetic field near the surface of the workpiece being inspected. When the eddy current flows inside the material, if it encounters a defect, it will cause changes in the intensity and distribution of the eddy current. The location and nature of the defect are identified by detecting these changes. Usually, a probe is used to scan the surface of the workpiece for detection.

[0005] With the development of computer technology, computer vision has begun to be applied to defect detection on the surface of strip steel, which is mainly divided into traditional machine learning methods and deep learning methods. In the detection of defects on the surface of strip steel, traditional machine learning methods use feature engineering and classification algorithms to identify defects. First, features are extracted from the image of the strip surface. These features may include color, texture, shape, edge and other information. Common feature extraction methods include gray-level co-occurrence matrix, local binary pattern and edge detection. Then, these features are used to construct feature vectors and input them into the classifier for training and prediction. Common classification algorithms include support vector machines (SVM), decision trees, random forests and K nearest neighbors (KNN). By training the classifier, the system can learn the characteristic patterns of different types of defects, and then automatically identify and classify defects in new images. This method relies on the quality of feature engineering and the performance of the classifier and is suitable for relatively simple and stable detection tasks.

[0006] Manual visual inspection has disadvantages such as low efficiency, poor accuracy and consistency, high labor intensity, strong subjectivity, high training costs, difficulty in detecting subtle defects and strong environmental dependence. These limitations make it difficult to perform in high-speed, mass production and inspection tasks requiring high standardization, and it is gradually being replaced by automated and intelligent inspection technologies.

[0007] Traditional technologies for strip surface defect detection include magnetic particle testing, liquid penetrant testing and eddy current testing. Magnetic particle testing: can only be used for ferromagnetic materials, ineffective for non-ferromagnetic materials; insensitive to deep defect detection; requires demagnetization, affecting detection efficiency; greatly affected by environmental conditions, such as humidity and dust; Liquid penetrant testing: can only detect surface opening defects, ineffective for internal defects; the detection process is cumbersome, requires multiple steps, and takes a long time; penetrants and developers may be harmful to the environment and operators; poor results on rough or porous surfaces; Eddy current testing: only suitable for conductive materials, ineffective for non-conductive materials; difficult to detect workpieces with complex shapes and large thickness variations, and complex results to interpret; requires high-precision equipment and professional operators, with high initial costs; has requirements for the electromagnetic properties of materials, and limited detection depth.

[0008] Traditional machine learning relies on manual design and feature extraction, which requires expertise and a large number of experiments to determine the optimal feature set, resulting in a complex and time-consuming feature selection process. In addition, traditional methods perform poorly when processing large-scale and high-dimensional data, and are unable to automatically learn complex feature relationships from the data, resulting in relatively low detection accuracy and robustness.

[0009] The training and inference of existing deep learning technologies require high-performance computing resources such as GPUs and TPUs, which may be difficult to configure and maintain in some industrial environments. The generalization ability of the model is also a problem. When the model processes new data with a different distribution from the training data, its performance may drop significantly. Summary of the invention

[0010] 1. Technical issues to be resolved

[0011] In view of the shortcomings of the prior art, the present invention provides a strip surface defect detection method based on deep learning, which solves the problems raised in the above background technology.

[0012] (II) Technical solution

[0013] To achieve the above objectives, the present invention is implemented through the following technical solutions: a strip surface defect detection method based on deep learning, the detection method specifically includes the following steps;

[0014] S1. Data collection and preprocessing: Collect defect images and annotate defect images through data collection device design and data annotation;

[0015] S2, deep learning model; use deep neural network to train data and output defect detection results;

[0016] S3, loss function; positioning loss, classification loss and confidence loss,

[0017] S4, post-processing operations: The deep model generates a large number of prediction boxes during the inference process, each of which contains the category probability, confidence, and bounding box coordinates. In order to extract meaningful detection results, a series of post-processing steps are required, including threshold filtering, non-maximum suppression, bounding box adjustment, and multi-classification processing.

[0018] Preferably, in S1, a DALSA linear array scanning camera is used to continuously capture the surface image of the steel strip at high speed and high resolution. When the steel strip passes through the detection area of ​​the camera and the light source, the linear array camera will continuously capture images at a high frame rate, and the image data is transmitted to a computer via a high-speed interface GigE for storage and processing.

[0019] Preferably, in S1, the data collected by the data acquisition device are classified and stored according to the defect category, and the collected images are manually annotated using an annotation tool. During the annotation process, the location and type of each defect need to be identified, and a corresponding annotation file is generated for deep learning model training.

[0020] Preferably, the overall structure of the deep model in S2 is a network structure improved based on YOLOv8, which includes the following steps:

[0021] A1, feature extraction network;

[0022] A2, multi-scale fusion network;

[0023] A3, model detection head;

[0024] The feature extraction network of the deep learning model in A1 adopts the CSPDarknet53 model architecture, which has 10 layers in total.

[0025] The feature extraction network in A2 outputs three different scale features from its second layer C2f layer, third layer C2f layer and PASPP layer, and the corresponding three scale features are defined as large-scale feature FL, medium-scale feature FM and small-scale feature FS;

[0026] The model detection head in A3 is the TDADH detection head, which uses the task alignment idea in TOOD. In the process of decoupling, different task branches can better learn the specific features of their respective tasks and improve the model detection accuracy.

[0027] Preferably, the positioning loss in S3 is used to measure the difference between the predicted box and the true box, and is measured using IoU and its improved version CIOU. The classification loss is used to measure the difference between the category of the target contained in each predicted box and the true category. Generally, cross entropy loss or binary cross entropy loss is used to calculate the accuracy of category prediction. The confidence loss is used to measure the difference between the confidence of whether the predicted box contains an object and the actual situation, and is usually calculated using cross entropy loss or binary cross entropy loss.

[0028] Preferably, the loss function benchmark function formula is:

[0029] Loss = λ coord Localization_Loss+λ cls Classification_Loss+λ conf Confidence_Loss;

[0030] Among them, λ coord , cls and λ conf is the weight coefficient used to balance the impact of different losses.

[0031] Preferably, the processing method in S4 includes the following steps:

[0032] B1, threshold filtering;

[0033] B2, non-maximum suppression;

[0034] B3, bounding box adjustment;

[0035] B4. Multi-classification processing.

[0036] Preferably, the threshold filtering performs preliminary filtering on the prediction boxes output by the model, each prediction box contains a confidence score, indicating the possibility that the box contains the target, and by setting a confidence threshold, only the prediction boxes with confidence higher than the threshold are retained, and invalid boxes with low confidence are removed;

[0037] The non-maximum suppression sorts the remaining prediction boxes from high to low according to the confidence, selects the box with the highest confidence from the sorted list as the current box, calculates the IoU between the current box and all the remaining boxes, sets an IoU threshold, and deletes the boxes with IoU greater than the threshold from the list, because these boxes are considered to overlap too much with the current box and are redundant boxes. The IoU threshold needs to be adjusted according to the specific application scenario, and is usually set to 0.5-0.6 to find the best balance between detection accuracy and recall rate.

[0038] Preferably, after the bounding box adjustment is processed by non-maximum suppression, the retained prediction box needs to be further adjusted to improve the positioning accuracy. The prediction box output by the deep learning method is usually expressed in the form of relative coordinates and needs to be converted into absolute coordinates in the image;

[0039] In the multi-classification processing, each prediction box may contain the probability of multiple defect types. It is necessary to set a category probability threshold, only retain categories with probabilities higher than the threshold, and select the category with the highest probability as the final category.

[0040] (III) Beneficial effects

[0041] The present invention provides a strip steel surface defect detection method based on deep learning. It has the following beneficial effects:

[0042] (1) This strip surface defect detection method based on deep learning achieves high detection accuracy: the improved YOLOv8 network can accurately identify and classify various strip surface defects, significantly improve detection accuracy, reduce missed detection and false detection rates, and ensure product quality.

[0043] (2) The deep learning-based strip surface defect detection method achieves high detection efficiency: the optimized algorithm has a high detection speed while maintaining high accuracy, which can meet the real-time detection needs of high-speed production lines and reduce production stagnation and bottlenecks.

[0044] (3) This deep learning-based strip surface defect detection method achieves algorithm scale optimization: the improved YOLOv8 network is optimized in terms of computing resources and model size, reducing computing costs and making it more suitable for deployment and operation in actual industrial environments.

[0045] (4) The deep learning-based strip surface defect detection method has strong robustness: the improved algorithm shows stronger robustness and wider adaptability when dealing with various complex production environments and diverse defect types.

[0046] (5) The deep learning-based strip surface defect detection method achieves scalability and versatility: The patented algorithm has good scalability and can be further optimized and adjusted according to actual needs to adapt to other similar industrial inspection tasks.

[0047] (6) This strip surface defect detection method based on deep learning reduces operating costs: by optimizing the algorithm structure and improving detection efficiency, it reduces dependence on high-performance computing resources and reduces overall operating and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of the working principle of the present invention;

[0049] Figure 2 It is a schematic diagram of the principle scheme in the system of the present invention;

[0050] Figure 3 A schematic diagram of a depth model in the system of the present invention;

[0051] Figure 4 Schematic diagram of the PASPP framework structure in the system of the present invention

[0052] Figure 5 It is a schematic diagram of the structure of the SPPF module and the CPCA module in the system of the present invention;

[0053] Figure 6 This is a schematic diagram of the MFE module architecture structure in the system of the present invention;

[0054] Figure 7 This is a schematic diagram of the TDADH module architecture structure in the system of the present invention;

[0055] Figure 8 Schematic diagram of an example of detection results in the system of the present invention. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] See also Figure 1-8The embodiment of the present invention provides a technical solution: a strip surface defect detection method based on deep learning includes collecting defect images using a data acquisition device, annotating the defect images, using a deep neural network to train data, and outputting defect detection results. The solution is as follows: Figure 1 and Figure 2 As shown, the specific steps are:

[0058] 1. Data collection and preprocessing;

[0059] Design of data acquisition device; DALSA line scan camera can continuously capture the surface image of strip at high speed and high resolution, which is very suitable for real-time detection on the production line. DALSA camera ensures image quality with its high sensitivity and low noise characteristics. Coaxial red light source: Coaxial red light source provides uniform and stable lighting, effectively eliminates reflections and shadows, ensures the clarity and contrast of defects in the image, and places the coaxial red light source on the running strip. The coaxial red light source is installed coaxially with the line scan camera to ensure that the light source is consistent with the camera's visual axis, providing the best lighting effect. When the strip passes through the detection area of ​​the camera and the light source, the line scan camera will continuously capture images at a high frame rate, and the image data is transmitted to the computer through the high-speed interface GigE for storage and processing.

[0060] Data labeling: The data collected by the data acquisition device is classified and stored according to the defect category, and the collected images are manually labeled using labeling tools (such as Labelimg). During the labeling process, the location and type of each defect need to be identified, and the corresponding labeling file (TXT format) is generated during the deep learning model training.

[0061] 2. Deep learning model;

[0062] The overall structure of the deep model is based on the improved network structure of YOLOv8 (You Only Look Once version 8), such as Figure 3 , (in the module, k is the convolution kernel size, s is the convolution step size, n is the number of repetitions of the bottleneck structure of the C2f module, and True indicates the residual connection in the bottleneck structure);

[0063] Feature extraction network: The feature extraction network of the deep learning model adopts the CSPDarknet53 model architecture. The feature extraction network has a total of 10 layers, and its structure is as follows: the first Conv layer → the second Conv layer → the first C2f layer → the third Conv layer → the second C2f layer → the fourth Conv layer → the third C2f layer → the fifth Conv layer → the fourth C2f layer → the PASPP layer.

[0064] The output channels of the Conv layers from the first to the fifth layers are 16, 32, 64, 128, and 256, respectively. The convolution kernels of all Conv layers are , and the step size is 2. Convolution operations, batch normalization operations, and SiLU activation processing are performed in the Conv layers. When the step size in the Conv layer is 2, it is a downsampling operation, and the width and height of the image will be half of the input image resolution.

[0065] The bottleneck structure in C2f is two consecutive Conv layers with a convolution kernel of and a step size of 1. Finally, the input features are residually connected with the output features of the two Conv layers (when the residual parameter of the C2f layer is True). The structure of the C2f layer is a Conv layer with a convolution kernel of and a step size of 1. The output of the Conv layer is then divided into two parts through Split. A part of the features after the division passes through n bottleneck structures, and the two parts obtained by Split are concatenated with the features obtained from each bottleneck structure through Concat. The concatenated features pass through the last Conv layer with a convolution kernel of and a step size of 1 to output the result of the C2f layer. The resolution and number of channels of the feature map output by the C2f layer are consistent with the input features, that is, the output features of the Conv layer on the previous layer of C2f.

[0066] The PASPP (Perceptual Attention Spatial PyramidPooling) layer at the last layer of the feature extraction network is the perception attention module. The PASPP structure is as follows: Figure 4 The PASPP module divides the output features of the previous C2f layer into two branches. The first branch uses SPPF for multi-scale pooling operations, and the second branch uses the CPCA attention mechanism. The structure of the SPPF module and the CPCA module is as follows: Figure 5 The features of the second branch after being processed by the CPCA attention mechanism are subjected to Sigmoid operation to obtain a weight feature between 0 and 1, which reflects the activation degree of the input feature after the CPCA attention mechanism. The features with values ​​of 0 to 1 are multiplied element by element with the output results of the SPPF structure of the first branch, which helps the network learn better feature representation. The multiplication result is added to the first branch to enhance the perception ability of the features in the multi-scale pooling branch.

[0067] The SPPF module first passes through a Conv operation with a convolution kernel of 1 and a step size of 1. The Conv output is divided into two branches, one of which is output to three Maxpool operations with a pooling operation size of 1 and a step size of 1. The initial Conv output is concatenated with the three Maxpool operation outputs using Concat, and finally the concatenated result is output through a Conv output with a convolution kernel of 1 and a step size of 1. The output feature resolution is consistent with the output of the fourth C2f layer.

[0068] The CPCA attention mechanism is mainly divided into channel attention part and spatial attention part. For channel attention, the input features are respectively extracted from the feature map through average pooling and maximum pooling operations. This process generates two independent channel attention vectors, i.e., feature maps with a length and width of 1 and the same number of channels as the input channels. Then, these vectors are input into the shared multi-layer perceptron Shared MLP. The output of the shared MLP is combined by element-by-element summation to obtain the channel attention map. For spatial attention, in order to avoid forcing the consistency of the spatial attention map on each channel, deep convolution is used to capture the spatial information between features to preserve the relationship between channels. Subsequently, a multi-scale structure is used to enhance the ability of convolution operations to capture spatial relationships. The resulting spatial attention map is subjected to channel mixing operations by the 1×1 convolution at the tail to generate a more refined attention map.

[0069] Multi-scale fusion network; the feature extraction network outputs three different scale features from its second layer C2f layer, third layer C2f layer and PASPP layer, and the corresponding three scale features are defined as large-scale features FL, medium-scale features FM and small-scale features FS. The large-scale features correspond to feature maps with larger resolution but fewer channels, while the small-scale features have smaller resolution and larger channels. The large-scale features contain rich spatial information suitable for positioning, while the small-scale features contain rich semantic information suitable for classification. The feature multi-scale fusion network will fuse features of different scales into one scale, and use the rich information of various scales to enhance the model's feature extraction capability for defect images, thereby improving the model's detection accuracy.

[0070] Multi-scale fusion uses the MFE structure to fuse the three scales into the intermediate scale. The MFE module is as follows: Figure 6 ,The MFE module is divided into two parts. The first part is the FAB module, which is ,mainly used for feature fusion, and the second part is the IRB module, which ,mainly strengthens the features after feature fusion.

[0071] In the FAB part of the figure, the L, M and S features correspond to the input large-scale feature FL, medium-scale feature FM and small-scale feature FS. For the large-scale feature, the number of channels is 0.5C and the resolution is 2R. Downsampling is performed to reduce the scale of the feature map. First, the Avgpool operation with a pooling operation size and a step size of 1 is performed, and then the output is divided into two parts. The number of channels of the two parts is half of the number of channels of the Avgpool output, and the scale size is the same. The first part is downsampled by the Maxpool operation with a pooling operation size and a step size of 2, and then the Conv operation with a convolution kernel of , a step size of 1 is performed to output the feature information of the first part. The second part is subjected to a Conv operation with a convolution kernel of , a step size of 2, and the feature information of the second part is output. The output is concat spliced ​​to obtain feature information with a channel number of 0.5C and a resolution of 1R. For medium-scale features, the channel number is 1C and the resolution is 1R. Only after convolution verification and a Conv operation with a step size of 1, the channel number is output as 0.5C, and the resolution remains unchanged. For small-scale features, the channel number is 2C and the resolution is 0.5R. First, the feature resolution is doubled to 1R through an upsampling operation of the nearest neighbor interpolation. Subsequently, a convolution kernel is connected with a Conv operation with a step size of 1, and the output channel number is 0.5C and the resolution is 1R. At the end of the FAB module, the processed large-scale features, medium-scale features, and small-scale features are concat spliced ​​in the channel number dimension to obtain features with a channel number of 1.5C and a resolution of 1R.

[0072] The IRB part first performs a multi-branch operation on the output features of the FAB part, namely, depthwise convolution (DwConv) with convolution kernels of,,, and five branches that do not perform any operation (direct connection). The results of these five branches are directly added together, and the added feature information is subjected to a Conv operation with a convolution kernel of, and a step size of 1. Finally, the Conv output is added to the output of the FAB part to obtain features with 1.5C channels and 1R resolution.

[0073] The MFE1 module aggregates the three scale features of the feature extraction network to the intermediate scale, and the output feature is FM1. After the nearest neighbor upsampling operation, FM1 is concatenated with the output result of the second layer C2f layer of the feature extraction network by Concat1. The output feature is passed through C2f5 to obtain the large-scale feature FL1. FM1 is subjected to the Conv operation with a convolution kernel of and a step size of 2. The output of Conv is concatenated with the output feature of the PASPP layer through Concat2. The output feature is passed through C2f6 to obtain the small-scale feature FS1.

[0074] The MFE2 module aggregates the three scale features FL1, FM1 and FS1 obtained above to a unified scale to obtain the medium-scale feature FM2. After the nearest neighbor upsampling operation is performed on FM2, the features obtained by the nearest neighbor upsampling operation on FM1 and the large-scale feature FL1 are concatenated by Concat3 to obtain the large-scale feature FL2. The features obtained by passing the Conv operation with a convolution kernel of 2 on FM2 are concatenated with the features obtained by passing the Conv operation with a convolution kernel of 2 on FM1 and FS1 by Concat4 to obtain the small-scale feature FS2.

[0075] Finally, the output feature of FL2 after the C2f7 layer is feature P3, the output feature of FM2 after the Conv operation with a convolution kernel of step size 1 is feature P4, and the output feature of FS2 after the C2f8 layer is feature P5. P3, P4 and P5 are three different scale features of the multi-scale feature fusion network output to the end of the model to detect the head, which helps the model detect defect features of different scales.

[0076] Model detection head; In the conventional decoupled head structure, due to the interaction between the single-designed task features, that is, the convolution process of the head preprocessing, it is inevitable to introduce certain feature conflicts in the two tasks of classification and positioning, which affects the model detection effect. Therefore, this paper proposes a TDADH detection head, which uses the task alignment idea in TOOD. In the process of decoupling, different task branches can better learn the specific features of their respective tasks and improve the model detection accuracy. The TDADH structure is as follows Figure 7 shown.

[0077] First, the three scale features output by the multi-scale fusion network are subjected to two consecutive shared convolution operations with a step size of 1, in which the parameters of the convolution kernel are shared. The output features of the second shared convolution are concatenated with the output features of the first shared convolution to obtain the feature FGN. The use of shared convolution blocks can also effectively reduce the number of parameters in the detection head and improve computational efficiency.

[0078] Subsequently, the interactive feature FGN is decoupled into a positioning branch and a classification branch. Both branches pass through a task decomposition block. The task decomposition block is a layer attention mechanism. First, the input is passed through Avgpool to obtain a vector, and then the Conv operation and Sigmoid operation are used to obtain a vector with a value of 0 to 1. The vector is multiplied with the input of the task decomposition block to output the features for task decomposition.

[0079] Since the localization and classification tasks have different focuses, the task decomposition block dynamically calculates unique task features through layer attention to decompose the tasks, so that the classification and localization tasks can focus on their respective feature extraction to avoid introducing conflicts between the two task features, which helps to optimize the feature representation of specific tasks and improve the accuracy of classification and localization.

[0080] For the positioning branch, the DCNV2 module is introduced after the task decomposition module. The DCNV2 module can adaptively adjust the position of the convolution kernel to better capture the geometric shape and position information of the target, improve the expression ability of the positioning feature, and convolve the interactive feature FGN to extract the feature offset Offset required by DCNV2 and adjust the offset weight Mask, which is used to guide DCNV2 for the feature reorganization effect after task decomposition and strengthen the model's analysis of feature details. For the classification branch, after the task decomposition block, the dynamic feature selection is performed with the interactive feature FGN to obtain the feature weighting. The dynamic selection block DSM undergoes convolution operation, SiLU activation, convolution operation and Sigmoid operation. The dynamic selection weighting process adaptively adjusts the importance of each region feature, suppresses noise features, and improves classification accuracy. Finally, task alignment learning is used to further guide the head to perform task alignment.

[0081] The TDADH detection head has significantly improved model expression ability, task specificity, positioning accuracy and classification effect through more efficient feature sharing, targeted task decoupling and task alignment, and dynamic spatial adjustment. These advantages enable it to demonstrate higher performance and stronger adaptability when dealing with diverse scenarios and complex target detection tasks.

[0082] 3. Loss function;

[0083] The loss function mainly includes three parts: positioning loss, classification loss and confidence loss. Positioning loss (Localization_Loss): used to measure the difference between the predicted box and the real box, using IoU (Intersection over Union) and its improved version CIOU for measurement. Classification loss (Classification_Loss): used to measure the difference between the category of the target contained in each predicted box and the real category. Generally, cross entropy loss or binary cross entropy loss is used to calculate the accuracy of category prediction. Confidence loss (Confidence_Loss): used to measure the difference between the confidence of whether the predicted box contains an object and the actual situation. It is usually calculated using cross entropy loss or binary cross entropy loss.

[0084] The loss function benchmark function formula is:

[0085] Loss = λ coordLocalization_Loss+λ cls Classification_Loss+λ conf Confidence_Loss;

[0086] Among them, λ coord , cls and λ conf is the weight coefficient used to balance the impact of different losses.

[0087] 4. Post-processing operations;

[0088] The deep model generates a large number of prediction boxes during the inference process. Each prediction box contains the category probability, confidence, and bounding box coordinates. In order to extract meaningful detection results, a series of post-processing steps are required, including threshold filtering, non-maximum suppression (NMS), bounding box adjustment, and multi-classification processing.

[0089] Threshold Filtering: First, the prediction boxes output by the model need to be initially filtered. Each prediction box contains a confidence score, which indicates the possibility that the box contains the target. By setting a confidence threshold, only the prediction boxes with confidence higher than the threshold are retained, and the invalid boxes with low confidence are removed. The purpose of this step is to reduce the number of candidate boxes and reduce the amount of calculation for subsequent processing. The confidence threshold setting is based on the specific application scenario and requirements. Set a reasonable confidence threshold (such as 0.5), and the prediction boxes with confidence scores lower than the threshold will be filtered out.

[0090] Non-Maximum Suppression (NMS): After threshold filtering, the remaining prediction boxes may still contain a large number of redundant boxes, especially when the target object is large, multiple prediction boxes may overlap. Non-maximum suppression (NMS) is used to remove these redundant overlapping boxes and retain only the best detection results. First, sort the remaining prediction boxes from high to low by confidence, select the box with the highest confidence from the sorted list as the current box, calculate the IoU (Intersection over Union) between the current box and all other boxes, set an IoU threshold (such as 0.5), and delete the boxes with IoU greater than the threshold from the list, because these boxes are considered to overlap too much with the current box and are redundant boxes. The IoU threshold needs to be adjusted according to the specific application scenario, usually 0.5-0.6, to find the best balance between detection accuracy and recall.

[0091] Bounding Box Refinement: After non-maximum suppression (NMS) processing, the retained prediction box needs to be further adjusted to improve positioning accuracy. The prediction box output by the deep learning method is usually expressed in the form of relative coordinates and needs to be converted into absolute coordinates in the image. The center coordinates and width and height of the bounding box are relative values ​​relative to the grid unit. The absolute coordinates are obtained after conversion through the sigmoid function and the exponential function. These coordinates are then converted into the coordinates of the upper left and lower right corners of the bounding box to draw the detection box in the image.

[0092] Multi-classification processing: For multi-classification processing, each prediction box may contain the probability of multiple defect types. It is necessary to set a class probability threshold, retain only the classes with probabilities higher than the threshold, and select the class with the highest probability as the final class. Through these post-processing steps, the deep learning method can provide accurate and reliable strip surface defect detection results. Examples of detection results are as follows: Figure 8 shown.

[0093] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions.

[0094] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A strip surface defect detection method based on deep learning, characterized in that: The detection method specifically comprises the following steps: S1. Data collection and preprocessing: Collect defect images and annotate defect images through data collection device design and data annotation; S2, deep learning model; use deep neural network to train data and output defect detection results; S3, loss function; positioning loss, classification loss and confidence loss, S4, post-processing operations; the deep model will generate a large number of prediction boxes during the inference process. Each prediction box contains the category probability, confidence and bounding box coordinates. In order to extract meaningful detection results, a series of post-processing steps are required, including threshold filtering, non-maximum suppression, bounding box adjustment and multi-classification processing.

2. The method for detecting surface defects of steel strips based on deep learning according to claim 1, characterized in that: In the S1, the DALSA line array scanning camera is used to continuously capture the surface image of the strip at high speed and high resolution. When the strip passes through the detection area of ​​the camera and the light source, the line array camera will continuously capture images at a high frame rate, and the image data is transmitted to the computer through the high-speed interface GigE for storage and processing.

3. The method for detecting surface defects of steel strips based on deep learning according to claim 1, characterized in that: In S1, the data collected by the data acquisition device are classified and stored according to the defect category, and the collected images are manually annotated using an annotation tool. During the annotation process, the location and type of each defect need to be identified, and a corresponding annotation file is generated during deep learning model training.

4. The method for detecting surface defects of steel strips based on deep learning according to claim 1, characterized in that: The overall structure of the deep model in S2 is based on the improved network structure of YOLOv8. The steps include: A1, feature extraction network; A2, multi-scale fusion network; A3, model detection head; The feature extraction network of the deep learning model in A1 adopts the CSPDarknet53 model architecture, which has 10 layers in total. The feature extraction network in A2 outputs three different scale features from its second layer C2f layer, third layer C2f layer and PASPP layer, and the corresponding three scale features are defined as large-scale feature FL, medium-scale feature FM and small-scale feature FS; The model detection head in A3 is the TDADH detection head, which uses the task alignment idea in TOOD. In the process of decoupling, different task branches can better learn the specific features of their respective tasks and improve the model detection accuracy.

5. The method for detecting surface defects of strip steel based on deep learning according to claim 1, characterized in that: The positioning loss in S3 is used to measure the difference between the predicted box and the true box, and is measured using IoU and its improved version CIOU. The classification loss is used to measure the difference between the category of the target contained in each predicted box and the true category. Generally, cross entropy loss or binary cross entropy loss is used to calculate the accuracy of category prediction. The confidence loss is used to measure the difference between the confidence of whether the predicted box contains an object and the actual situation, and is usually calculated using cross entropy loss or binary cross entropy loss.

6. The method for detecting surface defects of strip steel based on deep learning according to claim 5, characterized in that: The loss function benchmark function formula is: Loss=λ coord ·Localization_Loss+λ cls ·Classification_Loss+λ conf ·Confidence_Loss; Among them, λ coord , cls and λ conf is the weight coefficient used to balance the impact of different losses.

7. The method for detecting surface defects of steel strips based on deep learning according to claim 1, characterized in that: The processing method in S4 includes the following steps: B1, threshold filtering; B2, non-maximum suppression; B3, bounding box adjustment; B4. Multi-classification processing.

8. The method for detecting surface defects of steel strips based on deep learning according to claim 7, characterized in that: The threshold filtering performs preliminary filtering on the prediction boxes output by the model. Each prediction box contains a confidence score, which indicates the possibility that the box contains the target. By setting a confidence threshold, only the prediction boxes with confidence higher than the threshold are retained, and the invalid boxes with low confidence are removed. The non-maximum suppression sorts the remaining prediction boxes from high to low according to the confidence, selects the box with the highest confidence from the sorted list as the current box, calculates the IoU between the current box and all the remaining boxes, sets an IoU threshold, and deletes the boxes with IoU greater than the threshold from the list, because these boxes are considered to overlap too much with the current box and are redundant boxes. The IoU threshold needs to be adjusted according to the specific application scenario, and is usually set to 0.5-0.6 to find the best balance between detection accuracy and recall rate.

9. The method for detecting surface defects of steel strips based on deep learning according to claim 7, characterized in that: After the bounding box adjustment is processed by non-maximum suppression, the retained prediction box needs to be further adjusted to improve the positioning accuracy. The prediction box output by the deep learning method is usually expressed in the form of relative coordinates and needs to be converted into absolute coordinates in the image; In the multi-classification processing, each prediction box may contain the probability of multiple defect types. It is necessary to set a category probability threshold, only retain categories with probabilities higher than the threshold, and select the category with the highest probability as the final category.

Citation Information

Cited By

  • Dike surface defect detection method based on deep learning image processing

    CN119919418A

  • Multi-scale metal surface defect detection method and system fusing dynamic convolution and attention

    CN120431027A