Strip steel surface defect detection method and system and electronic equipment

Abstract

The invention relates to the field of deep learning, and provides a strip steel surface defect detection method and system, and electronic equipment, and the method comprises the steps: obtaining the state information of a to-be-detected strip steel, the state information at least comprising a surface image of the to-be-detected strip steel; inputting the state information into a pre-constructed detection model for defect detection, wherein the detection model is constructed by fusing an improved spatial pyramid module and a polarization self-attention mechanism on the basis of a Yolov8 model; the improved spatial pyramid module comprises an SPPCSPC layer and a variable convolutional layer, and the polarization self-attention mechanism comprises a channel branch and a spatial branch; and determining the surface defect of the to-be-detected strip steel based on the detection information output by the defect detection model. The method and the device are used for overcoming the defect of poor detection precision when defect detection is carried out on the surface of the strip steel in the prior art, and compared with an existing Yolov8 model, the scheme provided by the invention has the advantage that the detection precision is obviously improved when the defect detection is carried out on the surface of the strip steel.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a strip surface defect detection method and system, and electronic equipment. Background Art

[0002] Strip steel is a relatively narrow but long steel product, usually produced by hot rolling or cold rolling. During the production process, defects such as plaques, scratches, and inclusions may appear on the surface of the strip steel, which will affect the performance of the strip steel and reduce the quality of the strip steel. In addition, surface defects may also cause serious problems such as strip breakage and accumulation, causing huge losses to steel production. Therefore, the rapid and accurate detection of strip steel surface defects during the production process directly affects product quality and production efficiency, and is of great significance to the research of related technologies.

[0003] In related technologies, the deep learning algorithm of the YOLO system is used to assist in the detection of surface defects of strip steel. However, the existing surface defect detection algorithm for strip steel still has the problem of poor detection accuracy, which is not conducive to practical application deployment. Summary of the invention

[0004] The present invention provides a strip surface defect detection method and system, and an electronic device, which are used to solve the problem of poor detection accuracy when performing strip surface defect detection in the related art. The solution of the present application is improved on the basis of the existing Yolov8 model by combining an improved spatial pyramid module and a polarization self-attention mechanism. Compared with the existing Yolov8 model, the detection accuracy when performing strip surface defect detection is significantly improved.

[0005] The present invention provides a method for detecting surface defects of a strip steel, comprising:

[0006] Acquiring state information of the steel strip to be inspected, wherein the state information at least includes a surface image of the steel strip to be inspected;

[0007] Input the state information into a pre-built detection model for defect detection, wherein the detection model is constructed by fusing an improved spatial pyramid module and a polarization self-attention mechanism on the basis of a Yolov8 model, wherein the improved spatial pyramid module includes an SPPCSPC layer and a variability convolution layer, and the polarization self-attention mechanism includes a channel branch and a spatial branch, and the output of the polarization self-attention mechanism is obtained by parallel fusion of the outputs of the channel branch and the spatial branch;

[0008] Based on the detection information output by the defect detection model, the surface defects of the steel strip to be detected are determined.

[0009] According to the strip surface defect detection method provided by the present invention, the channel branch is also used for:

[0010] Obtaining an input first feature map, and converting the first feature map into a query vector and a value vector by convolution;

[0011] After the query vector is fully compressed, it is enhanced using the Softmax operator;

[0012] Determine a key vector based on the feature graph, and perform a matrix multiplication operation on the query vector and the key vector;

[0013] The output value is calculated through the activation function.

[0014] According to the strip surface defect detection method provided by the present invention, the space branch is also used for:

[0015] Obtaining an input second feature map, and converting the second feature map into a query vector and a value vector by convolution;

[0016] After compressing the query vector in spatial dimension, a normalization function is used to enhance the query vector;

[0017] The output value is calculated through the activation function.

[0018] According to the strip surface defect detection method provided by the present invention, the SPPCSPC layer includes several parallel maximum pooling layers, and the SPPCSPC layer is used to perform data classification on the status information of the strip to be detected.

[0019] According to the strip surface defect detection method provided by the present invention, the variable convolution layer adjusts the position and shape of the convolution kernel through a sparse attention mechanism to improve the robustness and generalization of the detection model;

[0020] The deformable convolution layer processes memory access instructions of several channels through a single thread.

[0021] According to the strip surface defect detection method provided by the present invention, the training process of the detection model includes:

[0022] Acquire a number of sample data, wherein the sample data reflects status information of the strip steel;

[0023] After marking the sample data, determining the defect situation corresponding to each sample data;

[0024] Constructing a training set and a test set based on the sample data;

[0025] The pre-built primary detection model is trained based on the training set, and the performance of the trained detection model is tested based on the test set.

[0026] According to the strip surface defect detection method provided by the present invention, the performance of the trained detection model is detected based on the test set, including:

[0027] Inputting the test set into the detection model to determine the detection result output by the detection model;

[0028] The performance of the detection model is evaluated based on the performance parameters of the detection results corresponding to the test set, and the performance parameters include precision, recall rate, and average precision mean.

[0029] According to the strip surface defect detection method provided by the present invention, the training process of the detection model further includes:

[0030] The generalization ability of the detection model is evaluated by a loss function, wherein the loss function includes a weighted intersection-over-union function;

[0031] The weighted intersection-over-union function improves the authenticity of the evaluation process through a region alignment mechanism and a weighting mechanism.

[0032] The present invention also provides a strip steel surface defect detection system, comprising:

[0033] An information acquisition module, used for acquiring status information of the steel strip to be detected, wherein the status information at least includes a surface image of the steel strip to be detected;

[0034] A model detection module, used for inputting the state information into a pre-built detection model for defect detection, wherein the detection model is constructed by fusing an improved spatial pyramid module and a polarization self-attention mechanism on the basis of a Yolov8 model, wherein the improved spatial pyramid module includes an SPPCSPC layer and a variability convolution layer, and the polarization self-attention mechanism includes a channel branch and a spatial branch, and the output of the polarization self-attention mechanism is obtained by parallel fusion of the outputs of the channel branch and the spatial branch;

[0035] The defect determination module is used to determine the surface defects of the strip to be detected based on the detection information output by the defect detection model.

[0036] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned strip surface defect detection methods is implemented.

[0037] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the computer program implements any of the above-mentioned strip surface defect detection methods.

[0038] The present invention also provides a computer program product, including a computer program, which implements any of the above-mentioned strip surface defect detection methods when executed by a processor.

[0039] In the strip surface defect detection method provided by the solution of the present application, an improved detection model can be used to perform data detection on the collected strip status information. The detection model, based on the Yolov8 model, introduces an improved spatial pyramid module consisting of an SPPCSPC layer and a variable convolutional layer, which can effectively integrate multi-scale information and reduce background interference. In addition, the polarization self-attention mechanism is also applied in the detection model, which can reduce noise interference while reducing information loss and improving the detection accuracy of small targets and complex scenes through the fusion of channel branches and spatial branches. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 This is one of the flow diagrams of the strip steel surface defect detection method provided by the present invention;

[0042] Figure 2 is a schematic diagram of the structure of a detection model provided by an embodiment of the present invention;

[0043] Figure 3 A schematic diagram of the structure of the polarized self-attention mechanism provided by an embodiment of the present invention;

[0044] Figure 4 It is a schematic diagram of the structure of the SPPCSPC provided by an embodiment of the present invention;

[0045] Figure 5 is a schematic diagram of the structure of the variable convolution provided by an embodiment of the present invention;

[0046] Figure 6 is a schematic diagram of the structure of the receptive field provided by an embodiment of the present invention;

[0047] Figure 7 is a geometric diagram of a loss function provided by an embodiment of the present invention;

[0048] Figure 8 is a schematic diagram of the structure of the attention mechanism provided by an embodiment of the present invention;

[0049] Fig. 9 is a schematic diagram of the structure of a neck attention module provided by an embodiment of the present invention;

[0050] Fig.10 is a schematic diagram of the structure of a bidirectional feature pyramid network provided by an embodiment of the present invention;

[0051] Fig.11 is a schematic structural diagram of a strip steel surface defect detection system provided by an embodiment of the present invention;

[0052] Fig.12 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] Figure 1 It is one of the flow diagrams of the strip surface defect detection method provided by the present invention.

[0055] like Figure 1 As shown, this embodiment provides a method for detecting surface defects of a strip steel, comprising:

[0056] Step 101, obtaining state information of a steel strip to be inspected, wherein the state information at least includes a surface image of the steel strip to be inspected;

[0057] Step 102, inputting the state information into a pre-built detection model for defect detection, wherein the detection model is constructed by fusing an improved spatial pyramid module and a polarization self-attention mechanism on the basis of a Yolov8 model; the improved spatial pyramid module includes an SPPCSPC layer and a variable convolution layer, the polarization self-attention mechanism includes a channel branch and a spatial branch, and the output of the polarization self-attention mechanism is obtained by parallel fusion of the outputs of the channel branch and the spatial branch;

[0058] Step 103: determining the surface defects of the steel strip to be detected based on the detection information output by the defect detection model.

[0059] In practical applications, the status information of the steel strip may be image information of the steel strip to be detected, or may be other information that can reflect the surface condition of the steel strip.

[0060] Figure 2 It is a schematic diagram of the structure of the detection model provided by an embodiment of the present invention.

[0061] In practical applications, see Figure 2 The structural diagram of the detection model shown in Figure 2 It can be seen that compared with the previous YOLO system model, the detection model introduces the C2f model backbone and decoupled model head, and adopts the Anchor-free splitting method to improve the accuracy and efficiency of the detection process.

[0062] The SPPCSPC layer used in the improved spatial pyramid module in this embodiment integrates the respective advantages of the traditional SPP module and the CSPC module to improve the performance of target detection and image classification. Specifically, the SPP module can use multi-scale pooling to obtain multi-scale features of the input image, thereby enhancing the detection capability of targets of different scales. The CSPC module introduces cross-stage partial connections, reduces redundant features, improves feature utilization efficiency, promotes gradient flow, and alleviates the problem of gradient disappearance. The SPPSPC layer in this embodiment combines multi-scale feature extraction and efficient feature connection methods, which can better characterize image details and complex structures, thereby improving the overall detection and classification effects.

[0063] In implementation, the self-attention mechanism in related technologies usually needs to calculate the attention of all positions to all other positions, which will lead to increased computational complexity and extended model training time. The polarized self-attention mechanism used in this embodiment is an improved self-attention mechanism that aims to improve the efficiency and accuracy of the detection model. The polarized self-attention mechanism reduces the attention between position pairs that need to be calculated by introducing a polarization factor, thereby improving the efficiency of the self-attention model. This method can improve the efficiency of the self-attention model by reducing the amount of computation while maintaining accuracy.

[0064] In the strip surface defect detection method provided in this embodiment, the improved detection model can be used to perform data detection on the collected strip status information. The detection model introduces an improved spatial pyramid module consisting of an SPPCSPC layer and a variable convolutional layer on the basis of the Yolov8 model, which can effectively integrate multi-scale information and reduce background interference. In addition, the polarization self-attention mechanism is also applied in the detection model, which can reduce noise interference while reducing information loss and improving the detection accuracy of small targets and complex scenes through the fusion of channel branches and spatial branches.

[0065] Figure 3 A schematic diagram of the structure of the polarized self-attention mechanism provided in an embodiment of the present invention.

[0066] like Figure 3 As shown, in the exemplary embodiment, the channel branch is also used for:

[0067] Obtaining an input first feature map, and converting the first feature map into a query vector and a value vector by convolution;

[0068] After the query vector is fully compressed, it is enhanced using the Softmax operator;

[0069] Determine a key vector based on the feature graph, and perform a matrix multiplication operation on the query vector and the key vector;

[0070] The output value is calculated through the activation function.

[0071] In practical applications, in the channel branch of the polarized self-attention mechanism, the first input feature map can be converted into a query vector (Query, Q) and a value vector (Value, V) by 1×1 convolution. Q is fully compressed and enhanced by the normalization function (Softmax). At the same time, V still maintains the number of channels of C / 2. Q is matrix multiplied with the key vector (Key, K) and then passed through a 1×1 convolution and normalization layer to increase the dimension to C. Finally, the activation function is used to calculate the output value. The calculation formula of the channel branch weight Ach(X) is as follows:

[0072]

[0073] Among them, X is the input feature, FSG represents the nonlinear activation function, Wz is the learnable weight matrix, and θ1 is the related hyperparameter of this weight matrix. σ1 is the first activation function, Wv(X) is the linear transformation of X, Wv is the weight matrix, FSM represents the Softmax function, σ2 is the second activation function, and Wq(X) generates a query-related representation for the input X.

[0074] In an exemplary embodiment, Figure 3 As shown, the space branch is also used for:

[0075] Obtaining an input second feature map, and converting the second feature map into a query vector and a value vector by convolution;

[0076] After compressing the query vector in spatial dimension, a normalization function is used to enhance the query vector;

[0077] The output value is calculated through the activation function.

[0078] like Figure 3 As shown in the figure, the spatial branch Asp(X) of the polarized self-attention mechanism also uses 1×1 convolution to convert the second feature map of the input into Q and V. Global pooling () is used to compress the spatial dimension of Q, and then the normalization function is used to enhance Q. V remains unchanged. The process of calculating the output value through the activation function conforms to the following formula:

[0079] A sp (X) = F SG {σ3(F SM (σ1(F GP (W q (X))))×σ2(W ν (X)))} (2)

[0080] Among them, FSG represents a nonlinear activation function, σ1, σ2, σ3 are activation functions for processing linear transformations, FGP is a global pooling operation, and Wq(X) and Wv(X) are linear transformation matrices.

[0081] In practical applications, after the output values ​​are calculated by the channel branch and the spatial branch respectively, the output values ​​of the two branches can be output in parallel. By splicing the outputs of the two branches, a comprehensive attention representation combining channel and spatial information is finally obtained. The final combined output result PSAp(X) is as follows:

[0082] PSA p (X) = Concat(A ch (X),A sp (X)) (3)

[0083] Figure 4 It is a schematic diagram of the structure of the SPPCSPC provided in an embodiment of the present invention.

[0084] like Figure 4 As shown, in an exemplary embodiment, the SPPCSPC layer includes several parallel maximum pooling layers, and the SPPCSPC layer is used to perform data classification on the state information of the strip to be detected.

[0085] In the implementation, in the SPPCSPC layer, the input feature map first introduces hole convolution and parallel convolution through the RFB module to enhance the model feature extraction capability. For some small target detection, the maximum pooling only selects the maximum value as the output, and small targets usually contain more details and local features, and the maximum pooling will ignore some small information. In contrast, average pooling can better retain local detail information, and has relatively little impact on noise, so it is more suitable for small target detection. Therefore, the maximum pooling layer is replaced by average pooling, and the pooling kernel is reduced to (3,5,9) to better retain the local details of small targets. The DCNv4 variable convolution is introduced to replace the standard convolution layer, which can more flexibly adapt to the shape and size of the target and further improve the feature extraction capability.

[0086] Figure 5 It is a schematic diagram of the structure of the variable convolution provided by an embodiment of the present invention.

[0087] like Figure 5 As shown, in an exemplary embodiment, the variable convolution layer adjusts the position and shape of the convolution kernel through a sparse attention mechanism to improve the robustness and generalization of the detection model;

[0088] The deformable convolution layer processes memory access instructions of several channels through a single thread.

[0089] In practical applications, Figure 5 The example illustrates the intra-group shared processing of Deformable Convolutionv4 (DCNv4). DCNv4 combines the sparse attention mechanism with the convolution operation, removes the softmax normalization in spatial aggregation, and improves dynamic properties and expressiveness. Through the sparse attention mechanism, DCNv4 can adaptively adjust the position and shape of the convolution kernel, and improve the sampling mechanism, so that the model has better robustness and generalization in complex situations such as target deformation and scale change. In DCNv4, a single thread is used to process multiple channels that share sampling offsets and aggregation weights in the same group. This can reduce workloads such as memory reads and bilinear interpolation coefficient calculations, and can merge multiple memory access instructions.

[0090] In practice, DCNv4 shows stronger robustness and adaptability when dealing with complex computer vision tasks. Figure X , the convolution kernel W and the bias b introduce an offset Δp k And the attention weight Δmk, the process of obtaining the output feature map Y conforms to the following formula:

[0091]

[0092] Among them, K is the number of sampling points of the convolution kernel, p is the pixel coordinate of the image feature map, and pk is the relative position of the kth sampling point in the convolution kernel.

[0093] Figure 6 Schematic diagram of the structure of the receptive field provided by an embodiment of the present invention.

[0094] like Figure 6 As shown, in an exemplary embodiment, the improved spatial pyramid module also includes a receptive field layer, and the receptive field layer includes a convolution layer, a batch normalization layer and an activation function layer, which are used to expand the receptive field.

[0095] In practical applications, the receptive field block (RFB) can also enable the detection model to capture richer semantic information by introducing multi-scale feature responses. Compared with traditional convolution operations, the RFB in this embodiment can effectively enhance the perception ability of the network without increasing the number of parameters and computational costs, so that the detection model can be more adaptable to processing target objects of various sizes and shapes, thereby improving the performance of target detection and semantic segmentation tasks.

[0096] In an exemplary embodiment, the training process of the detection model includes:

[0097] Acquire a number of sample data, wherein the sample data reflects status information of the strip steel;

[0098] After marking the sample data, determining the defect situation corresponding to each sample data;

[0099] Constructing a training set and a test set based on the sample data;

[0100] The pre-built primary detection model is trained based on the training set, and the performance of the trained detection model is tested based on the test set.

[0101] In an exemplary embodiment, the testing of the performance of the trained detection model based on the test set includes:

[0102] Inputting the test set into the detection model to determine the detection result output by the detection model;

[0103] The performance of the detection model is evaluated based on the performance parameters of the detection results corresponding to the test set, and the performance parameters include precision, recall rate, and average precision mean.

[0104] In an exemplary embodiment, the training process of the detection model further includes:

[0105] The generalization ability of the detection model is evaluated by a loss function, wherein the loss function includes a weighted intersection-over-union function;

[0106] The weighted intersection-over-union function improves the authenticity of the evaluation process through a region alignment mechanism and a weighting mechanism.

[0107] The weighted intersection over union (WIoU) function provided in this embodiment is a loss function with dynamic non-monotonic focused border regression proposed based on the conventional IoU function. By introducing better region alignment and weighting mechanism, it can better reflect the actual performance of target detection.

[0108] Figure 7 It is a geometric diagram of the loss function provided by an embodiment of the present invention.

[0109] like Figure 7 As shown, C B (x, y) represents the geometric center of the prediction box, C A (x gt ,y gt ) represents the geometric center of the target box, Wg and Hg represent the width and height of the minimum outer rectangular box, to prevent R WIoU The anchor box loss produces gradients that hinder convergence. * indicates a separation operation.

[0110] L WIoUv1 =R WIoU L IoU (5)

[0111]

[0112] Among them, LIoU is a loss function based on IoU, which adjusts the IoU loss weight in linear space, and RWIoU is a reweighted IoU loss, which is used to dynamically adjust the weight to solve the problem of adaptability of the loss to different types of samples. Combining the two types of loss functions, we can obtain a loss function LWIoUv1 with higher robustness in order to more comprehensively optimize the performance of the prediction box.

[0113] In order to make BBR focus more on anchor boxes of normal quality and improve its classification performance, a small gradient gain is assigned to the anchor box to effectively prevent the generation of large harmful gradients in low quality.

[0114]

[0115]

[0116] Among them, LIoU is an exponential moving average with momentum m, L*IoU is an IoU loss function with a dynamic weighting mechanism, which dynamically adjusts the weight according to the target attribute, and the anchor box outlier β is the ratio of the two. The smaller the outlier value, the higher the quality of the anchor box. A non-monotonic focusing coefficient LWIoUv3 is constructed through β for WIoUv1, and a smaller gradient gain is assigned to the anchor box with a larger outlier value. Among them, γ is the gradient gain, which is controlled by the hyperparameters α and δ.

[0117] Figure 8 Schematic diagram of the structure of the attention mechanism provided by an embodiment of the present invention.

[0118] Fig. 9 Schematic diagram of the structure of the neck attention module provided by an embodiment of the present invention.

[0119] In an exemplary embodiment, an attention mechanism C2f-BAM is also introduced into the detection model, such as Figure 8 As shown in the figure, the attention mechanism in this embodiment introduces the neck attention module BAM at the bottleneck of the C2f architecture to enhance the feature representation ability of the model. Through reasonable configuration and experimental verification, a good balance is achieved between performance and efficiency. By directly adding BAM to the bottleneck module, the powerful function of the attention mechanism can be fully utilized while maintaining the advantages of the original structure, thereby improving the performance of the model in various tasks.

[0120] like Fig. 9 As shown in the figure, in the neck attention module, for the input feature map F, the channel dimension features are generated using global average pooling in the channel attention module (CA), and then the input channel weights are learned through two layers of fully connected layers FC, and finally the C×1×1 size channel attention weight M is obtained. c (F), the process conforms to the following formula:

[0121] M c (F)=BN(MLP(AvgPool(F)))=BN(W1(W0AvgPool(F)+b0)+b1)(9)

[0122] The spatial attention module uses 1×1 convolution to reduce the dimension, and then learns the spatial weight of the feature map by repeated dilated convolution operations. Finally, the feature weight is reduced to 1 through 1×1 convolution. The spatial attention weight M s (F), the process conforms to the following formula:

[0123]

[0124] The spatial and channel attentions are summed element by element, activated using the activation function, and finally the original feature map is weighted using the comprehensive attention weight, which conforms to the following formula:

[0125]

[0126] Fig.10 It is a schematic diagram of the structure of a bidirectional feature pyramid network provided by an embodiment of the present invention.

[0127] In an exemplary embodiment, Fig.10As shown in the figure, the detection model also introduces a bidirectional feature pyramid network BiFPN, which is an improvement on FPN. BiFPN realizes the bidirectional flow and multiple fusion of feature information while maintaining computational efficiency, and is a multi-scale feature fusion method for target detection. By introducing a weighted bidirectional feature fusion mechanism, features at different levels can be efficiently fused, improving detection accuracy and speed. The BiFPN structure is shown in the figure. Fig.10 As shown in Figure 2, BiFPN considers each bidirectional path as a feature network layer. The weight parameter w of the i-th input feature map is i Get the normalized weight Use the normalized weights to adjust the i input feature maps x i After weighted fusion, the feature map obtained after weighted fusion conforms to the following formula:

[0128]

[0129] Finally, the weighted fused feature maps are concatenated along the specified dimension. For each weighted feature map, they are concatenated along the channel dimension to form a new feature map.

[0130] The strip surface defect detection system provided by the present invention is described below. The strip surface defect detection system described below and the strip surface defect detection method described above can be referenced to each other.

[0131] Fig.11 It is a schematic diagram of the structure of a strip steel surface defect detection system provided in an embodiment of the present invention.

[0132] like Fig.11 As shown, the strip surface defect detection system provided in this embodiment includes:

[0133] An information acquisition module 1101 is used to acquire status information of the steel strip to be detected, wherein the status information at least includes a surface image of the steel strip to be detected;

[0134] A model detection module 1102 is used to input the state information into a pre-built detection model for defect detection. The detection model is constructed by fusing an improved spatial pyramid module and a polarization self-attention mechanism on the basis of a Yolov8 model. The improved spatial pyramid module includes an SPPCSPC layer and a variability convolution layer. The polarization self-attention mechanism includes a channel branch and a spatial branch. The output of the polarization self-attention mechanism is obtained by parallel fusion of the outputs of the channel branch and the spatial branch.

[0135] The defect determination module 1103 is used to determine the surface defects of the strip to be detected based on the detection information output by the defect detection model.

[0136] The performance of the detection model is also experimentally verified in the solution of the present application, as shown in the following examples.

[0137] A dataset of surface defects of strip steel was obtained. The dataset contains six types of defects, namely scale (RS), plaque (Pa), crack (Cr), pit (PS), inclusion (In) and scratch (Sc), with a total of 1800 grayscale images in JPG format. There are 300 images of each defect type, with a resolution of 200×200 pixels. The dataset is randomly divided into training set and validation set in a ratio of 7:3, with 1260 test sets and 540 validation sets. The experimental environment uses GPU to train the data, the operating system is Ubuntu22.04.3, the graphics card is NVIDIA GeForce RTX 3090, the video memory is 24GB, the software environment is Cuda12.1, the pytorch version 2.1.1 deep learning framework is selected, and the python3.10.14 version is used for programming. The number of iterations is 120, the image size is 640×640 resolution, the batch size is 16, the learning rate lr0 is 0.01, the weight parameter is 0.0005, and the learning rate momentum is 0.937.

[0138] When evaluating the detection accuracy and speed of defect detection models, commonly used evaluation indicators include: precision P, recall R, mean average precision mAP, frames per second FPS, model calculation GFLOPs, and model parameters Parameters to measure detection accuracy.

[0139] The precision p, recall R, average precision mAP, and frames per second FPS satisfy the following formulas respectively:

[0140]

[0141] Among them, TP, TN, FP, and FN represent the number of correctly detected defect targets, correctly detected non-defect samples, the number of incorrectly detected defect targets, and the number of missed defect targets, respectively. Framenum is the total number of detected images, and ElapsedTime is the total time spent on detection.

[0142] When comparing the performance of feature extraction modules, in order to verify the superiority of SCRD in the solution of this application, the performance impact of feature extraction modules on defect detection is compared. SPPF, SimSPPF, SPPELAN, and SPPCSPC are selected for comparison.

[0143] Table 1 Comparison of feature extraction modules

[0144]

[0145]

[0146] As can be seen from the table, the proposed method SCRD in this paper is 3.9% higher than SPPCSPC in terms of accuracy. The recall rate of SCRD is 2.1% higher than that of SPPF, but 0.7% lower than that of SPPCSPC. However, the average precision of SCRD is 1.3% higher than that of SPPF and 0.6% higher than that of SPPCSPC. SCRD has a large amount of calculation, but the processing speed is fast. Therefore, the effectiveness of the improved module SCRD is proved.

[0147] When conducting the attention mechanism experiment, in order to verify the superiority of C2f_BAM in the solution of this application, the influence of different attention mechanisms on the experimental results is compared. This paper selects the attention mechanisms GAM, BoT3, MHSA, CBAM, MSDA, ShuffleAttention, and SENet for comparison.

[0148] Table 2 Comparison of attention mechanisms

[0149]

[0150] According to the table, after embedding GAM, BoT3, MHSA, CBAM, ShuffleAttention, MSDA, and SENet attention mechanisms, the precision, recall rate, and average precision all decreased to varying degrees. After embedding the C2f_BAM attention mechanism, the mAP increased by 0.5%, the precision increased by 0.6%, and the recall rate increased by 0.8% compared with the baseline C2f. This proves the effectiveness of the C2f_BAM attention mechanism.

[0151] When comparing loss functions, in order to verify the usefulness of introducing the WIoU loss function, the commonly used loss functions GIoU, CIoU, EIoU, SIoU and DIoU were selected for comparative experiments.

[0152] Table 3. Comparison of loss functions

[0153]

[0154] According to the experimental results, compared with other loss functions, the introduction of WIoU loss function can obtain higher total precision, recall rate and mAP. For the six defects, the average precision of CIoU and WIoU for Sc reached 97.9%. In addition, for other defect types, the detection results of WIoU are better than those of other loss functions. WIoU adopts a dynamic non-monotonic focusing mechanism, which enables the model to focus more on the anchor box of the quality of ordinary defect samples, thereby ensuring the balance of model performance. As a regression loss, WIoU loss improves the convergence speed of the model, thereby improving the prediction effect of the model. Therefore, WIoU is selected as the loss function.

[0155] In this embodiment, the overall effectiveness of the detection model is also verified by ablation experiments. Based on YOLOv8n, an ablation experiment is designed on the public dataset of Northeastern University NEU-DET for comparative analysis. During the experiment, all parameters are kept consistent. 1 is the yolov8 benchmark model, 2 is the SCRD module, 3 is the PSA module, 4 is the C2f-BAM module, 5 is the BiFPN module, and 6 is the WIoU loss function. 1+2 means the introduction of the SCRD module alone, 1+3 means the citation of the PSA module alone, 1+4 means the introduction of the C2f-BAM module alone, 1+5 means the citation of the BiFPN module alone, and 1+6 means the citation of the WIoU loss function alone. 1+2+3 represents the results after the introduction of the SCRD and PSA modules, with an overall accuracy increase of 1.7%, a recall rate increase of 2.6%, and an average accuracy increase of 2.0%. 1+...+4 represents the results after the addition of SCRD, PSA and C2f-BAM, with accuracy, recall and average accuracy increased by 2.3%, 4.3% and 2.9% respectively. 1+...+5 represents the result after adding SCRD, PSA, C2f-BAM and BiFPN. The precision and average precision are improved by 0.6% and 0.6% compared with 1+...+4, but the recall rate is reduced by 0.5%. 1+...+6 represents the result after introducing 1+...+4 module and WIoU. The overall precision is the highest, and the average precision of the detection of six defects is higher than that of the baseline model. The average precision of RS, Sc, Cr, Ps, Pa, and In is improved by 3.3%, 1.8%, 5.6%, 34.3%, 26.2%, and 3.4%, respectively.

[0156] Table 4 Ablation experiment

[0157]

[0158] It can be seen from the above table that the original model had missed detections and low detection accuracy, while the improved model has improved accuracy and can detect more targets, verifying the improvement of the improved model in defect sample detection.

[0159] To verify the performance of the improved model, we conducted comparative experiments with the current mainstream models YOLOv5n, YOLOv5s, YOLOX, YOLOv7, YOLOv7-tiny, Fast-RCNN

[23] , and SSD

[24] algorithms. The experimental results are shown in Table 5.

[0160] Table 5 Comparison of different models

[0161]

[0162] It can be seen from the table that Faste-RCNN and SSD have slow detection speed and low accuracy, and neither of them is suitable for current detection needs. Compared with other methods, the algorithm in this paper is better in accuracy, recall rate and average precision. Compared with the benchmark YOLOv8n, the algorithm has improved by 4.9%, 4.3% and 4.0%, respectively, proving the effectiveness of the improved algorithm. Compared with other methods, the accuracy is improved by 11.3%, 9.0% and 4.0% compared with YOLOv5n, YOLOv6n and YOLOX, the number of parameters is reduced by 1.2M, 2.8M and 3.5M, and the number of model parameters is increased by 2.6M. Due to the introduction of complex module structure, the FPS of the algorithm in this paper is lower than that of YOLOv8n detection frame number, but it is still suitable for real-time detection of strip surface defects. In summary, the improved YOLOv8n-BSPB model has good detection performance in strip surface defect detection.

[0163] In order to verify the effectiveness of the model, the experimental environment remains unchanged, and the data set is randomly divided into a training set and a validation set in a ratio of 7:3. The number of iterations is 120, the image size is 640×640 resolution, the batch size is 16, and the learning rate lr0 is 0.01. The training uses the kaggle strip steel data set and the defect data set (Iron) collected in the actual factory for model testing. There are four types of defects in the kaggle data set. There are 6666 defect images in the data set, 4666 test sets and 2000 validation sets.

[0164] The defect dataset collected in the factory includes two types of defects: gap (FL) and edge damage (ED). There are 608 defect images in the dataset, 426 test sets and 182 validation sets.

[0165] The verification experiment results are shown in Table 6, where the four types of defect names in the Kaggle dataset correspond to 1, 2, 3, and 4 in the table respectively.

[0166] Table 6 Comparison of tests on different data sets

[0167]

[0168] As can be seen from the table, for the Kaggle dataset, the precision, recall rate and average precision of the improved Yolov8n model increased by 2.1%, 2.1% and 2.3%, and the average precision of cracks, inclusions, plaques and scratches increased by 4.7%, 4.0%, 0.9% and 7.2%. For the Iron dataset, the precision, recall rate and average precision of the improved model increased by 10.0%, 1.9% and 5.5%, and the average precision of scratches and edge damage increased by 11.0% and 3.4%. In summary, the improved model has good generalization.

[0169] In summary, the improvement strategy proposed in this embodiment is aimed at the problems existing in the actual detection of existing strip steel. It is improved on the basis of yolov8n, and the BAM attention mechanism is introduced in the backbone to enhance the model feature expression ability. It is proposed to improve the spatial pyramid pooling feature extraction module SCRD to extract multi-scale features. And the PSA polarization attention mechanism is introduced to optimize information transmission and suppress background interference. The BiFPN bidirectional feature pyramid network is introduced in the neck to improve the detection speed of the model while maintaining accuracy. Finally, the WIoU loss function is introduced to reduce error propagation and improve the accuracy and efficiency of target detection. The feasibility of the improved algorithm is verified through a large number of comparative tests, ablation experiments and data set tests. It has high detection accuracy and small parameter amount, which has positive significance in the real-time detection of surface defects of strip steel. In future work, we will continue to study the lightweighting of the model while maintaining high accuracy, and improve the detection speed while ensuring detection accuracy.

[0170] Fig.12 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.12 As shown, the electronic device may include: a processor 1210, a communications interface 1220, a memory 1230 and a communication bus 1240, wherein the processor 1210, the communications interface 1220 and the memory 1230 communicate with each other through the communication bus 1240. The processor 1210 may call the logic instructions in the memory 1230 to execute the strip surface defect detection method, which includes:

[0171] Acquiring state information of the steel strip to be inspected, wherein the state information at least includes a surface image of the steel strip to be inspected;

[0172] The state information is input into a pre-built detection model for defect detection, wherein the detection model is constructed by fusing an improved spatial pyramid module and a polarization self-attention mechanism on the basis of a Yolov8 model; the improved spatial pyramid module includes an SPPCSPC layer and a variability convolution layer, the polarization self-attention mechanism includes a channel branch and a spatial branch, and the output of the polarization self-attention mechanism is obtained by parallel fusion of the outputs of the channel branch and the spatial branch;

[0173] Based on the detection information output by the defect detection model, the surface defects of the steel strip to be detected are determined.

[0174] In addition, the logic instructions in the above-mentioned memory 1230 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0175] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the strip surface defect detection method provided by the above methods, the method comprising:

[0176] Acquiring state information of the steel strip to be inspected, wherein the state information at least includes a surface image of the steel strip to be inspected;

[0177] The state information is input into a pre-built detection model for defect detection, wherein the detection model is constructed by fusing an improved spatial pyramid module and a polarization self-attention mechanism on the basis of a Yolov8 model; the improved spatial pyramid module includes an SPPCSPC layer and a variability convolution layer, the polarization self-attention mechanism includes a channel branch and a spatial branch, and the output of the polarization self-attention mechanism is obtained by parallel fusion of the outputs of the channel branch and the spatial branch;

[0178] Based on the detection information output by the defect detection model, the surface defects of the steel strip to be detected are determined.

[0179] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the strip surface defect detection method provided by the above methods, the method comprising:

[0180] Acquiring state information of the steel strip to be inspected, wherein the state information at least includes a surface image of the steel strip to be inspected;

[0181] The state information is input into a pre-built detection model for defect detection, wherein the detection model is constructed by fusing an improved spatial pyramid module and a polarization self-attention mechanism on the basis of a Yolov8 model; the improved spatial pyramid module includes an SPPCSPC layer and a variability convolution layer, the polarization self-attention mechanism includes a channel branch and a spatial branch, and the output of the polarization self-attention mechanism is obtained by parallel fusion of the outputs of the channel branch and the spatial branch;

[0182] Based on the detection information output by the defect detection model, the surface defects of the steel strip to be detected are determined.

[0183] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0184] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0185] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting surface defects of a strip steel, characterized in that: include: Acquiring state information of the steel strip to be inspected, wherein the state information at least includes a surface image of the steel strip to be inspected; Inputting the state information into a pre-built detection model for defect detection, wherein the detection model is constructed by integrating an improved spatial pyramid module and a polarized self-attention mechanism on the basis of a Yolov8 model; The improved spatial pyramid module includes an SPPCSPC layer and a variable convolution layer, the polarized self-attention mechanism includes a channel branch and a spatial branch, and the output of the polarized self-attention mechanism is obtained by parallel fusion of the outputs of the channel branch and the spatial branch; Based on the detection information output by the defect detection model, the surface defects of the steel strip to be detected are determined.

2. The method for detecting surface defects of a steel strip according to claim 1, characterized in that: The channel branch is also used for: Obtaining a first feature map of input, and converting the first feature map into a query vector and a value vector by convolution; After the query vector is fully compressed, a normalization function is used to enhance it; Determine a key vector based on the feature graph, and perform a matrix multiplication operation on the query vector and the key vector; The output value is calculated through the activation function.

3. The method for detecting surface defects of a steel strip according to claim 1, characterized in that: The spatial branch is also used for: Obtaining an input second feature map, and converting the second feature map into a query vector and a value vector by convolution; After compressing the query vector in spatial dimension, a normalization function is used to enhance the query vector; The output value is calculated through the activation function.

4. The method for detecting surface defects of a steel strip according to claim 1, characterized in that: The SPPCSPC layer includes several parallel maximum pooling layers, and the SPPCSPC layer is used to perform data classification on the state information of the strip to be detected.

5. The method for detecting surface defects of a steel strip according to claim 1, characterized in that: The variable convolution layer adjusts the position and shape of the convolution kernel through a sparse attention mechanism to improve the robustness and generalization of the detection model; The deformable convolution layer processes memory access instructions of several channels through a single thread.

6. The method for detecting surface defects of a steel strip according to claim 1, characterized in that: The training process of the detection model includes: Acquire a number of sample data, wherein the sample data reflects status information of the strip steel; After marking the sample data, determining the defect situation corresponding to each sample data; Constructing a training set and a test set based on the sample data; The pre-built primary detection model is trained based on the training set, and the performance of the trained detection model is tested based on the test set.

7. The method for detecting surface defects of a steel strip according to claim 6, characterized in that: The testing of the performance of the trained detection model based on the test set includes: Inputting the test set into the detection model to determine the detection result output by the detection model; The performance of the detection model is evaluated based on the performance parameters of the detection results corresponding to the test set, and the performance parameters include precision, recall rate, and average precision mean.

8. The method for detecting surface defects of a steel strip according to claim 6, characterized in that: The training process of the detection model also includes: The generalization ability of the detection model is evaluated by a loss function, wherein the loss function includes a weighted intersection-over-union function; The weighted intersection-over-union function improves the authenticity of the evaluation process through a region alignment mechanism and a weighting mechanism.

9. Strip steel surface defect detection system, characterized in that: include: An information acquisition module, used for acquiring status information of the steel strip to be detected, wherein the status information at least includes a surface image of the steel strip to be detected; A model detection module, used for inputting the state information into a pre-built detection model for defect detection, wherein the detection model is constructed by fusing an improved spatial pyramid module and a polarization self-attention mechanism on the basis of a Yolov8 model, wherein the improved spatial pyramid module includes an SPPCSPC layer and a variability convolution layer, and the polarization self-attention mechanism includes a channel branch and a spatial branch, and the output of the polarization self-attention mechanism is obtained by parallel fusion of the outputs of the channel branch and the spatial branch; The defect determination module is used to determine the surface defects of the strip to be detected based on the detection information output by the defect detection model.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for detecting surface defects of a steel strip as described in any one of claims 1 to 8 is implemented.

Images