Lightweight surface defect detection method and system, medium and terminal

Abstract

The invention discloses a lightweight surface defect detection method and system, a medium and a terminal, and the method comprises the steps: obtaining a data set of surface defects, and carrying out the preprocessing of a surface defect image sample in the data set; inputting the preprocessed surface defect image sample into a double-pooling mechanism backbone network for feature extraction to obtain a plurality of semantic information feature maps of different scales; inputting a plurality of semantic information feature maps with different scales into the improved progressive feature pyramid neck network for feature fusion to obtain fused features; and inputting the fused features into a head network for surface defect detection, and outputting a detection result. According to the method, the low-contrast surface camouflage defect can be identified with high precision and high efficiency in an industrial scene.

Description

Technical Field

[0001] The present invention relates to the technical field of surface defect detection, and in particular to a lightweight surface defect detection method, system, medium and terminal. Background Art

[0002] In recent years, with the rapid development of my country's industrial sheet metal industry, and the continued growth in demand for steel products, the industry's requirements for steel quality and appearance have become increasingly stringent. In particular, during the production of sheet steel, surface quality directly impacts subsequent processing and the performance of the final product. Detecting surface defects remains a crucial step in ensuring product quality and improving production efficiency. However, due to the thin nature of sheet steel, the rapid rolling process can easily lead to defects. Firstly, due to the extremely high temperatures experienced during hot rolling and the metal's high plasticity, the sheet steel surface is susceptible to cracks and scratches due to uneven roll gaps and roll surface damage. Secondly, friction and collisions between the steel surface and air, water, or equipment during rapid rolling can cause surface irregularities and scratches. Furthermore, factors such as uneven molten steel composition and unstable charge quality can also lead to defects such as cracks and inclusions in the metal during rolling. In high-temperature environments, the surface of sheet steel easily absorbs impurities such as dust and oil, which can further damage the surface and even affect subsequent processing or coating processes.

[0003] However, although deep learning has shown great potential in defect detection, it still faces some challenges in its application in the detection of surface defects on steel strips. First, low-contrast defects on the surface of steel strips are often similar to factors such as the brightness and color of the background. This makes it difficult for deep learning models to fully identify these features during training, resulting in insufficient sensitivity of the model to low-contrast defects. At the same time, the intra-class differences in surface defects of steel strips are large, while the inter-class differences are small. That is, defects of the same type may show significant differences due to factors such as lighting and angle, while defects of different types may have similar appearance features, which further increases the difficulty of model classification. In addition, the morphological diversity of defects, large scale differences, and complex background interference also pose great challenges to the detection task, requiring the model to have both strong feature representation capabilities and anti-interference robustness.

[0004] Secondly, while deep learning can handle complex defect types, striking a balance between speed and accuracy remains a challenge. Especially on high-speed production lines, achieving both real-time performance and high accuracy requires model optimization and adjustment. In actual deployment, the number of parameters and computational complexity of deep learning models can be limiting factors, especially when these models are deployed on hardware platforms with limited computing resources, such as embedded devices. The large number of model parameters and high computational complexity can lead to insufficient processing power, impacting real-time response and detection efficiency. In this context, optimizing the model's parameters and computational complexity while ensuring accuracy, reducing computing resource consumption, and improving inference speed are key considerations for deep learning defect detection systems. Summary of the Invention

[0005] In response to the deficiencies in the background technology, the present invention provides a lightweight surface defect detection method, system, medium and terminal, wherein the method solves the problem that the existing technology is difficult to simultaneously meet the requirements of high-precision and high-efficiency identification of low-contrast surface defects in industrial scenarios.

[0006] In a first aspect, the present invention provides a lightweight surface defect detection method, comprising:

[0007] S1: Obtain a dataset of surface defects and preprocess image samples of surface defects in the dataset;

[0008] S2: The pre-processed surface defect image samples are input into the dual-pooling mechanism backbone network for feature extraction to obtain multiple semantic information feature maps of different scales;

[0009] S3: Multiple semantic information feature maps of different scales are used as input and input into the improved progressive feature pyramid neck network for feature fusion to obtain the fused features;

[0010] S4: Input the fused features into the head network for surface defect detection and output the detection results.

[0011] Furthermore, the dual-pooling mechanism backbone network in S2 includes a backbone input layer, a hybrid mobile inverse bottleneck convolution module and a mobile inverse bottleneck convolution module;

[0012] The hybrid moving inverse bottleneck convolution module includes multiple serial hybrid moving inverse bottleneck convolutions; wherein, the hybrid moving inverse bottleneck convolution extracts features in the shallow network through 3×3 convolution, the SE attention module performs channel attention weighting, and then uses 1×1 convolution to compress the number of channels and reduce the dimension. Finally, the output and the original features are added together in combination with random depth to obtain shallow features;

[0013] The mobile inverse bottleneck convolution module includes multiple serial mobile inverse bottleneck convolutions, each of which outputs a semantic feature information feature map of the corresponding scale; wherein, the mobile inverse bottleneck convolution first expands the number of channels of the feature image initially extracted by the backbone input layer by 1×1 convolution, then extracts features through 3×3 depth convolution, the SE attention module performs channel attention weighting, and then uses 1×1 convolution to compress the number of channels and reduce the dimensionality. Finally, the output and the original features are added in combination with random depth to obtain the semantic information feature map.

[0014] Furthermore, the SE attention module in the moving inverse bottleneck convolution and hybrid moving inverse bottleneck convolution is to combine average pooling and maximum pooling, process the results of the two pooling methods through the shared layer perceptron respectively, and then merge them by element-by-element summation, and finally generate the channel attention weight through the Sigmoid function, specifically:

[0015] s c =σ(out1+out2)

[0016]

[0017] Among them, s c is the attention weight of channel c; σ is the Sigmoid function; out1 is the local feature; out2 is the global feature; θ1 and θ2 are both weights representing the fully connected layer; δ is the activation function; is the global maximum pooling result of channel c; is the global average pooling result of channel c; x c (i, j) is the value of the feature map at position (i, j); H is the height of the feature map; W is the width of the feature map; max i,j Indicates taking the maximum value in the spatial dimension.

[0018] Furthermore, the improved progressive feature pyramid neck network includes a basic fusion module, a segmentation fusion module and a segmentation scale enhancement module;

[0019] The basic fusion module performs feature fusion operation on multiple semantic information feature maps obtained by S2 by upsampling or downsampling, and then performs convolution operation to obtain the basic fused feature map. The specific operation is:

[0020]

[0021] in, Feature map for basic fusion; is the feature vector of the feature map at (i, j); n is the number of semantic information feature layers;

[0022] The segmentation and fusion module divides the basic fusion feature map into s feature subsets, and the i-th feature subset is recorded as X i (i∈{1,2,...,s}), except X1, the feature subset X i It will be combined with the output Y of the previous convolution operation i-1 After addition, convolution operation K i , get the multi-granularity feature representation Y i ; where each feature subset X i Keep the same spatial dimension as the feature map of the base fusion, but reduce the number of channels to 1 / s;

[0023] The segmentation scale enhancement module connects the multi-granularity feature representation obtained by the segmentation fusion module with convolution in series, and adds the result to the feature map of the basic fusion through residual connection to obtain the fused feature.

[0024] Furthermore, the specific process of S4 is as follows:

[0025] S41: Input the fused features obtained in S3 into the ghost shuffled convolution decoupling head to obtain the surface defect bounding box classification and regression task;

[0026] S42: Considering the overlapping area, center point distance and aspect ratio of the bounding box, a CIoU loss function is constructed to optimize the bounding box regression task;

[0027] S43: Based on the results of the CIoU loss function, the non-maximum suppression algorithm is used to process the predicted bounding boxes in the model inference stage, redundant detection boxes are removed, and the retained surface defect detection results are used as the optimal detection results.

[0028] Furthermore, the specific operation of the ghost shuffle convolution decoupling head is: the fused features obtained by S3 are subjected to channel dimensionality reduction through 1×1 convolution, and then passed through two parallel ghost shuffle convolution branches. The first branch performs classification through 1×1 convolution, and the second branch obtains the target box regression and confidence values ​​through different 1×1 convolutions.

[0029] Furthermore, the ghost shuffled convolution branch combines the feature information of standard convolution and depthwise separable convolution through channel splicing, and obtains depthwise separable convolution generated features that are evenly permeated with standard convolution generated information through shuffling operation.

[0030] In a second aspect, the present invention provides a lightweight surface defect detection system, comprising:

[0031] Data preprocessing module: used to obtain a data set of surface defects and preprocess the surface defect image samples in the data set;

[0032] Multi-scale semantic information feature map acquisition module: used to input the preprocessed surface defect image samples into the dual-pooling mechanism backbone network for feature extraction, and obtain multiple semantic information feature maps of different scales;

[0033] Feature fusion module: It is used to take multiple semantic information feature maps of different scales as input, input them into the improved progressive feature pyramid neck network for feature fusion, and obtain the fused features;

[0034] Surface defect result prediction module: used to input the fused features into the head network for surface defect detection and output the detection results.

[0035] In a third aspect, the present invention provides an electronic terminal comprising a processor and a memory, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of the above method.

[0036] In a fourth aspect, the present invention provides a readable storage medium storing a computer program, which is used by a processor to execute the steps of the method described above.

[0037] Beneficial effects

[0038] The present invention proposes a lightweight surface defect detection method, system, medium, and terminal. The method enhances the ability to extract texture and edge information of low-contrast defects, enriching feature expression and thus better distinguishing defect areas from background areas. The improved progressive feature pyramid structure achieves complementary fusion of shallow and deep features, builds a diverse receptive field, and improves the ability to extract multi-scale features, effectively solving the information loss or degradation problem that exists in the traditional feature pyramid network during the feature fusion process. The decoupled detection head based on ghost shuffle convolution reduces the computational complexity of the detection head, retains hidden connections between channels, and achieves improved classification and positioning accuracy. At the same time, it avoids mutual interference between tasks and improves detection performance. In addition, compared with existing defect detection algorithms, the present invention excels in lightweight design, greatly reduces the number of model parameters and computational complexity, significantly improves detection speed, and enables efficient operation on resource-constrained embedded devices or low-spec GPUs, better meeting the strict requirements of industrial sites for real-time performance and resource utilization efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0040] Figure 1 is a flow chart of a lightweight surface defect detection method provided by an embodiment of the present invention;

[0041] Figure 2 A schematic diagram of the overall model structure of a lightweight surface defect detection method provided by an embodiment of the present invention;

[0042] Figure 3 Schematic diagram of moving inverse bottleneck convolution in the lightweight surface defect detection method provided by an embodiment of the present invention;

[0043] Figure 4 Schematic diagram of hybrid moving inverse bottleneck convolution in the lightweight surface defect detection method provided by an embodiment of the present invention;

[0044] Figure 5 Schematic diagram of the improved SE attention module in the lightweight surface defect detection method provided by an embodiment of the present invention;

[0045] Figure 6 A schematic diagram of a segmentation and fusion module in a lightweight surface defect detection method provided by an embodiment of the present invention;

[0046] Figure 7 A schematic diagram of a segmentation scale enhancement module in a lightweight surface defect detection method provided by an embodiment of the present invention;

[0047] Figure 8 Schematic diagram of a coupling head and a decoupling head in a lightweight surface defect detection method provided by an embodiment of the present invention;

[0048] Figure 9 Schematic diagram of ghost shuffle convolution in the lightweight surface defect detection method provided by an embodiment of the present invention;

[0049] Figure 10 This is a graph showing the results of using SSD300 to detect surface defects on a steel strip, provided in an embodiment of the present invention;

[0050] Figure 11 A graph showing the results of detecting surface defects of steel strips using RTMDet provided in an embodiment of the present invention;

[0051] Figure 12 A graph showing the results of using YOLOv8n to detect surface defects on steel strips according to an embodiment of the present invention;

[0052] Figure 13 A graph showing the results of using YOLOv11n to detect surface defects on steel strips according to an embodiment of the present invention;

[0053] Figure 14A result diagram of detecting surface defects of steel strips using the lightweight surface defect detection method provided by the present invention is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of the present invention more apparent, the technical solutions of the present invention will be described in detail below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other implementations obtained by those of ordinary skill in the art without inventive effort are within the scope of protection of the present invention.

[0055] Example 1

[0056] like Figure 1 As shown, this embodiment provides a lightweight surface defect detection method that can identify surface defects on different industrial sheets without limiting the type of industrial sheets. This embodiment uses the detection of surface defects on plate and strip steel as an example to illustrate the method, which includes:

[0057] S1: Obtain a dataset of surface defects of steel strips and perform preprocessing on image samples of surface defects of steel strips in the dataset.

[0058] Specifically, the data set of surface defects of plate and strip steel can be selected according to the specific implementation situation and is not limited thereto. In this embodiment, the NEU-DET data set is selected as the data set of surface defects of plate and strip steel, and the data set is divided into a training set and a validation set, and the image data in the data set is preprocessed, and the preprocessed data is input into the network for training. Among them, the preprocessing includes data cleaning, standardization, data enhancement and other operations on the image data in the data set. The network constructed in this embodiment is a lightweight dual-pooled progressive decoupling network model, which is composed of a backbone network, a neck network and a head network. In order to further elaborate on the network model, the relevant introduction of the convolutional neural network model is introduced:

[0059] The i-th layer of the convolutional neural network is defined as the function:

[0060] Y i =F i (X i )

[0061] Among them, F i is the operation mode, where the operation mode includes upsampling, downsampling, convolution operation, etc.; X i and Y i The input tensor and output tensor are respectively, and the shape of the tensor is (H i ,W i ,C i), H i 、W i 、C i Represent the height, width, and number of channels of the feature map respectively. A convolutional neural network N can be represented by a combination of K layers:

[0062]

[0063] Convolutional neural networks are usually divided into multiple stages, and the layer structure within each stage is the same. A convolutional neural network is defined as follows:

[0064]

[0065] in, Represents the number of times the convolutional layer F is stacked in stage i; X is the input tensor.

[0066] In order to find the optimal configuration of the network, all layers are required to be scaled in the same proportion and maximize the accuracy of the model while meeting resource constraints. The problem can be described as follows:

[0067]

[0068] Memory(N)≤target_memory

[0069] FLOPS(N)≤target_flops

[0070] Among them, d, w, r are the coefficients used to scale the network depth, width and resolution respectively, and F i , L i , H i , W i , C i These are the preset baseline network parameters.

[0071] The composite coefficient φ is used to uniformly adjust the depth d, width w and resolution r of the network at the same time:

[0072] depth:d=α φ

[0073] width:w=β φ

[0074] resolution:r=γ φ

[0075] stα·β 2 γ 2 ≈2

[0076] α≥1,β≥1,γ≥1.

[0077] S2: The preprocessed image samples of the surface defects of the steel strip are input into the dual-pooling mechanism backbone network for feature extraction to obtain multiple semantic information feature maps of different scales.

[0078] Specifically, the dual-pooling mechanism backbone network includes a backbone input layer (Stem), a mobile inverse bottleneck convolution module, and a hybrid mobile inverse bottleneck convolution module. In specific implementation, the dual-pooling mechanism backbone network in this embodiment uses the EfficientNetV2 network. The combination of the two building blocks, the mobile inverse bottleneck convolution module and the hybrid mobile inverse bottleneck convolution module, can be adjusted according to actual conditions and is not limited. In this embodiment, through neural network architecture search, the hybrid mobile inverse bottleneck convolution module uses 5 groups of serial hybrid mobile inverse bottleneck convolutions (FusedMBConv). The mobile inverse bottleneck convolution module includes 6 groups of mobile inverse bottleneck convolutions (MBConv), and 3 outputs of different scales in the 6 groups of mobile inverse bottleneck convolutions are selected as the input of the neck network. The image features of the surface defect pictures of the plate and strip steel are efficiently extracted through the EfficientNetV2 network, and the dual-pooling mechanism is introduced to enhance the expression ability of the channel attention to maintain a low computational overhead and achieve lightweight model.

[0079] The hybrid mobile inverse bottleneck convolution module includes multiple serial hybrid mobile inverse bottleneck convolutions; Figure 4 As shown, the hybrid moving inverse bottleneck convolution extracts features in shallow networks through 3×3 convolutions. The SE attention module (SE+) performs channel attention weighting, and then uses 1×1 convolution to compress the number of channels and reduce the dimensionality. Finally, the output and the original features are added together in combination with random depth to obtain shallow features. Different from the existing technology, this embodiment replaces the moving inverse bottleneck convolution with 3×3 convolution, thereby better enabling feature extraction in shallow networks.

[0080] The mobile inverse bottleneck convolution module includes multiple serial mobile inverse bottleneck convolutions, each mobile inverse bottleneck convolution outputs a semantic feature information feature map of the corresponding scale; Figure 3 As shown in the figure, the mobile inverse bottleneck convolution first expands the number of channels of the feature image initially extracted by the backbone input layer by 1×1 convolution, then extracts features through 3×3 depth convolution, and the SE attention module (SE+) performs channel attention weighting. Finally, 1×1 convolution is used to compress the number of channels and reduce the dimensionality. Finally, the output and the original features are added together with random depth to obtain the semantic information feature map. Figure 3 The Dropout in represents the random depth, which means that the module trunk may be discarded and only the shortcut branch is left, thereby preventing overfitting and improving the robustness of the model.

[0081] The core idea of ​​the SE module is to compress the two-dimensional spatial information into a one-dimensional channel descriptor through the combination of average pooling and fully connected layers, and dynamically adjust the weight of each channel to enhance the feature response of important channels while suppressing unimportant channels. The mathematical expression is as follows:

[0082]

[0083] s c =σ(θ2·δ(θ1·z c ))

[0084] in represents the global average pooling result of channel c; H and W represent the height and width of the feature map respectively; x c (i, j) represents the value of the feature map at position (i, j); θ1 and θ2 represent the weights of the fully connected layer respectively; δ represents the activation function such as ReLU, and σ represents the Sigmoid function.

[0085] In this embodiment, maximum pooling is introduced on the basis of the SE module to enhance the expression ability of local features. The mathematical expression of global maximum pooling is as follows:

[0086]

[0087] in represents the global maximum pooling result of channel c; x c (i, j) is the value of the feature map at position (i, j); max i,j Indicates the maximum value in the spatial dimension. Global maximum pooling can capture significant local features in the feature map, and is particularly suitable for extracting details such as edges and textures. The results of the two pooling methods are processed by a shared multi-layer perceptron (Shared MLP) respectively, and then they (that is, the two pooling results are processed by SharedMLP respectively to obtain Out1 and Out2, as shown in Figure 2). Figure 5 The dual pooling mechanism captures both global and local features. The mathematical expressions for the two pooling outputs after processing by the shared multilayer perceptron (SharedMLP) are as follows:

[0088]

[0089] Among them, θ1 and θ2 are weights representing the fully connected layer; δ is an activation function such as ReLU; is the global maximum pooling result of channel c; is the global average pooling result of channel c.

[0090] Add the two pooling outputs and generate channel attention weights through the Sigmoid function:

[0091] s c =σ(out1+out2)

[0092] Among them, s c is the attention weight of channel c; σ is the Sigmoid function; out1 is the local feature; out2 is the global feature.

[0093] The improved SE module is as follows Figure 5 As shown in the figure, through the dual pooling mechanism, the network can simultaneously capture global and local features, enhancing the ability to extract texture and edge information of surface defects in steel strips. The maximum pooling feature can encode the most significant part of the information, which can usually make up for the global statistical information encoded by the average pooling feature.

[0094] S3: Multiple semantic information feature maps of different scales are used as input and input into the improved progressive feature pyramid neck network for feature fusion to obtain the fused features.

[0095] Specifically, the improved progressive feature pyramid neck network includes a basic fusion module (BasicFusion), a segmentation fusion module (SFM), and a segmentation scale enhancement module (SSEB). The improved progressive feature pyramid neck network can reduce the semantic gap between non-adjacent levels and further extract multi-scale defect features by combining multi-scale residual learning.

[0096] The backbone network consists of multiple hybrid moving inverse bottleneck convolutions and moving inverse bottleneck convolutions. The output of each hybrid moving inverse bottleneck convolution and each moving inverse bottleneck convolution has a feature map size of different sizes. As the network goes deeper, the layers of features obtained become deeper. Therefore, by integrating shallow features of different scales and using the integration result as the starting point of the fusion process, deeper features are gradually introduced into the fusion process as the network layers progress, until the deep features at the top layer of the network are integrated. This progressive fusion mechanism effectively alleviates the semantic gap problem between non-adjacent layers.

[0097] The adaptive spatial feature fusion operation is to adjust the level of each feature map from n to l by upsampling or downsampling and then performing a convolution operation. In this embodiment, three groups of outputs of different scales from the six groups of moving inverse bottleneck convolutions are selected as the input of the neck network. Therefore, n is a maximum of 3. The specific operation of fusion is performed in combination with the corresponding weights as follows:

[0098]

[0099] Where, Indicates that the feature map of level n at pixel (i, j) is converted to a feature vector of level l; (i, j) is the two-dimensional spatial coordinate on the feature map, corresponding to a specific position in the image; l is the level of the feature map, representing the feature layers of different scales or depths in the network; n→l means adjusting the feature map of layer n to the scale of layer l for cross-layer feature fusion; and are the weights of three different levels of feature maps corresponding to l, which are shared across all channels and satisfy and Use 1×1 convolution from x 1→l 、x 2→l and x 3→l Get the mapping and The result is obtained through the Softmax layer (that is, a 1×1 convolution is used on the feature map to obtain a mapped weight, which is Then combine the weights to get ). The adaptive spatial feature fusion operation consumes too many resources. Therefore, in this embodiment, the basic fusion module performs a feature fusion operation on the multiple semantic information feature maps obtained by S2 by upsampling or downsampling and then performing a convolution operation to obtain a basic fused feature map. Specifically, the weight of the fused part is set to a fixed average weight. After modification, the ability of multi-scale feature fusion can be maintained and the computational complexity can be reduced. The formula for basic fusion is as follows:

[0100]

[0101] in, Feature map for basic fusion; is to convert the feature map of level n at pixel (i, j) into a feature vector of level l; n is the number of semantic information feature layers.

[0102] The segmentation and fusion module divides the basic fusion feature map into s feature subsets, and the i-th feature subset is recorded as X i (i∈{1,2,...,s}), except X1, the feature subset X i It will be combined with the output Y of the previous convolution operation i-1 After addition, convolution operation K i , get the multi-granularity feature representation Y i ; where each feature subset X i The spatial dimension is kept the same as that of the basic fusion feature map, but the number of channels is reduced to 1 / s. The convolution in this embodiment uses 3×3, and the specific process of the segmentation and fusion module is as follows: Figure 6 As shown. The formula for the multi-granularity feature representation obtained by the segmentation fusion module is:

[0103]

[0104] With this design, each 3×3 convolution operation K i All feature subsets {X j ,j≤i}. With the feature subset X j Through sequential 3×3 convolution operations, the receptive field of the output features will gradually expand. By grouping the input feature maps and performing progressive convolution layer by layer, the segmentation fusion module generates multi-granularity feature representations that can capture contextual information at different scales.

[0105] The segmentation scale enhancement module connects the multi-granularity feature representation obtained by the segmentation fusion module with convolution in series, and adds the result to the basic fusion feature map through residual connection to obtain the fused feature. While ensuring low computational complexity, it effectively improves the network's ability to fuse multi-scale features. In specific implementation, the convolution size in series in this embodiment is 3×3, and the specific process of the segmentation scale enhancement module is as follows: Figure 7 shown.

[0106] S4: Input the fused features into the head network to detect surface defects of steel strips and output the detection results. The specific process is as follows:

[0107] S41: Input the fused features obtained in S3 into the ghost shuffled convolution decoupling head (GSCDHead) to obtain the surface defect bounding box classification and regression task of the strip steel.

[0108] like Figure 8 In the traditional coupling head shown above, the coupling head only performs a 1×1 convolution on the input features and then directly performs classification and regression tasks.

[0109] Figure 8 The basic structure of the decoupling head is shown below. Features are first reduced in channel dimension through convolution operations. Then, they pass through two parallel convolution branches. The upper branch performs classification through convolution, while the lower branch uses different convolution operations to obtain object bounding box regression and confidence values. The classification task requires capturing the semantic information of the object, while the regression task requires accurate prediction of the object's bounding box. By decoupling these two tasks, the decoupling head can better adapt to the needs of different tasks, thereby improving detection performance. However, traditional decoupling heads typically have a high number of parameters and computational complexity due to the 3×3 convolutions deep in the network. This is especially computationally expensive when processing high-resolution feature maps.

[0110] In order to further optimize the performance of the network, this embodiment uses ghost shuffle convolution for optimization, such as Figure 9As shown in the figure, the core design idea is to combine the advantages of standard convolution, depthwise separable convolution, and shuffle operations to reduce the computational complexity of the model while maintaining high accuracy. Compared to depthwise separable convolution, ghost shuffle convolution can better preserve the hidden connections between channels by introducing standard convolution and channel shuffle operations, making the output of ghost shuffle convolution as close as possible to standard convolution, reducing computational cost.

[0111] Specifically, the ghost shuffled convolution decoupling head operates as follows: the fused features obtained from S3 are subjected to channel dimensionality reduction through 1×1 convolution, and then pass through two parallel ghost shuffled convolution branches. The first branch performs classification through 1×1 convolution, and the second branch obtains target bounding box regression and confidence values ​​through different 1×1 convolutions. The ghost shuffled convolution branch combines the feature information of standard convolution and depthwise separable convolution through channel splicing, and through the shuffling operation, obtains depthwise separable convolution-generated features that are evenly permeated with standard convolution-generated information.

[0112] As for the computational cost, assuming that the size of the input feature map is C1×W1×H1, the size of the output feature map is C2×W2×H2, and the size of the convolution kernel is K1×K2, then the parameter amount of the standard convolution P1=C1×C2×K1×K2, the parameter amount of the ghost shuffle convolution P2=C1×(C2 / 2)×K1×K2+(C2 / 2)×K1×K2, then the parameter ratio is:

[0113]

[0114] Regarding the computational complexity, the computational complexity of standard convolution is Q1=W2×H2×C1×C2×K1×K2, and the computational complexity of ghost shuffle convolution is Q2=W2×H2×C1×(C2 / 2)×K1×K2+W2×H2×(C2 / 2)×K1×K2. The ratio of floating-point computational complexity is:

[0115]

[0116] In summary, the cost of ghost shuffled convolution is about 50% of that of standard convolution, but its contribution to model learning ability is as close as possible to that of standard convolution. In this way, ghost shuffled convolution can maximize the connection between channels and retain semantic information while maintaining low computational complexity, thereby effectively improving model accuracy.

[0117] S42: Considering the overlapping area, center point distance and aspect ratio of the bounding box, a CIoU loss function is constructed to optimize the bounding box regression task.

[0118] S43: Based on the results of the CIoU loss function, the non-maximum suppression algorithm is used to process the predicted bounding box in the model inference stage, redundant detection boxes are removed, and the retained plate and strip surface defect detection results are used as the optimal detection results.

[0119] In order to highlight the superiority of this embodiment, four other popular and commonly used lightweight methods (SSD300, RTMDet, YOLOv8n, and YOLOv11n) are introduced as a comparison group. The specific comparison is as follows.

[0120] As summarized in Table 1, the quantitative evaluation results of different methods were all retrained under the same experimental environment. As can be seen from the table, this embodiment achieved a maximum mAP of 81.3%, and the FPS was still relatively high at 67.5. This shows that the dual-pooled EfficientNetV2 backbone network, while reducing the computational complexity of the model, enhances channel attention and effectively improves the ability to extract low-contrast texture and edge features. The improved progressive feature pyramid can complement the fusion of the spatial information of shallow features and the semantic information of deep features, and construct a diverse receptive field to further optimize the fusion capability of multi-scale features. In addition, the decoupled detection head based on ghost shuffled convolution greatly reduces the computational overhead of the detection head and enhances the feature interaction capability, achieving simultaneous optimization of classification and positioning performance.

[0121] Table 1. Quantitative evaluation results of different methods.

[0122]

[0123] Table 2 Parameters and calculations of different methods.

[0124]

[0125] A detailed comparative analysis of the parameters and computational complexity of four other algorithms was conducted. The results are shown in Table 2. The data in the table show that this embodiment demonstrates extremely high efficiency and lightweightness among all algorithms, with the lowest parameter count of 2.5M and computational complexity of 5.0. Furthermore, this comparison shows that the method described in this embodiment significantly reduces model resource consumption while maintaining high detection accuracy. This allows the method described in this embodiment to run efficiently on devices with limited memory and computing resources, thereby better meeting the stringent real-time and resource efficiency requirements of industrial sites.

[0126] Please refer to the quantitative results Figure 10-14, the results of the present invention were compared with those of SSD300, RTMDet, YOLOv8n, and YOLOv11n. For cracks, the RTMDet and YOLOv8n algorithms have the problems of missed detection and too many detection area frames, respectively. For inclusions, SSD300, RTMDet, and YOLOv11n all have missed detections. It is worth noting that there is an inclusion defect in the upper right corner of the plaque. The positioning and classification of such mixed defects is a challenge for most detection algorithms because they need to process defect types with different shapes and characteristics at the same time. However, the method described in this embodiment successfully classified and located the plaques and inclusions, which shows that the method described in this embodiment has stronger feature extraction capabilities and robustness when processing complex scenes. In addition, for pitting surfaces, most models failed to accurately detect the actual boundaries of the defects and produced overly large frames, but the detection frame of the method described in this embodiment can fit the defect edge closely, showing higher detection accuracy and adaptability to complex shapes. For rolled-in scale, all models failed to fully detect the defect on the right, but compared to the defect identified on the left, the method described in this example located it more accurately and with higher confidence. The network described in this example was able to stably identify defects with varying shapes and scales, such as patches and scratches.

[0127] First, the present invention proposes a dual-pooling backbone network, improves the original compression and excitation network, proposes a dual-pooling mechanism, and uses global average pooling and global maximum pooling in parallel to enhance the expressive ability of channel attention.

[0128] Furthermore, an improved progressive feature pyramid structure is proposed to avoid information loss or degradation during multi-stage transmission and reduce the large semantic gaps between non-adjacent levels. A basic fusion module is designed to further improve the efficiency of feature fusion. Furthermore, a segmentation scale enhancement module balances the extraction of global and local information, thereby enhancing the ability to capture multi-scale defect features.

[0129] Finally, we propose a decoupling head based on ghost shuffled convolution. This uses deep convolution and channel shuffling to promote information interaction and maximize the preservation of inter-channel connections and semantic information. Applying this to the decoupling head significantly reduces the computational complexity of the detection head and improves classification and localization accuracy.

[0130] Compared with four popular and commonly used lightweight methods, the present invention significantly improves the recognition accuracy of various types of defects on the surface of strip steel while maintaining a high detection speed.

[0131] Example 2

[0132] This embodiment provides a lightweight strip steel surface defect detection system based on dual-pooling progressive decoupling, including:

[0133] Data preprocessing module: used to preprocess image samples of surface defects of steel strips and incorporate the processed image samples into the database;

[0134] Multi-scale semantic information feature map acquisition module: used to input defect image samples into the dual-pooling mechanism backbone network for feature extraction, and obtain multiple semantic information feature maps of different scales;

[0135] Feature fusion module: It is used to take multiple semantic information feature maps of different scales as input, input them into the improved progressive feature pyramid neck network for feature fusion, and obtain the fused features;

[0136] Plate and strip steel surface defect result prediction module: used to input the fused features into the head network for plate and strip steel surface defect detection and output the detection results.

[0137] Example 3

[0138] This embodiment provides an electronic terminal, which includes a processor and a memory, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of the above method.

[0139] Example 4

[0140] This embodiment provides a readable storage medium that stores a computer program, which is used by a processor to execute the steps of the method described above.

[0141] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0142] It should be understood that in the embodiments of the present invention, the processor referred to may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The memory may include a read-only memory and a random access memory, and provides instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0143] The readable storage medium is a computer-readable storage medium, which may be an internal storage unit of the controller described in any of the aforementioned embodiments, such as a hard disk or memory of the controller. The readable storage medium may also be an external storage device of the controller, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the controller. Furthermore, the readable storage medium may also include both an internal storage unit of the controller and an external storage device. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0144] Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned readable storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0145] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A lightweight surface defect detection method, characterized in that: include: S1: Obtain a dataset of surface defects and preprocess the surface defect image samples in the dataset; S2: The preprocessed surface defect image samples are input into the dual-pooling mechanism backbone network for feature extraction to obtain multiple semantic information feature maps of different scales; S3: Multiple semantic information feature maps of different scales are used as input and input into the improved progressive feature pyramid neck network for feature fusion to obtain the fused features; S4: Input the fused features into the head network for surface defect detection and output the detection results.

2. The lightweight surface defect detection method according to claim 1, characterized in that: The dual-pooling mechanism backbone network in S2 includes a backbone input layer, a hybrid mobile inverse bottleneck convolution module and a mobile inverse bottleneck convolution module; The hybrid moving inverse bottleneck convolution module includes multiple serial hybrid moving inverse bottleneck convolutions; wherein, the hybrid moving inverse bottleneck convolution extracts features in the shallow network through 3×3 convolution, the SE attention module performs channel attention weighting, and then uses 1×1 convolution to compress the number of channels and reduce the dimension. Finally, the output and the original features are added together in combination with random depth to obtain shallow features; The mobile inverse bottleneck convolution module includes multiple serial mobile inverse bottleneck convolutions, each of which outputs a semantic feature information feature map of the corresponding scale; wherein, the mobile inverse bottleneck convolution first expands the number of channels of the feature image initially extracted by the backbone input layer by 1×1 convolution, then extracts features through 3×3 depth convolution, the SE attention module performs channel attention weighting, and then uses 1×1 convolution to compress the number of channels and reduce the dimensionality. Finally, the output and the original features are added in combination with random depth to obtain the semantic information feature map.

3. The lightweight surface defect detection method according to claim 2, characterized in that: The SE attention module in the moving inverse bottleneck convolution and hybrid moving inverse bottleneck convolution is to combine average pooling and maximum pooling, process the results of the two pooling methods through the shared layer perceptron respectively, and then merge them by element-by-element summation. Finally, the channel attention weight is generated by the Sigmoid function, which is specifically: s c =σ(out1+out2) Among them, s c is the attention weight of channel c; σ is the Sigmoid function; out1 is the local feature; out2 is the global feature; θ1 and θ2 are both weights representing the fully connected layer; δ is the activation function; is the global maximum pooling result of channel c; is the global average pooling result of channel c; x c (i, j) is the value of the feature map at position (i, j); H is the height of the feature map; W is the width of the feature map; max i,j Indicates taking the maximum value in the spatial dimension.

4. The lightweight surface defect detection method according to claim 1, characterized in that: The improved progressive feature pyramid neck network includes a basic fusion module, a segmentation fusion module and a segmentation scale enhancement module; The basic fusion module performs a feature fusion operation on multiple semantic information feature maps obtained by S2 by upsampling or downsampling and then performing a convolution operation to obtain a basic fused feature map. The specific operations are as follows: Where n→l means adjusting the feature map of layer n to the scale of layer l; (i, j) is the two-dimensional spatial coordinate on the feature map; l is the level of the feature map; Feature map for basic fusion; is the feature vector of the feature map at (i, j); n is the number of feature layers involved in average fusion; The segmentation and fusion module divides the basic fusion feature map into s feature subsets, and the i-th feature subset is recorded as X i (i∈{1,2,...,s}), except X1, the feature subset X i It will be combined with the output Y of the previous convolution operation i-1 After addition, convolution operation K i , get the multi-granularity feature representation Y i ; where each feature subset X i Keep the same spatial dimension as the feature map of the base fusion, but reduce the number of channels to 1 / s; The segmentation scale enhancement module connects the multi-granularity feature representation obtained by the segmentation fusion module with convolution in series, and adds the result to the feature map of the basic fusion through residual connection to obtain the fused feature.

5. The lightweight surface defect detection method according to claim 1, characterized in that: The specific process of S4 is as follows: S41: Input the fused features obtained in S3 into the ghost shuffled convolution decoupling head to obtain the surface defect bounding box classification and regression task; S42: Considering the overlapping area, center point distance and aspect ratio of the bounding box, a CIoU loss function is constructed to optimize the bounding box regression task; S43: Based on the results of the CIoU loss function, the non-maximum suppression algorithm is used to process the predicted bounding boxes in the model inference stage, redundant detection boxes are removed, and the retained surface defect detection results are used as the optimal detection results.

6. The lightweight surface defect detection method according to claim 5, characterized in that: The specific operation of the ghost shuffle convolution decoupling head is as follows: the fused features obtained by S3 are subjected to channel dimensionality reduction through 1×1 convolution, and then passed through two parallel ghost shuffle convolution branches. The first branch performs classification through 1×1 convolution, and the second branch obtains the target box regression and confidence values ​​through different 1×1 convolutions.

7. The lightweight surface defect detection method according to claim 6, characterized in that: The ghost shuffled convolution branch combines the feature information of standard convolution and depth-wise separable convolution through channel splicing, and obtains depth-wise separable convolution generated features uniformly permeated with standard convolution generated information through shuffling operation.

8. A lightweight surface defect detection system, characterized in that: include: Data preprocessing module: used to obtain a data set of surface defects and preprocess the surface defect image samples in the data set; Multi-scale semantic information feature map acquisition module: used to input the preprocessed surface defect image samples into the dual-pooling mechanism backbone network for feature extraction, and obtain multiple semantic information feature maps of different scales; Feature fusion module: It is used to take multiple semantic information feature maps of different scales as input, input them into the improved progressive feature pyramid neck network for feature fusion, and obtain the fused features; Surface defect result prediction module: used to input the fused features into the head network for surface defect detection and output the detection results.

9. An electronic terminal, characterized in that: The method comprises a processor and a memory, wherein the memory stores a computer program, and the processor calls the computer program to execute the steps of the method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: A computer program is stored, and when the computer program is called by a processor, it is used to execute: the steps of the method according to any one of claims 1 to 7.

Images