Steel plate surface defect detection method based on swin-transformer network structure and electronic device

Abstract

The application discloses a steel plate surface defect detection method based on a Swin-Transformer network structure and electronic equipment: a stage is used to extract and output a feature map for image scale characteristics of each layer; a scale feature map is obtained by convolving the feature map with different convolution kernels; a pooling vector is extracted by globally pooling the scale feature map, a fusion scale weight is generated according to the pooling vector, and a fusion feature map is obtained by weighted fusion; the fusion feature map is operated, a classification branch feature is obtained by introducing the fusion scale weight and weighted fusion, and a classification prediction result is obtained by inputting a classification full connection layer; the fusion feature map is globally average-pooled to obtain a global feature representation, the pooling vector is processed, the fusion scale weight is fused to form a regression feature representation, and a regression prediction result is obtained by inputting a regression full connection layer; the classification feature representation and the global feature representation are spliced to generate an interaction weight, and the classification prediction result and the regression prediction result are weighted to obtain a final classification output and a final regression output. The application solves the problem of surface defect detection.

Description

Technical Field

[0001] This invention belongs to the field of industrial automation and computer vision technology, specifically relating to a method and electronic device for detecting surface defects in steel plates based on a Swing-Transformer network structure. Background Technology

[0002] As a key link in industrial quality control, the detection of surface defects in objects has evolved from traditional manual feature extraction to deep learning-based automatic modeling. However, existing methods still have significant shortcomings when dealing with real-world industrial scenarios.

[0003] For example, in steel plate surface defect detection, early methods relying on manually designed textures and shapes had limited generalization ability in scenarios with high similarity between defects and background, varying scales, and complex shapes. In deep learning-based solutions, convolutional neural networks are limited by local convolution operations, making it difficult to effectively capture global context. While visual Transformers possess global modeling capabilities, they suffer from high computational complexity. Consequently, conventional surface defect detection solutions perform poorly in practical applications.

[0004] Therefore, there is an urgent need for a steel plate surface defect detection method and electronic device based on the Swin-Transformer network structure to solve the problem that conventional surface defect detection schemes are not effective in practical applications. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a method and electronic device for detecting surface defects in steel plates based on a Swing-Transformer network structure, thereby solving the problem that conventional surface defect detection schemes are not effective in practical applications.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for detecting surface defects in steel plates based on a Swin-Transformer network structure includes the following steps:

[0008] A hierarchical Swin-Transformer network is constructed, with several stages using window attention mechanism to extract features at different scales from the input steel plate surface image and output feature maps respectively.

[0009] Input each layer The output feature map is convolved with the input feature map using convolution kernels of different sizes to obtain features under different receptive fields and obtain the corresponding scale feature maps.

[0010] Pooling vectors for each scale feature map are extracted using global pooling. Fusion scale weights are generated based on the obtained pooling vectors, and a fused feature map at a unified scale is obtained through weighted fusion.

[0011] The fused feature map is normalized and convolutional to extract local details. A fusion scale weight is introduced to obtain classification branch features. The classification branch features are weighted and fused, and the fused classification feature representation is input into the classification fully connected layer to obtain the classification prediction result.

[0012] Global average pooling is performed on the fused feature map to obtain a global feature representation of the overall contour; perceptual processing is performed on each pooling vector, and combined with the fused scale weights to form a regression feature representation of the perceptual scale distribution; the regression feature representation is input into the regression fully connected layer to obtain the regression prediction result;

[0013] The classification feature representation and the global feature representation are concatenated. Based on the concatenated features, weighting factors for the classification prediction results and regression prediction results are generated and weighted to obtain the adjusted final classification output and final regression output, which are used to determine the surface defects of the steel plate.

[0014] To optimize the above technical solution, the specific measures also include:

[0015] Furthermore, the first layer of the several Stages is Stage1, which consists of convolutional coding and a Swin-Transformer layer. Each of the remaining Stages consists of a stack of a Swin-Transformer block and an attention block, and each attention block contains a window attention mechanism.

[0016] Furthermore, the several stages respectively utilize a window attention mechanism to extract features at different scales from the input steel plate surface image and output feature maps respectively, including the following steps:

[0017] The input image is divided into non-overlapping image patches, and initial feature maps are generated through Stage 1 convolutional encoding and a Swin-Transformer layer. Subsequently, each stage utilizes a window attention mechanism and performs large-scale feature extraction on the feature map output by the previous layer through layer-by-layer downsampling and feature aggregation, as shown in the following formula, and outputs the feature maps for each layer accordingly:

[0018]

[0019] in, Indicates the first Stage For the i-th layer Output feature map.

[0020] Furthermore, the input layer The output feature map is convolved with convolutional kernels of different sizes on the input feature map to obtain features under different receptive fields and obtain corresponding scale feature maps, including the following steps:

[0021] Using convolution kernels of different sizes to process the i-th input feature map Perform convolution to obtain the i-th feature map. Corresponding query ,key ,value matrix:

[0022]

[0023] in, These represent convolution operations with kernels of different sizes.

[0024] Then, the similarity between Q and K is calculated, and the attention weights are obtained through the Softmax function. :

[0025]

[0026] in, Indicator key The dimension of a vector express transpose,

[0027] Finally, attention weights and Perform feature fusion:

[0028]

[0029] Obtain the i-th feature map Scale feature map .

[0030] Furthermore, the step of extracting pooling vectors from feature maps at each scale through global pooling, generating fusion scale weights based on the obtained pooling vectors, and obtaining a fusion feature map at a unified scale through weighted fusion includes the following steps:

[0031] For each scale feature map Perform global pooling separately, extract the global statistical representation of features at each scale, and obtain the corresponding pooling vector. :

[0032]

[0033] in, Represent the feature map at the i-th scale, respectively. Height and width; These represent the feature maps at the i-th scale, respectively. The row and column coordinates in the data; Represents the feature map at the i-th scale In position eigenvectors;

[0034] The output pooling vectors ,pass The functions are concatenated and then input into the multilayer perceptron. Generate corresponding fusion scale weights :

[0035]

[0036] Softmax is used to ensure the fusion scale weights of the output. Functions that satisfy normalization constraints

[0037] The UpSample operation is used to sample each pooling vector. Align to a uniform spatial resolution and perform weighted blending:

[0038]

[0039] Obtain fused feature maps at a uniform scale .

[0040] Furthermore, the step of performing standard normalization and convolution operations on the fused feature map to extract local details, introducing fusion scale weights to obtain classification branch features, performing weighted fusion of the classification branch features, and inputting the fused classification feature representation into the classification fully connected layer to obtain the classification prediction result includes the following steps:

[0041] Standard normalization and convolution operations are performed on the fused feature map to extract local details:

[0042]

[0043] Where Norm represents batch normalization and Conv represents a convolutional layer with enhanced receptive field.

[0044] Introducing fusion scale weights For classification branch features Perform weighted fusion:

[0045]

[0046] Finally, the classification features fused from multiple scales are represented. Input to classification fully connected layer :

[0047]

[0048] Obtain classification prediction results .

[0049] Further, the step of performing a global average pooling operation on the fused feature map to obtain a global feature representation of the overall contour; performing perceptual processing on each pooling vector and combining it with the fused scale weights to form a regression feature representation of the perceptual scale distribution; and inputting the regression feature representation into the regression fully connected layer to obtain the regression prediction result includes the following steps:

[0050] For fused feature maps Perform a global average pooling operation to obtain a global feature representation of the overall contour. :

[0051]

[0052] in, Represents the fused feature map height, 'Represents fused feature map width, Indicates the location of the fused feature map eigenvectors;

[0053] Then, for each pooling vector Perform perception processing and combine fusion scale weights To form a regression feature representation of the perceptual scale distribution :

[0054]

[0055] Where AnchorAware represents the pooling vectors... Perform sensory processing. Indicates the basic fully connected layer;

[0056] Then, the regression features are represented Input regression fully connected layer :

[0057]

[0058] Obtain regression prediction results .

[0059] Furthermore, the step of concatenating the classification feature representation with the global feature representation, generating weighting factors for the classification prediction result and regression prediction result based on the concatenated features, and then weighting them to obtain the adjusted final classification output and final regression output includes the following steps:

[0060] Representing classification features With global feature representation splicing together to form a joint representation :

[0061]

[0062] Then, the base fully connected layer FC is input, and the weighting factors for the classification and regression prediction results are generated using the Softmax function:

[0063]

[0064] in, For classification interaction weights, To regress the interaction weights,

[0065] Final classification prediction results and regression prediction results The adjusted task output is obtained through a weighted method:

[0066]

[0067] Get the final classification output and final regression output It is used by staff to determine and understand the defects in the input images.

[0068] Furthermore, the Swin-Transformer network is trained and optimized using the following loss function:

[0069] For classification, a focus loss function is used. Its definition is:

[0070]

[0071] in, These are the positive and negative sample weight coefficients. An index factor to adjust the level of attention given to samples of varying difficulty;

[0072] In terms of regression, IoU loss is introduced. The expression is:

[0073]

[0074] in, Represents the actual value;

[0075] Simultaneously, a binary cross-entropy loss is introduced. :

[0076]

[0077] in, Indicates the first pooling vectors Confidence component; Indicates and The corresponding N data in the current batch The true binary label corresponding to each data point includes 1 indicating that there is a defect at the corresponding location of the predicted box, and 0 indicating that there is no defect. Indicates the number of training samples in the current batch;

[0078] The total loss function is represented by a weighted combination:

[0079]

[0080] in, and These are the balancing factors for regression and object tasks, respectively.

[0081] An electronic device is characterized by comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the above-described method for detecting surface defects in steel plates based on a Swing-Transformer network structure.

[0082] The beneficial effects of this invention are:

[0083] This invention optimizes the network structure, reduces computational complexity, and preserves the global and local semantic information of defects, thus solving the problem of low computational efficiency of traditional Transformers and meeting the needs of real-time industrial detection.

[0084] This invention enhances the context of scale feature maps, effectively capturing long-distance dependencies between defects at different scales, improving multi-scale adaptability, and reducing missed detections of small targets and positioning errors of large targets.

[0085] This invention implements a dynamic scale weight allocation and cross-task interaction module, enabling adaptive fusion of multi-scale features and dynamic collaboration between classification and regression tasks, thereby improving the accuracy of defect detection.

[0086] This invention improves overall detection performance by jointly optimizing end-to-end loss functions, taking into account losses from classification, regression, and object detection, and optimizing the parameters of each module through joint training. Attached Figure Description

[0087] Figure 1 This is a schematic diagram of the process for a steel plate surface defect detection method based on a Swin-Transformer network structure proposed in this invention.

[0088] Figure 2 This diagram illustrates a comparison of the accuracy of the steel plate surface defect detection method based on the Swin-Transformer network structure proposed in this invention with other mainstream tracking algorithms. Detailed Implementation

[0089] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0090] As attached Figure 1 As shown in the figure, a method for detecting surface defects of steel plates based on a Swin-Transformer network structure according to an embodiment of the present invention includes the following steps:

[0091] A hierarchical Swin-Transformer network is constructed, with several stages utilizing a window attention mechanism to extract features at different scales from the input steel plate surface image and outputting feature maps respectively. A multi-scale Criss-Cross attention mechanism is also constructed, with each stage acting as a input to... The output feature map is convolved with convolutional kernels of different sizes on the input feature map to obtain features under different receptive fields and obtain corresponding scale feature maps. Global pooling is then performed on each scale feature map to extract the pooling vector of each scale feature map. An adaptive task decoupling head (ATD) is then used. The first step involves generating fusion scale weights based on the obtained pooling vectors. Weighted fusion is then used to obtain a fusion feature map at a unified scale. Standard normalization and convolution operations are performed on the fusion feature map to extract local details. Fusion scale weights are introduced to obtain classification branch features. These classification branch features are then weighted and fused, and the resulting classification feature representation is input into a fully connected classification layer to obtain the classification prediction result. Global average pooling is then performed on the fusion feature map to obtain a global feature representation of the overall contour. Anchor-Aware modules are designed to perform perceptual processing on each pooling vector. The fusion scale weights are integrated into the Anchor-Aware model to form a regression feature representation with a perceptual scale distribution. This regression feature representation is input into a fully connected regression layer to obtain the regression prediction result. Finally, a collaborative interaction module for classification and regression tasks is constructed. This module uses feature concatenation and attention weighting between the classification feature representation and the global feature representation to dynamically generate interaction weights based on the coupling degree between tasks. The classification and regression prediction results are then weighted to obtain adjusted final classification and regression outputs, which are used to determine the surface defects of the steel plate.

[0092] The above-described construction of a hierarchical Swin-Transformer network as the backbone structure includes the following steps:

[0093] The first layer of the multi-stage is Stage1, which consists of convolutional coding and a Swing-Transformer layer. Each of the remaining stages consists of a Swing-Transformer block and an attention block stacked together. Each attention block contains a window attention mechanism and a Feed-Forward Network (FFN).

[0094] Each of the aforementioned stages utilizes a window attention mechanism to extract features at different scales from the input steel plate surface image and outputs feature maps accordingly, including the following steps:

[0095] The input image is divided into non-overlapping image patches, and initial feature maps are generated through Stage 1 convolutional encoding and a Swin-Transformer layer. Subsequently, a window attention mechanism is used to model the output feature map of the previous stage. At the same time, through layer-by-layer downsampling and feature aggregation, small-scale local details are gradually expanded into larger-scale hierarchical features. Through layer-by-layer accumulation and expansion, feature maps of small defects and overall defect regions in the image are extracted simultaneously. The forward propagation form of the extraction stage is as follows:

[0096]

[0097] in, Indicates the first Stage For the i-th layer Output feature map.

[0098] Specifically, the Swin-Transformer backbone network consists of four stages. The second to fourth stages correspond to the extraction of local details at a small scale, the contours of defects at a medium scale, and the overall region at a large scale, respectively, thereby achieving multi-scale feature representation. The forward propagation form of the extraction stage is as follows:

[0099] ;

[0100] The output feature maps decrease in resolution layer by layer, achieving multi-scale defect representation from local to global.

[0101] The above-described multi-scale Criss-Cross attention mechanism is constructed by inputting each layer. The output feature map is convolved with convolutional kernels of different sizes on the input feature map to obtain features under different receptive fields and obtain corresponding scale feature maps, including the following steps:

[0102] Each layer Output feature map The multi-scale Criss-Cross attention module is input separately, and convolutional kernels of different sizes are used to process the i-th input feature map. Perform convolution to obtain the i-th feature map. Corresponding query ,key ,value matrix:

[0103]

[0104] in, These represent convolution operations with kernels of different sizes.

[0105] Specifically:

[0106] ;

[0107] in, These represent convolution operations with kernel sizes of 5, 3, and 1, respectively.

[0108] Then, the similarity between Q and K is calculated, and the attention weights are obtained through the Softmax function. :

[0109]

[0110] in, Indicator key The dimension of a vector express transpose,

[0111] Finally, attention weights and Perform feature fusion:

[0112]

[0113] Obtain scale feature map .

[0114] Specifically, the feature map output in step one After passing through a multi-scale Criss-Cross attention module, a new feature map representation is obtained. By leveraging features from different receptive fields, the ability to perceive defect regions at multiple scales is enhanced. In this way, long-distance dependencies are calculated in the horizontal and vertical directions of the feature map, thereby improving the ability to model complex defect shapes and contextual information.

[0115] The above-mentioned process involves performing global pooling on feature maps at each scale, extracting pooling vectors for each scale, using an Adaptive Task Decoupling Head (ATD Head), generating fusion scale weights based on the obtained pooling vectors, and then performing weighted fusion to obtain a fused feature map at a unified scale. This process includes the following steps:

[0116] Feature maps at various scales Perform global pooling separately, extract the global statistical representation of features at each scale, and obtain the corresponding pooling vector. :

[0117]

[0118] in, Represent the feature map at the i-th scale, respectively. Height and width; These represent the feature maps at the i-th scale, respectively. The row and column coordinates in the data are used to traverse all pixel positions; Represents the feature map at the i-th scale In position eigenvectors;

[0119] Specifically, for scale feature maps Perform global pooling to obtain the corresponding pooling vector. :

[0120] ;

[0121] The output pooling vectors The data is then spliced ​​and input into a multilayer perceptron. Generate corresponding fusion scale weights :

[0122]

[0123] The Softmax function is used to ensure the fusion scale weights of the output. Satisfying the normalization constraint indicates the importance of the fusion of features at each scale.

[0124] Specifically,

[0125] ;

[0126] The Softmax function ensures the output weights of the three fusion scales. Satisfy normalization constraints,

[0127] Due to the feature maps at various scales For different resolutions, an upsampling operation is used to pool the various pooling vectors. Align to a uniform spatial resolution and perform weighted blending:

[0128]

[0129] Obtain fused feature maps at a uniform scale .

[0130] Specifically, the UpSample operation is used to... Align to a uniform spatial resolution and perform weighted blending:

[0131] ;

[0132] Obtain fused feature maps at a uniform scale .

[0133] By employing the Adaptive Task Decoupling Head (ATD Head), fusion weights are generated based on the statistical features of the feature maps at each scale. Through weighted fusion of multi-scale features, adaptive perception of defects at different scales is achieved.

[0134] The above-described process involves standardizing and normalizing the fused feature map, performing convolutional operations to extract local details, introducing fusion scale weights to obtain classification branch features, weighting and fusing these classification branch features, and inputting the resulting classification feature representation into a fully connected classification layer to obtain the classification prediction result. This process includes the following steps:

[0135] Standard normalization and convolution operations are performed on the fused feature map to extract local details:

[0136]

[0137] Where Norm represents batch normalization, Conv represents a convolutional layer with enhanced receptive field, and the present invention uses the ReLU activation function;

[0138] Introducing fusion scale weights For classification branch features Perform weighted fusion:

[0139]

[0140] Specifically, a fusion scale weight is introduced. Weighted fusion of features from classification branches:

[0141] ;

[0142] Among them, classification feature representation It possesses the capabilities of multi-scale fusion, local enhancement, and weight adaptation, which can improve the perception of complex textures and category boundaries;

[0143] Finally, the classification features fused from multiple scales are represented. Input to classification fully connected layer :

[0144]

[0145] Obtain classification prediction results .

[0146] Therefore, in response to the need for defect type identification, a local convolution enhancement module is introduced to enhance the details of the fused features, improve the distinguishability of texture and edge information, and enhance the contribution of different scales in the classification task by combining fusion weights.

[0147] The above-described process involves performing global average pooling on the fused feature map to obtain a global feature representation of the overall contour. Anchor-Aware modules are then designed to perform perceptual processing on each pooling vector. The fused scale weights are integrated into the Anchor-Aware model to form a regression feature representation that perceives the scale distribution. This regression feature representation is then input into a fully connected regression layer to obtain the regression prediction result. The process includes the following steps:

[0148] To achieve accurate modeling for defect localization, the regression branch employs a combination of global feature aggregation and anchor point awareness mechanisms.

[0149] For fused feature maps Perform a global average pooling operation to obtain a global feature representation of the overall contour. :

[0150]

[0151] in, Represents the fused feature map height, 'Represents fused feature map width, Indicates the location of the fused feature map eigenvectors;

[0152] Then, an Anchor-Aware module was designed to process each pooling vector. Perform perception processing and fuse scale weights Integrating into the Anchor-Aware model to form a regression feature representation of the perceptual scale distribution. :

[0153]

[0154] Specifically, for each pooling vector ,include Perform perception processing and fuse scale weights. Integrating into the Anchor-Aware model to form a regression feature representation of the perceptual scale distribution. :

[0155] ;

[0156] AnchorAware refers to a sub-module for extracting regression features from preset anchor points, such as those of different sizes and aspect ratios. This represents the basic fully connected layer, which outputs regression box parameters, including center point coordinates, width, and height. This mechanism can adapt to defect targets of different sizes and shapes, improving positioning accuracy and robustness.

[0157] Then, the regression features are represented Input regression fully connected layer :

[0158]

[0159] Obtain regression prediction results .

[0160] Therefore, by using global pooling to extract the overall structural features of the image and combining it with an adaptive anchor point mechanism to generate regression targets, we can adapt to various defect sizes and shapes and achieve accurate boundary fitting.

[0161] The aforementioned collaborative interaction module for classification and regression tasks dynamically generates interaction weights based on the degree of coupling between tasks by concatenating classification feature representations and global feature representations and applying attention weights. It then adjusts the classification and regression prediction results using a weighted approach to obtain the final classification and regression outputs. The module includes the following steps:

[0162] To achieve effective collaboration between classification and regression tasks, this invention introduces a task interaction module. By analyzing the correlation between classification and regression features, it achieves dynamic weight allocation, thereby alleviating task conflicts and improving overall detection performance. Specifically, classification features are represented... With global feature representation splicing together to form a joint representation :

[0163]

[0164] Then, the base fully connected layer (FC) is input, and the weighting factors for the two tasks are generated using the Softmax function:

[0165]

[0166] in, For classification interaction weights, To regress the interaction weights,

[0167] Final classification prediction results and regression prediction results The adjusted task output is obtained through a weighted method:

[0168]

[0169] Get the final classification output and final regression output It is used by staff to determine and understand the defects in the input images.

[0170] This enables collaborative optimization between classification and regression tasks, improving overall detection performance. The module adaptively allocates focus based on the complexity of the defect category and the ambiguity of its boundaries, emphasizing classification when textures are clear and strengthening the regression branch when edges are unclear, thus enhancing the model's comprehensive perception of complex defects.

[0171] In another specific embodiment based on the above, the following steps are also included:

[0172] The design incorporates a joint loss function that integrates classification, regression, and object discrimination. It employs focus loss, IoU loss, and binary cross-entropy to form the overall optimization objective, and dynamically adjusts it in conjunction with task weights.

[0173] Specifically, it includes the following steps:

[0174] Experiments were conducted using the NEU-DET industrial defect dataset, which contains images representing several classes of defects. The training and test sets were divided in a 7:3 ratio, and the batch size for each training batch was set to N.

[0175] Specifically, the experiment was conducted using the NEU-DET industrial defect dataset, which contains 1800 images representing 6 categories of defects. The training and test sets were divided in a 7:3 ratio, with 1260 images randomly selected as the training set and the remainder as the test set. The batch size was set to 16, and the number of training rounds was 200.

[0176] To ensure that the proposed multi-scale feature extraction and fusion model can effectively learn and optimize parameters, this invention designs a unified optimization objective during the training phase. This optimization objective is achieved through a joint loss function, which optimizes the model by calculating the difference between the prediction and the true label after the model generates the prediction result in forward propagation, and uses this error to update the model parameters during backpropagation. This method constructs an end-to-end joint loss function and achieves balanced optimization among multiple tasks through dynamic weighting.

[0177] To achieve a unified training optimization goal, this method constructs an end-to-end joint loss function that comprehensively considers the errors in classification, boundary localization, and object confidence, and achieves balanced optimization among multiple tasks through dynamic weighting.

[0178] In classification, the Focal Loss function is used to address the class imbalance problem, and it is defined as follows:

[0179]

[0180] in, These are the positive and negative sample weight coefficients. To adjust the index factor for the attention given to easy and difficult samples; in the regression aspect, the IoU (Intersection over Union) loss is introduced to measure the degree of overlap between the predicted box and the ground truth box, expressed as:

[0181]

[0182] in, Represents the actual value;

[0183] Simultaneously, the binary cross-entropy loss from object detection is introduced:

[0184]

[0185] in, Indicates the first pooling vectors Confidence component; This indicates the Nth data in the current batch. The true binary label corresponding to each data point includes 1 indicating that there is a defect at the corresponding location of the predicted box, and 0 indicating that there is no defect. Indicates the number of samples in the current batch;

[0186] A confidence level assessment is performed to determine whether a defect exists. The three factors are then weighted and combined to form the total loss function:

[0187]

[0188] in, and These are the balance factors for regression and object tasks, respectively. Used for optimization.

[0189] Specifically, those skilled in the art can use deep learning algorithms to obtain the numerical range corresponding to the surface defects of the steel plate, for use in [the following context is missing from the original text]. To perform corresponding mappings, for example, regarding steel plate surface defects, relevant photos of six common steel surface defects—cracks, inclusions, patches, pitting, indentation oxidation, and scratches—are used as the training dataset. After training, the range intervals corresponding to the six defects are obtained for reference. Perform a corresponding analysis.

[0190] A system applying the above method includes a preprocessing module, a processing and splicing module, a classification prediction module, a regression prediction module, and a weighting module, wherein:

[0191] The preprocessing module uses a window attention mechanism to extract features at different scales from the input steel plate surface image through several stages and outputs feature maps accordingly.

[0192] The splicing module processes each layer The output feature map is convolved with convolution kernels of different sizes to obtain features under different receptive fields and corresponding scale feature maps. The pooling vector of each scale feature map is extracted by global pooling, and the fusion scale weight is generated based on the obtained pooling vector. The fusion is then weighted to obtain a fusion feature map at a unified scale.

[0193] The classification prediction module performs standard normalization and convolution operations on the fused feature map to extract local details, introduces fusion scale weights to obtain classification branch features, performs weighted fusion of classification branch features, and inputs the fused classification feature representation into the classification fully connected layer to obtain the classification prediction result.

[0194] The regression prediction module performs global average pooling on the fused feature map to obtain a global feature representation of the overall contour; it performs perceptual processing on each pooling vector and combines it with the fused scale weights to form a regression feature representation of the perceptual scale distribution; the regression feature representation is input into the regression fully connected layer to obtain the regression prediction result.

[0195] The weighting module concatenates the classification feature representation with the global feature representation, generates weighting factors for the classification prediction result and regression prediction result based on the concatenated features, and performs weighting to obtain the adjusted final classification output and final regression output, which are used to determine the surface defects of the steel plate.

[0196] This optimization strategy ensures that the model can simultaneously focus on classification accuracy, localization accuracy, and detection confidence during training, thereby improving overall detection performance and generalization ability. It is suitable for stable detection of various types of defects in industrial scenarios.

[0197] This invention integrates a Swing-Transformer backbone network, a multi-scale Criss-Cross attention mechanism, an adaptive task decoupling head (ATD Head), and a cross-task collaborative optimization strategy to achieve high-precision and real-time surface defect detection. Specifically, it utilizes the Swing-Transformer architecture to extract multi-scale features, combines a multi-scale Criss-Cross attention mechanism to enhance contextual modeling, and employs the adaptive task decoupling head (ATD Head) to achieve dynamic scale weight allocation and cross-task interaction in surface defect detection.

[0198] This invention optimizes the network structure using the Swing-Transformer, reducing computational complexity while preserving the global and local semantic information of defects. This solves the problem of low computational efficiency in traditional Transformers and meets the needs of real-time industrial detection.

[0199] This invention enhances the context of scale feature maps through a multi-scale Criss-Cross attention module, which can effectively capture the long-distance dependencies of defects at different scales, improve multi-scale adaptability, and reduce the cases of missed detection of small targets and localization errors of large targets.

[0200] This invention utilizes an Adaptive Task Decoupling Head (ATD Head) to achieve dynamic scale weight allocation and cross-task interaction modules, enabling adaptive fusion of multi-scale features and dynamic collaboration between classification and regression tasks, thereby improving the accuracy of defect detection.

[0201] This invention improves overall detection performance by jointly optimizing end-to-end loss functions, taking into account losses from classification, regression, and object detection, and optimizing the parameters of each module through joint training.

[0202] This invention was tested on the NEU-DET industrial defect dataset, which includes six common steel surface defects: cracks, inclusions, patches, pitting, indentation oxidation, and scratches. In the experiments, the method of this invention was compared with the performance of five existing techniques—AGLNet, Faster R-CNN, RetinaNet, FCOS, and Grid R-CNN—on the same dataset. The comparison results are available in the appendix. Figure 2 As shown in the figure. The results show that the mean accuracy (mAP) of the present invention reaches a high level, especially in the accuracy of small target defect detection, which is significantly improved compared with the above model, while the detection speed can meet the requirements of industrial real-time detection.

[0203] In another embodiment, the present invention proposes an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the above-described method for detecting surface defects in steel plates based on a Swing-Transformer network structure.

[0204] In another embodiment, the present invention provides a computer-readable storage medium storing a computer program that causes a computer to execute a steel plate surface defect detection method based on a Swin-Transformer network structure as described above.

[0205] In the embodiments disclosed in this application, a computer storage medium may be a tangible medium that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0206] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0207] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principle of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for detecting surface defects in steel plates based on a Swin-Transformer network structure, characterized in that, Includes the following steps: A hierarchical Swin-Transformer network is constructed, with several stages using window attention mechanism to extract features at different scales from the input steel plate surface image and output feature maps respectively. Construct a multi-scale Criss-Cross attention mechanism, inputting each layer The output feature map is convolved with the input feature map using convolution kernels of different sizes to obtain features under different receptive fields and obtain the corresponding scale feature maps. Pooling vectors for each scale feature map are extracted using global pooling. Fusion scale weights are generated based on the obtained pooling vectors, and a fused feature map at a unified scale is obtained through weighted fusion. The fused feature map is normalized and convolutional to extract local details. A fusion scale weight is introduced to obtain classification branch features. The classification branch features are weighted and fused, and the fused classification feature representation is input into the classification fully connected layer to obtain the classification prediction result. Perform global average pooling on the fused feature map to obtain the global feature representation of the overall contour; Each pooling vector is processed perceptually and combined with the fusion scale weights to form a regression feature representation of the perceptual scale distribution; the regression feature representation is input into the regression fully connected layer to obtain the regression prediction result; The classification feature representation and the global feature representation are concatenated. Based on the concatenated features, weighting factors for the classification prediction results and regression prediction results are generated and weighted to obtain the adjusted final classification output and final regression output, which are used to determine the surface defects of the steel plate.

2. The method for detecting surface defects of steel plates based on a Swing-Transformer network structure according to claim 1, characterized in that: The first stage of the multi-stage is Stage1, which consists of convolutional coding and a Swing-Transformer layer. Each of the remaining stages consists of a Swing-Transformer block and an attention block stacked together. Each attention block contains a window attention mechanism.

3. The method for detecting surface defects of steel plates based on a Swing-Transformer network structure according to claim 2, characterized in that, The several stages each utilize a window attention mechanism to extract features at different scales from the input steel plate surface image and output feature maps accordingly, including the following steps: The input image is divided into non-overlapping image patches, and initial feature maps are generated through Stage 1 convolutional encoding and a Swing-Transformer layer. Subsequently, each stage utilizes a window attention mechanism and performs large-scale feature extraction on the feature map output by the previous layer through layer-by-layer downsampling and feature aggregation, as shown in the following formula, and outputs the feature maps for each layer accordingly: in, Indicates the first Stage For the i-th layer Output feature map.

4. The method for detecting surface defects of steel plates based on a Swing-Transformer network structure according to claim 3, characterized in that, The input layer The output feature map is convolved with convolutional kernels of different sizes on the input feature map to obtain features under different receptive fields and obtain corresponding scale feature maps, including the following steps: Using convolution kernels of different sizes to process the i-th input feature map Perform convolution to obtain the i-th feature map. Corresponding query ,key ,value matrix: in, These represent convolution operations with kernels of different sizes. Then, the similarity between Q and K is calculated, and the attention weights are obtained through the Softmax function. : in, Indicator key The dimension of a vector express transpose, Finally, attention weights and Perform feature fusion: Obtain the i-th feature map Scale feature map .

5. The method for detecting surface defects of steel plates based on a Swing-Transformer network structure according to claim 4, characterized in that, The process of extracting pooling vectors from feature maps at each scale through global pooling, generating fusion scale weights based on the obtained pooling vectors, and then obtaining a fusion feature map at a unified scale through weighted fusion includes the following steps: For each scale feature map Perform global pooling separately, extract the global statistical representation of features at each scale, and obtain the corresponding pooling vector. : in, Represent the feature map at the i-th scale, respectively. Height and width; These represent the feature maps at the i-th scale, respectively. The row and column coordinates in the data; Represents the feature map at the i-th scale In position eigenvectors; The output pooling vectors ,pass The functions are concatenated and then input into the multilayer perceptron. Generate corresponding fusion scale weights : Softmax is used to ensure the fusion scale weights of the output. Functions that satisfy normalization constraints The UpSample operation is used to sample each pooling vector. Align to a uniform spatial resolution and perform weighted blending: Obtain fused feature maps at a uniform scale .

6. The method for detecting surface defects of steel plates based on a Swing-Transformer network structure according to claim 5, characterized in that, The process of extracting local details by performing standard normalization and convolution operations on the fused feature map, introducing fusion scale weights to obtain classification branch features, performing weighted fusion of the classification branch features, and inputting the fused classification feature representation into the classification fully connected layer to obtain the classification prediction result includes the following steps: Standard normalization and convolution operations are performed on the fused feature map to extract local details: Where Norm represents batch normalization and Conv represents a convolutional layer with enhanced receptive field. Introducing fusion scale weights For classification branch features Perform weighted fusion: Finally, the classification features fused from multiple scales are represented. Input to classification fully connected layer : Obtain classification prediction results .

7. The method for detecting surface defects of steel plates based on a Swing-Transformer network structure according to claim 6, characterized in that, The process involves performing global average pooling on the fused feature map to obtain a global feature representation of the overall contour; performing perceptual processing on each pooling vector and combining it with the fusion scale weights to form a regression feature representation of the perceptual scale distribution; and inputting the regression feature representation into the regression fully connected layer to obtain the regression prediction result, including the following steps: For fused feature maps Perform a global average pooling operation to obtain a global feature representation of the overall contour. : in, Represents the fused feature map height, 'Represents fused feature map width, Indicates the location of the fused feature map eigenvectors; Then, for each pooling vector Perform perception processing and combine fusion scale weights To form a regression feature representation of the perceptual scale distribution : Where AnchorAware represents the pooling vectors... Perform sensory processing. Indicates the basic fully connected layer; Then, the regression features are represented Input regression fully connected layer : Obtain regression prediction results .

8. The method for detecting surface defects of steel plates based on a Swing-Transformer network structure according to claim 7, characterized in that, The step of concatenating the classification feature representation with the global feature representation, generating weighting factors for the classification prediction result and the regression prediction result based on the concatenated features, and then weighting them to obtain the adjusted final classification output and final regression output includes the following steps: Representing classification features With global feature representation splicing together to form a joint representation : Then, the base fully connected layer FC is input, and the weighting factors for the classification and regression prediction results are generated using the Softmax function: in, For classification interaction weights, To regress the interaction weights, Final classification prediction results and regression prediction results The adjusted task output is obtained through a weighted method: Get the final classification output and final regression output It is used by staff to determine and understand the defects in the input images.

9. The method for detecting surface defects of steel plates based on a Swing-Transformer network structure according to claim 8, characterized in that, The Swin-Transformer network is trained and optimized using the following loss function: For classification, a focus loss function is used. Its definition is: in, These are the positive and negative sample weight coefficients. An index factor to adjust the level of attention given to samples of varying difficulty; In terms of regression, IoU loss is introduced. The expression is: in, Represents the actual value; Simultaneously, a binary cross-entropy loss is introduced. : in, Indicates the first pooling vectors Confidence component; Indicates and The corresponding N data in the current batch The true binary label corresponding to each data point includes 1 indicating that there is a defect at the corresponding location of the predicted box, and 0 indicating that there is no defect. Indicates the number of training samples in the current batch; The total loss function is represented by a weighted combination: in, and These are the balancing factors for regression and object tasks, respectively.

10. An electronic device, characterized in that, include: The present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements a method for detecting surface defects of steel plates based on a Swin-Transformer network structure as described in any one of claims 1-9.

Images