Industrial defect detection method based on multi-scale Transform

Abstract

The invention provides an industrial defect detection method based on a multi-scale Transform, and the method comprises the steps: employing a pre-training network to extract multi-scale deep features, introducing a multi-scale Transform encoder-decoder architecture, achieving the long-distance dependence modeling through a multi-head self-attention mechanism, and obtaining the global representation capability exceeding a convolution network. Compared with CNN-U-Net, lossless jump connection is designed, 512-dimensional features generated by an encoder can be completely transmitted to a decoder, and tiny defect information loss caused by a traditional dimension reduction mode is avoided; the calculation complexity of global attention is remarkably reduced through a three-level local attention decomposition mechanism and is only 0.47% of that of a standard Transform, and the industrial real-time detection requirement is met; and self-adaptive error normalization and multi-scale smooth fusion are combined, so that false alarms at complex textures are effectively inhibited, and patch boundary artifacts are eliminated. Finally, according to the method, a pixel-level abnormal score graph is generated based on reconstruction deviation, and high-precision positioning and detection of multi-scale defects are achieved.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary fields of computer vision, defect detection, and deep learning, and specifically relates to an industrial defect detection method based on multi-scale Transformer. Background Technology

[0002] In the fields of industrial automation and intelligent manufacturing, defect detection, as a crucial aspect of product quality control, has received widespread attention. Traditional defect detection methods largely rely on image processing techniques based on manual features. These methods are practical for scenarios with simple structures and fixed defect types. However, as industrial product structures become increasingly complex and defect forms more diverse, rule-based detection methods are no longer sufficient to meet practical needs, especially in complex backgrounds or unstructured samples, where the false positive and false negative rates increase significantly.

[0003] In recent years, with the development of deep learning, supervised defect detection methods based on convolutional neural networks have developed rapidly. However, supervised learning methods heavily rely on large-scale, high-quality defect samples as training data. In actual industrial scenarios, defect samples are often scarce, difficult to collect, unevenly distributed, and diverse in type, making it very difficult to construct a complete training set.

[0004] Unsupervised defect detection methods, as a technique trained solely on normal samples, have become an important direction for solving the aforementioned problems. Early unsupervised methods were mainly based on reconstruction models such as autoencoders (AEs) and generative adversarial networks (GANs). These methods assume that after training on normal samples, the model can reconstruct normal images well, but will produce significant errors in reconstructing abnormal images. However, these image-level reconstruction-based methods suffer from the following problems:

[0005] Training stability issues: Because the training data only contains normal samples and lacks diversity, the model has difficulty learning expressive feature representations, and the training process is unstable.

[0006] Generalization control is difficult: an undertrained autoencoder may fail to reconstruct a normal image, while an overtrained autoencoder may even reconstruct an abnormal region, making it difficult to guarantee the existence of a generalization gap.

[0007] Insufficient reconstruction accuracy: Convolution-based image reconstruction operations not only affect abnormal regions, but also their neighboring regions, making it difficult to accurately locate small-scale defects.

[0008] Lack of global modeling ability: Traditional convolutional networks mainly extract local features and lack the ability to model long-distance dependencies in images, which can easily lead to blurred reconstruction or incomplete features.

[0009] Recently, some methods have begun to utilize networks pre-trained on large-scale datasets to extract features, and then use statistical methods or distance metrics to identify anomalies. While these methods avoid the difficulty of learning feature representations on datasets containing only normal samples, they still lack universal feature analysis guidelines, and different methods use different statistical distances or comparison methods, resulting in insufficient robustness.

[0010] Furthermore, most existing methods employ single-scale or shallow reconstruction approaches, making it difficult to simultaneously preserve both global semantic information and local structural features of an image. This results in poor model perception of complex textures or minute defects. In particular, existing algorithms often struggle to achieve sufficient detection accuracy when dealing with multi-scale and multi-type defects.

[0011] Therefore, how to construct an unsupervised defect detection model that simultaneously possesses feature representation capabilities, global modeling capabilities, and multi-scale detection capabilities has become a pressing technical challenge. Summary of the Invention

[0012] To address the aforementioned issues, this invention discloses an industrial defect detection method based on multi-scale Transformer. It innovatively combines pre-trained feature extraction with multi-scale Transformer reconstruction, achieving lossless transfer of feature information and significantly improving the detection capability of minute defects and detailed features.

[0013] To achieve the above objectives, the technical solution of the present invention is as follows:

[0014] An industrial defect detection method based on multi-scale Transformer includes the following steps:

[0015] S1: Input image Input the pre-trained feature extraction network to extract multi-scale deep features;

[0016] S2: In feature representation Add global location encoding to preserve the spatial location information of features;

[0017] S3: Input the feature representation into a multi-scale Transformer to reconstruct the features and obtain the reconstructed features. ;

[0018] S4: Calculate the original features With reconstruction features The reconstruction error between them is normalized using a standard deviation estimation network;

[0019] S5: Perform multi-scale smoothing on the normalized error to generate the final outlier score map. ;

[0020] S6: Determine the location and type of defects based on the anomaly score chart.

[0021] Furthermore, step S1 specifically includes the following steps:

[0022] S1.1: Select ResNet-18 pre-trained on the ImageNet dataset as the feature extraction network;

[0023] S1.2: Extract the feature maps of the three intermediate layers (layer 1, layer 2, and layer 3) in ResNet-18, denoted as follows: , , ;

[0024] Furthermore, step S1.2 specifically includes the following steps:

[0025] S1.2.1: For feature maps of two adjacent scales and ,in The size is , The size is In the formula, B is the batch size, H and W are the height and width of the feature map, and C is the number of channels. The scale ratio is calculated. ;

[0026] S1.2.2: Use the s value calculated in S1.2.1 as the kernel size parameter for the unfold operation. Expand to (B, , , );

[0027] S1.2.3: Stitching along the channel dimension Expand results and ,get ;

[0028] S1.2.4: Reassemble the fusion result using the fold operation. Feature map;

[0029] S1.2.5: Fusing sequentially , , The final multi-scale feature representation is obtained. The number of channels is .

[0030] S1.3: Upsample the three feature maps to the same spatial size. ,in ;

[0031] S1.4: During the training phase, the parameters of the feature extraction network are kept frozen and used only for feature extraction, without participating in gradient updates.

[0032] Furthermore, step S2 specifically includes the following steps:

[0033] S2.1: Define the global positional encoding matrix ,in The number of feature channels, The feature map space size;

[0034] S2.2: The global position encoding adopts a non-learnable parameter form, and the optimal position representation is automatically learned through training;

[0035] S2.3: Integrating global location encoding with feature representation Add them together to obtain the location-aware features. .

[0036] Furthermore, step S3 specifically includes the following steps:

[0037] S3.1: The Transformer consists of three layers of encoders and decoders, corresponding to patch_sizes of... ;

[0038] S3.2: At each level The input feature map is segmented into There are 1 patch, and each patch contains 10 patches. 1 eigenvector;

[0039] S3.3: Add a zero-padding token to each patch as the overall representation of the patch;

[0040] S3.4: Add relative position encoding to the feature vectors within each patch. ;

[0041] S3.5: Include the patch file. By adding 1 token to the Transformer encoder at the input level, we get:

[0042] The Transformer encoder calculates the following feature latent vectors:

[0043] For hierarchy The nth patch has the following input: ,in pad the token with zero. For the first in the patch 1 eigenvector;

[0044] After passing through a multi-head self-attention network and a feedforward network, the output is... ;

[0045] in This serves as the overall semantic representation of the patch, and is passed down to the next level.

[0046] S3.6: The overall latent vector of the patch is used as the input to the next level; the feature latent vector is directly passed to the decoder at the same level without any dimensionality reduction processing.

[0047] S3.7: After the three levels of processing are completed, the underlying latent representation at the lowest level begins to be upsampled and reconstructed. The latent feature vector is directly passed to the decoder at the same level via skip connections. Specific process:

[0048] S3.7.1: Starting from the bottom layer (level 3), the output of the previous level decoder is used as the header input of the current level decoder. ;

[0049] S3.7.2: Obtaining the latent feature vector of the encoder from skip connections ;

[0050] S3.7.3: Concatenate the header input with the feature latent vector. As the memory input for the decoder;

[0051] S3.7.4: Use learnable query embeddings as query inputs to the decoder;

[0052] S3.7.5: Through a multi-head cross-attention mechanism, the query extracts information from memory to generate a reconstructed feature vector;

[0053] S3.7.6: Concatenate the reconstruction results of all patches to restore the complete feature map.

[0054] Furthermore, step S4 specifically includes the following steps:

[0055] S4.1: Calculate the original features Reconstructed features channel by channel Norm error:

[0056] ;

[0057] S4.2: Estimating the standard deviation of network error using standard deviation The network consists of multiple layers of convolutions and activation functions;

[0058] S4.3: Calculate the normalized error:

[0059] ;

[0060] in To prevent division by zero of small constants;

[0061] S4.4: Simultaneously optimize reconstruction loss during training phase Loss estimation by sum of standard deviation :

[0062] ;

[0063] ;

[0064] ;

[0065] Furthermore, step S5 specifically includes the following steps:

[0066] S5.1: Normalization error Perform average pooling operations at four different scales, with pooling kernel sizes of respectively ;

[0067] S5.2: Upsample the pooling results at the four scales to the original size. ;

[0068] S5.3: Perform feature stitching between the original error map and the smoothed results at four scales;

[0069] S5.4: Use Convolution performs weighted fusion of the splicing results:

[0070] ;

[0071] Weights set to ;

[0072] S5.5: Upsample the fused fractional image to the input image size to obtain the final anomaly fractional image. .

[0073] Furthermore, step S6 specifically includes the following steps:

[0074] S6.1: Anomaly score plot Normalization is performed, min-max normalization to... ;

[0075] S6.2: Calculate image-level anomaly scores: Take The maximum value is used as the anomaly score for the entire image;

[0076] S6.3: Calculate pixel-level anomaly localization: Areas exceeding a preset threshold are marked as defect locations.

[0077] S6.4: Calculate the AUROC index to evaluate the detection performance.

[0078] The present invention has the following beneficial effects:

[0079] 1. Traditional CNN-U-Net typically uses 1×1 convolutions to compress channels in skip connections, restricting the flow of low-level features into high-level structures. However, this results in the loss of high-frequency details crucial for detecting minute defects. The lossless skip connection proposed in this invention allows the 512-dimensional features output by the encoder to be directly passed to the decoder without any dimensionality reduction, fully preserving fine-grained structural information. Experiments show that this design significantly improves pixel-level AUROC and significantly reduces the false negative rate for 2–3 pixel-level holes and fine scratches, addressing the structural shortcomings of CNN-U-Net in detecting ultra-small defects.

[0080] 2. CNN-U-Net's convolutional operations are limited to local regions, making it difficult to capture long-range dependencies; while the standard Transformer, due to its O(n²) complexity, cannot be directly applied to high-resolution industrial images. This invention employs a three-level local attention decomposition strategy: breaking down global attention into local computations of different patches and levels, reducing the overall complexity to approximately 0.47% of the standard Transformer. This strategy simultaneously possesses: near-global semantic modeling capabilities (superior to CNN-U-Net); near real-time inference speed (approximately 30 FPS, meeting the needs of industrial production lines); fundamentally overcoming the bottleneck of CNN-U-Net's inability to balance global modeling and real-time performance.

[0081] 3. Adaptive error normalization and multi-scale smoothing fusion effectively suppress false alarms in textured regions and eliminate patch boundary artifacts.

[0082] CNN-U-Net's convolutional reconstruction-based defect detection typically suffers from two major problems: complex texture regions inherently have higher errors, leading to a high false positive rate; and discontinuities across patches or receptive fields result in unsmooth localization boundaries. This invention addresses these shortcomings through two key design features: a standard deviation estimation network that learns a "normal error baseline" for each spatial location, normalizing the reconstruction error and significantly reducing false positives in highly textured regions (such as carpet, leather, and grid). Multi-scale smoothing fusion utilizes average pooling at different scales (1, 4, 16, 64) across patch boundaries, achieving a balance between global consistency and local detail, resulting in smoother edges and a more continuous structure in the anomaly score map. Attached Figure Description

[0083] Figure 1 This is an overall structural diagram of the present invention.

[0084] Figure 2 This is a structural diagram of the layered encoder in a specific embodiment of the present invention.

[0085] Figure 3 This is a diagram of a multi-layered attention structure.

[0086] Figure 4 This is a diagram of the feedforward network structure. Detailed Implementation

[0087] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0088] like Figure 1 As shown, the core idea of ​​this invention, a multi-scale Transformer-based industrial defect detection method, is as follows: First, a pre-trained deep convolutional neural network is used to extract multi-scale deep features from the input image, avoiding the instability of learning feature representations on datasets containing only normal samples. Second, the extracted deep features are treated as discrete information units, and feature-level reconstruction is performed through an innovative multi-scale Transformer structure. The key innovation lies in using a lossless connection method to ensure the complete transmission of feature information. Finally, by calculating the difference between the original features and the reconstructed features, an accurate anomaly score map is generated, enabling defect detection and localization. The innovation of this invention is summarized as follows:

[0089] 1. Lossless skip connection: Abandoning the traditional channel dimensionality reduction, the 512-dimensional features output by the encoder are directly and losslessly transmitted to the decoder, avoiding the loss of high-frequency details.

[0090] 2. Hierarchical attention computation: By adopting a three-layer local attention decomposition strategy, the computational complexity is reduced to 0.47% of the standard method, achieving efficient training and inference and meeting the needs of real-time industrial detection.

[0091] 3. Adaptive Error Normalization: A standard deviation estimation network is designed to adaptively normalize the reconstruction error at its location, avoiding information leakage. This mechanism significantly reduces the false positive rate for complex texture categories, improving image-level detection accuracy.

[0092] 4. Multi-scale smooth fusion: By asymmetric weighted fusion of multi-scale pooling results, boundary artifacts caused by patch processing are eliminated. This method makes the boundaries of the defect localization mask more coherent.

[0093] 5. Zero-sensitivity alignment design: Leveraging the semantically driven nature of Transformer self-attention and combining it with multi-scale feature fusion, the model becomes insensitive to transformations such as rotation and translation. Performance is not degraded in non-alignment tests, and plug-and-play deployment is supported.

[0094] Specifically, such as Figure 1 As shown, the present invention includes a feature extraction and reconstruction stage and a defect discrimination stage.

[0095] Image reconstruction includes the following steps:

[0096] S1: Input image Input the pre-trained feature extraction network to extract multi-scale deep features, as follows:

[0097] S1.1: Select ResNet-18 pre-trained on the ImageNet dataset as the feature extraction network.

[0098] This invention selects ResNet-18 as the feature extractor. ResNet-18 contains approximately 11.7M parameters, mitigates gradient vanishing through residual connections, and its hierarchical structure captures multi-scale features. ImageNet pre-training provides a general visual representation, avoiding the instability of training on small-scale normal sample sets.

[0099] S1.2: Extract the feature maps of the three intermediate layers (layer 1, layer 2, and layer 3) in ResNet-18, denoted as follows: ;

[0100] Extract the feature maps from the first three layers, configured as follows:

[0101] Space dimensions Number of channels ;

[0102] Space dimensions: 32×32, number of channels ;

[0103] Space dimensions 16×16, number of channels ;

[0104] The three-layer Transformer achieves a two-order-of-magnitude reduction in computational cost while maintaining accuracy through a hierarchical attention mechanism, making it particularly suitable for high-resolution defect detection needs in industrial scenarios.

[0105] Furthermore, step S1.2 specifically includes the following steps:

[0106] S1.2.1: For feature maps of two adjacent scales and ,in The size is The dimensions are, Calculation scale ratio ;

[0107] Different scale features correspond to different receptive fields. Each pixel corresponds to the original Figure 4 ×4 area (Before upsampling) Corresponding 8×8 region. Calculate the original scale ratio to guide fusion:

[0108] ;

[0109] In the formula, "original" represents the original scale image. and : For fusion features and : .

[0110] S1.2.2: Use the s value calculated in S1.2.1 as the kernel size parameter for the unfold operation. Expand to (B, , , );

[0111] The `unfold` operation splits and expands the feature map into windows. implement :

[0112] ;

[0113] The 64×64 space is divided into 32×32 2×2 windows, and the 64-dimensional features at 4 positions in each window are expanded into 256 dimensions.

[0114] S1.2.3: Stitching along the channel dimension Expand results and ,get After unfolding and (Pooled to 32×32) splicing:

[0115] ;

[0116] S1.2.4: Reassemble the fusion result using the fold operation. Feature map

[0117] The fold operation is the inverse of the unfold operation, reorganizing the space structure (kernel_size=2, stride=2, output_size=(64, 64));

[0118] ;

[0119] S1.2.5: Fusing sequentially , , The final multi-scale feature representation Z is obtained, with C=448 channels;

[0120] right and Perform the second fusion. Unfold operation (kernel_size=4, stride=4):

[0121] ;

[0122] and Piece together and fold:

[0123] ;

[0124] The number of channels is too high, so it is compressed to 448 using a 1×1 convolution:

[0125] ;

[0126] The convolution weights are initialized using Kaiming: ,in .

[0127] S1.3: Upsample the three feature maps to the same spatial size H×W, where H=W=64;

[0128] Using bilinear interpolation Upsampled to 64×64:

[0129] ;

[0130] Bilinear interpolation calculation formula:

[0131] ;

[0132] S1.4: During the training phase, the parameters of the feature extraction network are frozen and used only for feature extraction, without participating in gradient updates.

[0133] During training, all parameters of ResNet-18 are frozen, i.e. Since the training data only contains normal samples, allowing parameter updates can lead to overfitting and reduce the feature generalization ability.

[0134] Furthermore, regarding the image reconstruction stage, step S2 specifically includes the following steps:

[0135] S2.1: Define the global positional encoding matrix Where C is the number of feature channels and H×W is the feature map space size. The Transformer is position-insensitive and requires explicit encoding. Definition It provides a 448-dimensional position vector for each spatial location.

[0136] S2.2: The global position encoding adopts a learnable parameter form, and the optimal position representation is automatically learned through training.

[0137] Unlike fixed sinusoidal coding, learnable coding is used to capture task-related location preferences. Initialize to During training, updates are performed using gradient descent.

[0138] S2.3: Add the global location code to the feature representation Z to obtain the location-aware feature. ;

[0139] Element-wise addition incorporates positional information:

[0140] ;

[0141] Adjusted to 512 dimensions using 1×1 convolution and LayerNorm:

[0142] ;

[0143] ;

[0144] in Output .

[0145] Furthermore, regarding the image reconstruction stage, step S4 specifically includes the following steps:

[0146] S3.1: The Transformer consists of three levels of encoders and decoders, corresponding to patch_sizes of [4, 4, 4] respectively;

[0147] Three-tier configuration: Number of Transformer blocks at each level .

[0148] S3.2: At each level The input feature map is segmented into There are 1 patch, and each patch contains 10 patches. 1 eigenvector;

[0149] Level 1: The 64×64 feature map is divided into 16×16=256 patches, each patch containing 4×4=16 feature vectors. This is achieved through reshape and permute operations.

[0150] ;

[0151] Level 2: The 256 patch tokens are reorganized into 16×16, which is then divided into 4×4=16 patches.

[0152] Level 3: 16 tokens as a single patch.

[0153] S3.3: Add a zero-padding token to each patch as the overall representation of the patch;

[0154] Add zero initialization As a whole, the patch design is inspired by BERT's [CLS] token design.

[0155] S3.4: Add relative position encoding to the feature vectors within each patch. All patches at the same level share this feature. Each patch's input sequence consists of 17 tokens.

[0156] ;

[0157] S3.5: The Transformer encoder processes the 17 tokens from each patch into a 4-layer Transformer encoder, and the output includes:

[0158] The overall latent vector (header token) of a single patch. and 16 feature latent vectors The specific structure of the multi-head self-attention mechanism is as follows: Figure 3 As shown. The core operations of the encoder include multi-head self-attention (8 heads), a feedforward network (2048-dimensional hidden layers), and residual connections. The overall architecture is as follows. Figure 2 As shown.

[0159] S3.6: The overall latent vector of the patch is used as the input to the next level; the feature latent vector is directly passed to the decoder at the same level without any dimensionality reduction processing, as follows: (header token) The latent feature vectors are passed to the next level directly to the decoder at the same level via skip connections, while the dimension remains unchanged at 512.

[0160] S3.7: After the three levels of processing are completed, the underlying latent representation at the lowest level begins to be upsampled and reconstructed. The latent feature vector is directly passed to the decoder at the same level via skip connections. Specific process:

[0161] S3.7.1: Starting from the bottom layer (level 3), the output of the previous level decoder is used as the header input of the current level decoder. The third-level decoder initializes 16 learnable query embeddings. Initialized to .

[0162] S3.7.2: Obtaining the latent feature vector of the encoder from skip connections 16 512-dimensional feature latent vectors are obtained from the encoder.

[0163] S3.7.3: Concatenate the header input with the feature latent vector. Memory input for the decoder

[0164] Building memory:

[0165] ;

[0166] S3.7.4: Use learnable query embeddings as query inputs to the decoder.

[0167] Query initialized as ;

[0168] S3.7.5: Through a multi-head cross-attention mechanism, the query extracts information from memory to generate a reconstructed feature vector. The specific process is as follows:

[0169] The decoder consists of three sub-layers. Sub-layer 1 is query self-attention:

[0170] ;

[0171] Sublayer 2 uses query-memory cross-attention:

[0172] ;

[0173] ;

[0174] Residual connection:

[0175] ;

[0176] Sublayer 3 is a feedforward network, with the specific structure as follows: Figure 4 As shown:

[0177] ;

[0178] Output after 4 decoder blocks: ;

[0179] S3.7.6: Concatenate the reconstruction results of all patches to restore the complete feature map.

[0180] The third-level output is reshaped to (B, 512, 4, 4), and the nearest neighbor interpolation is upsampled to (B, 512, 16, 16). The second level processes 16 patches, outputting (B, 512, 16, 16), and upsampled to (B, 512, 64, 64). The first level processes 256 patches, outputting (B, 512, 64, 64).

[0181] Restored to 448 channels using 1×1 convolution:

[0182] ;

[0183] Furthermore, regarding the image reconstruction stage, step S5 specifically includes the following steps:

[0184] S4.1: Calculate the original features Reconstructed features Channel-by-channel L2 norm error: Calculate the L2 distance of the 448 channels position by position:

[0185] ;

[0186] Output ;

[0187] S4.2: Estimating the standard deviation of network error using standard deviation The network consists of multiple layers of convolutions and activation functions, and a standard deviation estimation network is designed to learn a "normal error baseline":

[0188] ;

[0189] The network structure consists of 4 layers of 1×1 convolutions: Each layer is followed by BatchNorm and LeakyReLU.

[0190] S4.3: Calculate the normalized error: ,in To prevent small constants from being divided by zero

[0191] Normalization eliminates position-dependent bias:

[0192] ;

[0193] S4.4: Simultaneously optimize reconstruction loss during training phase Loss estimation by sum of standard deviation ;

[0194] ;

[0195] ;

[0196] ;

[0197] Cut off during training arrive gradient flow, i.e. ;

[0198] Furthermore, regarding the defect identification stage, step S6 specifically includes the following steps:

[0199] S5.1: Normalization error Four different scales of average pooling are performed, with kernel sizes of 1, 4, 16, and 64, respectively, since features are reconstructed by patch. Multi-scale average pooling is then executed.

[0200] ;

[0201] S5.2: Upsample the pooling results at the four scales to the original size (64×64), and perform bilinear interpolation upsampling:

[0202] ;

[0203] S5.3: Perform feature concatenation between the original error map and the smoothed results at four scales, concatenating along the channel dimension:

[0204] ;

[0205] S5.4: Use 1×1 convolution to perform weighted fusion of the splicing results: The weights are set to The kernel weights are fixed. :

[0206] ;

[0207] The original error weights are 0, and only 4 smoothing scales are used. Large-scale pooling spans the patch boundaries, eliminating discontinuities.

[0208] S5.5: Upsample the fused fractional image to the input image size to obtain the final anomaly fractional image S;

[0209] Bilinear interpolation upsampling to the input size (e.g., 256×256):

[0210] ;

[0211] S6.1: Normalize the outlier score map S by mini-maximum normalization to [0, 1]:

[0212] ;

[0213] S6.2: Calculate image-level anomaly scores: Take The maximum value is used as the anomaly score for the entire image.

[0214] ;

[0215] Determine whether an image contains defects by comparing it with a threshold.

[0216] S6.3: Calculate pixel-level anomaly localization: Areas exceeding a preset threshold are marked as defect locations.

[0217] Generate a binary mask:

[0218] ;

[0219] in This is a preset threshold.

[0220] S6.4: Calculate the AUROC index to evaluate detection performance

[0221] AUROC calculation:

[0222] ;

[0223] Where TPR is the true positive rate and FPR is the false positive rate. Image-level AUROC (in terms of...) is calculated separately. For prediction) and pixel-level AUROC (in the form of) Each pixel is a prediction.

[0224] It should be noted that the above content merely illustrates the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, various improvements and modifications can be made without departing from the principle of the present invention, and all such improvements and modifications fall within the scope of protection of the claims of the present invention.

Claims

1. An industrial defect detection method based on multi-scale Transformer, characterized in that: Includes the following steps: S1: Input image Input the pre-trained feature extraction network to extract multi-scale deep features; S2: In feature representation Add global location encoding to preserve the spatial location information of features; S3: Input the feature representation into a multi-scale Transformer to reconstruct the features and obtain the reconstructed features. ; S4: Calculate the original features With reconstruction features The reconstruction error between them is normalized using a standard deviation estimation network; S5: Perform multi-scale smoothing on the normalized error to generate the final outlier score map. ; S6: Determine the location and type of defects based on the anomaly score chart.

2. The industrial defect detection method based on multi-scale Transformer according to claim 1, characterized in that: Step S1 specifically includes the following steps: S1.1: Select ResNet-18 pre-trained on the ImageNet dataset as the feature extraction network; S1.2: Extract the feature maps of the three intermediate layers (layer 1, layer 2, and layer 3) in ResNet-18, denoted as follows: , , ; S1.3: Upsample the three feature maps to the same spatial size. ,in ; S1.4: During the training phase, the parameters of the feature extraction network are kept frozen and used only for feature extraction, without participating in gradient updates.

3. The industrial defect detection method based on multi-scale Transformer according to claim 2, characterized in that: Step S1.2 specifically includes the following steps: S1.2.1: For feature maps of two adjacent scales and ,in The size is , The size is In the formula, B is the batch size, H and W are the height and width of the feature map, C is the number of channels, and the scale ratio is calculated. ; S1.2.2: Use the s value calculated in S1.2.1 as the kernel size parameter for the unfold operation. Expand to (B, , , ); S1.2.3: Stitching along the channel dimension Expand results and ,get ; S1.2.4: Reassemble the fusion result using the fold operation. Feature map; S1.2.5: Fusing sequentially , , The final multi-scale feature representation is obtained. The number of channels is .

4. The industrial defect detection method based on multi-scale Transformer according to claim 1, characterized in that: Step S2 specifically includes the following steps: S2.1: Define the global positional encoding matrix ,in The number of feature channels, The feature map space size; S2.2: The global position encoding adopts a non-learnable parameter form, and the optimal position representation is automatically learned through training; S2.3: Integrating global location encoding with feature representation Add them together to obtain the location-aware features. .

5. The industrial defect detection method based on multi-scale Transformer according to claim 1, characterized in that: Step S3 specifically includes the following steps: S3.1: The Transformer consists of three layers of encoders and decoders, corresponding to patch_sizes of... ; S3.2: At each level The input feature map is segmented into There are 1 patch, and each patch contains 10 patches. 1 eigenvector; S3.3: Add a zero-padding token to each patch as the overall representation of the patch; S3.4: Add relative position encoding to the feature vectors within each patch. ; S3.5: Include the patch file. By adding 1 token to the Transformer encoder at the input level, we get: The calculation process of the Transformer encoder in step S3.5 is as follows: (The text contains several incomplete sentences and fragments, making a coherent translation impossible. A more accurate translation would require the full context.) For hierarchy The nth patch has the following input: ,in pad the token with zero. For the first in the patch 1 eigenvector; After passing through a multi-head self-attention network and a feedforward network, the output is... ; in The overall semantic representation of the patch is passed to the next level; S3.6: The overall latent vector of the patch is used as the input to the next level; the feature latent vector is directly passed to the decoder at the same level without any dimensionality reduction processing. S3.7: After the three levels of processing are completed, the lowest-level latent representation begins upsampling and reconstruction. The feature latent vector is directly passed to the decoder at the same level through a skip connection.

6. The industrial defect detection method based on multi-scale Transformer according to claim 5, characterized in that: The specific process of step S3.7 is as follows: S3.7.1: Starting from the bottom layer (level 3), the output of the previous level decoder is used as the header input of the current level decoder. ; S3.7.2: Obtaining the latent feature vector of the encoder from skip connections ; S3.7.3: Concatenate the header input with the feature latent vector. As the memory input for the decoder; S3.7.4: Use learnable query embeddings as query inputs to the decoder; S3.7.5: Through a multi-head cross-attention mechanism, the query extracts information from memory to generate a reconstructed feature vector; S3.7.6: Concatenate the reconstruction results of all patches to restore the complete feature map.

7. The industrial defect detection method based on multi-scale Transformer according to claim 1, characterized in that: Step S4 specifically includes the following steps: S4.1: Calculate the original features Reconstructed features channel by channel Norm error: S4.2: Estimating the standard deviation of network error using standard deviation The network consists of multiple layers of convolutions and activation functions; S4.3: Calculate the normalized error: in To prevent division by zero of small constants; S4.4: Simultaneously optimize reconstruction loss during training phase Loss estimation by sum of standard deviation : 。 8. The industrial defect detection method based on multi-scale Transformer according to claim 1, characterized in that: Step S5 specifically includes the following steps: S5.1: Normalization error Perform average pooling operations at four different scales, with pooling kernel sizes of respectively ; S5.2: Upsample the pooling results at the four scales to the original size. ; S5.3: Perform feature stitching between the original error map and the smoothed results at four scales; S5.4: Use Convolution is used to weight and fuse the splicing results: Weights set to ; S5.5: Upsample the fused fractional image to the input image size to obtain the final anomaly fractional image. .

9. The industrial defect detection method based on multi-scale Transformer according to claim 1, characterized in that: Step S6 specifically includes the following steps: S6.1: Anomaly score plot Normalization is performed, min-max normalization to... ; S6.2: Calculate image-level anomaly scores: Take The maximum value is used as the anomaly score for the entire image; S6.3: Calculate pixel-level anomaly localization: Areas exceeding a preset threshold are marked as defect locations. S6.4: Calculate the AUROC index to evaluate the detection performance.

Images