A method for constructing a surface defect detection model based on RecTransformer

Abstract

The present invention discloses a method for constructing a surface defect detection model based on RecTransformer, which belongs to the field of image processing. By establishing a global attention mechanism, global information is fully utilized to complete defect samples. Specifically, the input image is embedded into the latent space, global and local position embedding codes are established, and the input image is serialized into a sequence of image blocks; multiple Transformer encoder modules are stacked to learn latent space features; starting from the global image, local pixel values ​​are inferred, and global feature information is fully utilized to guide the network to identify defect areas; further, a multi-head feature attention mechanism is established to analyze the degree to which each patch in the patch sequence is affected, and the image reconstruction task is converted into patch-level reconstruction. In this way, the present invention can accurately reconstruct defects using a small amount of defect samples as training data.

Description

Technical Field

[0001] The present invention belongs to the field of image processing, and more specifically, relates to a method for constructing a surface defect detection model based on RecTransformer. Background Art

[0002] Today, people are increasingly reliant on display screens. With the proliferation of displays in everything from televisions and smartphones to car dashboards and VR headsets, the demand for display quality and clarity is growing stronger. For example, mura refers to uneven brightness on a display, resulting in various artifacts. Mura on a display screen, also known as brightness non-uniformity, detracts from the user's viewing experience and may hinder the display's performance or functionality.

[0003] Mura is a potential side effect of today's display screen (LCD and OLED) manufacturing processes. Displays are typically composed of multiple layers of materials and substrates bonded together. Bonding all of these layers with absolute precision is nearly impossible every time; seams, migrations, contaminants, bubbles, or other imperfections can creep in. Another cause of mura can be tension in the display panel itself, which can be caused by mechanical stress, vibration, shock, or squeezing or twisting during assembly of the display panel into the device housing or in embedded display production environments. As displays become increasingly miniaturized, such as those used in ultra-thin panels and smart wearables, the potential for mura during manufacturing will only increase. Therefore, timely and accurate detection of various defects on the display surface, including mura, is crucial.

[0004] As GPU processing power becomes increasingly powerful, large and deep neural networks are becoming the mainstream approach in anomaly detection. Existing methods are generally based on pre-trained models, further analyzing the extracted features to analyze defect attributes and determine defect locations. This type of method is often referred to as a latent space embedding method. This type of method introduces additional training sets, which typically improves the expressive power of the pre-trained feature extractor and allows for fine-tuning on a specific dataset to adapt to the detection of various surface anomalies. However, the training and testing costs of this type of method are typically high, and pre-trained feature extractors are typically composed of larger backbone networks, such as ResNet-50 and ResNet-101. Although the detection results are impressive, their time efficiency is low and the network complexity is high, making them difficult to meet the needs of some specific industrial production environments. Summary of the Invention

[0005] In response to the defects and improvement needs of the existing technology, the present invention provides a method for constructing a surface defect detection model based on RecTransformer. While using a small number of defect samples, various types of surface defects including mura can be accurately detected. RecTransformer simplifies the defect detection problem into a patch-level image completion problem. Without using convolution, the block image is completed into a defect-free reconstructed image using the Transformer model. This application establishes global semantic information, establishes an attention mechanism in the patch sequence, and uses position encoding to determine the spatial information of the patch to complete the reconstruction of the global image.

[0006] To achieve the above objectives, in a first aspect, the present invention provides a method for constructing a surface defect detection model based on RecTransformer, comprising:

[0007] Build a surface defect detection model including a position embedding module, a backbone network, and a reconstruction module;

[0008] The position embedding module is used to encode each patch of the original image to obtain a patch sequence of each patch; for any patch, a global position embedding vector is established using the position relationship of the patch in the original image; L×L patches containing the patch are selected to form a window, and a local position embedding vector is established using the position relationship of the patch in the window; the patch sequence, the global position embedding vector and the local position embedding vector of the patch are fused to obtain a patch embedding sequence of the patch; and the patch embedding sequences of all patches are stacked;

[0009] The backbone network consists of multiple standard Transformer encoders stacked in series, each of which includes a multi-head feature attention layer and a multi-layer perceptron;

[0010] The multi-head feature attention layer is used to obtain the query vector q, key vector k and value vector v according to the stacked patch embedding sequence, and then split the query vector q, key vector k and value vector v into h parts; for each query vector q i , key vector k i Find the similarity, as the corresponding value vector v i The weights of, where i∈{1,...,h}, all the value vectors v containing weights i Combining to obtain a feature Z containing global information; reducing the dimension of the feature Z to the dimension of the stacked patch embedding sequence;

[0011] The multi-layer perceptron is used to decode the features after dimensionality reduction to obtain a new patch embedding sequence as the input of the next encoder;

[0012] The reconstruction module is used to reconstruct each patch according to the patch embedding sequence output by the backbone network;

[0013] A surface defect detection model is trained by randomly selecting a patch with the goal of minimizing the pixel value difference and structural similarity between the reconstructed patch and the original patch.

[0014] Furthermore, the multi-head feature attention layer is further used to fuse the reduced-dimensional features with the input of the multi-head feature attention layer as the input of the multi-layer perceptron;

[0015] The multi-layer perceptron is also used to fuse the new patch embedding sequence with the input of the multi-layer perceptron as the input of the next encoder.

[0016] Furthermore, each encoder also includes two layer normalizations, one layer normalization is located before the multi-head feature attention layer, and is used to normalize the input of the multi-head feature attention layer; the other layer normalization is located before the multi-layer perceptron, and is used to normalize the input of the multi-layer perceptron.

[0017] Furthermore, the positional relationship of the patch in the original image is expressed as:

[0018] f g (x,y)=(x-1)·N+y

[0019] The position relationship of the patch in the window is expressed as:

[0020] f l (x,y)=(xr)·L+y-c+1

[0021] Where x and y represent the row and column numbers of the patch in the grid into which the original image is divided, respectively; N = H / K represents the number of grids decomposed per row, H represents the width of the original image, and K represents the size of the square patch; (r, c) represents the row and column coordinates of the patch in the upper left corner of the window in the original image.

[0022] Furthermore, based on the position embedding method of sine and cosine functions, a global position embedding vector and a local position embedding vector are obtained.

[0023] Furthermore, each patch is reconstructed according to the patch embedding sequence output by the backbone network, specifically:

[0024] The patch embedding sequence output by the backbone network is projected and mapped into image blocks as the reconstructed patch.

[0025] In a second aspect, the present invention provides a surface defect detection method based on RecTransformer, characterized by comprising:

[0026] Divide the image to be inspected into several patches, and input the patches into the surface defect detection model constructed by the method for constructing a surface defect detection model based on RecTransformer described in the first aspect, to obtain a reconstructed patch for each patch;

[0027] All reconstructed patches are combined to obtain a reconstructed image, and the reconstructed image is subtracted from the image to be detected to obtain the detection result.

[0028] In a third aspect, the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium includes a stored computer program, wherein, when the computer program is run by a processor, it controls the device where the storage medium is located to execute the method for constructing a surface defect detection model based on RecTransformer as described in the first aspect and / or the surface defect detection method based on RecTransformer as described in the second aspect.

[0029] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects:

[0030] (1) The present invention refers to the general form of Transformer and stacks multiple Transformer sub-blocks to analyze image features. Starting from the global image, the local pixel values ​​are inferred, and the global feature information is fully utilized to guide the network to identify defect areas; further, a multi-head feature attention mechanism is established to analyze the degree to which each patch in the patch sequence is affected, and the image reconstruction task is converted into patch-level reconstruction. Unlike convolutional neural networks that increase the receptive field through layer-by-layer convolution kernels, resulting in insufficient utilization of global information, this application can solve the problem of poor expression of large-scale defects in local perception.

[0031] (2) The present invention uses a stack of multiple encoder modules to form a backbone network. PEM serializes the input image into patches and performs position embedding at both local and global scales, thereby enhancing the efficiency of feature utilization. Multi-head feature attention (MFSA) helps the network establish an attention mechanism and use the entire image information to reconstruct the patch, thereby obtaining a complete reconstructed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1It is a schematic diagram of the structure of the RecTransformer network designed by the present invention;

[0033] Figure 2 Schematic diagram of image block position encoding in the RecTransformer network designed by the present invention, where (a) is global position encoding and (b) is local position encoding;

[0034] Figure 3 This is a schematic diagram of the encoder structure in the RecTransformer network designed by the present invention;

[0035] Figure 4 This is a schematic diagram of the multi-head feature attention MFSA calculation process in the RecTransformer network provided by the present invention. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0037] In the present invention, the terms "first", "second", etc. (if any) in the present invention and the drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0038] The Transformer originated in the field of Natural Language Processing (NLP). After BERT, using the Transformer, demonstrated strong dominance across various NLP tasks, this model architecture quickly came to dominate the field. Inspired by its capabilities, researchers discovered that the architecture also exhibited impressive performance in computer vision tasks. In the decade since CNNs dominated the field of vision, the Transformer has emerged as a powerful emerging research framework.

[0039] The Transformer's overall architecture consists of an encoder-decoder module, each composed of several stacked encoders and decoders. Typically, the encoder consists of a self-attention layer and a feedforward neural network (FNN); the decoder consists of a masked self-attention layer, a self-attention layer, and a feedforward neural network. Unlike the local perception of convolutional networks, global attention allows the Transformer to express features more comprehensively and specifically in certain situations.

[0040] The self-attention mechanism is the core of the Transformer. First, the self-attention mechanism calculates three vectors, denoted as query, key, and value. In the Transformer, these three vectors are obtained by multiplying the input embedding vector (Embedding) and the parameter matrix, as shown in the following formula:

[0041]

[0042] Among them, W Q 、W K and W V Represents the parameter matrix of the corresponding vector. Its parameters are randomly initialized during training and are updated as training progresses. Secondly, the score of self-attention is calculated, that is, the degree of attention paid to other parts of the input sequence after encoding the input at a certain position. The calculation of attention is as follows:

[0043]

[0044] Among them, K T represents the transposed matrix of K; d k is the dimension of the vector.

[0045] Transformers are gradually emerging in every subtask of vision, with the Swin Transformer (Shifted Windows Transformer) achieving exceptional results across all three major vision tasks. These methods offer a broader research perspective, employing pre-training on large datasets and fine-tuning on specific downstream tasks, achieving similarly remarkable results.

[0046] However, this performance improvement comes at a doubled cost. All Transformers have large parameter sizes, making complex models unsuitable for industrial inspection needs and prohibitively expensive models from being deployed on-site. A lightweight and efficient Transformer network is suitable for detecting various surface defects, including mura defects in new display devices.

[0047] The key to large-scale surface defect detection is to be able to analyze abnormal features from global features in order to detect various types of defects, including mura. Mura defects often have variable shapes, and most mura defects exhibit large-scale characteristics. Therefore, accurate detection of defects of different scales becomes the key. The use of global information is a shortcoming of CNN-based methods. Due to its local perception characteristics, it is less effective for large-scale defect detection. In response to the above problems, the present invention proposes a method for constructing a surface defect detection model based on RecTransformer. By using the attention mechanism of the Transformer network, accurate detection of various types of surface defects, including mura, is achieved using a small number of real defect samples.

[0048] like Figure 1 As shown in Figure 1, the surface defect detection model, namely the RecTransformer network, consists of a position embedding module, a backbone network, and a reconstruction module. The following is a detailed description of each module.

[0049] (1) Position Embedding Module (PEM)

[0050] Unlike text input, which inherently has a sequential order, image patch sequences do not. Therefore, positional information must be encoded into the original input to form a position embedding, enabling the network to fully utilize the positional information between patches. PEM establishes local and global position embedding vectors to improve feature utilization.

[0051] Specifically, for the original image I n , the present invention satisfies Where H and W represent the original image I n The width and height of the image, C represents the original image I n In this invention, K is set as the size of the square patch, so the original image is decomposed into N and M small patches in the width and height directions respectively. Each patch can be expressed as:

[0052]

[0053] Where N = H / K and M = W / K represent the number of decomposed grids. In general, Represents the input image I n After the grid is divided, the patch at row i and column j.

[0054] The present invention aims to reconstruct a local feature through global information, that is, in the original image I nIn , L×L patches are selected again to form a window, and all patches are used to reconstruct the features corresponding to a specific patch in the window to achieve patch-level image completion.

[0055] Therefore, the window in the L×L area can be expressed as follows:

[0056] Window={r,...,r+L-1}×{c,...,c+L-1} (4)

[0057] Generally, (r,c) represents the window in the original image I n The relative coordinates in the original image I, that is, the patch in the upper left corner of the window n The corresponding row and column coordinate positions in .

[0058] like Figure 2 As shown, within the window, a local patch position relationship can be established; when looking at the entire image, the patches within the window can establish a global position relationship. Regarding the position embedding method, experiments in ViT have shown that different position embedding methods are not much different. Therefore, this invention also uses the simplest one-dimensional position embedding. The global and local one-dimensional sequence mapping can be expressed as:

[0059] f g (i,j)=(i-1)·N+j (5)

[0060] f l (i,j)=(ir)·L+j-c+1 (6)

[0061] To ensure that a fixed-length sequence is fed into the Transformer network, the position information needs to be added after the patch is serialized and converted into a latent space vector with dimension D. This paper uses the classic trigonometric function position embedding method, and the position embedding of the patch sequence can be expressed as follows:

[0062]

[0063] Here, k is the position of a specific patch in the sequence, k∈{1,...,L·L} in the local sequence and k∈{1,...,N·M} in the global sequence. D is the embedding dimension of the Transformer network input.

[0064] Therefore, the input to the Transformer network can be represented as a linear projection of the position embedding as follows:

[0065]

[0066] in, Represents the parameter matrix of the linear projection. Let the global coordinates of the reconstructed patch be (a, b), then (i, j) represents all patches in the patch sequence that do not contain the coordinates (a, b). In order to reconstruct a specific patch with the coordinates (a, b), a similar position embedding method is used to process it, as shown in the following formula:

[0067]

[0068] From formula (8) and formula (9), we can get: and Both are D-dimensional latent space embeddings. Therefore, the final patch embedding sequence of the input Transformer network can be obtained:

[0069] y=concat[c;y 1 ;...;y N·M ] (10)

[0070] Among them, concat[·] means embedding each input into a stack. Represents a sequence of length N·M, and then the embedding composed of all input patches is used as the input of the backbone network.

[0071] (2) Backbone Network

[0072] The backbone network uses several Transformer encoders to stack. In this invention, a total of n encoders are designed to be stacked. Figure 3 As shown, each encoder is designed with a multi-head feature attention layer and a multi-layer perceptron. Layer normalization is designed before each parameter layer. To accelerate convergence and improve the network's ability to express details, a residual connection is performed after the two parameter layers. Each encoder module is calculated as follows:

[0073]

[0074] Among them, y in =PEM(flatten(I n )), represents the original image I n Flattening into a patch sequence and performing position embedding; LN(·) represents the layer normalization operation. Since the image is based on feature information perceived by the global field of view, the Transformer in the field of image processing usually does not require mask attention.

[0075] Each MLP layer is designed as a three-layer perception network, and the nonlinear transformation of the features is And use the GELU activation function for activation. GELU can be seen as a combination of dropout and RELU activation. Its expression is as follows:

[0076]

[0077] The stacking of multiple encoder modules leads to a deeper network. In the early stages of the network, the attention weights are also concentrated on the spatial background, and the complete input image has been integrated in the shallow layers. In order to pass the feature information of the shallow layers of the network to the deeper parts, the backbone network establishes a cross-connection between the encoder modules. The U-net structure connecting the first module and the last module is used for easy expression. Figure 1 Spanning connections are not shown.

[0078] like Figure 4 As shown in Figure 1, self-attention is the most important component of Transformer and an important operation for establishing patch completion in this invention. Self-attention is applied to the latent space embedding y after image serialization.

[0079] Specifically, the input of the RecTransformer network is The length of the patch embedding sequence is N·M. Referring to the attention mechanism of the original Transformer, the query vector q, key vector k and value vector v are defined in this invention and obtained by the following formula:

[0080]

[0081] in, Represents the corresponding parameter matrix. The query vector q, key vector k and value vector v can be divided into h parts, that is, Where D′ = D / h. Attention calculation is performed on each small part after segmentation, so h represents the "number of heads".

[0082] For each head i∈{1,...,h}, the split query vector q, key vector k and value vector v can be expressed as The attention calculation of each head is as follows:

[0083]

[0084]

[0085] in, Represents the attention score matrix calculated by each head. To implement multi-head attention, the attention scores calculated by all heads are stacked as follows:

[0086] Z=concat[Z1;Z2;...;Z h ] (16)

[0087] The multi-head feature attention mechanism can be expressed as:

[0088] MSA(y)=ZW MSA =[Z1; Z2; ...; Z h ]W MSA (17)

[0089] Among them, W MSA Represents a learnable parameter matrix that satisfies The image completion hypothesis of the present invention can be implemented using the aforementioned multi-head attention mechanism. However, in anomaly detection, especially mura defect detection, after the image is segmented into a sequence of patches, it is found that each patch that does not contain a defect exhibits great similarity. This results in the calculated value of Equation (14) not being very discriminative, reducing the effectiveness of the attention mechanism.

[0090] This paper optimizes the attention score calculation of the query vector q and the key vector k, increasing the parameter complexity of both to alleviate the "underfitting" problem that arises here. As shown below:

[0091]

[0092] The above formula shows that the full expression of features is achieved by designing a multi-layer perceptron. The MLP set in this invention contains only one hidden layer and is still activated by the GELU function shown in formula (12). D″ is used to represent the dimension after dimensionality reduction, and the number of heads D″ = D / h′. For each head i∈{1,...,h′}, the query vector q and key vector k after segmentation can be expressed as Then equations (14) and (15) can be expressed as:

[0093]

[0094]

[0095] Therefore, according to formula (17), the multi-head feature attention used in the present invention can be specifically expressed as:

[0096] MFSA(y)=Z′W MSA =[Z′1;Z′2;...;Z′ h ]W MFSA (twenty one)

[0097] The multi-head attention mechanism establishes relationships across the entire image scale. Unlike the local perception characteristics of CNN networks, the RecTransformer network uses global information to reconstruct a specific patch with the help of the multi-head feature attention mechanism, which makes better use of features, especially for more effective detection of large-scale mura defects.

[0098] (3) Reconstruction module

[0099] The reconstruction module is used to reconstruct each patch according to the patch embedding sequence output by the backbone network.

[0100] Specifically, the input and output dimensions of each encoder are consistent. After the equal-dimensional transformation of several encoders, in order to restore the image information, another projection vector is used to map the feature vector to an image block, thereby calculating the loss. In the present invention, a fully connected layer is designed to map the feature back to the image block size.

[0101] On the other hand, this application provides the training and testing process of the RecTransformer network:

[0102] During training, RecTransformer randomly selects a patch as the reconstruction target, and the window used for local position embedding is a random window containing the target patch, which is then trained through the RecTransformer network.

[0103] During training, all patches of normal samples are self-completed. For real samples with defects included in the training, their corresponding pixel-level labels are added for auxiliary training. Defect-free locations in the patch are reconstructed, while defective locations are guided to produce large reconstruction errors, thereby drawing more attention to the defects. This application uses pixel-level loss and SSIM loss as training loss functions, constraining reconstruction quality at different levels and emphasizing detailed reconstruction information.

[0104] Specifically, the Structural Similarity Index (SSIM) can be used as a metric to measure the similarity between two given images. It can more accurately measure defects, especially surface texture defects. In this application, pixel-level loss and SSIM loss are used as training loss functions to constrain the reconstruction quality at different levels and highlight the reconstructed details.

[0105] Structural similarity measures the similarity between two patches from three aspects: illumination, contrast, and structure. Contrast and structure This loss can be expressed as follows:

[0106]

[0107] Among them, α, β and γ represent the weight hyperparameters of the three similarities. Generally, brightness By the average value μ between patches p and To reflect; contrast Mainly reflected in the variance of different patches and Structural differences Use the covariance of two patches The above three measurement indicators can be expressed as follows:

[0108]

[0109] The constants c1 and c2 are set to avoid zero values. In the present invention, c1=0.01 and c2=0.03 are set. Contrast and structural differences Taking all three into consideration, the calculation formula of SSIM loss can be obtained from Equation (22) and Equation (23):

[0110]

[0111] Given the original block image p and the reconstructed block image Finally, the loss function of RecTransformer during training is designed to be the sum of two reconstruction losses, as shown in the following formula:

[0112]

[0113] After network training is complete, RecTransformer uses the same residual method to segment defects during the inference phase. This involves reconstructing the entire test image and then subtracting it from the input image to identify areas with significant differences as defects. For any patch with coordinates (a, b), where a∈N and b∈M, RecTransformer reconstructs all patches in the original image to obtain a complete reconstructed image.

[0114] Specifically, the hyperparameters of RecTransformer mainly include n, K, L, D and h.

[0115] n is the number of stacked encoder modules in the RecTransformer backbone network. A larger value for n increases the number of network parameters and model complexity. Defect detection faces high repetitiveness in image backgrounds. To balance detection accuracy and training efficiency, this paper sets n = 8.

[0116] K is the size of each patch in the sequence of patches fed into the RecTransformer. The size of K affects the attention calculations, as it determines the size of the area covered by each patch in the sequence. In this paper, K is set to 16.

[0117] L represents the size of the window used in the local position embedding calculation. The size of L determines the number of patches covered. Local position embedding should use a small window size; a large window size will have a similar effect to global position embedding. In this invention, L is set to 5.

[0118] D is the dimension of the latent space. D measures the characteristic quantity of the latent space. A too large D will increase the complexity of the model and consume more computing resources. In this invention, D=512 and D″=256 are set.

[0119] h is the number of heads in the multi-head feature attention mechanism. This draws on the idea of ​​CNNs, embedding patches in a lower dimension and simultaneously calculating attention in multiple feature spaces, making it easier to learn rich feature information. In this paper, h is set to 8.

[0120] In summary, to address the problem of defect detection on surfaces with complex textured backgrounds when there are only a few defect samples, a surface defect detection model based on RecTransformer is proposed. By establishing a global attention mechanism, global information is fully utilized to complete the defect samples. Specifically, the input image is embedded into the latent space, global and local position embedding codes are established, and the input image is serialized into a sequence of image blocks; multiple Transformer encoder modules are stacked to learn latent space features; a multi-head feature attention mechanism is proposed to fully learn global features, improve information utilization, complete and restore the defect position, and achieve feature learning of surface defects and accurate background reconstruction. Using a small number of defect samples as training data, the RecTransformer algorithm can accurately reconstruct defects and achieve the highest pixel-level segmentation AuROC score of 97.6% in the dataset. Experiments on a general surface defect dataset have demonstrated the effectiveness of the RecTransformer algorithm.

[0121] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for constructing a surface defect detection model based on RecTransformer, characterized in that: include: Build a surface defect detection model including a position embedding module, a backbone network, and a reconstruction module; The position embedding module is used to encode each patch of the original image to obtain a patch sequence of each patch; for any patch, a global position embedding vector is established using the position relationship of the patch in the original image; L×L patches containing the patch are selected to form a window, and a local position embedding vector is established using the position relationship of the patch in the window; the patch sequence, the global position embedding vector and the local position embedding vector of the patch are fused to obtain a patch embedding sequence of the patch; and the patch embedding sequences of all patches are stacked; The backbone network consists of multiple standard Transformer encoders stacked in series, each of which includes a multi-head feature attention layer and a multi-layer perceptron; The multi-head feature attention layer is used to obtain the query vector q, key vector k and value vector v according to the stacked patch embedding sequence, and then split the query vector q, key vector k and value vector v into h parts; for each query vector q i , key vector k i Find the similarity, as the corresponding value vector v i The weights of, where i∈{1,...,h}, all the value vectors v containing weights i Combining to obtain a feature Z containing global information; reducing the dimension of the feature Z to the dimension of the stacked patch embedding sequence; The multi-layer perceptron is used to decode the features after dimensionality reduction to obtain a new patch embedding sequence as the input of the next encoder; The reconstruction module is used to reconstruct each patch according to the patch embedding sequence output by the backbone network; A surface defect detection model is trained by randomly selecting a patch with the goal of minimizing the pixel value difference and structural similarity between the reconstructed patch and the original patch.

2. The method for constructing a surface defect detection model based on RecTransformer according to claim 1, characterized in that: The multi-head feature attention layer is further used to fuse the reduced-dimensional features with the input of the multi-head feature attention layer as the input of the multi-layer perceptron; The multi-layer perceptron is also used to fuse the new patch embedding sequence with the input of the multi-layer perceptron as the input of the next encoder.

3. The method for constructing a surface defect detection model based on RecTransformer according to claim 1 or 2, characterized in that: Each encoder also includes two layer normalizations, one layer normalization is located before the multi-head feature attention layer, which is used to normalize the input of the multi-head feature attention layer; the other layer normalization is located before the multi-layer perceptron, which is used to normalize the input of the multi-layer perceptron.

4. The method for constructing a surface defect detection model based on RecTransformer according to claim 1, characterized in that: The position relationship of the patch in the original image is expressed as: f g (x,y)=(x-1)·N+y The position relationship of the patch in the window is expressed as: f l (x,y)=(x-r)·L+y-c+1 Where x and y represent the row and column numbers of the patch in the grid into which the original image is divided, respectively; N = H / K represents the number of grids decomposed per row, H represents the width of the original image, and K represents the size of the square patch; (r, c) represents the row and column coordinates of the patch in the upper left corner of the window in the original image.

5. The method for constructing a surface defect detection model based on RecTransformer according to claim 1 or 4, characterized in that: Based on the position embedding method of sine and cosine functions, the global position embedding vector and the local position embedding vector are obtained.

6. The method for constructing a surface defect detection model based on RecTransformer according to claim 1, characterized in that: The patch embedding sequence output by the backbone network is used to reconstruct each patch, specifically: The patch embedding sequence output by the backbone network is projected and mapped into image blocks as the reconstructed patch.

7. A surface defect detection method based on RecTransformer, characterized in that: include: Dividing the image to be inspected into a plurality of patches, inputting the patches into a surface defect detection model constructed by the method for constructing a surface defect detection model based on RecTransformer according to any one of claims 1 to 6, and obtaining a reconstructed patch for each patch; All reconstructed patches are combined to obtain a reconstructed image, and the reconstructed image is subtracted from the image to be detected to obtain the detection result.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed by a processor, the device where the storage medium is located is controlled to execute the method for constructing a surface defect detection model based on RecTransformer as described in any one of claims 1 to 6 and / or the surface defect detection method based on RecTransformer as described in claim 7.

Images