Industrial defect detection method based on reverse distillation learning

Abstract

The invention discloses an industrial defect detection method based on reverse distillation learning, which relates to the technical field of industrial defect detection, and comprises the steps of generating and filtering a noise picture, and fusing reverse distillation learning with an attention mechanism. According to the method, the fine defect detection and positioning capability under the complex background is improved, and particularly, the defects of small defect area and low background contrast ratio under the condition of complex background texture or relatively high noise level are overcome; according to the performance, the image level AUROC is 99.0%, the pixel level AUROC is 98.23%, the AUPRO can reach 95.02%, and the AUROC and the AUPRO are far higher than other indexes based on distillation learning. In the aspect of real-time performance, the detection speed of a single picture is high and can reach 25ms, the real-time performance is in the front, and the defect detection performance and the defect positioning performance are higher; noise synthesis and filtering are used for improving the small defect detection rate and reducing the false detection rate; the MFF + attention mechanism is used for improving the logic defect detection rate.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial defect detection, and in particular to an industrial defect detection method based on reverse distillation learning. Background Art

[0002] Unsupervised learning mainly includes methods based on image reconstruction, methods based on knowledge distillation, methods based on feature distance, and methods based on self-supervision. Image reconstruction is mainly based on AE, VAE, and GAN. The parameters of the model are only obtained by training normal samples. The model can only reconstruct normal samples well, but will produce large reconstruction errors in the defective areas of defective samples. In the knowledge distillation method, the teacher-student framework is used. First, the representation ability of the large pre-trained network is transferred to the lightweight teacher network through knowledge distillation. Then, multiple randomly initialized student networks are trained only on the normal data set so that their expression of normal samples is the same as that of the teacher. In order to further enhance the effect, the asymmetry of the results between the teacher and the student is also used to introduce student-teacher feature pyramid matching. The method based on feature distance is SPADE, which is based on the KNN algorithm. First, the CNN model pre-trained on ImageNet is used to extract the feature vectors of the training set to build a normal sample database. Then, KNN is used to retrieve the K normal samples that are most semantically similar to the sample to be tested. Finally, the feature pyramid method is used for multi-scale feature alignment. Another method is to use a pre-trained CNN to generate a feature library, and then model the normality of a multivariate Gaussian distribution for each position. In the test phase, the Mahalanobis distance between the feature of each position and the "standard feature template" is measured to generate the final anomaly map. Self-supervised learning is a type of unsupervised learning. There are two main types of methods based on self-supervised learning: image restoration and image synthesis. The main idea of ​​image restoration is to explicitly train the model to restore the transformed input image to the original normal image. In the test phase, the pixels in the normal area remain basically unchanged, and the abnormal area is erased, and then defect detection is achieved based on the reconstruction error. The defect synthesis-based method constructs defects in normal images to obtain defect samples and their labels, thereby training the model using contrast learning or supervised methods.

[0003] Compared with traditional learning methods, various methods based on unsupervised learning have greatly improved accuracy and performance, and the requirements for data sets are greatly reduced compared to supervised learning. Due to its ability to detect unknown defects and the advantage of not requiring pixel-level annotation, it has great advantages in complex industrial scenarios. However, there are still some problems, such as poor detection and positioning of subtle defects in complex backgrounds, poor detection of logical defects, and high demands for real-time and accuracy in industrial scenarios.

[0004] In view of this, an industrial defect detection method based on reverse distillation learning is provided to overcome the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide an industrial defect detection method based on reverse distillation learning to solve the problems raised in the above background technology.

[0006] In order to solve the above technical problems, the present invention provides an industrial defect detection method based on reverse distillation learning, including the generation and filtering of noise images, and the fusion attention mechanism of reverse distillation learning;

[0007] The generation and filtering of noise images includes the following steps:

[0008] Step 1: Generate abnormal images;

[0009] Step 2: Noise filtering: Design a filter layer to filter abnormal features;

[0010] Reverse distillation learning fusion attention mechanism includes the following steps:

[0011] S1, reverse distillation learning: The teacher-student model is defined as a teacher encoder-student decoder structure. After the image passes through the teacher encoder, three groups of feature maps E1, E2, and E3 are generated respectively, and the corresponding channel numbers are 256, 512, and 1024 respectively. Then, the image passes through the filter layer F (E1, E2, E3) to obtain , filter out potential abnormal features; fuse the features of each layer obtained by filtering; add an attention mechanism module; finally, filter the feature information by using the fourth layer structure (ConvBlock) of ResNet, remove a large amount of redundant information, and project the high-dimensional representation of the teacher model into a low-dimensional space. The compact features of the low-dimensional space serve as an information bottleneck to further prevent the propagation of abnormal features to the student model; after passing through the student decoder, three groups of feature maps D1, D2, and D3 are generated respectively. A distillation loss function is designed to optimize the features reconstructed by the student decoder. The formula is as follows:

[0012]

[0013] Among them, K is the number of feature layers used in training, (h, w) is the height and width of the corresponding feature map; during training, the total loss function is as follows:

[0014] L total =αL recon +L KD

[0015] Among them, L recon is the reconstruction loss, L KD is the distillation loss between teacher and student, α is the regularization parameter;

[0016] S2, attention module: CA decomposes channel attention into two one-dimensional feature encoding processes, aggregating features along two spatial directions respectively; then the generated feature maps are encoded into a pair of direction-aware and position-sensitive attention maps respectively; given the input X, a pooling kernel of size (H, 1) or (1, W) is used to encode each channel along the horizontal and vertical coordinates; therefore, the c-th channel with height h can be expressed as:

[0017]

[0018] Similarly, the output of the cth channel with width w can be expressed as:

[0019]

[0020] These two transformations allow the attention module to capture long-term dependencies along one spatial direction and preserve precise position information along another spatial direction; after the transformation in the information embedding, this part connects the above transformations and then changes them using the 1*1 convolution transformation function F1, as follows:

[0021] f=δ(F1([z h ,z w ]))

[0022] Where [,] is the connection operation along the spatial dimension, δ is the nonlinear activation function, and f is the intermediate feature map that encodes the spatial information in the horizontal and vertical directions; then f is decomposed into two separate tensors f along the spatial dimension h ∈R C / r×H and f w ∈R C / r×W , r is the reduction rate; using another two 1*1 convolution F h and F w f h and f w Changing to a tensor with the same number of channels, we get:

[0023] g h =σ(F h (f h ))

[0024] g w =σ(F w (f w ))

[0025] Where σ is the sigmoid activation function; the final output of CA can be expressed as:

[0026]

[0027] Through CA attention, the global dependencies of the feature maps are obtained and passed to the student decoder.

[0028] Furthermore, in the process of generating abnormal images, we define (h noise ,w noise ) is the noise level, (Hh noise , Ww noise ) is the position coordinate of the noise in the image, and a random function is used to insert it into different positions to simulate natural defects; in the process of generating simplex noise, the coordinate transformation is first performed. In 3D space, x'=x+(x+y+z)*F, y'=y+(x+y+z)*F, z'=z+(x+y+z)*F, where (x', y', z') are the coordinates of the positive superlattice, (x, y, z) are the coordinates of the original superlattice, and F is the skew coefficient. The formula is as follows:

[0029]

[0030] Where n represents the dimension.

[0031] Furthermore, one of the 3D superlattices is composed of six 3D simplexes. Therefore, it is necessary to determine in 3D which of the six 3D simplexes the input point is located, and to determine which simplex it is located in by comparing the values.

[0032] Furthermore, in the process of generating abnormal images, after obtaining the vertices of the simplex, the vertex gradient vector in the permutation arrangement table is obtained by calculating the hash value, and finally the distance vector from the input point to the vertex is calculated. A radial attenuation function is used to calculate the contribution value of each vertex and then sum them up. The function for calculating the vertex contribution value is as follows:

[0033] (max(0,r 2 -|dist| 2 )) 4 ×dot(dist,grad)

[0034] Where dist is the distance vector from the input point to the vertex, grad is the stored pseudo-random gradient vector, and r 2 The value is 0.5 or 0.6, and the result is normalized.

[0035] Furthermore, in step 2, first, the teacher model and the noise model use the same network architecture to extract feature information at each level. The network architecture uses pre-trained ResNet and WideResNet. After the teacher model and the noise model extract the corresponding first three layers of features, the feature information of each layer will be input into the corresponding filter layer, where the filter layer is composed of 3*3 convolution, InstanceNorm2D, and ReLU, with a total of four groups.

[0036] Furthermore, we set f i,k , are the features of the normal image output by the encoder and the features of the abnormal image output by the encoder;

[0037] f i,k =E k (x i ), k represents the output features of different layers, x i represents the normal input image, ε(x i ) represents a pseudo-abnormal image synthesized by noise; in order to enable the filter layer to filter the abnormal feature information and make the normal feature information complete and compact, a loss function is designed as follows:

[0038]

[0039] During the training process, abnormal signals are injected into the feature space through pseudo-abnormal input images, and the filter layers are encouraged to learn how to reconstruct normal features from pseudo-abnormal regions.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] Improve the ability to detect and locate subtle defects in complex backgrounds, especially those with small defect areas and low contrast with the background when the background texture is complex or the noise level is high.

[0042] In terms of performance, the image-level AUROC is 99.0%, the pixel-level AUROC is 98.23%, and AUPRO can reach 95.02%, which are much higher than other indicators based on distillation learning.

[0043] In terms of real-time performance, the detection speed of a single image is fast, reaching 25ms, ranking among the top in real-time performance, with higher defect detection and defect location performance.

[0044] Noise synthesis and filtering are designed to improve the detection rate of small defects and reduce the false detection rate; the MFF+attention mechanism is designed to improve the detection rate of logical defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1A schematic diagram of an industrial defect detection method based on reverse distillation learning according to the present invention;

[0046] Figure 2 A schematic diagram of simplex vertex selection of an industrial defect detection method based on reverse distillation learning according to the present invention;

[0047] Figure 3 A schematic diagram of a noise filtering layer in an industrial defect detection method based on reverse distillation learning according to the present invention;

[0048] Figure 4 A schematic diagram of a feature fusion layer in an industrial defect detection method based on reverse distillation learning according to the present invention;

[0049] Figure 5 Schematic diagram of CoordinateAttention in an industrial defect detection method based on reverse distillation learning in the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] See also Figure 1-Figure 5 , the present invention provides a technical solution:

[0052] See also Figure 1 As shown in the figure, an industrial defect detection method based on reverse distillation learning adopts two technologies: the generation and filtering of noise images, and the fusion of reverse distillation learning and attention mechanism.

[0053] (1) Generation and filtering of noise images:

[0054] Step 1: Abnormal image generation. In order to add noise to the normal image to simulate subtle defects, various noises are analyzed, including Gaussian noise, Perlin noise, simplex noise, etc. After comprehensive analysis, simplex noise has much less parameter calculation than Perlin noise, and is better than Gaussian noise in simulating abnormal distribution based on power law, and the noise it generates is more natural. Definition (h noise ,w noise ) is the noise level, (Hh noise , Ww noise) is the position coordinate of the noise in the image, and a random function is used to insert it into different positions to simulate natural defects. In the process of generating simplex noise, the coordinate transformation is first performed. In 3D space, x'=x+(x+y+z)*F, y'=y+(x+y+z)*F, z'=z+(x+y+z)*F, where (x', y', z') are the coordinates of the positive superlattice, (x, y, z) are the coordinates of the original superlattice, and F is the skew coefficient. The formula is as follows:

[0055]

[0056] Where n represents the dimension. Then determine the simplex. A 3D superlattice is composed of six 3D simplexes. Therefore, in 3D, it is necessary to determine which of the six 3D simplexes the input point is in. By comparing the size of the (x, y, z) values, see Figure 2 shown.

[0057] After obtaining the vertices of the simplex, the vertex gradient vector in the permutation arrangement table is obtained by calculating the hash value, and finally the distance vector from the input point to the vertex is calculated. A radial attenuation function is used to calculate the contribution value of each vertex and then sum them. The function for calculating the vertex contribution value is as follows:

[0058] (max(0,r 2 -|dist| 2 )) 4 ×dot(dist,grad)

[0059] Where dist is the distance vector from the input point to the vertex, grad is the stored pseudo-random gradient vector, and r 2 The value is 0.5 or 0.6, and the result is normalized.

[0060] Step 2: Noise filtering.

[0061] In order to ensure that the feature information generated by the teacher encoder only passes normal features to the student decoder and avoid subtle defect features being passed to the student decoder, causing the student to reconstruct abnormal features, a filtering layer is designed to filter abnormal features.

[0062] First, the teacher model and the noise model use the same network architecture to extract feature information at each level. The network architecture uses pre-trained ResNet and WideResNet. After the teacher model and the noise model extract the corresponding first three layers of features, the feature information of each layer will be input into the corresponding filter layer. The filter layer is composed of 3*3 convolution, InstanceNorm2D, and ReLU. There are four groups in total. Figure 3 As shown. Set f i,k , is the feature of the normal image output by the encoder and the feature of the abnormal image output by the encoder. i,k =E k (x i ), k represents the output features of different layers, x i represents the normal input image, ε(x i ) represents a pseudo-abnormal image synthesized by noise. In order to enable the filter layer to filter the abnormal feature information and make the normal feature information complete and compact, a loss function is designed as follows:

[0063]

[0064] During the training process, the abnormal signals are injected into the feature space through pseudo-abnormal input images, and the filter layer is encouraged to learn how to reconstruct normal features from the pseudo-abnormal regions. By optimizing this learning objective, the ability of the filter layer to suppress abnormal information during the inference process is accelerated.

[0065] (2) Reverse distillation learning fusion attention:

[0066] S1: Reverse distillation learning.

[0067] Conventional distillation learning is divided into two modules, one is the teacher module and the other is the student module. The teacher module is a pre-trained large network, and the student network uses a neural network similar to or the same as the teacher model. It accepts raw digital images as input, and the goal is to match its features with those of the teacher. When detecting abnormal samples, the student model is expected to differ from the teacher. However, this difference sometimes disappears, resulting in missed anomalies. This problem is attributed to the similar architecture of the teacher and student networks and the same data flow during the TS knowledge transfer process. Reverse distillation learning is based on the idea of ​​distillation learning. It defines the teacher-student model as a teacher encoder-student decoder structure. After the image passes through the teacher encoder, three groups of feature maps E1, E2, and E3 will be generated respectively, with the corresponding number of channels being 256, 512, and 1024, respectively, and then passed through the filter layer F (E1, E2, E3) to obtain Filter out potential abnormal features and make normal features more compact.

[0068] The features of each layer obtained by filtering are fused, see Figure 4 As shown in the figure, through multi-scale feature fusion (MFF), the low-level information extracted from the shallow layer, such as color, texture, edge, etc., can be fused with the deep high-level information, global semantics and structural information, and all the effective information of the image can be passed to the student decoder.

[0069] After MFF, in order to better detect logical defects, an attention mechanism module is added to enable it to better obtain global features. Finally, the feature information is filtered by using the fourth layer structure (ConvBlock) of ResNet to remove a large amount of redundant information. At the same time, the high-dimensional representation of the teacher model is projected into a low-dimensional space. The compact features of the low-dimensional space serve as an information bottleneck to further prevent the propagation of abnormal features to the student model. After passing through the student decoder, three groups of feature maps D1, D2, and D3 are generated respectively. A distillation loss function is designed to optimize the features reconstructed by the student decoder. The formula is as follows:

[0070]

[0071] Among them, K is the number of feature layers used in training, and (h,w) is the height and width of the corresponding feature map. During training, the total loss function is as follows:

[0072] L total =αL recon +L KD

[0073] Among them, L recon is the reconstruction loss, L KD is the distillation loss between teacher and student, and α is the regularization parameter.

[0074] S2: Attention module.

[0075] In order to improve the ability to detect and locate logical defects, which are often caused by misplaced or redundant objects, it is necessary to capture global semantic and structural information for students to learn. When a logical defect exception occurs, the teacher model will extract features normally, while the student model will reconstruct the missing or misplaced objects due to its full understanding of the global structure. The difference between the two is the location of the defect. In order to ensure real-time performance, the attention module used should be simple and have a small number of parameters. For SENet, it has a significant effect on improving model performance, but they usually ignore position information, which is very important for generating spatial selective attention maps. CBAM is based on channel attention and spatial attention, and introduces spatial information encoding through large-scale kernel convolution. It can fully capture spatial information, but it also has some problems, such as over-weighting the global structural weight of a certain class, resulting in poor generalization and easy false detection.

[0076] For Coordinate Attention (CA), it takes into account the positional relationship on the basis of channel attention and integrates channel attention with spatial attention. After experiments, this module has the best effect on logic defect detection and will not cause false detection. The CA flow chart is as follows: Figure 5As shown. CA decomposes channel attention into two 1D feature encoding processes, aggregating features along two spatial directions respectively. In this way, long-range dependencies can be captured along one spatial direction, while precise position information can be retained along another spatial direction. The generated feature maps are then encoded into a pair of direction-aware and position-sensitive attention maps, respectively, which can be complementarily applied to the input feature map to enhance the representation of the object of attention. The specific steps are as follows: given an input X, each channel is encoded along the horizontal and vertical coordinates using a pooling kernel of size (H, 1) or (1, W). Therefore, the cth channel with a height of h can be expressed as:

[0077]

[0078] Similarly, the output of the cth channel with width w can be expressed as:

[0079]

[0080] These two transformations allow the attention module to capture long-term dependencies along one spatial direction and save precise location information along another spatial direction, which helps the network locate the target of interest more accurately. After the transformation in information embedding, this part connects the above transformations and then uses the 1*1 convolution transformation function F1 to change it. The formula is as follows:

[0081] f=δ(F1([z h ,z w ]))

[0082] Where [,] is the connection operation along the spatial dimension, δ is the nonlinear activation function, and f is the intermediate feature map that encodes the spatial information in the horizontal and vertical directions. Then f is decomposed into two separate tensors f along the spatial dimension. h ∈R C / r×H and f w ∈R C / r×W , r is the reduction rate. Using another two 1*1 convolution F h and F w f h and f w Changing to a tensor with the same number of channels, we get:

[0083] g h =σ(F h (f h ))

[0084] g w =σ(F w (f w ))

[0085] Where σ is the sigmoid activation function. The final output of CA can be expressed as:

[0086]

[0087] Through CA attention, the global dependencies of the feature maps are obtained and passed to the student decoder.

[0088] The test results on the industrial dataset MvTec_AD are as follows, which is divided into fifteen categories:

[0089] Image_AUROC Pixel_AUROC AUPRO bottle 100％ 98.78％ 96.95％ cable 98.69％ 98.39％ 94.14％ capsule 97.21％ 98.63％ 95.56％ carpet 100％ 99.26％ 97.86％ grid 100％ 99.31％ 97.67％ hazelnut 100％ 99.17％ 96.3％ leather 100％ 99.43％ 99.19％ metal_nut 100％ 97.87％ 92.27％ Pill 98.06％ 98.12％ 96.84％ screw 95.18％ 99.49％ 97.97％ tile 98.34％ 96.48％ 92.18％ toothbrush 100％ 99.13％ 93.80％ transistor 98.62％ 94.9％ 85.07％ wood 99.47％ 95.73％ 93.41％ zipper 97.56％ 98.7％ 96.06％ Mean 98.93％ 98.23％ 95.02％

[0090] Comparison of single image reasoning time:

[0091] Model Reasoning time STFPM 32ms Fastflow 40ms Draem 15ms Patchcore 179ms RD 33.6ms Ours 25ms

[0092] Summarize:

[0093] Improve the ability to detect and locate subtle defects in complex backgrounds, especially those with small defect areas and low contrast with the background when the background texture is complex or the noise level is high.

[0094] In terms of performance, the image-level AUROC is 99.0%, the pixel-level AUROC is 98.23%, and AUPRO can reach 95.02%, which are much higher than other indicators based on distillation learning.

[0095] In terms of real-time performance, the detection speed of a single image is fast, reaching 25ms, ranking among the top in real-time performance, with higher defect detection and defect location performance.

[0096] Noise synthesis and filtering are designed to improve the detection rate of small defects and reduce the false detection rate; the MFF+attention mechanism is designed to improve the detection rate of logical defects.

Claims

1. An industrial defect detection method based on reverse distillation learning, characterized in that: Including the generation and filtering of noisy images, reverse distillation learning and fusion attention mechanism; The generation and filtering of noise images includes the following steps: Step 1: Generate abnormal images; Step 2: Noise filtering: Design a filter layer to filter abnormal features; Reverse distillation learning fusion attention mechanism includes the following steps: S1, reverse distillation learning: The teacher-student model is defined as a teacher encoder-student decoder structure. After the image passes through the teacher encoder, three groups of feature maps E1, E2, and E3 are generated respectively, and the corresponding channel numbers are 256, 512, and 1024 respectively. Then, the image passes through the filter layer F (E1, E2, E3) to obtain , filter out potential abnormal features; fuse the features of each layer obtained by filtering; add an attention mechanism module; finally, filter the feature information by using the fourth layer structure (ConvBlock) of ResNet, remove a large amount of redundant information, and project the high-dimensional representation of the teacher model into a low-dimensional space. The compact features of the low-dimensional space serve as an information bottleneck to further prevent the propagation of abnormal features to the student model; after passing through the student decoder, three groups of feature maps D1, D2, and D3 are generated respectively. A distillation loss function is designed to optimize the features reconstructed by the student decoder. The formula is as follows: Among them, K is the number of feature layers used in training, (h, w) is the height and width of the corresponding feature map; during training, the total loss function is as follows: L total =αL recon +L KD Among them, L recon is the reconstruction loss, L KD is the distillation loss between teacher and student, α is the regularization parameter; S2, attention module: CA decomposes channel attention into two one-dimensional feature encoding processes, aggregating features along two spatial directions respectively; then the generated feature maps are encoded into a pair of direction-aware and position-sensitive attention maps respectively; given an input X, a pooling kernel of size (H, 1) or (1, W) is used to encode each channel along the horizontal and vertical coordinates; therefore, the c-th channel with height h can be expressed as: Similarly, the output of the cth channel with width w can be expressed as: These two transformations allow the attention module to capture long-term dependencies along one spatial direction and preserve precise position information along another spatial direction; after the transformation in the information embedding, this part connects the above transformations and then changes them using the 1*1 convolution transformation function F1, as follows: f=δ(F1([z h ,z w ])) Where [,] is the connection operation along the spatial dimension, δ is the nonlinear activation function, and f is the intermediate feature map that encodes the spatial information in the horizontal and vertical directions; then f is decomposed into two separate tensors f along the spatial dimension h ∈R C / r×H and f w ∈R C / r×W , r is the reduction rate; using another two 1*1 convolution F h and F w f h and f w Changing to a tensor with the same number of channels, we get: g h =σ(F h (f h )) g w =σ(F w (f w )) Where σ is the sigmoid activation function; the final output of CA can be expressed as: Through CA attention, the global dependencies of the feature maps are obtained and passed to the student decoder.

2. The industrial defect detection method based on reverse distillation learning according to claim 1, characterized in that: In the process of generating abnormal images, define (h noise ,w noise ) is the noise level, (Hh noise ,Ww noise ) is the position coordinate of the noise in the image, and a random function is used to insert it into different positions to simulate natural defects; in the process of generating simplex noise, the coordinate transformation is first performed. In 3D space, x'=x+(x+y+z)*F, y'=y+(x+y+z)*F, z'=z+(x+y+z)*F, where (x', y', z') are the coordinates of the positive superlattice, (x, y, z) are the coordinates of the original superlattice, and F is the skew coefficient. The formula is as follows: Where n represents the dimension.

3. The industrial defect detection method based on reverse distillation learning according to claim 2, characterized in that: One of the 3D superlattices is composed of six 3D simplexes. Therefore, it is necessary to determine in 3D which of the six 3D simplexes the input point is located, and to determine which simplex it is located in by comparing the values.

4. The industrial defect detection method based on reverse distillation learning according to claim 3, characterized in that: In the process of generating abnormal images, after obtaining the vertices of the simplex, the vertex gradient vector in the permutation arrangement table is obtained by calculating the hash value, and finally the distance vector from the input point to the vertex is calculated. A radial attenuation function is used to calculate the contribution value of each vertex and then sum them. The function for calculating the vertex contribution value is as follows: (max(0,r 2 -|dist| 2 )) 4 ×dot(dist,grad) Where dist is the distance vector from the input point to the vertex, grad is the stored pseudo-random gradient vector, and r 2 The value is 0.5 or 0.6, and the result is normalized.

5. The industrial defect detection method based on reverse distillation learning according to claim 1, characterized in that: In step 2, first, the teacher model and the noise model use the same network architecture to extract feature information at each level. The network architecture uses pre-trained ResNet and WideResNet. After the teacher model and the noise model extract the corresponding first three layers of features, the feature information of each layer will be input into the corresponding filter layer, where the filter layer is composed of 3*3 convolution, InstanceNorm2D, and ReLU, with a total of four groups.

6. The industrial defect detection method based on reverse distillation learning according to claim 5, characterized in that: Set f respectively i,k , is the feature of the normal image output by the encoder and the feature of the abnormal image output by the encoder; where f i,k =E k (x i ), k represents the output features of different layers, x i represents the normal input image, ε(x i ) represents a pseudo-abnormal image synthesized by noise; in order to enable the filter layer to filter the abnormal feature information and make the normal feature information complete and compact, a loss function is designed as follows: During the training process, abnormal signals are injected into the feature space through pseudo-abnormal input images, and the filter layers are encouraged to learn how to reconstruct normal features from pseudo-abnormal regions.

Images