A reverse distillation industrial anomaly detection model and a construction method and a detection method thereof

Abstract

The application belongs to the field of anomaly detection, and specifically discloses a reverse distillation industrial anomaly detection model and a construction method and a detection method thereof, which comprises the following steps: training an anomaly detection model through a training set, wherein the anomaly detection model comprises a teacher encoder, a student decoder and an auxiliary classifier, the teacher encoder is pre-trained through normal images, and the student decoder adopts a lightened teacher encoder architecture; the training set comprises an abnormal image, an abnormal label and a category label; during training, the teacher encoder extracts features from the abnormal image to obtain a feature map A; the student decoder performs reverse knowledge distillation based on the feature map A to reconstruct a feature map B; based on the difference between the feature maps A and B, an abnormal position on the abnormal image is determined, i.e., the abnormal label; and the auxiliary classifier obtains the category label based on intermediate features generated by each convolutional layer in the student decoder. The application improves the anomaly detection robustness and classification accuracy in a complex industrial scene.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of anomaly detection, and more particularly relates to a reverse distillation industrial anomaly detection model and a construction method and a detection method thereof. BACKGROUND

[0002] In recent years, the importance of industrial anomaly detection in intelligent manufacturing and quality control has become increasingly prominent. Traditional methods mainly rely on manual inspection or supervised learning models based on deep learning, but face the following problems:

[0003] Manual detection is low in efficiency, high in cost and susceptible to subjective factors; supervised learning requires a large number of labeled anomaly samples, but anomaly samples are scarce in industrial scenarios and labeling costs are high; existing unsupervised methods (such as autoencoders and knowledge distillation) do not require anomaly samples, but have difficulty in distinguishing complex anomaly types and are not sensitive enough to minor defects.

[0004] Reverse distillation (RD) uses a heterogeneous teacher encoder-student decoder architecture to locate abnormal areas using normal data reconstruction error, but has the following limitations:

[0005] Insufficient feature discriminability: traditional RD relies too much on reconstruction loss and ignores explicit modeling of abnormal features; lack of classification ability: unable to distinguish different types of abnormal items; insufficient anomaly localization accuracy: details are blurred due to information loss during propagation. SUMMARY

[0006] To overcome the above defects or improve the needs of the prior art, the present application provides a reverse distillation industrial anomaly detection model and a construction method and a detection method thereof, which aims to achieve high-precision anomaly detection and classification of industrial images.

[0007] To achieve the above purpose, according to the first aspect of the present application, a construction method of a reverse distillation industrial anomaly detection model is provided, comprising the following steps:

[0008] Training the anomaly detection model through a training set to obtain a trained anomaly detection model;

[0009] The anomaly detection model comprises a teacher encoder, a student decoder and an auxiliary classifier, the teacher encoder is pre-trained through normal images, and the student decoder adopts a lightweight processed teacher encoder architecture;

[0010] The training set comprises abnormal images and corresponding abnormal labels and category labels; during training:

[0011] The teacher encoder extracts features from the abnormal images to obtain a feature map A;

[0012] The student decoder reconstructs the feature map B by reverse knowledge distillation through reverse feature propagation based on the feature map A; and determines the abnormal position on the abnormal image, i.e., the abnormal label, based on the difference between the feature map A and the feature map B.

[0013] The auxiliary classifier obtains the class label based on the intermediate features generated by each convolutional layer in the student decoder.

[0014] As a further optimization, when training the abnormality detection model, the loss function L total is:

[0015] L total = L pix + αL cls

[0016] wherein L pix is the feature reconstruction loss of the student decoder, L cls is the classification loss of the auxiliary classifier, and α is a dynamic weight.

[0017] As a further optimization, the feature reconstruction loss L pix uses cosine similarity to measure the difference between the feature map A and the feature map B.

[0018] As a further optimization, the classification loss L cls uses a cross-entropy loss function.

[0019] As a further optimization, the student decoder uses a skip connection to pass the intermediate features generated by the convolutional layers of the teacher encoder to the corresponding convolutional layers of the student decoder, thereby realizing cross-layer feature fusion.

[0020] As a further optimization, the teacher encoder is based on ResNet-50.

[0021] As a further optimization, the lightweight processing is to adjust the channel dimension of the feature map in the teacher encoder through 1x1 convolution, and to use feature normalization to unify the feature distribution of the teacher encoder and the student decoder.

[0022] As a further optimization, in the auxiliary classifier, the weights of the intermediate features generated by each convolutional layer of the student decoder are dynamically allocated, and the sensitivity to image features is enhanced through a spatial attention mechanism.

[0023] According to a second aspect of the present application, there is provided an abnormality detection model constructed using the above-mentioned method for constructing an abnormality detection model through reverse distillation.

[0024] According to a third aspect of the present application, there is provided an abnormality detection method, comprising the following steps: inputting an image obtained by industrial detection into the above-mentioned abnormality detection model to obtain an abnormality label and a class label.

[0025] Overall, compared with the prior art, the above technical solutions conceived by the present application mainly have the following technical advantages:

[0026] 1. The present application realizes high-precision anomaly detection and classification through a bidirectional knowledge interaction architecture: a reverse distillation path is adopted, the student decoder reconstructs the teacher features layer by layer from deep to shallow through reverse feature propagation, captures the normal data distribution, and then obtains the abnormal position; the forward classification path inputs the intermediate layer features of the decoder into the auxiliary classifier, focuses on the abnormal features extracted by the intermediate layer of the student decoder, and explicitly separates the abnormal features; thereby realizing bidirectional knowledge transfer in the student decoder part, not only reconstructing features from deep to shallow, but also updating parameters through forward transmission from shallow to deep, which significantly improves the robustness of anomaly detection and the classification accuracy under complex industrial scene images.

[0027] 2. The present application combines feature reconstruction loss and classification loss to realize joint training, which can optimize the detection and classification tasks simultaneously, and only a small amount of abnormal data is needed to complete the training.

[0028] 3. The student decoder realizes cross-layer feature fusion through a skip connection to enhance the recovery ability of multi-scale features, especially the capture of shallow details; specifically, the skip connection directly transmits the intermediate layer features of the teacher encoder to the corresponding level of the student decoder, solving the problem of detail blurring caused by information loss in the reverse propagation process of traditional decoders, which can avoid the loss of shallow texture, edge and other information in the upsampling process, and also combine deep semantic and shallow local features to improve the accuracy of abnormal positioning. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The figure is a schematic diagram of the overall architecture of the reverse distillation industrial anomaly detection model of the embodiment of the present application.

[0030] Figure 2 The figure is a specific architecture diagram of each module of the anomaly detection model of the embodiment of the present application.

[0031] Figure 3 The figure is a curve showing the influence of the hyperparameter alpha on the detection performance of the embodiment of the present application.

[0032] Figure 4 The figure is a comparison chart of abnormal positioning visualization, where (a) is the original image, (b) is the result of the traditional RD, and (c) is the result of the embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of the present application clearer, further detailed explanations will be given below in combination with the accompanying drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and should not be used to limit the present application. In addition, the technical features involved in the various embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0034] The method for constructing a reverse distillation industrial anomaly detection model provided by the embodiment of the present application comprises the following steps:

[0035] (1) Construction of training set.

[0036] An abnormal image for industrial detection is obtained, and an abnormal label and a category label are obtained by corresponding labeling to construct a training set for joint training. The abnormal label is used to represent the abnormal position on the abnormal image, and the category label is used to represent the category of the article in the abnormal image, such as nut, zipper, etc.

[0037] (2) Abnormal detection model architecture.

[0038] The abnormal detection model comprises a teacher encoder, a student decoder and an auxiliary classifier, as shown in Figure 1 , wherein:

[0039] The teacher encoder extracts features from the input image to obtain a feature map A.

[0040] The student decoder adopts a reverse feature propagation path based on the output of the teacher encoder to perform reverse knowledge distillation, reconstructs the feature map layer by layer from deep to shallow, and obtains a feature map B. Then, based on the difference between the feature maps A and B, an abnormal label can be obtained, which is an abnormal heat map displayed at the pixel level.

[0041] The input of the auxiliary classifier is the intermediate layer feature of the student decoder, which enhances the discriminative ability of the abnormal feature through a contrast learning mechanism to obtain a category label.

[0042] Specifically, the auxiliary classifier outputs as follows:

[0043]

[0044] Wherein, y represents the output, Classifier() represents the classifier, Downsample() represents down-sampling, o0, o1 and o2 represent the features of the three adjacent layers in the student decoder, respectively.

[0045] The auxiliary classifier adopts a cross-layer feature pyramid fusion structure, as shown in Figure 2As shown, the student decoder dynamically allocates the weight of the multi-scale intermediate features generated by each convolutional layer of the student decoder, and enhances the sensitivity of the model to image features through a spatial attention mechanism, so that the student decoder can better understand the features of normal images and improve the effect of anomaly detection.

[0046] Specifically, the student decoder adopts a similar but lighter architecture to the teacher encoder. It should be noted that the features in the teacher encoder are propagated forward from the shallow layer to the deep layer, while the student decoder is used in reverse, reconstructing the features from the deep layer to the shallow layer. When lightening, the channel alignment mechanism is adopted, the channel dimension of the feature map is adjusted through 1x1 convolution, and the feature normalization is used to unify the feature distribution of the teacher encoder and the student decoder, as follows:

[0047] Adjust the channel dimension:

[0048] Feature normalization unifies the distribution:

[0049] wherein, , represent the channel dimension in the student decoder and the teacher encoder, respectively, represents the weight of the 1x1 convolution kernel, and b represents the offset; represents the feature normalization, f K represents the channel dimension, μ and σ are statistical measures of central tendency and dispersion of features, and γ and β are learnable parameters.

[0050] Further, the teacher encoder is constructed based on ResNet-50 and pre-trained. Specifically, ResNet-50 is trained for image classification on the imagenet dataset as pre-training, and after pre-training, the main part of ResNet-50 is intercepted as the teacher encoder, i.e. the image classification part is shielded; then the teacher encoder can output four scale feature maps when detecting anomalies.

[0051] Further, the teacher encoder and the student decoder structure are asymmetric, which can avoid feature homogenization. The student decoder realizes cross-layer feature fusion through a skip connection, enhancing the ability to recover multi-scale features, especially capturing shallow details. Specifically, the skip connection directly transmits the intermediate layer features of the teacher encoder to the corresponding level of the student decoder, so that the intermediate features generated by the convolutional layers of the student decoder are fused with the intermediate features generated by the corresponding convolutional layers of the teacher encoder, and then input into the next convolutional layer of the student decoder. This solves the problem of detail blurring caused by information loss in the reverse propagation process of the traditional decoder, avoids the loss of shallow texture, edge and other information in the upsampling process, and also combines deep semantic and shallow local features to improve the accuracy of anomaly positioning. For example, after the image is input into the teacher encoder, it successively passes through the first, second and third convolutional layers, and then the feature map A passes through the third, second and first convolutional layers in the student decoder. In the decoding process, the output of the third convolutional layer of the decoder is combined with the output of the third convolutional layer of the encoder, and then input into the second convolutional layer of the decoder.

[0052] In addition, the anomaly detection model further includes a neck module, and the features obtained by each convolutional layer of the teacher encoder are input into the neck module to obtain the feature map A. As shown in Figure 2 , the feature a obtained by the first convolutional layer (shallow layer) of the teacher encoder is convolved twice in the neck module, the feature b obtained by the second convolutional layer is convolved once in the neck module, and then combined with the feature c of the third convolutional layer (deep layer) to obtain the feature map A.

[0053] (3) Model training.

[0054] The anomaly detection model is jointly trained by the training set of the joint training, and a trained anomaly detection model is obtained.

[0055] Specifically, the loss function L used in joint training is as follows: total

[0056] L total = L pix + αL cls

[0057] Wherein, L pix is the feature reconstruction loss based on cosine similarity, which is determined based on the difference between the feature maps A and B; L cls is the classification cross-entropy loss of the auxiliary classifier, and α is a dynamic weight.

[0058] The classification loss L cls is specifically expressed as follows:

[0059]

[0060] Wherein, C represents the number of categories, y​c an encoding representing the c-th class label, p c an encoding representing the class probability predicted by the model.

[0061] Further, the α balance between reconstruction and classification tasks is dynamically adjusted during joint training, preferably α = 0.1, as shown in Figure 3 An Adam optimizer is used, preferably with a learning rate of 0.005 and a batch size of 32.

[0062] The model of the present application has a bidirectional knowledge integration architecture, which can simultaneously achieve abnormal positioning and accurate classification, and achieves an image-level AUROC of 99.5% and a classification accuracy of 100% on the MVTec-AD dataset; and is suitable for use in the fields of industrial product quality inspection, equipment fault diagnosis, medical image auxiliary analysis, network security monitoring, etc.

[0063] The following are specific embodiments:

[0064] Taking the "metal nut" class of the MVTec AD dataset as an example: the input is a 256x256 RGB image; the output is a pixel-level abnormal heat map and a class label, and the abnormal heat map is as shown in Figure 4

[0065] The PRO score of the results obtained in this embodiment is 98.6%, which is 3.61% higher than the baseline RD++, indicating that the performance of the model of the present application in positioning abnormal areas is better.

[0066] Those skilled in the art will readily understand that the above description is only a preferred embodiment of the present application and is not intended to limit the present application, and any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.​

Claims

1. A method for constructing an anomaly detection model for reverse distillation industry, characterized in that, Includes the following steps: The anomaly detection model is trained using the training set to obtain a well-trained anomaly detection model. The anomaly detection model includes a teacher encoder, a student decoder, and an auxiliary classifier. The teacher encoder is pre-trained on normal images, and the student decoder adopts a lightweight teacher encoder architecture. The training set includes abnormal images and corresponding abnormal labels and category labels; during training: The teacher encoder extracts features from the abnormal image to obtain feature map A; The student decoder reconstructs feature map B by performing reverse knowledge distillation through inverse feature propagation based on feature map A; based on the difference between feature maps A and B, it determines the abnormal location on the abnormal image, i.e., the abnormal label. The auxiliary classifier obtains class labels based on the intermediate features generated by each convolutional layer in the student decoder. In the auxiliary classifier, weights are dynamically assigned to the intermediate features generated by each convolutional layer in the student decoder, and a spatial attention mechanism is used to enhance its sensitivity to image features. The output of the auxiliary classifier is as follows: Where y represents the output of the auxiliary classifier, Classifier() represents the classifier, Downsample() represents downsampling, and o0, o1, and o2 represent the features of the three adjacent layers in the middle of the student decoder, respectively. The loss function L used when training the anomaly detection model is... total for: L total =L pix +αL cls Among them, L pix For the feature reconstruction loss of the student decoder, L cls The classification loss of the auxiliary classifier is α, where α is the dynamic weight.

2. The method for constructing the reverse distillation industrial anomaly detection model as described in claim 1, characterized in that, Feature Reconstruction Loss L pix Cosine similarity is used to measure the difference between feature maps A and B.

3. The method for constructing the anomaly detection model for the reverse distillation industry as described in claim 1, characterized in that, Classification loss L cls The cross-entropy loss function is used.

4. The method for constructing the anomaly detection model for the reverse distillation industry as described in claim 1, characterized in that, The student decoder uses skip connections to pass intermediate features generated by the teacher encoder's convolutional layer to the corresponding convolutional layer of the student decoder, thus achieving cross-layer feature fusion.

5. The method for constructing the anomaly detection model for the reverse distillation industry as described in claim 1, characterized in that, The teacher encoder is built on ResNet-50.

6. The method for constructing the anomaly detection model for the reverse distillation industry as described in claim 1, characterized in that, The lightweighting process involves adjusting the feature map channel dimension in the teacher encoder using 1×1 convolution and unifying the feature distribution between the teacher encoder and the student decoder using feature normalization.

7. An anomaly detection model, characterized in that, It was constructed using the method described in any one of claims 1-6 for constructing an anomaly detection model for the reverse distillation industry.

8. An anomaly detection method, characterized in that, The process includes the following steps: inputting the image acquired by industrial inspection into the anomaly detection model as described in claim 7 to obtain anomaly labels and category labels.

Images