Knowledge distillation anomaly detection method based on segmentation guidance

Abstract

The invention discloses a knowledge distillation anomaly detection method based on segmentation guidance, and relates to the technical field of anomaly detection, and particularly relates to a pre-trained teacher network, a noise-reduced student network and a segmentation network. Taking the extracted feature image data as input data of a student network; performing denoising processing on the input data by using a student network, and learning and integrating wide features from different scales; and inputting fusion features from the teacher network and the de-noised student network into a segmentation network, and generating discrimination features of an abnormal region by the segmentation network by using a student-teacher framework to realize accurate anomaly detection. According to the method, a teacher-student framework and a segmentation network are integrated into one model, training can only be carried out from normal images, a multi-scale feature fusion strategy is adopted, a de-noising student network is enhanced, higher-level detail reservation and noise suppression are achieved, and accurate detection of anomalies is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of anomaly detection, and in particular to a segmentation-guided knowledge distillation anomaly detection method. Background Art

[0002] Anomaly detection is an important task in computer vision and has a wide range of applications in product quality inspection, medical disease detection, video surveillance and other fields. At present, in product quality inspection, it is a challenge to generate a large number of abnormal samples in a short period of time due to the completeness of the production process and the high yield rate. At the same time, the labeling cost of samples is high, incomplete or excessive labeling, and differences in labeling of different individuals may affect the detection performance. Therefore, it is crucial to use normal samples for anomaly detection. Industrial image anomaly detection processing methods based on unsupervised learning can be divided into two categories: reconstruction-based and feature embedding-based.

[0003] GAN and VAE are the most popular reconstruction-based anomaly detection methods. A variety of GAN-based and VAE-based anomaly detection methods have been proposed, which can detect abnormal regions at the pixel level by comparing the difference between normal images and reconstructed images. During the inference process, normal samples can still be reconstructed effectively, but the reconstruction effect of abnormal samples is poor due to distribution differences. Therefore, anomaly scores can be obtained through pixel-level comparison to determine whether the sample is abnormal. However, if the normal image cannot be reconstructed well by the network, it is difficult to detect abnormal regions. In recent years, feature embedding-based methods have gained significant attention due to their effectiveness in capturing high-dimensional visual features and converting them into compact and meaningful representations. These embeddings facilitate the discovery of outliers in image datasets by exploiting the underlying structure and patterns in visual data. The main difference from reconstruction methods is that it performs anomaly detection in high-dimensional feature space instead of RGB image space. These methods have achieved remarkable results in unsupervised anomaly detection, but still cannot accurately segment abnormal regions.

[0004] In order to solve the above problems and inspired by DeSTSeg, an improved method called a segmentation-guided knowledge distillation anomaly detection method (KDSeg) is proposed, which can achieve accurate segmentation and detection of abnormal areas. Summary of the invention

[0005] In view of the deficiency that the above-mentioned prior art cannot accurately segment abnormal areas, the present invention provides a segmentation-guided knowledge distillation anomaly detection method (KDSeg). During the segmentation process, this method makes full use of the differences in feature extraction between the teacher network and the student network to achieve accurate detection of anomalies.

[0006] The present invention provides the following technical solution: a knowledge distillation anomaly detection method based on segmentation guidance, comprising a pre-trained teacher network, a denoised student network and a segmentation network; wherein: A, pre-trained teacher network: the teacher network is ResNet18 pre-trained with ImageNet, and the last block is deleted, and the output feature map of the teacher network is three blocks, respectively marked as T F1 、T F2 and T F3 , and with different resolutions;

[0007] B. Denoising Student Network: The core principle of the denoising student network is that it can not only effectively denoise the input data, but also utilize different feature information across multiple scales, and can learn and integrate a wide range of features from different scales, ensuring a comprehensive understanding of local details and global context in the data;

[0008] The denoising student network has an independent encoder-decoder architecture. The encoder is a randomly initialized ResNet18. All blocks are represented as and The decoder is a reversed ResNet18 with four residual blocks, denoted as and It replaces all downsampling with bilinear upsampling, and enhances the decoder process by taking the encoder's last layer features as the only input to the decoder and using the encoder's features at all scales to increase the decoder's information richness at all levels;

[0009] C. Segmentation network: The segmentation network consists of two residual blocks and a spatial pyramid pooling module without upsampling or downsampling. The segmentation network adaptively fuses ST network features to detect abnormal areas;

[0010] The segmentation-guided knowledge distillation anomaly detection method comprises the following steps:

[0011] Step 1: Use the pre-trained teacher network to extract feature image data from the original image, and use the extracted feature image data as input data for the student network;

[0012] Step 2: Use the student network to denoise the input data and learn and integrate a wide range of features from different scales;

[0013] Step 3: The fused features from the teacher network and the denoising student network are input into the segmentation network. The segmentation network uses the student-teacher framework to generate discriminative features of abnormal areas to achieve accurate anomaly detection.

[0014] Preferably, except for the initial layer, each subsequent decoder layer receives its input by combining two sources: the upsampled features from the previous decoder layer and the corresponding scaled features extracted directly from the encoder, using formulas as shown in (1) to (4):

[0015]

[0016] Preferably, in the training phase of the student network, the weights of the teacher network are fixed, and the weights of the student network are trainable. Paired abnormal and normal images are fed to the denoised student network and the teacher network, respectively, to encourage the student network to generate abnormal features different from the teacher. The loss is the sum of the distances at three different feature levels. Fi and The losses used are shown in equations (5) to (7):

[0017]

[0018] Preferably, in the training phase of the segmentation network, the weights of the teacher network and the denoising student network are frozen, the synthesized abnormal image is used as the input of the two ST networks, the corresponding binary abnormal mask is the ground truth, and the segmentation training is optimized using the focal loss and L1 loss. In the testing phase, the test image is fed into the teacher and student networks at the same time, and finally the abnormal score map is predicted by the segmentation network, and the segmentation training is optimized using the focal loss and L1 loss shown in Eq. (8) to Eq. (10):

[0019]

[0020] L seg =L focal +L l1 (10).

[0021] Preferably, during the training phase, Perlin noise is used to generate simulated abnormal images based on normal images to enhance the learning ability of the student network; the student network is trained to reconstruct normal image features by matching normal image features with features extracted by the teacher network, and at the same time, the student network learns from the normal image features extracted by the teacher network.

[0022] Preferably, the method integrates the student-teacher network and the segmentation network into one model and can be trained only from normal images.

[0023] Preferably, when training the segmentation network, the difference between the features extracted by the teacher network and the features extracted by the enhanced denoising student network is fully considered, and the fusion features of the student-teacher network are input into the segmentation network for optimization, so as to facilitate the learning of the segmentation network features.

[0024] Compared with the prior art, the present invention has the following beneficial effects:

[0025] 1. This segmentation-guided knowledge distillation anomaly detection method integrates the teacher-student framework and the segmentation network into one model, and can only be trained from normal images. This method makes full use of multi-scale information, improves the ability of the student network to reconstruct normal images, and increases the ability of the student network to learn normal features from the teacher. During the segmentation process, this method makes full use of the differences in feature extraction between the teacher network and the student network to achieve accurate detection of anomalies.

[0026] 2. The proposed model based on segmentation-guided knowledge distillation anomaly detection method shows strong robustness and effectiveness in anomaly detection and localization, achieves excellent performance in multiple categories, and has great potential in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a model network structure diagram of the anomaly detection method in the present invention.

[0028] Figure 2 is the Enhanced Denoising Student Network (ADSN);

[0029] Figure 3 is an abnormality detection result diagram of an embodiment of the present invention;

[0030] Figure 4 Table of pixel-level anomaly localization AUC / AP (%) for the MVTec AD dataset;

[0031] Figure 5 Table 2 shows the image-level anomaly detection AUC (%) on the MVTec AD dataset, with the results for all categories averaged. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] The present invention provides a segmentation-guided knowledge distillation anomaly detection method, comprising the following steps:

[0034] Step 1: Use the pre-trained teacher network to extract feature image data from the original image, and use the extracted feature image data as input data for the student network;

[0035] Step 2: Use the student network to denoise the input data and learn and integrate a wide range of features from different scales;

[0036] Step 3: The fused features from the teacher network and the denoising student network are input into the segmentation network. The segmentation network uses the student-teacher network (ST network) framework to generate discriminative features of abnormal areas to achieve accurate anomaly detection.

[0037] In order to further describe the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0038] Figure 1 It is a model network structure diagram of the anomaly detection method in the present invention. The anomaly detection method provided by the present invention is inspired by DeSTSeg and is called KDSeg. KDSeg uses three networks for anomaly detection: a pre-trained teacher network, a denoising student network, and a segmentation network.

[0039] Among them: A. Pre-trained teacher network: The teacher network is ResNet18 pre-trained with ImageNet, and the last block is deleted. The output feature map of the teacher network is three blocks, marked as T F1 、T F2 and T F3 , and have different resolutions.

[0040] B. Denoising Student Network: The core principle of the denoising student network is that it can not only effectively denoise the input data, but also utilize different feature information across multiple scales, and can learn and integrate a wide range of features from different scales, ensuring a comprehensive understanding of local details and global context in the data;

[0041] The denoising student network has an independent encoder-decoder architecture. The encoder is a randomly initialized ResNet18. All blocks are represented as and The decoder is a reversed ResNet18 with four residual blocks, denoted as and It replaces all downsampling with bilinear upsampling and enhances the decoder process by taking the encoder's last layer features as the only input to the decoder and utilizing the encoder's features at all scales to increase the decoder's information richness at all levels.

[0042] This enhanced decoder process functions as follows: instead of relying solely on the highest level abstract features, each layer of the decoder now benefits from customized feature fusion. Specifically, except for the initial layer, each subsequent decoder layer receives its input by combining two sources: the upsampled features from the previous decoder layer and the corresponding scaled features extracted directly from the encoder, using formulas as shown in (1) to (4):

[0043]

[0044]

[0045] C. Segmentation network: The segmentation network consists of two residual blocks and a spatial pyramid pooling module without upsampling or downsampling. The segmentation network adaptively fuses the ST network features and uses a student-teacher network framework to generate discriminative features of abnormal areas to achieve accurate anomaly detection.

[0046] The method integrates the teacher-student framework and the segmentation network into one model, and can only be trained from normal images. The multi-scale feature fusion strategy is adopted, and the enhanced denoising student network achieves a higher level of detail retention and noise suppression. In addition, in the training stage, the model uses Perlin noise to generate simulated abnormal images based on normal images to enhance the learning ability of the student network. By matching the normal image features with the features extracted by the teacher network, the training network reconstructs the normal image features. At the same time, the student network learns from the normal image features extracted by the teacher network. When training the segmentation network, the difference between the features extracted by the teacher network and the enhanced denoising student network is fully considered. The fused features of these networks are input into the segmentation network for optimization, which facilitates the learning of the segmentation network features. In the inference stage, the fused features from the teacher network and the denoising student network are input into the segmentation network to achieve accurate anomaly detection. In one embodiment of the present application, the student network learning rate is set to 0.001, the batch is set to 32, the training rounds of the student network are set to 2000, and the training rounds of the segmentation network are set to 8000.

[0047] In the training phase of the student network, the weights of the teacher network are fixed, and the weights of the student network are trainable. The paired abnormal and normal images are fed into the denoising student network and the teacher network respectively, prompting the student network to generate abnormal features different from the teacher. The loss is the sum of the distances at three different feature levels. Fi and The losses used are shown in equations (5) to (7):

[0048]

[0049]

[0050] During the training phase of the segmentation network, the weights of the teacher network and the denoising student network are frozen. The synthesized anomaly image is used as the input of the two ST networks, and the corresponding binary anomaly mask is the ground truth. The segmentation training is optimized using the focal loss and L1 loss. During the testing phase, the test image is fed into the teacher and student networks at the same time, and finally the anomaly score map is predicted by the segmentation network. The segmentation training is optimized using the focal loss and L1 loss shown in Eq. (8) to Eq. (10):

[0051]

[0052] L seg =L focal +L l1 (10).

[0053] Figure 2 This novel architecture is the Augmented Denoising Student Network (ADSN), which is carefully designed to exploit different feature information across multiple scales, which is the key factor in its superior performance in identifying deviations from expected patterns.

[0054] Example

[0055] Evaluation of the effectiveness of anomaly detection methods based on segmentation-guided knowledge distillation.

[0056] The MVtec anomaly dataset is used for evaluation. The dataset includes 15 categories, each of which includes normal samples used only for training and normal and abnormal samples with various types of defects for testing. It also provides pixel-level annotations for defective test images and uses the same accuracy metrics as pixel-level and image-level evaluations.

[0057] The results of KDSeg are compared with STPM, Cut and Paste, DRAEM, PatchCore and DeSTSeg. The pixel-level AUC and image-level AUC results of different methods are shown in Figure 4 and Figure 5 As shown in the figure, the accuracy indicators of other algorithms are from the literature. In addition, in order to further analyze the advantages and disadvantages of the algorithm proposed in this paper, Figure 3 Several visualization examples of our approach under different categories are given.

[0058] Figure 4 The pixel-level anomaly detection AUC and AP values ​​are shown. Figure 5The image-level anomaly detection AUC for all categories is shown. The proposed method outperforms the state-of-the-art techniques in all categories at both pixel and image levels. In the pixel-level evaluation, the proposed method performs well in multiple categories, especially in "leather", "metal nuts", and "capsules", indicating its robustness in pixel-level anomaly detection. In the image-level evaluation, the proposed method achieves an average AUC of 99.1%, slightly outperforming other state-of-the-art methods, demonstrating its effectiveness in identifying anomalies at the image level, and the proposed method achieves 98.25% pixel-level accuracy. Figure 3 The anomaly detection results of the embodiment are shown, and the results provided clearly show that the method introduced in this paper has significant advantages over the destseg method in processing and identifying the segmentation of abnormal regions. Through careful comparative analysis, it is revealed that this new method not only shows higher accuracy in accurately distinguishing the boundaries between normal and abnormal regions, but also shows enhanced robustness and sensitivity when dealing with complex and fuzzy edges of abnormal regions. In addition, although Figure 4 It shows that the pixel-wise AUC metric for some categories, such as “bottle” and “cable”, is not the highest, but Figure 3 The results shown demonstrate that the proposed method effectively improves the accuracy of abnormal region boundary segmentation.

[0059] In summary, the proposed model shows strong robustness and effectiveness in anomaly detection and localization, achieving excellent performance in multiple categories. These results verify the effectiveness of the proposed improvements and demonstrate its great potential in practical applications.

[0060] Through the accuracy evaluation of the abnormal experimental results at the pixel level and image level, the method proposed in this paper is superior to STPM, CutPaste, DRAEM, DSR, Patchcore and DestSeg, and obtains a more accurate abnormal area. The average pixel-level AUC increases by 0.5%, and the image-level AUC increases by 0.5%. This method is superior in image-level, pixel-level and instance-level anomaly detection tasks.

[0061] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in the field. Although the embodiments of the present invention have been shown and described, it is understood by ordinary technicians in the field that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention. The scope of the present invention is defined by the attached claims and their equivalents.

Claims

1. A knowledge distillation anomaly detection method based on segmentation guidance, Features: It includes a pre-trained teacher network, a denoising student network and a segmentation network; A. Pre-trained teacher network: The teacher network is ResNet18 pre-trained with ImageNet, and the last block is deleted. The output feature map of the teacher network is three blocks, marked as T F1 , T F2 and T F3 , and with different resolutions; B. Denoising Student Network: The core principle of the denoising student network is that it can not only effectively denoise the input data, but also utilize different feature information across multiple scales, and can learn and integrate a wide range of features from different scales, ensuring a comprehensive understanding of local details and global context in the data; The denoising student network has an independent encoder-decoder architecture. The encoder is a randomly initialized ResNet18. All blocks are represented as and The decoder is a reversed ResNet18 with four residual blocks, denoted as and It replaces all downsampling with bilinear upsampling, and enhances the decoder process by taking the encoder's last layer features as the only input to the decoder and using the encoder's features at all scales to increase the decoder's information richness at all levels; C. Segmentation network: The segmentation network consists of two residual blocks and a spatial pyramid pooling module without upsampling or downsampling. The segmentation network adaptively fuses ST network features to detect abnormal areas; The segmentation-guided knowledge distillation anomaly detection method comprises the following steps: Step 1: Use the pre-trained teacher network to extract feature image data from the original image, and use the extracted feature image data as input data for the student network; Step 2: Use the student network to denoise the input data and learn and integrate a wide range of features from different scales; Step 3: The fused features from the teacher network and the denoising student network are input into the segmentation network. The segmentation network uses the student-teacher framework to generate discriminative features of abnormal areas to achieve accurate anomaly detection.

2. According to claim 1, a segmentation-guided knowledge distillation anomaly detection method, Features: Except for the initial layer, each subsequent decoder layer receives its input by combining two sources: the upsampled features from the previous decoder layer and the corresponding scaled features extracted directly from the encoder, using formulas shown in (1) to (4):

3. According to claim 1, a segmentation-guided knowledge distillation anomaly detection method, Features: In the training phase of the student network, the weights of the teacher network are fixed, and the weights of the student network are trainable. The paired abnormal and normal images are fed into the denoising student network and the teacher network respectively, prompting the student network to generate abnormal features different from the teacher. The loss is the sum of the distances at three different feature levels. Fi and The losses used are shown in equations (5) to (7):

4. The segmentation-guided knowledge distillation anomaly detection method according to claim 1, Features: During the training phase of the segmentation network, the weights of the teacher network and the denoising student network are frozen. The synthesized anomaly image is used as the input of the two ST networks, and the corresponding binary anomaly mask is the ground truth. The segmentation training is optimized using the focal loss and L1 loss. During the testing phase, the test image is fed into the teacher and student networks at the same time, and finally the anomaly score map is predicted by the segmentation network. The segmentation training is optimized using the focal loss and L1 loss shown in Eq. (8) to Eq. (10): L seg =L focal +L l1 (10) 5. According to claim 3, a segmentation-guided knowledge distillation anomaly detection method, Features: During the training phase, Perlin noise is used to generate simulated abnormal images based on normal images to enhance the learning ability of the student network. The student network is trained to reconstruct normal image features by matching normal image features with features extracted by the teacher network. At the same time, the student network learns from the normal image features extracted by the teacher network.

6. The segmentation-guided knowledge distillation anomaly detection method according to claim 1, Features: This method integrates the student-teacher network and the segmentation network into one model and can only be trained from normal images.

7. The segmentation-guided knowledge distillation anomaly detection method according to claim 4, Features: When training the segmentation network, the differences between the features extracted by the teacher network and the features extracted by the enhanced denoising student network are fully considered, and the fused features of the student-teacher network are input into the segmentation network for optimization, which facilitates the learning of the segmentation network features.

Images