Contrastive Distillation for Fast Pixel-Level Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current anomaly detection methods in industrial manufacturing require significant manpower for manual defect annotation, lead to missed detections of unknown defects, and result in oversized models due to underfitting and data inefficiencies.
Innovation Solution
A three-level network architecture based on contrastive representation distillation uses a pre-trained expert network to learn normal sample features, constructs defective samples through data augmentation, and employs knowledge distillation to train a student network for fast and robust anomaly detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation of defects is used for anomaly detection, then detection accuracy can be improved, but manpower resources consumption increases significantly
Solution Approach 1:
The system uses self-supervised learning where the model automatically learns feature representations from normal samples without requiring manual annotation of defective samples. The contrastive representation distillation enables the student network to automatically capture anomaly patterns by comparing feature distributions between normal and synthetic defective samples, eliminating the need for manual defect annotation while maintaining detection accuracy
Solution Approach 2:
The patent employs knowledge distillation where a teacher network trained on normal samples transfers learned representations to a student network. The teacher network learns contrastive representations from normal samples and synthetic defective samples, then distills this knowledge to the student network, allowing the student network to detect anomalies without requiring manual annotation of all defective samples
2Measurement precision
If conventional anomaly detection methods are used, then known defects can be detected, but unknown defective samples may be missed
Solution Approach 1:
The patent changes the training approach by using contrastive representation distillation that learns general anomaly patterns rather than specific defect types. The student network is trained to distinguish normal from abnormal patterns through contrastive loss, enabling it to detect unknown defect types that were not explicitly annotated during training by capturing fundamental anomaly characteristics in the feature space
3Reliability
If manual annotation and traditional training methods are used, then model can be trained, but model size becomes oversized due to underfitting
Solution Approach 1:
The patent applies contrastive representation distillation that changes how the model learns from data. Instead of training on manually annotated defective samples which requires large datasets and complex models, the method uses contrastive loss to learn compact feature representations from normal samples and synthetic defects, resulting in smaller, more efficient models that avoid underfitting while maintaining reliability
Solution Approach 2:
Knowledge distillation copies the learned representations from a teacher network to a student network. The teacher network, trained on contrastive representations from normal and synthetic defective samples, provides distilled knowledge that enables the student network to achieve comparable or superior performance with fewer parameters, eliminating the need for oversized models
4Adaptability or versatility
If data augmentation is used to construct defective samples, then training data diversity improves, but computational time increases
Solution Approach 1:
The patent performs data augmentation and contrastive representation learning during the training phase of the teacher network. By pre-computing the contrastive representations and synthetic defective samples during training, the system avoids the need for time-consuming manual annotation and real-time augmentation during deployment, achieving fast inference while maintaining data diversity
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
The present application provides a method and system for anomaly detection. The method includes: acquiring a picture of an object to be detected; inputting the acquired picture to each of a trained teacher network and a student network distilled from the teacher network, to obtain a feature map output by the teacher network and a feature map output by the student network, where the teacher network is trained by constructing a defective sample to learn feature distribution of normal samples from a pre-trained expert network; and determining a greatest anomalous pixel in a difference map between the feature map output by the teacher network and the feature map output by the student network as an anomaly value of the acquired picture, to output an anomaly detection result.