Long-Tailed Image Anomaly Detection with Learnable Class Embeddings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection models struggle with scalability and generalization across multiple object classes, particularly in long-tailed distributions, leading to overfitting on popular classes and ignoring less popular ones, and rely on ambiguous or unknown class names, resulting in inaccurate detection.
Innovation Solution
A combined reconstruction and semantic technique using a transformer-based reconstruction model and a binary classifier with a pretrained visual-language foundation model to learn pseudo class names, enabling anomaly detection across multiple classes without relying on dataset class names.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional anomaly detection models are trained on specialized datasets with real-world normal images, then detection accuracy improves, but the model requires different models per image class which limits scalability
Solution Approach 1:
The patent develops a universal anomaly detection model that can handle multiple image classes simultaneously without requiring separate models for each class. The model uses a class-agnostic approach with learnable class embeddings that enable it to generalize across different object categories, thus achieving multi-functionality while maintaining detection accuracy.
Solution Approach 2:
The patent introduces learnable class embeddings as adjustable parameters that allow the model to adapt to different image classes dynamically. By changing the embedding parameters rather than the model structure, the system achieves scalability across multiple classes without retraining entire models for each class.
2Adaptability or versatility
If anomaly detection models use reconstruction techniques to project images into a manifold of normal images, then generalization across object classes improves, but detection accuracy decreases for certain regions with anomalies
Solution Approach 1:
The patent introduces a projector operator as an intermediary component that bridges the reconstruction model and the anomaly detection process. This projector maps features into a space where anomalies can be more effectively detected, mediating between the generalization capability of reconstruction and the precision requirements of anomaly detection.
Solution Approach 2:
The patent combines multiple techniques (reconstruction, semantic understanding, and projector-based feature mapping) into a composite anomaly detection system. This composite approach leverages the strengths of each component while mitigating their individual weaknesses, achieving both generalization and accuracy.
3Ease of operation
If anomaly detection models rely on predefined text prompts with class names, then semantic understanding improves, but detection accuracy decreases when class names are ambiguous or unknown
Solution Approach 1:
The patent enables the model to learn class embeddings autonomously from the data without relying on predefined class names or external knowledge. The system performs self-service by automatically adapting to the semantic structure of each dataset, eliminating the need for manual class name specification while maintaining semantic understanding.
Solution Approach 2:
Instead of using predefined class names to guide detection, the patent inverts the approach by learning class representations from the data itself. The model discovers semantic structures bottom-up rather than imposing top-down class labels, which resolves the issue of ambiguous or unknown class names.
4Productivity
If anomaly detection models are trained on long-tailed datasets with imbalanced class distributions, then productivity increases by handling multiple classes, but detection accuracy decreases for less popular classes due to overfitting on popular classes
Solution Approach 1:
The patent applies data augmentation and sampling strategies that give appropriate attention to minority classes despite their limited representation. By performing partial actions (selective sampling, augmentation) on underrepresented classes, the model compensates for the imbalance and prevents overfitting to majority classes while maintaining productivity across all classes.
Data Source
AI summary
Embodiments of the present disclosure provide a method for anomaly detection in a patch of an image. The method comprises collecting a first text encoding of a first text prompt in a latent space, collecting a second text encoding of a second text prompt in the latent space, encoding the image to produce features of the image, partitioning the features of the image into feature patches, projecting each of the feature patches into the latent space using a projector operator, and comparing the projection of each of the feature patches with the first text encoding and the second text encoding to detect the anomaly when the projection of a feature patch from the feature patches is closer to the second text encoding than to the first text encoding.