Contrastive Affinity Learning for Novel Image Category Discovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing contrastive learning methods struggle with false negatives in unlabeled data, leading to semantic representation degradation and class collision issues, particularly in generalized novel category discovery tasks where unlabeled data contain both known and novel classes.
Innovation Solution
The Prompt-based Contrastive Affinity Learning (PromptCAL) framework employs Discriminative Prompt Regularization (DPR) loss and Contrastive Affinity Learning (CAL) process to discover semantic clusters by generating reliable pseudo positives through affinity graphs, enhancing semantic discriminativeness and iteratively improving model representations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If contrastive learning is applied to semi-supervised learning, then performance on labeled data improves, but false negatives occur in unlabeled data leading to semantic representation degradation
Solution Approach 1:
The patent introduces an intermediary mechanism (affinity graph with pseudo-positive samples) between the contrastive learning objective and the unlabeled data. This intermediary filters out false negatives by selectively identifying true positive pairs through affinity computation, thereby preserving semantic representation while maintaining the benefits of contrastive learning.
Solution Approach 2:
The patent implements a feedback loop where affinity graphs are constructed from current model predictions, pseudo-positive samples are identified, and these are used to update the contrastive learning objective. This iterative feedback process continuously refines the semantic representation by correcting false negatives based on the latest model state.
2Measurement precision
If contrastive learning pushes apart different classes, then class discrimination improves, but class collision occurs when unlabeled data contains both known and novel classes
Solution Approach 1:
The patent segments the contrastive learning process into two distinct phases: (1) affinity graph construction that identifies true positive pairs without class collision, and (2) contrastive loss computation that pushes apart only verified different classes. This segmentation allows precise class discrimination while avoiding false collisions between known and novel classes.
Solution Approach 2:
The patent applies local quality by computing affinity scores locally for each data pair and selectively applying contrastive pressure only where appropriate. Instead of uniformly pushing all pairs apart, the method adapts the contrastive force based on local affinity measurements, thereby achieving precise class discrimination without causing class collision.
3Measurement precision
If supervised learning uses labeled data, then classification accuracy improves, but human resources and labeling costs increase significantly
Solution Approach 1:
The patent enables the system to serve itself by automatically constructing affinity graphs and identifying pseudo-positive samples from unlabeled data without human intervention. The model uses its own predictions and affinity computations to generate training signals, thereby achieving high classification accuracy while minimizing the need for manually labeled data.
Solution Approach 2:
The patent performs preliminary action by pre-computing affinity scores and identifying pseudo-positive samples before the main contrastive learning training. This preliminary processing of unlabeled data creates a ready-to-use training set that reduces the need for extensive human labeling while maintaining high classification accuracy.
Data Source
AI summary
A system and method of fine-grained image category discovery with few human annotations includes a camera and a trained machine learning model, which predicts a label for an object in a captured image and outputs the predicted label. The machine learning model is trained by contrastive affinity learning, including retrieving images having an object, a warm-up stage in which semi-supervised contrastive learning is performed based on projected features of a class token and an ensembled prompt, respectively. In a contrastive affinity learning stage, a student model and an exponentially moving averaged teacher model are forwarded with different augmented views of the retrieved images. Teacher embeddings are enqueued into a token-specific memory. A semi-supervised contrastive loss is computed on a current batch and a contrastive affinity learning loss for student embeddings and the teacher embeddings with pseudo-labels from a affinity graph dynamically generated by semi-supervised affinity generation.