Teacher-Student Video Representation Learning Reducing False Negatives
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing contrastive learning methods for unsupervised learning from unlabeled videos often select semantically-similar samples as negatives, leading to false negative results and requiring a large number of negative samples.
Innovation Solution
A teacher/student training framework is introduced, where two teacher models operate in parallel, with one model using weakly-augmented samples to create an artificial label, which is then used by the other model to mine potential positives from negative samples, reducing false negative results.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If contrastive learning selects semantically-similar samples as negatives, then the model can learn from available data, but false negative results increase and representation quality deteriorates
Solution Approach 1:
The teacher model performs preliminary prediction on negative samples before the student model processes them. By pre-computing prediction probabilities for negative samples using the teacher model, the system can identify and filter out potential positives beforehand, preventing false negatives from propagating through the training process
Solution Approach 2:
The teacher model acts as an intermediary between the data distribution and the student model. It processes negative samples first and provides prediction probabilities that guide the student model in distinguishing true negatives from potential positives, thereby mediating the contrastive learning process to reduce false negatives
2Quantity of substance
If a large number of negative samples are used, then coverage of data distribution improves, but computational complexity and memory requirements increase
Solution Approach 1:
The teacher model performs preliminary filtering and prediction on negative samples before they reach the student model. This pre-processing step allows the system to work with a manageable subset of informative negative samples while maintaining good data distribution coverage, reducing the computational burden on the student model
Solution Approach 2:
The teacher model serves itself by processing negative samples and generating prediction probabilities that are then used to guide the student model. This self-service mechanism allows the system to efficiently utilize negative samples without requiring excessive computational resources from the student model
Data Source
AI summary
Embodiments described herein provide systems and methods for learning representation from unlabeled videos. Specifically, a method may comprise generating a set of strongly-augmented samples and a set of weakly-augmented samples from the unlabeled video samples; generating a set of predictive logits by inputting the set of strongly-augmented samples into a student model and a first teacher model; generating a set of artificial labels by inputting the set of weakly-augmented samples to a second teacher model that operates in parallel to the first teacher model, wherein the second teacher model shares one or more model parameters with the first teacher model; computing a loss objective based on the set of predictive logits and the set of artificial labels; updating student model parameters based on the loss objective via backpropagation; and updating the shared parameters for the first teacher model and the second teacher model based on the updated student model parameters.