Credibility Vectors for Semi-Supervised Label Propagation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Semi-supervised learning faces errors in label propagation due to variability in similarity metrics across transformations and iterative training, leading to incorrect label assignment and propagation, especially when samples are similar to multiple samples with different labels, and is prone to errors due to the sensitivity of cross-entropy loss functions.
Innovation Solution
The introduction of credibility vectors, which assign certainty values to class labels (−1, 0, or 1), normalized and clipped to [0,1], to address ambiguity and reduce error propagation, allowing for robust label propagation by averaging and subsampling these vectors across training iterations and batches, and using them in tandem with similarity metrics to learn representations and labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If similarity metrics are used for label propagation in semi-supervised learning, then labels can be assigned to unlabeled samples, but errors in similarity metrics lead to incorrect label assignment and propagation
Solution Approach 1:
The patent introduces credibility vectors with values in [-1, 1] to replace traditional similarity metrics for label propagation. These credibility vectors capture not only similarity but also uncertainty about label correctness, allowing the system to propagate labels more reliably while maintaining efficiency. The transformation of similarity measures into credibility vectors resolves the contradiction by adding reliability information without sacrificing productivity.
Solution Approach 2:
The patent implements iterative updates of credibility vectors based on training progress and label propagation results. This feedback mechanism allows the system to correct and refine label assignments over time, reducing error propagation while maintaining high label propagation efficiency through the iterative nature of the process.
2Ease of manufacture
If cross-entropy loss function is used for classification, then training can be performed, but the loss function is sensitive to label errors leading to incorrect classifications
Solution Approach 1:
The patent introduces credibility vectors as an intermediary between the training process and classification. These vectors mediate the relationship by providing uncertainty-weighted label information that reduces the sensitivity of cross-entropy loss to label errors. The credibility vectors act as a buffer that protects the classification system from propagating training errors, maintaining both training simplicity and classification reliability.
3Adaptability or versatility
If samples similar to multiple samples with different labels are processed, then comprehensive label propagation occurs, but ambiguity in class assignment leads to incorrect labels
Solution Approach 1:
The patent transforms binary similarity relationships into multi-value credibility vectors that explicitly represent uncertainty. When a sample is similar to multiple samples with different labels, the credibility vector captures this ambiguity through fractional values in [-1, 1], allowing the system to maintain comprehensive label propagation coverage while correctly identifying uncertain cases that should not be assigned labels.
Data Source
AI summary
A contrastive credibility propagation trainer (“trainer”) trains a representation neural network to learn credibility vectors for partially labeled data samples that represent certainty of samples belonging to each of a set of classes. The representation neural network is trained according to a loss function that accounts for both the credibility vectors and similarity of representations generated by the neural network itself. Using the credibility vectors as soft labels, the trainer trains a classifier neural network to generate labels for unlabeled samples in the partially labeled samples.