Multiple Pre-Trained Models for Graph-Based Pseudo-Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The availability of labeled training data in sufficient quantity and diversity is a significant limiting factor for training new deep learning models, particularly in computer vision tasks, where image acquisition and manual labeling are costly and time-consuming.
Innovation Solution
Utilizing multiple pre-trained models to analyze unlabeled data and apply graph-based measures to organize it into clusters with high probability of belonging to the same class, reducing the need for extensive human labeling by providing labels for each identified cluster.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If multiple pre-trained models are used to analyze unlabeled data and apply graph-based clustering, then the quantity of manually labeled samples needed is reduced, but the device complexity increases
Solution Approach 1:
The patent segments the labeling task by dividing unlabeled data into multiple clusters using graph-based clustering algorithms. Each cluster represents a group of data points with similar characteristics, allowing selective labeling of only representative samples from each cluster rather than labeling all data points individually. This segmentation approach significantly reduces the total number of manual labels required while maintaining data diversity.
Solution Approach 2:
The patent applies preliminary action by using multiple pre-trained models to analyze and organize unlabeled data into clusters before the actual labeling process. This preliminary organization identifies representative samples that best characterize each cluster, so that manual labeling efforts are focused only on these key representatives rather than the entire dataset. The pre-trained models perform feature extraction and initial classification to guide subsequent labeling decisions.
2Reliability
If manual labeling is performed on all data samples, then the reliability of training data is maximized, but the time and cost required increases significantly
Solution Approach 1:
The patent implements self-service by enabling the system to automatically identify and select representative samples that best represent each cluster structure. The graph-based clustering algorithm autonomously determines which unlabeled data points are most informative for characterizing each cluster, eliminating the need for manual review of every sample. This self-service mechanism maintains reliability by ensuring that selected representatives are objectively determined based on their positional importance within the cluster structure.
Solution Approach 2:
The patent replaces the mechanical manual labeling process with an automated computational system. Instead of manually examining and labeling all data samples, the system uses graph-based clustering algorithms and pre-trained models to automatically identify representative samples. This substitution maintains data reliability through algorithmic selection while dramatically reducing the time and human effort required, as the computational system efficiently processes and analyzes the data structure.
3Productivity
If a single pre-trained model is used for pseudo-labeling, then the process is simpler and faster, but the accuracy of pseudo-labeling decreases
Solution Approach 1:
The patent merges the outputs of multiple pre-trained models to generate pseudo-labels for unlabeled data. Instead of relying on a single model's predictions, the system combines the results from multiple models, allowing them to complement each other's strengths and compensate for individual weaknesses. This merging approach improves the accuracy of pseudo-labeling by achieving more reliable consensus predictions, while the graph-based clustering framework efficiently organizes this multi-model output.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method of labeling a dataset of input samples for a machine learning task includes selecting a plurality of pre-trained machine learning models that are related to a machine learning task. The method further includes processing a plurality of input data samples through each of the pre-trained models to generate a set of embeddings. The method further includes generating a plurality of clusterings from the set of embeddings. The method further includes analyzing, by a processing device, the plurality of clusterings to extract superclusters. The method further includes assigning pseudo-labels to the input samples based on analysis.