Pseudo-label Generation Ensemble Model Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing database systems face challenges in accurately inferring data labels from input features due to overfitting when trained with limited data, and the process of generating sufficient labeled data is time-consuming and resource-intensive.
Innovation Solution
The method involves training multiple models on a labeled dataset to generate pseudo-labels for unlabeled features, which are then combined to create a pseudo-labeled dataset, allowing a final model to be trained on both labeled and pseudo-labeled data, improving accuracy and reducing resource requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If a model is trained using limited training data, then the training time and resources are reduced, but the model becomes overfit to the training data and loses the ability to accurately infer labels from new input features
Solution Approach 1:
The system performs preliminary actions by training multiple diverse models on the limited labeled training data before deployment. These pre-trained models serve as an ensemble that can handle new input features more robustly, preventing overfitting while maintaining quick training on the original limited dataset. The ensemble of pre-trained models captures diverse patterns from the limited data, improving generalization without requiring additional training time.
2Reliability
If sufficient volume of training data is obtained to avoid overfitting, then the model's ability to accurately infer labels improves, but the cost and time required for data collection and labeling increase significantly
Solution Approach 1:
The system enables self-service by allowing the trained models to generate their own training data. The models process unlabeled input features and generate synthetic labeled data pairs, which are then added to the training dataset. This self-generated training data expands the volume of training data without requiring external data collection or manual labeling, thereby improving model accuracy while avoiding the time and resource costs of traditional data labeling processes.
3Quantity of substance
If features are used as training data without corresponding labels, then the volume of available training data increases, but the model cannot learn the relationship between features and target labels
Solution Approach 1:
The system introduces trained models as intermediaries between unlabeled features and label information. These models process the unlabeled features and generate predicted labels, creating synthetic feature-label pairs. The models act as mediators that transform unlabeled data into labeled training data, enabling the system to utilize the full volume of available features while recovering the necessary label information through the models' predictive capabilities.
Data Source
AI summary
Systems and methods include training of each of a plurality of models based on a first set of training data comprising a first plurality of pairs, each of the first plurality of pairs comprising a feature and a corresponding label, inputting of each of a plurality of features into each of the plurality of trained models to generate, for each feature of the plurality of features, a plurality of output labels, determining, for each of the plurality of features, a pseudo-label based on the plurality of output labels generated for the feature, determining a second set of training data comprising a second plurality of pairs, each of the second plurality of pairs comprising one of the plurality of features and a pseudo-label determined for the one of the plurality of features, and training an inference model to output an inferred label based on the first set of training data and the second set of training data.