Model Training Using Labeled and Unlabeled Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The high labor and cost associated with data labeling in training computer models, particularly in scenarios where labeled data is limited, hinder the efficiency and speed of model training in fields like computer vision and deep learning.
Innovation Solution
A method that utilizes both labeled and unlabeled data to optimize the target model by calculating loss values from both datasets, allowing the model to be trained effectively even with a limited amount of labeled data, and incorporating noise information to enhance data robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If only labeled data is used for training the model, then the model can learn accurately from verified data, but the training process becomes labor-intensive and costly
Solution Approach 1:
The patent combines labeled data and unlabeled data into a unified training framework. The model processes both types of data through the same network architecture, using labeled data for supervised learning and unlabeled data for self-supervised learning, thereby merging the benefits of both approaches to improve training efficiency while maintaining accuracy
Solution Approach 2:
The patent implements self-supervised learning where the model generates its own labels from unlabeled data by predicting relationships between data samples. This allows the model to learn from unlabeled data autonomously without requiring manual annotation, significantly reducing labor costs and improving training productivity
2Reliability
If more data is collected for training, then the model can improve its performance, but the cost and time required for data labeling increases
Solution Approach 1:
The patent creates pseudo-labels by copying and transforming unlabeled data into a format that can be used for training. The model generates predicted labels from unlabeled data samples, which are then used to create training pairs, effectively copying the labeling function to automate the data preparation process
Solution Approach 2:
The patent performs preliminary processing on unlabeled data by pre-computing features, embeddings, and potential labels before the main training process. This preliminary action reduces the computational burden during training and minimizes the time required for data preparation and labeling
3Productivity
If the model is trained with limited labeled data, then the training cost decreases, but the model's generalization ability may be compromised
Solution Approach 1:
The patent transitions from two-dimensional supervised learning (input-output pairs) to three-dimensional learning by incorporating unlabeled data as a third dimension. The model learns representations from unlabeled data and uses these to enhance its understanding of the data distribution, improving generalization ability without requiring additional labeled examples
Solution Approach 2:
The patent creates a composite training approach that combines supervised learning from labeled data and self-supervised learning from unlabeled data. This composite learning methodology leverages the strengths of both approaches, using labeled data for precise pattern recognition and unlabeled data for robust representation learning, thereby improving overall model generalization
Data Source
AI summary
A method for generating a model includes: obtaining training data for training a target model, in which the training data includes labeled data and unlabeled data; obtaining a first output result by the target model based on the labeled data; obtaining a second output result by the target model based on the unlabeled data; and obtaining an optimized target model by optimizing the target model based on the first output result and the second output result.