Teacher Model Label Classification for Student Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for training neural networks require manual extraction of information from input data to generate training data, which is time-consuming and inefficient, and often include inaccurate information that can confuse the student model during training.
Innovation Solution
A method and apparatus that input data to a teacher model to acquire label values, classify them into intervals based on threshold probabilities, and generate training data for a student model, where the label values are refined to indicate whether the data matches a class, does not match, or is ambiguous, thereby reducing unnecessary confusion and improving training efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual extraction of information from input data is used to generate training data, then training data can be generated, but the process is time-consuming and inefficient
Solution Approach 1:
A teacher model is introduced as an intermediary between the input data and the student model. The teacher model automatically generates training data by processing input data and producing label values, eliminating the need for manual information extraction while providing structured training data for the student model
Solution Approach 2:
The manual mechanical process of information extraction is replaced with an automated neural network-based system. The teacher model uses machine learning algorithms to automatically extract and structure information from input data, substituting human manual work with computational processes
2Reliability
If manual extraction of information is used to generate training data, then training data can be created, but inaccurate information may be included that confuses the student model
Solution Approach 1:
The teacher model serves as a filtering intermediary that processes raw input data and outputs only high-quality, accurate label values. This intermediary layer ensures that the student model receives reliable training data without the inaccuracies that might arise from manual extraction
Solution Approach 2:
The system uses the teacher model's trained knowledge to generate training data for the student model, creating a feedback loop where accurate patterns learned by the teacher model are transferred to guide the student model's training, reducing confusion and improving reliability
3Quantity of substance
If all label values are used for training, then more training data is available, but ambiguous label values may reduce training effectiveness
Solution Approach 1:
Different label values are treated differently based on their quality and confidence levels. High-confidence label values are used for training, while ambiguous or low-confidence label values are filtered out or weighted differently, ensuring that only high-quality training signals contribute to model learning
Solution Approach 2:
The system changes the parameter of label value confidence by comparing against threshold probabilities. Label values are classified into different categories (positive, negative, ignore) based on their confidence levels, allowing selective use of training data that optimizes both quantity and quality
Data Source
AI summary
Disclosed is a training data generating apparatus and method to change output data inappropriate for training a student model to an ignore value such that the inappropriate output data is not used to train the student model, change output data appropriate for training the student model such that the student model outputs an improved result in comparison to output data of a teacher model, and change a label value in a form of probability to an identifier corresponding to intervals divided based on threshold label values using the teacher model based on input data.