Machine-Learning Model Generation with Minimum Data Requirements
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The generation of machine-learning models is computationally expensive and resource-intensive due to the need for large training datasets and multiple iterations of hyper-parameter tuning, often resulting in overfitted models that waste resources and fail to achieve target accuracy.
Innovation Solution
A system and method for determining minimum data requirements by generating sample models with different datasets, identifying minimum viable models that achieve a threshold accuracy, and estimating the number of samples needed, thereby reducing the amount of computing resources required for new model generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If large training datasets are used to train machine-learning models, then model accuracy is improved, but computational resource consumption increases
Solution Approach 1:
The system performs preliminary analysis of the training dataset to determine the minimum viable subset size before actual model training begins. This preliminary action identifies the optimal data quantity needed to achieve target accuracy, preventing wasteful consumption of computational resources on excessive data processing while ensuring sufficient data is used for accurate model training.
2Measurement precision
If multiple iterations of hyper-parameter tuning are performed, then model accuracy is improved, but training time increases
Solution Approach 1:
The system performs a partial evaluation of hyper-parameter tuning iterations by using the determined minimum viable training subset to assess model performance. Instead of exhaustively testing all possible hyper-parameter combinations on the full dataset, the system evaluates whether the minimum viable subset achieves target accuracy with reasonable tuning, thereby reducing training time while maintaining sufficient model accuracy.
3Productivity
If progressively less training data is used for secondary models, then resource efficiency is improved, but model accuracy deteriorates
Solution Approach 1:
The system implements a feedback mechanism where secondary models trained on progressively smaller data subsets are evaluated against the target accuracy threshold. When model accuracy falls below the threshold, the system feedbacks this information to adjust the minimum viable subset size upward. This feedback loop ensures that resource efficiency is maximized while maintaining the necessary model accuracy, as the system dynamically adjusts data quantity based on performance outcomes.
Data Source
AI summary
A system for determining data requirements to generate machine-learning models. The system may include one or more processors and one or more storage devices storing instructions. When executed, the instructions may configure the one or more processors to perform operations including: receiving a sample dataset, generating a plurality of data categories based on the sample dataset; generating a plurality of primary models of different model types using data from the corresponding one of the data categories as training data; generating a sequence of secondary models by training the corresponding one of the primary models with progressively less training data; identifying minimum viable models in the sequences of secondary models; determining a number of samples required for the minimum viable models; and generating entries in the database associating: model types; corresponding data categories; and corresponding numbers of samples in the training data used for the minimum viable models.