Machine Learning Parameter Optimization via Performance Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning methods face challenges in efficiently determining optimal parameter values for hyperparameters, especially when training data size varies, leading to increased learning time and suboptimal prediction performance.
Innovation Solution
A computer-readable medium and apparatus that calculates expected values and variance degrees of prediction performance for different parameter values and training data sizes, allowing for the estimation of prediction performance without actual machine learning, thereby efficiently searching for optimal parameter values.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If progressive sampling method is used to train model with small-size training data first, then learning time is reduced, but prediction performance may be insufficient
Solution Approach 1:
The system performs preliminary machine learning with small-size training data to obtain initial parameter values before final model training. This preliminary action allows early evaluation of parameter performance without committing to large-scale training, enabling efficient parameter selection that balances learning time reduction with prediction performance maintenance
Solution Approach 2:
The system creates a simplified copy of the machine learning process using small-size training data to evaluate parameter values. This copy model allows rapid experimentation with different parameters to identify promising candidates, which are then validated in the final training phase with larger data, avoiding the need to test all parameter combinations exhaustively
2Measurement precision
If multiple parameter values are tested to find optimal parameters, then prediction performance improves, but learning time increases
Solution Approach 1:
The parameter optimization process is segmented into two phases: initial parameter value acquisition using small-size training data, and final parameter validation using larger training data. This segmentation allows rapid screening of multiple parameter values in the first phase without incurring the full time cost of comprehensive training for each parameter, thereby evaluating many parameters while controlling overall learning time
Solution Approach 2:
The system performs partial machine learning with small-size training data to obtain parameter values, rather than completing full training for each parameter evaluation. This partial action provides sufficient information to compare parameter performance and select candidates, avoiding the excessive time investment of complete training cycles for parameter search while still achieving effective parameter optimization
3Measurement precision
If training data size is increased to improve prediction performance, then model accuracy improves, but learning time increases
Solution Approach 1:
The system performs preliminary machine learning with small-size training data to identify promising parameter values before committing to large-scale training. This preliminary action with limited data provides early performance indicators that guide subsequent training decisions, ensuring that full training resources are allocated only to parameter configurations showing potential, thereby achieving high prediction performance while minimizing unnecessary learning time
Solution Approach 2:
The system changes the training data size parameter dynamically: using small-size training data for initial parameter value acquisition and evaluation, then transitioning to larger training data for final model training only with selected parameter values. This parameter change strategy allows the system to leverage the computational efficiency of small data for parameter search while utilizing large data's predictive power only when necessary, optimizing the trade-off between prediction performance and learning time
Data Source
AI summary
A processor obtains a first measurement value representing prediction performance of a model that has been learned by using a first parameter value and training data of first size. The processor calculates a first expected value and a first variance degree of prediction performance of a model that would be learned by using the first parameter value and training data of second size. The processor also obtains a second measurement value representing prediction performance of a model that has been learned by using a second parameter value and training data of the first size. The processor calculates a second expected value and a second variance degree of prediction performance of a model that would be learned by using the second parameter value and training data of the second size.