Hyperparameter Predictors for Automated ML Algorithm Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Selecting the best machine learning algorithm for a specific task is challenging due to the numerous algorithms available, with existing automatic approaches incurring significant computational overhead and often failing to consider algorithm hyperparameters, which significantly affect performance.
Innovation Solution
A two-level approach using trained mini-models and hyperparameter predictors to optimize machine learning algorithm selection, where mini-models with predicted hyperparameters improve the accuracy of scores, allowing for more efficient and accurate ranking and selection of algorithms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If training and testing each machine learning algorithm to find the best performing one, then selection accuracy is improved, but computational overhead increases significantly
Solution Approach 1:
The patent segments the algorithm selection process into multiple independent components: (1) a scoring module that evaluates algorithms based on dataset characteristics without full training, (2) a filtering module that eliminates poor-performing algorithms, and (3) a final selection module that chooses from remaining candidates. This segmentation allows accurate selection while reducing computational overhead by avoiding complete training of all algorithms.
Solution Approach 2:
The patent performs preliminary actions by pre-computing dataset characteristics and algorithm metadata before the actual selection process. The scoring module uses pre-extracted features to evaluate algorithms quickly, and the filtering module pre-eliminates poor candidates before any significant training occurs. This preliminary processing reduces the computational burden of the final selection while maintaining accuracy.
2Device complexity
If using a single regressor for predicting algorithm performance, then device complexity is reduced, but prediction accuracy decreases due to algorithm interference
Solution Approach 1:
The patent replaces the single regressor approach with segmented, algorithm-specific predictors. Each algorithm has its own dedicated predictor model that evaluates only that specific algorithm's performance based on dataset characteristics. This segmentation eliminates interference between different algorithms while maintaining manageable complexity through modular, independent predictors.
Solution Approach 2:
The patent introduces an intermediary scoring module that acts as a mediator between dataset characteristics and algorithm performance prediction. This scoring module evaluates algorithms based on pre-computed features and metadata, providing accurate predictions without requiring direct interaction between multiple algorithm models, thus avoiding interference while maintaining system complexity at acceptable levels.
3Ease of operation
If not considering algorithm hyperparameters, then ease of operation is improved, but selection accuracy decreases due to significant impact of hyperparameters on performance
Solution Approach 1:
The patent performs preliminary action by pre-determining optimal hyperparameters for each algorithm based on dataset characteristics before the actual selection process. The scoring module incorporates hyperparameter recommendations into its evaluation, allowing the system to consider hyperparameter impacts without requiring manual tuning or complex hyperparameter optimization during the selection process itself.
Solution Approach 2:
The patent implements self-service by having the system automatically determine and apply appropriate hyperparameters for each algorithm based on the dataset characteristics and algorithm-specific metadata. The algorithm-specific predictors automatically select and apply optimal hyperparameter settings without human intervention, maintaining ease of operation while significantly improving selection accuracy through hyperparameter consideration.
Data Source
AI summary
Techniques are provided for selection of machine learning algorithms based on performance predictions by using hyperparameter predictors. In an embodiment, for each mini-machine learning model (MML model), a respective hyperparameter predictor set that predicts a respective set of hyperparameter settings for a data set is trained. Each MML model represents a respective reference machine learning model (RML model). Data set samples are generated from the data set. Meta-feature sets are generated, each meta-feature set describing a respective data set sample. A respective target set of hyperparameter settings are generated for said each MML model using a hypertuning algorithm. The meta-feature sets and the respective target set of hyperparameter settings are used to train the respective hyperparameter predictor set. Each hyperparameter predictor set is used during training and inference to improve the accuracy of automatically selecting a RML model per data set.