Automated Hyperparameter Selection via Secondary Model Ranking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning algorithms face inefficiencies in determining optimal hyperparameters, as manual approaches are inconsistent and unreliable, while brute force searches require excessive computing time due to exponential complexity and require tuning of their own hyperparameters.
Innovation Solution
An automated system generates metadata from datasets to select suitable machine learning models and identify hyperparameter influences, using a secondary model to rank and search for optimal hyperparameter values with variable granularity based on performance metrics, reducing the search space and computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual heuristics are used to discover optimal hyperparameters, then the approach is simple and requires minimal computing resources, but the results are inconsistent and unreliable across varying datasets and algorithms
Solution Approach 1:
The system enables self-service by allowing the machine learning system to automatically select its own hyperparameters through the secondary model that predicts optimal values based on dataset metadata, eliminating the need for manual intervention while maintaining consistency across different datasets and algorithms
2Measurement precision
If brute force search is used to find optimalhyperparameters, then all possible combinations can be exhaustively searched, but the computing time required increases exponentially with the number ofhyperparameters
Solution Approach 1:
The system performs preliminary action by using a secondary model to predict optimalhyperparameter values before actual model training begins. The secondary model processes dataset metadata and returns pre-computed hyperparameter recommendations, eliminating the need for exhaustive brute force search and significantly reducing computing time while maintaining optimization precision
Solution Approach 2:
The secondary model acts as an intermediary between the dataset metadata and the finalhyperparameter selection. It processes the metadata through a trained model that has learned the relationship between data characteristics and optimal hyperparameters, providing efficient predictions without requiring exhaustive search of all possible hyperparameter combinations
3Reliability
If brute force search is used, then optimalhyperparameters can be found, but the search algorithm requires its ownhyperparameters to be tuned, adding significant time requirements
Solution Approach 1:
The system extracts the hyperparameter selection function from the main training process by using a separate secondary model that independently predicts optimalhyperparameter values. This separation eliminates the need for the search algorithm to have its own hyperparameters to tune, reducing the overall complexity of the system while maintaining reliable search results
Data Source
AI summary
Methods and systems are provided to determine suitable hyperparameters for a machine learning model and/or feature engineering process. A suitable machine learning model and associated hyperparameters are determined by analyzing a dataset. Suitable hyperparameter values for compatible machine learning models having one or more hyperparameters in common and a compatible dataset schema are identified. Hyperparameters may be ranked according to each of their respective influences on a model performance metrics, and hyperparameter values identified as having greater influence may be more aggressively searched.