Automated Machine Learning Model Selection Hierarchy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Choosing the most effective combination of data processing techniques, feature selection techniques, machine learning algorithms, and hyperparameters for building a machine learning model can be complex and requires significant expertise, making it challenging for non-specialists to achieve optimal model performance.
Innovation Solution
A service provider system that automatically determines a suitable machine learning model by performing an iterative model selection process, using a selection hierarchy to optimize data processing packages, feature selection packages, machine learning platforms, algorithms, and hyperparameters based on performance metrics, thereby abstracting out the complexity of model construction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If automatic model selection is implemented, then ease of operation is improved, but device complexity increases
Solution Approach 1:
The patent introduces an automatic model selection system that acts as an intermediary between the user and the complex machine learning model construction process. This system automatically selects appropriate data processing techniques, feature selection methods, machine learning algorithms, and hyperparameters based on the input data characteristics, thereby shielding users from the underlying complexity while delivering optimized models.
Solution Approach 2:
The system enables self-service by allowing the automatic model selection process to autonomously determine the optimal machine learning model configuration without requiring user expertise. The system evaluates multiple algorithms and parameters automatically, selecting the best combination based on performance metrics, thus making the complex model building process accessible to non-experts.
2Manufacturing precision
If expertise-based model selection is required, then manufacturing precision is improved, but ease of operation deteriorates
Solution Approach 1:
The automatic model selection system performs the expertise-based evaluation and selection process autonomously. It automatically assesses different machine learning algorithms, data processing techniques, and hyperparameter combinations based on performance metrics, eliminating the need for user expertise while maintaining high model performance through systematic automated evaluation.
Solution Approach 2:
The system incorporates feedback mechanisms where performance metrics from training and validation are continuously monitored and used to guide the model selection process. This feedback loop enables the system to automatically refine its selections and identify the optimal model configuration based on actual performance data rather than relying on user expertise.
3Manufacturing precision
If comprehensive hyperparameter tuning is performed, then manufacturing precision is improved, but loss of time increases
Solution Approach 1:
The system performs preliminary action by pre-evaluating and ranking different machine learning algorithms and hyperparameter combinations based on historical performance data and data characteristics. This preliminary assessment allows the automatic model selection process to quickly identify promising candidates without exhaustively tuning every parameter for every algorithm, thereby reducing time while maintaining performance quality.
Solution Approach 2:
The system applies partial action by focusing hyperparameter tuning efforts only on the most promising algorithm candidates identified through preliminary evaluation. Rather than exhaustively tuning all parameters for all algorithms, the system concentrates computational resources on partial tuning of selected candidates, achieving good performance with reduced time investment.
Data Source
AI summary
A method includes receiving a set of training data and selecting a first machine learning platform based on a first optimization function that metrics past machine learning platforms used for training on the set of training data. The method also includes selecting a first algorithm supported by the first machine learning platform based on a second optimization function that metrics past algorithms used for training on the set of training data. Further, the method includes determining one or more hyperparameters supported by the first algorithm based on a third optimization function that metrics past combinations of hyperparameters from the set of hyperparameters used for training on the set of training data. The method also includes training a machine learning model on the set of training data using the first machine learning platform, the first algorithm, and the one or more hyperparameters.