Machine Learning Parameter Selection via Segmented Feature Tuning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning model training is hindered by the computational expense of selecting optimal feature sets and hyperparameters, as existing methods often require repeated training across high-dimensional spaces, making it challenging to achieve accurate predictions.
Innovation Solution
A method is introduced that systematically selects feature sets and hyperparameters by training models on a subset of combinations, using a parameter selection application to evaluate performance measures and identify the best configurations for predicting characteristics in a dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If repeated training of different models in high-dimensional spaces is performed to select feature sets and hyperparameters, then prediction accuracy is improved, but computational expense increases tremendously
Solution Approach 1:
The patent segments the feature set selection and hyperparameter tuning processes into separate, independent modules. Feature selection identifies the optimal subset of input variables, while hyperparameter tuning optimizes model parameters. This segmentation allows each module to be optimized independently, reducing the overall computational burden compared to simultaneously searching the entire parameter space.
Solution Approach 2:
The patent performs preliminary feature selection before hyperparameter tuning and model training. By first identifying the optimal feature set using statistical methods and domain knowledge, the system reduces the dimensionality of the subsequent hyperparameter search space. This preliminary action eliminates the need to train models across all possible feature combinations, significantly reducing computational expense.
2Use of energy by moving object
If feature selection and hyperparameter tuning are performed separately, then computational expense is reduced, but the integration of both optimizations may be suboptimal
Solution Approach 1:
The patent implements a feedback mechanism where the results of feature selection inform the hyperparameter tuning process, and vice versa. The optimal feature set identified in the first stage becomes the input for the second stage, and the performance metrics from hyperparameter tuning can inform potential refinements to feature selection. This feedback loop ensures that both optimizations work together synergistically to achieve the best overall model performance.
Solution Approach 2:
The patent introduces a performance evaluation metric as an intermediary between feature selection andhyperparameter tuning. This metric serves as a common interface that allows both modules to be optimized independently while ensuring their results are compatible and integrated. The performance metric guides both the feature selection andhyperparameter tuning processes, ensuring they work toward the same optimization goal without requiring direct coupling of the two complex processes.
Data Source
AI summary
A computing device selects a feature set and hyperparameters for a machine learning model to predict a value for a characteristic in a scoring dataset. A number of training model iterations is determined. A unique evaluation pair is selected for each iteration that indicates a feature set selected from feature sets and a hyperparameter configuration selected from hyperparameter configurations. A machine learning model is trained using each unique evaluation pair. Each trained machine learning model is validated to compute a performance measure value. An estimation model is trained with the feature set, the hyperparameter configuration, and the performance measure value computed for unique evaluation pair. The trained estimation model is executed to compute the performance measure value for each unique evaluation pair. A final feature set and a final hyperparameter configuration are selected based on the computed performance measure value.