Meta-Learning Feature Subset Selection for ML Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning (ML) model training is resource-intensive and prone to overfitting due to the need to process all features in a dataset, making it inefficient and time-consuming, especially for datasets with hundreds of thousands of features, as existing feature selection methods are either ineffective or computationally infeasible.
Innovation Solution
The use of meta-learning techniques to rank features based on relevance scores and predict an optimal subset of features to process, reducing the number of feature subsets to evaluate from exponential to linear or logarithmic, through exploratory landmark search and ensemble ranking, allowing for efficient feature selection and acceleration of training.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If all features are processed in ML model training, then model accuracy may be improved, but computational resources and training time increase substantially
Solution Approach 1:
The patent extracts and removes irrelevant or redundant features from the feature set before training the ML model. By identifying and eliminating features that do not contribute meaningfully to model accuracy, the system reduces computational resource consumption while maintaining model performance.
Solution Approach 2:
The patent applies different processing treatments to different features based on their individual characteristics and relevance. Rather than uniformly processing all features, the system selectively processes only those features that meet certain relevance criteria, optimizing the balance between accuracy and computational cost.
2Reliability
If all features are processed in ML model training, then model accuracy may be improved, but training time increases substantially
Solution Approach 1:
The patent extracts and removes irrelevant or redundant features from the feature set before training the ML model. By identifying and eliminating features that do not contribute meaningfully to model accuracy, the system reduces training time while maintaining model performance.
Solution Approach 2:
The patent performs feature selection and filtering as a preliminary step before actual model training. By pre-processing the feature set to remove irrelevant features, the system prepares optimized input data that reduces the computational burden during the training phase, thereby reducing overall training time.
3Reliability
If feature selection is performed using exhaustive evaluation of all possible subsets, then optimal feature subset can be found, but computational complexity becomes infeasible
Solution Approach 1:
The patent extracts and removes irrelevant or redundant features from the feature set before training the ML model. By identifying and eliminating features that do not contribute meaningfully to model accuracy, the system reduces computational resource consumption while maintaining model performance.
Solution Approach 2:
The patent applies different processing treatments to different features based on their individual characteristics and relevance. Rather than uniformly processing all features, the system selectively processes only those features that meet certain relevance criteria, optimizing the balance between accuracy and computational cost.
Data Source
AI summary
The present invention relates to dimensionality reduction for machine learning (ML) models. Herein are techniques that individually rank features and combine features based on their rank to achieve an optimal combination of features that may accelerate training and/or inferencing, prevent overfitting, and/or provide insights into somewhat mysterious datasets. In an embodiment, a computer ranks features of datasets of a training corpus. For each dataset and for each landmark percentage, a target ML model is configured to receive only a highest ranking landmark percentage of features, and a landmark accuracy achieved by training the ML model with the dataset is measured. Based on the landmark accuracies and meta-features values of the dataset, a respective training tuple is generated for each dataset. Based on all of the training tuples, a regressor is trained to predict an optimal amount of features for training the target ML model.