Feature Subset Selection Using Ranking and Parallel Search
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current machine learning (ML) model training is resource-intensive and prone to overfitting due to the need to process all features in a dataset, with feature selection being exponentially complex and inefficient, especially for large datasets, and existing methods are sequential, impractical for parallel processing, and lack an effective way to determine the ideal number of features to process.
Innovation Solution
The approach combines feature ranking with non-sequential search techniques, using multiple scoring functions to rank features by relevance and create subsets based on these rankings, allowing for parallel evaluation and selection of feature subsets, optimizing for parallel execution and reducing the number of evaluations from exponential to linear, thereby accelerating training and preventing overfitting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If all features are processed in ML model training, then model accuracy may be improved, but resource consumption and training time increase substantially
Solution Approach 1:
The patent extracts and removes irrelevant or redundant features from the dataset before training the ML model. By identifying and eliminating features that do not contribute meaningfully to model accuracy, the system reduces training time and resource consumption while maintaining model performance.
Solution Approach 2:
The feature set is segmented into relevant and irrelevant subsets using statistical analysis. The system divides the original feature space into meaningful groups, processing only the relevant segments for model training, thereby reducing overall computation time while preserving accuracy.
2Measurement precision
If feature selection is performed exhaustively to find optimal subsets, then model performance improves, but computational complexity becomes exponential and infeasible
Solution Approach 1:
The patent changes the approach from exhaustive search to statistical parameter-based selection. Instead of evaluating all 2^n possible feature subsets, the system uses statistical measures (such as correlation coefficients, variance analysis, or information gain) to rank and select features, reducing computational complexity from exponential to polynomial time while maintaining effective feature selection.
3Measurement precision
If sequential feature subset evaluation is used, then thorough assessment is achieved, but processing speed and scalability are limited
Solution Approach 1:
The feature evaluation process is segmented into independent parallel tasks. Different feature subsets are evaluated simultaneously using multiple processing units or cores, transforming the sequential evaluation into a parallel architecture that maintains thoroughness while dramatically improving processing speed and scalability.
Data Source
AI summary
The present invention relates to dimensionality reduction for machine learning (ML) models. Herein are techniques that individually rank features and combine features based on their rank to achieve an optimal combination of features that may accelerate training and/or inferencing, prevent overfitting, and/or provide insights into somewhat mysterious datasets. In an embodiment, a computer calculates, for each feature of a training dataset, a relevance score based on: a relevance scoring function, and statistics of values, of the feature, that occur in the training dataset. A rank based on relevance scores of the features is calculated for each feature. A sequence of distinct subsets of the features, based on the ranks of the features, is generated. For each distinct subset of the sequence of distinct feature subsets, a fitness score is generated based on training a machine learning (ML) model that is configured for the distinct subset.