Feature Selection via Statistical Segmentation for Regression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing feature selection algorithms for machine learning are computationally expensive and incapable of efficiently handling different types of features (numerical, categorical, integer, logical) in a unified manner, leading to impracticality in large-scale regression analysis due to the curse of dimensionality and overfitting.
Innovation Solution
A feature selection algorithm that estimates the significance of various feature types by splitting the feature range into subsets, calculating average target variable values, and determining the difference between maximum and minimum values to filter out irrelevant features, allowing for automated and computationally efficient selection of significant features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing feature selection algorithms are used, then feature selection can be performed, but processing time becomes excessively long and computational cost becomes prohibitive
Solution Approach 1:
The patent segments the feature selection process into distinct phases: (1) computing aggregate statistics (mean, variance, skewness, kurtosis) for each feature across the dataset, (2) computing pairwise correlation coefficients between features, (3) ranking features based on these pre-computed statistics, and (4) selecting top-ranked features. This segmentation allows each phase to be optimized independently and enables parallel computation, dramatically reducing processing time while maintaining selection accuracy.
Solution Approach 2:
The patent performs preliminary computations of aggregate statistics and correlation coefficients before the actual feature selection ranking. These pre-computed values serve as proxies for more computationally intensive model training processes. By performing this preliminary action, the system avoids repeated expensive computations during the selection process itself, achieving fast feature selection without sacrificing quality.
2Measurement precision
If existing feature selection algorithms are used, then feature selection can be performed, but computational cost becomes prohibitive for large-scale datasets
Solution Approach 1:
The patent replaces traditional model-training-based feature selection mechanisms with a statistical statistics-based mechanism. Instead of training multiple models to evaluate feature importance (computationally expensive), the system substitutes this with aggregate statistics computation (computationally efficient). This substitution maintains feature selection quality while dramatically reducing computational cost and energy consumption.
Solution Approach 2:
The patent changes the parameters used for feature evaluation from model performance metrics (requiring iterative training) to aggregate statistical parameters (mean, variance, skewness, kurtosis, correlation coefficients) that can be computed in a single pass through the data. This parameter change fundamentally reduces the computational complexity from O(n*m*k) to O(n*m) where n is samples, m is features, and k is iterations.
3Reliability
If all features are used in model construction, then model completeness is maintained, but the curse of dimensionality and overfitting occur
Solution Approach 1:
The patent extracts and removes irrelevant and redundant features from the feature set by computing their aggregate statistics and correlation coefficients. Features with low variance, high skewness, or high correlation with other features are identified and extracted (removed) from the candidate set. This leaving only the most relevant features for model construction, reducing model complexity while maintaining reliability.
Solution Approach 2:
The patent applies partial action by selecting only a subset of the most important features rather than using all available features. The system computes statistics for all features but then takes only the top-ranked portion (e.g., top 10% or top N features) for model construction. This partial action avoids the curse of dimensionality and overfitting while maintaining the essential information needed for reliable modeling.
Data Source
AI summary
Aspects of the subject disclosure may include, for example, dividing a feature range of a feature into a plurality of subsets that span the feature range, calculating an average target variable value for each subset of the plurality of subsets, resulting in a plurality of average target variable values, and estimating a measure of feature significance with respect to a target variable by determining a difference between a maximum average target variable value and a minimum average target variable value in the plurality of average target variable values. Other embodiments are disclosed.