Feature Selection via Augmented Matrix Sweep
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing feature selection methods for classification models are computationally expensive due to their polynomial or quadratic nature, leading to overfitting and insufficient generalization when dealing with large numbers of features, especially when selecting and ranking more than a small number of features.
Innovation Solution
A computer-implemented method that generates an augmented matrix from a training dataset and iteratively applies a sweep operation to identify the most-relevant features, allowing for efficient ranking and selection of features for classification models, thereby reducing computational cost and improving generalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional feature selection methods are used, then features can be selected for classification, but the computational cost increases polynomially or quadratically with the number of features
Solution Approach 1:
The patent transforms the feature selection problem from a sequential iterative process to a parallel matrix-based process. By changing the computational parameters from individual feature evaluations to matrix-wide operations, the method achieves linear scaling with the number of features while maintaining selection accuracy through systematic matrix decomposition and information theory-based scoring.
Solution Approach 2:
The patent creates a computational copy of the feature selection process through matrix representations. Instead of repeatedly training classification models for each feature combination, it constructs augmented matrices that encode all feature relationships simultaneously, allowing parallel computation of feature relevance scores without repeated model training.
2Reliability
If all features are used to train a classification model, then more information is available for classification, but the model becomes over fitted and insufficiently generalized
Solution Approach 1:
The patent extracts only the most relevant features from the complete feature set by computing feature scores based on matrix decomposition results. It identifies and removes irrelevant or redundant features that would contribute to overfitting, keeping only the subset of features that provide meaningful discriminative power for generalization to new observations.
Solution Approach 2:
The patent performs preliminary feature selection and ranking before training the classification model. By pre-computing feature relevance scores using matrix operations and information theory metrics, it prepares an optimized feature subset that prevents overfitting from the outset, rather than attempting to control model complexity during the training phase alone.
3Measurement precision
If forward feature selection is used to select features stepwise, then relevant features can be identified, but the process requires linear to quadratic number of estimations
Solution Approach 1:
The patent merges the sequential feature evaluation process into a single parallel matrix computation. Instead of evaluating features one at a time or in small batches through repeated model training, it combines all feature relationships into augmented matrices and computes relevance scores simultaneously through matrix decomposition, reducing the number of computational estimations from linear or quadratic to a constant number of matrix operations.
Solution Approach 2:
The patent maintains continuous useful computation throughout the feature selection process by using matrix operations that leverage previously computed values. The augmented matrix structure allows each computation to build on previous results, eliminating redundant calculations and maintaining high computational efficiency throughout the entire feature ranking process.
Data Source
AI summary
A computer-implemented method for selecting features for classification may include (1) generating a matrix X, a column vector Y, and a matrix Z from a training dataset that includes a plurality of samples with a plurality of features, (2) generating an augmented matrix from the matrix X, the column vector Y, and the matrix Z, (3) identifying one or more most-relevant features from the plurality of features by iteratively applying a sweep operation to the augmented matrix, and (4) training a classification model using the most-relevant features from the plurality of features rather than all of the plurality of features. Various other methods, systems, and computer-readable media may have similar features.