Sequential Feature Selection via Statistical Significance Testing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning models face challenges in determining the optimal feature set for accurate classification or prediction, as using too few features can compromise accuracy while using too many features increases computational cost without significant gain in accuracy.
Innovation Solution
The method involves using one or more processors to apply backward elimination or forward addition techniques, selecting a reduced feature set through statistical tests to determine if the model using the new feature set is adequate for classification, thereby optimizing the feature set for efficient and accurate performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a model uses a greater number of features, then classification accuracy is improved, but computational cost increases
Solution Approach 1:
The patent extracts and removes unnecessary features from the feature set through backward elimination. Statistical tests are applied to identify and eliminate features that do not contribute significantly to classification accuracy, thereby reducing computational cost while maintaining the essential accuracy-performance features.
Solution Approach 2:
Instead of starting with a small feature set and adding features (forward selection), the patent applies backward elimination by starting with a complete feature set and removing unnecessary features. This inverted approach allows the model to begin with high accuracy and systematically reduce complexity by eliminating redundant features based on statistical significance.
2Device complexity
If a model uses a lesser number of features, then computational cost is reduced, but classification accuracy is compromised
Solution Approach 1:
The patent changes the parameter of feature set size by applying statistical significance testing at each elimination step. The elimination process is controlled by a significance threshold parameter, allowing systematic reduction of features while monitoring and maintaining classification accuracy within acceptable bounds.
Solution Approach 2:
The patent implements feedback through statistical evaluation at each step of the backward elimination process. The significance test results provide feedback on whether removing a feature will adversely affect classification accuracy, allowing the process to stop when further removal would compromise performance, thus finding the optimal feature set size.
3Measurement precision
If unnecessary features are included in the model, then computational resources are wasted, but model accuracy may be maintained
Solution Approach 1:
The patent extracts and removes unnecessary features that waste computational resources. By applying statistical significance tests, the system identifies and eliminates features that do not contribute to classification accuracy, thereby removing the source of computational waste while preserving the accuracy-critical features.
Solution Approach 2:
The patent discards unnecessary features through systematic elimination based on statistical criteria. The process recovers computational resources by removing redundant features while maintaining the essential feature subset that provides the desired classification accuracy, effectively recovering resources without sacrificing performance.
Data Source
AI summary
Systems and methods for determining a reduced feature set for a model for classifying data are disclosed. In some embodiments, the method includes obtaining a first feature set for the model. The method may also include selecting a second feature set for the model, wherein the second feature set is a candidate for the reduced feature set. In some embodiments, the second feature set is a subset or a superset of the first feature set. In some embodiments, the selection includes applying a selection statistical test. The method may further include determining whether the model using the second feature set in place of the first feature set is adequate for classifying the data. In some embodiments, the determination includes applying an evaluation statistical test.