Recursive Feature Elimination Using SVM Feedback Loops
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing recursive feature elimination methods based on support vector machines (SVM-RFE) for data-intensive applications, such as disease gene selection, are inefficient due to independent rounds of SVM training and feature elimination, which do not effectively leverage dependency between rounds.
Innovation Solution
The proposed method implements a recursive feature elimination process where SVM training and feature elimination rounds are dependent on each other, using a decision logic, SVM learning machine, ranking criterion logic, and eliminating logic to iteratively train and rank features until the group becomes empty, outputting a feature elimination history or ranking list.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If independent rounds of SVM training and feature elimination are used, then the process is simple to implement, but the efficiency and accuracy of feature ranking deteriorates
Solution Approach 1:
The patent implements feedback by using the feature ranking results from one round to guide the feature selection in subsequent rounds. The SVM model's weight coefficients from each training round provide feedback on feature importance, which is then used to eliminate less important features before the next training round, creating a dependent and iterative process that improves efficiency.
Solution Approach 2:
The patent applies preliminary action by performing feature elimination before the next SVM training round based on the ranking from the current round. This preliminary feature selection prepares the feature set for the subsequent training, avoiding unnecessary computations and improving the overall efficiency of the feature elimination process.
2Quantity of substance
If independent rounds of SVM training and feature elimination are used, then the computational cost is low, but the accuracy and relevance of feature ranking deteriorates
Solution Approach 1:
The feedback mechanism uses the SVM model's weight coefficients and feature ranking results from each round to guide subsequent feature elimination and training. This feedback loop ensures that computationally expensive training rounds focus only on the most relevant features, improving ranking accuracy without proportionally increasing computational cost.
Solution Approach 2:
The patent introduces dynamics by making the feature set adaptive across rounds. The feature elimination criteria dynamically adjust based on the SVM model's learned weights from previous rounds, allowing the system to automatically focus computational resources on the most informative features and improve ranking accuracy efficiently.
Data Source
AI summary
Method, apparatus and system are described to perform a feature eliminating method based on a support vector machine. In some embodiments, a value for each feature in a group of features provided by a training data is determined. At least one feature is eliminated from the group by utilizing the value for each feature in the group. The value for each feature in the group is updated based upon a part of the training data that corresponds to the eliminated feature.