Joint Feature Subset-Classifier Selection via Relax-Greedy Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for feature selection and classifier selection in machine learning follow a two-stage approach, which is time-consuming and may not provide an optimal feature set-classifier pair, as they are isolated and do not account for the intrinsic relationship between feature subsets and classifiers, leading to suboptimal predictive performance.
Innovation Solution
A method and system for joint selection of a feature subset-classifier pair using a relax-greedy approach that computes an integrated relevant score, exploiting ensembling optimization and avoiding overfitting, allowing for simultaneous and automated selection of the best combination of features and classifiers based on performance objectives.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a two-stage approach is used for feature selection and classifier selection, then the process is simpler to implement, but the predictive performance is suboptimal and the process is time-consuming
Solution Approach 1:
The patent combines feature selection and classifier selection into a single integrated optimization process. Instead of performing feature selection first and then classifier selection separately, the system simultaneously optimizes both the feature subset and classifier choice together, treating them as interdependent variables in a unified search space. This merging eliminates the sequential overhead and allows the algorithm to find optimal combinations that consider the interaction between features and classifiers.
Solution Approach 2:
The patent creates a universal framework that handles both feature selection and classifier selection through a single metaheuristic optimization algorithm. The system is designed to work with multiple classifiers and feature subsets simultaneously, making the optimization process multi-functional. The same optimization routine can evaluate different feature subsets paired with different classifiers, eliminating the need for separate specialized procedures for each selection task.
2Reliability
If feature selection and classifier selection are performed in isolation, then each can be optimized independently, but the intrinsic relationship between feature subsets and classifiers is ignored leading to suboptimal pairs
Solution Approach 1:
The patent merges feature selection and classifier selection into a single integrated optimization process. Instead of performing feature selection first and then classifier selection separately, the system simultaneously optimizes both the feature subset and classifier choice together, treating them as interdependent variables in a unified search space. This merging eliminates the sequential overhead and allows the algorithm to find optimal combinations that consider the interaction between features and classifiers.
Solution Approach 2:
The patent introduces a performance objective function as an intermediary that evaluates and guides the joint optimization of feature subsets and classifiers. This objective function serves as a mediator that quantifies the quality of feature-set classifier pairs, allowing the metaheuristic algorithm to navigate the search space effectively. The objective function captures the intrinsic relationship between features and classifiers by evaluating their combined performance, thus enabling optimal pairing without requiring explicit knowledge of their interactions.
3Productivity
If manual intervention is used for feature selection and classifier selection, then domain expertise can be applied, but the process requires considerable time and effort for each disease identification problem
Solution Approach 1:
The patent implements a self-service system where the optimization algorithm automatically performs feature selection and classifier selection without requiring manual domain expertise. The metaheuristic algorithm independently explores the search space, evaluates candidate solutions using the performance objective function, and identifies optimal feature-set classifier pairs autonomously. This self-service capability eliminates the need for manual intervention while maintaining high productivity through automated optimization.
Solution Approach 2:
The patent creates a universal framework that handles both feature selection and classifier selection through a single integrated optimization algorithm. The system is designed to work with multiple classifiers and feature subsets simultaneously, making the optimization process multi-functional. The same optimization routine can evaluate different feature subsets paired with different classifiers, eliminating the need for separate specialized procedures for each selection task.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A method and system for a feature subset-classifier pair for a classification task. The classification task corresponds to automatically classifying data associated with a subject(s) or object(s) of interest into an appropriate class based on a feature subset selected among a plurality of features extracted from the data and a classifier selected from a set of classifier types. The method proposed includes simultaneously determining the feature subset-classifier pair based on a relax-greedy {feature subset, classifier} approach utilizing sub-greedy search process based on a patience function, wherein the feature subset-classifier pair provides an optimal combination for more accurate classification. The automatic joint selection is time efficient solution, effectively speeding up the classification task.