Bagged Filtering for Feature Selection in Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for feature selection in classifiers often lead to overfitting and fail to generalize well due to the inclusion of noisy or irrelevant features, especially when dealing with large datasets and confounding factors, limiting their ability to accurately classify samples across different data sets.
Innovation Solution
The method employs 'bagged filtering' by splitting a development sample set into subsets, applying a classifier to each subset, and using an ensemble average to filter features based on their performance across multiple realizations, allowing for the selection or deselection of features that contribute meaningfully to classifier performance, thereby reducing overfitting and improving generalizability.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional feature selection methods are applied to development sets, then classification performance may improve on training data, but overfitting occurs and generalization to other datasets deteriorates
Solution Approach 1:
The development set is divided into multiple subsets (e.g., 10 subsets) through random sampling. Feature selection is performed independently on each subset using the same selection method. This segmentation prevents overfitting to a single subset and improves generalization by ensuring features are consistently selected across different data partitions.
Solution Approach 2:
The results from multiple independent feature selection runs on different subsets are merged by counting how often each feature is selected. Features that appear frequently across subsets are identified as robust features. This combining approach aggregates evidence from multiple data partitions to identify features with genuine predictive power that generalize well.
2Loss of information
If many features are included in classifier development, then more information is available for classification, but noisy and irrelevant features degrade classifier performance
Solution Approach 1:
The method extracts and removes noisy and irrelevant features by performing feature selection on multiple random subsets and identifying features that consistently appear across subsets. Features that do not demonstrate consistent selection are discarded as noise. This extraction process retains only the most robust and informative features, improving classifier performance by eliminating detrimental features.
3Quantity of substance
If feature selection is performed on the entire development set, then all available data is used for selection, but the selected features may not generalize to other datasets
Solution Approach 1:
Instead of performing feature selection on the entire development set at once, the method segments the data into multiple random subsets and performs feature selection on each subset independently. This segmentation allows the method to utilize all available data while preventing overfitting to any single partition, thereby improving feature robustness and generalization.
Solution Approach 2:
The method performs preliminary feature selection on multiple random subsets before finalizing the feature set for the complete development set. This preliminary action on subsets identifies robust features that are then validated and confirmed when applied to the entire dataset, ensuring both data utilization and feature reliability.
Data Source
AI summary
Classifier generation methods are described in which features used in classification (e.g., mass spectral peaks) are selected, or deselected using bagged filtering. A development sample set is split into two subsets, one of which is used as a training set the other of which is set aside. We define a classifier (e.g., K-nearest neighbor, decision tree, margin-based classifier or other) using the training subset and at least one of the features (or subsets of two or more features in combination). We apply the classifier to a subset of samples. A filter is applied to the performance of the classifier on the sample subset and the at least one feature is added to a “filtered feature list” if the classifier performance passes the filter. We do this for many different realizations of the separation of the development sample set into two subsets, and, for each realization, different features or sets of features in combination. After all the iterations are performed the filtered feature list is used to either select features, or deselect features, for a final classifier.