Sparse Feature Selection for Interpretable Anomalous Subgroup Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning techniques for detecting anomalous subgroups in large datasets are computationally expensive and often result in less interpretable subgroups due to the use of the whole input feature set, leading to excessive computational resources and potential bias.
Innovation Solution
A model-free sparsity-based automatic feature selection (SAFS) framework that ranks and selects features using sparsity metrics like Gini-index and normalized odds ratios, reducing the search space and computational time while maintaining detection performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the whole input feature set is used for detecting anomalous subgroups, then detection performance is maintained, but computational cost increases and interpretability decreases
Solution Approach 1:
The patent extracts and removes irrelevant or redundant features from the complete feature set, keeping only the most informative features for subgroup detection. This reduction maintains detection performance while significantly lowering computational cost and improving result interpretability.
Solution Approach 2:
The patent applies different processing strategies to different features based on their individual characteristics and importance. High-importance features are retained with full processing, while low-importance features are reduced or removed, creating a non-uniform feature set that optimizes both performance and efficiency.
2Reliability
If the whole input feature set is used for detecting anomalous subgroups, then detection performance is maintained, but computational time increases
Solution Approach 1:
The patent extracts and removes irrelevant or redundant features from the complete feature set, keeping only the most informative features for subgroup detection. This reduction maintains detection performance while significantly lowering computational cost and improving result interpretability.
Solution Approach 2:
The patent performs feature selection and ranking as a preliminary step before the actual subgroup detection process. By pre-processing the feature set to identify and retain only the most important features, the system reduces the computational burden of subsequent detection operations, thereby reducing overall computational time.
3Reliability
If the whole input feature set is used for detecting anomalous subgroups, then detection performance is maintained, but interpretability of subgroups decreases
Solution Approach 1:
The patent extracts and removes irrelevant or redundant features from the complete feature set, keeping only the most informative features for subgroup detection. This reduction maintains detection performance while significantly lowering computational cost and improving result interpretability.
Solution Approach 2:
The patent applies different processing strategies to different features based on their individual characteristics and importance. High-importance features are retained with full processing, while low-importance features are reduced or removed, creating a non-uniform feature set that optimizes both performance and efficiency.
Data Source
AI summary
One or more computer processors discover an anomalous subset through sparsity-based automatic feature selection.