Iterative Feature Selection Using Explainability Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face challenges in efficiently selecting and recombining machine learning features using explainability vectors, due to difficulties in determining feature importance and high computation costs associated with training models with full feature sets.
Innovation Solution
The method involves dividing a full set of features into feature groups, training candidate models for each group, generating explainability vectors, and iteratively selecting the most impactful features until performance metrics are met, thereby generating an improved set of features in a computationally expedient manner.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a full feature set is used to train machine learning models, then model performance is improved, but computational costs increase
Solution Approach 1:
The full feature set is divided into multiple feature groups, and candidate models are trained separately on each group. This segmentation reduces the computational burden of training while maintaining model performance through iterative feature selection using explainability vectors.
Solution Approach 2:
The patent extracts only the most impactful features from each feature group using explainability vectors, creating a reduced feature set that maintains model performance while reducing computational costs. The extraction process identifies and removes redundant or less important features.
2Measurement precision
If explainability vectors are used for feature selection, then feature importance is determined, but computation costs increase
Solution Approach 1:
Explainability vectors are computed separately for each candidate model trained on a specific feature group, rather than computing one large explainability vector for all features. This segmentation reduces the computational complexity of generating explainability vectors while maintaining precise feature importance measurements.
3Use of energy by moving object
If the number of features is reduced, then computational costs are lowered, but model performance may deteriorate
Solution Approach 1:
The patent uses explainability vectors as feedback to iteratively refine the feature selection process. By analyzing the explainability vectors from candidate models, the system identifies which features to retain or remove, ensuring that model performance is maintained while reducing the feature set size.
Solution Approach 2:
The patent selects a reduced but sufficient subset of features that provides enough information to maintain model performance. Rather than using all features, it applies partial action by selecting only the most impactful features needed to achieve acceptable model performance.
Data Source
AI summary
Systems and methods for selecting machine learning features using iterative batch feature reduction. In some aspects, the system trains a plurality of candidate models based on a plurality of feature groups split from a first set of features. Each candidate model takes as input a feature group of no more than a first threshold number of features. For each candidate model in the plurality of candidate models, the system processes the candidate model to extract an explainability vector. Based on the explainability vector, the system selects a second threshold number of features from the feature group to generate a slim feature group. The system trains a slim candidate model which takes as input the slim feature group. The system generates a second set of features by combining features from a plurality of slim candidate models.