Deep Learning Feature Removal for Prediction Robustness
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models often include features that appear useful for prediction but are not truly related to the output, leading to poorer model performance over time.
Innovation Solution
A method is provided to identify and remove 'fake' features from deep learning models by comparing the performance of the model with and without each feature, using a difference in loss metric and outlier analysis to determine which features to remove.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning models include a large number of features to improve prediction capability, then the model can capture more patterns from training data, but the model performance deteriorates over time due to fake features that appear useful but are not truly related to the output
Solution Approach 1:
The patent extracts and removes fake features from the feature set by systematically comparing model performance with and without each feature. Features that show significant performance degradation when removed are identified as fake and excluded from the final model, thereby improving long-term reliability while maintaining predictive accuracy through the remaining genuine features
2Reliability
If features are removed from the model to eliminate fake features, then the model robustness improves, but the complexity of the feature selection process increases
Solution Approach 1:
The feature selection process is segmented into systematic steps: training multiple models with different feature combinations, comparing performance metrics, identifying fake features through statistical analysis of performance differences, and iteratively removing them. This segmentation transforms the complex task into manageable phases with clear decision criteria
Solution Approach 2:
The patent implements feedback mechanisms by continuously monitoring model performance when features are added or removed. Performance metrics from multiple training runs are fed back into the selection process, allowing automatic identification of fake features through statistical analysis of performance variations, thereby simplifying the overall complexity through automated decision-making
Data Source
AI summary
Methods, systems, and computer program products may obtain a machine learning model, a training dataset including a time range and a feature set including a number of features, and a number times to split the training dataset; for each feature in the feature set, determine, based on a difference between a first trained model including the machine learning model trained on the training dataset with that feature and a second trained model including the machine learning model trained on the training dataset without that feature, whether to update the training dataset to include an updated feature set by removing the feature from the feature set of the training dataset; train the machine learning model on the training dataset including the updated feature set to generate a trained machine learning model; and provide the trained machine learning model.