ML Feature Backward Stripping for Lower Training Resource Use
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The use of excessive features in machine learning can slow training and limit the robustness of models on unseen data, leading to inefficient resource consumption.
Innovation Solution
Implement automatic machine learning feature backward stripping, where features with low importance are identified and their activation status is propagated backwardly through the pipeline, reducing computation, storage, and network resources by refraining from gathering or generating inactive features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If excessive features are used in machine learning, then the model can capture more patterns and improve accuracy, but the training time increases and resource consumption increases
Solution Approach 1:
The system performs feature importance evaluation in advance before training to identify and remove irrelevant features. By pre-evaluating feature importance using techniques like permutation importance or SHAP values, the system determines which features to exclude from the training process, thereby reducing training time while maintaining model accuracy.
Solution Approach 2:
The patent extracts and removes unnecessary features from the feature set before training begins. Through feature selection methods, the system identifies and extracts only the most relevant features, eliminating redundant and irrelevant features that would otherwise increase training time and computational resource consumption.
2Measurement precision
If excessive features are used in machine learning, then the model can capture more patterns, but resource consumption (processor cycles, memory, network traffic) increases
Solution Approach 1:
The system performs feature importance evaluation in advance before training to identify and remove irrelevant features. By pre-evaluating feature importance using techniques like permutation importance or SHAP values, the system determines which features to exclude from the training process, thereby reducing training time while maintaining model accuracy.
Solution Approach 2:
The patent extracts and removes unnecessary features from the feature set before training begins. Through feature selection methods, the system identifies and extracts only the most relevant features, eliminating redundant and irrelevant features that would otherwise increase training time and computational resource consumption.
3Ease of manufacture
If all features are always processed, then the system is simple to implement, but it cannot adapt to changing data patterns or reduce resource consumption
Solution Approach 1:
The system dynamically adjusts the feature set based on changing data patterns and model performance requirements. Through continuous monitoring and re-evaluation of feature importance, the system can adaptively add or remove features from the training process, enabling it to respond to evolving data characteristics while optimizing resource consumption.
Solution Approach 2:
The patent implements feedback mechanisms that continuously monitor model performance and data characteristics. Based on this feedback, the system automatically adjusts feature selection to optimize both accuracy and efficiency, allowing the system to adapt to changing conditions without requiring manual intervention.
Data Source
AI summary
Features are used to train one or more ML models in a modelling layer. In a feature selection layer, each generated ML model is analyzed to determine, for each input feature, a degree of importance of the feature on the results generated by the ML model. Features with low importance are identified and the information is propagated backward to the data source and feature engineering layers. In response, the data source and feature engineering layers refrain from gathering or generating the unimportant features. Based on a confidence measure of the determination that each feature is important or unimportant, a number of periods between reevaluation of the feature importance is determined. After the number of periods has elapsed, a removed feature is restored to the pipeline.