Adaptive Multi-Threshold Feature Filtering for ML Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing feature filtering methods for machine learning models rely on manually set fixed thresholds, leading to inefficient training and inaccurate predictions due to the inability to dynamically adjust filtering based on training conditions.
Innovation Solution
An automatic multi-threshold feature filtering method that calculates a feature filtering threshold and correlation values iteratively, using the maximum gradient and sample data ratio, to filter features dynamically during each training iteration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a fixed threshold is manually set for feature filtering, then the filtering process is simple and easy to implement, but the filtering effect is unstable and prediction accuracy is reduced due to inability to adapt to changing training conditions
Solution Approach 1:
The patent transforms the static fixed threshold into a dynamic adaptive threshold that automatically adjusts during training iterations. The threshold is calculated based on gradient information and sample data ratios from previous iterations, enabling it to adapt to changing training conditions and maintain stable filtering effectiveness throughout the training process
Solution Approach 2:
The patent implements a feedback mechanism where the filtering threshold and correlation values are calculated based on the results from previous iteration rounds. This feedback loop allows the system to learn from previous training outcomes and adjust filtering parameters accordingly, improving both stability and accuracy of feature selection
2Device complexity
If a single fixed threshold is used for feature filtering, then the implementation is straightforward, but the threshold cannot be dynamically adjusted to filter all features well, resulting in unreliable models
Solution Approach 1:
The patent divides the filtering process into multiple stages with different thresholds corresponding to different filtering requirements. Instead of using a single threshold, it calculates multiple thresholds (first threshold for initial filtering, second threshold for refined filtering) based on gradient magnitudes and sample data ratios, allowing precise control over which features are retained at each stage
Solution Approach 2:
The patent dynamically changes filtering parameters (threshold values and correlation thresholds) during training based on calculated gradient information and data ratios. The thresholds are not fixed but are continuously adjusted according to the training progress and feature importance detected in each iteration, significantly improving filtering accuracy
3Adaptability or versatility
If manual threshold setting is used, then human experience can be applied to initial filtering, but the threshold cannot be automatically adjusted according to sample data, reducing training efficiency
Solution Approach 1:
The patent enables the system to automatically determine its own filtering thresholds without requiring manual intervention. The threshold calculation uses gradient information and sample data ratios computed during training, allowing the system to self-adjust and optimize feature filtering based on the actual characteristics of the training data, thereby improving training efficiency
Data Source
AI summary
An automatic multi-threshold feature filtering method and an apparatus thereof are provided. In an iterative process of training a machine learning model, the feature filtering method calculates a feature filtering threshold and feature correlation values of a current round of iteration based on a result of a previous iteration, and performs feature filtering on samples based on the calculated feature filtering threshold and the calculated feature correlation values. The feature filtering apparatus of the present disclosure includes a calculation module and a feature filtering module. The method and apparatus of the present disclosure can automatically generate different feature filtering thresholds at each iteration, which greatly improves an accuracy of a filtering threshold, and can greatly increase the training speed of automatic machine learning and an accuracy of a machine learning model compared with fixed and single thresholds nowadays.