Class-Specific Threshold Tuning for Imbalanced Multi-Class Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
State-of-the-art machine learning models are biased towards predicting the majority class in imbalanced datasets, leading to decreased accuracy in minority class predictions, especially in multiclass classification scenarios where common threshold tuning methods are intractable.
Innovation Solution
The approach optimizes a distinct classification threshold for each majority class using a validation dataset to evaluate candidate thresholds and select the one yielding the best validation score, allowing for post-processing calibration of class probabilities to improve classification accuracy by disqualifying majority classes based on learned minimum probability thresholds.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If state-of-the-art machine learning models are used for imbalanced multiclass classification, then the model training process is simplified, but the model becomes excessively biased towards predicting the majority class, decreasing accuracy for minority classes
Solution Approach 1:
The patent segments the classification problem by introducing separate classification thresholds for each class instead of using a single global threshold. This allows the model to handle majority and minority classes differently, applying more stringent thresholds to majority classes to reduce false positives and more lenient thresholds to minority classes to improve detection, thereby resolving the bias without retraining the model
Solution Approach 2:
The patent changes the parameter of classification thresholds from a single uniform value to multiple class-specific values. By optimizing distinct thresholds for each class based on validation data, the system adjusts the decision boundary parameters to balance the trade-off between majority class precision and minority class recall, eliminating the need for complex retraining while improving minority class accuracy
2Measurement precision
If common threshold tuning methods are applied to multiclass classification, then minority class accuracy improves, but the computational complexity becomes intractable for multiclass problems with more than a very few classes
Solution Approach 1:
The patent performs preliminary action by pre-optimizing class-specific thresholds using a validation dataset before deploying the model for production. This upfront calibration step establishes the optimal threshold for each class, allowing the main classification process to simply apply these pre-determined thresholds without complex real-time computations, making multiclass threshold tuning tractable
Solution Approach 2:
The system performs self-service by automatically optimizing its own classification thresholds using validation data without requiring manual intervention or complex external tuning processes. The model self-calibrates by evaluating different threshold configurations and selecting the optimal set that maximizes performance metrics, eliminating the need for sophisticated external threshold tuning mechanisms
3Productivity
If a single classification threshold is used for all classes, then the classification process is computationally efficient, but the model cannot adequately handle class imbalance, leading to biased predictions
Solution Approach 1:
The patent applies local quality by assigning different classification thresholds to different classes based on their specific characteristics and imbalance ratios. Instead of using a uniform global threshold, each class receives a locally optimized threshold that accounts for its prevalence in the dataset, allowing efficient classification while maintaining accuracy for both majority and minority classes
Data Source
AI summary
In an embodiment, a computer generates, from an input, an inference that contains multiple probabilities respectively for multiple mutually exclusive classes that contain a first class and a second class. The probabilities contain (e.g. due to overfitting) a higher probability for the first class that is higher than a lower probability for the second class. In response to a threshold exceeding the higher probability, the input is automatically and more accurately classified as the second class. One, some, or almost all classes may have a respective distinct threshold that can be concurrently applied for acceleration. Data parallelism may simultaneously apply a threshold to a batch of multiple inputs for acceleration.