Weighted Majority Vote Classifier for Imbalanced Data Precision
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning methods struggle to effectively handle imbalanced datasets, particularly in applications like fraud detection, medical diagnosis, and predictive maintenance, where the number of positive examples is significantly smaller than negative examples, leading to high false positive rates.
Innovation Solution
An ensemble learning method that involves generatively oversampling minority class examples using kNN and a multivariate probability distribution, generating subsamples, and learning a diversity-aware weighted majority vote classifier to minimize diversity among base classifiers on positive samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional classification methods are used on imbalanced datasets, then the model can be trained quickly with simple algorithms, but the false positive rate becomes unacceptably high
Solution Approach 1:
The patent segments the classification problem into multiple base classifiers that are trained on different subsamples of the data. Each base classifier focuses on specific regions of the feature space or different aspects of the minority class, allowing the ensemble to achieve high reliability without requiring any single classifier to be overly complex
Solution Approach 2:
The patent combines multiple base classifiers into an ensemble system where their predictions are aggregated. This merging allows the system to achieve high reliability through collective decision-making, while each individual base classifier remains relatively simple, thus resolving the contradiction between reliability and training complexity
2Measurement precision
If oversampling techniques are applied to balance the dataset, then the classification accuracy on minority class improves, but the diversity among base classifiers decreases
Solution Approach 1:
The patent applies local quality by creating different subsamples with varying compositions and characteristics. Each base classifier is trained on a specific subsample that captures local patterns and features, ensuring that while all classifiers achieve good accuracy on the minority class, they maintain diversity through their specialized focus on different data regions
Solution Approach 2:
The patent introduces dynamics by generating multiple different subsamples from the oversampled dataset, where each subsample has dynamic variations in composition. This allows base classifiers to adapt to different data distributions while maintaining overall diversity in the ensemble, resolving the contradiction between accuracy and diversity
3Reliability
If cost-sensitive approaches are used to penalize minority class misclassification, then the model pays more attention to positive samples, but the overall model complexity increases
Solution Approach 1:
The patent segments the cost-sensitive learning problem across multiple base classifiers rather than implementing a single complex cost-sensitive model. Each base classifier handles a portion of the minority class detection task with simpler learning algorithms, and the ensemble collectively achieves high reliability for minority class detection without requiring any single component to be overly complex
Data Source
AI summary
An ensemble learning based method is for a binary classification on an imbalanced dataset. The imbalanced dataset has a minority class comprising positive samples and a majority class comprising negative samples. The method includes: generatively oversampling the imbalanced dataset by synthetically generating minority class examples, thereby generating a generated dataset; using the generated dataset to generate subsamples, and learning a base classifier on each of the subsamples to determine a plurality of base classifiers; and learning a weighted majority vote classifier by combining outputs of the base classifiers. Each of the base classifiers is assigned a weight in such a way that a diversity between the base classifiers on the positive samples is minimized.