Weighted Ensemble Voting for Imbalanced Binary Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning methods struggle to effectively handle imbalanced datasets, particularly in applications like fraud detection, medical diagnosis, and predictive maintenance, where the number of positive examples is significantly smaller than negative examples, leading to high false positive rates.
Innovation Solution
An ensemble learning method that involves generatively oversampling minority class examples using kNN and multivariate probability distributions, learning base classifiers on subsamples, and combining them with a diversity-aware weighted majority vote classifier to minimize diversity on positive samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If standard classification methods are used on imbalanced datasets, then the model can be trained quickly and simply, but the false positive rate increases and classification precision deteriorates
Solution Approach 1:
The patent segments the imbalanced dataset into multiple balanced subsamples through stratified sampling, where each subsample contains equal representation of minority and majority classes. This segmentation allows the training process to focus on balanced class distributions across multiple subsets, improving precision while maintaining reliability through the collective judgment of multiple classifiers.
Solution Approach 2:
The patent combines multiple base classifiers into an ensemble system that aggregates their predictions. By merging the results of classifiers trained on different balanced subsamples, the system achieves both high precision (through balanced training) and high reliability (through ensemble voting that reduces false positives), resolving the contradiction between these two metrics.
2Quantity of substance
If oversampling techniques are applied to balance the dataset, then the minority class representation improves, but the computational complexity and training time increase
Solution Approach 1:
Instead of oversampling the entire dataset which would duplicate data and increase computational load, the patent segments the data into multiple balanced subsamples. Each subsample contains a manageable number of synthetic minority class samples generated through SMOTE, distributed across different folds. This segmentation maintains adequate minority class representation while controlling computational complexity through efficient use of available data.
Solution Approach 2:
The patent performs preliminary stratified sampling and synthetic sample generation before the actual classification training. By pre-balancing the dataset into k subsamples with equal class distribution, the computational overhead is incurred once during data preparation, rather than during each training iteration. This preliminary action reduces the computational burden during the actual model training and evaluation phases.
3Adaptability or versatility
If multiple base classifiers are trained on different subsamples, then the ensemble diversity increases and generalization improves, but the training time and computational resources increase
Solution Approach 1:
The patent divides the balanced dataset into k stratified subsamples, where each subsample maintains the same class distribution. Multiple base classifiers are trained on these segmented subsamples, creating diversity in the ensemble. The segmentation ensures that each classifier receives balanced data while the variation across subsamples promotes ensemble diversity, achieving both generalization improvement and controlled training time through parallelizable independent training.
Solution Approach 2:
The patent trains multiple base classifiers on different subsamples rather than training a single classifier on the entire dataset. This partial action approach distributes the computational workload across multiple smaller training tasks that can be executed in parallel. The excessive action of training more classifiers than strictly necessary is justified by the diversity gain, where each additional classifier contributes to reducing false positives through the voting mechanism.
Data Source
AI summary
An ensemble learning based method is for a binary classification on an imbalanced dataset. The imbalanced dataset has a minority class comprising positive samples and a majority class comprising negative samples. The method includes: generatively oversampling the imbalanced dataset by synthetically generating minority class examples, thereby generating a generated dataset; using the generated dataset to generate subsamples, and learning a base classifier on each of the subsamples to determine a plurality of base classifiers; and learning a weighted majority vote classifier by combining outputs of the base classifiers. Each of the base classifiers is assigned a weight in such a way that a diversity between the base classifiers on the positive samples is minimized.