Active Learning Data Selection for Imbalanced Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing active learning methods for imbalanced classification problems are inefficient as they often select unrepresentative data points, leading to biased model training and poor performance in classifying minority classes, especially when dealing with skewed datasets.
Innovation Solution
A method that scores unlabeled data samples using a combination of uncertainty, diversity, and class imbalance scores, selecting the most informative samples for labeling and iteratively augmenting the training dataset to improve model performance across all classes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing active learning methods select data points for labeling, then the training dataset is augmented, but the selected data points are unrepresentative leading to sampling bias
Solution Approach 1:
The patent changes the selection criteria parameters from simple uncertainty-based scoring to a composite scoring system that incorporates uncertainty, diversity, and class imbalance parameters. This multi-parameter approach ensures selected data points are representative across all dimensions, resolving the sampling bias problem while maintaining reliability.
Solution Approach 2:
The patent creates a composite selection criterion by combining multiple scoring components (uncertainty score, diversity score, class imbalance score) into a unified selection mechanism. This composite approach ensures that selected data points satisfy multiple quality requirements simultaneously, eliminating sampling bias while preserving representativeness.
2Productivity
If existing active learning methods select data points using simple criteria, then the selection process is computationally efficient, but the model training becomes biased
Solution Approach 1:
The patent transforms the data selection process by introducing multiple scoring parameters (uncertainty, diversity, class imbalance) that can be computed efficiently using the trained model. These parameter-based scores enable reliable model training while maintaining computational efficiency through vectorized operations and iterative refinement.
Solution Approach 2:
The patent replaces complex manual or heuristic data selection mechanisms with an automated scoring system based on model predictions and statistical measures. This substitution enables efficient computation of multiple criteria simultaneously, ensuring both productivity and training quality without requiring manual intervention.
3Reliability
If more data points are selected for labeling, then the model performance improves, but the cost and time for labeling increases
Solution Approach 1:
The patent optimizes the number of data points to be labeled by using class imbalance scores that identify the minimum necessary samples from minority classes. This parameter-driven approach ensures model performance improves with the smallest possible labeled dataset, reducing labeling time and costs while maintaining high reliability.
Solution Approach 2:
The active learning system automatically identifies and selects the most informative data points without manual intervention, using the composite scoring mechanism to determine optimal sample selection. This self-service approach minimizes human time investment in the labeling process while maximizing model performance gains.
4Device complexity
If existing methods use uncertainty-based selection only, then the selection process is simple, but minority classes are poorly classified
Solution Approach 1:
The patent combines uncertainty-based selection with diversity and class imbalance criteria in a composite scoring system. This composite approach maintains the simplicity of uncertainty-based methods while adding necessary components to improve minority class classification, achieving both low complexity and high reliability.
Solution Approach 2:
The patent applies different selection criteria with varying weights depending on the local characteristics of data points. For minority class samples, class imbalance scores are emphasized, while for majority classes, uncertainty and diversity scores dominate. This local adaptation ensures optimal classification performance across all classes without excessive overall complexity.
Data Source
AI summary
Using a first dataset of labeled data, a model is trained by adjusting a feature extractor parameter, a classifier parameter, and a discriminator parameter of the model. Using the discriminator parameter and a parametric function of the feature extractor parameter, a plurality of samples of a dataset of unlabeled data is scored. A subset of the scored plurality of samples is selected for labeling. Responsive to receiving a label of each of the selected subset of the scored plurality of samples, the first dataset of labeled data is augmented with the selected subset of the scored plurality of samples and the label of each of the selected subset of the scored plurality of samples. Using the augmented dataset of labeled data, the model is retrained. The retraining comprises further adjusting the feature extractor parameter, the classifier parameter, and the discriminator parameter of the model.