Adaptive Sampling for Imbalanced Dataset Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning pipelines face challenges in scalability and efficiency, particularly with large and imbalanced datasets, leading to prolonged training and validation processes and biased model predictions.
Innovation Solution
An adaptive data-sampling technique that iteratively subsamples datasets using a greedy approach, balancing between model performance and resource efficiency by adjusting sample sizes based on validation scores and customizable stopping criteria.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If under-sampling strategy is applied to decrease dataset size, then resource requirements are reduced, but sampling accuracy and model performance may deteriorate
Solution Approach 1:
The patent implements dynamic adaptive sampling that adjusts sampling rates based on class distribution and model performance metrics. The system continuously monitors validation scores and modifies sampling strategies iteratively, transitioning from static under-sampling to a dynamic process that adapts to dataset characteristics and model needs, thereby maintaining accuracy while reducing dataset size.
Solution Approach 2:
The system changes multiple parameters simultaneously including sampling rates for different classes, dataset composition ratios, and model hyperparameters. By coordinating changes in these parameters through iterative optimization, the system achieves both dataset size reduction and performance maintenance, resolving the contradiction between quantity reduction and quality preservation.
2Reliability
If larger datasets are used to improve model performance, then prediction accuracy increases, but training and validation time increase
Solution Approach 1:
The patent applies partial action by using adaptive sampling to select only the necessary portion of data required for achieving good model performance. Instead of using the complete dataset, the system identifies and uses representative subsets that provide sufficient training signal, thereby reducing training time while maintaining acceptable performance levels through optimized sampling strategies.
Solution Approach 2:
The system performs preliminary adaptive sampling and validation to identify optimal dataset configurations before full model training. By pre-processing and characterizing the data distribution, the system determines efficient sampling parameters in advance, enabling faster subsequent training while preserving performance, thus resolving the time-performance tradeoff.
3Productivity
If imbalanced datasets are used as-is, then data efficiency is improved, but model bias toward over-represented classes increases
Solution Approach 1:
The patent applies local quality by implementing differential sampling rates for different classes based on their specific characteristics and representation in the dataset. Instead of uniform processing, the system tailors sampling strategies to each class's needs, applying higher sampling rates to under-represented classes and lower rates to over-represented classes, thereby maintaining data efficiency while correcting model bias through localized quality adjustments.
Data Source
AI summary
According to an embodiment, a method includes generating a first dataset sample from a dataset, calculating a first validation score for the first dataset sample and a machine learning model, and determining whether a difference in validation score between the first validation score and a second validation score satisfies a first criteria. If the difference in validation score does not satisfy the first criteria, the method includes generating a second dataset sample from the dataset. If the difference in validation score does satisfy the first criteria, the method includes updating a convergence value and determining whether the updated convergence value satisfies a second criteria. If the updated convergence value satisfies the second criteria, the method includes returning the first dataset sample. If the updated convergence value does not satisfy the second criteria, the method includes generating the second dataset sample from the dataset.