Class-Specific Data Generation for Imbalanced ML Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning models require large amounts of training data, which can be scarce and imbalanced, leading to sampling bias due to under-representation or over-representation of certain classes, particularly in fields like medicine.
Innovation Solution
A system and method for generating class-specific data using a probabilistic generative model that approximates an initial probability distribution, samples from identified portions likely to belong to specific classes, and utilizes uncertainty information to create more data for training, employing a normalizing, flow-based model and a classification model to enhance dataset balance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If raw data is used for training machine learning models, then the model can learn from real-world examples, but the data may be scarce and imbalanced leading to sampling bias
Solution Approach 1:
The patent uses probabilistic generative models to create synthetic copies of training data that replicate the statistical properties and uncertainty characteristics of real data. These synthetic samples are generated by sampling from learned parameter distributions, effectively copying the essential features of real-world data while augmenting the training dataset
Solution Approach 2:
The patent transforms raw data into a different representation space by learning parameter distributions (e.g., Gaussian distributions with learned means and covariances) that characterize each class. By sampling from these parameter distributions, the system generates new data points with controlled parameter variations that maintain class-specific characteristics
2Reliability
If more training data is collected to reduce sampling bias, then class representation improves, but data collection time and resources increase
Solution Approach 1:
The patent performs preliminary learning of class-specific parameter distributions from available raw data before actual data collection or model training. This preliminary action captures the essential statistical characteristics and uncertainty patterns of each class, enabling subsequent synthetic data generation without requiring additional real data collection
Solution Approach 2:
Instead of collecting more real data, the system creates synthetic copies by sampling from learned parameter distributions. This copying approach efficiently generates additional training samples that reflect the underlying data distribution without the time and resource costs of real data collection
3Ease of manufacture
If synthetic data is generated without uncertainty information, then data augmentation is simple, but the generated data lacks realistic variation and reliability
Solution Approach 1:
The patent enhances synthetic data generation by learning and sampling from parameter distributions (means, covariances, and other statistical parameters) rather than using fixed parameters. This approach introduces realistic variation by sampling from distributions that capture the inherent uncertainty and variability of each class, while maintaining computational simplicity through standardized sampling procedures
Data Source
AI summary
A system and related methods for generating class-specific data are disclosed. The data belong to an input space with an unknown initial probability distribution and a known classification scheme. From a relatively small, unbalanced dataset of samples with respect to the classification scheme in the input space, the system is programmed to learn a series of invertible transformations from the input space to a target space, a target probability distribution for the samples in the target space, and a trainable parameter probability distribution for each parameter of the target probability distribution to represent uncertainty information related to the target probability distribution. The system is programmed to further identify how to sample from each parameter probability distribution, which determine how to sample from the target probability distribution, to generate samples in the input space that are more likely to belong to specific classes.