Combined Feature Embeddings for Minority Class Upsampling

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional machine learning classification systems face inaccuracies and inefficiencies due to class imbalance in training data, leading to over-fitting and misclassification issues, especially when dealing with sparse training samples, and they are inflexible in handling limited training resources.

Innovation Solution

The system generates combined feature embeddings for minority class upsampling by merging training samples from majority and minority classes using a sampled combination ratio, creating synthetic training samples that maintain the original class distribution, allowing for efficient and accurate model training even with sparse data.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If conventional systems oversample minority class training data, then the quantity of minority class samples increases, but the deviation from minority class distribution increases and accuracy decreases

Engineering Contradiction:

Improvequantity of minority class samplesVSAvoidaccuracy of minority class modeling

Core Design Contradiction:

Quantity of substanceVSManufacturing precision

Solution Approach 1:

The patent combines feature embeddings from majority class samples and minority class samples to generate synthetic minority class samples. This merging approach allows increasing the quantity of minority class samples while maintaining the underlying class distribution characteristics, thereby improving model accuracy for minority class detection without the deviation problems caused by simple oversampling.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent transforms training samples into feature embeddings and uses interpolation in the embedding space to generate synthetic samples. By changing the parameter space from raw data to feature embeddings, the system can generate diverse synthetic minority class samples that maintain distributional properties, thus increasing sample quantity while preserving modeling accuracy.

Inventive Principle:

Principle #35Parameter changes

2Quantity of substance

If conventional systems use Generative Adversarial Networks to generate synthetic training data, then the quantity of training samples increases, but computational resource consumption increases significantly

Engineering Contradiction:

Improvequantity of training samplesVSAvoidcomputational resource consumption

Core Design Contradiction:

Quantity of substanceVSUse of energy by moving object

Solution Approach 1:

Instead of using complex Generative Adversarial Networks, the patent creates synthetic minority class samples by copying and interpolating feature embeddings from existing majority class samples. This simpler copying approach in the embedding space achieves sample augmentation with significantly reduced computational resource consumption while maintaining sample diversity and distributional properties.

Inventive Principle:

Principle #26Copying

3Productivity

If conventional systems train machine learning models with imbalanced training data, then the training process is simple, but the model accuracy for minority class classification deteriorates

Engineering Contradiction:

Improvetraining efficiencyVSAvoidminority class classification accuracy

Core Design Contradiction:

ProductivityVSManufacturing precision

Solution Approach 1:

The patent performs preliminary action by generating synthetic minority class samples through feature embedding interpolation before the actual model training process. This pre-augmentation of minority class data ensures that the training dataset has balanced representation, allowing the model to learn minority class patterns effectively without compromising training efficiency, as the synthetic samples are generated quickly in the embedding space.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11631029B2Generating combined feature embedding for minority class upsampling in training machine learning models with imbalanced samples

Publication Date: 2023.04.18 ADOBE INC

AI summary

Systems, methods, and non-transitory computer-readable media are disclosed for generating combined feature embeddings for minority class upsampling in training machine learning models with imbalanced training samples. For example, the disclosed systems can select training sample values from a set of training samples and a combination ratio value from a continuous probability distribution. Additionally, the disclosed systems can generate a combined synthetic training sample value by modifying the selected training sample values using the combination ratio value and combining the modified training sample values. Moreover, the disclosed systems can generate a combined synthetic ground truth label based on the combination ratio value. In addition, the disclosed systems can utilize the combined synthetic training sample value and the combined synthetic ground truth label to generate a combined synthetic training sample and utilize the combined synthetic training sample to train a machine learning model.