Adversarial Image Perturbation for Few-Shot Class Imbalance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning models trained on imbalanced datasets tend to be biased towards classes with more training images, leading to poor performance on classes with only a handful of training images.
Innovation Solution
The method generates additional images for few-shot classes by computing image perturbations using a gradient-ascent-based technique, allowing the model to classify images from few-shot classes as confusing classes, thereby improving generalization performance without requiring additional networks or complex training procedures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If additional networks such as GANs and VAEs are used to generate images for few-shot classes, then the model's performance on few-shot classes is improved, but the device complexity and training time increase significantly
Solution Approach 1:
The patent extracts the image generation functionality from separate complex networks (GANs, VAEs) and integrates it directly into the existing deep neural network classification model. This is achieved by computing gradients with respect to input images and using these gradients to generate perturbed images that are then fed back into the same network for training, eliminating the need for additional generator networks while maintaining performance on few-shot classes
Solution Approach 2:
The patent makes the deep neural network perform multiple functions: both classification and image generation. By computing gradients with respect to input images and using them to generate perturbed images that are then classified by the same network, the model becomes multi-functional, serving as both a classifier and an image generator without requiring separate specialized networks
2Reliability
If additional networks such as GANs and VAEs are used to generate images for few-shot classes, then the model's performance on few-shot classes is improved, but the training time increases to weeks of GPU training time
Solution Approach 1:
The patent merges the image generation process with the classification training process into a single unified workflow. By computing gradients with respect to input images and using them to generate perturbed images that are immediately fed into the same network for classification training, the method combines what were previously separate processes (image generation via GANs/VAEs and classification training) into one efficient training loop, dramatically reducing training time
Solution Approach 2:
The patent enables continuous useful action by using the same deep neural network continuously for both generating perturbed images and training on them. The gradient computation and image perturbation happen in the same training loop as classification, allowing the model to continuously improve on few-shot classes without the interruption and additional time required by separate generator network training
3Reliability
If additional networks such as GANs and VAEs are used to generate images for few-shot classes, then the model's performance on few-shot classes is improved, but the computational resources required increase significantly
Solution Approach 1:
The patent extracts the image generation capability from resource-intensive separate networks (GANs, VAEs) and implements it using only the existing deep neural network's gradient computation mechanism. By taking out the generation function from dedicated generator networks and embedding it in the classification model itself, the method eliminates the redundant computational overhead of maintaining and training separate generator networks
Solution Approach 2:
The patent enables the deep neural network to serve itself by using its own gradient computation mechanism to generate perturbed images for training. The same network that performs classification also generates the training images through gradient-based perturbation, making the system self-sufficient and eliminating the need for external generator networks that consume additional computational resources
Data Source
AI summary
A method of balancing a dataset for a machine learning model includes identifying confusing classes of few-shot classes for a machine learning model during validation. One of the confusing classes and an image from one of the few-shot classes are selected. An image perturbation is computed such that the selected image is classified as the selected confusing class. The selected image is modified with the computed perturbation. The modified selected image is added to a batch for training the machine learning model.