Visual Feature Augmentation Using Attribute Difference Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional visual data augmentation methods based on interpolation lead to biased sampling in imbalanced class-specific training data, particularly in long-tailed distributions, resulting in poor classifier performance.
Innovation Solution
A method and apparatus for augmenting visual features by calculating a difference vector between a class and an attribute class, embedding these vectors into a text space, projecting the difference vector into a visual space, and using it to augment the visual feature, allowing for semantic perturbation that maintains class labels.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If interpolation-based visual data augmentation is used, then semantic perturbation between classes is achieved, but biased sampling occurs in imbalanced class distributions leading to poor classifier performance
Solution Approach 1:
The patent changes the fundamental parameter of data augmentation from interpolation-based semantic perturbation to attribute-based feature modification. By calculating difference vectors between class embeddings and attribute class embeddings, then projecting these into visual space to modify features, the method achieves unbiased sampling that works effectively across imbalanced and balanced distributions, resolving the contradiction between adaptability and reliability.
2Adaptability or versatility
If interpolation-based augmentation is used, then sample labels are augmented through semantic perturbation, but the method is limited to balanced class distributions with large sample numbers
Solution Approach 1:
The patent extracts the problematic interpolation operation from the data augmentation process and replaces it with a direct attribute-based feature modification approach. By taking out the interpolation step and substituting it with difference vector calculation and projection into visual space, the method eliminates the sampling bias mechanism while maintaining the ability to generate diverse samples, thereby improving adaptability to imbalanced data.
3Reliability
If difference vector projection into visual space is applied, then feature space is densified and classifier performance improves, but additional processing steps are required
Solution Approach 1:
The patent introduces a difference vector as an intermediary element that bridges the gap between class information and visual features. By calculating the difference between class embedding and attribute class embedding, then projecting this intermediary into visual space to modify features, the method achieves feature space densification and improved classifier performance while keeping the process computationally manageable through efficient vector operations.
Data Source
AI summary
There is provided a method for augmenting a visual feature, comprising: extracting a visual feature from an input image; embedding into a text space respectively, a class of the input image and an attribute class formed by reflecting attribute information onto the class; calculating a difference vector between an embedded vector of the class and an embedded vector of the attribute class; and augmenting the visual feature corresponding to the input image based on the difference vector, wherein the apparatus includes: an encoder that extracts the visual feature from the input image; and a predictor that generates predicted class of the input image based on the augmented the visual feature in order to compare whether the predicted class is matched with the class of the input image.