Tabular Data Augmentation Using PCA-Based Semantic Perturbation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmentation techniques for tabular data, such as adding Gaussian random noise or overwriting with arbitrary sample values, are ineffective due to the significant semantic changes that even small value alterations can cause in tabular data, making it difficult to maintain the original data's class and semantic integrity.
Innovation Solution
A method involving principal component analysis (PCA) or independent component analysis (ICA) to determine correlations between vectors in the dataset, calculating eigenvectors and eigenvalues, and adding perturbations scaled by hyperparameters to generate an augmented dataset that maintains the original dataset's semantic.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If Gaussian random noise or arbitrary sample values are added to tabular data for augmentation, then the data volume increases, but the semantic integrity and class classification of the original data deteriorate
Solution Approach 1:
The patent transforms tabular data into image format, changing the data representation parameters from tabular to spatial. This allows the application of image-based augmentation techniques that preserve semantic relationships while increasing data volume. The transformation maintains the underlying data semantics by representing tabular rows as image rows with consistent spatial patterns.
Solution Approach 2:
The patent introduces an intermediary transformation process that converts tabular data to image format and then back to tabular format after augmentation. This intermediary representation enables the use of effective augmentation techniques while preserving the original data semantics and structure in the final augmented dataset.
2Adaptability or versatility
If small value changes are made to tabular data, then data variation is introduced, but the data may change class or have completely different semantic
Solution Approach 1:
By transforming tabular data to image format, the patent changes the parameter space in which variations occur. Image-based augmentations operate on spatial patterns and visual features rather than direct numerical values, allowing data variation while maintaining class consistency through preserved spatial relationships and patterns.
Solution Approach 2:
The patent adds a spatial dimension to tabular data by representing it as images. This dimensional transformation allows augmentations to operate in the spatial domain rather than the numerical domain, enabling data variation that preserves semantic meaning through maintained spatial patterns and structures.
3Adaptability or versatility
If conventional augmentation techniques are applied to tabular data, then data diversity increases, but the effectiveness of augmentation is reduced
Solution Approach 1:
The patent changes the fundamental parameter representation from tabular to image format, enabling the application of diverse and effective image-based augmentation techniques. This parameter transformation allows the use of proven augmentation methods while maintaining the integrity and effectiveness of the augmented tabular data.
Solution Approach 2:
The intermediary image representation serves as a bridge that enables effective augmentation techniques to be applied to tabular data. This intermediate format allows diverse augmentations to be performed while ensuring the results can be effectively transformed back to maintain augmentation effectiveness in the final tabular dataset.
Data Source
AI summary
There are provided a method and apparatus for augmenting a dataset through the steps of determining a correlation between a plurality of vectors included in the dataset, determining perturbation based on the correlation, and generating an augmented dataset by adding the perturbation to the dataset.