Denoising Autoencoder Image Captioning Data Augmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Image caption generation models require substantial training data, which is often unavailable or prohibitively expensive, especially for uncommon languages, leading to poor performance due to overfitting and underfitting, and increased costs in creating and maintaining caption datasets.
Innovation Solution
The method involves using a denoising autoencoder to generate additional captions from ground truth captions, calculating importance weights using Consensus-based Image Description Evaluation (CIDEr), and training a caption model with these new captions to reduce training requirements and improve performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If substantial training data is used for image caption generation models, then model accuracy and precision improve, but data collection and maintenance costs increase
Solution Approach 1:
The patent creates synthetic training data by copying and transforming existing ground truth captions through a denoising autoencoder framework. The system generates multiple noisy versions of original captions, encodes them, and decodes to create new training examples, thereby multiplying the utility of limited original data without incurring additional data collection costs
Solution Approach 2:
The patent applies parameter changes by introducing controlled noise parameters to the training process. The denoising autoencoder learns to reconstruct clean captions from noisy inputs by adjusting weights and biases, effectively transforming the training paradigm from clean-to-clean to noisy-to-clean reconstruction, which improves model robustness and accuracy
2Reliability
If more training data is collected for uncommon languages, then model performance improves, but resource requirements increase
Solution Approach 1:
For uncommon languages with limited available data, the system creates synthetic training examples by copying and transforming existing captions. The denoising autoencoder generates multiple variations of limited ground truth captions, effectively amplifying the training dataset size without requiring additional manual annotation resources
Solution Approach 2:
The system performs self-service data augmentation by automatically generating its own training data from existing ground truth captions. The denoising autoencoder framework enables the model to create its own training examples through encoded-decoded reconstruction, reducing dependency on external data sources and manual annotation efforts
3Measurement precision
If ground truth captions are used for training, then model accuracy improves, but data maintenance costs increase
Solution Approach 1:
The patent creates multiple synthetic training samples by copying and transforming ground truth captions through noise addition and denoising reconstruction. Each original caption generates multiple training examples with varying noise levels, multiplying the training value of each ground truth annotation without requiring additional maintenance
Solution Approach 2:
The system performs preliminary data processing by pre-noising and pre-encoding ground truth captions to create a robust training set before actual model training. This preliminary action transforms the ground truth data into a more resilient training format that requires less ongoing maintenance while maintaining high accuracy standards
Data Source
AI summary
In an approach to augmenting a caption dataset by leveraging a denoising autoencoder to sample and generate additional captions from the ground truth captions, one or more computer processors generate a plurality of new captions utilizing an autoencoder fed with one or more noisy captions, wherein the autoencoder is trained with a dataset comprising a plurality of ground truth captions. The one or more computer processors calculate an importance weight for each new caption in the plurality of generated new captions as compared to a plurality of associated ground truth captions based on a consensus metric. The one or more computer processors train a caption model with the generated plurality of new captions and associated calculated weights.