AEGAN Architecture for Synthetic Gaze Data Generation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze prediction models using convolutional neural networks face challenges with accuracy due to the need for a large and diverse set of training images, often resulting in overfitting when the image set is not comprehensive, leading to instability and low quality in synthetic data generation by generative models.
Innovation Solution
An autoencoding generative adversarial network (AEGAN) is employed, which includes an autoencoder network and a generative adversarial network sharing a generator, trained with specific loss terms to stabilize and improve the generation of high-quality, diverse synthetic training data by mapping real data to a latent space and iteratively updating the data distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large and diverse set of training images is used to train the convolutional neural network for gaze prediction, then the accuracy of gaze prediction is improved, but the complexity of data collection and processing increases
Solution Approach 1:
The patent uses a generative adversarial network to create synthetic copies of training images. The GAN generates artificial eye images that mimic real eye appearances, providing diverse training data without requiring actual photographing of eyes in various conditions. This copying approach resolves the contradiction by supplying abundant training data (improving accuracy) through computational generation rather than physical data collection (reducing complexity).
Solution Approach 2:
The patent introduces an intermediary generative model that acts as a bridge between limited real eye images and the diverse training data needed for accurate gaze prediction. This intermediary GAN system translates simple input images into complex, diverse training examples, thereby improving prediction accuracy while avoiding the complexity of direct large-scale data collection.
2Device complexity
If a limited set of training images is used, then the complexity of data collection is reduced, but overfitting occurs and generalization performance deteriorates
Solution Approach 1:
The GAN generates synthetic copies of training images to augment the limited real data. By creating artificial variations of the available eye images, the system maintains low data collection complexity while significantly improving model generalization performance through exposure to diverse synthetic examples during training.
Solution Approach 2:
The patent applies parameter changes by transforming the limited training images through the GAN's latent space manipulation. The generator modifies image parameters (lighting, angle, expression, background) to create diverse variations, enabling the model to generalize better without collecting more real data, thus maintaining low complexity while improving reliability.
3Adaptability or versatility
If generative models are used to create synthetic training data, then data diversity is improved, but the quality and stability of synthetic data generation deteriorates due to training instability
Solution Approach 1:
The patent merges the autoencoder and GAN into a unified AEGAN architecture where both components share the same generator. This merging allows the autoencoder's reconstruction capability to stabilize the GAN's generation process, maintaining data diversity while improving training stability and synthetic data quality through the combined objectives of both networks.
Solution Approach 2:
The AEGAN implements feedback mechanisms where the discriminator provides gradient feedback to guide the generator toward producing more realistic and stable synthetic images. The autoencoder's reconstruction loss also provides feedback to constrain the generator, ensuring that synthetic data maintains both diversity and stability throughout training.
Data Source
AI summary
Techniques for using a deep generative model to generate synthetic data sets that can be used to boost the performance of a discriminative model are described. In an example, an autoencoding generative adversarial network (AEGAN) is trained to generate the synthetic data sets. The AEGAN includes an autoencoding network and a generative adversarial network (GAN) that share a generator. The generator learns how to the generate synthetic data sets based on a data distribution from a latent space. Upon training the AEGAN, the generator generates the synthetic data sets. In turn, the synthetic data sets are used to train a predictive model, such as a convolutional neural network for gaze prediction.