Gaze Estimation DNN Generalization via Augmented Images
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing appearance-based gaze estimation technologies using deep neural networks (DNNs) face challenges in generalizing gaze estimation across new portrait images and varying environments, leading to overfitting and the need for extensive relearning processes.
Innovation Solution
The proposed method enhances generalization performance by creating multiple augmented images from an original image, using a DNN to output gaze estimation values, and calculating a total loss through gaze consistency regularization (GCR) using a spherical gaze distance (SGD), which updates the DNN parameters to improve domain generalization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a DNN is trained on appearance-based gaze estimation data, then gaze estimation can be performed without calibration, but the model overfits and fails to generalize to new portrait images and environments
Solution Approach 1:
The patent applies preliminary action by creating multiple augmented images from a single original image before training the DNN. These augmented images (with varying transformations) are used to pre-train the model, enabling it to handle diverse portrait images and environments without requiring calibration or extensive relearning when deployed to new domains.
Solution Approach 2:
The patent changes the parameter of using multiple augmented images instead of a single original image for training. This parameter change enables the DNN to learn robust features that generalize across different domains while maintaining the ease of operation benefit of calibration-free gaze estimation.
2Adaptability or versatility
If a DNN is trained on extensive learning data to improve generalization, then domain adaptation improves, but the cost of building and relearning increases
Solution Approach 1:
The patent uses copying by creating multiple augmented images from a single original image. These augmented images serve as copies that capture diverse variations of the same portrait, allowing the model to learn generalization capabilities without requiring extensive diverse training data. This significantly reduces the quantity of learning data needed while maintaining high domain generalization performance.
Solution Approach 2:
The patent achieves universality by training the DNN on augmented images that represent multiple domains and conditions within a single training process. The model learns universal gaze estimation capabilities that work across different portrait styles, lighting conditions, and environments without requiring separate training datasets for each domain, thereby reducing overall data requirements.
3Adaptability or versatility
If the DNN is trained on diverse portrait images to improve generalization, then domain adaptability improves, but the model becomes overly complex and difficult to train
Solution Approach 1:
The patent applies segmentation by dividing the training process into two distinct stages: a pre-training stage using augmented images from a single original image to learn generalization features, and a fine-tuning stage using actual portrait images for domain-specific optimization. This segmentation reduces overall training complexity by breaking down the complex task of training on diverse portraits into manageable phases, while still achieving high domain adaptability.
Data Source
AI summary
Disclosed herein are a deep neural network (DNN) learning method for generalizing appearance-based gaze estimation and an apparatus for the same. The deep neural network (DNN) learning method includes creating multiple augmented images based on an original image, inputting the multiple augmented images to a DNN to output a gaze estimation value, calculating a total loss between a gaze ground truth of the original image and the gaze estimation value through gaze consistency regularization (GCR) using a spherical gaze distance (SGD), and updating parameters of the DNN by backpropagation of the total loss.