Head Pose And Gaze Estimation Using Synthetic Eye Texture Swaps
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Accurate head pose and gaze estimation in computer vision is challenging due to the independence of head and eye movements, and existing datasets are insufficient for effective training of neural networks, leading to unreliable detection systems.
Innovation Solution
A method involving a first neural network to decouple head and eye movements by mapping 2D face images to a facial position image, constructing a facial texture image, and training a second neural network with transformed 2D face images to estimate head pose and gaze angles, using a database of eye texture images to enhance accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing datasets are used for training neural networks, then the training process can be performed, but the estimation accuracy of head pose and gaze angles is insufficient due to the independence of head and eye movements
Solution Approach 1:
The patent segments the face image processing into distinct components: head pose estimation and gaze angle estimation. By using a 3D morphable face model, the system separately models head orientation parameters and eye gaze parameters, allowing independent optimization of each estimation task while maintaining their anatomical relationships.
Solution Approach 2:
The patent introduces a 3D morphable face model as an intermediary between 2D face images and the estimation parameters. This model serves as a bridge that captures the complex relationships between head pose, eye gaze, and facial appearance, enabling more accurate joint estimation by modeling the underlying 3D geometry and texture variations.
2Device complexity
If head and eye movements are treated as independent, then the estimation complexity is reduced, but the accuracy of simultaneous head pose and gaze estimation deteriorates
Solution Approach 1:
The patent merges head pose estimation and gaze angle estimation into a unified neural network framework. The model jointly processes facial images to predict both head orientation and eye gaze parameters simultaneously, capturing their interdependencies while maintaining a computationally efficient architecture through shared feature extraction layers.
Solution Approach 2:
The patent transforms the estimation problem by changing parameters from independent 2D image features to coupled 3D morphable model parameters. By representing head pose and gaze angles as parameters of a 3D face model, the system captures their natural correlations and anatomical constraints, improving joint estimation accuracy without proportionally increasing complexity.
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
The disclosure relates to technology for head pose and gaze angle estimation that includes using a face alignment method with an image of a two-dimensional (2D) face to generate a facial UV position image and facial UV texture image of the 2D face in which to decouple movement of the head and eyes of the 2D face. An eye UV texture image is extracted from the facial UV texture image and stored in a database. The eye region of the facial UV texture image may then be replaced with the eye UV texture image stored in the database to generate a modified facial UV texture image. The modified facial UV texture image is reconstructed to generate a modified 2D face image as training data stored in the database, and the head pose and gaze angles of a person are simultaneously estimated by training a convolutional neural network with the training data.