Eye Flow Field Gaze Correction for Dynamic Head Postures
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze correction methods in images are limited to fixed head postures and cannot effectively handle scenarios where head postures change in real time, such as video conferences and video calls.
Innovation Solution
A gaze correction method that involves acquiring an eye image, determining an eye movement flow field based on a target gaze direction, adjusting pixel positions in the eye image using the flow field, and generating a corrected gaze face image, utilizing a trained student gaze correction model through knowledge distillation from teacher models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If gaze correction is performed on fixed head posture images, then gaze correction accuracy is improved, but adaptability to dynamic head postures deteriorates
Solution Approach 1:
The patent applies dynamics by transitioning from static gaze correction to dynamic gaze correction. The system processes video frames sequentially, updating eye gaze estimates and flow fields in real-time as head posture changes. This allows the gaze correction to adapt dynamically to varying head positions while maintaining accuracy through continuous optimization.
Solution Approach 2:
The patent uses preliminary action by pre-training teacher models with diverse head postures and using knowledge distillation to create a student model that can handle various head posture conditions. The system prepares flow fields and gaze corrections in advance for different scenarios, enabling accurate gaze correction without requiring real-time retraining.
2Productivity
If real-time gaze correction is implemented for video conferencing, then communication effectiveness is improved, but computational complexity increases
Solution Approach 1:
The patent applies copying by using knowledge distillation to create a student model that replicates the capabilities of more complex teacher models. The student model learns to generate flow fields and corrected eye images by copying the knowledge patterns from teacher models trained on diverse head postures, enabling real-time processing with reduced computational complexity.
Solution Approach 2:
The patent applies segmentation by dividing the gaze correction process into separate modules: eye gaze estimation, flow field generation, and image correction. Each module can be processed independently and optimized separately, reducing overall computational complexity while maintaining real-time performance for video conferencing applications.
Data Source
AI summary
In the field of artificial intelligence technologies, a gaze correction method and apparatus for a face image, a device, a computer-readable storage medium, and a computer program product are provided. The method includes: acquiring an eye image from a face image; determining an eye movement flow field based on the eye image and a target gaze direction, the target gaze direction being a gaze direction to which an eye gaze in the eye image is to be corrected; adjusting a pixel position in the eye image based on the eye movement flow field, to obtain a corrected eye image; and generating a face image with a corrected gaze based on the corrected eye image.