Mobile Gaze Correction via Eye Region Extraction and 3D Model Transformation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional methods for correcting the gaze difference between a camera and a display unit in video calls require sophisticated face outline separation technology and two cameras, making real-time video calls on mobile devices challenging due to increased computation time and complexity.
Innovation Solution
A method that extracts the eye outline using Active Shape Model (ASM) for reduced computation and applies a 2D or 3D eye model to transform and insert a virtual eye, allowing for gaze correction on mobile devices with simplified processing and natural background integration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional face outline separation technology is used to correct gaze direction, then gaze correction accuracy is improved, but computation time and device complexity increase significantly
Solution Approach 1:
The patent extracts only the essential eye region features (eye outer points, eye shape parameters) needed for gaze correction, rather than performing complete face outline separation. This selective extraction of critical information maintains gaze correction functionality while dramatically reducing computation time and complexity for mobile devices.
Solution Approach 2:
The patent segments the face processing task into two parts: (1) simple eye region detection using eye outer points, and (2) gaze correction based on eye shape parameters. This segmentation allows the system to focus computational resources only on the eye region, avoiding the need for complex full-face outline separation while achieving effective gaze correction.
2Measurement precision
If two cameras are installed to capture multiple angles for gaze correction, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent uses a single camera to capture the user's face, then creates virtual multiple-viewpoint information through image processing and 3D modeling techniques. By generating synthetic eye shapes and gaze vectors from the single camera input, the system achieves gaze correction functionality equivalent to multi-camera systems without the hardware complexity.
Solution Approach 2:
The patent introduces intermediate processing steps (eye outer point detection, eye shape parameter extraction, 3D eye model generation) that mediate between the single camera input and the final gaze correction output. These intermediary computations enable the system to derive accurate gaze direction information from a single camera view, eliminating the need for multiple physical cameras.
3Manufacturing precision
If sophisticated face outline separation is performed in real-time, then gaze correction quality is improved, but processing speed decreases
Solution Approach 1:
The patent employs computationally inexpensive eye detection methods (detecting eye outer points and basic eye shape parameters) that can be executed rapidly in real-time. Rather than using expensive, time-consuming full-face outline separation algorithms, the system uses simpler, faster eye-specific detection techniques that maintain sufficient accuracy for gaze correction while enabling real-time processing on mobile devices.
Data Source
AI summary
A method and mobile terminal for correcting a gaze of a user in an image includes setting eye outer points that define an eye region of the user in an original image, transforming the set eye outer points to a predetermined reference camera gaze direction, and transforming the eye region of the original image based on the transformed eye outer points.