Online Meeting Eye Contact Imaging with 3D Gaze Correction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional online video meeting systems fail to maintain effective eye contact among participants due to the misalignment of camera and display locations, leading to unnatural gaze shifts and impaired communication.
Innovation Solution
Employ multiple cameras to capture images from different angles, construct a three-dimensional model of the participant's face, determine gaze direction, and generate a two-dimensional eye contact image perpendicular to the gaze direction for real-time rendering on a display, ensuring aligned gaze with remote participants.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single camera and display are used at each location, then the system is simple and cost-effective, but eye contact cannot be maintained because the camera and display are not co-located
Solution Approach 1:
The patent transitions from a 2D display plane to a 3D spatial model by capturing images from multiple cameras at different angles and constructing a three-dimensional model of the participant's face. This enables determination of gaze direction in three-dimensional space, allowing the system to generate a corrected 2D image that appears to look at the remote participant, thus resolving the non-co-location issue between camera and display.
Solution Approach 2:
The patent introduces a three-dimensional model as an intermediary between the captured images and the final displayed image. This intermediate representation allows the system to calculate gaze direction and generate a corrected image that maintains eye contact, acting as a mediator that transforms the raw camera input into an eye-contact-preserving output.
2Reliability
If the participant looks directly at the camera when speaking, then the captured image shows proper eye contact, but the participant cannot simultaneously view the remote participants on the display
Solution Approach 1:
The patent separates the functions of capturing the participant's image and displaying remote participants into different spatial locations. The participant views the display showing remote participants, while a separate camera system captures the participant's face from multiple angles to determine gaze direction and generate a corrected image that appears to maintain eye contact with remote participants.
Solution Approach 2:
The patent creates a corrected copy of the participant's face image that is synthesized to appear as if the participant is looking at the remote participants. This virtual copy is generated from multiple camera views and three-dimensional modeling, allowing the remote participants to see an image that maintains eye contact while the local participant continues to view the display naturally.
3Measurement precision
If multiple cameras are used to capture images from different angles, then a three-dimensional model can be constructed to determine gaze direction, but the device complexity and processing requirements increase
Solution Approach 1:
The patent uses multiple cameras positioned at different angles to capture images of the participant's face, then constructs a three-dimensional model from these 2D images. This transition to 3D space enables precise determination of gaze direction by analyzing the spatial relationships between facial features in three dimensions, achieving high measurement precision through dimensional transformation.
Data Source
AI summary
Methods and systems for establishing and maintaining eye contact during an online video meeting are described herein. Multiple cameras capture images of a meeting participant from a number of different angles. A three dimensional model is determined based on the collected images. The three dimensional model is evaluated to determine the gaze direction of the participant. A two dimensional eye contact image of the participant is generated from the three dimensional model such that the plane of the two dimensional image is perpendicular to the gaze direction of the participant. This two dimensional eye contact image is then rendered on a display viewed by a remotely located meeting participant. Eye contact images of each participant are generated as described herein and presented to the remotely located meeting participant in real time. As a result, eye contact between both participants is established and maintained during the video meeting.