3D Video Conferencing View Adjustment via Gaze Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video conferencing systems lack the ability to provide a realistic three-dimensional representation of remote users, failing to effectively mimic face-to-face interactions by not accounting for gaze direction and head orientation, which are crucial non-verbal cues in personal communication.
Innovation Solution
A system and method for video conferencing that captures and processes stereoscopic image data, including range data, to reconstruct a three-dimensional image of remote users, and dynamically adjusts the view based on their gaze direction and head orientation, using techniques such as time-of-flight or structured light to enhance the perception of depth and realism.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional two-dimensional video conferencing systems are used, then device complexity is low and ease of operation is high, but the sense of presence and realism is insufficient
Solution Approach 1:
The patent transitions from traditional two-dimensional video conferencing to three-dimensional immersive video conferencing by capturing depth information through stereoscopic cameras and range data sensors. This dimensional enhancement creates a more realistic sense of presence while managing system complexity through integrated processing pipelines.
Solution Approach 2:
The system dynamically adjusts the displayed view of remote users based on real-time detection of gaze direction and head orientation. This dynamic adaptation enhances the sense of presence by making the remote user appear to naturally follow the local user's attention, creating a more engaging and realistic interaction experience.
2Adaptability or versatility
If static display orientation is used, then device complexity is low, but the ability to mimic natural face-to-face interaction is poor
Solution Approach 1:
The system implements feedback mechanisms by detecting the local user's gaze direction and head orientation through imaging systems, then using this information to dynamically adjust the displayed orientation of remote users. This closed-loop feedback creates a more natural face-to-face interaction experience where the remote user appears to naturally follow the local user's attention.
Solution Approach 2:
The displayed orientation of remote users is dynamically adjusted based on real-time detection of local user behavior (gaze and head orientation). This dynamic adaptation allows the system to mimic natural face-to-face interactions while managing processing complexity through efficient real-time computation.
3Reliability
If three-dimensional image reconstruction is implemented, then the sense of presence is improved, but the use of energy and computational resources increases
Solution Approach 1:
The system reconstructs three-dimensional images from two-dimensional video feeds by incorporating range data from depth sensors and stereoscopic camera pairs. This dimensional enhancement improves visual realism and sense of presence while managing computational energy through optimized processing pipelines that leverage the additional depth information efficiently.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution provides a more immersive and realistic video conferencing experience by synchronizing the displayed orientation of remote users with their gaze and head movements, thereby mimicking a three-dimensional reality, improving the sense of presence and engagement in virtual communication.
Implementation Method 1
capturing stereoscopic image data
Implementation Method 2
calculating range data by a time-of-flight technique
Implementation Method 3
calculating range data by a triangulation technique, such as, but not limited to, a structured light technique
Implementation Method 4
at least one infrared light source constituted for illuminating at least a portion of the local scene
Data Source
AI summary
A system for video conferencing is disclosed. The system comprises a data processor which receives from a remote location a stream of imagery data of a remote user, and displays an image of the remote user on a display device. The data processor also receives a stream of imagery data of an individual in a local scene in front of the display device, and extracts a gaze direction and/or a head orientation of the individual. The data processor varies a view of the image responsively to the gaze direction and/or the head orientation.