Virtual Camera Pose Determiner for Natural Eye Contact
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video communication systems fail to provide natural eye contact due to the fixed orientation of virtual cameras, leading to an unnatural appearance of participants when they are not directly in front of the camera.
Innovation Solution
A virtual camera pose determiner is used to calculate the position and orientation of a virtual camera based on the display position of a remote participant and the local participant's position, ensuring that the virtual view is rendered as if viewed from the display position, incorporating 3D analysis and eye tracking for precise rendering of critical face regions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a virtual camera is placed at the position where users are looking at on the display, then eye contact is provided, but the local participant appears static or unnatural when not directly in front of the camera
Solution Approach 1:
The virtual camera's position and orientation are made dynamic rather than fixed. The camera pose is continuously adjusted based on the local participant's head position and gaze direction, allowing the system to adapt to different viewing scenarios and maintain natural appearance while preserving eye contact
Solution Approach 2:
The system changes the parameters of the virtual camera (position and orientation) based on detected head pose and gaze direction. By dynamically modifying these camera parameters, the system resolves the contradiction between maintaining eye contact and ensuring natural appearance across different participant positions
2Ease of manufacture
If a fixed orientation virtual camera is used, then the system is simple to implement, but the local participant appears unnatural when not directly in front of the camera
Solution Approach 1:
The system transitions from a static fixed-orientation camera to a dynamic camera system that adjusts its pose based on real-time head tracking data. This dynamic adaptation enables the system to handle various participant positions naturally while maintaining implementation feasibility through algorithmic control
3Stability of the object's composition
If the virtual camera always looks straight in a fixed direction, then the camera orientation is constant and simple, but the rendered image does not match the user's actual line of sight
Solution Approach 1:
The camera orientation parameters are dynamically adjusted to match the user's actual line of sight by incorporating head pose estimation and gaze direction detection. This ensures the virtual camera's viewing direction precisely aligns with where the user is actually looking, improving measurement precision while maintaining orientation stability through controlled parameter adjustment
Data Source
AI summary
A virtual camera pose determiner is configured to determine a position and an orientation of a virtual camera. The position of the virtual camera is determined on the basis of a display position of a displayed representation of a remote participant on a display. The orientation of the virtual camera is determined on the basis of a geometrical relation between the display position of the remote participant on the display, and a position of a local participant. The virtual camera is configured to transmit an image or a sequence of images to the remote participant, so that an image provided by the virtual camera has the view on the local participant as is if viewed from the display position. Further embodiments provide a video communication system having a virtual camera pose determiner for providing a virtual camera pose on basis of the display position and the position of the local participant.