Immersive Video Conference System Using Virtual Viewpoint Synthesis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Remote video conferences lack the immersive and interactive elements of face-to-face meetings, such as eye contact and natural interaction, which hinders efficient communication.
Innovation Solution
The system determines a conference mode for the video conference, which defines the layout of a virtual conference space. It then uses viewpoint information to generate a first view of one participant based on the perspective of another participant, and sends this view to the participant's conference device to display a conference image.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional video conference layouts are used, then device complexity is reduced, but immersion and authenticity of interaction deteriorate
Solution Approach 1:
The system creates a virtual copy of the physical conference space with virtual avatars representing each participant. These virtual copies replicate the spatial relationships and interactions of the real meeting, allowing participants to experience face-to-face interaction dynamics in a virtual environment. The virtual conference space includes virtual cameras positioned to capture views as if from actual camera locations, preserving natural interaction patterns.
Solution Approach 2:
The system transitions from traditional 2D video feeds to a 3D virtual conference space where participants interact in a spatial environment. Virtual cameras are positioned at specific locations in the virtual space to capture views from multiple angles, creating a three-dimensional experience that adds depth and spatial context to the interaction, enabling more natural eye contact and head movement dynamics.
2Adaptability or versatility
If fixed camera views are used, then ease of operation is improved, but flexibility of conference space layout deteriorates
Solution Approach 1:
The system dynamically adjusts camera views and virtual camera positions based on the determined conference mode and participant actions. Rather than fixed static views, the virtual cameras can be repositioned within the virtual conference space to optimize viewing angles and capture relevant interactions. The system automatically adapts the layout according to the number of participants, conference type, and spatial relationships.
Solution Approach 2:
The system continuously monitors participant positions, orientations, and interactions in the virtual conference space, and automatically adjusts camera views and virtual camera positions in response. This feedback mechanism ensures that the most relevant views are captured and displayed, maintaining natural interaction dynamics while adapting to changing conference conditions without requiring manual intervention.
3Measurement precision
If multiple virtual cameras are deployed, then measurement precision of participant views is improved, but device complexity increases
Solution Approach 1:
The system uses a single physical camera to capture the participant's view, then synthesizes multiple virtual camera perspectives through computational processing. The virtual cameras are software constructs that can be positioned anywhere in the virtual conference space, eliminating the need for multiple physical cameras. This multi-functional approach allows one physical device to provide the functionality of multiple virtual cameras.
Solution Approach 2:
The system replaces the mechanical system of multiple physical cameras with a computational approach using virtual cameras. Instead of deploying multiple physical devices to capture different viewpoints, the system uses image processing and virtual reality techniques to synthesize multiple perspectives from a single camera feed, substituting mechanical complexity with computational processing.
Data Source
AI summary
According to implementations of the subject matter described herein, there is provided a solution for an immersive video conference. In the solution, a conference mode for the video conference is determined at first, the conference mode indicating a layout of a virtual conference space for the video conference, and viewpoint information associated with the second participant in the video conference is determined based on the layout. Furthermore, a first view of the first participant is determined based on the viewpoint information and then sent to a conference device associated with the second participant to display a conference image to the second participant. Thereby, on the one hand, it is possible to enable the video conference participants to obtain a more authentic and immersive video conference experience, and on the other hand, to obtain a desired virtual conference space layout according to needs more flexibly.