Multi-Camera Terminal Layout for Realistic Multi-User Telepresence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing virtual face-to-face communication technologies lack the realism in representing multiple users in a virtual space, particularly when using large displays.
Innovation Solution
A terminal apparatus equipped with an overhead camera and multiple in-screen cameras, along with a controller, generates precise 2D or 3D model images of users based on their positions, enabling improved representation and realism in virtual face-to-face calls.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single camera is used to capture images for virtual face-to-face calls, then the device complexity is low, but the measurement precision of user position and orientation is insufficient
Solution Approach 1:
The camera system is segmented into multiple specialized cameras: a first imager positioned above the display for capturing user position and orientation, and multiple second imagers positioned behind the display for capturing detailed user images from different angles. This segmentation allows each camera to specialize in specific measurement tasks, improving overall measurement precision while maintaining manageable device complexity through functional division.
Solution Approach 2:
The system transitions from a single-camera 2D perspective to a multi-camera 3D spatial arrangement. The first imager positioned in the vertical dimension above the display captures overhead views for position and orientation, while second imagers positioned behind the display capture frontal views. This dimensional expansion enables precise three-dimensional localization and orientation detection of users in the virtual space.
2Adaptability or versatility
If multiple users are represented on a large display, then the communication capacity is improved, but the realism of individual user representation deteriorates
Solution Approach 1:
The display surface is segmented into multiple independent display regions, each dedicated to representing a specific user. The controller assigns each detected user to a separate display region and selects appropriate second imagers to capture images for that user's model. This segmentation ensures that even when multiple users are present, each user receives dedicated imaging resources and display space, maintaining individual representation quality and realism.
Solution Approach 2:
Different regions of the display are assigned different local qualities and functions based on user positions. Each display region is optimized for its specific user with dedicated image capture from the nearest second imager. This local quality approach ensures that each user experiences high-fidelity representation in their own region, while the overall system supports multiple users simultaneously on the large display.
3Reliability
If 3D model images are generated for all users, then the realism is improved, but the processing complexity and time increase
Solution Approach 1:
The system dynamically adjusts the type of model image generated (2D or 3D) based on real-time conditions such as user position, distance from the display, and communication context. The controller selectively generates 3D models when they are most beneficial for realism while using 2D models for other cases. This dynamic approach optimizes the balance between realism and processing time, generating computationally intensive 3D models only when necessary rather than for all users continuously.
Data Source
AI summary
A terminal apparatus includes a display configured to be capable of displaying an image to a user in front, a first imager provided around the display, a plurality of second imagers provided behind the display, a communication interface, and a controller configured to communicate by the communication interface, wherein the controller is configured to transmit, to another terminal apparatus, information for generating a model image representing the user, based on a captured image by a second imager that corresponds to the position of the user included in a captured image by the first imager, so that the other terminal apparatus displays the model image.