Telepresence System Using Camera Array for Photorealistic Avatars
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current video conferencing solutions fail to effectively replicate the realism, directionality, and social cues of in-person communication, leading to negative impacts on mental and emotional health due to lack of immersion and realism.
Innovation Solution
A remote telepresence communication system using a camera array to capture a user's light field from multiple views, relighting users in a shared environment, and employing social view synthesis to create realistic user representations at specified distances and perspectives, ensuring directionality and social context.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional video conferencing is used, then communication can be conducted remotely, but realism and social cues are lost
Solution Approach 1:
The system creates photorealistic 3D avatars that copy and replicate the visual appearance, expressions, and characteristics of real users. These digital twins are generated from photographs and video feeds, allowing remote participants to see themselves and others as if physically present, thereby maintaining realism while enabling remote communication.
Solution Approach 2:
The patent transitions from 2D video feeds to 3D photorealistic avatars, adding spatial depth and dimensional realism to virtual meetings. This dimensional enhancement allows users to perceive depth, perspective, and spatial relationships, recreating the immersive experience of in-person communication while maintaining remote accessibility.
2Reliability
If video conferencing emulates in-person communication, then social cues can be preserved, but immersion and directionality are reduced
Solution Approach 1:
The system implements view-dependent rendering that adapts the visual presentation based on each user's specific location and orientation in the virtual space. Each participant sees avatars and environmental elements from their unique perspective, preserving directional information and spatial context that flat video feeds cannot convey, thereby maintaining both social cues and immersive directionality.
3Reliability
If multiple views of user are captured, then realistic representation is achieved, but device complexity increases
Solution Approach 1:
Instead of requiring complex multi-camera setups at each location, the system captures user imagery once using a camera array, then creates photorealistic 3D avatar copies that can be viewed from multiple angles. This single capture process generates reusable digital twins that provide realistic multi-view representation without requiring ongoing complex hardware at each user site.
Data Source
AI summary
Methods, apparatus, systems, and articles of manufacture are disclosed to provide remote telepresence communication. At least one non-transitory machine-readable medium comprises instructions that, when executed, cause a processor to identify features from a plurality of images, the plurality of images representing a first user and a second user, create a first representation of the first user, a second representation of the second user, the representations created using the plurality of images, the representations representing the respective users at specified distances and specified perspectives from a viewer, and construct a first image, the first image including the second model at a specified location within a shared environment, the first image to be presented on a first display.