Stereoscopic Video Calling via Planar Proxy Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current artificial reality video calling systems either rely on resource-intensive three-dimensional rendering, which is not feasible for average consumers due to complex hardware requirements, or settle for two-dimensional representations that lack depth perception and immersive experience.
Innovation Solution
The system captures and renders image data from multiple perspectives using techniques such as planar proxies or 3D meshes within a virtual environment, allowing for a three-dimensional video calling experience without the need for full 3D reconstruction, thereby improving image quality and depth perception while reducing power and bandwidth requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If three-dimensional rendering is used to provide immersive video calling experience, then depth perception and image quality are improved, but hardware complexity and power consumption increase significantly
Solution Approach 1:
The patent segments the three-dimensional rendering process into multiple two-dimensional perspective views captured by separate camera sensors. Instead of requiring a single complex 3D rendering pipeline, the system divides the visual information into multiple 2D projections that can be captured independently and then synthesized to create the immersive experience, reducing hardware complexity while maintaining depth perception capabilities
Solution Approach 2:
The patent transitions from traditional three-dimensional spatial rendering to a multi-dimensional approach by capturing multiple two-dimensional perspectives simultaneously. This dimensionality change allows the system to encode depth and spatial information across multiple 2D planes rather than requiring full 3D reconstruction, thereby reducing hardware requirements while preserving immersive visual qualities
2Measurement precision
If three-dimensional rendering is used to provide immersive video calling experience, then depth perception and image quality are improved, but power consumption increases significantly
Solution Approach 1:
The patent segments the computationally intensive three-dimensional rendering task into multiple simpler two-dimensional camera captures. By dividing the rendering workload across multiple independent 2D perspective acquisitions rather than performing full 3D rendering, the system significantly reduces power consumption while maintaining the depth perception and immersive experience through synthesis of the multiple 2D views
Solution Approach 2:
The patent employs partial action by capturing multiple two-dimensional perspectives that provide sufficient depth information without completing a full three-dimensional reconstruction. This partial approach to 3D rendering achieves the necessary depth perception for immersive video calling while avoiding the excessive power consumption associated with complete 3D rendering pipelines
3Measurement precision
If multiple perspective image data is rendered and displayed, then immersive experience and depth perception are improved, but data bandwidth requirements increase
Solution Approach 1:
The patent uses multiple two-dimensional camera sensors to capture perspective views that serve as copies of the scene from different angles. These 2D image copies contain embedded depth and spatial information that can be synthesized to create immersive experiences, reducing the need to transmit large amounts of full 3D data while maintaining depth perception through the use of these lighter-weight 2D image copies
Data Source
AI summary
A method includes receiving video data of a user, the video data comprising a first captured image and a second captured image, generating a two-dimensional planar proxy of the user, determining a pose comprising a location and orientation of the two-dimensional planar proxy within a three-dimensional virtual environment, rendering one or more display images for one or more displays of an artificial-reality device based on the two-dimensional planar proxy having the determined pose and at least one of the first and second captured images, displaying the rendered one or more display images using the one or more displays, respectively, determining that a viewing angle of the artificial-reality device relative to the two-dimensional planar proxy exceeds a predetermined maximum threshold, and based on the determination that the viewing angle exceeds the predetermined maximum threshold, ceasing to display the one or more display images.