Client-Server Feedback for Volumetric AR Content Streaming
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing client-side rendering of volumetric video objects in augmented reality (AR) devices requires significant processing power, leading to power consumption and bulkiness, while server-side rendering needs improved feedback mechanisms for optimizing data transmission and user experience.
Innovation Solution
A client device streams visual content to a server, providing indications about viewpoints, latency, user interactions, rendering processes, and 3D scene properties to adapt the content delivery, allowing the server to optimize data transmission and quality based on client capabilities and user behavior.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If client-side rendering is used for volumetric video objects in AR devices, then processing capability and realism are improved, but power consumption and device bulkiness increase
Solution Approach 1:
The rendering process is segmented between server and client: the server performs complex volumetric rendering and transmits only essential data (point clouds, camera parameters, timing information), while the client performs lightweight integration with real-world camera feeds. This division allows high rendering quality without requiring full client-side rendering power.
Solution Approach 2:
The patent introduces an intermediary data format that bridges server-side volumetric rendering and client-side AR integration. The server renders volumetric content offline and transmits compressed representations, which the client then integrates with real-time camera captures. This intermediary approach avoids transmitting full-resolution volumetric data while maintaining visual fidelity.
2Weight of moving object
If server-side rendering is used for volumetric video objects, then device portability is improved, but feedback mechanisms for optimizing data transmission are insufficient
Solution Approach 1:
The patent implements a feedback mechanism where the client sends timing information, camera parameters, and synchronization data back to the server. This feedback loop allows the server to optimize future renderings based on actual client performance, network conditions, and user behavior, ensuring efficient data transmission while maintaining portability.
Solution Approach 2:
The server performs preliminary rendering of volumetric content offline before transmission. By pre-rendering and compressing volumetric video objects into optimized data formats, the server reduces the computational burden on portable devices while ensuring high-quality visual output when needed.
3Reliability
If high-quality volumetric video streaming is provided, then user experience is improved, but data transmission requirements increase
Solution Approach 1:
Instead of transmitting original high-resolution volumetric video data, the patent transmits compressed representations and essential parameters (point cloud data, camera intrinsics, timing information). The client reconstructs the visual experience by integrating this compressed data with real-time camera feeds, significantly reducing transmission requirements while maintaining user experience quality.
Solution Approach 2:
The patent applies local quality optimization by transmitting only the most critical data elements at high fidelity (camera parameters, timing synchronization) while using compressed representations for volumetric content. This selective quality approach ensures reliable user experience without requiring proportionally high data transmission volumes.
Data Source
AI summary
A concept for transmitting information from a client to a server is described, the client being for streaming visual content from the server and for integrating the visual content into a 3D scene of a user application, and the client being for obtaining a rendered view of the 3D scene.