Client-Server Visual Streaming for 3D Scene Latency-Fidelity Tradeoffs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing client-server communication in streaming of volumetric video objects for augmented reality scenarios faces challenges in achieving a balance between low data rate and realistic user experience, particularly due to the need for high processing power and network latency, especially with lightweight clients like smart glasses.
Innovation Solution
A client device integrates visual content into a 3D scene and provides messages to a server with indications about viewpoints, latency, user-object interaction, rendering processes, and scene properties, allowing the server to adapt the visual content transmission accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If rendering is performed on the server-side to reduce client processing requirements, then client device complexity and power consumption are reduced, but network latency and data transmission requirements increase
Solution Approach 1:
The system performs preliminary rendering of virtual objects into 2D image sequences on the server-side before transmission to the client. This preliminary action transfers the computational burden from the client to the server, enabling lightweight clients like smart glasses to function without local rendering capabilities while managing network latency through efficient pre-processing and transmission of ready-to-display image data
Solution Approach 2:
The patent introduces an intermediary representation format (2D image sequences with associated metadata) that bridges the server-side rendering output and client-side display requirements. This intermediary format allows the server to prepare content in advance while providing the client with easily processable data that minimizes latency during actual playback and interaction
2Manufacturing precision
If high-fidelity visual content is transmitted to maintain realistic user experience, then image quality and immersion are improved, but data transmission rate and network bandwidth requirements increase
Solution Approach 1:
The system dynamically changes transmission parameters including resolution, frame rate, and compression level based on network conditions, client capabilities, and scene complexity. This allows the server to maintain high visual fidelity when conditions permit while reducing data transmission requirements when network bandwidth is constrained, optimizing the trade-off between quality and data rate
Solution Approach 2:
The patent applies different quality levels to different regions or elements within the visual content based on their importance to the user experience. Critical elements such as focal points or interactive objects are transmitted with higher fidelity while less important background elements use lower resolution, thereby maintaining realistic user experience while reducing overall data transmission requirements
3Productivity
If client-side metrics reporting is implemented to optimize streaming performance, then service performance evaluation and adaptation are improved, but communication overhead and protocol complexity increase
Solution Approach 1:
The system implements a feedback mechanism where the client reports metrics such as rendering performance, network conditions, and user interaction data back to the server. This feedback enables the server to adaptively optimize content delivery parameters, adjust quality levels, and improve streaming performance while keeping the reporting protocol relatively simple by focusing on essential metrics rather than comprehensive system state
Data Source
AI summary
A concept for transmitting information from a client to a server is described, the client being for streaming visual content from the server and for integrating the visual content into a 3D scene of a user application, and the client being for obtaining a rendered view of the 3D scene.