Hybrid Cloud Local Rendering for VR Latency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Virtual reality and augmented reality applications face challenges in balancing graphical realism with responsiveness due to high computational and bandwidth demands, often resulting in rendering lag and user discomfort like motion sickness.
Innovation Solution
A hybrid rendering approach where remote servers prioritize and partially render a virtual environment based on components' complexity, proximity, and motion, transmitting the rendered parts to client devices for compositing with locally rendered elements to achieve low latency and high responsiveness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If cloud gaming executes video game application on remote servers and transmits video stream to client device, then high-end hardware requirements are reduced for client device, but network bandwidth consumption increases and latency is introduced
Solution Approach 1:
The patent segments the rendering workload by dividing the virtual environment into multiple components (foreground objects, background objects, static elements, dynamic elements) and selectively rendering only priority components on remote servers while leaving lower-priority components to be rendered locally by the client device. This segmentation reduces the amount of data that needs to be transmitted over the network, thereby reducing bandwidth consumption while still providing cloud gaming functionality.
2Manufacturing precision
If remote servers render complete virtual environment for VR/AR applications, then graphical realism is improved, but rendering latency increases causing user discomfort
Solution Approach 1:
The patent applies local quality by assigning different rendering priorities to different spatial regions and object types within the virtual environment. High-priority components such as foreground objects and dynamic elements that directly impact user experience and realism are rendered on remote servers with high graphical fidelity, while lower-priority background and static elements are rendered locally. This ensures that graphical realism is maintained for critical visual elements while reducing overall rendering latency.
Solution Approach 2:
Instead of rendering the complete virtual environment on remote servers, the patent implements partial rendering by selectively processing only the most visually critical components (foreground objects, dynamic elements, objects near the user's viewpoint). This partial action approach achieves sufficient graphical realism for the most important visual elements while dramatically reducing the computational load and rendering time required, thereby minimizing latency.
3Device complexity
If all virtual environment components are rendered by remote servers, then centralised control is simplified, but network transmission data volume increases causing bandwidth bottlenecks
Solution Approach 1:
The patent segments the virtual environment into multiple components with different rendering priorities (foreground, background, static, dynamic objects). Remote servers are responsible for rendering only high-priority components, while low-priority components are handled by the client device. This segmentation significantly reduces the volume of rendered data that needs to be transmitted over the network, alleviating bandwidth bottlenecks while maintaining simplified centralized control for the critical visual elements.
4Manufacturing precision
If high computational power is allocated to remote rendering servers, then rendering quality is improved, but responsiveness to user input deteriorates due to network latency
Solution Approach 1:
The patent implements partial rendering where remote servers with high computational power focus exclusively on rendering the most visually critical components (foreground objects, dynamic elements, objects in the user's immediate viewpoint) that require high rendering quality. The client device handles lower-priority rendering tasks locally, which are less sensitive to quality variations. This approach ensures that high rendering quality is maintained for elements that matter most to user experience while improving overall responsiveness by reducing network dependency for critical visual updates.
Data Source
AI summary
Information is received by one or more servers regarding a plurality of components of a virtual environment to be presented for display by a remote client device. A set of remote rendering prioritization values, indicating a degree of prioritization for rendering individual components of the plurality of components by the one or more server computing devices, is determined based on the received information. Based on the set of remote rendering prioritization values, a partial scene of the virtual environment is generated at the one or more servers by rendering a set of first components of the plurality of components. The generated partial scene of the virtual environment is transmitted to the remote client device for compositing with a set of second components of the plurality of components to be rendered by the remote client device.