Cloud VR Display Latency Reduction via Predictive Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud VR solutions experience excessively high display latency due to the serial processing of logical operations and image rendering on cloud devices, which negatively impacts user experience by prolonging the time between posture and location changes of VR devices and the display of corresponding images.
Innovation Solution
The proposed method involves a terminal device sending posture and location information to a cloud device, which renders a field of view image and sends it back, allowing the terminal device to convert and display the image within its updated field of view range based on posture and location changes, thereby reducing latency by parallelizing image rendering and display processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If cloud device performs logical operation and image rendering sequentially, then processing completeness is ensured, but display latency increases
Solution Approach 1:
The cloud device performs image rendering in advance based on predicted user posture and location changes, before the actual display moment arrives. This preliminary rendering action reduces the time gap between user movement and image display, lowering display latency while ensuring processing completeness through predictive pre-computation
Solution Approach 2:
The rendering process is segmented into multiple sub-tasks that can be executed in parallel: logical operation processing, image rendering, and depth information generation are separated into independent modules. This segmentation allows concurrent execution of multiple processing streams, reducing overall processing time while maintaining complete processing of all required functions
2Manufacturing precision
If cloud device renders high-resolution field of view images, then image quality improves, but processing time and bandwidth consumption increase
Solution Approach 1:
Instead of uniformly rendering the entire field of view at high resolution, the system applies different quality levels to different regions: the central vision area (fovea) is rendered at high resolution while peripheral areas are rendered at lower resolution. This local quality differentiation maintains perceived image quality while significantly reducing processing time and bandwidth consumption
Solution Approach 2:
The system renders only the necessary portion of the field of view at full quality based on the user's current gaze direction and head pose, rather than rendering the entire scene at maximum resolution. This partial action approach focuses computational resources on the visually critical regions, achieving acceptable overall image quality with reduced processing time
3Adaptability or versatility
If terminal device performs image conversion and display processing, then display flexibility increases, but device complexity increases
Solution Approach 1:
Complex processing functions such as image rendering, depth map generation, and field of view conversion are extracted from the terminal device and relocated to the cloud device. The terminal device retains only lightweight functions for receiving processed images and performing simple display adjustments, thereby reducing its processing complexity while maintaining display flexibility through cloud-based adaptability
Data Source
AI summary
A terminal device sends first information to a cloud device, where the first information indicates a posture and a location of the terminal device at a first moment. Then, the terminal device receives information about a first field of view image from the cloud device, where the first field of view image is a field of view image corresponding to the posture and the location of the terminal device at the first moment. The terminal device displays an image within a field of view range of the terminal device at a second moment based on the information about the first field of view image and a posture change and a location change of the terminal device from the first moment to the second moment.