Remote Display Compositing for Wireless VR Latency Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional virtual reality systems require wired connections and frequent frame re-composition, which can break immersion and pose safety risks due to cable management issues, and are inefficient with limited bandwidth wireless connections.
Innovation Solution
A method for compositing frames on a display control device using sensor feedback, where sections of image data are received and combined based on 'camera' location and angle, allowing for efficient compression and independent editing of layers, reducing the need for frequent re-rendering and data transmission.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a wired connection is used between the virtual reality headset and the host, then data transmission reliability is improved, but user safety and immersion are worsened due to cable management issues and potential dangers
Solution Approach 1:
The patent extracts the frame composition function from the host device and relocates it to the display control device (headset). This allows the headset to operate more independently, reducing the critical need for a stable wired connection while maintaining data transmission reliability for essential data only.
Solution Approach 2:
The patent segments the image data into multiple layers (depth layers) that can be transmitted separately and recombined at the display control device. This segmentation allows for more efficient wireless transmission of compressed layer data rather than complete frames, reducing bandwidth requirements.
2Manufacturing precision
If the host re-composes each frame every time the view is refreshed, then image quality is improved, but bandwidth efficiency is worsened when using a wireless connection with limited bandwidth
Solution Approach 1:
The patent applies preliminary action by having the display control device prepare and store multiple depth layers in advance. When the view needs to be refreshed, the device can quickly composite new frames by combining these pre-prepared layers with sensor data, avoiding the need for complete frame re-rendering at the host.
Solution Approach 2:
The patent applies local quality by transmitting and processing only the necessary layer data rather than complete frames. Different layers can be compressed at different quality levels based on their importance and the user's current viewing direction, optimizing bandwidth usage while maintaining perceived image quality.
3Manufacturing precision
If complete frames are transmitted to the display device, then image quality is improved, but data transmission efficiency is worsened
Solution Approach 1:
The patent segments complete frames into multiple independent depth layers that can be transmitted separately. Each layer represents a specific depth range in the scene and can be compressed independently, allowing for more efficient bandwidth utilization while maintaining the ability to reconstruct high-quality images.
Solution Approach 2:
The patent applies partial action by transmitting only the essential layer data needed for frame composition rather than complete rendered frames. The display control device performs the additional composition work locally using sensor feedback, reducing the data transmission burden while maintaining image quality.
Data Source
Figure 1
Figure 2
Figure 3A~3D
AI summary
A method for compositing display data at a remote device to form an image for display involves the remote device receiving (S62) elements of the image, where the image includes display data forming a background layer, display elements forming a foreground layers and an overlay data layer. The remote device receives (S63) sensor information indicating one or more of a position of an eye of a viewer, a direction of focus of the eye of the viewer, and/or a position of a display. It then determines (S64) movement of a line of sight between the eye of the viewer and the display, determines an estimate of a future position of the line of sight at a future time based on the determined movement of the line of sight, and determines (S65) an adjustment to be made to the image based on the future position of the line of sight. The display elements of the foreground layer are composited (S66) relative to the display data of the background layer according to the determined adjustment and the composited image is forwarded (S67) for display at the future time.