HMD Image Delivery via Remote Graphics Offloading
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current head-mounted displays for virtual reality require substantial hardware, software, and bandwidth to maintain a seamless viewing experience, particularly due to distortion issues when the user moves their head, as not all pixels are displayed simultaneously, leading to disorienting changes in the virtual environment.
Innovation Solution
A head-mounted display system with remote graphics processing and computational resources, utilizing a low-latency communication bus to offload significant image processing and rendering, allowing for direct display of the field of view without local processing, and pre-storing image data outside the current field of view for immediate delivery when the user's gaze shifts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If image processing and rendering are performed locally on the HMD, then the viewing experience is seamless with no latency, but the hardware requirements and device complexity increase substantially
Solution Approach 1:
The patent extracts the complex image processing and rendering functions from the HMD device itself and relocates them to remote servers. The HMD only retains basic display capabilities and receives pre-processed image data through wireless communication, thereby reducing device complexity while maintaining viewing experience quality.
Solution Approach 2:
The patent introduces an intermediary system (remote servers connected via network) between the content source and the HMD display. This intermediary performs the computationally intensive image processing and rendering tasks, allowing the HMD to function with minimal local hardware while still delivering high-quality visual output.
2Reliability
If all pixels are displayed simultaneously to avoid distortion, then the image quality is maintained, but the bandwidth requirements increase substantially
Solution Approach 1:
The patent applies preliminary action by pre-processing and preparing image data on remote servers before transmission to the HMD. The system anticipates display needs and prepares appropriate image resolutions and formats in advance, reducing the bandwidth required for real-time transmission while maintaining image quality.
Solution Approach 2:
The patent implements local quality by transmitting different resolutions and quality levels for different regions of the display field. Areas requiring higher detail receive higher resolution data, while peripheral areas receive lower resolution data, optimizing bandwidth utilization while maintaining overall image quality where needed.
3Loss of time
If the HMD processes image data locally, then the latency is minimized, but the software requirements and computational resources increase substantially
Solution Approach 1:
The patent extracts complex software processing requirements from the HMD and relocates them to remote servers. The HMD software is simplified to handle only basic display functions and data reception, while sophisticated image processing, rendering, and optimization algorithms run remotely, reducing both latency and local software complexity.
Solution Approach 2:
The patent replaces the mechanical/computational processing system within the HMD with a network-based distributed computing system. Instead of local computational resources handling all processing tasks, the system substitutes remote computational power delivered through communication protocols, reducing local hardware and software requirements.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A head-mounted display image delivery system in combination with a head- mounted display (HMD) device, the image delivery system and the HMD device are local to one another: the HMD device comprising: one or more displays; one or more sensors to determine movement, position and attitude of the display; and one or more image inputs configured to receive image data for direct display on the one or more displays without image processing by the HMD from the image delivery system, the image delivery system being configured to communicate a stereoscopic image to the HMD system to convey image information to a wearer, wherein the image delivery system comprises: one or more inputs to receive sensor data from the sensors; one or more image inputs configured to receive pre-processed image data for direct display of the field of view on the one or more displays without image processing by the image delivery system; one or more image outputs to deliver image data for direct display on the one or more displays of the field of view without image processing by the HMD system.