Image Re-projection for Head-Mounted Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Head-mountable displays (HMDs) face challenges in providing an immersive experience due to latency issues and the need for accurate real-time image rendering and re-projection, especially when the user's head position changes, leading to potential disorientation and nausea.
Innovation Solution
The implementation of a technique that captures and re-projects images using metadata to adjust for differences in the user's head position and orientation, ensuring that the displayed image is rendered correctly in real-time, even when the user's viewpoint changes, by combining high and low field of view images and using depth maps for accurate re-positioning of image features.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time image rendering and re-projection is implemented to maintain accurate image orientation when user head position changes, then user disorientation and nausea are reduced, but system complexity and processing requirements increase
Solution Approach 1:
The system performs preliminary actions by capturing images and generating depth maps in advance, storing them with associated metadata before display is needed. This allows the rendering system to have pre-processed image data ready, reducing the computational burden during real-time display and minimizing latency while maintaining accurate orientation.
Solution Approach 2:
The system segments the image processing task by separating high FOV and low FOV representations, and processing different regions of interest independently. This segmentation allows selective rendering and optimization of specific image portions, reducing overall system complexity while maintaining orientation accuracy through targeted processing.
2Manufacturing precision
If high FOV and low FOV images are combined with depth maps for accurate re-projection, then image rendering precision is improved, but processing time and computational resources increase
Solution Approach 1:
The system generates depth maps and processes high FOV and low FOV image representations in advance, storing them with metadata before display is required. This preliminary processing eliminates the need for complex real-time computation during display, reducing processing time while maintaining high rendering precision through pre-computed depth information.
Solution Approach 2:
The system applies partial processing by focusing computational resources on regions of interest within the image, rather than processing the entire image at full resolution. By combining high FOV and low FOV representations selectively, the system achieves sufficient rendering precision for the displayed region while minimizing overall processing time and computational resource requirements.
3Loss of time
If latency is reduced through optimized image processing, then immersive experience is improved, but image quality or rendering accuracy may deteriorate
Solution Approach 1:
The system performs all necessary image processing, depth map generation, and rendering computations in advance, storing processed images with associated metadata before display is needed. This eliminates real-time processing delays, achieving minimal latency while maintaining full image quality since all processing is completed beforehand without time constraints.
Solution Approach 2:
The system segments image processing into distinct stages (high FOV processing, low FOV processing, depth map generation, compositing) that can be performed independently and in parallel during the preliminary processing phase. This segmentation allows comprehensive image quality processing to be completed offline, with only simple retrieval and display operations occurring in real-time, thus maintaining quality while minimizing latency.
Data Source
AI summary
Aspects of the technology encompass image encoding, which includes generating image content according to a viewpoint defined by image viewpoint data. Successive output images are generated such that each output image includes image content generated according to one or more viewpoints, and encoding metadata associated with each output image which indicates each viewpoint relating to image content contained in that output image, and which defines which portions of that output image were generated according to each of those viewpoints. An image display method for generating successive display images from successive input images is also provided, which includes re-projecting portions of each input image to form a respective display image, according to any differences between a desired display viewpoint and a particular viewpoint defined for that portion by metadata associated with a respective input image. The metadata indicating viewpoints relating to image content contained in the respective input images.