HMD Reprojection Unit and Combining Unit for Accurate UI Positioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In head-mounted displays with head tracking for virtual reality, there is a delay between the generation and display of video, causing 'VR sickness' and inconvenience due to the reprojection process, which affects the positioning of superimposed images and user interfaces.
Innovation Solution
An image generation technique that applies a reprojection process to images rendered by another device and combines a superimposed image not subject to reprojection with the reprojected image, ensuring accurate positioning and reducing user discomfort.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a reprojection process is applied to correct the image based on orientation difference, then the accuracy of the virtual reality video display is improved, but superimposed images such as sight marks and user interfaces are incorrectly moved
Solution Approach 1:
The patent divides the image processing into two separate segments: the reprojection process handles the virtual reality video content, while the superimposed images are processed and positioned separately. This segmentation allows each element to be handled according to its specific requirements, preventing the mixing of transformation matrices that would otherwise cause incorrect positioning of UI elements.
Solution Approach 2:
The patent introduces an intermediary approach where the superimposed images are positioned using a separate transformation matrix that is not affected by the reprojection process. This intermediary positioning system acts as a mediator between the reprojected video content and the final display, ensuring that UI elements maintain their correct positions regardless of the reprojection transformations applied to the video.
2Adaptability or versatility
If head tracking function is used to generate virtual reality video by changing viewpoint and gaze direction, then the sense of immersion is improved, but delay between head motion and video display causes VR sickness
Solution Approach 1:
The patent applies reprojection process that calculates and corrects the image based on the difference between orientation data at generation time and orientation data at display time. This preliminary correction action compensates for the delay inherent in the rendering pipeline, predicting and adjusting the image to match the user's actual head orientation when the video is finally displayed.
Solution Approach 2:
The system implements a feedback loop where orientation data is continuously collected from the head tracking function, and this feedback is used to drive the reprojection process. The reprojection unit constantly adjusts the displayed image based on the difference between assumed and actual head orientations, creating a closed-loop system that responds to user motion in real-time despite the inherent processing delays.
Data Source
AI summary
A reprojection unit (60) of a head-mounted display (100) receives an original image rendered by a rendering apparatus (200) and applies a reprojection process to the original image. A combining unit (66) combines a superimposed image that is not subject to the reprojection process with an image after the reprojection process by the reprojection unit (60) to generate a combined image. The head-mounted display (100) displays the combined image.