HMD Frame-Shift Latency Reduction via Sensor Timing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional head-mounted displays (HMDs) experience significant motion-to-motion latency, exceeding 50 ms, leading to user discomfort, simulator sickness, and degraded response, as they struggle to efficiently reflect user motion on the screen.
Innovation Solution
The HMD incorporates a sensor system to measure motion at two time points, using first sensing information to generate image data and second sensing information to frame-shift the data just before output, incorporating a display device with image processing units and a timing controller to adjust the data application signal, thereby reducing latency by shifting the image data on the display panel.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional HMD processes and displays image data through complete generation and rendering cycles, then image quality and completeness are maintained, but motion-to-motion latency increases to 50 ms or more causing simulator sickness
Solution Approach 1:
The patent applies preliminary action by measuring sensor information at a second time point (immediately before output) and using this data to frame-shift the image data that was generated at an earlier first time point. This allows the system to prepare and adjust the image data in advance based on the most recent motion state, reducing the latency between user motion and screen reflection without compromising image quality.
2Loss of time
If HMD uses frame-shifting technology to reduce latency, then motion-to-motion latency decreases to around 16.7 ms, but device complexity increases due to additional image processing units and timing controllers
Solution Approach 1:
The patent segments the image processing function into distinct modules: a first image processing unit that removes distortion based on lens correction information, a frame shifting unit that shifts the image data based on pixel shift information calculated from sensor data, and a second image processing unit that applies distortion. This segmentation allows each module to perform its specific function efficiently, reducing overall processing latency while maintaining manageable system complexity.
3Productivity
If HMD processes a large number of image data frames to maintain smooth display, then display smoothness is improved, but motion-to-motion latency increases exceeding 50 ms
Solution Approach 1:
The patent introduces pixel shift information as an intermediary element that bridges the gap between sensor measurements and image display. By calculating pixel shift information from sensor data and using it to frame-shift the image data, the system can adjust the displayed image to reflect current motion state without waiting for complete re-rendering of image frames. This intermediary mechanism enables smooth display adaptation while maintaining low latency.
Data Source
AI summary
A head mounted display (HMD) may include: a sensor configured to measure a motion of the HMD; a controller configured to generate image data based on first sensing information measured by the sensing unit at a first time point; and a display device configured to frame-shift the generated image data, and output the frame-shifted image data to a display panel. The display device may frame-shift the image data using second sensing information measured by the sensor at a second time point after the first time point, and the second time point may correspond to a time point immediately before the controller outputs the image data to the display panel after generating the image data.