Eye Tracking Dynamic Render Time for HMD Visual Artifacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
VR systems using the rolling band display driving technique suffer from unwanted visual artifacts like judder near the top and bottom of the display, caused by inaccuracies in head pose prediction and re-projection adjustments, leading to ghosting effects of moving objects.
Innovation Solution
The system dynamically targets a render time for each frame based on eye tracking data to minimize re-projection adjustments, ensuring accurate image rendering at the location where the user is looking, thereby reducing visual artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a single target render time is chosen for every frame based on middle row illumination, then the total amount of re-projection adjustments is minimized, but unwanted judder artifacts appear near the top and bottom of the display where users are looking
Solution Approach 1:
The patent applies different target render times to different regions of the display based on where the user is looking. Instead of using a single global target render time for the entire frame, the system divides the display into multiple regions (e.g., top, middle, bottom) and selects the target render time corresponding to the illumination time of the region where the user's gaze is detected, thereby optimizing image quality locally where it matters most.
Solution Approach 2:
The system uses eye tracking data as feedback to dynamically adjust the target render time for each frame. By continuously monitoring where the user is looking and using this information to select the appropriate target render time, the system adapts to the user's viewing behavior in real-time, minimizing re-projection artifacts in the region of interest.
2Measurement precision
If re-projection adjustments are applied to correct head pose inaccuracies, then image alignment is improved, but visual artifacts like ghosting and judder are introduced near the display edges
Solution Approach 1:
The patent applies re-projection adjustments selectively based on the user's gaze location. By identifying the region where the user is looking and using the target render time specific to that region, the system minimizes re-projection adjustments in the critical viewing area while still maintaining proper alignment, thereby reducing ghosting and judder artifacts where they would be most noticeable.
Solution Approach 2:
The system performs preliminary selection of the target render time based on predicted eye gaze location before applying re-projection adjustments. By anticipating where the user will be looking and pre-selecting the appropriate target render time, the system prepares the optimal rendering parameters in advance, reducing the need for corrective re-projection adjustments that would introduce artifacts.
3Use of energy by moving object
If rolling band display driving technique is used to illuminate rows of pixels in sequence, then power consumption is reduced, but judder artifacts appear at the top and bottom of the display
Solution Approach 1:
The patent maintains the power-efficient rolling band display driving technique but compensates for its artifact-producing effects by dynamically selecting the target render time based on eye tracking data. This allows the system to keep the low-power rolling illumination scheme while minimizing re-projection artifacts in the region where the user is actually looking, effectively decoupling the power consumption benefit from the artifact problem.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A head-mounted display (HMD) with a rolling illumination display panel can dynamically target a render time for a given frame based on eye tracking. Using this approach, re-projection adjustments are minimized at the location of the display(s) where the user is looking, which mitigates unwanted, re-projection-based visual artifacts in that "region of interest." For example, logic of the HMD may predict a location on the display panel where a user will be looking during an illumination time period for a given frame, determine a time, within that illumination time period, at which an individual subset of the pixels that corresponds to the predicted location will be illuminated, predict a pose that the HMD will be in at the determined time, and send pose data indicative of this predicted pose to an application for rendering the frame.