Pre-warping Correction Mesh for Eye Motion Distortion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Artificial reality systems face image distortion due to rolling updates and user eye motion, leading to shearing distortions and position shifts, which negatively impact user experience.
Innovation Solution
A correction mesh is generated using eye tracking data to predict eye velocity and apply pre-warping to images, counteracting distortions caused by rolling updates and eye motion, ensuring accurate image projection and reduced distortion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by moving object
If rolling update display is used to reduce power consumption and simplify hardware, then energy efficiency improves, but image distortion and position shifts occur due to eye motion
Solution Approach 1:
The system performs preliminary warping of the image content before display based on predicted eye motion. By anticipating the user's eye movement and pre-adjusting the image geometry, the system compensates for the distortion that would otherwise occur during rolling update, thereby maintaining image accuracy without requiring simultaneous full-frame refresh
Solution Approach 2:
The system uses eye tracking data as feedback to dynamically adjust the warping parameters. By continuously monitoring actual eye position and velocity, the system refines its prediction model and adjusts the pre-warping transformation in real-time, creating a closed-loop control that maintains image accuracy while using power-efficient rolling update
2Manufacturing precision
If full frame refresh is used to eliminate image distortion, then image projection accuracy improves, but power consumption and processing load increase
Solution Approach 1:
Instead of performing costly full-frame refresh operations, the system applies a computationally efficient pre-warping transformation to the image content before rendering. This preliminary geometric adjustment compensates for expected distortion, achieving similar visual accuracy to full refresh but with significantly reduced processing requirements and power consumption
Solution Approach 2:
The system changes the parameter space by working in predicted eye velocity space rather than directly in image pixel space. By transforming the problem into predicting and compensating for eye motion parameters, the system can apply efficient 2D displacement corrections per pixel row rather than performing comprehensive full-frame re-rendering
3Manufacturing precision
If eye tracking system is added to predict eye motion, then image distortion compensation improves, but device complexity increases
Solution Approach 1:
The system introduces an intermediary prediction layer that translates eye tracking data into corrective transformations. Rather than directly using complex eye motion data, the system uses a prediction algorithm as an intermediary to convert eye velocity measurements into simplified 2D displacement parameters that can be efficiently applied during image rendering
Solution Approach 2:
The system performs preliminary conversion of eye tracking data into prediction parameters before the main image rendering process. By pre-computing the relationship between eye velocity and required image displacement, the system reduces the complexity of real-time distortion compensation during frame rendering
Data Source
AI summary
In one embodiment, a computing system may render an image to be output by a display which sequentially outputs pixel rows of the image at a row-to-row velocity. The system may predict, based on eye-tracking data, an eye-motion velocity within a timeframe for displaying the image. The system may determine a predicted retina projection displacement for each pixel row based on the row-to-row velocity and eye-motion velocity. The system may determine a first correction displacement for each pixel row based on the predicted retina projection displacement. The system may determine a cumulative correction displacement based on the first correction displacement. The system may determine, for each pixel row, a second correction displacement based on the first correction displacement of that pixel row and the cumulative correction displacement. The system may warp each pixel row using the associated second correction displacement and output the warped pixel rows using the display.