Eye Tracking Warping for Laser Scanning Display Artefacts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Laser beam scanning display technologies in mixed reality systems face challenges in maintaining image quality when a user changes their gaze, leading to visual artefacts such as slew and misstitch.
Innovation Solution
The method involves using eye tracking data to transform image frames based on the position of a viewer's pupils, shifting pixel sets by calculated shift values to ensure accurate alignment with the user's gaze, thereby eliminating artefacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If laser beam scanning display is used to present virtual image information, then immersive mixed reality experience is achieved, but visual artefacts such as slew and misstitch appear when user gaze changes
Solution Approach 1:
The system performs preliminary warping of the image frame based on predicted future eye position before the actual display occurs. By anticipating where the user's eyes will be when different portions of the display are scanned, the system pre-adjusts the image geometry to compensate for expected gaze movements, thereby preventing slew and misstitch artefacts from appearing in the final displayed image
Solution Approach 2:
The system continuously monitors actual eye position using eye tracking sensors and uses this feedback to dynamically adjust and refine the warping parameters. By comparing predicted eye position with actual measured position, the system can correct deviations and optimize the warping transformation in real-time, improving image alignment accuracy during gaze changes
2Manufacturing precision
If eye tracking is used to adjust image frame based on pupil position, then image quality is improved, but computational complexity increases
Solution Approach 1:
The system divides the image frame into multiple regions corresponding to different scan areas, and applies differential warping parameters to each region based on predicted eye position. This segmentation allows the complex warping operation to be broken down into manageable regional transformations, reducing overall computational complexity while maintaining image quality
Solution Approach 2:
The system dynamically changes warping parameters based on eye tracking data, adjusting the transformation matrix according to predicted pupil position. By parameterizing the warping operation and pre-calculating transformation matrices for different gaze positions, the system reduces real-time computational burden while maintaining high image quality
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments transform an image frame based on a position of pupils of a viewer to eliminate visual artefacts formed on an image frame displayed on a scanning-type display device. An MR system obtains a first image frame corresponding to a first view perspective associated with a first pupil position. The system receives data from an eye tracking device, determines a second pupil position, and generates a second image frame corresponding to a second view perspective associated with the second pupil position. A first set of pixels of the second image frame are shifted by a first shift value, and a second set of pixels of the second image frame are shifted by a second shift value, where the shift values are calculated based on at least the second pupil position. The system transmits the second image frame to a near-eye display device to be displayed thereon.