MEMS Mirror XR Projection with Coaxial Eye-Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current extended reality (XR) technologies face challenges in enhancing user experience through effective eye-tracking and foveated rendering, as they struggle to accurately detect and track the fovea position in real-time, leading to inefficient dynamic adjustment of image resolution.
Innovation Solution
An image projection system that includes a coaxial scanning system with MEMS mirrors and an eye-tracking sensor, which generates and transmits pixel light pulses and infrared light pulses to project images onto the eye, allowing for the determination of the fovea region and dynamic adjustment of image resolution based on the detected focal direction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If uniform high resolution is applied across the entire projection image, then image quality is improved, but computational resources and processing time are increased
Solution Approach 1:
The patent applies different resolution levels to different regions of the projection image based on the user's gaze position. The fovea region (where the user is looking) receives high resolution rendering, while peripheral regions receive lower resolution rendering. This local differentiation optimizes resource allocation by concentrating computational resources only where the user's visual attention is focused, rather than uniformly applying high resolution across the entire field of view.
2Productivity
If eye-tracking is implemented to enable foveated rendering, then resource allocation is optimized, but system complexity is increased
Solution Approach 1:
The patent combines the eye-tracking sensor, scanning system, and image projection system into an integrated apparatus. The eye-tracking sensor and scanning system share common optical paths and control mechanisms, allowing the same hardware infrastructure to serve multiple functions: projecting the XR image and simultaneously tracking the user's gaze position. This merging reduces overall system complexity compared to having separate independent systems for each function.
3Productivity
If real-time fovea tracking is implemented, then dynamic resolution adjustment is enabled, but measurement precision requirements are increased
Solution Approach 1:
The system implements a feedback loop where the eye-tracking sensor continuously monitors the user's gaze position, and this information is fed back to the image projection system in real-time. Based on this feedback, the system dynamically adjusts the rendering resolution and redirects the projection beam to maintain the focal point at the center of the user's fovea. This closed-loop control enables accurate real-time tracking and dynamic adaptation without requiring excessively high measurement precision throughout the entire field of view.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables real-time foveated rendering by increasing image resolution only where the user is focusing, thereby improving the XR experience by optimizing resource allocation and enhancing visual clarity.
Implementation Method 1
a coaxial scanning system arranged along the transmission path and the reception path, the coaxial scanning system including at least one oscillator structure that enables the coaxial scanning system to steer the pixel light pulses and the IR light pulses
Implementation Method 2
a second transmitter configured to generate infrared (IR) light pulses transmitted along the transmission path and to be projected onto the eye and reflected back therefrom as reflected IR light pulses on a reception path; an eye-tracking sensor configured to receive the reflected IR light pulses from the coaxial scanning system
Data Source
AI summary
An image projection system includes a first transmitter configured transmit pixel light pulses along a transmission path to be projected onto an eye to render a projection image thereon; a second transmitter configured to generate infrared (IR) light pulses transmitted along the transmission path to be projected onto the eye and reflected back therefrom as reflected IR light pulses on a reception path; a coaxial scanning system arranged along the transmission and reception paths; an eye-tracking sensor configured to generate a retina image of the eye based on reflected IR light pulses, and process the retina image to determine a fovea region location of the eye; and a system controller configured to render the projection image based on the fovea region location, wherein the projection image is rendered with a higher resolution in the fovea region and is rendered with a lower resolution outside of the fovea region.