Gaze-Based Camera Blur Compensation for Mixed Reality Displays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing MR devices suffer from motion blur in captured images due to user gaze and head movement, leading to noisy and blurry visual experiences, with current techniques like short exposure times and optical flow estimation being inadequate.
Innovation Solution
A method and apparatus that utilize gaze-tracking to predict and compensate for motion blur by controlling camera operations before image capture, employing gaze and head tracking data to determine and adjust shutter settings and image sensor movements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If short exposure times are used to capture images during gaze movement, then motion blur is reduced, but the images become noisy due to reduced light capture
Solution Approach 1:
The system performs preliminary gaze tracking and motion prediction before image capture to anticipate the direction and magnitude of gaze movement. This allows the camera to pre-adjust its parameters (such as exposure time, shutter speed, or capture timing) to compensate for expected motion, thereby reducing motion blur without requiring extremely short exposure times that would introduce noise.
2Manufacturing precision
If optical flow estimation is used to reduce motion blur, then some motion compensation is achieved, but inaccuracies and uncertainties in optical flow cause insufficient compensation or introduce larger motion blur
Solution Approach 1:
The patent introduces gaze tracking data as an intermediary between the user's visual system and the camera capture process. Instead of relying solely on post-capture optical flow estimation from image data, the system uses direct gaze measurements (from eye tracking sensors) to predict motion and guide capture timing or camera movement compensation. This intermediary gaze signal provides more reliable and direct motion information than optical flow alone.
3Manufacturing precision
If gaze tracking is implemented to predict motion blur, then accurate compensation is achieved, but device complexity increases
Solution Approach 1:
The patent integrates gaze tracking functionality into the existing mixed reality headset platform, where the same eye tracking sensors and processing infrastructure serve multiple purposes: gaze-based UI interaction, foveated rendering, and motion blur compensation. This multi-functionality approach allows the system to leverage existing hardware and software investments rather than adding dedicated separate systems for each function, thereby reducing the net increase in device complexity.
Data Source
AI summary
Disclosed is a method that includes detecting a beginning of a movement of a user's gaze by processing gaze-tracking data, collected by a gaze-tracking means; predicting a motion blur in an image which is to be captured by at least one camera during the movement of the user's gaze, using a portion of the gaze-tracking data that corresponds to the beginning of the movement of the user's gaze; and compensating for the predicted motion blur while capturing the image by controlling the at least one camera.