Gaze-Based Camera Focusing Using Depth Maps to Prevent Flicker
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing camera systems for mixed reality applications suffer from latency and visual flickering due to the need for trial and error in selecting the correct focus distance, which is inefficient and provides an unpleasant viewing experience.
Innovation Solution
A computer-implemented method and system for gaze and depth-based camera focusing that uses gaze-tracking to estimate optical depths and convergence distances, allowing for the accurate and efficient selection of a focus distance using a variable-focus camera.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If trial and error method is used to select focus distance by trying multiple focus distances, then the correct focus distance can be found, but latency and visual flickering occur due to multiple frames required
Solution Approach 1:
The system performs preliminary actions by using gaze tracking to predict the region of interest before actual focusing is needed, and by pre-calculating depth information from depth maps. This allows the focus distance to be determined in advance based on where the user is looking, rather than waiting to find it through trial and error after the fact.
Solution Approach 2:
The patent replaces the mechanical trial-and-error focusing system with an information-based system using gaze tracking and depth map analysis. Instead of mechanically adjusting focus through multiple attempts, the system uses computational methods to directly calculate the optimal focus distance based on eye gaze direction and depth information from the depth map.
2Measurement precision
If trial and error method is used to select focus distance, then the correct focus distance can be found, but visual flickering occurs during the focusing process
Solution Approach 1:
The system determines the optimal focus distance in advance using gaze tracking and depth map analysis before the actual image capture occurs. This preliminary determination eliminates the need for multiple focus adjustments that cause visual flickering, as the focus is set once based on predicted user attention.
Solution Approach 2:
The system uses real-time feedback from gaze tracking to continuously monitor where the user is looking and adjusts the focus distance accordingly. The depth map provides feedback about scene geometry, allowing the system to maintain stable focus by constantly aligning with the user's visual attention without requiring multiple trial adjustments.
3Measurement precision
If manual region-of-interest selection is required by tapping on display screen, then precise region can be selected, but ease of operation deteriorates
Solution Approach 1:
The system performs region selection automatically using gaze tracking technology. Instead of requiring manual user input through tapping, the system detects the user's eye gaze direction and automatically determines the region of interest, making the operation transparent and convenient while maintaining precise region selection.
Solution Approach 2:
The patent replaces the manual mechanical interaction (tapping on screen) with an automatic physiological measurement system (gaze tracking). The eye gaze direction serves as natural input, eliminating the need for manual tapping operations while precisely identifying the region of interest through computational analysis of gaze data and depth maps.
Data Source
Figure 1
Figure 2
Figure 3~4
AI summary
Disclosed is computer-implemented method comprising marching first ray (200, 300, 400) and second ray (202, 302, 402), along first gaze direction and second gaze direction that are estimated using gaze-tracking means (506, 606), from given viewpoint (204, 304, 404) into depth map (206, 306, 406), to determine first optical depth and second optical depth corresponding to first eye and second eye, respectively; calculating gaze convergence distance, based on first gaze direction and second gaze direction; detecting whether first optical depth lies within predefined threshold percent from second optical depth; and when it is detected that first optical depth lies within predefined threshold percent from second optical depth, selecting given focus distance as an average of at least two of: first optical depth, second optical depth, gaze convergence distance; and employing given focus distance for capturing given image using at least one variable-focus camera (502).