Stereo Matching Using Gaze Coordinates for AR Depth Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current stereo matching methods in augmented and virtual reality environments face challenges in accuracy and speed, particularly when dealing with environments containing repeated patterns or subjects at close distances, leading to increased processing time and errors.
Innovation Solution
An image processing device and method that utilizes gaze coordinates to rapidly and accurately perform stereo matching by extracting feature points and generating gaze coordinate information, restricting the search range for corresponding feature points based on gaze information, and employing this information to enhance the accuracy and speed of stereo matching.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional stereo matching methods are used, then stereo matching can be performed, but processing time increases and accuracy decreases in environments with repeated patterns or close subjects
Solution Approach 1:
The patent applies preliminary action by accumulating gaze coordinates over time before performing stereo matching. The system collects and stores gaze point information from the eye-tracking sensor during a predetermined time period, then uses this pre-accumulated data to define a restricted search range for feature point matching. This preliminary accumulation of gaze data enables faster and more accurate stereo matching by providing prior information about the user's visual attention, thereby resolving the contradiction between accuracy and processing time.
2Reliability
If the search range for feature points is not restricted, then all possible matches are considered, but processing time increases significantly
Solution Approach 1:
The patent applies local quality by using gaze information to create a localized restricted search range around the gaze point coordinates. Instead of performing stereo matching across the entire image, the system defines a specific region of interest based on accumulated gaze data and limits the feature point search to this local area. This localized approach maintains matching reliability within the relevant region while dramatically improving processing speed by eliminating unnecessary comparisons in unrelated areas.
3Measurement precision
If gaze coordinate information is accumulated over time, then accuracy of stereo matching improves, but system complexity increases
Solution Approach 1:
The patent applies universality by using the eye-tracking sensor and gaze coordinate accumulation mechanism for multiple purposes. The same gaze tracking infrastructure serves both to control the AR/HMD system and to provide precision positioning information for stereo matching. By making the gaze tracking system multi-functional, the patent improves matching precision without proportionally increasing system complexity, as the additional functionality leverages existing hardware and processing capabilities.
Data Source
AI summary
An image processing device is provided. The image processing device includes a camera configured to obtain a stereo image, an eye-tracking sensor configured to obtain gaze information of a user, a memory storing one or more instructions, and at least one processor. The at least one processor is configured to, by executing one or more instructions, extract feature points from the stereo image and generate gaze coordinate information in which gaze coordinates corresponding to the gaze information of the user are accumulated on the stereo image, and perform stereo matching based on the feature points and the gaze coordinate information.