Computer Vision Scene Segmentation via Gaze Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer vision systems face challenges in accurately identifying specific objects within a scene, particularly in determining the focus point of a user's gaze in augmented and virtual reality applications, where distinguishing between objects and backgrounds is difficult.
Innovation Solution
A computer vision system that combines biometric and gaze tracking devices to analyze physiological readings and determine semantic boundaries, allowing it to segment scenes into meaningful components and identify the user's focus point by associating relative gaze locations with semantic boundaries, creating a relatedness data structure to define relationships between segments.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If computer vision systems use traditional object recognition methods, then they can identify objects in images, but they cannot accurately determine the specific focus point of a user's gaze within a scene
Solution Approach 1:
The system segments the scene into multiple semantic regions (e.g., sky, ground, objects) and uses neurological readings to determine which segment the user is focusing on. This breakdown of the scene into manageable segments allows precise focus point identification without requiring overly complex processing of the entire scene at once.
Solution Approach 2:
The patent introduces neurological readings (eye tracking data, pupillary response, fixation patterns) as an intermediary mechanism between the user's gaze and the computer vision system's object identification. These physiological signals serve as a mediator that directly indicates the user's focus point, bridging the gap between traditional computer vision and user intent.
2Measurement precision
If computer vision systems analyze physiological readings and gaze data to determine semantic boundaries, then they can accurately identify user focus points, but the device complexity increases
Solution Approach 1:
The system uses a multi-functional approach where the same computer vision framework processes both traditional visual data and neurological readings. The semantic segmentation algorithm serves multiple purposes: it organizes the scene for general understanding and simultaneously identifies user focus points by correlating with gaze data, reducing the need for separate specialized processing systems.
Solution Approach 2:
The system implements feedback loops where neurological readings continuously inform and adjust the semantic segmentation process. As the user's gaze moves, the system receives real-time physiological feedback and dynamically updates which semantic boundaries are relevant, allowing adaptive focus point identification without requiring exhaustive analysis of all scene elements.
3Reliability
If the system integrates multiple data sources (physiological readings, gaze tracking, computer vision), then it can accurately segment scenes into semantic components, but the processing time and computational resources increase
Solution Approach 1:
The system performs preliminary semantic segmentation of the scene based on computer vision data before integrating neurological readings. This pre-processing step creates a structured framework of semantic regions that can be quickly queried and adjusted based on gaze data, avoiding the need to process all raw visual data in real-time when the user's focus changes.
Solution Approach 2:
Rather than continuously processing all available data sources at full resolution, the system applies partial processing strategies. It uses lower-resolution or summarized versions of physiological and visual data for initial analysis, only invoking full-detail processing when the user's gaze indicates interest in specific semantic regions, thereby reducing overall processing time while maintaining segmentation reliability.
Data Source
AI summary
Computer vision systems for segmenting scenes into semantic components identify a differential within the physiological readings from the user. The differential corresponds to a semantic boundary associated with the user's gaze. Based upon data gathered by a gaze tracking device, the computer vision system identifies a relative location of the user's gaze at the time of the identified differential. The computer vision system then associates the relative location of the user's gaze with a semantic boundary.