Gaze-Based Entity Selection Using Leaky Attention Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing gaze-based user input systems struggle to accurately associate commands with intended real or virtual objects due to saccadic eye movements, leading to incorrect associations and unnatural user experiences.
Innovation Solution
A spatial intent model using a leaky integrator to calculate time-dependent attention values for entities in a viewed scene, allowing user inputs to be correctly associated with the intended entity by considering gaze samples over time, even during saccadic eye motion.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If gaze direction is used to identify objects for user input association, then user input can be associated with gazed objects, but saccadic eye movements cause incorrect associations between commands and intended objects
Solution Approach 1:
The system performs preliminary actions by continuously tracking gaze direction over time and pre-calculating attention values before a command is issued. The leaky integrator accumulates gaze samples in advance, building up attention values that reflect sustained viewing rather than momentary saccades. This preliminary accumulation of gaze data allows the system to distinguish between intentional fixation and transient eye movements before the user actually issues a command.
Solution Approach 2:
The patent introduces an intermediary mechanism - the leaky integrator - that sits between the raw gaze direction data and the final object identification decision. This intermediary processes the gaze stream by integrating samples over time while applying a decay factor, effectively filtering out the harmful saccadic movements while preserving meaningful fixation patterns. The attention value computed by this intermediary serves as a mediator that bridges the gap between noisy gaze data and reliable object selection.
2Productivity
If gaze samples are processed in real-time, then user input can be quickly associated with objects, but saccadic eye movements during processing lead to unreliable attention determination
Solution Approach 1:
The system maintains continuous processing of gaze samples through the leaky integrator, which constantly accumulates and decays attention values even as the user moves their eyes. This continuous action ensures that processing never stops, maintaining productivity while the mathematical integration process inherently filters out transient saccades. The continuous updating of attention values allows the system to adapt dynamically to changing gaze patterns without interruption.
Solution Approach 2:
The leaky integrator changes the parameter of time by integrating gaze samples over a temporal window rather than processing them instantaneously. This parameter transformation from discrete time points to continuous temporal accumulation allows the system to maintain fast response while filtering noise. The decay factor in the integration introduces a time-based weighting that naturally emphasizes recent sustained gaze while diminishing the impact of older or transient samples.
Data Source
Figure 1~2
Figure 3A
Figure 3B
AI summary
One example provides a computing device comprising instructions executable to receive information regarding one or more entities in the scene, to receive eye tracking a plurality of eye tracking samples, each eye tracking sample corresponding to a gaze direction of a user and, based at least on the eye tracking samples, determine a time-dependent attention value for each entity of the one or more entities at different locations in a use environment, the time-dependent attention value determined using a leaky integrator. The instructions are further executable to receive a user input indicating an intent to perform a location- dependent action, associate the user input to with a selected entity based at least upon the time-dependent attention value for each entity, and perform the location-dependent action based at least upon a location of the selected entity.