Gaze and Gesture Interaction for Input Bandwidth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current human-computer interaction methods, particularly with graphical user interfaces, suffer from bandwidth asymmetry as input devices like keyboards and mice have not kept pace with the advancements in output technologies, making eye-tracking solutions for interaction cumbersome and inefficient.
Innovation Solution
Combining gaze-based input with gesture-based user commands to enable intuitive and natural interaction with graphical user interfaces, allowing users to interact with computers in a touchscreen-like manner without a traditional touchscreen, using a control module that acquires user input from gesture detection and gaze tracking data to execute actions based on determined gaze points and gestures.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If eye tracking is used for object selection and activation, then input bandwidth is improved, but user stress and difficulty in controlling eye movements increase
Solution Approach 1:
The patent combines head pose detection with eye tracking to create a more natural interaction modality. The head pose detector identifies the user's intended target area, and eye tracking refines the selection within that area, merging two biological functions (head movement and eye movement) to solve the limitations of using either alone.
Solution Approach 2:
The system introduces an intermediary processing layer that detects head pose and uses it to define a region of interest, which then guides the eye tracking selection process. This intermediary head pose detection acts as a mediator between the user's natural head movements and the precise eye-gaze-based object selection.
2Ease of operation
If traditional input devices like keyboard and mouse are used, then ease of operation is maintained, but bandwidth asymmetry between input and output increases
Solution Approach 1:
The patent replaces mechanical input devices (keyboard and mouse) with a biological sensing system that detects head pose and eye movements. This substitution eliminates the need for physical pointing devices and enables a more direct mapping between user intent and system response, thereby reducing bandwidth asymmetry.
3Speed
If eye gaze tracking is used to position the cursor, then interaction speed is improved, but precision in selecting small or distant objects decreases
Solution Approach 1:
The patent segments the selection process into two distinct stages: first, head pose detection identifies a broad region of interest or target area, and second, eye tracking within that region performs precise object selection. This segmentation allows the system to leverage the speed of head movements for coarse positioning and the precision of eye tracking for fine-grained selection.
Solution Approach 2:
The system adds a spatial dimension to the selection process by using head pose to define a three-dimensional region of interest, which then constrains the two-dimensional eye tracking search space. This dimensional approach allows rapid navigation to general areas while maintaining precision for specific object selection.
Data Source
AI summary
The present invention provides improved methods and systems for assisting a user when interacting with a graphical user interface by combining gaze based input with gesture based user commands. The present invention provide systems, devices and method that enable a user of a computer system without a traditional touch-screen to interact with graphical user interfaces in a touch-screen like manner using a combination of gaze based input and gesture based user commands. Furthermore, the present invention offers a solution for touch-screen like interaction using gaze input and gesture based input as a complement or an alternative to touch-screen interactions with a computer device having a touch-screen, such as for instance in situations where interaction with the regular touch-screen is cumbersome or ergonomically challenging. Further, the present invention provides systems, devices and methods for combined gaze and gesture based interaction with graphical user interfaces to achieve a touchscreen like environment in computer systems without a traditional touchscreen or in computer systems having a touchscreen arranged ergonomically unfavourable for the user or a touchscreen arranged such that it is more comfortable for the user to use gesture and gaze for the interaction than the touchscreen.