Eye-Tracking AAC Keyboard Layout With In-Focus Word Suggestions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmentative and alternative communication (AAC) devices with on-screen keyboards suffer from slow input speeds and high error rates due to inaccuracies in eye-tracking cameras and the need for users to search for additional elements outside their focus area.
Innovation Solution
An optimized on-screen keyboard that displays information and action elements within the user's focus area, using eye-tracking to generate context-sensitive word suggestions and adjustments based on previous inputs, environmental data, and user-specific frequencies to enhance input speed and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If word suggestions are displayed outside the focus area (conventional layout), then the keyboard layout remains simple, but users must search for elements outside their focus area, reducing input speed
Solution Approach 1:
The patent embeds word suggestions within the existing letter buttons by adding a second dimension of information display. Instead of placing suggestions in separate areas outside the focus area, the suggestions are integrated into the button structure itself, allowing users to access both letter selection and word suggestions within the same visual space without expanding the keyboard's horizontal or vertical footprint.
Solution Approach 2:
The word suggestions are nested within the letter buttons, similar to a nested doll structure. Each button contains not only the primary letter information but also embedded word suggestions that appear when the button is focused. This nesting allows multiple levels of information to coexist within the same spatial boundary, eliminating the need for separate suggestion areas.
2Ease of operation
If eye-tracking camera is used to capture gaze position, then the device enables hands-free operation, but inaccuracies in gaze position capture lead to high error rates
Solution Approach 1:
The system performs preliminary actions by predicting which word suggestions are most likely to be selected based on the current input context and language models. These predicted suggestions are pre-loaded and highlighted within the focus area before the user actually selects them, allowing the system to compensate for slight inaccuracies in gaze position tracking by anticipating the user's intent in advance.
Solution Approach 2:
The system implements feedback mechanisms where the display dynamically adjusts word suggestions based on detected gaze patterns. When the eye-tracking camera detects prolonged focus on a particular button or region, the system provides feedback by highlighting or prioritizing certain word suggestions, allowing users to correct or confirm their selection intent even if the initial gaze position measurement was imprecise.
3Adaptability or versatility
If the keyboard displays only basic letter buttons, then the interface remains simple, but users cannot access context-sensitive word suggestions efficiently
Solution Approach 1:
The keyboard interface is made dynamic by automatically adjusting the displayed word suggestions based on the current input context. As users type or select letters, the system continuously updates the suggestion list within each button to reflect the most relevant context-sensitive words. This dynamic adaptation provides versatility without requiring users to manually navigate through static, comprehensive word lists.
Solution Approach 2:
The patent merges the basic letter button function with the context-sensitive word suggestion function into a single integrated interface element. Each button simultaneously serves as both a letter selector and a context-aware suggestion display, combining what would traditionally be separate interface components into one unified structure that adapts its content based on the input context.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The optimized keyboard significantly increases input speed and reduces errors by intuitively presenting word suggestions and actions within the user's focus area, allowing users to participate more actively in conversations.
Implementation Method 1
a speech computer that uses an eye-tracking camera to monitor the user's gaze on an on-screen keyboard
Data Source
Figure 1
Figure 2a~2b
Figure 2c~2d
AI summary
The present invention relates to a method for generating an optimized on-screen keyboard for a device (20), in particular an augmentative and alternative communication (AAC) device, wherein the device is configured to display an on-screen keyboard (1) that can be operated by a user using eye tracking, the method comprising the following steps: (a) displaying an on-screen keyboard (1) having a plurality of buttons on the device (20); (b) receiving at least one user input with respect to the on-screen keyboard (1), in particular focusing on a button using eye tracking; (c) determining context information, at least based on the user input;(d) Generating a modified on-screen keyboard based on the context information, wherein the modified on-screen keyboard includes at least one modified information and/or action element located in a focus area of the user; and (e) Displaying the modified on-screen keyboard on the device (1). A computer-readable storage medium containing instructions that cause at least one processor to implement the method, and a corresponding device, in particular for supported communication, are also specified.