Line-of-Sight Command Selection Without Mouse
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional pointing support technologies face inefficiencies in speech recognition, particularly with short utterances, and lack user convenience, as they allocate different numbers to the same button each time and overlay numbers with GUI elements, making it difficult for physically disabled or elderly users to accurately select commands without a mouse.
Innovation Solution
A pointing support apparatus that detects a user's line-of-sight position on a screen, extracts commands within a search range, generates a command dictionary table associating commands with speech recognition data, and decides the selected command based on speech input, allowing for efficient and accurate command selection without a mouse.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If conventional speech recognition technology is used for pointing, then it can enable operation without a mouse, but it has difficulty recognizing short utterances accurately
Solution Approach 1:
The system performs preliminary actions by detecting the user's line-of-sight position before speech recognition, extracting commands within the visual field, and generating a command dictionary table in advance. This preparation reduces the recognition burden during actual speech input, enabling accurate recognition of short utterances without requiring a mouse.
2Ease of operation
If numbers are allocated to GUI elements for speech recognition, then pointing without a mouse becomes possible, but the same button receives different numbers each time reducing reliability
Solution Approach 1:
The system replaces the conventional number-allocation mechanism with a line-of-sight-based command extraction mechanism. Instead of assigning arbitrary numbers to GUI elements, the system detects where the user is looking and extracts the corresponding command directly from the visual field, ensuring consistent and reliable command identification across multiple uses.
3Ease of operation
If numbers are overlaid on GUI elements to assist selection, then users can see selection targets, but it creates visual clutter making accurate selection difficult
Solution Approach 1:
The system extracts and removes the distracting number overlays from the GUI display. Instead of showing numbers on the screen, the system uses line-of-sight detection to identify commands and presents a clean command dictionary table separately, eliminating visual clutter while maintaining accurate selection capability.
4Adaptability or versatility
If speech recognition processes all commands on screen, then complete coverage is achieved, but processing load increases significantly
Solution Approach 1:
The system applies local quality by focusing speech recognition resources only on commands within the user's visual field rather than processing all commands on the screen. Line-of-sight detection identifies the relevant region, and the command dictionary table is generated only for commands in that region, reducing processing load while maintaining comprehensive coverage of accessible commands.
Data Source
AI summary
A pointing support apparatus includes a memory, and a processor coupled to the memory and configured to detect a line-of-sight position of a user on a screen, extract a command included in a search range of the screen with reference to the line-of-sight position, generate a table in which the command and speech information of the command are associated with each other, and decide, when speech information from outside is accepted, a command based on comparison of the recognized speech information and the speech information of the table.