Keyboard Control of Graphical User Interface via Movement Path Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Many users are unable to operate graphical user interfaces without a pointing device or multi-touch interaction hardware, as they are not aware of the existence or usage of shortcut keys, and lack access to specialized hardware for gesture-based control.
Innovation Solution
A keyboard is used to control graphical user interfaces by interpreting sequences of key-presses as movement paths, which are then mapped to coordinates in the user interface, enabling single-touch and multi-touch input without the need for a pointing device or multi-touch hardware.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a pointing device or multi-touch hardware is used, then the ease of operation for graphical user interfaces is improved, but the device complexity increases
Solution Approach 1:
The keyboard is made multi-functional by enabling it to serve both its traditional text input purpose and a new pointing device function. The system detects sequences of key presses and interprets them as movement paths, allowing the keyboard to control cursor movement and manipulate on-screen objects without requiring additional pointing device hardware.
Solution Approach 2:
The keyboard serves itself by utilizing its existing structure and key press detection capability to provide pointing device functionality. Instead of requiring external specialized hardware, the system repurposes the keyboard's inherent key press sequence data to generate movement paths and control the graphical user interface.
2Adaptability or versatility
If specialized multi-touch interaction hardware is provided, then the adaptability of the system to support gesture-based control is improved, but the ease of manufacture deteriorates
Solution Approach 1:
The keyboard is transformed into a multi-functional device that supports both traditional text input and gesture-based control. By detecting sequences of key presses and interpreting them as movement paths, the system enables multi-touch-like functionality using existing keyboard hardware, eliminating the need for specialized multi-touch interaction hardware.
3Productivity
If shortcut keys are provided in graphical user interfaces, then the productivity is improved, but the ease of operation deteriorates
Solution Approach 1:
The system introduces an intermediary layer that translates keyboard movement sequences into graphical user interface control commands. This mediator interprets the physical movement path of key presses and maps it to corresponding on-screen cursor movements and object manipulations, bridging the gap between keyboard input and graphical interface control.
Data Source
AI summary
User interface control using a keyboard is described. In an embodiment, a user interface displayed on a display device is controlled using a computer connected to a keyboard. The keyboard has a plurality of alphanumeric keys that can be used for text entry. The computer receives data comprising a sequence of key-presses from the keyboard, and generates for each key-press a physical location on the keyboard. The relative physical locations of the key-presses are compared to calculate a movement path over the keyboard. The movement path describes the path of a user's digit over the keyboard. The movement path is mapped to a sequence of coordinates in the user interface, and the movement of an object displayed in the user interface is controlled in accordance with the sequence of coordinates.