UI Control Mapping to Keyboard Areas for Accessibility
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Visually impaired computer users face difficulties in interacting with user interfaces due to the time-consuming and inefficient methods of using the Tab key or shortcut keys, which provide little directional knowledge about the layout of UI controls.
Innovation Solution
A method that recognizes UI controls on a screen, determines their position relationships, and maps these controls to corresponding areas on a keyboard, aligning the keyboard layout with the screen layout, allowing specific keys to be associated with individual UI controls for direct access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If Tab key is used to navigate UI controls, then accessibility is provided, but operation time increases significantly
Solution Approach 1:
The patent creates a virtual keyboard layout that copies the spatial arrangement of UI controls onto a standard QWERTY keyboard. Each UI control is associated with a corresponding key area on the physical keyboard, allowing users to access controls by pressing keys in the same spatial pattern as their visual arrangement on the screen, eliminating the need to press Tab repeatedly through intervening controls.
Solution Approach 2:
The patent transitions from linear sequential navigation (Tab key moving through controls in document order) to two-dimensional spatial mapping (keyboard layout reflecting UI control positions on screen). This dimensional change allows users to jump directly to controls based on their visual position rather than navigating sequentially through all controls.
2Ease of operation
If shortcut keys grouped by classification are used, then access to controls is enabled, but directional layout knowledge is lost
Solution Approach 1:
The patent assigns different key areas on the keyboard to different UI controls based on their specific positions and types. Instead of grouping all controls of the same type together, each control receives a dedicated key area that preserves its unique spatial location information, allowing users to infer the layout structure from the keyboard key positions themselves.
3Adaptability or versatility
If traditional keyboard navigation is used, then compatibility is maintained, but operation efficiency decreases
Solution Approach 1:
The patent builds upon the universal QWERTY keyboard layout, making it serve multiple functions: traditional text input and enhanced UI navigation. The virtual keyboard mapping layer operates on top of the standard keyboard, allowing the same physical device to provide both conventional typing and efficient UI control access through spatially-mapped key associations.
Data Source
AI summary
According to an exemplary embodiment of the present invention, a method is provided. The method includes identifying UI controls in a GUI for display on a screen of a computing device. The method further includes determining a position relationship of the UI controls. The method further includes mapping the UI controls to keyboard areas such that a position relationship of the keyboard areas corresponds with the position relationship of the UI controls. One or more keys within the individual areas are associated with the UI controls that are mapped to the respective keyboard areas.