Cross-OS UI Element Mapping for Consistent Assistive Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Features native to different operating systems often do not interoperate when running on the same OS, leading to inconsistent user interface interactions and incompatibilities, particularly affecting assistive technologies and accessibility tools.
Innovation Solution
The techniques enable bridging UI elements between different operating systems by using UI automation components and libraries to map and translate UI events and commands, ensuring consistent and predictable interactions across systems.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If features native to different operating systems are allowed to run on the same OS, then the number of available applications and features increases, but UI interoperability and consistency deteriorate
Solution Approach 1:
The patent introduces a UI automation component as an intermediary layer between the host OS and guest OS applications. This mediator translates UI events from the host OS into corresponding UI events for the guest OS, enabling seamless interaction without direct compatibility between the two systems. The intermediary handles the complexity of cross-OS UI translation, allowing diverse applications to run while maintaining consistent UI behavior.
2Ease of operation
If assistive tools are configured to interpret UI features of one operating system, then assistive functionality is optimized for that OS, but compatibility with applications from other operating systems deteriorates
Solution Approach 1:
The UI automation component is designed with universal functionality that works across multiple operating systems. It implements a standardized interface that can interpret UI features from different OSes (Android, iOS, Windows, etc.) and translate them into a common format that assistive tools can understand. This multi-functional approach allows assistive tools to work effectively with applications from any OS without requiring OS-specific configuration.
3Ease of operation
If UI commands are forwarded from proxy application to guest application, then user control capability is enabled, but proper interpretation of UI commands by guest application deteriorates due to OS incompatibility
Solution Approach 1:
The UI automation component performs parameter transformation by converting UI event parameters from the host OS format into the corresponding format expected by the guest OS. When a UI command is generated on the host OS, the automation component analyzes the event type, target element, and parameters, then translates these into the appropriate parameter set for the guest OS application. This parameter adaptation ensures that UI commands are not only forwarded but also properly interpreted and executed on the target OS.
Data Source
AI summary
Techniques are disclosed for enabling use of user interface (UI) elements across multiple operating systems. An indication of a UI element is selected via an interface of the first operating system. A mapping between UI elements of the first operating system and the second operating system is accessed. Based on the mapping, a corresponding UI element is provided to the second operating system. The corresponding UI element provides an equivalent UI element in the second operating system that corresponds to the UI element entered via the interface of the first operating system.