Dynamic Voice Command Extraction from UI Markup
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing voice recognition systems face challenges such as speaker dependence, disambiguation between similar-sounding words, recognition of domain-specific terms, and real-time issues, particularly in dynamic user interface settings, where legacy approaches like pre-coded voice commands and large dictionaries are cumbersome and inefficient.
Innovation Solution
A method to generate a dynamic dictionary of voice commands by parsing the user interface description to map keyboard or mouse commands to voice commands, allowing for real-time extraction and recognition of commands from the user interface, enabling voice control without pre-coding and reducing the need for extensive dictionaries.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large domain-specific dictionary is used to improve speech recognition accuracy, then recognition precision improves, but system complexity and real-time performance deteriorate
Solution Approach 1:
The patent extracts only the necessary voice commands from the user interface description rather than using a comprehensive domain-specific dictionary. The system parses the UI description to identify actionable elements and generates a minimal set of voice commands specific to the current interface context, eliminating the need for large pre-defined dictionaries while maintaining recognition accuracy.
Solution Approach 2:
The patent implements dynamic generation of voice commands based on the current user interface state. Rather than relying on a static large dictionary, the system continuously parses the UI description and adapts the available voice commands to match the current interface elements and operations, making the command set both context-aware and computationally efficient.
2Adaptability or versatility
If pre-coded voice commands are used to enable voice control, then voice recognition capability is provided, but developer flexibility and adaptability to dynamic interfaces deteriorate
Solution Approach 1:
The patent enables the user interface to generate its own voice commands automatically through parsing the UI description. The system extracts command information directly from the interface elements and their associated operations, eliminating the need for developers to manually pre-code voice commands. This self-service approach allows the interface to adapt to changes automatically without requiring developer intervention.
Solution Approach 2:
The patent implements dynamic generation of voice commands based on the current user interface state. Rather than relying on a static large dictionary, the system continuously parses the UI description and adapts the available voice commands to match the current interface elements and operations, making the command set both context-aware and computationally efficient.
3Measurement precision
If keyword prefixes are required to distinguish commands, then command disambiguation improves, but user convenience and ease of operation deteriorate
Solution Approach 1:
The patent applies context-specific parsing to extract voice commands directly from the user interface description. By analyzing the local context of each interface element and its associated operations, the system generates semantically meaningful commands that are inherently disambiguated by their context rather than requiring artificial keyword prefixes. Each command is tailored to its specific interface context, providing both precision and ease of use.
Data Source
AI summary
A method, system, and computer program product for human interface design. Embodiments proceed upon receiving a markup language description of user interface pages (e.g., HTML pages), then, without modifying the user interface page, parsing the markup language description to identify user interface objects configured to perform an operation responsive to a keyboard or mouse or pointing device. One or more mapping techniques serve to relate the parsed-out operation(s) to one or more voice commands. In some embodiments, the parser recognizes interface objects in forms such as a button, a textbox, a checkbox, or an option menu, and the voice commands correspond to an aspect that is displayed when rendering the interface object (e.g., a button label, a menu option, etc.). After receiving a user utterance, the utterance is converted into a text representation which in turn is mapped to voice commands that were parsed from the user interface page.