Browser Agent Contextual Command Sheet Voice Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Many web applications lack native voice-interface functionality, making it difficult to enable voice support for web pages with elements that are not identifiable from the Document Object Model (DOM) and requiring complex methods to determine which elements to speech-enable.
Innovation Solution
A system that includes a voice application and an agent executing within the browser, which monitors events and uses a Contextual Command Sheet (CCS) to determine the current context of the web application and enable voice interactions for specific capabilities, even if they are not exposed by the browser.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If complex methods are used to determine which elements to speech-enable, then voice support can be enabled for web pages with elements not identifiable from the DOM, but the device complexity and implementation difficulty increase
Solution Approach 1:
The patent introduces an intermediary information source (such as a metadata file or external service) that stores pre-defined voice interaction information for web applications. This intermediary layer bridges the gap between the web application and the speech recognition system, providing a standardized interface without requiring complex analysis of web page elements. The speech-enabled web application queries this information source to retrieve supported voice interactions, thereby enabling voice support while avoiding the complexity of directly analyzing and determining which DOM elements should be speech-enabled.
2Ease of operation
If voice interactions are enabled for all web application elements, then user interaction capability improves, but the quantity of voice interaction information required increases
Solution Approach 1:
The patent extracts and separates the voice interaction information from the web application code itself, storing it in an external information source. This allows the speech-enabled web application to selectively retrieve only the voice interaction information it needs, rather than having to process or store all possible voice interaction data for all web applications. The extraction principle enables the system to obtain necessary voice interaction capabilities while minimizing the quantity of information that must be processed and stored locally.
3Ease of manufacture
If voice input is enabled without contextual information, then implementation is simpler, but voice recognition accuracy decreases
Solution Approach 1:
The patent implements preliminary action by having the speech-enabled web application query the information source for supported voice interactions before processing voice input. The system retrieves and stores the relevant voice interaction information in advance, so when voice input is received, the speech recognition system already has the contextual information needed to accurately interpret the user's intent. This preliminary retrieval of contextual data maintains implementation simplicity while significantly improving voice recognition accuracy.
Data Source
AI summary
Methods and apparatus for voice-enabling a web application, wherein the web application includes one or more web pages rendered by a web browser on a computer. At least one information source external to the web application is queried to determine whether information describing a set of one or more supported voice interactions for the web application is available, and in response to determining that the information is available, the information is retrieved from the at least one information source. Voice input for the web application is then enabled based on the retrieved information.