Browser Agent for Dynamic Voice Interaction in Web Applications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Many web applications lack native voice-interface functionality, making it difficult to enable voice interactions as browsers do not expose voice capabilities for these applications, and identifying elements in the Document Object Model (DOM) is insufficient to determine which elements to speech-enable.
Innovation Solution
A system that includes a voice application and an agent executing within the browser to monitor events and identify the current context of web applications, using a Contextual Command Sheet (CCS) to determine supported voice interactions and enable voice input based on the context, even for web applications without native voice interfaces.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If web applications are developed without native voice-interface functionality, then development complexity and time are reduced, but voice interaction capability is lost
Solution Approach 1:
The patent introduces an intermediary component (voice enabling module or agent) that acts as a bridge between the web application and the speech recognition system. This intermediary monitors DOM events, determines application context, and dynamically configures speech recognition grammar without requiring changes to the original application code, thus preserving development simplicity while enabling voice interaction capability
Solution Approach 2:
The patent creates a universal voice enabling mechanism that can be applied to multiple different web applications without requiring application-specific customization. The system provides multi-functional capabilities by supporting various types of web applications (e.g., email clients, document editors, search engines) through a single standardized interface that dynamically adapts to each application's context
2Adaptability or versatility
If speech recognition is enabled for all web applications, then voice interaction capability is improved, but system complexity and resource consumption increase
Solution Approach 1:
The patent implements dynamic speech recognition configuration where the system continuously monitors DOM events and adjusts the active grammar based on the current application context. Instead of enabling all possible voice commands simultaneously, the system dynamically activates only the relevant commands for the current state, reducing system complexity while maintaining voice interaction capability
Solution Approach 2:
The patent applies local quality by configuring speech recognition grammar specific to each application context rather than using a single universal grammar. The system monitors DOM events and activates context-specific voice commands (e.g., email-specific commands when in email composition mode, document editing commands when in document mode), reducing overall system complexity by enabling only locally relevant functionality
3Ease of manufacture
If DOM element identification is used to determine speech-enabled elements, then implementation simplicity is improved, but accuracy in identifying appropriate voice interaction targets is insufficient
Solution Approach 1:
The patent implements feedback mechanisms by continuously monitoring DOM events and using this information to refine and update the identification of speech-enabled elements. The system doesn't rely solely on static DOM structure but uses event feedback (clicks, hovers, form submissions) to dynamically determine which elements should respond to voice commands, improving identification accuracy while maintaining implementation simplicity through event-driven architecture
Data Source
AI summary
Methods and apparatus for voice-enabling a web application, wherein the web application includes one or more web pages rendered by a web browser on a computer. At least one information source external to the web application is queried to determine whether information describing a set of one or more supported voice interactions for the web application is available, and in response to determining that the information is available, the information is retrieved from the at least one information source. Voice input for the web application is then enabled based on the retrieved information.