Web Speech Interaction via Browser Plugin and Element Tagging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer user interfaces often lack efficient speech interaction capabilities, with existing implementations being difficult to develop and deploy, limiting natural communication modes for users.
Innovation Solution
Enabling speech interaction on web user interfaces by analyzing webpage coding to identify interactable elements, associating descriptive identifiers with these elements in a data structure, and matching speech inputs to perform corresponding actions, using techniques like n-gram matching and browser plugins or add-ons for speech recognition and parsing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If speech interaction is added to web user interfaces, then user interaction efficiency is improved, but development complexity increases
Solution Approach 1:
The patent introduces a browser plugin as an intermediary component that bridges speech recognition and web user interfaces. The plugin captures speech commands, processes them through speech-to-text conversion, and translates them into actionable selections for interactable elements on web pages, thereby enabling speech interaction without requiring complex native integration into the web browser itself
Solution Approach 2:
The system achieves universality by making web user interfaces accessible through multiple input modalities - both traditional keyboard/mouse interaction and speech-based interaction. The same web interface can be controlled by different users through their preferred input method, and the speech recognition system can identify and execute various types of commands (navigation, selection, input) across different web applications
2Speed
If speech recognition processing is performed locally in the browser, then response time is improved, but processing power requirements increase
Solution Approach 1:
The patent extracts the speech recognition and natural language processing functions from the browser and places them in a separate, specialized service. This allows the browser to remain lightweight while delegating computationally intensive tasks to a dedicated processing system that can handle complex speech analysis and command translation more efficiently
Data Source
AI summary
One or more associated identifiers are determined based on one or more associated tag types of an interactable element of web content. The determined one or more associated identifiers are associated with the interactable element. Based on the association of the determined one or more associated identifiers with the interactable element, one of the interactable element is matched to a received speech input. An action is performed with respect to the interactable element based on the matching.