Multimodal Browser Speech Disambiguation via Visibility Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In multimodal applications, speech recognition grammars often produce ambiguous results when users interact with limited screen displays, leading to unintended activation of links due to the search algorithm's traversal of the grammar tree from the leaf node upwards, causing user intent to be misinterpreted.
Innovation Solution
A multimodal browser maintains a record of visibility for each voice-activated hyperlink, including current and past visibility, and the sequence in which they were scrolled off the display, allowing it to select the appropriate hyperlink for activation based on the recognized speech and visibility record.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a speech recognition grammar is used to recognize user speech in a multimodal application, then voice interaction capability is improved, but ambiguous results occur when multiple links match the same speech input
Solution Approach 1:
The patent introduces an intermediary disambiguation mechanism that sits between speech recognition and link activation. When speech matches multiple links, the system uses additional criteria (link visibility, scroll position, user interaction history) to select the intended link, preventing incorrect activation while maintaining voice interaction capability
Solution Approach 2:
The system changes the parameters used for link selection from simple speech matching to a multi-parameter evaluation including speech confidence score, link visibility status, scroll position, and temporal factors. This parameter expansion resolves ambiguity while preserving voice interaction versatility
2Ease of manufacture
If the grammar tree is traversed from leaf node upwards to match speech input, then speech recognition processing is simplified, but incorrect links are activated due to ambiguous matching
Solution Approach 1:
The system performs preliminary actions by collecting all potential matching links before final selection. Instead of activating the first match found during tree traversal, it gathers all candidates and then applies disambiguation criteria to select the correct link, ensuring precision while maintaining processing simplicity
3Adaptability or versatility
If multiple voice-activated links are provided on the display, then user interaction options are increased, but user intent cannot be accurately determined when links are scrolled off the display
Solution Approach 1:
The patent adds temporal and positional dimensions to link tracking. Instead of only considering currently visible links, the system maintains a history of link visibility states and scroll positions, allowing it to infer user intent even when links move in and out of the visible display area
Data Source
AI summary
Disambiguating a speech recognition grammar in a multimodal application, the multimodal application including voice activated hyperlinks, the voice activated hyperlinks voice enabled by a speech recognition grammar characterized by ambiguous terminal grammar elements, including maintaining by the multimodal browser a record of visibility of each voice activated hyperlink, the record of visibility including current visibility and past visibility on a display of the multimodal device of each voice activated hyperlink, the record of visibility further including an ordinal indication, for each voice activated hyperlink scrolled off display, of the sequence in which each such voice activated hyperlink was scrolled off display; recognizing by the multimodal browser speech from a user matching an ambiguous terminal element of the speech recognition grammar; selecting by the multimodal browser a voice activated hyperlink for activation, the selecting carried out in dependence upon the recognized speech and the record of visibility.