Tapered Prompts in Multimodal Browser Applications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
User interaction with small devices through keyboards or styluses has become cumbersome due to their increasing miniaturization, despite having sufficient processing power for multimodal access, which combines multiple input modes like speech recognition, keyboards, touch screens, and styluses, limiting ease of use in multimodal applications.
Innovation Solution
Implementing tapered prompts in multimodal applications using multimodal browsers that identify prompt elements and attributes, allowing speech prompts to change based on user interaction, such as becoming more detailed for help or terser for familiar tasks, to enhance user experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional keyboard or stylus input methods are used on miniaturized devices, then device portability is maintained, but user interaction becomes cumbersome and difficult
Solution Approach 1:
The patent combines multiple input modes (speech recognition, keyboard, touch screen, stylus) into a unified multimodal input system. The browser integrates these different input channels to work together seamlessly, allowing users to switch between or combine input methods during interaction, thereby improving ease of operation without requiring complex separate systems for each input type.
Solution Approach 2:
The multimodal browser is designed to handle multiple types of input through a single unified interface. It can process speech, keyboard entries, touch screen gestures, and stylus input all through the same browser environment, making the device versatile and eliminating the need for separate specialized input systems for each interaction type.
2Ease of operation
If speech recognition is added to multimodal applications, then user interaction becomes more natural, but system complexity increases
Solution Approach 1:
The patent introduces a speech recognition intermediary layer that translates spoken language into commands the browser can process. This intermediary handles the complexity of speech processing separately from the core browser functionality, allowing speech recognition to be integrated without fundamentally complicating the browser's core architecture. The speech recognition system acts as a mediator between the user's natural speech and the application's command structure.
3Reliability
If prompts are always detailed and helpful, then user assistance is maximized, but interaction efficiency decreases for familiar tasks
Solution Approach 1:
The patent implements dynamic prompt adjustment where the level of prompt detail automatically adapts based on the user's interaction history and demonstrated familiarity with the application. For new users or unfamiliar tasks, the system provides detailed, helpful prompts. As users become more familiar with the application through repeated interactions, the prompts automatically become more concise, thereby maintaining reliability for new users while improving productivity for experienced users.
Data Source
AI summary
Methods, apparatus, and computer program products are described for invoking tapered prompts in a multimodal application implemented with a multimodal browser and a multimodal application operating on a multimodal device supporting multiple modes of user interaction with the multimodal application, the modes of user interaction including a voice mode and one or more non-voice modes. Embodiments include identifying, by a multimodal browser, a prompt element in a multimodal application; identifying, by the multimodal browser, one or more attributes associated with the prompt element; and playing a speech prompt according to the one or more attributes associated with the prompt element.