Multimodal Interface Combining Touch and Audio for Video Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current portable communication devices lack effective systems and modalities to allow users to easily interact with and access large volumes of video content, particularly on handheld devices with smaller screens.
Innovation Solution
A multimodal user interface system that combines tactile input from a touch screen and audio input via a microphone to initiate queries, using speech recognition and natural language understanding to retrieve and display video content, leveraging a multimodal portal and media processing engine for efficient content access.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If portable communication devices use traditional single-mode interfaces (pushbuttons, microphones, or touch screens separately), then device complexity is reduced, but ease of operation deteriorates when accessing large volumes of video content
Solution Approach 1:
The patent combines multiple input modalities (tactile input from touch screen display and audio input from microphone) into a unified multimodal interface system. This merging allows users to interact with video content through natural combinations of speech and touch, significantly improving ease of operation when accessing large volumes of content without requiring complex manual navigation through traditional single-mode interfaces
Solution Approach 2:
The multimodal interface system provides universal access methods that work across diverse video content types and search scenarios. The system can handle various input combinations (speech-only, touch-only, or combined speech and touch) to perform content retrieval, making it adaptable to different user preferences and content access needs while maintaining a consistent interface framework
2Volume of moving object
If portable devices have smaller screens to maintain portability, then device portability is improved, but ease of operation deteriorates for accessing and navigating video content
Solution Approach 1:
The patent replaces mechanical/manual navigation systems (manual scrolling, searching, and selecting through touch screen gestures) with speech-based interaction systems. Users can verbally query and navigate video content libraries, eliminating the need for extensive manual screen interaction and making content access equally easy on small portable screens as on larger displays
Solution Approach 2:
The system introduces speech recognition and natural language processing as intermediary layers between the user and the video content database. This intermediary translates user intent expressed through speech into appropriate content retrieval operations, bridging the gap between limited screen real estate and comprehensive content navigation capabilities
3Ease of operation
If portable devices increase processing capabilities to handle multimodal input, then ease of operation improves, but use of energy deteriorates
Solution Approach 1:
The system implements selective processing where speech and tactile inputs are processed based on context and user needs rather than always processing both modalities simultaneously. The interface can operate in speech-only mode, touch-only mode, or combined mode depending on the situation, reducing energy consumption when full multimodal processing is not necessary while maintaining ease of operation when needed
Data Source
AI summary
A portable communication device has a touch screen display that receives tactile input and a microphone that receives audio input. The portable communication device initiates a query for media based at least in part on tactile input and audio input. The touch screen display is a multi-touch screen. The portable communication device sends an initiated query and receives a text response indicative of a speech to text conversion of the query. The portable communication device then displays video in response to tactile input and audio input.