Multimodal Interaction Suggestion via Intent-State Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In Multi Device Experience (MDE) scenarios, users often interact with multiple devices using inappropriate modalities, leading to inefficient and unnatural interactions, especially in varying conditions such as driving or medical emergencies, where existing approaches fail to suggest alternative modalities.
Innovation Solution
An electronic device detects the user's intent and state by analyzing interaction history and device data, determining a suitable alternative modality and device to continue the interaction, and provides a suggestion through a Natural User Interface (NUI) on the associated devices, using an intent-state-modality mapping database and NUI template database.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If users interact with devices using a specific modality (e.g., touch) irrespective of conditions, then the interaction method is simple and consistent, but the interaction becomes unnatural and inefficient in varying situations
Solution Approach 1:
The system dynamically adjusts the interaction modality based on real-time analysis of user state, device state, and environmental context. Instead of using a fixed modality, the system selects from multiple modalities (touch, voice, gesture, etc.) according to the current situation, making the interaction adaptable and efficient across varying conditions
Solution Approach 2:
The system changes the interaction parameters (modality type) based on analyzed context factors such as user activity, device usage pattern, and environmental conditions. This allows the system to optimize interaction efficiency by selecting appropriate modalities for different scenarios while maintaining a consistent underlying architecture
2Ease of operation
If the system provides modality suggestions based on user intent and state analysis, then interaction efficiency and naturalness improve, but system complexity increases
Solution Approach 1:
The system performs self-analysis of user intent and state by monitoring interaction patterns, device usage, and contextual data. This self-service capability allows the system to autonomously determine appropriate modalities without requiring complex external analysis systems, thereby improving interaction naturalness while managing system complexity through self-contained intelligence
Solution Approach 2:
The system implements feedback loops where user interactions are continuously analyzed, and suggestions are provided based on this analysis. The system learns from user responses to suggestions and refines its intent-state analysis over time, improving interaction naturalness while using the feedback mechanism to manage complexity through iterative optimization rather than static complex rules
Data Source
AI summary
Provided are methods and systems for suggesting an enhanced multimodal interaction. The method for suggesting at least one modality of interaction, includes: identifying, by an electronic device, initiation of an interaction by a user with a first device using a first modality; detecting, by the electronic device, an intent of the user and a state of the user based on the identified initiated interaction; determining, by the electronic device, at least one of a second modality and at least one second device, to continue the initiated interaction, based on the detected intent of the user and the detected state of the user; and providing, by the electronic device, a suggestion to the user to continue the interaction with the first device using the determined second modality, by indicating the second modality on the first device or the at least one second device.