Multimodal Query Processing via Context Summarization and Reformulation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing multimodal cognitive systems face inefficiencies in interpreting user queries and providing responses, often requiring users to repeat conversations with experts and failing to accurately summarize context, leading to incorrect or unsatisfactory responses.
Innovation Solution
A method and system that split multimodal user queries into sub-queries, determine response availability, and when necessary, summarize context and historical conversation data to reformulate queries for expert input, ensuring accurate and efficient response generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the system redirects the user to directly communicate with the expert when unable to provide responses, then the expert can provide direct assistance, but the user has to repeat the full conversation to the expert, wasting time and system resources
Solution Approach 1:
The system performs preliminary summarization of the conversation between user and system before redirecting to the expert. The summary module creates a condensed version of the conversation context, which is then provided to the expert along with the reformulated query, eliminating the need for the user to repeat the entire conversation.
Solution Approach 2:
The patent introduces an intermediary summarization mechanism that acts as a bridge between the user-system conversation and the expert interaction. This intermediary process transforms the full conversation into a concise summary that captures essential context, which is then transmitted to the expert to facilitate efficient communication without requiring user repetition.
2Productivity
If the system provides a summary of the conversation based on keywords to the expert, then the information exchange is reduced, but the keywords may ignore the context of the user queries, leading to incorrect interpretation
Solution Approach 1:
The system incorporates feedback mechanisms where the summarization module continuously refines the conversation summary based on contextual understanding. The reformulated query module uses this feedback to ensure that the summarized content accurately reflects the user's intent and context, rather than merely relying on keyword extraction, thus maintaining both efficiency and accuracy.
Solution Approach 2:
The patent transforms the summarization approach by changing the parameters from simple keyword-based summarization to context-aware summarization. The system adjusts the summarization parameters to include contextual information, historical conversation data, and semantic understanding, thereby maintaining accuracy while achieving efficient information exchange.
3Reliability
If the system requires expert intervention for all unanswerable queries, then complete responses can be provided, but system resources are wasted and user experience deteriorates due to unnecessary expert involvement
Solution Approach 1:
The system applies partial action by involving the expert only when absolutely necessary. The query analysis module first attempts to handle queries using available resources and historical data, and only redirects to the expert when the query truly requires expert knowledge. This partial involvement of the expert maintains response completeness while avoiding unnecessary resource consumption.
Solution Approach 2:
The system enables self-service capabilities through the reformulated query module, which attempts to autonomously reformulate and answer queries using summarized conversation context and historical data. This self-service approach handles routine queries without expert intervention, reserving expert resources for complex cases that genuinely require human expertise.
Data Source
AI summary
Disclosed herein is method and system for processing multimodal user queries. The method comprises determining availability of one or more responses to each of one or more sub-queries, wherein the one or more sub-queries are formed by splitting the multimodal user queries. The method detects requirement of an expert to provide the one or more responses upon determining at least one of unavailability of the one or more responses by the response generation system or based predefined conditions. Thereafter, a summarized content is generated by summarizing context of the one or more sub-queries and historical conversation data associated with the one or more sub-queries. Based on the summarized content, the one or more sub-queries are reformulated. Finally, the one or more responses received, from the expert, for the reformulated one or more sub-queries are collated provided as the one or more responses for the multimodal user queries.