Dialog Control Flow for Information Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing conversation service applications, such as virtual assistants and chatbots, face limitations in flexibility and responsiveness due to one-to-one mapping with natural language processing (NLP) services, reliance on static knowledge bases, and the inability to accurately handle new or post-training data queries with large language models (LLMs).
Innovation Solution
The implementation of a dynamic and adaptive dialog control flow system that allows for the selection of the most suitable NLP service based on comprehension scores and the delegation to generalized language processing services when necessary, along with the augmentation of LLM prompts with context data and external information to enhance response accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a one-to-one mapping between conversation service applications and NLP services is used, then the system structure is simple, but the flexibility and responsiveness of the application is restricted
Solution Approach 1:
The patent segments the NLP service layer by introducing multiple specialized NLP services (e.g., intent classification, entity extraction, sentiment analysis) that can be independently selected and combined. This allows the conversation service application to flexibly compose different NLP service combinations based on specific task requirements, thereby improving adaptability while maintaining manageable system complexity through modular architecture.
Solution Approach 2:
The patent implements a universal NLP service framework where a single NLP service can perform multiple functions through different service instances or configurations. The system allows one NLP service to serve multiple conversation service applications with different domains and purposes, enhancing versatility without requiring separate dedicated services for each application type.
2Loss of time
If a static knowledge base is used to respond to user inquiries, then the response time is reduced for familiar questions, but the system cannot accurately handle new or post-training data queries
Solution Approach 1:
The patent introduces a dynamic knowledge base that can adapt its content and structure based on the query context and data recency. The system dynamically selects between static knowledge base queries for well-known questions and dynamic data retrieval for new or updated topics. This dynamic approach allows the system to maintain fast response times for familiar questions while accurately handling new data through real-time or recently updated knowledge sources.
Solution Approach 2:
The patent implements preliminary actions by pre-processing and indexing data from multiple sources before actual user queries arrive. The system performs preliminary data retrieval, filtering, and organization of knowledge base content and external data sources in advance, enabling rapid query response while ensuring the most relevant and up-to-date information is readily available for both static and dynamic queries.
3Adaptability or versatility
If LLMs are used to generate dynamic responses to newly-seen customer inquiries, then the system can handle new data, but the training process is expensive and time intensive
Solution Approach 1:
The patent uses copying by leveraging pre-trained LLM weights and architectures as a foundation, rather than training from scratch. The system copies and adapts existing LLM knowledge through fine-tuning on domain-specific data, which significantly reduces training time and computational costs while maintaining the ability to handle new and specialized data queries effectively.
Solution Approach 2:
The patent applies parameter changes by adjusting LLM parameters such as temperature, top-k sampling, and prompt engineering to optimize responses for different types of queries. The system dynamically modifies these parameters based on the query context, data recency, and desired response characteristics, enabling effective handling of new data without requiring retraining of the LLM itself.
4Ease of manufacture
If LLMs are trained on historical data up to a cut-off period, then the model is ready for deployment, but it cannot accurately answer queries involving post-training data
Solution Approach 1:
The patent introduces an intermediary layer between the pre-trained LLM and the query processing pipeline. This intermediary component acts as a mediator that enhances LLM outputs by injecting recent knowledge, facts, and context from external sources or updated knowledge bases. The intermediary processes queries involving post-training data by combining LLM-generated responses with recent information, thereby improving accuracy without requiring LLM retraining.
Solution Approach 2:
The patent implements feedback mechanisms where the system continuously monitors query responses and user interactions, using this feedback to update knowledge bases and refine LLM prompts. The feedback loop enables the system to learn from post-training data and improve its accuracy over time, maintaining model readiness while adapting to new information through continuous learning and knowledge updates.
Data Source
AI summary
Methods and apparatuses are described for dialog control flow in information retrieval applications. A server establishes a chat-based communication session between an information retrieval application and a client device. The server determines a user intent from utterances received from a user of the client device and initiates a first dialog workflow associated with the user intent. The server invokes NLP services using the utterances to determine a comprehension score for the user intent and identifies a first one of the NLP services to continue the first dialog workflow when the comprehension score is at or above a threshold value, including generating a response to the utterances using the first NLP service. The server delegates the communication session to a second dialog workflow when the comprehension score is below the threshold value, including invoking a generalized language processing service associated with the second dialog workflow using the user intent to generate a response to the utterances. The server transmits the generated response to the client device as part of the chat-based communication session.