Topic-Segmented Conversation History for Relevant LLM Context
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing large language models (LLMs) generate inaccurate outputs when conversation topics change due to irrelevant portions of the conversation history being included, leading to resource exhaustion and reduced relevance, particularly in multi-modal contexts.
Innovation Solution
Implement a system that segments conversation history into topics and filters relevant segments for LLM input, maintaining contextual accuracy while reducing token and computational resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the entire conversation history is provided as input to the LLM, then the LLM can maintain contextual awareness, but the token resources are rapidly exhausted and computational resources are wasted
Solution Approach 1:
The patent segments the conversation history into multiple chunks or blocks, where each chunk contains a subset of the conversation turns. Instead of providing the entire conversation history at once, the system divides it into manageable segments that can be processed separately, reducing the token burden on the LLM while maintaining contextual awareness through selective chunking strategies.
Solution Approach 2:
The patent extracts only the most relevant portions of the conversation history that are necessary for generating the current response. By identifying and removing redundant or less important conversation turns, the system reduces the overall token count while preserving the essential context needed for accurate LLM responses.
2Reliability
If the entire conversation history is provided as input to the LLM, then contextual information is available, but computational resources are wasted processing irrelevant tokens
Solution Approach 1:
The patent extracts and removes irrelevant or redundant conversation turns from the input prompt before sending it to the LLM. By identifying and excluding portions of the conversation history that do not contribute to the current query, the system reduces the computational burden on the LLM while maintaining access to necessary contextual information.
Solution Approach 2:
The patent applies partial action by providing only a subset of the full conversation history to the LLM, specifically selecting the most relevant portions needed for the current task. This avoids the excessive processing of entire conversation histories while ensuring sufficient context is provided for accurate responses.
3Reliability
If all prior messages are included in the prompt, then the LLM can understand the full conversation context, but the output relevance decreases when topics change
Solution Approach 1:
The patent extracts and removes conversation turns that are unrelated to the current topic or query. By identifying topic boundaries and eliminating irrelevant conversation segments, the system ensures that the LLM receives only the context necessary for generating relevant outputs, preventing confusion from unrelated prior discussions.
Solution Approach 2:
The patent segments the conversation history into topic-specific blocks, allowing the system to selectively include only the relevant topic segments in the prompt. This segmentation strategy maintains contextual understanding within each topic while preventing interference from unrelated topics, thereby improving output relevance.
Data Source
AI summary
Methods and systems for segmenting a conversation session and providing context to a generative language model are described. A conversation history is maintained for an ongoing conversation session. The conversation history contains conversation segments, where each conversation segment is associated with at least one topic and includes previous message(s) in the conversation session. A new message is received for the conversation session, and topic(s) associated with the new message are determined. The conversation history is filtered based on relevance to the topic(s) associated with the new message. The filtered conversation history has a relevant conversation segment associated with a topic that is relevant to the topic(s) associated with the new message. A prompt is provided to a generative language model based on the filtered conversation history and the new message. A message is outputted based on output generated by the generative language model in response to the prompt.