LLM Chat Session Partitioning for Multi-Topic Summarization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current LLMs lack memory capabilities, leading to independent queries and limited token size, and struggle with overloaded or mixed-topic interactions, resulting in low-quality summaries and ineffective summarization of multiple subjects, and existing technologies fail to effectively manage conversations with multiple topics, leading to low-quality summaries and inefficient token usage.
Innovation Solution
Implement a community detection algorithm to partition LLM chat sessions into semantically similar groups, using a two-phase process to map interactions into graphs or n-dimensional spaces, enabling effective summarization and token management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If past interactions are included in the current query to provide memory, then the LLM generates more accurate responses, but the number of tokens available in the current query is reduced
Solution Approach 1:
The patent extracts only the essential information from past interactions by generating concise summaries of previous conversations. Instead of including full past interactions, the system extracts key points and topics, thereby providing memory functionality while consuming fewer tokens than including complete interaction histories.
Solution Approach 2:
The patent transforms the representation of past interactions from detailed conversational text to compressed summary representations. By changing the parameter of how past interactions are stored and retrieved (from full text to summaries), the system maintains response accuracy while reducing token consumption significantly.
2Adaptability or versatility
If multiple subjects are considered in a single conversation, then the chat session handles diverse topics, but the summarization quality deteriorates when multiple subjects are mixed
Solution Approach 1:
The patent segments a multi-topic conversation into multiple sub-conversations, each dedicated to a single topic or subject. By dividing the overloaded chat session into topic-specific segments, the system enables high-quality summarization for each individual topic while maintaining the ability to handle diverse subjects overall.
Solution Approach 2:
The patent applies different summarization approaches to different topics within the same conversation based on their specific characteristics. Each topic receives specialized summarization treatment appropriate to its nature, thereby maintaining high summarization quality across multiple diverse subjects rather than applying a one-size-fits-all approach.
3Productivity
If the query size is limited to a few thousand tokens, then the LLM operates efficiently, but the ability to store and process past interactions is constrained
Solution Approach 1:
The patent extracts essential information from past interactions and stores only these extracted elements as summaries. This extraction approach allows the system to retain the most important information from past conversations while keeping the token footprint well within the few thousand token limit, maintaining operational efficiency without significant information loss.
Solution Approach 2:
Instead of trying to store all past interactions within the token limit, the patent inverts the approach by storing compressed representations (summaries) that capture the essence of past interactions. This inversion allows efficient operation with limited tokens while preserving the necessary information for context-aware responses.
Data Source
AI summary
One example method includes for each of one or more interactions of a chat session between a user and a chatbot, performing a building phase that comprises mapping the interaction into either a graph or an n-dimensional space, performing a verification phase that comprises partitioning the chat session, and using partitions of the chat session obtained during the verification phase to generate a subject-wise summarization of the chat session and/or to generate multiple chat sessions.