Segmented Conversation History Compression for LLM Context Accuracy

Solution Overview

Problem

Large language models (LLMs) face challenges in maintaining coherence and adherence to context over long conversations due to the computational expense and performance degradation caused by lengthy conversation histories, and existing methods like the sliding window strategy can disrupt continuity or preserve irrelevant information.

Innovation Solution

The conversation history is segmented into multiple segments, with varying levels of compression and detail, and reconstructed for each user request by prioritizing relevant segments with high detail and condensing less pertinent segments, using LLMs to determine relevance and apply appropriate compression levels.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If the entire conversation history is provided to the LLM, then context accuracy is maintained, but computational cost and response time increase significantly

Engineering Contradiction:

Improvecontext accuracyVSAvoidresponse time

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The conversation history is divided into multiple segments based on temporal recency and relevance. The system identifies important transition points in the conversation and creates segments that group related exchanges together. This segmentation allows the LLM to process only necessary portions of the history rather than the entire conversation, reducing computational load while preserving critical context.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different segments of conversation history are assigned different levels of detail and processing priority. Recent and relevant segments are provided in full detail to the LLM, while older or less relevant segments are summarized or omitted. This local quality approach ensures that computational resources are focused on the most important contextual information rather than treating all history uniformly.

Inventive Principle:

Principle #3Local quality

2Reliability

If the entire conversation history is provided to the LLM, then context accuracy is maintained, but computational expense increases

Engineering Contradiction:

Improvecontext accuracyVSAvoidcomputational expense

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The conversation history is divided into multiple segments based on temporal recency and relevance. The system identifies important transition points in the conversation and creates segments that group related exchanges together. This segmentation allows the LLM to process only necessary portions of the history rather than the entire conversation, reducing computational load while preserving critical context.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Different segments of conversation history are assigned different levels of detail and processing priority. Recent and relevant segments are provided in full detail to the LLM, while older or less relevant segments are summarized or omitted. This local quality approach ensures that computational resources are focused on the most important contextual information rather than treating all history uniformly.

Inventive Principle:

Principle #3Local quality

3Productivity

If the sliding window strategy is used to truncate conversation history, then computational cost is reduced, but continuity and context coherence are disrupted

Engineering Contradiction:

Improvecomputational efficiencyVSAvoidcontext coherence

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The conversation history is divided into multiple segments based on temporal recency and relevance. The system identifies important transition points in the conversation and creates segments that group related exchanges together. This segmentation allows the LLM to process only necessary portions of the history rather than the entire conversation, reducing computational load while preserving critical context.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs preliminary analysis of the conversation history to identify relevant segments and their importance before presenting them to the LLM. By pre-processing the history to determine which segments are essential for maintaining context coherence, the system avoids the discontinuities caused by simple sliding window truncation while still reducing the overall input size.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS20260079977A1Segmented and compressed conversation history for large language model (LLM) driven agents

Publication Date: 2026.03.19 SALESFORCE INC

AI summary

Disclosed are some implementations of systems, apparatus, methods and computer program products for recreating a conversation history to be processed by an artificial intelligent (AI) agent. A prompt is divided into a plurality of segments.For at least one segment, one or more corresponding compressed segments are generated and stored. A user query is received. For at least a first segment of the plurality of segments, a level of relevance to the query is estimated. A compressed segment is selected from a set of segments including the compressed segments, based, at least in part, on the level of relevance to the query. A conversation history is recreated using segments including the selected compressed segment. The query and recreated conversation history are provided to the AI agent.