AI Agent Multi-Turn Memory Using Summaries and Original Messages
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing AI agents using multi-turn methods face issues such as increased message length exceeding context length, higher costs due to increased input tokens, and data exploration tasks being repeatedly performed, leading to inefficiencies and potential information loss.
Innovation Solution
The AI agent maintains only the most recent j turns as original messages and uses summary messages for previous turns, storing original messages externally to minimize data exploration and reduce latency, while ensuring accurate responses by accessing original messages when needed.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multi-turn method is used to provide accurate answers by repeating questions and answers, then answer accuracy is improved, but message length increases exceeding context length and cost increases due to increased input tokens
Solution Approach 1:
The patent segments the chat history into two distinct parts: original messages (keeping only the most recent j turns) and summary messages (compressing earlier turns). This segmentation allows the system to maintain answer accuracy by preserving recent detailed interactions while controlling message length by summarizing older content, thus resolving the contradiction between accuracy and length.
Solution Approach 2:
The patent changes the parameter of message representation by introducing a hybrid format where some messages are kept in original form (recent j turns) and others are transformed into summaries (earlier turns). This parameter change enables the system to adapt message length dynamically while maintaining the necessary information for accurate responses.
2Measurement precision
If multi-turn method is used to provide accurate answers by repeating questions and answers, then answer accuracy is improved, but cost increases due to increased input tokens
Solution Approach 1:
The patent extracts and removes redundant information from the chat history by summarizing earlier turns and keeping only essential recent interactions in original form. This extraction reduces the total number of input tokens fed to the LLM, thereby lowering the cost while maintaining answer accuracy by preserving critical information.
Solution Approach 2:
By changing the representation parameter of historical messages from full original form to summarized form for earlier turns, the system reduces input token count and associated costs while maintaining sufficient information for accurate responses.
3Reliability
If data exploration task is repeatedly performed to acquire data from database, then data availability is ensured, but latency increases and efficiency decreases
Solution Approach 1:
The patent performs preliminary action by maintaining relevant data from original messages in memory (specifically in the chat history) rather than relying solely on repeated database queries. This preliminary preparation of data reduces latency by having information readily available when needed, while ensuring data availability through the preservation of original messages.
Solution Approach 2:
The patent creates copies of relevant data by storing original messages in the chat history, making them immediately accessible without requiring repeated database exploration tasks. This copying mechanism reduces latency while ensuring data availability for generating accurate responses.
Data Source
AI summary
There is provided a method for providing a service by an artificial intelligence(AI) agent with a multi-turn. A cost for performing tasks of an AI agent is reduced and a speed of performing the tasks is improved by maintaining only messages for the most recent j turns as originals and using summary message for each of messages of previous turns of the most recent j turns. Further, a risk of information loss in a multi-turn of the AI agent is eliminated by storing original messages corresponding to the summary messages in an external storage and thus allowing the original messages to be accessed when necessary.