Dynamic LLM Memory Architecture with Confidence-Based Inference Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automated assistants using large language models (LLMs) face challenges in memorizing and utilizing user information effectively, with transient memorization and rigid database storage methods leading to inaccurate responses and increased computational resource use.
Innovation Solution
Implement an inference generator and reconciler to dynamically generate and modify confidence indicators for user inferences based on additional data, allowing for flexible and accurate information management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If information is stored in a database with a high threshold for memorization, then the reliability of stored information is improved, but the adaptability of the automated assistant deteriorates
Solution Approach 1:
The patent implements dynamic confidence indicators that are continuously updated based on new information, allowing the system to adapt its belief in stored information over time. The confidence indicator is not static but evolves as the automated assistant processes additional user inputs and dialogs, enabling flexibility while maintaining reliability through evidence-based updates.
Solution Approach 2:
The system changes the parameter of confidence indication from a binary stored/not-stored state to a continuous confidence indicator that can be modified. This allows the automated assistant to adjust its reliance on stored information based on accumulating evidence, resolving the contradiction between maintaining high reliability and achieving adaptability.
2Adaptability or versatility
If an LLM memorizes information during dialogs, then the adaptability of the automated assistant is improved, but the reliability of information retention deteriorates
Solution Approach 1:
The system implements feedback mechanisms where the automated assistant continuously evaluates new information against stored inferences and updates confidence indicators accordingly. This feedback loop ensures that information retention is reliable by systematically incorporating new evidence while maintaining adaptability through dynamic confidence adjustments.
Solution Approach 2:
The confidence indicator serves as a dynamic mechanism that allows the system to retain information flexibly. Rather than rigidly storing or discarding information, the system dynamically adjusts confidence levels based on ongoing dialogs, achieving both reliable retention and adaptability.
3Reliability
If the automated assistant uses a high threshold for determining factual correctness, then the reliability of memorized information is improved, but the productivity of the automated assistant deteriorates
Solution Approach 1:
The system applies partial action by using confidence indicators to selectively process information. Rather than applying a uniform high threshold to all information, the system processes information proportionally to its confidence level, improving productivity while maintaining reliability for high-confidence inferences.
Solution Approach 2:
The system changes the parameter from a fixed high threshold to a dynamic confidence indicator that varies based on the strength of evidence. This allows the automated assistant to efficiently process high-confidence information while still maintaining reliability standards, thereby improving overall productivity.
4Reliability
If information is rigidly maintained in memory once memorized, then the reliability of stored information is improved, but the adaptability of the automated assistant deteriorates
Solution Approach 1:
The patent implements dynamic confidence indicators that allow stored information to be flexibly updated or discarded based on new evidence. This dynamic approach maintains reliability by systematically evaluating information while achieving adaptability through confidence-based adjustments, directly resolving the contradiction between rigid maintenance and flexibility.
Data Source
AI summary
Processor(s) can receive a first natural language (NL) based input as part of a dialog between a user of a client device and an automated assistant that is accessible at the client device; process, using a first machine learning (ML) model, the first NL based input to generate a first inference associated with the user and a confidence indicator for the first inference, the confidence indicator for the first inference indicative of a degree of confidence that the first inference is true; store the first inference and the confidence indicator for the first inference in a database; process, using a second ML model, the first inference and additional data to generate a modified confidence indicator for the first inference and store the modified confidence indicator in the database; and determine, based at least on the modified confidence indicator for the first inference, whether to process, using a first large language model (LLM), the first inference with a second NL based input to generate a first NL based response.