Retrieval-Augmented Thought for Accurate Agentic LLM Responses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing retrieval augmented generation (RAG)-based systems are computationally inefficient and inaccurate due to excessive resource consumption and reliance on overgeneralized, biased databases, leading to inaccurate and irrelevant responses, especially in specific domains like genealogy.
Innovation Solution
A model-modification system utilizing retrieval-augmented thought (RAT) that accesses a RAT-store database of expert-generated instructions and examples to generate responses without fine-tuning, employing a dynamic graph-based agentic framework with a RAT replanner to execute processes sequentially and improve accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If fine-tuning is performed to generate task-specific responses, then response accuracy improves, but computational resource consumption increases excessively
Solution Approach 1:
The system performs preliminary retrieval of relevant information from external databases and knowledge bases before generating responses. By pre-fetching and caching task-specific information, the system avoids the need for computationally expensive fine-tuning while maintaining response accuracy. The retrieval-augmented generation approach prepares relevant context in advance, eliminating the need for repeated fine-tuning operations.
Solution Approach 2:
The patent introduces an intermediary retrieval layer between the user query and the language model. This intermediary component fetches relevant information from external sources and presents it to the model as context, allowing the model to generate accurate responses without requiring task-specific fine-tuning. The intermediary acts as a bridge that provides necessary information without the computational cost of fine-tuning.
2Adaptability or versatility
If wide-ranging databases are used for broad coverage, then output versatility improves, but response accuracy deteriorates due to overgeneralization and hallucinations
Solution Approach 1:
The system applies local quality by retrieving specific, task-relevant information from databases rather than relying on the model's general knowledge. Each response is augmented with locally relevant data fetched based on the specific query context, ensuring high accuracy for that particular task while maintaining the ability to handle diverse queries through the retrieval mechanism.
Solution Approach 2:
The system dynamically adapts the retrieval strategy based on the specific query and task requirements. Rather than using a static, broad knowledge base, the system dynamically fetches relevant information tailored to each query, allowing it to maintain versatility across different tasks while ensuring accuracy through context-specific information retrieval.
3Reliability
If multiple iterations of fine-tuning are performed, then model performance improves, but training time and computational cost increase significantly
Solution Approach 1:
Instead of repeatedly fine-tuning the model through multiple iterations, the system creates copies of relevant information from external databases and knowledge bases. These copied contexts are injected into the model prompt for each query, achieving performance improvement without the time-consuming fine-tuning process. The system copies necessary information rather than retraining the model.
Solution Approach 2:
The system performs preliminary retrieval and preparation of relevant information before query processing. By pre-fetching and organizing task-specific data, the system eliminates the need for multiple fine-tuning iterations, achieving model performance improvement through advance information preparation rather than repeated training cycles.
Data Source
AI summary
The present disclosure is directed toward systems, methods, and non-transitory computer-readable media for a retrieval augmented thought approach to generating responses using neural networks. For example, the disclosed systems can receive a prompt instruction a large language model to generate a response. The disclosed systems can identify a retrieval-augmented-thought (RAT) store item corresponding to the prompt from among RAT-store items stored in a repository accessible by the large language model. In some cases, the disclosed systems can retrieve the RAT-store item and can generate a response using the large language model to execute processes defined by the RAT-store item. Further, in some cases, the disclosed systems utilize a dynamic graph-based agentic framework that incorporates one or more functional adapters and a RAT replanner to execute a sequence of processes indicated by a RAT-store item in a sequential manner.