Backup-Data Vector Retrieval for Context-Rich LLM Responses
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing retrieval augmented generation (RAG) techniques that leverage live enterprise data for improving large language model (LLM) responses require significant IT investment and can disrupt the live data, posing potential risks and complications.
Innovation Solution
Utilizing backup data managed by a data management system (DMS) to generate vectors from snapshots, which are stored in a vector database, allowing retrieval augmented generation (RAG) without affecting the live system, and enabling more streamlined and customizable communication applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If live enterprise data is used for retrieval augmented generation, then the accuracy and relevance of LLM responses is improved, but the system complexity and IT investment requirements increase significantly
Solution Approach 1:
The patent creates a copy of enterprise data from backup systems to populate the vector database. Instead of directly accessing live enterprise data systems, the invention uses replicated data copies that are periodically updated from backup sources, thereby reducing system complexity while maintaining response accuracy.
Solution Approach 2:
The patent introduces a data management system as an intermediary layer between the LLM and enterprise data. This mediator handles data retrieval, processing, and vectorization, simplifying the overall system architecture by centralizing data access operations and reducing direct connections to complex enterprise systems.
2Measurement precision
If live enterprise data is accessed for RAG, then contextual relevance is improved, but disruption to the live system occurs
Solution Approach 1:
The patent performs data retrieval and vectorization operations on backup data before they are needed for LLM queries. By pre-processing and storing vectors from backup data in advance, the system avoids real-time disruptions to live systems while maintaining the ability to provide contextually relevant responses when queries are made.
Solution Approach 2:
The patent uses copies of enterprise data from backup systems rather than accessing live data sources. These data copies are periodically updated and processed independently, allowing RAG operations to proceed without interfering with the stability and reliability of the live enterprise systems.
3Reliability
If backup data is used for vector database population, then system disruption is avoided, but data freshness may be compromised
Solution Approach 1:
The patent implements periodic updates of the vector database from backup data sources. Rather than continuous real-time synchronization, the system periodically refreshes its vector database with new backup data, balancing system stability with reasonable data freshness for RAG operations.
Data Source
AI summary
Methods, apparatuses, and computer readable media are configured to perform operations comprising: receiving, by a data management system (DMS), a query for a large language model (LLM) via an application; and retrieving, by the DMS and based at least in part on contextual information associated with the query, information from a vector database accessible to the DMS, wherein the vector database comprises one or more vectors comprising data associated with one or more snapshots obtained by the DMS of a computing system, wherein a prompt for the LLM is generated based at least in part on the query and the information, and a response to the query is provided based at least in part on the prompt and the LLM.