Federated Data Retrieval for Decentralized Generative AI Pipelines
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional large language models (LLMs) are unable to incorporate new or proprietary data, and retrieval-augmented generation (RAG) techniques have limited ability to utilize decentralized data sources.
Innovation Solution
A generative AI pipeline that interfaces with a plurality of decentralized data sources, retrieving unstructured and structured data to augment user queries for LLMs, using intent determination and metadata to select relevant data sources and extract information for improved responses.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional RAG techniques are used to supplement user queries with relevant supplied data, then LLMs can provide improved responses, but the ability to identify and use data from decentralized data sources is limited
Solution Approach 1:
The system segments the data source access process into distinct components: a federated data source manager that handles multiple external data sources, an intent classification module that categorizes queries, and a data retrieval module that fetches relevant information. This segmentation enables the system to efficiently manage decentralized data sources while maintaining response accuracy through specialized handling of each data source type.
Solution Approach 2:
The patent introduces a federated data source manager as an intermediary layer between the LLM and multiple external data sources. This intermediary component coordinates query routing, manages data retrieval from diverse sources, and consolidates results, thereby enabling the system to access decentralized data sources effectively while maintaining reliable response generation.
2Adaptability or versatility
If organizations train and operate their own LLMs with new or proprietary data, then the models can incorporate proprietary data, but it is not feasible for most organizations
Solution Approach 1:
The patent extracts the data training function from the LLM itself and separates it into a distinct RAG pipeline component. Instead of training the LLM with proprietary data, the system extracts relevant information from external data sources during query processing and provides it to the pre-trained LLM. This extraction approach enables proprietary data incorporation without the complexity of model training infrastructure.
Solution Approach 2:
The system performs preliminary actions by pre-processing and storing information from external data sources in structured formats before queries are submitted. The federated data source manager pre-establishes connections to multiple data sources and prepares retrieval mechanisms in advance, so that when a query arrives, the relevant proprietary information can be quickly fetched and provided to the LLM without requiring model retraining.
3Ease of operation
If a comprehensive centralized data source is maintained for a generative AI pipeline, then all data can be accessed from one location, but it is less efficient than using decentralized data sources
Solution Approach 1:
The system segments the centralized data repository into multiple specialized external data sources, each optimized for specific types of data. The federated data source manager then segments the query processing workflow to route different types of queries to appropriate data sources. This segmentation enables parallel data retrieval from multiple sources simultaneously, improving productivity while maintaining ease of operation through the unified manager interface.
Solution Approach 2:
The patent transitions from a single-dimensional centralized data storage model to a multi-dimensional federated architecture where data sources are distributed across different dimensions (external sources, different formats, different locations). The intent classification module adds another dimension by categorizing queries along semantic dimensions, enabling efficient routing to appropriate data sources. This dimensional transformation improves retrieval efficiency while the federated manager maintains operational simplicity.
Data Source
AI summary
The systems and methods described herein relate to improvements to generative artificial intelligence systems through the use of generative artificial intelligence pipelines to supply external information to pre-trained large language models for use in answering queries. To improve the efficiency and accuracy of large language models in responding to user queries, according to various aspects described herein, such queries may be modified and augmented with additional relevant information from decentralized data sources. The retrieval-augmented generation pipeline may interface with one or more decentralized data sources and retrieve the external information, including unstructured text and/or structured data.