RAG Function Calling for Multi-Source Data Retrieval Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing retrieval augmented generation (RAG) systems are limited by the breadth and relevance of documents stored within a single document index, leading to reduced performance and computational inefficiencies when interacting with multiple knowledge sources, and struggle with accurate out-of-domain responses due to complex instructions and high error rates.
Innovation Solution
Implement a RAG assistant that leverages function calling to delegate data source selection to a generative AI model, allowing it to select relevant data sources and execute function calls to retrieve data chunks, while separating the assessment of relevance and response generation into distinct prompts to reduce hallucinations and latency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a single document index is used to store all knowledge sources, then the system structure is simple, but the breadth and relevance of retrieved documents are limited
Solution Approach 1:
The patent divides the single document index into multiple separate data sources, each storing specific knowledge domains. The RAG assistant maintains a function list with function definitions that map to these separate data sources, enabling selective retrieval from multiple sources based on the query topic while maintaining manageable system architecture through structured organization.
Solution Approach 2:
The patent introduces a function list as an intermediary layer between the RAG assistant and multiple data sources. The function list includes function definitions with function descriptors that describe each data source's content, allowing the RAG assistant to select appropriate functions (data sources) without directly managing complex multi-source connections.
2Adaptability or versatility
If complex instructions are provided to the LLM for multi-source retrieval, then the system can access multiple knowledge sources, but the error rate increases
Solution Approach 1:
The patent segments the retrieval process into distinct functional steps: (1) the RAG assistant generates a function selection instruction prompt with a function list, (2) the LLM selects appropriate functions by evaluating function descriptors, and (3) the RAG assistant executes the selected functions to retrieve data. This segmentation simplifies the LLM's task to function selection rather than complex multi-step retrieval coordination.
Solution Approach 2:
The patent replaces complex mechanical retrieval instructions with a generative approach where the LLM processes natural language function selection based on function descriptors. The function descriptors provide structured information about data sources, enabling the LLM to make intelligent selections without complex procedural instructions.
3Productivity
If the LLM processes both relevance assessment and response generation in a single prompt, then the workflow is simple, but latency increases and hallucinations occur
Solution Approach 1:
The patent segments the LLM's processing into two distinct phases: (1) function selection phase where the LLM evaluates function descriptors and selects relevant data sources, and (2) response generation phase where the LLM generates the final response based on retrieved data. This segmentation reduces latency by avoiding redundant processing and minimizes hallucinations by separating selection and generation functions.
4Reliability
If the LLM evaluates all data sources for relevance, then comprehensive information is retrieved, but computational overhead increases
Solution Approach 1:
The patent applies partial action by having the LLM evaluate only the function descriptors (not all actual data content) to determine relevance. The function descriptors provide sufficient information for the LLM to select appropriate data sources without requiring full evaluation of all data sources, reducing computational overhead while maintaining accuracy.
Data Source
AI summary
A generative artificial intelligence system includes a retrieval augmented generation (RAG) assistant that utilizes function calling to facilitate multi-source data retrieval to enhance user queries transmitted to a large language model (LLM). The RAG assistant Transmits, to the LLM, a function selection instruction prompt that includes conversation history data, a function list including function definitions that each correspond to a data source, and instructions directing the LLM to return a function call to at least one function defined on the function list identified as relevant to the conversation history data based on a corresponding function descriptor. In response to receiving a function selection response from the LLM that includes the function call, the RAG assistant selects and executes a conditional operation based on a name of the at least one function.