RAG Function Calling for Multi-Source Retrieval and Lower Hallucination
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing retrieval augmented generation (RAG) assistants in generative AI systems are limited by the breadth and relevance of documents stored within a single document index, leading to reduced performance and increased computational costs when interacting with multiple knowledge sources, and struggle with accurate out-of-domain responses due to complex instructions and high error rates.
Innovation Solution
Implementing a RAG assistant with function calling capabilities to delegate data source selection to a generative AI model, allowing it to select relevant data sources and execute function calls to retrieve data chunks, and employing conditional logic to reduce hallucinations and latency by separating relevance assessment from document mining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a single document index is used to store all knowledge sources, then the system structure is simple, but the retrieval accuracy and relevance decrease when dealing with multiple knowledge sources
Solution Approach 1:
The patent divides the single document index into multiple separate data sources, each with its own index. The system segments knowledge sources by type (e.g., technical documentation, corporate policies, HR policies) and creates separate indexes for each, allowing more precise and relevant retrieval for different query types while maintaining manageable system complexity through modular architecture.
Solution Approach 2:
The patent adds a new dimension to the system architecture by introducing a function selection layer that sits between the query and the data sources. This dimension enables the system to dynamically select appropriate data sources based on query analysis, transforming the flat single-index structure into a multi-layered, adaptive retrieval system.
2Reliability
If all data sources are queried simultaneously to ensure comprehensive coverage, then the retrieval completeness is improved, but the computational latency increases
Solution Approach 1:
The patent performs preliminary action by analyzing the query and predicting which data sources are most likely to contain relevant information before actually executing the retrieval. The function selection model pre-identifies suitable data sources based on query characteristics, allowing the system to query only the necessary sources rather than all available sources, thus reducing latency while maintaining completeness.
Solution Approach 2:
The system employs self-service through the function selection model that autonomously determines which data sources to query based on the query content. This self-service mechanism eliminates the need for manual configuration or exhaustive querying of all data sources, enabling the system to automatically optimize its retrieval strategy for each query.
3Measurement precision
If complex instructions are provided to handle out-of-domain responses, then the response accuracy is improved, but the error rate increases due to model confusion
Solution Approach 1:
The patent segments the instruction set into distinct, clear categories that guide the model through specific decision pathways. Instead of providing one complex set of instructions, the system breaks down the guidance into separate functional categories (e.g., domain matching, data source selection, response generation) that reduce cognitive load and minimize confusion, thereby lowering error rates while maintaining accuracy.
Solution Approach 2:
The patent introduces an intermediary function selection model that acts as a mediator between the query and the main LLM. This intermediary analyzes the query, selects appropriate data sources, and prepares context before passing everything to the LLM. This intermediary layer simplifies the instructions given to the LLM by pre-processing and structuring the input, reducing the complexity of instructions needed and thereby reducing errors.
Data Source
Figure 1
Figure 2A
Figure 2B
AI summary
A generative artificial intelligence system includes a retrieval augmented generation (RAG) assistant that utilizes function calling to facilitate multi-source data retrieval to enhance user queries transmitted to a large language model (LLM). The RAG assistant transmits, to the LLM, a function selection instruction prompt that includes conversation history data, a function list including function definitions that each correspond to a data source, and instructions directing the LLM to return a function call to at least one function defined on the function list identified as relevant to the conversation history data based on a corresponding function descriptor. In response to receiving a function selection response from the LLM that includes the function call, the RAG assistant selects and executes a conditional operation based on a name of the at least one function.