Multi-Index Query Resolution Under LLM Context Limits
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional query resolution systems are inefficient when dealing with queries that require searching multiple indexes, leading to labor-intensive, time-consuming processes, increased computing resources, and inaccurate responses due to limited processing capabilities of Large Language Models (LLMs).
Innovation Solution
The system processes user queries to identify multiple relevant indexes, summarizes data from these indexes within a permissible content length, and uses existing LLMs to generate comprehensive responses, ensuring contextually relevant data is processed without exceeding the LLM's capacity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data from multiple indexes is retrieved to improve query accuracy, then response reliability improves, but processing time and computational resources increase
Solution Approach 1:
The patent segments the data retrieval process by identifying and retrieving data from multiple specific indexes (e.g., financial data index, operational data index, market data index) rather than searching the entire database. This targeted segmentation allows the system to access relevant information from predetermined locations, improving reliability while controlling processing time through selective retrieval rather than comprehensive search.
2Quantity of substance
If LLM context size is increased to process more data, then data processing capability improves, but computational resources and costs increase
Solution Approach 1:
The patent applies preliminary action by pre-identifying and pre-retrieving data from multiple indexes before the LLM processing step. The system determines which indexes are relevant to the query and retrieves only that specific data in advance, so that when the LLM processes the information, the data is already organized and ready. This eliminates the need for the LLM to handle raw, unorganized data from multiple sources simultaneously, reducing computational requirements while maintaining comprehensive data processing capability.
3Device complexity
If direct LLM integration is used to simplify the system, then device complexity decreases, but response accuracy and context relevance deteriorate
Solution Approach 1:
The patent introduces an intermediary layer between the user query and the LLM. This intermediary component (the query resolution system) first analyzes the query, identifies relevant indexes, retrieves specific data from those indexes, and prepares the data in a format suitable for LLM processing. This intermediary layer handles the complexity of multi-index navigation and data selection, keeping the LLM itself simple while significantly improving response accuracy and context relevance through the structured data retrieval process.
Data Source
AI summary
The present disclosure provides approaches for enhanced query resolution. Initially, a user query is processed to determine a contextual data and identifying one or more indexes from a plurality of indexes. The indexes may be searched to respond to the user query. For each identified index, a permissible content length for a summary of data available in the index is calculated based on the number of indexes determined to be searched and the context size of a query resolution model to be used for processing the user query. Summary of the information available in each index is generated using a summarization model. The sum of content lengths of summaries of each of the indexes does not exceed the context size of the query resolution model. User query and the generated summaries are then provided to the query resolution model for generating a response to the user query.