LLM Query Expansion Using Synthetic Queries and Few-Shot Prompts
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing information retrieval systems face challenges in handling variable queries without labeled training data, particularly in zero-shot learning scenarios, leading to suboptimal retrieval of relevant documents due to issues like missing keywords, ambiguity, or specificity.
Innovation Solution
Utilizing a large language model (LLM) to generate synthetic queries related to a corpus of documents, followed by an adaptive few-shot prompting technique to refine user queries, incorporating relevant information and improving query expansion through an amended query generation process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If zero-shot learning is used for information retrieval without labeled training data, then the system can handle variable queries in fields with no relevant labeled training data, but the retrieval accuracy deteriorates due to missing keywords, ambiguity, or specificity in queries
Solution Approach 1:
The system performs preliminary action by generating synthetic queries and their corresponding relevant documents before the actual retrieval task. The LLM generates synthetic queries from documents and creates query-document pairs that can be used for training or as reference examples, enabling the system to improve retrieval accuracy for variable queries without requiring pre-existing labeled training data for each specific domain
Solution Approach 2:
The system creates synthetic query-document pairs that copy and replicate the structure and quality of ideal labeled training data. By generating these synthetic examples using the LLM, the system obtains sufficient training material or reference examples to improve retrieval accuracy while maintaining the ability to handle variable queries in domains without existing labeled data
2Measurement precision
If synthetic queries are generated for each document using LLM, then query expansion and retrieval accuracy improve, but computational time and processing resources increase
Solution Approach 1:
The system applies partial action by generating synthetic queries for a subset of documents rather than all documents, or by generating a limited number of synthetic queries per document. This selective approach maintains sufficient retrieval accuracy by focusing computational resources on the most important or relevant documents while reducing overall processing time and resource consumption
3Measurement precision
If adaptive few-shot prompting is used to generate amended queries, then the quality and relevance of retrieved documents improve, but the complexity of the query processing system increases
Solution Approach 1:
The system uses an intermediary approach by introducing a dedicated module or component that handles the adaptive few-shot prompting and synthetic query generation. This intermediary layer separates the complex LLM-based query expansion logic from the core retrieval system, allowing the complexity to be managed and isolated while maintaining improved document relevance through the sophisticated query amendment process
Data Source
AI summary
Systems and methods for performing query expansion. A computing system uses a large language model (LLM) to generate one or more synthetic queries for each document of a set of documents. For a user query, the computing system: selects one or more of the synthetic queries related to the user query; generates an adaptive few-shot prompt to instruct the LLM to generate a response to the query, wherein the adaptive few-shot prompt comprises an example query-response pair for each of the selected one more synthetic queries; provides the adaptive few-shot prompt to the LLM as an input; and generates an amended query based on the output of the LLM in response to the adaptive few-shot prompt.