Verified LLM Query Answering With Retrieval and Watermarking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative AI models struggle to answer recent internet data lack the ability to answer questions regarding private data (e.g., who won the 2021 NBA Finals?) and questions about particular network-based services regarding private information to train a generative AI model accurately, but they lack the ability to answer questions regarding private data (e.g., information specific to, and regarding, a network-based storage service provider's internal troubleshooting guidelines, a system's internal FAQs), and they lack the ability to answer questions regarding private network-based services regarding network-based storage service provider's internal troubleshooting guidelines, a system's internal FAQs), and they lack the ability to answer questions regarding private network-based services regarding network-based storage service provider's internal troubleshooting guidelines, a system's internal frequently asked questions (FAQs).
Innovation Solution
A network-based service utilizing an aggregator to retrieve passages and question-answer pairs from search systems, a large language model (LLM) for generating answers, and a verifier to ensure accuracy, with watermarking to protect proprietary information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If generative AI models are trained on public internet data, then they can answer general knowledge questions, but they cannot answer questions about private data or recent internet data
Solution Approach 1:
The patent introduces an intermediary component (search system/aggregator) that acts as a bridge between the generative AI model and private/recent data sources. The search system retrieves relevant passages from private data sources and recent internet data, then provides them to the LLM as context, enabling the model to answer questions about private data without being directly trained on it.
Solution Approach 2:
The system performs preliminary actions by pre-retrieving and storing relevant passages from private data sources and recent internet data before the LLM needs to answer questions. The aggregator component proactively gathers and organizes this information, making it readily available when queries are submitted, thus enabling the model to access private and recent information on demand.
2Adaptability or versatility
If a network-based service uses private information to train a generative AI model, then the model can answer service-specific questions, but the proprietary information may be compromised
Solution Approach 1:
The search system serves as an intermediary that allows the LLM to access private service-specific information without direct training on it. The system retrieves passages from private data sources through controlled search operations rather than incorporating the data into the model's training set, thus maintaining proprietary information security while enabling service-specific question answering.
Solution Approach 2:
Instead of training the model on private proprietary information (which would create a permanent copy in the model weights), the system creates temporary copies of relevant passages and provides them as context to the LLM during inference. This allows the model to utilize private information for answering questions without permanently storing or exposing the proprietary data.
3Productivity
If generative AI models are used without verification, then they can generate answers quickly, but they may produce inaccurate or hallucinated responses
Solution Approach 1:
The patent implements a feedback mechanism where the generated answer is verified against the retrieved passages and the original query. The verification component checks whether the answer is supported by the context and does not contain hallucinations, providing feedback that confirms or corrects the generated response before final output.
Solution Approach 2:
The system performs preliminary verification of the generated answer before presenting it to the user. The verification component proactively checks the answer's accuracy and consistency with the retrieved passages in advance, ensuring that only verified, accurate answers are outputted, thus maintaining both speed and reliability.
Data Source
AI summary
Systems and methods are provided for a natural language question answering service to provide answers to natural language questions regarding network-based services or computing domains. The natural language question answering service may receive the natural language question from a customer computing device. An aggregator of the natural language question answering service can retrieve passages from search systems based on the question and generate a prompt. A large language model (LLM) of the natural language question answering service may receive the prompt and provide an answer. The answer may be verified by a verifier of the natural language question answering service. Attribution may be applied to the answers and retrieved passages to produce references, inline citations, and similar questions. A watermarking module of the natural language question answering service may watermark the answer if it is verified.