Natural Language Query Answering With Retrieval and Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative AI models struggle to accurately answer questions about recent internet data and private network-based services due to training data limitations and lack of intent understanding, leading to inefficient resource use and incorrect answers.
Innovation Solution
A network-based service utilizing an aggregator to retrieve passages and QA pairs, a trained LLM for generating answers, a verifier to ensure accuracy, and a watermarking component to protect proprietary content, enhancing answer quality and authenticity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If generative AI models are trained on public internet data, then they can answer general questions, but they cannot answer questions about recent internet data or private network-based services
Solution Approach 1:
The system segments the data source into two parts: public internet data for general knowledge and private network-based service data for specific, recent information. The aggregator component separately retrieves passages from search systems and private data sources, then combines them to form a comprehensive knowledge base for the LLM, enabling the model to answer both general and specific questions accurately.
Solution Approach 2:
The patent introduces an aggregator component as an intermediary between the LLM and data sources. This aggregator retrieves relevant passages from search systems and private network services, processes them, and presents them to the LLM in a structured format. This intermediary layer enables the LLM to access recent and private data without being retrained, improving adaptability while maintaining reliability.
2Productivity
If generative AI models process natural language queries directly, then they can provide quick responses, but they lack intent understanding and generate incorrect answers
Solution Approach 1:
The system performs preliminary actions before the LLM generates answers: the aggregator component pre-retrieves relevant passages from search systems and private data sources, and the verifier component pre-validates the relevance and accuracy of these passages. This preliminary processing ensures that when the LLM receives the query, it has access to contextually relevant information, improving intent understanding without significantly impacting response speed.
Solution Approach 2:
The patent implements a feedback mechanism where the verifier component checks the LLM's generated answers against the retrieved passages and private data. If the answer is incorrect or lacks proper intent understanding, the system can refine the query or retrieve additional passages. This feedback loop improves measurement precision while maintaining productivity through automated validation.
3Ease of operation
If network-based services use LLM to generate answers, then they can provide natural language responses, but they cannot ensure answer accuracy or protect proprietary information
Solution Approach 1:
The patent introduces a verifier component as an intermediary between the LLM and the final output. This verifier checks the generated answers against the retrieved passages and private network-based service data to ensure accuracy. It also validates that proprietary information is appropriately handled and not exposed. This intermediary layer maintains ease of operation by allowing natural language processing while ensuring reliability through automated verification and protection mechanisms.
4Measurement precision
If the system retrieves and processes大量 passages from search systems, then it can improve answer accuracy, but it increases processing time and resource consumption
Solution Approach 1:
The aggregator component implements partial action by retrieving only the most relevant passages from search systems based on the query's intent and context. Rather than processing all available data, it selects a focused subset of passages that are most likely to contain the answer. This approach maintains high answer accuracy while significantly reducing processing time and resource consumption compared to exhaustive search methods.
Data Source
AI summary
Systems and methods are provided for a natural language question answering service to provide answers to natural language questions regarding network-based services or computing domains. The natural language question answering service may receive the natural language question from a customer computing device. An aggregator of the natural language question answering service can retrieve passages from search systems based on the question and generate a prompt. A large language model (LLM) of the natural language question answering service may receive the prompt and provide an answer. The answer may be verified by a verifier of the natural language question answering service. Attribution may be applied to the answers and retrieved passages to produce references, inline citations, and similar questions. A watermarking module of the natural language question answering service may watermark the answer if it is verified.