Natural Language Query Answering With Retrieval and Answer Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative AI models struggle to accurately answer questions about recent internet data and private network-based services due to training data limitations and lack of intent understanding, leading to inefficient resource use and incorrect answers.
Innovation Solution
A network-based service utilizing an aggregator to retrieve passages and QA pairs, a trained LLM for generating answers, a verifier to ensure accuracy, and a watermarking component to protect proprietary content, enhancing answer quality and authenticity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If generative AI models are trained on public internet data, then they can answer general questions, but they cannot answer questions about recent internet data or private network-based services
Solution Approach 1:
The system performs preliminary actions by retrieving relevant passages from search systems and incorporating them into the prompt before the LLM generates answers. This ensures the model has access to up-to-date and proprietary information without retraining, resolving the contradiction between adaptability to recent data and answer reliability.
Solution Approach 2:
The patent introduces an intermediary retrieval-augmented generation framework that mediates between the LLM and external data sources. The search systems and passage retrieval mechanisms act as intermediaries, providing the model with access to recent and private data while maintaining answer accuracy through verified context.
2Productivity
If generative AI models process natural language queries directly, then they respond quickly, but they generate incorrect answers due to lack of intent understanding
Solution Approach 1:
The patent segments the query processing into distinct components: intent analysis, passage retrieval, prompt construction, and answer generation. This segmentation allows each component to specialize, improving both response speed through efficient processing pipelines and accuracy through dedicated intent understanding and verified context retrieval.
Solution Approach 2:
The system implements feedback mechanisms where the LLM's generated answers are evaluated against the retrieved passages and original queries. This feedback loop ensures answer accuracy by verifying that generated responses are supported by the context, while the structured feedback maintains efficient processing speeds.
3Extent of automation
If network-based services use generative AI models, then they provide automated responses, but they waste resources on multiple question-answer turns without correct answers
Solution Approach 1:
The system performs preliminary intent analysis and context retrieval before committing to full LLM generation. This preliminary action filters out queries that can be answered from context without extensive generation, reducing computational resource waste while maintaining high automation for complex queries that truly require generative capabilities.
Solution Approach 2:
The patent applies partial action by retrieving only the necessary passages relevant to each query rather than processing all available data. This selective approach reduces computational resources wasted on unnecessary processing while maintaining automated response capabilities for queries requiring full generative processing.
Data Source
AI summary
Systems and methods are provided for a natural language question answering service to provide answers to natural language questions regarding network-based services or computing domains. The natural language question answering service may receive the natural language question from a customer computing device. An aggregator of the natural language question answering service can retrieve passages from search systems based on the question and generate a prompt. A large language model (LLM) of the natural language question answering service may receive the prompt and provide an answer. The answer may be verified by a verifier of the natural language question answering service. Attribution may be applied to the answers and retrieved passages to produce references, inline citations, and similar questions. A watermarking module of the natural language question answering service may watermark the answer if it is verified.