Question Answering System Follow-on Inquiries for Accuracy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Question answering systems often produce incorrect responses due to missing information and marginal evidence, leading to low confidence scores and inefficiencies in providing accurate answers.
Innovation Solution
The system generates follow-on inquiries to external sources to gather missing information, which is then validated and integrated into the QA system to enhance confidence scores and improve answer accuracy by using crowd sourcing and external expert communities.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If the QA system uses pre-structured databases or document collections to answer questions, then the system can provide automated answers, but the system produces incorrect responses due to missing information and marginal evidence
Solution Approach 1:
The system generates follow-on inquiries based on failures in answering original questions, uses responses from these inquiries to improve the corpus of data, and continuously refines the QA system's knowledge base. This feedback loop transforms incorrect answers into learning opportunities, systematically improving reliability while maintaining automation.
Solution Approach 2:
The system proactively identifies missing information and generates follow-on inquiries before finalizing answers. By anticipating knowledge gaps and seeking additional information in advance, the system prevents incorrect answers from being produced in the first place, improving reliability while maintaining automated operation.
2Reliability
If the QA system generates follow-on inquiries to external sources, then the accuracy of answers is improved, but the complexity of the system increases
Solution Approach 1:
The system uses an intermediary layer of follow-on inquiries that mediates between the QA system and external knowledge sources. Instead of directly accessing complex external databases, the system generates simplified inquiry representations that can be answered by external sources, then integrates these answers back into the corpus. This intermediary approach improves accuracy while managing complexity.
Solution Approach 2:
The system segments the knowledge acquisition process into distinct components: identifying missing information, generating follow-on inquiries, receiving responses from external sources, and integrating answers into the corpus. This segmentation allows each component to be optimized independently, improving overall accuracy while keeping individual components manageable in complexity.
3Adaptability or versatility
If the system uses marginal evidence to generate candidate answers, then the system can provide answers with limited information, but the confidence scores remain low
Solution Approach 1:
The system uses responses from follow-on inquiries as feedback to refine and re-score candidate answers. By iteratively seeking additional evidence through follow-on inquiries and using these responses to update confidence scores, the system transforms low-confidence answers based on marginal evidence into high-confidence answers backed by robust evidence, while maintaining the ability to generate answers from limited initial information.
Data Source
AI summary
A computerized device for enhancing the accuracy of a question-answer system is disclosed. The computerized device comprises a question-answer system comprising software for performing a plurality of question answering processes. A receiver receives a question into the question-answer system. A processor that generates a plurality of candidate answers to the question is connected to the question-answer system. The processor determines a confidence score for each of the plurality of candidate answers. The processor evaluates sources of evidence used to generate the plurality of candidate answers. The processor identifies missing information from a corpus of data. The missing information comprises any information that improves a confidence score for a candidate answer. The processor generates at least one follow-on inquiry based on the missing information. A network interface outputs the at least one follow-on inquiry to external sources separate from the question-answer system.