Evidence Evaluation System Using Question Answering for Hypothesis Verification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current intelligence analysis systems rely on extracted patterns to verify facts, which are costly and limited in unrestricted domains, and require pre-defined question types, making it difficult to confirm or refute hypotheses due to the vastness of data and varied expressions of facts.
Innovation Solution
A system and method for evidence evaluation based on question answering that converts information into questions, determines answers, and marks facts as supported or refuted, without relying on pre-extracted patterns, allowing for continuous confirmation and refutation of assertions in real-time, and enabling collaboration and incremental evaluation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If extracted patterns are used to verify facts, then hypothesis confirmation is possible, but the system is costly and limited in unrestricted domains
Solution Approach 1:
Instead of extracting patterns from text to verify hypotheses, the system inverts the approach by using a question-answering system to directly query the text corpus for evidence supporting or refuting hypotheses. This eliminates the need for complex pattern extraction while maintaining hypothesis verification capability.
Solution Approach 2:
The patent introduces a question-answering system as an intermediary between the hypothesis verification task and the text corpus. This QA system automatically generates relevant questions from hypotheses and searches for answers in the text, serving as a mediator that simplifies the verification process without requiring complex pattern matching.
2Measurement precision
If extracted patterns are used to verify facts, then specific relations can be identified, but the specification of extraction patterns is costly
Solution Approach 1:
The system enables self-service by automatically generating questions from hypotheses and autonomously searching the text corpus for answers. This eliminates the manual time-consuming process of specifying extraction patterns, as the QA system adapts to different hypotheses dynamically without requiring pre-defined pattern specifications.
3Ease of operation
If pre-defined question types are used, then structured analysis is possible, but it is difficult to confirm or refute hypotheses due to varied expressions of facts
Solution Approach 1:
The system transitions from static pre-defined question types to dynamic question generation. The QA system adapts its questions based on the specific hypotheses being tested and the content of the text corpus, allowing it to handle varied expressions of facts while maintaining structured analysis through the systematic question-answering process.
Data Source
AI summary
An evidence evaluation method and system based on question answering converts a report of analyzed information and/or a model of information into a collection of questions, determines answers for the collection of questions. A fact in the report is marked as being supported if one or more of the answers for the collection of questions support the fact. A fact in the report of analyzed facts is marked as being refuted if one or more of the answers for the collection of questions refute the fact. The method and system also may collect the answers as evidence and add the evidence to the model of information to create an updated model of information. The steps may be repeated using the updated report and updated model.