Permanence-Based Confidence Ranking in Question Answering Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Intelligence statements in question answering systems often lose confidence over time due to factors like age and source reliability, requiring a method to assess the permanence of data and adjust confidence scores accordingly.
Innovation Solution
The approach identifies permanence data for terms in a question answering system, establishing time-based confidence scores for candidate answers based on the permanence of the data, which influences the ranking and scoring of answers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Duration of action of stationary object
If intelligence statements are retained in the system for longer periods, then the system has more data to work with, but the confidence in those statements decays over time
Solution Approach 1:
The system pre-assigns permanence values to different types of intelligence data before they are stored. This preliminary classification allows the system to automatically apply appropriate confidence decay rates to each data type, preparing the groundwork for future confidence calculations without requiring real-time analysis of data characteristics.
Solution Approach 2:
The system dynamically adjusts confidence scores based on the age of intelligence statements by applying permanence-based decay functions. As statements grow older, their confidence scores automatically decrease according to their assigned permanence characteristics, allowing the system to maintain data retention while managing confidence levels through parameter transformation.
2Measurement precision
If the system discounts older intelligence statements, then confidence accuracy is improved, but the loss of potentially valuable information increases
Solution Approach 1:
The system applies different permanence values and confidence decay rates to different types of intelligence data based on their specific characteristics. Rather than uniformly discounting all older statements, the system treats each data type locally according to its permanence profile, preserving valuable information while maintaining confidence accuracy for each category.
3Measurement precision
If permanence data is assigned to all terms in the corpus, then confidence ranking accuracy is improved, but the system complexity increases
Solution Approach 1:
The system divides the corpus of intelligence data into distinct segments or categories, each assigned a specific permanence value. This segmentation allows the system to manage complexity by handling different data types separately with their own confidence decay parameters, rather than treating the entire corpus as a single complex entity.
Data Source
AI summary
An approach is provided to identify permanence data corresponding to terms included in a corpus of a question answering (QA) system. Based on the identified permanence, a time-based confidence of each of the terms is established. Terms are identified as a plurality of candidate answers to a question posed to the QA system. Each of the plurality of candidate answers are scored with the scoring being at least partially based on the time-based confidence established for each of the terms.