Lexical Answer Type Confidence Estimation in QA Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current question answering systems face challenges in accurately answering factual questions due to the lack of a computer program capable of understanding queries and extracting correct answers from structured or unstructured information sources, often resulting in wrong lexical answer types (LATs) that reduce system performance.
Innovation Solution
A computing infrastructure and methodology that automatically estimates the confidence of detected LATs, combining this confidence score with the degree of match between the LAT and candidate answers to provide a more accurate overall score for the candidate answer, thereby improving the quality of answer generation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Extent of automation
If the system uses automated LAT detection to identify answer types from queries, then the speed and automation of question answering is improved, but the accuracy of LAT detection deteriorates leading to wrong lexical answer types
Solution Approach 1:
The system implements feedback by computing a confidence score for the detected LAT and using this score to evaluate and rank candidate answers. The confidence score feedback mechanism allows the system to identify when LAT detection may be erroneous and adjust answer selection accordingly, resolving the contradiction between automation and accuracy.
Solution Approach 2:
The system changes the parameter of LAT detection by introducing a confidence score dimension. Instead of relying solely on automated LAT detection results, the system evaluates the confidence level of detected LATs and uses this additional parameter to improve overall answer accuracy, effectively managing the trade-off between automation speed and detection precision.
2Device complexity
If the system relies solely on match degree between LAT and candidate answer to score answers, then the simplicity of the scoring mechanism is maintained, but the accuracy of answer selection deteriorates due to LAT detection errors
Solution Approach 1:
The system enhances the scoring mechanism by incorporating confidence score feedback. Instead of using only the match degree between LAT and candidate answer, the system multiplies the match degree by the confidence score of the detected LAT, creating a more accurate composite score that accounts for LAT detection uncertainty while maintaining computational efficiency.
3Productivity
If the system does not account for LAT detection confidence, then the computational efficiency is maintained, but the reliability of answer generation deteriorates
Solution Approach 1:
The system improves reliability by adding a confidence score parameter to the LAT detection process. This parameter change allows the system to weigh candidate answers based on both match degree and LAT detection confidence, significantly improving answer generation reliability while maintaining computational efficiency through efficient score computation and ranking.
Data Source
AI summary
A system, method and computer program product for automatically estimating the confidence of a detected LAT to provide a more accurate overall score for an obtained candidate answer. A confidence “score” or value of each detected LAT is obtained, and the system and method performs combining the confidence score with a degree of match between a LAT and an AnswerType of the candidate answer to provide improved overall score for the candidate answer.