Type Evaluation in Question-Answering Systems
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing question answering systems face challenges in type evaluation for open-domain natural language questions, where the lexical answer type and candidate answer often do not exist in the same taxonomy, leading to inefficiencies in determining valid answers.
Innovation Solution
A dynamic infrastructure and methodology that uses syntactic analysis to extract type instances of the lexical answer type from a text corpus, maps them to multiple taxonomies, and clusters them into significant supertypes, allowing for scoring of candidate answers without requiring a combined taxonomy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If a single taxonomy is used for type evaluation, then the system structure is simple, but the mapping success rate between LAT and candidate answer decreases
Solution Approach 1:
The patent divides the taxonomy resource into multiple separate domain-specific taxonomies (e.g., DBpedia, Freebase, Wikidata) rather than using a single unified taxonomy. Each taxonomy maintains its own structure and characteristics, allowing LAT and candidate answers to be mapped to appropriate taxonomies based on their domain, thereby increasing mapping success rate without requiring a complex merged taxonomy structure.
Solution Approach 2:
The patent creates a universal type evaluation framework that can work with multiple different taxonomies simultaneously. The framework includes domain identification mechanisms and adaptive mapping strategies that enable the system to select and adapt to the most suitable taxonomy for each specific LAT-candidate answer pair, achieving high mapping success rate across diverse domains without consolidating all taxonomies into one structure.
2Reliability
If multiple taxonomies are used to increase mapping chances, then the mapping success rate improves, but the system complexity increases
Solution Approach 1:
The patent implements a dynamic taxonomy selection mechanism that adapts to each query context. The system identifies the domain of the LAT and candidate answer, then dynamically selects the most appropriate taxonomy for mapping. This dynamic adaptation allows the system to use multiple taxonomies when needed while avoiding unnecessary complexity for simpler cases, maintaining efficiency while improving mapping success rate.
Solution Approach 2:
The patent introduces domain identification and adaptive mapping strategies as intermediary components between the LAT-candidate answer pair and the multiple taxonomies. These intermediaries analyze the semantic domain and select the most suitable taxonomy, acting as a mediator that simplifies the interaction between the system and multiple taxonomies, thereby managing complexity while maximizing mapping success rate.
3Ease of operation
If taxonomies are merged to create a unified structure, then the type evaluation becomes simpler, but information loss occurs during merging
Solution Approach 1:
The patent maintains separate, unmerged taxonomy structures for different domains (DBpedia, Freebase, Wikidata, etc.), preserving the original information and structural characteristics of each taxonomy. Instead of merging them into a unified structure that would cause information loss, the system segments the taxonomies and selects the appropriate one for each mapping task, thereby avoiding information loss while keeping the evaluation process manageable through automated selection.
Solution Approach 2:
The patent creates virtual mappings and projections of LAT and candidate answers onto different taxonomy structures without physically merging the taxonomies. The system copies the relevant semantic information into the appropriate taxonomy framework for evaluation, then retrieves the results. This copying approach allows the system to operate with multiple taxonomies as if unified, while preserving the original taxonomy information without actual merging or information loss.
Data Source
AI summary
A system and method for automatically mapping LATs and candidate answers to multiple taxonomies without a need to merge these taxonomies. The method includes using a syntactic analysis of a corpus to extract all type instances of the LAT. The extracted instances are then mapped to a given taxonomy and clustered in a set of supertypes. Each supertype receives a score based on the coverage of LAT instances in the corpus. The method includes mapping the candidate answer to the same taxonomy to determine if the candidate answer is an instance of a significant supertype. Then the score of a candidate answer is obtained by aggregating or taking a maximum of the score of the matched significant supertypes. This score evaluates the type match between the LAT and candidate answer for a taxonomy. Multiple taxonomies can be used to increase the chance of LAT and candidate answer mapping.