Semi-structured Data Answer Extraction via Tree Probability Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing question answering (QA) systems face difficulties in efficiently extracting relevant answers from semi-structured data sources, such as HTML documents, due to the lack of structural organization, which makes it challenging to identify correct answers amidst unrelated information.
Innovation Solution
The proposed method organizes HTML data into a tree structure, allowing for the computation of three probability models: answer prior, answer likelihood, and answer context models, to determine the relevance of candidate answers based on their structure, content, and context within the global structure, and weights these probabilities for accurate answer retrieval.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If QA systems search through semi-structured data sources without structural organization, then they can access all potential answers, but the accuracy of answer extraction deteriorates due to inability to identify correct answers amidst unrelated information
Solution Approach 1:
The patent segments the semi-structured data into a hierarchical tree structure with global and local structures. This segmentation organizes the unorganized data into manageable components (nodes and edges) with defined relationships, enabling the system to navigate and extract answers systematically rather than searching through all data indiscriminately.
Solution Approach 2:
The patent introduces probability models as intermediary computational mechanisms that mediate between the raw semi-structured data and the final answer extraction. These models (answer prior, answer likelihood, answer context) act as intermediaries that evaluate candidate answers based on structural, content, and contextual features, thereby improving extraction accuracy without requiring manual analysis of all data.
2Measurement precision
If QA systems organize data into structured formats, then answer extraction accuracy improves, but the complexity of processing semi-structured data increases
Solution Approach 1:
The patent performs preliminary organization of semi-structured data into a tree structure before answer extraction. By pre-processing the data into a hierarchical format with defined global and local structures, the system prepares the data in advance for efficient querying, reducing the computational burden during actual answer extraction operations.
Solution Approach 2:
The patent applies different processing approaches to different parts of the data structure. The global structure provides overall organizational context while local structures around candidate answers provide detailed evaluation criteria. This local quality approach allows the system to focus computational resources on relevant portions of the data rather than uniformly processing entire datasets.
3Measurement precision
If multiple probability models are computed for candidate answers, then the accuracy of answer selection improves, but the computational time and complexity increase
Solution Approach 1:
The patent computes multiple probability models (answer prior, answer likelihood, answer context) for candidate answers, which represents an excessive action approach. By evaluating more aspects than minimally required and combining them through weighted scoring, the system achieves higher answer selection accuracy. The weighted combination allows the system to balance computational effort with accuracy requirements, adjusting the contribution of each model based on its importance.
Data Source
AI summary
A frequently-asked-question (FAQ)-based system receives question(s) from a user and generates answer(s) based on data about the question(s). In one embodiment, a method includes retrieving, from a memory, a global structure and candidate answers therein. The method can include computing a first, second, and third probability of a candidate answer based on a local structure of the candidate answer within the global structure, content of the candidate answer given content of a query and context of the candidate answer given the content of the query, respectively. The method can include providing a combined probability of the candidate answer based on the first probability, second probability, and third probability. The method can improve efficiency of a FAQ-based system by automating organization of semi-structured data in a database. Therefore, a human user does not need to manually generate the database when it is already generated in semi-structured form, a semi-structured HTML document.