Question Answering System Token Mapping for Structured Data Retrieval
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing question answering systems face inefficiencies in utilizing unstructured content from structured sources, requiring extensive preparatory work and training data for effective answer retrieval.
Innovation Solution
A method that maps structured sources, such as tables or database records, to a list of tokens to simulate a passage for comparison with the question, using an answer scoring model to identify relevant information and return ordered entries as answers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If extensive preparatory work and training data are used for translating questions into database queries, then answer retrieval effectiveness is improved, but system complexity and resource requirements increase
Solution Approach 1:
The patent creates unstructured text representations (copies) of structured database records that can be processed by natural language processing algorithms. Instead of requiring complex query translation, the system generates narrative versions of database contents that can be directly compared with questions using standard NLP techniques, thereby reducing system complexity while maintaining retrieval effectiveness
Solution Approach 2:
The patent introduces an intermediary representation layer between structured data and question answering. By translating structured records into unstructured text narratives as an intermediate form, the system enables seamless integration with NLP-based question answering mechanisms without requiring direct complex query translation or extensive training data
2Ease of operation
If structured sources are translated into unstructured text for question answering, then ease of processing is improved, but information loss may occur
Solution Approach 1:
The patent dynamically adjusts the level of detail and granularity in unstructured text generation based on the specific question being asked. By changing parameters such as narrative depth, included fields, and level of aggregation, the system optimizes the balance between ease of processing and information preservation for different query scenarios
3Measurement precision
If token matching is performed across all records and headers, then answer accuracy is improved, but processing time increases
Solution Approach 1:
The patent segments the question into individual tokens and processes matching against records and headers in a systematic manner. By dividing the processing into discrete token-level operations rather than holistic comparisons, the system achieves thorough accuracy while enabling efficient parallel processing and optimization of individual matching operations
Data Source
AI summary
A method, system and computer program product are provided for implementing unstructured content utilization from a structured source in a system capable of answering questions. A question is received by the system. The question is provided as a list of tokens T (T1, T2, . . . , Tn). A structured source is received containing records R (R1, R2, . . . , Rk). Headers H (H1, H2, . . . , Hi) are mapped to records. Each record R1, R2, . . . , Rk is searched for any token Ti in the list of tokens T (T1, T2, . . . , Tn) in the question. Responsive to identifying the token Ti in the question in a record Rj, the token Ti is added to a tracking structure. Any header mapped to the record Rj is compared to the list of tokens T (T1, T2, . . . , Tn) in the question. Responsive to identifying a header Hj mapped to the record Rj in the question, the header Hj is added to tracking structure. The tracking structure is sorted according a token order found in the question and ordered entries from the tracking structure are returned as an answer to the question.