Semantic Parsing of Mathematical Formulas Using Question Answering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing techniques for extracting definitions of identifiers from mathematical formulas in documents are error-prone due to assumptions about noun phrases and sentence structure, leading to incorrect tagging and splitting of text.
Innovation Solution
A method that extracts identifiers from mathematical formulas and corresponding text passages, using a question answering model to generate candidate answers and select definitions based on confidence scores, without relying on error-causing assumptions, and facilitates transfer learning across different source formats.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing techniques use assumptions about noun phrases and sentence structure to extract definitions, then the process is simpler and faster, but the accuracy and reliability of extracted definitions deteriorate
Solution Approach 1:
The patent introduces a question-answering model as an intermediary component between the identifier extraction module and the definition selection module. This QA model takes identifier-passage pairs as input and generates candidate answers, which are then ranked using confidence scores to select the most likely definition. This intermediary approach eliminates the need for error-prone assumptions about noun phrases and sentence structure while maintaining systematic processing through structured modules.
2Measurement precision
If manual feature selection is used, then the quality and precision of features are improved, but the time consumption and labor requirements increase
Solution Approach 1:
The patent implements automated feature engineering where the system performs its own feature extraction and definition selection without requiring manual intervention from data scientists or domain experts. The automated pipeline includes: (1) extracting identifiers from mathematical formulas, (2) extracting text passages containing identifiers, (3) generating candidate answers using the QA model, and (4) selecting definitions based on confidence scores. This self-service approach maintains high precision through systematic processing while dramatically reducing time consumption compared to manual selection.
3Manufacturing precision
If error-prone assumptions about text structure are used, then the extraction process is more straightforward, but incorrect tagging and splitting of text occurs
Solution Approach 1:
The patent replaces the mechanical approach of relying on fixed assumptions about noun phrases and sentence structure with a computational approach using a question-answering model. Instead of using grammatical rules or pattern matching that are prone to errors, the system uses the QA model to understand the semantic relationship between identifiers and their definitions in context. This substitution maintains ease of operation through automated processing while significantly improving the precision of text tagging and splitting by avoiding error-prone structural assumptions.
Data Source
AI summary
A method, system, and computer program product are disclosed. The method includes extracting at least one identifier from a formula in a document and extracting text passages in the document that contain the identifier(s). The method also includes selecting an identifier and extracted text passages containing the identifier, as well as generating identifier-passage pairs for the selected text passages and the identifier. Further, the method includes submitting the identifier-passage pairs to a question answering (QA) model, which generates candidate answers from the selected text passages. A definition of the identifier is then selected from the candidate answers.