Machine Reading Comprehension Encoding for Accurate Answer Span Extraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing question answering systems face challenges in efficiently extracting answer spans from text-based documents using machine reading comprehension technology, particularly when user questions relate to device functions or common knowledge.
Innovation Solution
A method for training a machine reading comprehension model involves generating input sequences from user questions and document passages, performing word-level and sentence-level encoding, and adjusting model weights based on loss values to accurately identify answer spans.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a machine reading comprehension model is used to extract answer spans from text-based documents, then the ability to answer user questions is improved, but the complexity of the system increases
Solution Approach 1:
The model is divided into multiple encoding layers (first encoding layer for word-level encoding, second encoding layer for sentence-level encoding) that process information at different granularities. This segmentation allows the system to handle complex extraction tasks through specialized sub-components rather than a monolithic structure.
Solution Approach 2:
The patent introduces multi-dimensional processing by performing both word-level and sentence-level encoding simultaneously. The model operates in multiple dimensional spaces (token embeddings, word-level hidden states, sentence-level hidden states) to capture different aspects of the answer span extraction problem.
2Measurement precision
If multiple encoding layers are used to improve answer span detection, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The encoding process is segmented into distinct levels: the first encoding layer processes word-level information to identify precise token boundaries, while the second encoding layer processes sentence-level information to determine the broader context. Each layer has a specialized function that contributes to the overall precision without requiring the other to be overly complex.
Solution Approach 2:
The first encoding layer acts as an intermediary that transforms raw token embeddings into word-level representations, which then serve as input to the second encoding layer. This intermediate processing stage enables the system to build precision incrementally rather than attempting to achieve it in a single complex transformation.
3Manufacturing precision
If loss values are calculated for multiple positions (starting position, ending position, sentence position), then manufacturing precision is improved, but loss of time increases
Solution Approach 1:
The training objective is segmented into three separate loss calculations, each targeting a specific aspect of answer span identification: starting position, ending position, and sentence position. This segmentation allows the model to learn each aspect independently and simultaneously, achieving comprehensive precision without requiring sequential processing of each objective.
Solution Approach 2:
All three loss values are calculated and used for weight adjustment in a continuous training process. The model receives gradient updates from all three objectives simultaneously, maintaining continuous learning across all aspects of answer span extraction rather than alternating between different training phases.
Data Source
AI summary
The disclosure relates to a method for training a machine reading comprehension model, a computer-readable recording medium storing the method, and a question answering system using a trained machine reading comprehension model.The method for training a machine reading comprehension model includes: obtaining features of input sentences; generating an input sequence including a plurality of tokens based on at least one feature among the obtained features; embedding the input sequence using a pre-trained language model; inputting an output embedding, obtained by embedding the input sequence, to a first encoding layer to perform word-level encoding; and inputting a portion of hidden states of the first encoding layer to a second encoding layer to perform sentence-level encoding.