Global normalized reader systems and methods

Abstract

Presented herein are systems and methods for question answering (QA). In embodiments, extractive question answering (QA) is cast as an iterative search problem through the document's structure: selectthe answer's sentence, start word, and end word. This representation reduces the space of each search step and allows computation to be conditionally allocated to promising search paths. In embodiments, globally normalizing the decision process and back-propagating through beam search makes this representation viable and learning efficient. Various model embodiments, referred to as Globally Normalized Readers (GNR), achieve excellent performance. Also introduced are embodiments of data-augmentation to produce semantically valid examples by aligning named entities to a knowledge base and performing swaps new entities of the same type. This methodology also improved the performance of GNR models and is of independent interest for a variety of natural language processing (NLP) tasks.

Description

technical field [0001] The present disclosure generally relates to systems and methods for computer learning that provide improved computer performance, features and usage. Background technique [0002] Question answering (QA) and information extraction systems have proven to be of great value in applications such as collection of medical information about drugs and genes, large-scale health impact research, or educational material development. [0003] Recent advances in neural network-based extractive question answering models are rapidly closing the gap with human performance in several benchmark QA tasks and providing a smarter, more responsive connection between information discovery and its usability in high-stakes decision-making , wherein the benchmark QA task is such as Stanford Question Answering Dataset (SQuAD), Microsoft Machine Reading Comprehension Dataset (MS MARCO) or NewsQA. [0004] However, current extractive question answering methods face many limitatio...

AI Technical Summary

Problems solved by technology

[0004] However, current extractive question answering methods face many limitations

First, computation is evenly distributed across the entire document regardless of answer position, making it impossible to ignore or focus computation on specific parts, limiting applicability to longer documents

Second, they generally rely on costly bidirectional attention mechanisms or must line up all possible answer spans (Span)

And third, although data extensions for question answering have been proposed, current methods still do not provide training data that can improve the performance of existing systems, nor do they allow explicit use of question nature or item types to control generation

Images