Question and answer task downstream task processing method and model

Abstract

The invention discloses a question and answer task downstream task processing method and model, and the method comprises the steps: obtaining a context representation HCKey perceived by key information and a question representation HQKey perceived by key information, and generating a context representation G perceived by a question; calculating an update vector z and a memory weight g based on G, and updating G to obtain an output vector Gg; generating a context granularity vector GC and a sequence granularity vector GCLS, generating an output vector Cout, using softmax to calculate the probability that each character in the context serves as an answer starting and ending position, and extracting a continuous subsequence with the maximum probability to serve as an answer. The invention provides a two-way cascading attention mechanism, and constructs a mechanism taking fine reading and slight reading as a whole and a multi-granularity module based on a granular computing thought, so that the model effectively pays attention to and screens effective information, better understands texts under various granularities, gives out more accurate answers, and improves the accuracy of text analysis. And the performance is improved on the basis of a baseline model.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and in particular relates to a downstream task processing method and model of a question answering task. Background technique [0002] Machine reading comprehension is a challenging task in natural language processing, which aims to determine the correct answer to a question based on a given context. Common machine reading comprehension tasks are divided into cloze, multiple choice, fragment extraction and free answer according to the form of the answer. The newly developed pre-trained language model has achieved a series of successes in various natural language understanding tasks by virtue of its powerful text representation ability. These pre-trained language models are used as encoders of deep learning language models to extract language-associated features of relevant texts and fine-tune them in combination with task-specific downstream data processing structures. With ...

AI Technical Summary

Problems solved by technology

[0003] To sum up, the existing deep learning language model has the following shortcomings: (1) it will pay attention to the unimportant parts of the text, while ignoring the important parts; The impact of interfering sentences in texts with multiple identical vocabulary can only be matched through the text itself, but not semantically matched

Images