Omission recovery method for short text understanding

Abstract

The invention discloses an omission recovery method for short text understanding. The model is mainly divided into an embedding layer, a coding layer and a decoding layer. The embedding layer is usedfor obtaining distributed representation of discrete words; the coding layer is used for mining features of the text; and the decoding layer generates a result after completion omission by using the features extracted by the coding layer. The method has the beneficial effects that the given model assumes that words in the sentence are possibly omitted, the structure information of the to-be-recovered sequence is fully considered when the model is trained and predicted, and the problem of 'sick sentences' caused by seq2seq is effectively solved. Besides, in the aspect of short text understanding, the model fuses a cross attention mechanism and a self-attention mechanism, and compared with a seq2seq model, more and deeper text features can be extracted.

Description

technical field [0001] The invention relates to the field of language processing, in particular to an omission recovery method for short text understanding. Background technique [0002] As a linguistic phenomenon, ellipsis is ubiquitous in natural languages. According to statistics, about 96% of English subjects are explicit subjects, while only 64% of them are explicit subjects in Chinese. It can be seen that ellipsis is more frequent in Chinese than in English. In addition, in a short text application scenario, such as a dialogue system, the two parties in the dialogue have the same dialogue background, and some information is shared in multiple rounds of the dialogue, so ellipsis is more common in the dialogue. Similarly, in a series of question answering systems, there is a correlation between multiple consecutive questions, and there is also a certain correlation between the follow-up questions and the answers to the previous questions. They may share some informatio...

AI Technical Summary

Problems solved by technology

[0058] In the seq2seq model proposed by Zheng et al., the structural information of the sequence to be restored comes from the coding information of Bi-LSTM. The seq2seq model uses MLE to implicitly learn the semantic structure of the sequence to be restored, which leads to loss of the model during prediction. Part of the source sequence structure information, resulting in a low accuracy rate of complete matching

Images