Text simplification method based on word vector query model

Abstract

The invention provides a text simplification method based on a word vector query model. Based on a sequence and a sequence model, when decoding is conducted, the correlation between the hidden state of a decoder and the word vectors of all vocabularies is obtained through the reference of an attention mechanism to serve as a measurement of the possibility of the words to be generated in the next step. The method includes the following steps that a text encoder is designed, and an original text is compressed; a text simplification decoding generator is designed, and the current hidden layer vector and the context vector at every moment are calculated circularly; the retrieval correlation of each word in a word list is obtained, the predicted words at the current moment are output, and a complete simplified text is obtained; a model for generating the simplified text is trained, and the log likelihood of the predicted words and actual target words is minimized; after training, the complete simplified text is generated. The method can improve the quality and accuracy of the generated text, greatly reduce the number of parameters of the existing method, and reduce the training time andthe memory usage.

Description

technical field [0001] The invention belongs to the technical field of natural language processing and relates to a text simplification method, in particular to a text simplification method based on a word vector query model. Background technique [0002] Many existing text simplification algorithms use sequence-to-sequence-based generative models. These text simplification algorithms are based on deep learning technology and evolved from neural network machine translation algorithms. By observing large-scale source language to The training corpus of the target language can automatically simplify the text after a certain period of training. However, because the traditional sequence-to-sequence algorithm only maps the hidden layer representation to the vocabulary list through a large matrix multiplication when generating words in the decoder, the semantics of the words are not fully utilized. Moreover, mapping through a large matrix makes the entire network structure use a h...

AI Technical Summary

Problems solved by technology

However, since the traditional sequence-to-sequence algorithm generates words in the decoder, it only maps the hidden layer representation to the vocabulary list through a large matrix multiplication, and the semantics of the words are not fully utilized.

Moreover, mapping through a large matrix makes the entire network structure use a huge number of parameters (the vocabulary is generally large), resulting in slower model training and convergence, longer training time, and more memory resources. Inefficiency of text simplification

Images