A Transfer Learning Method Based on Machine Reading to Sequence Model

Abstract

The invention discloses a transfer learning method based on machine reading to sequence model, comprising the following steps: (1) pre-training a machine reading model, the machine reading model includes a coding layer and a model layer based on a recurrent neural network; (2) ) set up a sequence model, the sequence model includes an encoder and a decoder based on a recurrent neural network; (3) extract the parameters of the encoding layer and the model layer in the trained machine reading model, and migrate to the sequence model to be trained, As part of the initialization parameters when training the sequence model; (4) train the sequence model until the model converges; (5) use the trained sequence model for text sequence prediction tasks. By using the present invention, the information contained in the text can be more deeply excavated, and the quality of the generated text sequence can be improved.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and in particular relates to a transfer learning method based on machine reading to sequence model. Background technique [0002] Machine reading is one of the hottest and thorniest problems in natural language processing, which requires models to understand natural language and be able to apply existing knowledge. The most popular task at present is generally given an article and a question, and we need to find the answer from the article according to the question. With the release of several high-quality datasets in recent years, the performance of neural network-based models on machine reading is getting better and better, even surpassing humans on some datasets. An efficient machine reading model can be widely used in many fields based on semantic understanding, such as dialogue robots, question answering systems and search engines. [0003] The sequence model with atten...

AI Technical Summary

Problems solved by technology

[0005] However, the existing transfer learning methods for natural language processing rarely transfer multi-layer neural networks to other tasks, and only transferring the coding layer will lose a lot of information of the original pre-trained model.

Images