A Transfer Learning Method Based on Machine Reading to Sequence Model

A transfer learning and model technology, applied in the field of transfer learning based on machine reading to sequence models, can solve the problem of loss of pre-training model information, etc., and achieve the effect of simple and intuitive models, improved quality, and in-depth content
CN109508457BActive Publication Date: 2020-05-29ZHEJIANG UNIV
8 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN Β· China
Patent Type
Patents(China)
Current Assignee / Owner
ZHEJIANG UNIV
Publication Date
2020-05-29

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a transfer learning method based on machine reading to sequence model, comprising the following steps: (1) pre-training a machine reading model, the machine reading model includes a coding layer and a model layer based on a recurrent neural network; (2) ) set up a sequence model, the sequence model includes an encoder and a decoder based on a recurrent neural network; (3) extract the parameters of the encoding layer and the model layer in the trained machine reading model, and migrate to the sequence model to be trained, As part of the initialization parameters when training the sequence model; (4) train the sequence model until the model converges; (5) use the trained sequence model for text sequence prediction tasks. By using the present invention, the information contained in the text can be more deeply excavated, and the quality of the generated text sequence can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention belongs to the technical field of natural language processing, and in particular relates to a transfer learning method based on machine reading to sequence model. Background technique

[0002] Machine reading is one of the hottest and thorniest problems in natural language processing, which requires models to understand natural language and be able to apply existing knowledge. The most popular task at present is generally given an article and a question, and we need to find the answer from the article according to the question. With the release of several high-quality datasets in recent years, the performance of neural network-based models on machine reading is getting better and better, even surpassing humans on some datasets. An efficient machine reading model can be widely used in many fields based on semantic understanding, such as dialogue robots, question answering systems and search engines.

[0003] The sequence model with atten...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More