Unlock instant, AI-driven research and patent intelligence for your innovation.

A transfer learning method based on machine reading to sequence model

A transfer learning and model technology, applied in the field of transfer learning based on machine reading to sequence model, can solve the problem of loss of pre-training model information, etc., achieve the effect of simple and intuitive model, deep content, and improved accuracy

Active Publication Date: 2019-03-22
ZHEJIANG UNIV
View PDF8 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] However, the existing transfer learning methods for natural language processing rarely transfer multi-layer neural networks to other tasks, and only transferring the coding layer will lose a lot of information of the original pre-trained model.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A transfer learning method based on machine reading to sequence model
  • A transfer learning method based on machine reading to sequence model
  • A transfer learning method based on machine reading to sequence model

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] In order to make the object, technical solution and beneficial technical effect of the present invention clearer, the technical content and specific implementation methods of the present invention will be further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementations described in this specification are only for explaining the present invention, not for limiting the present invention.

[0038] Such as figure 1 As shown, a transfer learning method based on machine reading to sequence model, including the following steps:

[0039] S01, pre-training a machine reading model.

[0040] We use the Stanford Question Answering Dataset SQuAD, a large-scale, high-quality corpus, as the training set. Our task is to predict an answer given an article and a question, which is a continuous field in the article.

[0041] The structure of the machine reading model see figure 2, we use the existing word vector...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a migration learning method based on a machine reading to a sequence model, comprising the following steps: (1) pre-training a machine reading model, wherein the machine reading model comprises a coding layer and a model layer based on a cyclic neural network; (2) establishing a sequence model, wherein the sequence model comprises an encoder and a decoder based on a loop neural network; (3) extracting the parameters of the coding layer and the model layer in the trained machine reading model and migrating them to the sequence model to be trained as part of the initialization parameters of the training sequence model; (4) training the sequence model until the model converges; (5) Using the trained sequence model to predict the text sequence. By utilizing the invention, the implied information of the text can be more deeply mined, and the quality of the generated text sequence can be improved.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and in particular relates to a transfer learning method based on machine reading to sequence model. Background technique [0002] Machine reading is one of the hottest and thorniest problems in natural language processing, which requires models to understand natural language and be able to apply existing knowledge. The most popular task at present is generally given an article and a question, and we need to find the answer from the article according to the question. With the release of several high-quality datasets in recent years, the performance of neural network-based models on machine reading is getting better and better, even surpassing humans on some datasets. An efficient machine reading model can be widely used in many fields based on semantic understanding, such as dialogue robots, question answering systems and search engines. [0003] The sequence model with atten...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/27G06F16/332G06N3/04G06N3/08
CPCG06N3/08G06F40/30G06N3/045
Inventor 潘博远蔡登李昊陈哲乾赵洲何晓飞
Owner ZHEJIANG UNIV