A machine reading comprehension method based on knowledge-guided attention

A technology of reading comprehension and attention, applied in instruments, digital data processing, natural language data processing, etc., can solve problems such as inability to fully understand content

Active Publication Date: 2021-06-29
ZHEJIANG UNIV
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

When people read and understand the content of an article, the process of reasoning is almost everywhere. Without reasoning, people cannot fully understand the content, and the same is true for machines.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A machine reading comprehension method based on knowledge-guided attention
  • A machine reading comprehension method based on knowledge-guided attention
  • A machine reading comprehension method based on knowledge-guided attention

Examples

Experimental program
Comparison scheme
Effect test

Embodiment

[0135] Taking the CNN\Daily Mail dataset as an example, apply the above method to the reading comprehension task. The specific parameters and practices in each step are as follows:

[0136] 1. Using the CNN\Daily Mail dataset, since the CNN\Daily Mail original dataset is stored in the form of one piece of data and one file, in order to facilitate subsequent processing, it is merged and redundant field information is removed, and only (Question, Context, Answer), using natural language processing tools to segment articles and questions into sentences and words, the vocabulary size is 118497 / 208045, and the average number of entities in CNN and Daily Mail articles is about 26;

[0137] 2. Use the 600 million Stanford GloVe 300-dimensional vectors that have been trained and the vocabulary in 1 to form a 300-dimensional word vector. In order to train the model, this paper counts the word frequencies in the training set, sorts them in descending order, and selects the first 50k word...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a machine reading comprehension method based on knowledge guiding attention. The method includes the following steps: (1) use the pre-trained word embedding matrix to obtain the word vector of the text sequence; (2) use the bidirectional GRU network to model the context information of each word in the text; (3) convert the The context representation is input into the one-way GRU network as the initial hidden layer state, and the GRU network uses the attention-based look-back mechanism to iteratively perform the search steps to collect information in the article that may be used to predict the answer; (4) use external knowledge as long-term memory Add a look-back mechanism to guide the focus of attention during the look-back process, and the model will redistribute the attention scores; (5) Get the predicted answer through the pointer network at the output of the one-way GRU network. The present invention is an end-to-end model that does not require data preprocessing other than pre-trained word vectors in the unlabeled corpus, so the present invention can be widely used in reading comprehension in different languages ​​and fields.

Description

technical field [0001] The invention relates to natural language processing, in particular to a machine reading comprehension method based on knowledge guiding attention. Background technique [0002] Natural Language Processing (NLP) is an interdisciplinary subject integrating linguistics and computer science. Reading comprehension (Reading Comprehension) is a fundamental task in natural language processing, usually by asking the system to answer questions, inferring answers from a given text or context. With the advent of the Internet age, the information on the Internet has exploded, including text data in various languages ​​and forms, such as news from Sina and Daily Mail, articles from Baidu and Wikipedia, Zhihu and Answers from question-and-answer communities like Quora. These corpora become the basis for constructing large-scale machine reading comprehension datasets. Teaching machines to read, process and understand human language is one of the core tasks of natu...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F40/205G06F40/289G06F40/30G06N3/04
CPCG06N3/045
Inventor 庄越挺浦世亮汤斯亮谭洁郝雷光吴飞
Owner ZHEJIANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap