Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Novel answer selection model based on GRU attention mechanism

A technology of attention and mechanism, applied in special data processing applications, instruments, unstructured text data retrieval, etc., can solve problems such as multi-noise, and achieve the effect of improving algorithm stability and algorithm stability

Inactive Publication Date: 2019-09-13
SUN YAT SEN UNIV
View PDF5 Cites 1 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

IARNN-GATE does not screen the input information, which will result in more noise in the candidate output hidden state, and it is difficult to remove all the noise through an update gate inside the GRU

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Novel answer selection model based on GRU attention mechanism
  • Novel answer selection model based on GRU attention mechanism
  • Novel answer selection model based on GRU attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0027] figure 1 is a schematic structural diagram of a recurrent neural network RNN ​​according to an embodiment of the present invention. The cyclic neural network can be expressed as a function, and the common form of the neural network can be divided into an input layer, a hidden layer, and an output layer. No matter how many layers there are in the hidden layer, it can be abstracted into a large hidden layer as a whole. The hidden layer can also be expressed in the form of a function, which takes the data of the input layer as an independent variable and calculates the output dependent variable. The output layer is also a function that takes the output of the dependent variable from the hidden layer as input. RNN has important applications in many natural language processing tasks. RNN has achieved very good results in tasks such as language model, text generation, machine translation, language recognition and image description generation.

[0028] according to figure ...

Embodiment

[0084] The present invention has carried out accuracy comparison and analysis experiment based on the attention mechanism model inside GRU to above-mentioned method, specifically as follows:

[0085] This experiment uses InsuranceQA dataset and WikiQA dataset.

[0086]Firstly, the InsuranceQA data set is used. The InsuranceQA data set is divided into three parts: training set, verification set and test set. The test set is divided into two small test sets (Test1 and Test2). Each part has the same compositional format: each question-answer pair consists of 1 question and 11 answers, where 11 answers include 1 correct answer and 10 distracting answers. In the model training phase, for each question in the training set, one of the corresponding 10 interference answers is randomly selected as the interference answer during training; in the testing phase, the distance between each question and its corresponding 11 answers is calculated. similarity score. Before the experiment, th...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Answer selection (AS) is an important subtask in question and answer system design, and the problem is mainly solved by using a deep learning method at present. A traditional attention mechanism is more biased to the following state characteristics, and an internal attention mechanism is provided based on the following state characteristics, so that the problem of weight distribution deviation isavoided. However, a model does not screen the input information, which results in more noise contained in the candidate output hidden state. The invention relates to an algorithm for an answer selection model of a question and answer system. The invention has the following effects: (1) an input gate in front of an attention model is added in a GRU to filter useless information; (2) an answer selection model is improved in the question and answer system by using the new attention mechanism, and accuracy is improved compared with that of an original GRU-based internal attention mechanism model;and (3) the method provided by the invention is greatly improved in the aspects of accuracy, algorithm stability and the like, and can be better suitable for practical engineering work.

Description

technical field [0001] The invention relates to the field of natural language processing, that is, an algorithm for selecting the optimal reply in an answer selection model of a question answering system. Background technique [0002] Answer selection (Answer selection, AS) is an important sub-task in the design of question answering system. Its function is to select the best answer from a series of candidate responses for a given question. The accuracy of answer selection during dialogue plays a key role in the performance of question answering systems. In the past few years, answer selection has received a lot of attention. Among them, the use of neural network models to solve answer selection tasks has achieved great success. However, when the semantic vector is generated by the cyclic neural network, the question and the answer are encoded separately, and the information related to the question in the answer is ignored, resulting in the generated answer semantic vector...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/332G06F16/33
CPCG06F16/3329G06F16/3344
Inventor 王慧刘璨戴宪华
Owner SUN YAT SEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com