Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Answer selection model based on internal attention mechanism of GRU neural network

A neural network and model selection technology, applied in the field of algorithms for selecting optimal responses, to achieve the effect of improving the stability of the algorithm

Pending Publication Date: 2019-11-26
SUN YAT SEN UNIV
View PDF3 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In summary, the traditional attention mechanism is more biased towards the latter state features

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Answer selection model based on internal attention mechanism of GRU neural network
  • Answer selection model based on internal attention mechanism of GRU neural network
  • Answer selection model based on internal attention mechanism of GRU neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0026] figure 1 is a schematic structural diagram of a recurrent neural network RNN ​​according to an embodiment of the present invention. Recurrent Neural Networks (RNN) is a common artificial neural network. The connection lines between nodes in the network form a directed ring. RNN has important applications in many natural language processing tasks. Different from the feed-forward neural network (Feed-forward Neural Networks, FNN) in which the input and output are independent of each other, RNN can effectively use the output of the previous moment. Therefore, RNN is more suitable for processing sequence data. In theory, RNNs can handle arbitrarily long sequences, but in practice they cannot. RNN has achieved very good results in tasks such as language model, text generation, machine translation, language recognition and image description generation.

[0027] figure 2 is a schematic structural diagram of a traditional GRU according to an embodiment of the present inven...

Embodiment

[0054] The present invention has carried out precision comparison and analysis experiment to above-mentioned method and traditional method, specifically as follows:

[0055] This experiment uses Chinese data translated from the insuranceQA corpus. Before the experiment, the questions and answers were first cut into words, and then word2vec was used to pre-train the questions and answers. Since this experiment uses a fixed-length GRU, the questions and answers need to be truncated (too long) or supplemented (too short).

[0056] Experimental modeling generates input data. This experiment is modeled in the form of a question-answer triplet (q, a+, a-), where q represents a question, a+ represents a positive answer, and a- represents a negative answer. The training data in insuranceQA already contains questions and positive answers, so it is necessary to select the negative answers. In the experiment, we randomly selected the negative answers and combined them into the form of ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Answer selection is a very important sub-question in a question and answer system, and at present, a deep learning method is mainly used for solving the question, and a very good effect is achieved. According to the deep learning method, an end-to-end attention mechanism is mainly used, but the attention mechanism has defects and is more deviated to subsequent state characteristics, so that the characteristics close to the time sequence tail end are easier to select out due to the fact that the characteristics contain all previous information. The invention relates to an algorithm for selecting an optimal reply in an answer selection model of a question-answering system. The algorithm comprises the following steps: (1) introducing an action vector into a GRU network, and proposing a new attention mechanism; (2) improving an answer selection model in the question-answering system by using the new attention mechanism, and improving the accuracy by about 4 percent points compared with that of a traditional answer selection model; and (3) being able to be applied to an intelligent customer service system of an e-commerce platform, being greatly improved in the aspects of accuracy, algorithm stability and the like, and being better applied to actual engineering work.

Description

technical field [0001] The invention relates to the field of natural language processing, that is, an algorithm for selecting the optimal reply in an answer selection model of a question answering system. Background technique [0002] Answer selection is a very important sub-question in the question answering system. In the past few years, answer selection has received a lot of attention. Among them, using the method of deep learning to solve the answer selection task has achieved good results. Among them, the end-to-end attention mechanism has achieved the best results on this problem. End-to-end attention computes word weights between answers and questions. However, the end-to-end attention mechanism treats the entire sentence as an ordered string and processes it sequentially to generate a representation of the sentence. This representation does not take into account the syntactic information between sentences and the more complex inter-sentence contact. At the same ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/332G06F16/33G06N3/04
CPCG06F16/3329G06F16/3344G06N3/044
Inventor 熊海泉王慧戴宪华
Owner SUN YAT SEN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com