Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Multi-round dialogue model construction method based on hierarchical attention mechanism

A construction method and attention technology, applied in special data processing applications, instruments, electrical digital data processing, etc., can solve the problems of training speed influence, meaningless reply, etc.

Pending Publication Date: 2018-03-06
HARBIN INST OF TECH
View PDF9 Cites 28 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0027] The purpose of the present invention is to solve the problem that the existing man-machine dialogue system relies on large-scale corpus, the training speed is affected by the scale of corpus, and because the reply generated by the dialogue is not unique, the Seq2Seq model always tends to generate general and meaningless In view of the shortcomings of replies, a multi-round dialogue model construction method based on a hierarchical attention mechanism is proposed, including:

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-round dialogue model construction method based on hierarchical attention mechanism
  • Multi-round dialogue model construction method based on hierarchical attention mechanism
  • Multi-round dialogue model construction method based on hierarchical attention mechanism

Examples

Experimental program
Comparison scheme
Effect test

specific Embodiment approach 1

[0049] The present invention provides a method for constructing a multi-round dialogue model based on a hierarchical attention mechanism, such as figure 1 shown, including:

[0050] Step 1. Receive n+1 sentences and input c o ,c 1 ,... c n .

[0051] Step 2, for each sentence c i , starting from the first word to calculate the encrypted implicit function h i,t =f(x i,t ,h i,t-1 ), where x i,t stands for c i the tth word; where h i,0 Recorded as a preset parameter; and the last calculated h i,t denoted as sentence c i The encrypted implicit function h i .

[0052] Step 3. Calculate the Attention weight of the i-th sentence where e i =v T tanh (Wh i +Uh n ); v, W, and U are all preset parameters in the Attention mechanism.

[0053] Step 4. Calculate the topic context representation vector T=∑α i h i .

[0054] Step 5. Calculate and decrypt the implicit function s t =f(y t-1 ,s t-1 ,T), y t-1 Indicates the iterative input amount at time t-1, 0 is the d...

specific Embodiment approach 2

[0059] The present invention also provides another method for constructing a multi-round dialogue model based on a hierarchical attention mechanism, including:

[0060] Step 1. Receive n+1 sentences and input c o ,c 1 ,... c n .

[0061] Step 2, for each sentence c i , starting from the first word to calculate the encrypted implicit function h i,t =f(x i,t ,h i,t-1 ), where x i,t stands for c i the tth word; where h i,0 Recorded as a preset parameter; and the last calculated h i,t denoted as sentence c i The encrypted implicit function h i .

[0062] Step 3. Calculate the Attention weight of the t-th word in the i-th sentence where e it =v T tanh (Wh i +Us t-1 ); v, W, and U are all preset parameters in the Attention mechanism; s t-1 is the state of the hidden layer at time t-1.

[0063] Step 4. Calculate the dynamic representation vector D t = α it h i .

[0064] Step 5. Calculate and decrypt the implicit function s t =f(y t-1 ,s t-1 ,D t ), y t-1...

example 1

[0083] Above:

[0084] Now I'm going to try the development version from the PPA and see if it crashes again.

[0085] Are you looking at your computer's CPU temperature?

[0086] No, I haven't had a problem with the temperature....where can I see it's temperature?

[0087] topic model: you can try to delete your config file and try again

[0088] dyna model: try running lspci from terminal, there is a list

[0089] hred model: System -> Preferences -> Power Management

[0090] vhred model: I don't understand what you mean

[0091] LSTM model: I don't understand

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a multi-round dialogue model construction method based on a hierarchical attention mechanism. The invention aims to put forward the multi-round dialogue model construction method based on the hierarchical attention mechanism in order to solve the defects that an existing man-machine conversation system depends on large-scale corpora, training speed is influenced by the scale of the corpora, in addition, a reply generated by the dialogue is not unique to cause that a Seq2Seq model always tends to generate a universal and meaningless reply. The method comprises the following steps that: receiving sentence input, aiming at each sentence, beginning to calculate an encrypted implicit function from a first word, calculating the Attention weight of each sentence, calculating a topic context representation vector, finally, calculating a decrypted implicit function, and meanwhile, outputting a result. The method is suitable for the chatting robot system of an open domain.

Description

technical field [0001] The invention relates to a man-machine dialogue system, in particular to a method for constructing a multi-round dialogue model based on a hierarchical attention mechanism. Background technique [0002] 1. The status quo of foreign technology [0003] (1) Dialogue system based on artificial template [0004] The technology based on artificial templates manually sets the dialogue scenes, and writes some targeted dialogue templates for each scene. The templates describe the possible questions of the user and the corresponding answer templates. [0005] Weizenbaum et al. (1966) developed the earliest chat robot ELIZA. ELIZA pre-designed the corresponding language template according to the language situation that may appear in the dialogue. The text generator will embed the important information in the input into the template according to the user's input. , eventually getting a reply. [0006] They both limit the chat to specific scenarios or specific ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F17/30
CPCG06F16/3329
Inventor 张伟男汪意发朱庆福刘挺
Owner HARBIN INST OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com