Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Dialogue generation method and device based on two-stage decoding, medium and computing equipment

A stage and decoder technology, applied in computing, biological neural network models, special data processing applications, etc., can solve the problems of model lack of information, such as reply, single, etc., to improve relevance and information, easy to control, have an interpretable effect

Pending Publication Date: 2021-06-18
SOUTH CHINA UNIV OF TECH
View PDF3 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In recent years, a large number of solutions to this problem have been proposed, such as: adding personalized information, topic information or external knowledge information, so that the model can better understand the semantics of the context, etc. However, these methods still only use a single decoding device, without distinction between content words and function words, generate the entire dialogue reply at one time
In this way, the model will tend to generate function words with less semantic information but higher frequency than content words with more semantic information but lower frequency, which will still cause the model to generate general and uninformative replies

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Dialogue generation method and device based on two-stage decoding, medium and computing equipment
  • Dialogue generation method and device based on two-stage decoding, medium and computing equipment
  • Dialogue generation method and device based on two-stage decoding, medium and computing equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0074] This embodiment discloses a dialogue generation method based on two-stage decoding. The method generates a dialogue through a two-stage decoding dialogue generation model, which can be applied to various language response systems in the field of human-computer interaction, such as intelligent robots that can chat , voice-controlled smart home products, etc.

[0075] The structure of the dialog generation model is as follows figure 2 As shown, it mainly includes two self-attention encoders (Self-Attention Encoder, SAE): context self-attention encoder (Post Self-Attention Encoder, PSAE) and content-word sequence self-attention encoder (Content-words Self-Attention Encoder) Attention Encoder, CwSAE), which are used to extract the features of the context sentence U and the content word sequence C, respectively, and two Transformer decoders: the first stage Transformer decoder, the second stage Transformer decoder. The contextual self-attention encoder is connected to the ...

Embodiment 2

[0125] This embodiment discloses a dialog generation device based on two-stage decoding, which can implement the dialog generation method in Embodiment 1. The dialogue generation device includes a sequentially connected mapping module, a context self-attention encoder, a first-stage Transformer decoder, a content word sequence self-attention encoder and a second-stage Transformer decoder, and the context self-attention encoder is also connected to the second stage. Two-stage Transformer decoder.

[0126] Among them, for the mapping module, it takes the text of the dialogue context as input and can be used to map each word in the text to a word embedding vector;

[0127] For the context self-attention encoder, it takes the sentence as the unit and takes the word embedding vector as input, which can be used to extract the feature vector of the context;

[0128] For the first-stage Transformer decoder, it takes the contextual feature vector as input, which can be used to decode ...

Embodiment 3

[0133] This embodiment discloses a computer-readable storage medium, which stores a program. When the program is executed by a processor, the method for generating a dialog based on two-stage decoding described in Embodiment 1 is implemented, specifically:

[0134] (1) Input the text of the dialogue context in the model, and map each word in the text to a word embedding vector;

[0135] (2) Take the sentence as a unit, input the word embedding vector into the context self-attention encoder, and extract the context feature vector through the context self-attention encoder;

[0136] (3) Input the obtained context feature vector into the Transformer decoder in the first stage, and decode to generate a content word sequence, which expresses the main semantic information in the final reply;

[0137] (4) input the obtained content word sequence into the content word sequence self-attention encoder to obtain the feature vector of the content word sequence;

[0138] (5) Input the enc...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a dialogue generation method and device based on two-stage decoding, a medium and computing equipment, and the method comprises the steps of dividing a dialogue reply generation process into two decoding stages, firstly inputting a dialogue context into a dialogue generation model, and mapping the dialogue context into a word embedding vector; inputting a word vector into a context self-attention encoder to obtain a feature vector of a dialogue context, inputting the feature vector into a first-stage Transformer decoder, and decoding to generate a notional word sequence; inputting the notional word sequence into a notional word sequence encoder to obtain a feature vector of the notional word sequence; and finally, inputting the context and the feature vector of the notional word sequence into a second-stage Transformer decoder, and decoding to generate a final reply. Through the two-stage decoding process, interference of the virtual words which are high in frequency but lack semantic information on the notional words is prevented, and therefore reply relevance and information amount are improved.

Description

technical field [0001] The present invention relates to the technical field of natural language processing, in particular to a two-stage decoding-based dialogue generation method and device, medium and computing equipment. Background technique [0002] In recent years, with the development of deep learning technology and the emergence of a large number of dialogue data sets, it is possible to use deep learning technology to build dialogue systems for open fields, which greatly expands the application scenarios of dialogue systems. [0003] In the field of dialogue generation in the open field, the current mainstream approach is based on an end-to-end generation framework: use an encoder to encode the dialogue context into a feature vector, and then use a decoder to decode and generate dialogue responses based on the previously generated vector. However, basic end-to-end dialogue generation models tend to generate generic, uninformative responses. Such as "OK", "I don't know...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/33G06F16/332G06N3/04
CPCG06F16/3344G06F16/3347G06F16/3329G06N3/047G06N3/045
Inventor 蔡毅钟志成孔俊生
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com