Training method and decoding method for decoder of generative dialogue system

Abstract

The invention discloses a training method and a decoding method for a decoder of a generative dialogue system. The method comprises the following steps: 1) for each question code in a question code set, predicting the question code by using forward and backward neural networks to respectively obtain vector representation results; 2) calculating the difference between prediction results of each step of the forward and backward neural networks, and taking the difference as a loss function of the encoder of the generative dialogue system; 3) calculating the state difference of each step of the forward and backward neural networks to serve as the local difference of the forward and backward neural networks; 4) calculating sentence vector representations of vector representation results generated by the forward and backward neural networks, and calculating a difference between the two sentence vector representations as an integral difference between the two sentence vector representations;5) adding the local difference and the integral difference as penalty term functions into a loss function to obtain an integral penalty function as a loss function of the encoder of the generative dialogue system, and 6) predicting the question codes by using the trained forward neural network to generate reply contents.

Description

technical field [0001] The invention relates to a training method and a decoding method for a decoder of a generative dialogue system based on a bidirectional recursive neural network, belonging to the technical field of computer software. Background technique [0002] Traditional decoders usually use neural network structures such as LSTM to generate responses corresponding to questions verbatim based on the vector representation of the questions generated by the encoder. The generation of the current word mainly depends on the historical information in front of it in the sentence, and its main feature is one-way decoding. [0003] The traditional decoder has been able to generate the corresponding answer according to the vector representation of the question, but because the encoder is unidirectional, it can only refer to the previous historical information when generating the t-th word. However, under normal circumstances, the following words are also helpful for the pre...

AI Technical Summary

Problems solved by technology

And if only one-way decoding is used to decode the encoded vector generated by the encoder, the decoder is prone to generate wrong words

Images