A generative dialogue system decoder training method and decoding method
A dialogue system and generative technology, applied in the direction of neural learning methods, instruments, biological neural network models, etc., can solve the problem that the decoder is easy to generate wrong words, etc., and achieve the effect of improving the quality of reply generation
Active Publication Date: 2022-05-03
INST OF INFORMATION ENG CHINESE ACAD OF SCI
View PDF5 Cites 0 Cited by
- Summary
- Abstract
- Description
- Claims
- Application Information
AI Technical Summary
Problems solved by technology
And if only one-way decoding is used to decode the encoded vector generated by the encoder, the decoder is prone to generate wrong words
Method used
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View moreImage
Smart Image Click on the blue labels to locate them in the text.
Smart ImageViewing Examples
Examples
Experimental program
Comparison scheme
Effect test
specific example
[0052] 1) Using the question code sent by the encoder, use two neural networks to predict the content of the reply from the front to the back and from the back to the front respectively, and get two replies, of which the reply from the front to the back is mainly generated Rely on the historical information of the reply, while the reply generation from back to front mainly depends on the future information of the reply;
[0053] 2) Use the cross-entropy loss function to calculate the difference between the prediction results of each step of the forward neural network and the backward neural network, as the loss function of the decoder of the generative dialogue system;
[0054] 3) Calculate the difference between the states of each step of the two neural networks, as the local difference between the two, the specific calculation method is to subtract the hidden layer states corresponding to each step of the two, and then obtain the corresponding F norm;
[0055] 4) Use the att...
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More PUM
Login to View More Abstract
The invention discloses a decoder training method and a decoding method of a generative dialogue system. The method is as follows: 1) For each question code in the question code set, use the forward and backward neural network to predict the question code to obtain a vector representation result respectively; 2) calculate the forward and backward neural network The difference between the prediction results of each step of the network is used as the loss function of the encoder of the generative dialogue system; 3) the difference between the states of each step of the forward and backward neural network is calculated as the local difference between the two; 4) the forward and backward The sentence vector representation of the vector representation result generated by the neural network, and calculate the difference between the vector representations of the two sentences as the overall difference between the two; 5) Add the local difference and the overall difference as a penalty function to the loss function to obtain the overall penalty The function is used as the loss function of the encoder of the generative dialogue system; 6) The trained feed-forward neural network is used to predict the encoding of the question and generate the reply content.
Description
technical field [0001] The invention relates to a training method and a decoding method for a decoder of a generative dialogue system based on a bidirectional recursive neural network, belonging to the technical field of computer software. Background technique [0002] Traditional decoders usually use neural network structures such as LSTM to generate responses corresponding to questions verbatim based on the vector representation of the questions generated by the encoder. The generation of the current word mainly depends on the historical information in front of it in the sentence, and its main feature is one-way decoding. [0003] The traditional decoder has been able to generate the corresponding answer according to the vector representation of the question, but because the encoder is unidirectional, it can only refer to the previous historical information when generating the t-th word. However, under normal circumstances, the following words are also helpful for the pre...
Claims
the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More Application Information
Patent Timeline
Login to View More Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/332G06N3/04G06N3/08
CPCG06F16/3329G06N3/08G06N3/044G06N3/045
Inventor 林政付鹏刘欢王伟平
Owner INST OF INFORMATION ENG CHINESE ACAD OF SCI