Fully parallelized end-to-end multi-turn dialogue system and method with domain scalability

Abstract

The present invention discloses a fully parallelized end-to-end multi-round dialogue system and method with domain scalability, wherein the system includes: an input embedding layer, which is used to generate a vector representation after user input information is input into the input embedding layer The input information; the encoder is used to encode the input information to obtain the abstract representation of the user input; the dialog state decoder is used to obtain the dialog state representation according to the abstract representation; the database is used to obtain the query result according to the dialog state representation query; The machine response decoder is used to generate a machine answer after the query result and the dialog state are jointly input to the machine response decoder. The system realizes the integration of various modules, can realize end-to-end training in the true sense, and simplifies the structure of the model while improving the training speed of the model, and has domain scalability, which is simple and easy to implement.

Description

technical field [0001] The invention relates to the technical fields of information technology and data business, in particular to a fully parallelized end-to-end multi-round dialogue system and method with field scalability. Background technique [0002] At present, the mainstream task-driven multi-round dialogue system in the industry is mainly designed based on the traditional assembly line method. This method has a complex structure and involves mutual coordination between multiple modules, but it is difficult for the end user's feedback to be transmitted to the upstream module. , one component adjustment requires significant labor costs to make corresponding changes to all components. Such a highly interdependent structure of various modules affects the performance and efficiency of the dialog system, and brings users a bad experience. In the related art, a method and system for multi-round dialogue is designed based on this pipeline architecture, which is mainly divid...

AI Technical Summary

Problems solved by technology

[0003] Based on this status quo, more and more research has begun to invest in end-to-end task-based multi-turn dialogue systems, but most of these systems are based on complex recurrent neural network structures, which aggravate the complexity of the model. The model encounters the problems of time dependence and sequence dependence, and cannot realize a fully parallelized architecture to give full play to the performance of GPU (Graphics Processing Unit, image processor)

In the related technology, there is an end-to-end hierarchical decoding task-based dialogue system, but the system only relies on database retrieval, and does not completely divide the dialogue state tracking and natural language generation modules, which may make it difficult to apply in complex environments, and is still highly Relying on two-way long-term short-term memory network (a variant of RNN (Recurrent Neural Network, cyclic neural network) network, which can capture long-distance dependencies in sentences), it has huge parameters and complex structure, which also makes the model The training speed becomes very slow, which greatly affects the performance of the model

[0004] In addition, most of the existing end-to-end models only adopt the method of joint training to make each module weakly related, but do not establish the real connection between each module, and do not consider the cross influence between each module. The process will involve the transfer of many variables, and has not realized the true end-to-end

In addition, the existing model is only applicable to a single domain, and it is necessary to redefine the labels of intent and slot values ​​during domain migration, which does not have domain scalability

In the related art, a method of realizing a question-and-answer robot based on a seq2seq model, the method first extracts subject words according to the context information of the question-and-answer sentence, and then puts the processed user question sentence into the seq2seq model for training, and the user question sentence processing part It needs to be trained separately and does not achieve true end-to-end training, which further aggravates the complexity of the model, and it is difficult for simple question and answer to adapt to multi-round dialogue scenarios

In addition, when the model undergoes domain migration, data labels (intent, slot value, etc.) need to be redefined, which brings difficulties to domain migration

Images