Chinese Intelligent Dialogue Method Based on Transformer

Abstract

The invention discloses a Transformer-based Chinese intelligent dialogue method. First, a large-scale Chinese chat corpus LCCC (Large-scale Cleaned Chinese Conversation) is used to construct a training data set as the input of the Transformer model, and then the Transformer is trained based on deep learning. The model is trained until the Transformer model converges. Finally, the input sequence to be dialogued is input into the Transformer model, and the expected output sequence is output in real time through the Transformer model, thereby realizing intelligent real-time dialogue in Chinese.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and more specifically relates to a Transformer-based Chinese intelligent dialogue method. Background technique [0002] With the rapid development of deep learning technology, more and more deep learning technology has been introduced in the field of NLP (Nature language model, natural language processing), and its effect has made great progress compared with traditional rule-based or traditional statistical methods. progress. The pre-trained language representation model using the Transformer model, such as BERT (Bidirectional Encoder Representations from Transformers), has achieved better results in various NLP tasks than traditional methods, because Transformer has improved the most criticized training of RNN. specialty. The self-attention mechanism is used to achieve fast parallelism, and the Transformer can be increased to a very deep depth to fully explore the characte...

AI Technical Summary

Problems solved by technology

[0008] Large-scale, high-quality Chinese dialogue data plays an important role in the model. Currently known question answering systems can only answer questions mechanically, and the answers are often irrelevant, and have not reached the level of intelligent understanding of semantics and context.

Images