A Generative Model for Small-Sample Multi-round Dialogue for E-commerce Live Scenarios

Abstract

The invention discloses a small sample multi-round dialogue generation model for e-commerce live broadcast scenarios, including: using a unigram language model to construct a Chinese vocabulary containing words and words, and using jieba to refer to the vocabulary to segment the input text; Characterize the input with the words and words obtained after the word segmentation; the sum of the word or word, role, round number and position embedding is input to the model as the embedded representation; the model contains a total of 12 Transformer blocks, and each block combines the decoder and The encoders are fused together to achieve context understanding and generate responses to achieve parameter sharing; two self-attention masks are used in each block to control the access of the current word to the context word; words in the context position can be seen All the context words, the words in the reply position, can only see the words before it; the hidden state corresponding to each word is output in the last layer. The invention uses the dialogue of the real scene in the process of live broadcast of the e-commerce, adopts the prompt method, and realizes the dialogue system on the data set based on a small number of samples.

Description

technical field [0001] The invention belongs to the technical field of dialogue systems, and more particularly relates to a generation model of small-sample multi-round dialogues used in e-commerce live broadcast scenarios. Background technique [0002] Current dialogue systems, whether they are chat systems (e.g., Blender-Bot) or task-oriented dialogue systems (e.g., MinTL), require large dialogue datasets to fine-tune language generation models. Fine-tuning these generative models with large datasets is expensive, and collecting a large number of domain-specific datasets takes a lot of human and material resources, as well as large computing resources and a lot of time. In order to solve the problem of using a large number of training samples and the overhead of fine-tuning, we adopt a learning method that does not require fine-tuning of the gradient, but uses a small number of samples as the context of the generated model. The learning method of the prompt. [0003] Exi...

AI Technical Summary

Problems solved by technology

[0003] Existing open-domain pre-trained generative models are trained based on datasets such as persona-chat, DailyDialog, and Wizard of Internet (WiT), which do not contain data from e-commerce live broadcast scenarios, resulting in the inability of these pre-trained generative models to Good solution to dialogue tasks in e-commerce live broadcast scenarios

Images