Generation model for small sample multi-round dialogues in e-commerce live broadcast scene

Abstract

The invention discloses a small sample multi-round dialogue generation model for an e-commerce live broadcast scene, which comprises the following steps of: constructing a Chinese word list containing characters and words by using a unary language model, carrying out word segmentation on an input text by referring to the word list by using jieba, and representing the input by using the characters and the words obtained after word segmentation; the sum of the characters or words, the roles, the number of rounds and the positions is used as an embedding representation to be input into the model; the model totally comprises 12 Transform blocks, and a decoder and an encoder are fused together in each block, so that context understanding and reply generation can be realized, and parameter sharing can be realized; controlling the access of the current word to the context word in each block by using two self-attention mask modes; the words in the context position can see all the words in the context, and the words in the reply position can only see the previous words; and outputting the hidden state corresponding to each word in the last layer. The dialogue system is realized on a data set based on a small number of samples by using dialogues of a real scene in an e-commerce live broadcast process and adopting a prompt mode.

Description

technical field [0001] The invention belongs to the technical field of dialog systems, and more specifically relates to a generation model of small-sample multi-round dialogs used in e-commerce live broadcast scenarios. Background technique [0002] Current dialogue systems, whether chat systems (e.g., Blender-Bot) or task-oriented dialogue systems (e.g., MinTL), require large dialogue datasets for fine-tuning on language generation models. Fine-tuning these generative models with large datasets is expensive, and collecting large domain-specific datasets takes a lot of manpower and material resources, and also requires large computing resources and a lot of time. In order to solve the problem of using a large number of training samples and the overhead caused by fine-tuning, we have adopted a learning method that does not require fine-tuning the gradient, but uses a small number of samples as the context of the generated model, that is, using a small number of samples. Prom...

AI Technical Summary

Problems solved by technology

[0003] Existing open-domain pre-trained generative models are trained based on datasets such as persona-chat, DailyDialog, and Wizard of Internet (WiT), which do not contain data from e-commerce live broadcast scenarios, resulting in the inability of these pre-trained generative models to Good solution to dialogue tasks in e-commerce live broadcast scenarios

Images