Model training method and device, dialogue system evaluation method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a model training method, device and equipment, and the method comprises the steps: obtaining a pre-trained dialogue generation model which comprises an encoder and a decoder; constructing a dialogue system evaluation model, wherein the dialogue system evaluation model takes questions and replies as input and takes scores corresponding to the replies as output; initializing parameters of an encoder in the dialogue system evaluation model according to the parameters of the encoder in the dialogue generation model; and according to the first training sample set, training the initialized dialogue system evaluation model to obtain a dialogue system evaluation model meeting a training ending condition, each training sample in the first training sample set comprising a question, a reply and a label score corresponding to the reply. The dialogue system evaluation model obtained by training through the method can evaluate the reply quality of the dialogue system from the perspective of semantic correlation, and the reliability of dialogue reply evaluation is improved.

Description

technical field [0001] The present application relates to the technical field of artificial intelligence, and in particular to a model training method, a dialogue system evaluation method, a device, a device, and a storage medium. Background technique [0002] As an important application of artificial intelligence technology, chatbots have been widely used in various specific scenarios, such as customer service, personal assistants, knowledge quizzes, and accompanying chats. At present, chatbots can be divided into task-type and non-task-type chatbots according to their purposes, and the realization technologies of chatbots for different purposes are not the same. [0003] Among them, non-task-based chatbots interact with users through non-task-based dialogue systems. There are usually two commonly used evaluation methods for the evaluation of the response quality of non-task-based dialogue systems. One is the manual evaluation method, which requires a lot of manpower. Manu...

AI Technical Summary

Problems solved by technology

[0003] Among them, non-task-based chatbots interact with users through non-task-based dialogue systems. There are usually two commonly used evaluation methods for the evaluation of the response quality of non-task-based dialogue systems. One is the manual evaluation method, which requires a lot of manpower. Manually evaluate the quality of system responses. Manual evaluation is highly reliable, but the efficiency of manual evaluation is high. Especially with the explosive growth of data volume, the labor cost and time cost of system evaluation will also increase, which is difficult to meet the needs of actual business. need

[0004] The other is to use BELU (Bilingual Evaluation understanding) value and other technical indicators for automatic evaluation, and the BELU value is calculated by comparing the coincidence degree between the system reply and the existing (manual) reply, specifically comparing the two types of reply respectively. The coincidence degree of any unit fragment (n-gram), the larger the value, the higher the coincidence degree of the two types of replies and the better the reply quality, and the smaller the value, the lower the coincidence degree of the two types of replies and the better the reply quality. The worse it is, the BELU value alone can only literally evaluate the similarity between the system reply and the existing (manual) reply, that is, it only has the ability to express the literal correlation of the sentence, which leads to the inaccurate evaluation of the reply quality of the system by using the current technical indicators

Images