Multi-round dialogue model training method and device, electronic equipment and storage medium

Abstract

The invention provides a multi-round dialogue model training method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: constructing a sample dialogue training set with a first label for indicating whether a corresponding sample dialogue statement needs to be rewritten or not, and carrying out iterative training on an initial multi-round dialogue model; in this way, the problem that in the prior art, whether the current dialogue statement needs to be rewritten or not is judged in an unsupervised mode, and consequently the judgment accuracy of a judgment model is low is solved. Moreover, the initial multi-round dialogue model not only can judge whether the current dialogue statement needs to be rewritten, but also can rewrite the current dialogue statement on the basis of at least one historical dialogue statement associated with the current dialogue statement when it is determined that the current dialogue statement needs to be rewritten, so that joint modeling of statement judgment and statement rewriting is realized; that is to say, the discrimination and rewriting of the initial multi-round dialogue model are trained, so that the dialogue rewriting effect of the multi-round dialogue model is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a training method, device, electronic device and storage medium for a multi-round dialogue model. Background Art

[0002] With the development of deep learning and big data technology, human-computer dialogue systems have become an important research work in the field of artificial intelligence, and have been widely used in intelligent customer service, chatbots, intelligent assistants, etc. Among them, according to the number of rounds of human-computer interaction, human-computer dialogue systems can be divided into: multi-round dialogue systems and single-round dialogue systems.

[0003] In a multi-round dialogue system, for the sake of simplicity and efficiency, reference and / or omission usually occur in the dialogue. For example, the name of an object that appeared in the previous round of dialogue sentences may be referred to by a person in the current round of dialogue sentences; for another example, the target entity (such as a place) that appeared in the previous round of dialogue sentences may be omitted in the current round of dialogue sentences. In this case, it is usually necessary to combine the information of the previous round of dialogue sentences to clarify the semantic information of the current round of dialogue sentences, so as to respond to the content of the current round of dialogue.

[0004] It can be seen that if there are omissions and references in the above multi-round dialogue, it may affect the machine's understanding of the language, and ultimately lead to poor results in the multi-round dialogue. Therefore, in order to improve the machine's understanding of the language, it is necessary to resolve references and / or complete omissions in dialogue sentences with omissions and / or references.

[0005] For example, in the related art, see Figure 1 As shown, firstly, a preset discrimination model is used to determine whether the current dialogue sentence needs to be rewritten (i.e., reference resolution and / or omission completion). If it is determined that the current dialogue sentence needs to be rewritten, the current dialogue sentence can be rewritten using the preset rewriting model and the context information of the current dialogue sentence.

[0006] However, the above method may result in low accuracy of the discriminant model because the discriminant model usually determines whether the current dialogue sentence needs to be rewritten in an unsupervised manner. In addition, since the discriminant model and the rewriting model use a pipeline working mode and the two models are independent of each other, the effect of dialogue rewriting is poor. Summary of the invention

[0007] The embodiments of the present application provide a method, device, electronic device and storage medium for training a multi-round dialogue model to improve the accuracy of rewriting judgment and the effect of dialogue rewriting.

[0008] In a first aspect, an embodiment of the present application provides a method for training a multi-round dialogue model, the method comprising:

[0009] Constructing a sample dialogue training set; the sample dialogue training set includes: at least one sample dialogue sentence with a first label, the first label of each sample dialogue sentence is used to indicate whether the corresponding sample dialogue sentence needs to be rewritten;

[0010] The initial multi-round dialogue model is iteratively trained multiple times based on the sample dialogue training set until the model loss value of the initial multi-round dialogue model meets the preset convergence condition, thereby obtaining a target multi-round dialogue model; the target multi-round dialogue model is used to determine whether the current dialogue sentence needs to be rewritten, and when it is determined that the current dialogue sentence needs to be rewritten, the current dialogue sentence is rewritten based on at least one historical dialogue sentence associated with the current dialogue sentence.

[0011] In an optional embodiment, constructing a sample dialogue training set includes:

[0012] For at least one sample dialogue sentence, perform the following operations:

[0013] Acquire at least one historical dialogue sentence associated with the first sample dialogue sentence; wherein the first sample dialogue sentence is any one of the at least one sample dialogue sentence;

[0014] Based on a standard dialogue sentence corresponding to at least one historical dialogue sentence, a first discrimination result of a first sample dialogue sentence is determined; wherein each standard dialogue sentence is a dialogue sentence that does not omit an entity and / or does not refer to an entity, and the first discrimination result indicates whether the first sample dialogue sentence is a dialogue sentence to be rewritten;

[0015] The first sample dialogue sentence is labeled based on the first discrimination result, and the first sample dialogue sentence with the first label is saved in a preset sample dialogue training set.

[0016] In an optional embodiment, performing iterative training on the initial multi-round dialogue model based on the sample dialogue training set includes:

[0017] During each training iteration, the following operations are performed:

[0018] Determine, by using the initial multi-round dialogue model, a second discrimination result of a second sample dialogue sentence selected from the sample dialogue training set, and when the second discrimination result indicates that the second sample dialogue sentence is a dialogue sentence to be rewritten, rewrite the second sample dialogue sentence based on at least one historical dialogue sentence associated with the second sample dialogue sentence;

[0019] Determine a first loss value between the predicted label corresponding to the second discrimination result and the first label of the second sample dialogue sentence, and determine a second loss value between the rewritten second sample dialogue sentence and the standard dialogue sentence corresponding to the second sample dialogue sentence;

[0020] Parameters of the initial multi-round dialogue model are adjusted based on the first loss value and the second loss value.

[0021] In an optional embodiment, rewriting the second sample dialogue sentence based on at least one historical dialogue sentence associated with the second sample dialogue sentence includes:

[0022] Using a preset character labeling method, a second label is added to a plurality of characters included in at least one historical dialogue sentence and a second sample dialogue sentence; wherein the second label corresponding to each character is used to indicate whether the corresponding character is used to determine a sentence rewriting position and a sentence rewriting content of the second sample dialogue sentence;

[0023] The second sample dialogue sentence is rewritten based on the second labels corresponding to the plurality of characters.

[0024] In an optional embodiment, determining a second loss value between the rewritten second sample dialogue sentence and a standard dialogue sentence corresponding to the second sample dialogue sentence includes:

[0025] A loss value obtained based on the predicted label and the true label respectively corresponding to at least one character included in the second sample dialogue sentence is used as the second loss value.

[0026] In an optional embodiment, if one of the following conditions is met, it is determined that the model loss value meets the preset convergence condition:

[0027] The first loss value is less than or equal to a preset first loss value threshold, and the second loss value is less than or equal to a preset second loss value threshold; or,

[0028] A third loss value obtained based on the first loss value and the second loss value is less than or equal to a preset third loss value threshold.

[0029] In an optional embodiment, after obtaining the target multi-turn dialogue model, the method further includes:

[0030] The target dialogue sentence is input into the target multi-turn dialogue model to obtain the rewritten target dialogue sentence and the rewriting confidence of the target dialogue sentence; the rewriting confidence represents: the accuracy of the target multi-turn dialogue model in determining that the target dialogue sentence is the sentence to be rewritten.

[0031] In a second aspect, an embodiment of the present application further provides a training device for a multi-round dialogue model, the device comprising:

[0032] A sample construction module is used to construct a sample dialogue training set; the sample dialogue training set includes: at least one sample dialogue sentence with a first label, and the first label of each sample dialogue sentence is used to indicate whether the corresponding sample dialogue sentence needs to be rewritten;

[0033] The model training module is used to perform multiple iterative training on the initial multi-round dialogue model based on the sample dialogue training set until the model loss value of the initial multi-round dialogue model meets the preset convergence condition, thereby obtaining a target multi-round dialogue model; the target multi-round dialogue model is used to determine whether the current dialogue sentence needs to be rewritten, and when it is determined that the current dialogue sentence needs to be rewritten, the current dialogue sentence is rewritten based on at least one historical dialogue sentence associated with the current dialogue sentence.

[0034] In an optional embodiment, when constructing a sample dialogue training set, the sample construction module is specifically used to:

[0035] For at least one sample dialogue sentence, perform the following operations:

[0036] Acquire at least one historical dialogue sentence associated with the first sample dialogue sentence; wherein the first sample dialogue sentence is any one of the at least one sample dialogue sentence;

[0037] Based on a standard dialogue sentence corresponding to at least one historical dialogue sentence, a first discrimination result of a first sample dialogue sentence is determined; wherein each standard dialogue sentence is a dialogue sentence that does not omit an entity and / or does not refer to an entity, and the first discrimination result indicates whether the first sample dialogue sentence is a dialogue sentence to be rewritten;

[0038] The first sample dialogue sentence is labeled based on the first discrimination result, and the first sample dialogue sentence with the first label is saved in a preset sample dialogue training set.

[0039] In an optional embodiment, when the initial multi-round dialogue model is iteratively trained based on the sample dialogue training set, the model training module is specifically used to:

[0040] During each training iteration, the following operations are performed:

[0041] Determine, by using the initial multi-round dialogue model, a second discrimination result of a second sample dialogue sentence selected from the sample dialogue training set, and when the second discrimination result indicates that the second sample dialogue sentence is a dialogue sentence to be rewritten, rewrite the second sample dialogue sentence based on at least one historical dialogue sentence associated with the second sample dialogue sentence;

[0042] Determine a first loss value between the predicted label corresponding to the second discrimination result and the first label of the second sample dialogue sentence, and determine a second loss value between the rewritten second sample dialogue sentence and the standard dialogue sentence corresponding to the second sample dialogue sentence;

[0043] Parameters of the initial multi-round dialogue model are adjusted based on the first loss value and the second loss value.

[0044] In an optional embodiment, when rewriting the second sample dialogue sentence based on at least one historical dialogue sentence associated with the second sample dialogue sentence, the model training module is specifically used to:

[0045] Using a preset character labeling method, a second label is added to a plurality of characters included in at least one historical dialogue sentence and a second sample dialogue sentence; wherein the second label corresponding to each character is used to indicate whether the corresponding character is used to determine a sentence rewriting position and a sentence rewriting content of the second sample dialogue sentence;

[0046] The second sample dialogue sentence is rewritten based on the second labels corresponding to the plurality of characters.

[0047] In an optional embodiment, when determining the second loss value between the rewritten second sample dialogue sentence and the standard dialogue sentence corresponding to the second sample dialogue sentence, the model training module is specifically used to:

[0048] A loss value obtained based on the predicted label and the true label respectively corresponding to at least one character included in the second sample dialogue sentence is used as the second loss value.

[0049] In an optional embodiment, the model training module determines that the model loss value satisfies a preset convergence condition if one of the following conditions is met:

[0050] The first loss value is less than or equal to a preset first loss value threshold, and the second loss value is less than or equal to a preset second loss value threshold; or,

[0051] A third loss value obtained based on the first loss value and the second loss value is less than or equal to a preset third loss value threshold.

[0052] In an optional embodiment, after obtaining the target multi-round dialogue model, the device further includes a dialogue rewriting module, and the dialogue rewriting module is specifically used to:

[0053] The target dialogue sentence is input into the target multi-turn dialogue model to obtain the rewritten target dialogue sentence and the rewriting confidence of the target dialogue sentence; the rewriting confidence represents: the accuracy of the target multi-turn dialogue model in determining that the target dialogue sentence is the sentence to be rewritten.

[0054] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0055] Processor; and

[0056] Memory for storing programs,

[0057] The program includes instructions, which, when executed by a processor, cause the processor to execute the training method for a multi-round dialogue model as described in the first aspect.

[0058] In a fourth aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the training method for a multi-round dialogue model as described in the first aspect.

[0059] In a fifth aspect, the present application provides a computer program product, which, when called by a computer, enables the computer to execute the steps of the training method for the multi-round dialogue model as described in the first aspect.

[0060] The beneficial effects of this application are as follows:

[0061] In the training method of the multi-round dialogue model provided in the embodiment of the present application, the initial multi-round dialogue model is iteratively trained by constructing a sample dialogue training set with a first label for indicating whether the corresponding sample dialogue sentence needs to be rewritten; this avoids the problem of low discrimination accuracy of the discrimination model caused by the unsupervised method in the prior art for judging whether the current dialogue sentence needs to be rewritten. In addition, since the initial multi-round dialogue model can not only judge whether the current dialogue sentence needs to be rewritten, but also rewrite the current dialogue sentence based on at least one historical dialogue sentence associated with the current dialogue sentence when it is determined that the current dialogue sentence needs to be rewritten, the joint modeling of sentence judgment and sentence rewriting is realized, that is, the judgment and rewriting of the initial multi-round dialogue model are both trained, thereby improving the dialogue rewriting effect of the multi-round dialogue model.

[0062] In addition, other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described here are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0064] Figure 1 A logic diagram of dialogue sentence rewriting provided in an embodiment of the present application;

[0065] Figure 2 A schematic diagram of an optional system architecture applicable to the embodiments of the present application;

[0066] Figure 3 A schematic diagram of an implementation flow of a method for training a multi-round dialogue model provided in an embodiment of the present application;

[0067] Figure 4 A flowchart of a method for constructing a sample conversation training set provided in an embodiment of the present application;

[0068] Figure 5 A specific example schematic diagram of reference resolution and omission completion provided in an embodiment of the present application;

[0069] Figure 6 A schematic diagram of an implementation flow of an iterative training method for an initial multi-round dialogue model provided in an embodiment of the present application;

[0070] Figure 7 A specific example schematic diagram of a dialogue sentence and its corresponding second label provided in an embodiment of the present application;

[0071] Figure 8 A schematic diagram of a specific application scenario of dialogue sentence rewriting provided in an embodiment of the present application;

[0072] Fig. 9 A specific example schematic diagram of constructing a first tag and a second tag provided in an embodiment of the present application;

[0073] Fig.10 A logic diagram of dialogue sentence rewriting provided in an embodiment of the present application;

[0074] Fig.11 A schematic diagram of the structure of a training device for a multi-round dialogue model provided in an embodiment of the present application;

[0075] Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0077] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0078] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0079] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0080] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0081] Some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0082] 1. Artificial intelligence (AI): It is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0083] Artificial intelligence technology is a comprehensive discipline, which includes both hardware-level and software-level technologies. Among them, pre-trained models are also called large models and basic models. After fine-tuning, they can be widely used in downstream tasks in various directions of artificial intelligence. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies.

[0084] Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.

[0085] 2. Natural language processing (NLP): It is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing involves natural language, that is, the language used by people in daily life, which is closely related to linguistics research; at the same time, it involves computer science and mathematics. The pre-trained model, an important technology for model training in the field of artificial intelligence, is developed from the large language model in the field of natural language processing. After fine-tuning, the large language model can be widely used in downstream tasks. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph and other technologies. The pre-trained model is the latest development of deep learning, which integrates the above technologies.

[0086] 3. Machine learning: It is a multi-disciplinary interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. It is applied in all fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by teaching.

[0087] 4. Pretrained model: refers to a model trained on a large-scale corpus, usually using unsupervised learning methods, such as autoencoders, language models, etc. The basic idea of ​​the pretrained model is to use a large-scale corpus and unsupervised learning methods to allow the model to learn a large amount of general knowledge and rules, so as to serve as a basic model for various natural language processing tasks. As an example, the initial multi-round dialogue model involved in the embodiments of the present application can be a pretrained model, specifically any deep learning model; for example: a generative pre-trained transformer (Chat GPT) model, a Transformer (bidirectional encoder representations from transformers, BERT) model based on bidirectional encoding representations, etc.

[0088] 5. Transformer: It is a common deep learning model architecture and has been widely used in various fields such as natural language processing, computer vision (CV) and speech processing.

[0089] When Transformer was first proposed, it was a sequence-to-sequence model architecture for machine translation. It consists of an encoder and a decoder. Both encoder and decoder are composed of a series of Transformer blocks with the same structure. Each Transformer block consists of a multi-head self-attention layer and a feedforward neural network layer. Currently, Transformer has become a common architecture in natural language processing and is often used as a pre-training model. In addition to language-related applications, Transformer is also used in computer vision, audio processing and other fields.

[0090] 6. Language model: A model used to model natural language. Its purpose is to predict the next word or character in a given text sequence. Language models can be used for a variety of natural language processing tasks, such as text semantic extraction, text generation, machine translation, speech recognition, etc. At present, Transformer-based pre-trained language models (PLM) are more common in various natural language processing tasks and usually achieve relatively good results. For example, more common pre-trained language models include BERT models and generative pre-trained Transformer models (GPT).

[0091] 7. Large language model (LLM): refers to a natural language processing model with large-scale parameters and training data. The training process of a large language model usually adopts an unsupervised learning method, that is, the model is trained through a large-scale text corpus to learn the probability distribution and language rules of the language.

[0092] During the training process, large language models usually use the language model as the objective function, that is, optimizing the model parameters by maximizing the prediction probability of the next word. For example, the GPT series model based on the Transformer model structure is trained on a large-scale corpus and can generate high-quality natural language texts such as articles and conversations.

[0093] 8. Tokenizer: A tool that converts natural language text into a sequence of characters, words, or subwords. In the Transformer model, tokenizer usually refers to a tool that converts natural language text into a sequence of tokens (segmented words) required for model input. It usually uses a word or sub-token-based segmentation method, such as byte pair encoding (BPE) or sentence piece, which can split words or sub-words into smaller units so that the model can better handle uncommon words or words that are not in the vocabulary.

[0094] 9. Token: A minimal semantic unit, also known as a segmentation or a word unit. In the embodiments of the present application, it is the minimal unit for modeling subsequent text sequences, such as a word or character in a sentence. For ease of description and understanding, in the following text, tokens are uniformly described as characters.

[0095] 10. End-to-end: Given an input, the final output is obtained directly through the model.

[0096] 11. Pipeline: It is a pipeline of multiple models, where the input of the next model depends on the output of the previous model.

[0097] 12. Co-reference resolution: Co-reference refers to the use of demonstrative pronouns or abstract concepts to replace entities or objects in the previous information (historical dialogue sentences); resolution is to classify different references representing the same entity into the same set. For example, in dialogue rewriting, based on the previous information "Do you know xxx?", the current sentence "He is a singer" is restored to "xxx is a singer".

[0098] 13. Omission completion: Omission, also known as zero reference, means omitting the previous information (i.e. the historical dialogue sentence); completion means completing the current dialogue sentence. For example, in dialogue rewriting, the current dialogue sentence "I also like" can be restored to "I also like xxx" based on the previous information "I like xxx".

[0099] Based on the above-mentioned nouns and related terminology explanations, the design concept of the embodiments of the present application is briefly introduced below:

[0100] In order to communicate concisely and efficiently, humans often use references and omissions when conducting multiple rounds of dialogue in daily conversations. Since humans have the ability to remember, they can reconstruct the information that is referred to or omitted in the current dialogue sentence based on historical dialogue information (i.e., the previous dialogue information). Therefore, in (human-computer) dialogue systems, in order to complete information (i.e., omission completion) while reducing redundancy (i.e., reference resolution, for example, summarizing and extracting the previous multi-round dialogue information and rewriting it into a single-round question), it is particularly important to improve the question-answering effect of the dialogue system.

[0101] It can be seen that multi-round dialogue rewriting is an important task in the dialogue system. The core of multi-round dialogue rewriting is to resolve the reference and complete the omitted information of the current dialogue sentence. The current dialogue sentence rewriting is generally divided into two steps: first, the reference and omission are judged (the judgment model), and then the reference is resolved and the omission is completed (the rewriting model).

[0102] Among them, the role of the discriminant model is to determine whether the current dialogue sentence needs to be rewritten. The discriminant method can use syntactic analysis to determine the completeness of sentence components, or measure the topic similarity between the current dialogue sentence and the context. The rewriting model can rewrite the current dialogue sentence in combination with the above. The rewriting method can be divided into generative and extractive methods based on the model structure. The generative encoder-decoder can be based on long short-term memory (LSTM), Transformer and other models, and the extractive text encoder can be based on Word2vec, BERT and other models.

[0103] However, in the above method, since the discrimination stage usually uses a "heuristic" strategy to determine whether the current dialogue sentence needs to be rewritten in an unsupervised manner, this method relies on manually formulated rules and has a low accuracy rate; in the rewriting stage, the generation of the generative model is relatively uncontrollable, and the model training relies on a large amount of annotated corpus; in the extractive model based on pointer annotation, there are no strong constraints between the starting position pointers of multiple extracted entities during model training, resulting in poor model performance. In addition, there is no interaction between the models in the two stages, which usually leads to error accumulation, resulting in poor overall performance.

[0104] In addition, although LLM can also perform a variety of traditional NLP tasks according to user instructions, such as dialogue rewriting, etc., calling LLM consumes more computing resources. Considering performance factors, LLM is not suitable for high-concurrency scenarios.

[0105] In view of this, in order to solve or improve the above-mentioned problems, an embodiment of the present application proposes an end-to-end dialogue model training method, which can jointly model the two tasks of discrimination and rewriting, that is, an embodiment of the present application proposes a multi-round dialogue model training method, which specifically includes: first, constructing a sample dialogue training set; wherein the sample dialogue training set includes at least one sample dialogue sentence with a first label, and the first label of each sample dialogue sentence is used to indicate whether the corresponding sample dialogue sentence needs to be rewritten; finally, based on the sample dialogue training set, the initial multi-round dialogue model is iteratively trained for multiple times until the model loss value of the initial multi-round dialogue model meets the preset convergence condition, thereby obtaining a target multi-round dialogue model; the target multi-round dialogue model is used to discriminate whether the current dialogue sentence needs to be rewritten, and when it is determined that the current dialogue sentence needs to be rewritten, the current dialogue sentence is rewritten based on at least one historical dialogue sentence associated with the current dialogue sentence. In this way, by constructing a sample dialogue training set with a first label for indicating whether the corresponding sample dialogue sentence needs to be rewritten, the initial multi-round dialogue model is iteratively trained; this avoids the problem of low discrimination accuracy of the discrimination model caused by the unsupervised method in the prior art for determining whether the current dialogue sentence needs to be rewritten.

[0106] Furthermore, since the initial multi-round dialogue model can not only determine whether the current dialogue sentence needs to be rewritten, but also, when it is determined that the current dialogue sentence needs to be rewritten, rewrite the current dialogue sentence based on at least one historical dialogue sentence associated with the current dialogue sentence, joint modeling of sentence determination and sentence rewriting is achieved, that is, both the determination and rewriting of the initial multi-round dialogue model are trained, thereby improving the dialogue rewriting effect of the multi-round dialogue model.

[0107] In particular, the preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.

[0108] See also Figure 2As shown, it is a schematic diagram of a system architecture applicable to an embodiment of the present application, and the system architecture may include: a terminal device (201a, 201b) and a server 202. The terminal device (201a, 201b) and the server 202 may exchange information through a communication network, wherein the communication mode adopted by the communication network may include: a wireless communication mode and a wired communication mode. Exemplarily, the terminal device (201a, 201b) may access the network through cellular mobile communication technology and communicate with the server 202. Wherein, the cellular mobile communication technology, for example, includes the fifth generation mobile communication (5th generation mobile networks, 5G) technology or the next generation mobile communication technology. Optionally, the terminal device (201a, 201b) may access the network through a short-range wireless communication mode and communicate with the server 202. Wherein, the short-range wireless communication mode, for example, includes wireless fidelity (wireless fidelity, Wi-Fi) technology.

[0109] The embodiment of the present application does not impose any restriction on the number of communication devices involved in the above system architecture. For example, the above system architecture may include more terminal devices, or fewer terminal devices, or other network devices. Figure 2 As shown, only the terminal devices (201a, 201b) and the server 202 are described as examples, and the above communication devices and their respective functions are briefly introduced below.

[0110] The terminal device (201a, 201b) is a device that can provide voice and / or data connectivity to users, and can be a device that supports wired and / or wireless connection.

[0111] Exemplarily, the terminal devices (201a, 201b) may include, but are not limited to: mobile phones, tablet computers, laptop computers, PDAs, mobile internet devices (MID), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminal devices in industrial control, wireless terminal devices in unmanned driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0112] In addition, a related client may be installed on the terminal device (201a, 201b), and the client may be software, for example, an application (APP), a browser, a short video software, etc., or a web page, a mini-program, etc.; it should be noted that in an embodiment of the present application, the terminal device (201a, 201b) may enable the above-mentioned client related to the training of the multi-round dialogue model to send at least one sample dialogue sentence and / or at least one historical dialogue sentence corresponding to at least one sample dialogue sentence to the server 202, so as to perform subsequent multi-round dialogue model training and other method steps.

[0113] Optionally, the terminal device (201a, 201b) can enable the above-mentioned client related to the training of the multi-round dialogue model to send the target dialogue sentence of the target object to the server 202, so as to subsequently determine whether the target dialogue sentence needs to be modified, and modify the target dialogue sentence when it is determined that the target dialogue sentence needs to be modified.

[0114] Server 202 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0115] It should be noted that if Figure 2 As shown, in the embodiment of the present application, a pre-trained language model (i.e., an initial multi-round dialogue model, such as a BERT-based dialogue model) is deployed on the server 202. After the server 202 completes the training of the initial multi-round dialogue model, the target initial multi-round dialogue model obtained can realize the reference resolution and / or omission completion of the current dialogue sentence. Optionally, the server 202 can obtain the rewritten target dialogue sentence and the rewriting confidence of the target dialogue sentence when inputting the target dialogue sentence into the target multi-round dialogue model.

[0116] It is worth mentioning that in the embodiment of the present application, the server 202 is used to construct a sample dialogue training set of at least one sample dialogue sentence with a first label, so as to perform multiple iterative training on the initial multi-round dialogue model based on the sample dialogue training set until the model loss value of the initial multi-round dialogue model meets the preset convergence condition, thereby obtaining a target multi-round dialogue model. The first label of each sample dialogue sentence is used to indicate whether the corresponding sample dialogue sentence needs to be rewritten.

[0117] The following describes the training method of the multi-round dialogue model provided by the exemplary embodiment of the present application in combination with the above-mentioned system architecture and with reference to the accompanying drawings. It should be noted that the above-mentioned system architecture is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect.

[0118] See also Figure 3 As shown, it is a schematic diagram of the implementation process of a training method for a multi-round dialogue model provided in an embodiment of the present application. The execution subject takes a server as an example. The specific implementation process of the method is as follows:

[0119] S301: Construct a sample dialogue training set.

[0120] The sample dialogue training set may include: at least one sample dialogue sentence with a first label, and the first label of each sample dialogue sentence may be used to indicate whether the corresponding sample dialogue sentence needs to be rewritten. In other words, the first label may be a binary classification label. In the embodiment of the present application, the specific form of the first label is not limited. For example, the first label may be any one or a combination of uppercase English letters, lowercase English letters, Arabic numerals, etc.

[0121] Taking the example of the first label represented by Arabic numerals, "1" indicates that the sample dialogue sentence is a positive example (i.e., a positive sample dialogue sentence), that is, the sample dialogue sentence needs to be rewritten; "0" indicates that the sample dialogue sentence is a negative example (i.e., a negative sample dialogue sentence), that is, the sample dialogue sentence does not need to be rewritten. Optionally, "1" can be called a positive label, and "0" can be called a negative label. For example, if sample dialogue sentence 1 in the above sample dialogue training set needs to be rewritten, the first label of sample dialogue sentence 1 is "1"; if sample dialogue sentence 2 in the above sample dialogue training set does not need to be rewritten, the first label of sample dialogue sentence 2 is "0".

[0122] Whether the sample dialogue sentence needs to be rewritten can be determined based on whether the sample dialogue sentence has reference and / or omission. Optionally, if the sample dialogue sentence needs to be rewritten, it means that the sample dialogue sentence has reference and / or omission, and if the sample dialogue sentence does not need to be rewritten, it means that the sample dialogue sentence does not have reference and omission.

[0123] For example, the first label of sample dialogue sentence 1 is "1", indicating that sample dialogue sentence 1 has reference and / or omission, and the first label of sample dialogue sentence 2 is "0", indicating that sample dialogue sentence 1 does not have reference and omission.

[0124] In an optional implementation, when executing step S301, the server may label the at least one sample dialogue sentence with a first label according to the at least one sample dialogue sentence and the at least one historical dialogue sentence associated therewith, thereby constructing a sample dialogue training set. That is, the model training data can be constructed according to the open source dialogue rewriting corpus REWRITE.

[0125] The above historical dialogue sentences, i.e., the previous (question-and-answer) dialogue sentences of the sample dialogue sentences, can be used to determine whether the sample dialogue sentences contain references and / or omissions, i.e., to determine whether the sample dialogue sentences need to be rewritten. In other words, at least one historical dialogue sentence associated with each sample dialogue sentence is a dialogue sentence of the same dialogue scene.

[0126] For example, see Figure 4 As shown, the server can perform the following operations respectively for any one of the at least one sample dialogue sentence, that is, the first sample dialogue sentence:

[0127] S401: Obtain at least one historical dialogue sentence associated with a first sample dialogue sentence.

[0128] Taking the first sample dialogue sentence "Why is it always like this" as an example, the first sample dialogue sentence is associated with two historical dialogue sentences, such as "How is the weather today" and "Today is cloudy" respectively. For another example, taking the first sample dialogue sentence "I want to know the company's process" as an example, the first sample dialogue sentence is associated with a historical dialogue sentence, such as "How to ask for leave". It should be noted that the first sample dialogue sentence and at least one historical dialogue sentence associated with the first sample dialogue sentence can be dialogue sentences of the same object, or can be dialogue sentences of different objects, as shown in Table 1:

[0129] Table 1 Examples of objects corresponding to different dialogue sentences

[0130]

[0131] Based on the mapping relationship between dialogue sentences and objects recorded in Table 1 above, it can be seen that the object (user_B) corresponding to the historical dialogue sentence 2 "Today is cloudy" in dialogue example 1 is different from the object (user_A) corresponding to the historical dialogue sentence 1 "How is the weather today" and the first sample dialogue sentence "Why is it always like this?"

[0132] Furthermore, the object (user_A) corresponding to the historical dialogue sentence 1 "How to ask for leave" in dialogue example 2 is the same as the object (user_A) corresponding to the first sample dialogue sentence "I want to know the company's procedures".

[0133] It can be seen that, since the number of objects involved in the sample dialogue sentences and their associated historical dialogue sentences (e.g., a single object or multiple objects) is not specifically limited when constructing the sample dialogue sentences included in the sample dialogue training set, the types of sample dialogue sentences in the sample dialogue training set are enriched (i.e., sample dialogue sentences corresponding to single-object historical dialogue sentences and sample dialogue sentences corresponding to multiple-object historical dialogue sentences), thereby improving the generalization ability of the subsequent multi-round dialogue model, that is, not only can dialogue sentences for the same object be rewritten, but dialogue sentences can also be rewritten according to at least one dialogue sentence corresponding to multiple objects. Therefore, the rewriting effect of dialogue sentences is also improved to a certain extent.

[0134] S402: Determine a first discrimination result of a first sample dialogue sentence based on a standard dialogue sentence corresponding to at least one historical dialogue sentence.

[0135] Among them, each standard dialogue sentence can be a dialogue sentence without omitting an entity and / or referring to an entity, that is, the aforementioned standard dialogue sentence is a dialogue sentence with reference resolution and omission completion corresponding to the first sample dialogue sentence. It should be noted that the aforementioned entity can be a person, an object, or a place, etc., and this embodiment of the application is not limited to this.

[0136] Still taking the dialogue sentences in Table 1 as an example, the standard dialogue sentences corresponding to the historical dialogue sentence 1 "How is the weather today?", the historical dialogue sentence 2 "Today is cloudy" and the first sample dialogue sentence "Why is it always like this?" in the dialogue example 1 can be "Why is it always like this?" cloudy day In the dialogue example 2, the standard dialogue sentence corresponding to the historical dialogue sentence 1 "How to ask for leave" and the first sample dialogue sentence "I want to know the company's procedures" can be "I want to know the company's procedures". Ask for leave process".

[0137] See also Figure 5 As shown, if the above-mentioned at least one historical dialogue sentence is represented as H = [U1, U1, ..., Un-1], and the first sample dialogue sentence is represented as Un, then in order to achieve reference elimination and omission completion of the first sample sentence, that is, according to the above-mentioned at least one historical dialogue sentence H, the first sample dialogue sentence Un can be rewritten into a standard dialogue sentence R, which can be represented as (H, Un) → R. Figure 5 As shown, the server can implement reference resolution of the first sample dialogue sentence in dialogue example 1, and omission completion of the first sample dialogue sentence in dialogue example 1.

[0138] For another example, if historical dialogue sentence 1 is "xx tea", historical dialogue sentence 2 is "Is xx tea delicious?" and the first sample dialogue sentence is "Hmm, xx tea is delicious" (i.e., dialogue example 3), the standard dialogue sentence corresponding to the aforementioned historical dialogue sentence 1, historical dialogue sentence 2 and the first sample dialogue sentence in dialogue example 2 can also be "Hmm, xx tea is delicious".

[0139] The above-mentioned first judgment result can indicate whether the first sample dialogue sentence is a dialogue sentence to be rewritten, that is, if the first sample dialogue sentence is the same as the corresponding standard dialogue sentence, there is no need to rewrite the first sample dialogue sentence; on the contrary, if the first sample dialogue sentence is different from the corresponding standard dialogue sentence, the first sample dialogue sentence needs to be rewritten.

[0140] Exemplarily, the first judgment result of the above-mentioned dialogue example 1 is that the first sample dialogue sentence in dialogue example 1 "Why is it always like this" is a sentence to be rewritten, the first judgment result of the above-mentioned dialogue example 2 is that the first sample dialogue sentence in dialogue example 2 "I want to know the company's procedures" is also a sentence to be rewritten, and the first judgment result of the above-mentioned dialogue example 3 is that the first sample dialogue sentence in dialogue example 3 "Hmm, xx tea is delicious" is not a sentence to be rewritten.

[0141] S403: labeling the first sample dialogue sentence based on the first discrimination result, and saving the first sample dialogue sentence labeled with the first label into a preset sample dialogue training set.

[0142] Specifically, when executing step S403, after the server obtains the first discrimination result of the first sample dialogue sentence, if the first discrimination result indicates that the first sample dialogue sentence is a sentence to be rewritten, the server marks the first sample dialogue sentence with a first label (for example, the first label is "1") for indicating that the first sample dialogue sentence needs to be rewritten; otherwise, if the first discrimination result indicates that the first sample dialogue sentence is not a sentence to be rewritten, the server marks the first sample dialogue sentence with a first label (for example, the first label is "0") for indicating that the first sample dialogue sentence does not need to be rewritten.

[0143] For example, taking the above-mentioned dialogue example 1 as an example, since the first judgment result of the first sample dialogue sentence "Why is it always like this" in dialogue example 1 is that the first sample dialogue sentence is a sentence to be rewritten, the server can mark the first sample dialogue sentence in dialogue example 1 with a first label "1". Taking the above-mentioned dialogue example 3 as an example, since the first judgment result of the first sample dialogue sentence "Hmm, xx tea tastes good" in dialogue example 3 is that the first sample dialogue sentence is not a sentence to be rewritten, the server can mark the first sample dialogue sentence in dialogue example 1 with a first label "0".

[0144] Furthermore, after the server tags the first sample dialogue sentence with the first label, it can save the first sample dialogue sentence with the first label to a preset sample dialogue training set. The preset sample dialogue training set is used to perform multiple iterations of training on the initial multi-round dialogue model (i.e., the multi-round dialogue model to be trained) in the future. It should be understood that if the first sample dialogue sentence is the first sample dialogue sentence with the first label among the at least one sample dialogue sentence, the preset sample dialogue training set is empty, i.e., does not contain the sample dialogue sentence with the first label.

[0145] It should be noted that the above-mentioned preset sample dialogue training set may also include at least one historical dialogue sentence associated with at least one sample dialogue sentence, so that there is no need to obtain the historical dialogue sentence associated with each sample dialogue sentence from other databases later. Of course, the above-mentioned preset sample dialogue training set may also not include at least one historical dialogue sentence associated with at least one sample dialogue sentence, and the embodiment of the present application is not limited to this.

[0146] Therefore, by respectively executing the method steps described in S401 to S403 for the at least one sample dialogue sentence, a sample dialogue training set including at least one sample dialogue sentence with a first label can be constructed. In this way, the server can perform supervised training on the discrimination stage of the initial multi-round dialogue model based on the sample dialogue training set, thereby improving the accuracy of the subsequent target multi-round dialogue model (i.e., the trained multi-round dialogue model) in discriminating whether a dialogue sentence needs to be rewritten.

[0147] S302: Perform multiple iterations of training on the initial multi-round dialogue model based on the sample dialogue training set until the model loss value of the initial multi-round dialogue model meets a preset convergence condition, thereby obtaining a target multi-round dialogue model.

[0148] The target multi-turn dialogue model can be used to determine whether the current dialogue sentence needs to be rewritten, and when it is determined that the current dialogue sentence needs to be rewritten, the current dialogue sentence is rewritten based on at least one historical dialogue sentence associated with the current dialogue sentence. In other words, the target multi-turn dialogue model has the ability to both sentence determination and sentence rewriting.

[0149] In an optional implementation, when executing step S302, the server may perform multiple iterative training of discrimination and rewriting on the initial multi-round dialogue model according to at least one sample dialogue sentence with a first label included in the sample dialogue training set. Figure 6 As shown, the server performs the following operations during each iteration of training:

[0150] S601: Determine a second discrimination result of a second sample dialogue sentence selected from a sample dialogue training set through an initial multi-round dialogue model, and when the second discrimination result indicates that the second sample dialogue sentence is a dialogue sentence to be rewritten, rewrite the second sample dialogue sentence based on at least one historical dialogue sentence associated with the second sample dialogue sentence.

[0151] The second sample dialogue sentence is any one of the at least one sample dialogue sentence, and the embodiment of the present application does not specifically limit the selection method of the second sample dialogue sentence. For example, the second sample dialogue sentence can be randomly selected from the at least one sample dialogue sentence, and of course, the second sample dialogue sentence can also be selected from multiple sample dialogue sentences that have never participated in the initial multi-round dialogue model training.

[0152] It should be noted that the above-mentioned method of determining the second discrimination result of the second sample dialogue sentence selected from the sample dialogue training set through the initial multi-round dialogue model is the same as the method of determining the first discrimination result of the first sample dialogue sentence in the process of constructing the sample dialogue training set, and the embodiments of the present application will not be repeated here.

[0153] In addition, when the server determines that the second sample dialogue sentence is the dialogue sentence to be rewritten according to the second discrimination result, the server can rewrite the second sample dialogue sentence according to at least one historical dialogue sentence associated with the second sample dialogue sentence, thereby rewriting the second sample dialogue sentence into a standard dialogue sentence corresponding to the aforementioned at least one historical dialogue sentence. Exemplarily, if the second sample dialogue sentence is the first sample dialogue sentence "Why is it always like this" in the above-mentioned dialogue example 1, the server can rewrite the second sample dialogue sentence "Why is it always like this" into the corresponding standard dialogue sentence "Why is it always cloudy" according to the historical dialogue sentence 1 "How is the weather today" and the historical dialogue sentence 2 "Today is cloudy" in the dialogue example 1, thereby solving the problem of reference in the second sample dialogue sentence, that is, realizing reference resolution of the second sample dialogue sentence.

[0154] In order to reduce the complexity of sentence rewriting and save server resource overhead, when the server rewrites the second sample dialogue sentence based on at least one historical dialogue sentence associated with the second sample dialogue sentence, the server can also use a preset character labeling method to add a second label to multiple characters included in at least one historical dialogue sentence and the second sample dialogue sentence, thereby rewriting the second sample dialogue sentence based on the second labels corresponding to the multiple characters.

[0155] Optionally, the above-mentioned preset character labeling method can be a "BIOS" sequence labeling method, where "B" stands for Begin, which can represent the character at the beginning of the entity segment; "I" stands for Intermediate, which can represent the character in the middle or at the end of the entity segment; "S" stands for Single, which can represent that the entity segment has only a single character; "O" stands for Other, which can represent other characters that are not entities. Exemplarily, the above-mentioned second label can include: "B-replace, I-replace, S-replace" for replacing entities, "B-coref, I-coref" for referring to entities, "S-insert" for inserting before, and "O" for not being an entity, etc. See. Figure 7 As shown, the server can add second labels to multiple characters included in the historical dialogue sentences and the sample dialogue sentences according to the aforementioned multiple types of second labels.

[0156] It should be noted that the above-mentioned preset character annotation method is not limited in the embodiments of the present application. For example, the above-mentioned preset character annotation method can be a "BIO" sequence annotation method, where "B" stands for Begin, which can represent the character at the beginning of the entity segment; "I" stands for Intermediate, which can represent the character in the middle or at the end of the entity segment; "O" stands for Other, which can represent other characters that are not entities. Of course, the above-mentioned preset character annotation method can also be other character annotation methods.

[0157] The second label corresponding to each character may be used to indicate whether the corresponding character is used to determine the sentence rewriting position and sentence rewriting content of the second sample dialogue sentence. Figure 7 Taking the "BIOS" sequence annotation method shown as an example, the second label "B-coref, I-coref, I-coref, I-coref" corresponding to "xxxx" in dialogue example a and the second label "S-replace" corresponding to "she" can determine the sentence rewriting position as: "she" and the sentence rewriting content as: rewrite "she" to "xxxx". The second label "B-coref, I-coref, I-coref" corresponding to "xx tea" that appears for the second time in dialogue example b and the second label "S-insert" corresponding to "good" can determine the sentence rewriting position as: "good" and the sentence rewriting content as: insert the characters "xx tea" before "good". The second label "B-coref, I-coref, I-coref" corresponding to "ABC" in dialogue example c, the second label "B-coref, I-coref, I-coref" corresponding to "photo", and the second label "S-insert" corresponding to the empty character after "take a look" can determine the sentence rewriting position as: the empty character after "take a look" and the sentence rewriting content as: add "ABC" to the empty character.

[0158] In an alternative implementation, see Figure 8 As shown, the server can encode the current conversation sentence (such as the second sample conversation sentence) and the historical conversation sentence based on the BERT model, where [CLS] and [SEP] are special symbols, [CLS] is used as the overall representation of the input text (i.e., the current conversation sentence cur_query: "I want to know the company's process" and the historical conversation sentence context: "How to ask for leave") to determine whether the current sentence needs to be rewritten, and [SEP] is the separator that separates the historical conversation and the current conversation. Figure 8 As shown, when the server adopts a preset character annotation method (such as the "BIOS" sequence annotation method), it can first extract word vector features from the input text to obtain word vectors corresponding to each character, for example, E_1 to E_16 in sequence; then, the second label can be annotated according to each word vector to improve the accuracy of label annotation; finally, according to the second labels corresponding to the current dialogue sentence and the historical dialogue sentence, the current dialogue sentence is rewritten to obtain the rewritten current dialogue sentence rewrite_query: "I want to know the company's leave application process". In addition, [CLS] can also be used to output the rewriting confidence corresponding to the discrimination task, such as Figure 8 The rewrite confidence in is 0.997.

[0159] Based on the above method, the server not only realizes the construction of the identification task label (i.e., the first label), but also realizes the construction of the rewriting task label (i.e., the second label). Fig. 9 As shown, the server can determine whether the current query is a positive example or a negative example (i.e., whether the current query needs to be rewritten) based on the difference between the current query (i.e., the current dialogue sentence) and the ground truth (i.e., the standard dialogue sentence), thereby constructing a discrimination task label; and, it can also construct a rewriting task label by combining the previous query and the previous answer (i.e., the historical dialogue sentence) of the current query.

[0160] S602: Determine a first loss value between the predicted label corresponding to the second discrimination result and the first label of the second sample dialogue sentence, and determine a second loss value between the rewritten second sample dialogue sentence and the standard dialogue sentence corresponding to the second sample dialogue sentence.

[0161] The first loss value can be used to characterize the accuracy of the initial multi-round dialogue model or the target multi-round dialogue model in determining whether a sentence is a sentence to be rewritten. For example, the first loss value can be expressed as Loss_1. The second loss value can be used to characterize the rewriting accuracy of the sentence rewriting result of the initial multi-round dialogue model or the target multi-round dialogue model. For example, the second loss value can be expressed as Loss_2.

[0162] Exemplarily, when executing step S602, the server can use the cross entropy loss function to calculate the first loss value Loss_1 and the second loss value Loss_2. Of course, other loss function calculation methods can also be used to calculate the first loss value Loss_1 and the second loss value Loss_2. In addition, the loss function for calculating the first loss value Loss_1 and the loss function for calculating the second loss value Loss_2 can be the same or different, and this embodiment of the application does not limit this.

[0163] In order to reduce the calculation complexity of the second loss value, in an optional implementation, the server may calculate the loss values ​​obtained by the predicted label and the true label respectively corresponding to at least one character included in the second sample dialogue sentence, and thereby use the loss values ​​obtained by the predicted label and the true label respectively corresponding to the aforementioned at least one character as the second loss value.

[0164] Optionally, the server may use a preset loss function (e.g., a cross entropy loss function) to calculate the sub-loss values ​​between the predicted (second) label and the true (second) label corresponding to at least one character included in the second sample dialogue sentence, thereby obtaining the loss value of the second sample dialogue sentence by weighted summation according to the obtained at least one sub-loss value, and then using the loss value of the second sample dialogue sentence as the second loss value. For example, taking the second sample dialogue sentence including 5 characters as an example, that is, taking 5 sub-loss values ​​as an example. The 5 loss values ​​are respectively: 0.042, 0.027, 0.035, 0.009 and 0.022, then the second loss value may be: (0.042+0.027+0.035+0.009+0.022) / 5=0.027.

[0165] Based on the above method, for the binary classification labeling task of determining whether the second sample dialogue sentence needs to be rewritten, a preset first loss function (e.g., a cross entropy loss function) can be used to calculate the first loss value Loss_1; and, for the sequence labeling task of the text sequence corresponding to the second sample dialogue sentence (i.e., the second labeling task), a preset second loss function (e.g., a cross entropy loss function) can also be used to calculate the second loss value Loss_2.

[0166] S603: Adjust parameters of the initial multi-round dialogue model based on the first loss value and the second loss value.

[0167] The above parameters are network model parameters related to determining whether a dialogue sentence needs to be rewritten and whether the dialogue sentence rewriting result is accurate in the initial multi-round dialogue model. The embodiment of the present application does not limit the specific content of the network model parameters.

[0168] Exemplarily, when executing step S603, the server may adjust at least one first parameter in the initial multi-round dialogue model (i.e., a network model parameter related to determining whether a dialogue statement needs to be rewritten) based on the first loss value, and may adjust at least one second parameter in the initial multi-round dialogue model (i.e., determining whether the dialogue statement rewriting result is accurate) based on the second loss value. Optionally, the server may also jointly adjust the aforementioned at least one first parameter and at least one second parameter based on the first loss value and the second loss value, for example, the aforementioned at least one first parameter and at least one second parameter may be adjusted based on a third loss value obtained from the first loss value and the second loss value.

[0169] In view of this, when executing the above step S302, if one of the following conditions is met, the server can determine that the model loss value meets the preset convergence condition:

[0170] Condition 1: The first loss value is less than or equal to a preset first loss value threshold, and the second loss value is less than or equal to a preset second loss value threshold.

[0171] For example, if it is assumed that the preset first loss value threshold is 0.01, the preset second loss value threshold is 0.005, and the first loss value is 0.008 and the second loss value is 0.0035, then the server can determine that the first loss value 0.008 is less than the preset first loss value threshold 0.01, and the second loss value 0.0035 is less than the preset first loss value threshold 0.005. That is, the server can determine that the model loss value meets the preset convergence condition, and then stop the training of the initial multi-round dialogue model.

[0172] Condition 2: The third loss value obtained based on the first loss value and the second loss value is less than or equal to a preset third loss value threshold.

[0173] Optionally, the third loss value may be the sum of the first loss value and the second loss value, that is, Loss_3 = Loss_1 + Loss_2. If it is assumed that the first loss value is 0.0075, the second loss value is 0.0028, and the preset third loss value threshold is 0.012, it can be determined that the third loss value is 0.0103, which is less than the preset third loss value threshold of 0.012. That is, the server can determine that the model loss value meets the preset convergence condition, and then stop the training of the initial multi-round dialogue model.

[0174] The above-mentioned method of performing multiple iterative training on the initial multi-round dialogue model can be understood as an optimization method based on gradient descent (that is, the mode loss value converges); in this way, supervised training of the initial multi-round dialogue model is achieved, thereby improving the accuracy of rewriting judgment and the effect of dialogue rewriting of the subsequently obtained target multi-round dialogue model.

[0175] Based on the method steps described in S301 to S302 above, refer to Fig.10 As shown, the server can obtain the current dialogue sentence and the historical dialogue sentence, as well as the target multi-round dialogue model (such as the BERT model), and complete the discrimination task for the current dialogue sentence (i.e., whether it needs to be rewritten), and the rewriting task of the current dialogue sentence (i.e., when it is determined that the current dialogue sentence needs to be rewritten, the current dialogue sentence is rewritten according to the historical dialogue sentence). In an optional implementation, after obtaining the target multi-round dialogue model, the server can also input the target dialogue sentence into the target multi-round dialogue model to obtain the rewritten target dialogue sentence and the rewriting confidence of the target dialogue sentence. Among them, the aforementioned rewriting confidence represents: the discrimination accuracy of the target multi-round dialogue model in judging the target dialogue sentence as the sentence to be rewritten. In this way, not only the sentence to be rewritten can be obtained, but also the confidence of the target multi-round dialogue model in judging the sentence to be rewritten can be obtained, so that the user can know the reliability of using the rewritten dialogue sentence, and then determine whether to use the rewritten dialogue sentence.

[0176] In addition, the two tasks of discrimination and rewriting are jointly modeled. On the one hand, the initial multi-round dialogue model is discriminated and trained based on the labeled data (i.e., the sample dialogue training set with the first label). The discrimination accuracy is greatly improved compared with the discrimination model of the traditional "heuristic" strategy. At the same time, the model outputs the rewriting confidence, which can provide a reference for the downstream tasks of the system (e.g., the application task of rewriting sentences), and facilitates the balance between accuracy and recall (e.g., the harmonic mean F1 of precision and recall). On the other hand, the use of sequence labeling (i.e., the sample dialogue training set of the present application) instead of pointer labeling naturally solves the constraint problem of the starting position of the entity, and improves the effect of dialogue rewriting without increasing the computational complexity.

[0177] In summary, in the training method of the multi-round dialogue model provided in the embodiment of the present application, by constructing a sample dialogue training set with a first label for indicating whether the corresponding sample dialogue sentence needs to be rewritten, the initial multi-round dialogue model is iteratively trained, thereby avoiding the problem of low discrimination accuracy of the discrimination model caused by the unsupervised method in the prior art for judging whether the current dialogue sentence needs to be rewritten. In addition, since the initial multi-round dialogue model can not only judge whether the current dialogue sentence needs to be rewritten, but also rewrite the current dialogue sentence based on at least one historical dialogue sentence associated with the current dialogue sentence when it is determined that the current dialogue sentence needs to be rewritten, the joint modeling of judgment and rewriting is realized, that is, the judgment and rewriting of the initial multi-round dialogue model are both trained, thereby improving the dialogue rewriting effect of the multi-round dialogue model.

[0178] Further, based on the same technical concept, the embodiment of the present application provides a training device for a multi-round dialogue model, and the training of the multi-round dialogue model is used to implement the above method flow of the embodiment of the present application. Fig.11 As shown, the training device 1100 of the multi-round dialogue model includes: a sample construction module 1101 and a model training module 1102, wherein:

[0179] The sample construction module 1101 is used to construct a sample dialogue training set; the sample dialogue training set includes: at least one sample dialogue sentence with a first label, and the first label of each sample dialogue sentence is used to indicate whether the corresponding sample dialogue sentence needs to be rewritten;

[0180] The model training module 1102 is used to perform multiple iterative training on the initial multi-round dialogue model based on the sample dialogue training set until the model loss value of the initial multi-round dialogue model meets the preset convergence condition, thereby obtaining a target multi-round dialogue model; the target multi-round dialogue model is used to determine whether the current dialogue sentence needs to be rewritten, and when it is determined that the current dialogue sentence needs to be rewritten, the current dialogue sentence is rewritten based on at least one historical dialogue sentence associated with the current dialogue sentence.

[0181] In an optional embodiment, when constructing a sample dialogue training set, the sample construction module 1101 is specifically used to:

[0182] For at least one sample dialogue sentence, perform the following operations:

[0183] Acquire at least one historical dialogue sentence associated with the first sample dialogue sentence; wherein the first sample dialogue sentence is any one of the at least one sample dialogue sentence;

[0184] Based on a standard dialogue sentence corresponding to at least one historical dialogue sentence, a first discrimination result of a first sample dialogue sentence is determined; wherein each standard dialogue sentence is a dialogue sentence that does not omit an entity and / or does not refer to an entity, and the first discrimination result indicates whether the first sample dialogue sentence is a dialogue sentence to be rewritten;

[0185] The first sample dialogue sentence is labeled based on the first discrimination result, and the first sample dialogue sentence with the first label is saved in a preset sample dialogue training set.

[0186] In an optional embodiment, when performing iterative training on the initial multi-round dialogue model based on the sample dialogue training set, the model training module 1102 is specifically used to:

[0187] During each training iteration, the following operations are performed:

[0188] Determine, by using the initial multi-round dialogue model, a second discrimination result of a second sample dialogue sentence selected from the sample dialogue training set, and when the second discrimination result indicates that the second sample dialogue sentence is a dialogue sentence to be rewritten, rewrite the second sample dialogue sentence based on at least one historical dialogue sentence associated with the second sample dialogue sentence;

[0189] Determine a first loss value between the predicted label corresponding to the second discrimination result and the first label of the second sample dialogue sentence, and determine a second loss value between the rewritten second sample dialogue sentence and the standard dialogue sentence corresponding to the second sample dialogue sentence;

[0190] Parameters of the initial multi-round dialogue model are adjusted based on the first loss value and the second loss value.

[0191] In an optional embodiment, when rewriting the second sample dialogue sentence based on at least one historical dialogue sentence associated with the second sample dialogue sentence, the model training module 1102 is specifically configured to:

[0192] Using a preset character labeling method, a second label is added to a plurality of characters included in at least one historical dialogue sentence and a second sample dialogue sentence; wherein the second label corresponding to each character is used to indicate whether the corresponding character is used to determine a sentence rewriting position and a sentence rewriting content of the second sample dialogue sentence;

[0193] The second sample dialogue sentence is rewritten based on the second labels corresponding to the plurality of characters.

[0194] In an optional embodiment, when determining the second loss value between the rewritten second sample dialogue sentence and the standard dialogue sentence corresponding to the second sample dialogue sentence, the model training module 1102 is specifically configured to:

[0195] A loss value obtained based on the predicted label and the true label respectively corresponding to at least one character included in the second sample dialogue sentence is used as the second loss value.

[0196] In an optional embodiment, the model training module 1102 determines that the model loss value satisfies a preset convergence condition if one of the following conditions is met:

[0197] The first loss value is less than or equal to a preset first loss value threshold, and the second loss value is less than or equal to a preset second loss value threshold; or,

[0198] A third loss value obtained based on the first loss value and the second loss value is less than or equal to a preset third loss value threshold.

[0199] In an optional embodiment, after obtaining the target multi-round dialogue model, the multi-round dialogue model training device 1100 further includes a dialogue rewriting module 1103, and the dialogue rewriting module 1103 is specifically used to:

[0200] The target dialogue sentence is input into the target multi-turn dialogue model to obtain the rewritten target dialogue sentence and the rewriting confidence of the target dialogue sentence; the rewriting confidence represents: the accuracy of the target multi-turn dialogue model in determining that the target dialogue sentence is the sentence to be rewritten.

[0201] Based on the description of the above method embodiment and device embodiment, the exemplary embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform the method according to the embodiment of the present invention when executed by the at least one processor.

[0202] An embodiment of the present application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method according to an embodiment of the present application.

[0203] An embodiment of the present application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method according to an embodiment of the present application.

[0204] See also Fig.12As shown, the structured block diagram of the electronic device 1200 that can be used as the server or client of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent the computer device of various forms of digital electronics, such as, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present application described herein and / or required.

[0205] like Fig.12 As shown, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 to a random access memory (RAM) 1203. In RAM 1203, various programs and data required for the operation of the device 1200 can also be stored. The computing unit 1201, ROM 1202 and RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0206] A plurality of components in the electronic device 1200 are connected to the I / O interface 1205, including: an input unit 1206, an output unit 1207, a storage unit 1208, and a communication unit 1209. The input unit 1206 may be any type of device capable of inputting information to the electronic device 1200, and the input unit 1206 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 1207 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1208 may include but is not limited to a disk, an optical disk. The communication unit 1209 allows the electronic device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a worldwide interoperability for microwave access (WiMax) device, a cellular communication device, and / or the like.

[0207] The computing unit 1201 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1201 performs the various methods and processes described above. For example, in some embodiments, the training method of the multi-round dialogue model described above may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1200 via the ROM 1202 and / or the communication unit 1209. In some embodiments, the computing unit 1201 may be configured to perform the training method of the multi-round dialogue model described above in any other appropriate manner (e.g., by means of firmware).

[0208] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0209] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0210] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0211] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tub (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0212] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0213] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0214] Furthermore, it should be understood that what is disclosed above is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope covered by the present application.

Claims

1. A training method for a multi-round dialogue model, characterized in that: include: Construct a sample dialogue training set; The sample dialogue training set includes: at least one sample dialogue sentence with a first label, the first label of each sample dialogue sentence being used to indicate whether the corresponding sample dialogue sentence needs to be rewritten; The initial multi-round dialogue model is iteratively trained multiple times based on the sample dialogue training set until a model loss value of the initial multi-round dialogue model meets a preset convergence condition, thereby obtaining a target multi-round dialogue model; the target multi-round dialogue model is used to determine whether a current dialogue sentence needs to be rewritten, and when it is determined that the current dialogue sentence needs to be rewritten, the current dialogue sentence is rewritten based on at least one historical dialogue sentence associated with the current dialogue sentence.

2. The method according to claim 1, characterized in that The step of constructing a sample dialogue training set includes: For the at least one sample dialogue sentence, the following operations are performed respectively: Acquire at least one historical dialogue sentence associated with the first sample dialogue sentence; wherein the first sample dialogue sentence is any one of the at least one sample dialogue sentence; Based on the standard dialogue sentence corresponding to the at least one historical dialogue sentence, a first discrimination result of the first sample dialogue sentence is determined; wherein each standard dialogue sentence is a dialogue sentence that does not omit an entity and / or does not refer to an entity, and the first discrimination result indicates whether the first sample dialogue sentence is a dialogue sentence to be rewritten; The first sample dialogue sentence is labeled based on the first discrimination result, and the first sample dialogue sentence labeled with the first label is saved in a preset sample dialogue training set.

3. The method according to claim 1, characterized in that The iterative training of the initial multi-round dialogue model based on the sample dialogue training set includes: During each training iteration, the following operations are performed: Determining, by means of the initial multi-round dialogue model, a second discrimination result of a second sample dialogue sentence selected from the sample dialogue training set, and rewriting the second sample dialogue sentence based on at least one historical dialogue sentence associated with the second sample dialogue sentence when the second discrimination result indicates that the second sample dialogue sentence is a dialogue sentence to be rewritten; Determining a first loss value between a predicted label corresponding to the second discrimination result and a first label of the second sample dialogue sentence, and determining a second loss value between the rewritten second sample dialogue sentence and a standard dialogue sentence corresponding to the second sample dialogue sentence; Parameters of the initial multi-round dialogue model are adjusted based on the first loss value and the second loss value.

4. The method according to claim 3, characterized in that The rewriting of the second sample dialogue sentence based on at least one historical dialogue sentence associated with the second sample dialogue sentence includes: Using a preset character labeling method, a second label is added to a plurality of characters included in the at least one historical dialogue sentence and the second sample dialogue sentence; wherein the second label corresponding to each character is used to indicate whether the corresponding character is used to determine a sentence rewriting position and a sentence rewriting content of the second sample dialogue sentence; The second sample dialogue sentence is rewritten based on the second labels respectively corresponding to the multiple characters.

5. The method according to claim 3, characterized in that The determining a second loss value between the rewritten second sample dialogue sentence and a standard dialogue sentence corresponding to the second sample dialogue sentence includes: A loss value obtained based on the predicted label and the true label respectively corresponding to at least one character included in the second sample dialogue sentence is used as the second loss value.

6. The method according to any one of claims 3 to 5, characterized in that If one of the following conditions is met, it is determined that the model loss value meets the preset convergence condition: The first loss value is less than or equal to a preset first loss value threshold, and the second loss value is less than or equal to a preset second loss value threshold; or, A third loss value obtained based on the first loss value and the second loss value is less than or equal to a preset third loss value threshold.

7. The method according to claim 1 or 2, characterized in that: After obtaining the target multi-round dialogue model, the method further includes: The target dialogue sentence is input into the target multi-turn dialogue model to obtain a rewritten target dialogue sentence and a rewriting confidence of the target dialogue sentence; the rewriting confidence represents: the accuracy of the target multi-turn dialogue model in determining that the target dialogue sentence is a sentence to be rewritten.

8. A training device for a multi-round dialogue model, characterized in that: include: Sample construction module, used to construct sample dialogue training set; The sample dialogue training set includes: at least one sample dialogue sentence with a first label, the first label of each sample dialogue sentence being used to indicate whether the corresponding sample dialogue sentence needs to be rewritten; A model training module is used to perform multiple iterative training on the initial multi-round dialogue model based on the sample dialogue training set until the model loss value of the initial multi-round dialogue model meets a preset convergence condition, thereby obtaining a target multi-round dialogue model; the target multi-round dialogue model is used to determine whether the current dialogue sentence needs to be rewritten, and when it is determined that the current dialogue sentence needs to be rewritten, rewrite the current dialogue sentence based on at least one historical dialogue sentence associated with the current dialogue sentence.

9. An electronic device, comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 8.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 8.

Images