Model generation method, dialogue generation method, device and electronic equipment

Abstract

Embodiments of the present application disclose a model generation method, a dialogue generation method, a device and an electronic device. The method comprises: obtaining training data, the training data comprising a plurality of dialogues, each dialogue comprising a question, a standard answer corresponding to the question, a plurality of background knowledge corresponding to the question, and a background label corresponding to the plurality of background knowledge, the background label representing the similarity between the plurality of background knowledge and the standard answer; and training an initial dialogue generation model based on the training data to obtain a dialogue generation model. In this way, the initial dialogue generation model can be trained based on the question, the standard answer, the plurality of background knowledge related to the question, and the background label representing the similarity between each piece of background knowledge and the standard answer, and the dialogue generation model can be obtained, so that the dialogue generation model can learn to accurately determine the similarity between the background knowledge and the answer based on the training process, thereby improving the accuracy of the dialogue generation model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and more particularly, to a model generation method, a dialogue generation method, an apparatus, and an electronic device. BACKGROUND

[0002] With the continuous development of artificial intelligence technology, open-domain dialogue systems have become one of the most important and most challenging tasks in the field of natural language processing. Unlike task-oriented dialogue, which has relatively definite and unique answers, open-domain dialogue does not limit the content of the answers, but places higher requirements on the fluency and information density of the answers. In related methods, an encoder-decoder (Encoder-Decoder) network architecture can be used to learn the mapping relationship between questions and answers from a large amount of dialogue data, so that when a question is obtained, a corresponding answer can be automatically generated. However, the answers generated by the related methods still have the problem of low accuracy. SUMMARY

[0003] In view of the above problems, the present application provides a model generation method, a dialogue generation method, an apparatus, and an electronic device to improve the above problems.

[0004] In a first aspect, the present application provides a model generation method, which comprises: obtaining training data, the training data comprising a plurality of dialogues, each of the dialogues comprising a question, a standard answer corresponding to the question, a plurality of background knowledge corresponding to the question, and background labels corresponding to the plurality of background knowledge, the background labels representing the similarity between the plurality of background knowledge and the standard answer; training an initial dialogue generation model based on the training data to obtain a dialogue generation model.

[0005] In a second aspect, the present application provides a dialogue generation method, which comprises: obtaining a question to be answered; obtaining a plurality of background knowledge corresponding to the question based on the question; inputting the question and the plurality of background knowledge into the dialogue generation model obtained based on any one of claims 1-6 to obtain predicted background labels corresponding to the plurality of background knowledge and a predicted answer.

[0006] In a third aspect, the present application provides a model generation apparatus, which comprises: a training data obtaining unit configured to obtain training data, the training data comprising a plurality of dialogues, each of the dialogues comprising a question, a standard answer corresponding to the question, a plurality of background knowledge corresponding to the question, and background labels corresponding to the plurality of background knowledge, the background labels representing the similarity between the plurality of background knowledge and the standard answer; a model generation unit configured to train an initial dialogue generation model based on the training data to obtain a dialogue generation model.

[0007] In a fourth aspect, the present application provides a dialogue generation apparatus, the apparatus comprising: a question obtaining unit configured to obtain a question to be answered; a background knowledge obtaining unit configured to obtain, based on the question, a plurality of pieces of background knowledge corresponding to the question; and an answer generation unit configured to input the question and the plurality of pieces of background knowledge into a dialogue generation model obtained based on the method described above, to obtain a predicted background label and a predicted answer corresponding to the plurality of pieces of background knowledge.

[0008] In a fifth aspect, the present application provides an electronic device comprising one or more processors and memory; one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs configured to perform the method described above.

[0009] In a sixth aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium storing program code, wherein the program code, when executed, performs the method described above.

[0010] The model generation method, dialogue generation method, apparatus, electronic device, and storage medium provided by the present application can be used to train an initial dialogue generation model based on a question, a standard answer corresponding to the question, a plurality of pieces of background knowledge related to the question, and a background label representing the similarity between each piece of background knowledge and the standard answer, to obtain a dialogue generation model. In this way, the dialogue generation model can be trained based on the question, the standard answer, the plurality of pieces of background knowledge related to the question, and the background label representing the similarity between each piece of background knowledge and the standard answer, so that the dialogue generation model can learn to accurately determine the similarity between background knowledge and an answer during the training process. Since a higher similarity indicates that an accurate answer is more likely to be obtained based on the background knowledge, the dialogue generation model can generate an answer based on the background knowledge with a high similarity, thereby improving the accuracy of the dialogue generation model. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0012] Figure 1 A flowchart of a model generation method according to an embodiment of the present application is shown;

[0013] Figure 2 A flowchart of a dialogue generation method according to an embodiment of the present application is shown; Figure 1A flow chart of an embodiment proposed in S120;

[0014] Figure 3 A flow chart of a model generation method proposed in another embodiment of the application is shown;

[0015] Figure 4 A schematic diagram of generating a first feature vector proposed in the application is shown;

[0016] Figure 5 A schematic diagram of generating a first feature vector proposed in the application is shown; Figure 3 A flow chart of an embodiment proposed in S260;

[0017] Figure 6 A schematic diagram of a calculation process of obtaining a local attention encoding vector proposed in an embodiment of the application is shown;

[0018] Figure 7 A flow chart of a model generation method proposed in another embodiment of the application is shown;

[0019] Figure 8 A schematic diagram of generating a second feature vector proposed in the application is shown;

[0020] Figure 9 A schematic diagram of generating a second feature vector proposed in the application is shown; Figure 7 A flow chart of an embodiment proposed in S370;

[0021] Figure 10 A schematic diagram of generating a predicted background label and a predicted answer proposed in the application is shown;

[0022] Figure 11 A flow chart of a dialogue generation method proposed in an embodiment of the application is shown;

[0023] Figure 12 A structural block diagram of a model generation apparatus proposed in an embodiment of the application is shown;

[0024] Figure 13 A structural block diagram of a dialogue generation apparatus proposed in an embodiment of the application is shown;

[0025] Figure 14 A structural block diagram of an electronic device proposed in the application is shown;

[0026] Figure 15 The storage unit for saving or carrying program codes for implementing the model generation method and the dialogue generation method according to the embodiments of the application is used in the embodiments of the application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of the present application.

[0028] With the continuous development of artificial intelligence technology, open-domain dialogue systems have begun to attract close attention from the industry and academia. In related manners, the mapping relationship between questions and answers can be learned from a large amount of dialogue data based on an encoder-decoder (Encoder-Decoder) network architecture, so that when a question is obtained, a corresponding answer to the question can be automatically generated. However, the answer generated by the related manner still has the problem of low accuracy.

[0029] However, the inventors found in related research that the answer generated by the related manner still has the problem of low accuracy. For example, in a dialogue data set, the question is often short, and when generating a viewpoint with rich information, the input text often cannot provide enough information to generate an ideal answer.

[0030] Therefore, the inventors propose a model generation method, a dialogue generation method, an apparatus, and an electronic device in the present application. After obtaining training data of a plurality of dialogues including questions, standard answers corresponding to the questions, a plurality of background knowledge corresponding to the questions, and background labels representing the similarity between the plurality of background knowledge and the standard answers, the initial dialogue generation model is trained based on the training data to obtain a dialogue generation model. Through the above manner, the initial dialogue generation model can be trained based on the question, the standard answer, the plurality of background knowledge related to the question, and the background label representing the similarity between each piece of background knowledge and the standard answer to obtain the dialogue generation model. The dialogue generation model can learn to accurately determine the similarity between the background knowledge and the answer based on the training process. Since the higher the similarity, the more likely it is to obtain an accurate answer based on the background knowledge, the dialogue generation model can generate an answer based on the background knowledge with high similarity, thereby improving the accuracy of the dialogue generation model.

[0031] In order to better understand the scheme of the embodiments of the present application, the technical terms used in the embodiments of the present application will be explained first.

[0032] Encoder-Decoder: can refer to a network structure for long sequence modeling, commonly used in translation, text summarization, question and answer, and other text generation fields. The network structure can use an encoder (Encoder) to encode the input sequence into a vector representation, and then input the vector representation into a decoder (Decoder) to obtain the output text.

[0033] Transformer: can refer to a network structure for encoding sequence type data (such as text, speech, etc.), and multiple Transformer modules are currently stacked to build a network architecture for various natural language processing tasks.

[0034] The embodiments in the present application will be described below with reference to the accompanying drawings.

[0035] Please refer to Figure 1 The model generation method provided in the present application comprises:

[0036] S110: obtaining training data, the training data comprising multiple dialogues, each of the dialogues comprising a question, a standard answer corresponding to the question, multiple pieces of background knowledge corresponding to the question, and background labels corresponding to the multiple pieces of background knowledge, the background labels representing the similarity between the multiple pieces of background knowledge and the standard answer.

[0037] The standard answer can refer to an artificially set answer conforming to language logic. The background knowledge can refer to knowledge related to the question. The related knowledge can refer to that the theme of the background knowledge can be the same as the theme corresponding to the question, and part of the content of the background knowledge can be similar to part of the content of the question. For example, the question can be "what do you think of war?", the theme corresponding to the question can be war, and the background knowledge can be content related to war. For another example, the question can be "what did you eat tonight?", and the background knowledge can be "today the weather is really good". The background label can be 0 or 1, 0 can represent that the background knowledge is irrelevant to the standard answer, and 1 can represent that the background knowledge is similar to the standard answer. That is, the background knowledge can be divided into positive class background knowledge (background label is 1) and negative class background knowledge (background label is 0) through the background label.

[0038] As a way, hot topics in daily life and work and comments corresponding to the hot topics can be collected from the network through a crawler technology, and then questions can be constructed based on the hot topics, background knowledge and standard answers can be obtained based on the comments, and background labels can be obtained based on the similarity between the background knowledge and the standard answers.

[0039] Optionally, the similarity between the background knowledge and the standard answer can be obtained based on a pre-trained semantic similarity model, the background label corresponding to the background knowledge with a similarity greater than a preset value can be set to 1, and the background label corresponding to the background knowledge with a similarity less than or equal to the preset value can be set to 0.

[0040] Optionally, the training data can be multi-turn dialogues and single-turn dialogues, and when the training data is multi-turn dialogues, the question of the training data can be a splicing result of questions of the multi-turn dialogues.

[0041] S120: training the initial dialogue generation model based on the training data to obtain the dialogue generation model.

[0042] The initial dialogue generation model can refer to a model based on a transformer structure for generating answers. The initial dialogue generation model can include an encoding network and a decoding network. The encoding network of the initial dialogue generation model can be used to encode the question and the plurality of pieces of background knowledge. The decoding network of the initial dialogue generation model can be used to generate the predicted background label and the predicted answer based on the output result of the encoding network, the standard answer, and the background label. That is, the decoding network of the initial dialogue generation model can learn how to accurately determine the background label of the input background knowledge and how to generate an accurate answer based on the output result of the encoding network, the standard answer, and the background label.

[0043] As a manner, as shown in Figure 2 training the initial dialogue generation model based on the training data to obtain the dialogue generation model includes:

[0044] S121: inputting the question and the plurality of pieces of background knowledge of each dialogue into the encoding network of the initial dialogue generation model corresponding to the current training process to obtain an encoding vector of each dialogue corresponding to the current training process.

[0045] The encoding vector can refer to a vector obtained by feature extraction of the question and the plurality of pieces of background knowledge of each dialogue by the encoding network of the initial dialogue generation model corresponding to the current training process.

[0046] As a manner, the question and the plurality of pieces of background knowledge of each dialogue can be represented by corresponding word vectors respectively. The word vectors representing the question and the plurality of pieces of background knowledge of each dialogue are arranged in the order of question, background knowledge 1, background knowledge 2, …, background knowledge K (K is a positive integer) and input into the encoding network of the initial dialogue generation model corresponding to the current training process to obtain the encoding vector of each dialogue corresponding to the current training process.

[0047] S122: inputting the standard answer, the background label, and the encoding vector of each dialogue into the decoding network of the initial dialogue generation model corresponding to the current training process to obtain the predicted answer and the predicted background label of each dialogue corresponding to the current training process.

[0048] As a manner, the standard answer and the background label of each dialogue can be respectively represented by a corresponding word vector, and the word vectors representing the standard answer and the background label of each dialogue are arranged in the order of the background label and the standard answer and input into the decoding network of the initial dialogue generation model corresponding to the current training process together with the encoding vector to obtain the word vectors representing the predicted answer and the predicted background label of each dialogue corresponding to the current training process.

[0049] S123: training the encoding network and the decoding network of the initial dialogue generation model corresponding to the current training process based on the standard answer, the background label, the predicted answer and the predicted background label of each dialogue corresponding to the current training process to obtain the trained dialogue generation model corresponding to the current training process.

[0050] As a manner, the vector representation of the standard answer and the background label of each dialogue corresponding to the current training process and the vector representation of the predicted answer and the predicted background label can be input into a loss function, and the encoding network and the decoding network of the initial dialogue generation model corresponding to the current training process are trained based on the value of the loss function to obtain the trained dialogue generation model corresponding to the current training process. The loss function can be used to make the difference between the predicted answer and the standard answer and the difference between the predicted background label and the background label as small as possible, and the loss function can be a cross-entropy loss function.

[0051] S124: if the trained dialogue generation model meets the target condition, the trained dialogue generation model is used as the dialogue generation model; if the target condition is not met, the next training process is entered, and the trained dialogue generation model is used as the initial dialogue generation model corresponding to the next training process.

[0052] The target condition can be various, for example, the target condition can be that the trained dialogue generation model reaches a convergence state, that is, the loss function reaches a local minimum. For another example, the target condition can be that the accuracy of the trained dialogue generation model reaches a preset accuracy.

[0053] As a manner, if the trained dialogue generation model meets the target condition, the trained dialogue generation model can be used as the dialogue generation model; if the target condition is not met, the next training process can be entered, and the trained dialogue generation model can be used as the initial dialogue generation model corresponding to the next training process.

[0054] Optionally, in the embodiments of the present application, the plurality of background knowledge can include positive and negative background knowledge, and the proportion of positive and negative background knowledge can be random within certain constraints, that is, the number of positive and negative background knowledge is an integer and their sum is equal to the preset number of background knowledge (such as K). By inputting the plurality of background knowledge into the encoding network, the model can learn the ability to organize and reconstruct the plurality of background knowledge and map it to the standard answer, that is, it can rewrite the plurality of background knowledge at the same time, thereby helping the dialogue generation model to generate new answers other than a single piece of background knowledge.

[0055] The model generation method provided in the embodiments can be used to train an initial dialogue generation model based on the question, the standard answer, the plurality of background knowledge related to the question, and the background label representing the similarity between each piece of background knowledge and the standard answer, so that the dialogue generation model can learn to accurately determine the similarity between the background knowledge and the answer during the training process. Since the higher the similarity indicates that it is more likely to get an accurate answer based on the background knowledge, the dialogue generation model can generate answers based on the background knowledge with high similarity, thereby improving the accuracy of the dialogue generation model.

[0056] Please refer to Figure 3 The model generation method provided in the embodiments includes the following steps:

[0057] S210: Obtain training data, the training data including a plurality of dialogues, each dialogue including a question, a standard answer corresponding to the question, a plurality of background knowledge corresponding to the question, and a background label corresponding to the plurality of background knowledge, the background label representing the similarity between the plurality of background knowledge and the standard answer.

[0058] In the embodiments of the present application, the question of each dialogue can correspond to a questioner identity and a dialogue turn, and the encoding network can include a first vector embedding module and an encoder. The first vector embedding module can be used to convert the word vector representing the question and the background knowledge into a dense vector, and the dense vector can have more rich semantic information for feature extraction. The first vector embedding module can include a question embedding module (token embedding), a first identity embedding module (role embedding), and a first turn embedding module (turn embedding).

[0059] S220: input the question of each dialogue and the plurality of pieces of background knowledge into the question embedding module to obtain a question background vector of each dialogue in a current training process.

[0060] As a manner, the word vectors representing the question and the plurality of pieces of background knowledge can be input into the question embedding module to obtain the question background vector of each dialogue. Illustratively, the question can be "what did you eat today", and the background knowledge can be "that hot pot is very good", "yesterday ate roast meat", etc., and the input of the question embedding module can be the word vectors representing "today, eat, what, that, hot pot, very, good, yesterday, eat, roast meat". Wherein, the comma can be a separator of a sentence, indicating the end of a sentence.

[0061] S230: input the questioner identity corresponding to the question of each dialogue into the first identity embedding module to obtain an identity vector corresponding to the question of each dialogue.

[0062] As a manner, the word vectors representing the questioner identity can be input into the first identity embedding module to obtain the identity vector corresponding to the question of each dialogue. Illustratively, the question can be "what did you eat today", and the identity of the questioner can be Xiaoming, and the word vector representing Xiaoming can be 1, and the input of the first identity embedding module can be "1, 1,..., 1".

[0063] Optionally, in order to accelerate the convergence speed of the model, the input vector dimension of the first identity embedding module can be the same as the input vector dimension of the question embedding module.

[0064] S240: input the dialogue round corresponding to the question of each dialogue into the first round embedding module to obtain a round vector corresponding to the question of each dialogue.

[0065] As a manner, the word vectors representing the dialogue round can be input into the first round embedding module to obtain the round vector corresponding to the question of each dialogue. Illustratively, the question can be "what did you eat today", and the round of the dialogue can be the first round, and the word vector representing the question of the first round dialogue can be 2, and the input of the first round embedding module can be "2, 2,..., 2".

[0066] Optionally, in order to accelerate the convergence speed of the model, the input vector dimension of the first round embedding module can be the same as the input vector dimension of the question embedding module and the first identity embedding module.

[0067] S250: feature fusion is performed on the question background vector, the identity vector and the round vector to obtain a first feature vector of each dialogue.

[0068] The first feature vector can refer to a dense vector representing the question, background knowledge, the identity of the questioner, and the dialogue turn after being encoded by an embedding technique.

[0069] As an example, as shown in Figure 4 The question background vector, the identity vector, and the turn vector at the same position can be added to obtain the first feature vector of each dialogue. For example, the question background vector can be {1, 2, 3, 4}, the identity vector can be {1, 1, 1, 1}, and the turn vector can be {2, 2, 2, 2}. Thus, the first feature vector can be {4, 5, 6, 7}.

[0070] S260: inputting the first feature vector into the encoder to obtain the encoded vector.

[0071] The encoder can include an attention network and a feedforward neural network, and the first feature vector includes a fusion question vector and a plurality of fusion background vectors.

[0072] As an example, as shown in Figure 5 inputting the first feature vector into the encoder to obtain the encoded vector, includes:

[0073] S261: inputting the first feature vector into the attention network to obtain a local attention encoded vector, which represents the fusion result of the question and the plurality of background knowledge, and the fusion result of the question and each of the background knowledge.

[0074] As an example, the first feature vector can be input into the attention network, and attention calculation can be performed on the fusion question vector and the plurality of fusion background vectors to obtain a global attention vector. Then, the global attention vector can be converted into a local attention encoded vector based on a mask matrix.

[0075] The attention calculation can refer to the multiplication of two vectors, and the multiplication of two vectors can obtain a corresponding matrix. The mask matrix can refer to a matrix used to filter out vectors in the global attention vector that interfere with the subsequent decoding process.

[0076] Optionally, the part of the mask matrix corresponding to the global attention vector that needs to be retained can be set as a first mask value, and the part of the mask matrix corresponding to the global attention vector that does not need to be retained can be set as a second mask value, wherein the first mask value is greater than the second mask value.

[0077] Optionally, in order not to change the global attention vector that needs to be retained, the first mask value can be 0, and in order to filter out the global attention vector that does not need to be retained as much as possible, the second mask value can be set to a negative number, such as -10000.

[0078] For example, the fusion question vector and each fusion background vector can be respectively calculated with all vectors in the first feature vector to obtain Figure 6 The calculation result in the above figure, that is, the global attention vector, is added to Figure 6 The calculation result in the above figure is added to Figure 6 The mask matrix in the middle figure is added to obtain Figure 6 The local attention matrix shown in the lower figure, and the result of each column of the local attention matrix is added to obtain the local attention encoding vector, so that the dimension of the local attention encoding vector is the same as that of the first feature vector.

[0079] In the embodiment of the present application, since the attention calculation between each two vectors can realize the feature fusion between the two vectors, and the result of one column of the local attention matrix can obtain part of the content of the local attention encoding vector, which can represent the feature fusion between each vector and other vectors, and since the input background knowledge can include background knowledge similar to the standard answer and background knowledge irrelevant to the standard answer, if the features of all background knowledge are fused together, it may not be conducive to the subsequent decoding network to generate accurate prediction background labels and prediction answers, and the feature fusion of the question and the background knowledge is conducive to mining more connections between the question and the background knowledge, therefore, the local attention calculation on the first feature vector by the mask matrix can effectively reduce the mutual interference between multiple background knowledge, so as to generate a local attention encoding vector that is more conducive to obtaining accurate prediction background labels and prediction answers, thereby improving the accuracy of the dialogue generation model.

[0080] S262: inputting the local attention encoding vector into the feedforward neural network to obtain the encoding vector.

[0081] The feedforward neural network can include a fully connected network and an activation function.

[0082] As a way, the local attention encoding vector can be input into the feedforward neural network to obtain the encoding vector.

[0083] Optionally, the encoder in the present application can be a structure of multiple encoders stacked.

[0084] S270: inputting the standard answer, the background label and the encoding vector of each dialogue into the decoding network of the initial dialogue generation model corresponding to the current training process to obtain the prediction answer and the prediction background label of each dialogue corresponding to the current training process.

[0085] S280: training the encoding network and the decoding network based on the standard answer, the background label, the predicted answer, and the predicted background label of each dialogue corresponding to the current training process, to obtain a trained dialogue generation model corresponding to the current training process.

[0086] S290: if the trained dialogue generation model meets a target condition, taking the trained dialogue generation model as a dialogue generation model; if the target condition is not met, entering a next training process and taking the trained dialogue generation model as an initial dialogue generation model corresponding to the next training process.

[0087] The model generation method provided in this embodiment can train an initial dialogue generation model based on a question, a standard answer, multiple pieces of background knowledge related to the question, and a background label representing the similarity between each piece of background knowledge and the standard answer, to obtain a dialogue generation model, so that the dialogue generation model can learn to accurately determine the similarity between background knowledge and an answer based on the training process. Since a higher similarity indicates that an accurate answer is more likely to be obtained based on the background knowledge, the dialogue generation model can generate an answer based on the background knowledge with a high similarity, thereby improving the accuracy of the dialogue generation model. In this embodiment, the first feature vector is obtained by fusing the question background vector, the identity vector, and the turn vector, which can improve the richness of the semantic information contained in the first feature vector, so that the encoding vector obtained by inputting the first feature vector into the encoding network can contain information useful for the subsequent decoding process, thereby improving the accuracy of the dialogue generation model.

[0088] Please refer to Figure 7 The model generation method provided in this embodiment includes:

[0089] S310: obtaining training data, the training data including multiple dialogues, each dialogue including a question, a standard answer corresponding to the question, multiple pieces of background knowledge corresponding to the question, and a background label corresponding to the multiple pieces of background knowledge, the background label representing the similarity between the multiple pieces of background knowledge and the standard answer.

[0090] Each dialogue's standard answer can correspond to an answerer identity and a dialogue turn, and the decoding network can include a second vector embedding module and a decoder, the second vector embedding module including an answer embedding module, a second identity embedding module, and a second turn embedding module.

[0091] S320: in a current training process, inputting the question and the multiple pieces of background knowledge of each dialogue into an encoding network of an initial dialogue generation model corresponding to the current training process, to obtain an encoding vector of each dialogue corresponding to the current training process.

[0092] S330: input the background label and the standard answer of each of the dialogues into the answer embedding module to obtain a label-answer vector of each of the dialogues.

[0093] As a manner, the word vector representing the background label and the standard answer of the dialogue can be input into the answer embedding module to obtain the label-answer vector of each dialogue. Exemplarily, the question can be "what did you eat today", the background knowledge can be "that hot pot is very good" (positive class) and "today the weather is really good" (negative class), and the standard answer can be "today I ate hot pot", the input of the answer embedding module can be the background label "1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0" and the word vector "today, day, eat, hot, pot".

[0094] S340: input the answerer identity corresponding to the question of each of the dialogues into the second identity embedding module to obtain an identity vector corresponding to the standard answer of each of the dialogues.

[0095] As a manner, the word vector representing the answerer identity can be input into the second identity embedding module to obtain the identity vector corresponding to the standard answer of each dialogue. Exemplarily, the standard answer can be "today I ate hot pot", the identity of the answerer can be small gang, the word vector representing small gang can be 2, and the input of the second identity embedding module can be "2, 2,..., 2".

[0096] Optionally, in order to accelerate the convergence speed of the model, the input vector dimension of the second identity embedding module can be the same as the input vector dimension of the answer embedding module.

[0097] S350: input the dialogue round corresponding to the standard answer of each of the dialogues into the second round embedding module to obtain a round vector corresponding to the standard answer of each of the dialogues.

[0098] As a manner, the word vector representing the dialogue round can be input into the second round embedding module to obtain the round vector corresponding to the standard answer of each dialogue. Exemplarily, the standard answer can be "today I ate hot pot", the round of the dialogue can be the first round, the word vector representing the answer of the first round dialogue can be 1, and the input of the second round embedding module can be "1, 1,..., 1".

[0099] Optionally, in order to accelerate the convergence speed of the model, the input vector dimension of the second round embedding module can be the same as the input vector dimension of the answer embedding module and the second identity embedding module.

[0100] S360: fuse the label-answer vector, the identity vector and the round vector to obtain a second feature vector of each of the dialogues.

[0101] The second feature vector can refer to a dense vector representing the background label, the standard answer, the questioner identity, and the dialogue turn, which are encoded by an embedding technique.

[0102] As a manner, as shown in Figure 8 The same position vectors in the label answer vector, the identity vector, and the turn vector can be added to obtain the second feature vector of each dialogue. For example, the label answer vector can be {1, 1, 1, 0, 0, 0, 2, 3, 4}, the identity vector can be {1, 1, 1, 1, 1, 1, 1, 1, 1}, and the turn vector can be {1, 1, 1, 1, 1, 1, 1, 1, 1}. Then the second feature vector can be {1, 1, 1, 2, 2, 2, 4, 5, 6}.

[0103] S370: inputting the second feature vector and the encoding vector into the decoder to obtain the predicted answer and the predicted background label.

[0104] As a manner, as shown in Figure 9 The second feature vector and the encoding vector are inputted into the decoder to obtain the predicted answer and the predicted background label.

[0105] S371: inputting the fusion label vector and the encoding vector into the decoder to obtain the predicted background label.

[0106] As a manner, as shown in Figure 10 The fusion label vector and the encoding vector can be inputted into the decoder by the first step to obtain the predicted background label.

[0107] S372: inputting the fusion label vector, the fusion answer vector, and the encoding vector into the decoder to obtain the predicted answer.

[0108] As a manner, as shown in Figure 10 The fusion label vector, the fusion answer vector, and the encoding vector can be inputted into the decoder by the subsequent step to obtain the predicted answer.

[0109] In the embodiment of the present application, since the encoding vector can represent multiple pieces of background knowledge, and the fused label vector can represent the similarity between each piece of background knowledge and the standard answer, in the training process, by inputting the encoding vector and the encoding vector into the decoder, the decoder can infer which background knowledge is conducive to generating a predicted answer close to the standard answer, so that the decoder can guide the generation of subsequent predicted answers based on the predicted label of S371, thereby improving the accuracy of the predicted answer. At the same time, the dialogue generation model of the present application can have the ability to screen background knowledge based on background labels and predicted background labels. Compared with other background knowledge screening methods, it is simpler and saves computing resources.

[0110] Optionally, the part of the encoding vector corresponding to the background knowledge irrelevant to the standard answer can be filtered out (e.g., set to 0) based on the predicted label of S371 to obtain an attention encoding vector, so that in the process of generating the predicted answer, the attention encoding vector and the fused answer vector can be input into the decoder to obtain the predicted answer.

[0111] Optionally, since the decoder also has an attention network, and the decoding process is carried out step by step, and the output result of each step can affect the subsequent output result, the part of the encoding vector corresponding to the background knowledge irrelevant to the standard answer can be filtered out based on the attention network of the decoder itself based on the predicted label of S371, so that in the process of generating the predicted answer, the predicted answer can be inferred as much as possible based on the background knowledge related to the standard answer, thereby improving the accuracy of the predicted answer.

[0112] Moreover, the screened background knowledge is similar to the standard answer, so that for a question with completely opposite views, the question can be input into the dialogue generation model of the present application at different time points, and the same view answer can be output.

[0113] Optionally, the encoder in the present application can be a structure in which multiple decoders are stacked.

[0114] S380: Training the encoding network and the decoding network based on the standard answer, the background label, the predicted answer, and the predicted background label of each dialogue corresponding to the current training process, to obtain a trained dialogue generation model corresponding to the current training process.

[0115] S390: If the trained dialogue generation model meets the target condition, the trained dialogue generation model is used as the dialogue generation model; if the target condition is not met, the next training process is entered, and the trained dialogue generation model is used as the initial dialogue generation model corresponding to the next training process.

[0116] The model generation method provided in this embodiment can train an initial dialogue generation model based on a question, a standard answer, a plurality of pieces of background knowledge related to the question, and background labels representing the similarity of each piece of background knowledge to the standard answer, to obtain a dialogue generation model, so that the dialogue generation model can learn to accurately determine the similarity of background knowledge to an answer based on the training process. Since a higher similarity indicates that an accurate answer is more likely to be obtained based on the background knowledge, the dialogue generation model can generate an answer based on background knowledge with a high similarity, thereby improving the accuracy of the dialogue generation model. In this embodiment, the second feature vector is obtained by fusing the label answer vector, the identity vector, and the round vector, which can improve the richness of the semantic information contained in the second feature vector, so that the decoding network can decode the predicted answer and the predicted background label that are closer to the standard answer and the background label based on the input second feature vector and the encoding vector, thereby improving the accuracy of the dialogue generation model.

[0117] Referring to Figure 11 The dialogue generation method provided in this application comprises:

[0118] S410: Obtain a question to be answered.

[0119] As one way, the question input by the user in the form of text, voice, etc. through the user terminal (such as a computer, a mobile phone, a tablet, etc.) can be obtained.

[0120] S420: Based on the question, obtain a plurality of pieces of background knowledge corresponding to the question.

[0121] As one way, the BM25 algorithm can be used to retrieve a plurality of similar theme articles related to the question in the database, the comments corresponding to these articles are aggregated to obtain a comment set, and the comment set is sorted based on signals such as the number of likes and the number of replies to obtain a plurality of high-quality comments as background knowledge.

[0122] S430: Input the question and the plurality of pieces of background knowledge into the dialogue generation model obtained based on the above method to obtain predicted background labels and predicted answers corresponding to the plurality of pieces of background knowledge.

[0123] As one way, the question and the plurality of pieces of background knowledge can be input into the dialogue generation model of this application to obtain corresponding predicted background labels and predicted answers, and the predicted answers can be output to the user in the form of voice, text, etc. as correct answers.

[0124] The dialogue generation method provided in the embodiment can obtain multiple pieces of background knowledge corresponding to a question from a database based on the question to be answered, so that the question and the multiple pieces of background knowledge corresponding to the question can be input into a dialogue generation model, so that an accurate answer can be obtained, and user experience is improved.

[0125] Please refer to Figure 12 The model generation device 600 provided in the present application comprises:

[0126] The training data acquisition unit 610 is configured to acquire training data, wherein the training data comprises multiple dialogues, each of the dialogues comprises a question, a standard answer corresponding to the question, multiple pieces of background knowledge corresponding to the question, and background labels corresponding to the multiple pieces of background knowledge, and the background labels represent the similarity between the multiple pieces of background knowledge and the standard answer.

[0127] The model generation unit 620 is configured to train an initial dialogue generation model based on the training data to obtain a dialogue generation model.

[0128] As a manner, the initial dialogue generation model comprises an encoding network and a decoding network, and the model generation unit 620 is specifically configured to input the question and the multiple pieces of background knowledge of each of the dialogues into the encoding network of the initial dialogue generation model corresponding to the current training process to obtain an encoding vector of each of the dialogues corresponding to the current training process; input the standard answer, the background labels and the encoding vector of each of the dialogues into the decoding network of the initial dialogue generation model corresponding to the current training process to obtain a predicted answer and a predicted background label of each of the dialogues corresponding to the current training process; train the encoding network and the decoding network of the initial dialogue generation model corresponding to the current training process based on the standard answer, the background labels, the predicted answer and the predicted background label of each of the dialogues corresponding to the current training process to obtain a trained dialogue generation model corresponding to the current training process; if the trained dialogue generation model meets a target condition, the trained dialogue generation model is used as the dialogue generation model; if the target condition is not met, the next training process is entered, and the trained dialogue generation model is used as the initial dialogue generation model corresponding to the next training process.

[0129] In the method, as one mode, each question of the dialogue corresponds to a questioner identity and a dialogue round, the encoding network comprises a first vector embedding module and an encoder, and the model generation unit 620 is specifically configured to input each question of the dialogue and the plurality of background knowledge into the question embedding module to obtain a question background vector of each question of the dialogue during the current training process; input the questioner identity corresponding to each question of the dialogue into the first identity embedding module to obtain an identity vector corresponding to each question of the dialogue; input the dialogue round corresponding to each question of the dialogue into the first round embedding module to obtain a round vector corresponding to each question of the dialogue; and perform feature fusion on the question background vector, the identity vector and the round vector to obtain a first feature vector of each dialogue; and input the first feature vector into the encoder to obtain the encoding vector.

[0130] Optionally, the encoder comprises an attention network and a feedforward neural network, and the first feature vector comprises a fusion question vector and a plurality of fusion background vectors, and the model generation unit 620 is specifically configured to input the first feature vector into the attention network to obtain a local attention encoding vector, the local attention encoding vector representing fusion results of the question and the plurality of background knowledge and fusion results of the question and each background knowledge; and input the local attention encoding vector into the feedforward neural network to obtain the encoding vector.

[0131] As another mode, each standard answer of the dialogue corresponds to an answerer identity and a dialogue round, the decoding network comprises a second vector embedding module and a decoder, the second vector embedding module comprises an answer embedding module, a second identity embedding module and a second round embedding module, and the model generation unit 620 is specifically configured to input the background label and the standard answer of each dialogue into the answer embedding module to obtain a label answer vector of each dialogue; input the answerer identity corresponding to each question of the dialogue into the second identity embedding module to obtain an identity vector corresponding to the standard answer of each dialogue; input the dialogue round corresponding to the standard answer of each dialogue into the second round embedding module to obtain a round vector corresponding to the standard answer of each dialogue; perform feature fusion on the label answer vector, the identity vector and the round vector to obtain a second feature vector of each dialogue; and input the second feature vector and the encoding vector into the decoder to obtain the predicted answer and the predicted background label.

[0132] Optionally, the second feature vector comprises a fusion label vector and a fusion answer vector, and the model generation unit 620 is specifically configured to input the fusion label vector and the encoding vector into the decoder to obtain the predicted background label; and input the fusion label vector, the fusion answer vector and the encoding vector into the decoder to obtain the predicted answer.

[0133] Please refer to Figure 13 The application provides a dialogue generation device 800, and the device 800 comprises:

[0134] A question acquisition unit 810 is configured to acquire a question to be answered.

[0135] A background knowledge acquisition unit 820 is configured to obtain a plurality of pieces of background knowledge corresponding to the question based on the question.

[0136] An answer generation unit 830 is configured to input the question and the plurality of pieces of background knowledge into a dialogue generation model obtained based on the above method to obtain predicted background labels and predicted answers corresponding to the plurality of pieces of background knowledge.

[0137] The application will be described below in combination with Figure 14 An electronic device provided by the application is described.

[0138] Please refer to Figure 14 Based on the above model generation method, dialogue generation method and device, the application further provides another electronic device 100 that can execute the above model generation method and dialogue generation method. The electronic device 100 comprises one or more (only one is shown in the figure) processors 102 and a memory 104 coupled with each other. The memory 104 stores programs that can execute the content in the above embodiments, and the processor 102 can execute the programs stored in the memory 104.

[0139] The processor 102 can include one or more processing cores. The processor 102 connects various parts within the electronic device 100 by various interfaces and lines, performs various functions of the electronic device 100 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 104, and calling data stored in the memory 104. Alternatively, the processor 102 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 102 can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 102, but can be implemented by a separate communication chip.

[0140] The memory 104 can include a random access memory (RAM) and can also include a read-only memory (ROM). The memory 104 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 104 can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing various method embodiments described below, etc. The data storage area can also store data created by the electronic device 100 in use (such as a phone book, audio and video data, chat record data, etc.).

[0141] Please refer to Figure 15 which shows a structural block diagram of a computer readable storage medium provided by the embodiments of the present application. The computer readable storage medium 1000 stores program codes therein, and the program codes can be called and executed by a processor to perform the methods described in the above method embodiments.

[0142] The computer-readable storage medium 1000 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk or a ROM. Optionally, the computer-readable storage medium 1000 comprises a non-transitory computer-readable storage medium. The computer-readable storage medium 1000 has storage space for program codes 1010 for performing any of the method steps of the above-described methods. These program codes can be read from or written to one or more computer program products. The program codes 1010 can be compressed, for example, in a suitable form.

[0143] To sum up, the model generation method, the dialogue generation method, the device and the electronic equipment provided in the application, after obtaining the training data of a plurality of dialogues including questions, standard answers corresponding to the questions, a plurality of background knowledge corresponding to the questions, and background tags representing the similarity between the plurality of background knowledge and the standard answers, the initial dialogue generation model is trained based on the training data to obtain a dialogue generation model. Through the above-mentioned manner, the initial dialogue generation model can be trained based on the question, the standard answer, the plurality of background knowledge related to the question, and the background tag representing the similarity between each piece of background knowledge and the standard answer to obtain the dialogue generation model, so that the dialogue generation model can learn to accurately judge the similarity between the background knowledge and the answer based on the training process. Since the higher the similarity is, the more likely it is to obtain an accurate answer based on the background knowledge, the dialogue generation model can generate an answer based on the background knowledge with high similarity, thereby improving the accuracy of the dialogue generation model.

[0144] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art will understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not drive the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. A model generation method, characterized in that, The method includes: Acquire training data, which includes multiple dialogues. Each dialogue includes a question, a standard answer to the question, multiple pieces of background knowledge to the question, and background labels corresponding to the multiple pieces of background knowledge. The background labels represent the similarity between the multiple pieces of background knowledge and the standard answer. During the current training process, the question and multiple pieces of background knowledge for each dialogue are input into the encoding network of the initial dialogue generation model corresponding to the current training process. The question embedding module within the first vector embedding module of the encoding network is used to obtain the question background vector for each dialogue. The first vector embedding module further includes a first identity embedding module and a first round embedding module. The questioner's identity corresponding to the question in each dialogue is input into the first identity embedding module to obtain the identity vector corresponding to the question in each dialogue. The dialogue round corresponding to the question in each dialogue is input into the first round embedding module to obtain the round vector corresponding to the question in each dialogue. The question background vector, the identity vector, and the round vector are fused to obtain a first feature vector for each dialogue. The first feature vector is input into the encoder in the encoding network to obtain the encoding vector for each dialogue. The standard answer, background label, and encoding vector of each dialogue are input into the decoding network of the initial dialogue generation model corresponding to the current training process to obtain the predicted answer and predicted background label of each dialogue corresponding to the current training process. Based on the standard answer, background label, predicted answer, and predicted background label of each dialogue in the current training process, the encoding and decoding networks of the initial dialogue generation model in the current training process are trained to obtain the trained dialogue generation model in the current training process. If the trained dialogue generation model meets the target conditions, the trained dialogue generation model is used as the dialogue generation model; if the target conditions are not met, the next training process begins, and the trained dialogue generation model is used as the initial dialogue generation model for the next training process.

2. The method according to claim 1, characterized in that, The encoder includes an attention network and a feedforward neural network. The first feature vector includes a fused question vector and multiple fused background vectors. Inputting the first feature vector into the encoder to obtain the encoded vector for each dialogue includes: The first feature vector is input into the attention network to obtain a local attention encoding vector, which represents the fusion result of the question and the multiple background knowledge, and the fusion result of the question and each of the background knowledge. The local attention encoding vector is input into the feedforward neural network to obtain the encoding vector.

3. The method according to claim 1, characterized in that, Each standard answer to the dialogue corresponds to the respondent's identity and the dialogue turn. The decoding network includes a second vector embedding module and a decoder. The second vector embedding module includes an answer embedding module, a second identity embedding module, and a second turn embedding module. The step of inputting the standard answer, background label, and encoded vector of each dialogue into the decoding network of the initial dialogue generation model corresponding to the current training process to obtain the predicted answer and predicted background label for each dialogue corresponding to the current training process includes: The background tags and standard answers for each dialogue are input into the answer embedding module to obtain the tag answer vector for each dialogue; The identity of the respondent corresponding to the question in each dialogue is input into the second identity embedding module to obtain the identity vector corresponding to the standard answer in each dialogue; The dialogue round corresponding to the standard answer of each dialogue is input into the second round embedding module to obtain the round vector corresponding to the standard answer of each dialogue. The tag response vector, the identity vector, and the round vector are fused to obtain a second feature vector for each dialogue. The second feature vector and the encoded vector are input into the decoder to obtain the predicted answer and the predicted background label.

4. The method according to claim 3, characterized in that, The second feature vector includes a fused label vector and a fused response vector. The step of inputting the second feature vector and the encoded vector into the decoder to obtain the predicted response and the predicted background label includes: The fused label vector and the encoded vector are input into the decoder to obtain the predicted background label; The fused label vector, the fused response vector, and the encoded vector are input into the decoder to obtain the predicted response.

5. A dialogue generation method, characterized in that, The method includes: Get the questions to be answered; Based on the problem, several pieces of background knowledge corresponding to the problem are obtained; The question and the multiple pieces of background knowledge are input into the dialogue generation model obtained based on any one of the methods described in claims 1-4, so as to obtain the predicted background labels and predicted answers corresponding to the multiple pieces of background knowledge.

6. A model generation apparatus, characterized in that, The device includes: The training data acquisition unit is used to acquire training data, which includes multiple dialogues. Each dialogue includes a question, a standard answer to the question, multiple pieces of background knowledge to the question, and background labels corresponding to the multiple pieces of background knowledge. The background labels represent the similarity between the multiple pieces of background knowledge and the standard answer. The model generation unit is used to input the question of each dialogue and multiple pieces of background knowledge into the encoding network of the initial dialogue generation model corresponding to the current training process during the current training process, so as to obtain the question background vector of each dialogue through the question embedding module in the first vector embedding module included in the encoding network, wherein the first vector embedding module further includes a first identity embedding module and a first round embedding module; input the questioner identity corresponding to the question of each dialogue into the first identity embedding module to obtain the identity vector corresponding to the question of each dialogue; input the dialogue round corresponding to the question of each dialogue into the first round embedding module to obtain the round vector corresponding to the question of each dialogue; perform feature fusion on the question background vector, the identity vector, and the round vector to obtain a first feature vector of each dialogue; input the first feature vector into the encoder in the encoding network to obtain the encoding vector of each dialogue; The model generation unit is further configured to input the standard answer, background label, and encoding vector of each dialogue into the decoding network of the initial dialogue generation model corresponding to the current training process, so as to obtain the predicted answer and predicted background label of each dialogue corresponding to the current training process; train the encoding network and decoding network of the initial dialogue generation model corresponding to the current training process based on the standard answer, background label, predicted answer, and predicted background label of each dialogue corresponding to the current training process, so as to obtain the trained dialogue generation model corresponding to the current training process; if the trained dialogue generation model meets the target conditions, the trained dialogue generation model is used as the dialogue generation model; if the target conditions are not met, the next training process is entered, and the trained dialogue generation model is used as the initial dialogue generation model corresponding to the next training process.

7. A dialogue generation device, characterized in that, The device includes: The question retrieval unit is used to retrieve questions to be answered. The background knowledge acquisition unit is used to obtain multiple pieces of background knowledge corresponding to the problem based on the problem. The answer generation unit is used to input the question and the multiple pieces of background knowledge into a dialogue generation model obtained based on any one of the methods of claims 1-4, so as to obtain the predicted background labels and predicted answers corresponding to the multiple pieces of background knowledge.

8. An electronic device, characterized in that, Includes one or more processors and memory; One or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs being configured to perform the method of any one of claims 1-4.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code, wherein the method described in any one of claims 1-4 is executed when the program code is run.

Images