Dialogue generation method and device, electronic equipment and computer readable storage medium

Abstract

The present application relates to the field of machine learning, and discloses a dialogue generation method, comprising: training a pre-constructed text generation model by using historical data, calculating a first loss value between a predicted reply text sequence output by the text generation model during training and a historical real reply sequence by a first loss function, calculating a second loss value between a predicted conversion rate output by the text generation model during training and a real conversion rate by a second loss function, adjusting parameters of the pre-constructed text generation model according to the first loss value and the second loss value, and obtaining a pre-trained text generation model; inputting target dialogue text, identity information of a questioning user in the target dialogue text, and identity information of a replying user into the pre-trained text generation model, and obtaining a target reply text sequence of the replying user to the target dialogue text. The present application also proposes a dialogue generation device, an electronic device and a medium. The present application can accurately generate personalized dialogue content based on natural language processing technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine learning and digital medicine, in particular to a dialogue generation method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, significant progress has been made in the field of natural language processing. Text generation models have become an important auxiliary tool, which can generate text with certain logic and grammatical structure. However, existing text generation models usually ignore the personalized needs and language style of users, resulting in generated text that does not match the preferences and habits of users, making it take a long time for dialogue participants to understand and adapt, increasing the response time of the dialogue. For example, in a question and answer platform or other system, the user (patient) on the user side or the user (medical assistant) on the doctor side is unfamiliar with the generated text expression, and needs to spend additional time to adapt to the language style and understand the text content, thereby increasing the response time of the question and answer. Therefore, it is urgent to propose a personalized dialogue generation method to accurately generate personalized dialogue content that meets individual preferences and habits. SUMMARY

[0003] The present application provides a dialogue generation method, device, electronic equipment and computer readable storage medium, which is mainly aimed at accurately generating personalized dialogue content.

[0004] To achieve the above purpose, the dialogue generation method provided by the present application comprises:

[0005] Obtain historical data, the historical data contains a plurality of historical dialogues, identity information of a first dialogue user in the historical dialogues, identity information of a second dialogue user in the historical dialogues, a historical real reply sequence corresponding to each of the historical dialogues, and a real conversion rate of the historical real reply sequence;

[0006] Obtain a pre-constructed text generation model, train the pre-constructed text generation model using a plurality of historical dialogues, identity information of a first dialogue user in the historical dialogues, and identity information of a second dialogue user in the historical dialogues, obtain a predicted reply text sequence corresponding to each of the historical dialogues output by the pre-constructed text generation model during training, and a predicted conversion rate of the predicted reply text sequence, the predicted reply text sequence comprising predicted reply texts output by the pre-constructed text generation model one by one;

[0007] a first loss value between the predicted reply text sequence and the historical real reply sequence is calculated by a first loss function, a second loss value between the predicted conversion rate and the real conversion rate is calculated by a second loss function, parameters of the pre-constructed text generation model are constantly adjusted according to the first loss value and the second loss value, and a pre-trained text generation model is obtained;

[0008] obtaining target dialogue text, identity information of a questioning user in the target dialogue text and identity information of a replying user;

[0009] inputting the target dialogue text, the identity information of the questioning user in the target dialogue text and the identity information of the replying user into the pre-trained text generation model to obtain a target reply text sequence corresponding to the target dialogue text output by the pre-trained text generation model.

[0010] Optionally, the second loss function comprises:

[0011]

[0012] wherein N is the total number of the plurality of historical dialogues, is a real conversion rate of a historical real reply sequence corresponding to the historical dialogue i, is a predicted conversion rate of a predicted reply text sequence corresponding to the historical dialogue i.

[0013] Optionally, the pre-constructed text generation model is GPT-2.

[0014] Optionally, after the historical data is obtained, the method further comprises:

[0015] performing data enhancement on the historical data.

[0016] Optionally, after the historical data is obtained, the method further comprises:

[0017] performing data cleaning on the historical data.

[0018] To solve the above problems, the application further provides a dialogue generation device, which comprises:

[0019] a first obtaining module configured to obtain historical data, wherein the historical data comprises a plurality of historical dialogues, identity information of a first dialogue user in the historical dialogues, identity information of a second dialogue user in the historical dialogues, a historical real reply sequence corresponding to each of the historical dialogues and a real conversion rate of the historical real reply sequence;

[0020] The training module is configured to obtain a pre-constructed text generation model, train the pre-constructed text generation model by using a plurality of historical dialogues, identity information of a first dialogue user in the historical dialogues, and identity information of a second dialogue user in the historical dialogues, obtain a predicted reply text sequence corresponding to each of the historical dialogues and a predicted conversion rate of the predicted reply text sequence output by the pre-constructed text generation model during training, and the predicted reply text sequence comprises predicted reply texts output by the pre-constructed text generation model in sequence.

[0021] The parameter adjustment module is configured to calculate a first loss value between the predicted reply text sequence and the historical real reply sequence by using a first loss function, calculate a second loss value between the predicted conversion rate and the real conversion rate by using a second loss function, and continuously adjust parameters of the pre-constructed text generation model according to the first loss value and the second loss value to obtain a pre-trained text generation model.

[0022] The second obtaining module is configured to obtain target dialogue text, identity information of a questioning user in the target dialogue text, and identity information of a reply user.

[0023] The dialogue prediction module is configured to input the target dialogue text, the identity information of the questioning user in the target dialogue text, and the identity information of the reply user into the pre-trained text generation model to obtain a target reply text sequence of the reply user for the target dialogue text output by the pre-trained text generation model.

[0024] Optionally, the second loss function comprises:

[0025]

[0026] wherein N is a total number of the plurality of historical dialogues, is a real conversion rate of the historical real reply sequence corresponding to the historical dialogue i, is a predicted conversion rate of the predicted reply text sequence corresponding to the historical dialogue i.

[0027] Optionally, the apparatus further comprises:

[0028] The data enhancement module is configured to perform data enhancement on the historical data after the historical data is obtained.

[0029] To solve the above problems, the present application further provides an electronic device, which comprises:

[0030] a memory configured to store at least one computer program; and

[0031] A processor executes a program stored in the memory to implement the dialogue generation method described above.

[0032] To solve the above problems, the application further provides a computer readable storage medium, wherein at least one computer program is stored in the computer readable storage medium, and the at least one computer program is executed by a processor in an electronic device to implement the dialogue generation method described above.

[0033] The embodiment of the application obtains historical data, the historical data including a plurality of historical dialogues, identity information of a first dialogue user in the historical dialogues, identity information of a second dialogue user in the historical dialogues, a historical real reply sequence corresponding to each of the historical dialogues, and a real conversion rate of the historical real reply sequence; obtains a pre-constructed text generation model, trains the pre-constructed text generation model by using the plurality of historical dialogues, the identity information of the first dialogue user in the historical dialogues, and the identity information of the second dialogue user in the historical dialogues, obtains a predicted reply text sequence corresponding to each of the historical dialogues output by the pre-constructed text generation model during training, and a predicted conversion rate of the predicted reply text sequence, the predicted reply text sequence including predicted reply texts output by the pre-constructed text generation model one by one; calculates a first loss value between the predicted reply text sequence and the historical real reply sequence by using a first loss function, calculates a second loss value between the predicted conversion rate and the real conversion rate by using a second loss function, and continuously adjusts parameters of the pre-constructed text generation model according to the first loss value and the second loss value to obtain a pre-trained text generation model; obtains a target dialogue text, identity information of a questioning user in the target dialogue text, and identity information of a reply user; inputs the target dialogue text, the identity information of the questioning user in the target dialogue text, and the identity information of the reply user into the pre-trained text generation model to obtain a target reply text sequence of the reply user for the target dialogue text output by the pre-trained text generation model. By concatenating the identity information of the first dialogue user, the identity information of the second dialogue user, and the historical dialogue in the historical dialogue to train the text generation model, the pre-trained text generation model can be personalized according to different users, thereby achieving the purpose of accurately generating personalized dialogue content. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 A flowchart of a dialogue generation method provided by an embodiment of the application is shown in the figure;

[0035] Figure 2 A detailed implementation flowchart of one step in the dialogue generation method provided by an embodiment of the application is shown in the figure;

[0036] Figure 3 A functional module diagram of the dialogue generation device provided by an embodiment of the present application is provided.

[0037] Figure 4 A structural schematic diagram of an electronic device for implementing the dialogue generation method provided by an embodiment of the present application is provided.

[0038] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0039] It should be understood that the specific embodiments described herein are merely intended to explain the present application and are not intended to limit the present application.

[0040] An embodiment of the present application provides a dialogue generation method. The execution subject of the dialogue generation method includes but is not limited to at least one of a server, a terminal and other electronic devices that can be configured to execute the method provided by the present application. In other words, the dialogue generation method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server can be a stand-alone server, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0041] Referring to Figure 1 FIG. 1 shows a flowchart of a dialogue generation method provided by an embodiment of the present application. In this embodiment, the dialogue generation method includes the following steps.

[0042] S1, obtaining historical data, the historical data including a plurality of historical dialogues, identity information of a first dialogue user in the historical dialogues, identity information of a second dialogue user in the historical dialogues, a historical real reply sequence corresponding to each of the historical dialogues, and a real conversion rate of the historical real reply sequence.

[0043] In this embodiment, the historical data is a collection of dialogues in the past period of time. For example, a collection of dialogues between patients and doctors in the past three years stored in a medical database.

[0044] In this embodiment, the number of historical conversations contained in the historical data is multiple, the first conversation user and the second conversation user are different users, and the first conversation user and the second conversation user can be the same or different between different historical conversations. For example, there is a historical conversation in the historical data, the first conversation user is user A, and the second conversation user is user B. There is another conversation in the historical data, the first conversation user is user C, and the second conversation user is user D. There is still another conversation in the historical data, the first conversation user is user A, and the second conversation user is user D.

[0045] In this embodiment, each historical conversation can contain only one question (such as the question "How is the weather today?" of the first conversation user), or can include at least one question and at least one reply (such as question 1 "How is the weather today?" of the first conversation user, reply 1 "Sunny" of the second conversation user, and question 2 "How about playing football outside?" of the first conversation user), and each historical conversation has a corresponding historical real reply sequence, and the identity information of the first conversation user, the identity information of the second conversation user, and the conversion rate of the historical real reply sequence corresponding to the historical conversation.

[0046] The identity information can be name, account, or other identification information that uniquely identifies the identity of the conversation participant. The real conversion rate is actually counted. During the conversation process or after the conversation ends, the questioner places an order to purchase goods, or subscribes to a service, or continues to click to proceed to the next step. For example, the conversion rate is 0 or 1, where 1 indicates a transaction, and 0 indicates no transaction, that is, if the conversion rate is 0, it indicates that the question user who raised the question did not place an order to purchase goods or subscribe to a service or further operation during the conversation process or after the conversation ended; that is, if the conversion rate is 1, it indicates that the question user who raised the question exists to place an order to purchase goods or subscribe to a service or further operation during the conversation process or after the conversation ends.

[0047] In this embodiment, the historical real reply sequence contains multiple reply texts (i.e., different single texts). For example, the historical real reply sequence contains "eat more vitamin D", where "eat", "eat", "vitamin", "D", "D", "D", and "D" are six different reply texts.

[0048] Specifically, the historical data can be collected in the form of , where the format of the data can be json format. , represents a party of a conversation and the content of the speech in a conversation (one) in the historical data, "text" stores the text content of the conversation itself, "sender" represents the conversation user sending the text message, "sender" is an int type, 0 represents the first conversation user (such as a patient), and 1 represents the second conversation user (such as a doctor); the text content belonging to the same conversation sequence (that is, one conversation can include one or more rounds of conversation, and the first conversation and the second conversation user take turns speaking in one or more rounds of conversation) is spliced into a long text, and the messages between different conversation users are separated by special characters [C] or [D], wherein [C] can represent the first conversation user (such as the inquiring party), and [D] can represent the second conversation user (such as the receiving party); the multi-round conversation messages belonging to a conversation sequence are spliced by a special character [next], and the beginning and end are added with start character and end character [CLS] and [SEP], and finally the continuous conversation (that is, multi-round conversation) of a conversation sequence is formed into the following conversation data:

[0049] [CLS][C]string1[D]string2[next]string3[C]string4[SEP]

[0050] Wherein, string1, string2, string3 and string4 are the contents of the text in the foregoing

[0051] Further, in an optional embodiment of the present application, after the historical data is obtained, the method further comprises:

[0052] data cleaning is performed on the historical data.

[0053] In this embodiment, data cleaning of the historical data includes duplicate removal processing, outlier detection processing and standardization processing of the historical data.

[0054] Duplicate removal processing of the historical data can reduce redundant data, outlier detection processing of the historical data can improve the accuracy and reliability of the historical data, and standardization processing of the historical data can improve the efficiency of data processing.

[0055] Therefore, in this embodiment, by performing data cleaning on the historical data, the efficiency and accuracy of model training can be improved.

[0056] Further, in an optional embodiment of the present application, after the historical data is obtained, the method further comprises:

[0057] data augmentation is performed on the historical data.

[0058] ​Specifically, the data augmentation on the historical data comprises:

[0059] obtaining synonyms of the plurality of source texts in the historical dialogue, replacing the plurality of source texts with the synonyms, and determining the replaced data as new data;

[0060] adding the new data to the historical data to obtain the data-augmented historical data.

[0061] In this embodiment, the data augmentation on the historical data can expand the data volume of the historical data, thereby improving the accuracy and robustness of the training.

[0062] S2, obtaining a pre-constructed text generation model, training the pre-constructed text generation model using a plurality of historical dialogues, identity information of a first dialogue user in the historical dialogues, and identity information of a second dialogue user in the historical dialogues, obtaining a predicted reply text sequence corresponding to each of the historical dialogues output by the pre-constructed text generation model during training and a predicted conversion rate of the predicted reply text sequence, the predicted reply text sequence comprising predicted reply texts output by the pre-constructed text generation model one by one.

[0063] In this embodiment, the pre-constructed text generation model can be a natural language processing model. For example, the pre-constructed text generation model is a BERT model, or the pre-constructed text generation model is a CNN, or an LSTM model.

[0064] During training of the pre-constructed text generation model, the identity information of the first dialogue user in the historical dialogue and the identity information of the second dialogue user in the historical dialogue are respectively vectorized to obtain a first dialogue user vector and a second dialogue user vector, and then the first dialogue user vector, the second dialogue user vector, and the historical dialogue (each character is mapped to a vector) are input to the pre-constructed text generation model, the pre-constructed text generation model outputs predicted reply texts one by one, and when outputting the next predicted reply text, the previous predicted reply text is combined for output, that is, when predicting the next predicted reply text, in addition to the first dialogue user vector, the second dialogue user vector, and the historical dialogue, the previous predicted reply text is also used for calculation to obtain the next predicted reply text.

[0065] And during training, the predicted conversion rate of the pre-constructed text generation model is trained, specifically, the predicted conversion rate of the pre-constructed text generation model is obtained at the same time through the foregoing input.

[0066] Further, in an optional embodiment of the present application, the pre-constructed text generation model is GPT-2.

[0067] GPT-2 is an open-source natural language generation model that boasts high prediction accuracy and speed. Therefore, it performs exceptionally well in platforms requiring high real-time performance and accuracy, such as medical consultations.

[0068] In this embodiment, a pre-built text generation model (such as GPT-2) can be trained using historical data.

[0069] like Figure 2 As shown, Figure 2 This is a diagram illustrating the output of predicted response text during GPT-2 training.

[0070] exist Figure 2 In GPT-2, the input data includes: the identity information of the first user in the historical dialogue (UU is a vector), the identity information P of the second user in the historical dialogue (P is a vector), the historical dialogue, and the predicted response text output by the pre-built text generation model (initially D, representing null values). The historical dialogue may contain only one question or one dialogue sequence (i.e., one or more rounds of dialogue). The predicted response text is the output data of GPT-2. Figure 2 In the GPT-2, the predicted response texts abcd constitute the predicted response text sequence. When the second output b is generated, the output a of the previous prediction is also input into GPT-2. Thus, when predicting the next text output, the content of the previous predicted text is referenced, resulting in a smooth and continuous predicted response text sequence.

[0071] For example, when user A1 has a conversation with doctor B1, and the conversation is that user A1 asks "What should I do if I am calcium deficient?", the vectors U and P obtained after vectorizing A1 and B2, and the initial prediction result D (D can be null) are input into GPT-2. The first predicted response text output by GPT-2 is "deficient". Then, the vectors U and P obtained after vectorizing A1 and B2, and the first predicted response text "deficient" output by GPT-2 are input into GPT-2. The second predicted response text output by GPT-2 is "calcium". Then, the vectors U and P obtained after vectorizing A1 and B2, and the first two outputs of GPT-2 "calcium deficient" are input into GPT-2. The second predicted response text output by GPT-2 is "can". And so on, to obtain the predicted response text sequence "Calcium deficiency can be treated with vitamin D supplementation".

[0072] S3, calculating a first loss value between the predicted reply text sequence and the historical real reply sequence by a first loss function, calculating a second loss value between the predicted conversion rate and the real conversion rate by a second loss function, and constantly adjusting parameters of the pre-constructed text generation model according to the first loss value and the second loss value to obtain a pre-trained text generation model.

[0073] In the embodiment, the first dialogue user identity, the second dialogue user identity and the historical dialogue are spliced together to train the training text generation model. Thus, when training, the generated text is also affected by the styles of the first dialogue user and the second dialogue user, so that the pre-trained text generation model can output according to different users.

[0074] In the embodiment, the first loss function can be a cross entropy loss function (Cross Entropy), and the second loss function can be a mean square error loss function.

[0075] Further, in an optional embodiment of the present application, the first loss function is maximum likelihood estimation.

[0076] Further, in an optional embodiment of the present application, the second loss function comprises:

[0077]

[0078] wherein N is the total number of the plurality of historical dialogues, is a real conversion rate of a historical real reply sequence corresponding to the historical dialogue i, is a predicted conversion rate of a predicted reply text sequence corresponding to the historical dialogue i.

[0079] Specifically, the historical dialogue i is one of the N historical dialogues.

[0080] Further, in an optional embodiment of the present application, the following objective function can also be constructed based on the first loss function and the second loss function: and then the parameters of the pre-constructed text generation model are adjusted according to the value of the objective function L.

[0081]

[0082] and then the parameters of the pre-constructed text generation model are adjusted according to the value of the objective function , wherein, is a coefficient of [0, 1], and specifically, The value of can be preset.

[0083] S4, obtaining a target dialogue text, identity information of a question user in the target dialogue text and identity information of a reply user.

[0084] In this embodiment, the target dialogue text can contain an input message of a questioning user, or contain at least one input message of a questioning user and at least one input message of a replying user.

[0085] In this embodiment, the identity information can be identity identification information, for example, an identity ID.

[0086] In this embodiment, the target dialogue question can be obtained from the dialogue text box.

[0087] S5, input the target dialogue text, the identity information of the questioning user and the identity information of the replying user in the target dialogue text into the pre-trained text generation model, to obtain the target reply text sequence of the replying user for the target dialogue text output by the pre-trained text generation model.

[0088] For example, in a new dialogue task, the questioning user of the target dialogue question is patient M, the content of the dialogue question is "when can you hang your number", and the dialogue object of the questioning user is doctor N. The doctor is the replying user. The target dialogue text "when can you hang your number", the identity information "patient M" of the questioning user and the identity information "doctor N" of the replying user are input into the pre-trained text generation model, to obtain the reply text input by the pre-trained text generation model in line with the language expression habit of the doctor N, that is, the target reply text sequence of the doctor N for the target dialogue text.

[0089] In the embodiment, historical data is acquired, the historical data including a plurality of historical dialogues, identity information of a first dialogue user in the historical dialogues, identity information of a second dialogue user in the historical dialogues, a historical real reply sequence corresponding to each of the historical dialogues, and a real conversion rate of the historical real reply sequence; a pre-constructed text generation model is acquired, the pre-constructed text generation model being trained by using the plurality of historical dialogues, the identity information of the first dialogue user in the historical dialogues, and the identity information of the second dialogue user in the historical dialogues, a predicted reply text sequence corresponding to each of the historical dialogues being output by the pre-constructed text generation model during training, and a predicted conversion rate of the predicted reply text sequence, the predicted reply text sequence including predicted reply texts output by the pre-constructed text generation model successively; a first loss value between the predicted reply text sequence and the historical real reply sequence is calculated by using a first loss function, a second loss value between the predicted conversion rate and the real conversion rate is calculated by using a second loss function, parameters of the pre-constructed text generation model are adjusted continuously according to the first loss value and the second loss value, and a pre-trained text generation model is obtained; target dialogue text, identity information of a questioning user in the target dialogue text, and identity information of a reply user are acquired; the target dialogue text, the identity information of the questioning user in the target dialogue text, and the identity information of the reply user are input into the pre-trained text generation model, and a target reply text sequence of the reply user for the target dialogue text output by the pre-trained text generation model is obtained. By training the text generation model by concatenating the identity information of the first dialogue user, the identity information of the second dialogue user, and the historical dialogues in the historical dialogues, the pre-trained text generation model can output individually according to different users, so that the purpose of accurately generating individualized dialogue content is achieved.

[0090] As Figure 3 shown in FIG. 1, which is a functional module diagram of a dialogue generation device provided by an embodiment of the present application.

[0091] The dialogue generation device 300 can be installed in an electronic device. According to the functions implemented, the dialogue generation device 300 includes a first acquisition module 101, a prediction module 102, a screening module 103, a second acquisition module 104, and a determination module 105. The modules of the present application can also be referred to as units, which refer to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, and are stored in the memory of the electronic device.

[0092] In the embodiment, the functions of the modules / units are as follows:

[0093] The first obtaining module 301 is configured to obtain historical data, wherein the historical data comprises a plurality of historical dialogues, identity information of a first dialogue user in the historical dialogues, identity information of a second dialogue user in the historical dialogues, a historical real answer sequence corresponding to each of the historical dialogues, and a real conversion rate of the historical real answer sequence.

[0094] In this embodiment, the historical data is a collection of dialogues in the past period of time. For example, a collection of dialogues between patients and doctors in the past three years stored in a medical database.

[0095] In this embodiment, the number of historical dialogues contained in the historical data is a plurality, the first dialogue user and the second dialogue user are different users, and the first dialogue user and the second dialogue user can be the same or different between different historical dialogues. For example, there is a historical dialogue in the historical data, the first dialogue user is user A, and the second dialogue user is user B. There is another dialogue in the historical data, the first dialogue user is user C, and the second dialogue user is user D. There is still another dialogue in the historical data, the first dialogue user is user A, and the second dialogue user is user D.

[0096] In this embodiment, each historical dialogue can only contain one question (such as the question "How is the weather today?" of the first dialogue user), or can include at least one question and at least one answer (such as the question 1 "How is the weather today?" of the first dialogue user, the answer 1 "Sunny" of the second dialogue user, and the question 2 "How about playing football outside?" of the first dialogue user), and each historical dialogue has a corresponding historical real answer sequence, identity information of the first dialogue user, identity information of the second dialogue user, and a conversion rate of the historical real answer sequence corresponding to the historical dialogue.

[0097] The identity information can be a unique identifier such as a name or an account number. The real conversion rate is actually counted. During the process of the dialogue or after the dialogue ends, the questioner places an order to purchase a product, or subscribes to a service, or continues to click to proceed to the next step. For example, the conversion rate is 0 or 1, where 1 indicates a transaction, and 0 indicates no transaction. That is, if the conversion rate is 0, it indicates that the question user who raised the question did not place an order to purchase a product or subscribe to a service or further operation during the dialogue process or after the dialogue ended. That is, if the conversion rate is 1, it indicates that the question user who raised the question placed an order to purchase a product or subscribed to a service or further operation during the dialogue process or after the dialogue ended.

[0098] In this embodiment, the historical true reply sequence contains multiple reply texts (i.e. different single texts). For example, the historical true reply sequence contains "eat more vitamin D", wherein "eat", "more", "vitamin", "D" are six different reply texts.

[0099] Specifically, the historical data can be collected in the form of , wherein, , represents a party in a conversation and the content of the speech in the historical data, "text" stores the text content of the conversation itself, "sender" represents the conversation user sending the text message, and "sender" is an int type, 0 represents the first conversation user (such as a patient), and 1 represents the second conversation user (such as a doctor); the text content belonging to the same conversation sequence (i.e. one conversation can include one or more rounds of conversation, and the first and second conversation users take turns speaking in one or more rounds of conversation) is spliced into a long text, and the messages between different conversation users are separated by special characters [C] or [D], wherein [C] can represent the first conversation user (such as the inquiring party), and [D] can represent the second conversation user (such as the receiving party); the multiple rounds of conversation messages belonging to a conversation sequence are spliced by a special character [next], and a start character [CLS] and an end character [SEP] are added at the beginning and the end, respectively, and finally the continuous conversation (i.e. multiple rounds of conversation) of a conversation sequence is formed into the following conversation data:

[0100] [CLS][C]string1[D]string2[next]string3[C]string4[SEP]

[0101] , wherein string1, string2, string3 and string4 are the contents of the text in the foregoing .

[0102] Further, in an optional embodiment of the application, the device further comprises:

[0103] A data cleaning module is configured to clean the historical data after obtaining the historical data.

[0104] In this embodiment, the data cleaning of the historical data includes de-duplication processing, outlier detection processing and standardization processing.

[0105] By removing duplicate data from historical data, the efficiency and accuracy of model training can be improved.

[0106] Therefore, in the embodiment, by cleaning the historical data, the efficiency and accuracy of model training can be improved.

[0107] Further, in an optional embodiment of the present application, the device further comprises:

[0108] The data enhancement module is configured to perform data enhancement on the historical data after obtaining the historical data.

[0109] Specifically, the data enhancement module is specifically configured to: obtain synonyms of a plurality of source texts in the historical conversation, replace the plurality of source texts with the synonyms, determine the replaced data as new data, and add the new data to the historical data to obtain data-enhanced historical data.

[0110] In the embodiment, by performing data enhancement on the historical data, the data volume of the historical data can be expanded, thereby improving the accuracy and robustness of training.

[0111] The training module 302 is configured to obtain a pre-built text generation model, train the pre-built text generation model using a plurality of historical conversations, identity information of a first conversation user in the historical conversations, and identity information of a second conversation user in the historical conversations, obtain a predicted reply text sequence corresponding to each of the historical conversations output by the pre-built text generation model during training, and a predicted conversion rate of the predicted reply text sequence, the predicted reply text sequence comprising predicted reply texts output by the pre-built text generation model one by one.

[0112] In the embodiment, the pre-built text generation model can be a natural language processing model. For example, the pre-built text generation model is a BERT model, or the pre-built text generation model is a CNN, or an LSTM model.

[0113] In training the pre-built text generation model, identity information of the first dialogue user in the historical dialogue and identity information of the second dialogue user in the historical dialogue are respectively vectorized to obtain a first dialogue user vector and a second dialogue user vector, and the first dialogue user vector, the second dialogue user vector and the historical dialogue (each character is mapped to a vector) are input into the pre-built text generation model, the pre-built text generation model outputs a predicted reply text one by one, and when the next predicted reply text is output, the previous predicted reply text is combined for output, that is, when the next predicted reply text is predicted, the previous predicted reply text is used for calculation in addition to the first dialogue user vector, the second dialogue user vector and the historical dialogue to obtain the next predicted reply text.

[0114] In addition, in the training, the prediction conversion rate of the pre-built text generation model is trained, specifically, the prediction conversion rate of the pre-built text generation model is obtained through the foregoing input.

[0115] Further, in an optional embodiment of the present application, the pre-built text generation model is GPT-2.

[0116] GPT-2 is an open source natural language generation model, which has high prediction accuracy and fast speed. Therefore, in a platform for medical diagnosis and the like which requires high real-time performance and high accuracy, GPT-2 can have good effects.

[0117] In the embodiment, the pre-built text generation model (such as GPT-2) can be trained through historical data.

[0118] As shown in Figure 2 , it is a schematic diagram of outputting a predicted reply text when training GPT-2. Figure 2 In

[0119] , the input data of GPT-2 includes: UU (UU is a vector) representing identity information of the first dialogue user in the historical dialogue, P (P is a vector) representing identity information of the second dialogue user in the historical dialogue, the historical dialogue, and the predicted reply text (initially D, representing null) output by the pre-built text generation model; wherein the historical dialogue can only include one question or one dialogue sequence (i.e. one or more rounds of dialogue); the predicted reply text is the output data of GPT-2. Figure 2 In Figure 2 , the predicted reply text abcd output by GPT-2 constitutes a predicted reply text sequence, and when the second output b is output, the previous predicted output a is also input into GPT-2, so that when the next text output is predicted, the content of the previous predicted text is also referred to, and a smooth and continuous predicted reply text sequence is obtained.

[0120] For example, when user A1 has a conversation with doctor B1, the conversation content is that user A1 asks "what should I do if I lack calcium", then "what should I do if I lack calcium", the vector U and the vector P obtained after vectorizing A1 and B2, and the initial prediction result D (D can be empty) are input into GPT-2, and the first prediction reply text output by GPT-2 is "lack"; then "what should I do if I lack calcium", the vector U and the vector P obtained after vectorizing A1 and B2, and the first prediction reply text "lack" output by GPT-2 are input into GPT-2, and the second prediction reply text output by GPT-2 is "calcium"; then "what should I do if I lack calcium", the vector U and the vector P obtained after vectorizing A1 and B2, and the output result "lack calcium" of GPT-2 in the previous two times are input into GPT-2, and the second prediction reply text output by GPT-2 is "can"; and so on, to obtain the prediction reply text sequence "lack calcium can supplement vitamin D".

[0121] The parameter adjustment module 303 is configured to calculate a first loss value between the prediction reply text sequence and the historical real reply sequence by using a first loss function, calculate a second loss value between the prediction conversion rate and the real conversion rate by using a second loss function, and constantly adjust parameters of the pre-constructed text generation model according to the first loss value and the second loss value, to obtain the pre-trained text generation model.

[0122] In this embodiment, the first conversation user identity, the second conversation user identity and the historical conversation are spliced together to train the text generation model, so that when training, the generated text is also affected by the styles of the first conversation user and the second conversation user, so that the pre-trained text generation model can output individually according to different users.

[0123] In this embodiment, the first loss function can be a cross entropy loss function (Cross Entropy), and the second loss function can be a mean square error loss function.

[0124] Further, in an optional embodiment of the present application, the first loss function is maximum likelihood estimation.

[0125] Further, in an optional embodiment of the present application, the second loss function includes:

[0126]

[0127] wherein N is the total number of the plurality of historical conversations, is a real conversion rate of a historical real reply sequence corresponding to the historical conversation i, is a prediction conversion rate of a prediction reply text sequence corresponding to the historical conversation i.

[0128] Specifically, the historical dialogue i is one of the N historical dialogues.

[0129] Further, in an optional embodiment of the present application, a target function can be constructed based on the first loss function and the second loss function, and then the parameters of the pre-constructed text generation model are adjusted according to the value of the target function L

[0130]

[0131] and the parameters of the pre-constructed text generation model are adjusted according to the value of the target function L , wherein, a is a coefficient in [0, 1], and specifically, the value of a can be preset.

[0132] The second acquisition module 304 is configured to acquire a target dialogue text, identity information of a questioning user in the target dialogue text, and identity information of a replying user.

[0133] In the embodiment, the target dialogue text can include an input message of the questioning user, or can include at least one input message of the questioning user and at least one input message of the replying user.

[0134] In the embodiment, the identity information can be identity identification information, for example, an identity ID.

[0135] In the embodiment, the target dialogue question can be acquired from a dialogue text box.

[0136] The dialogue prediction module 305 is configured to input the target dialogue text, the identity information of the questioning user in the target dialogue text, and the identity information of the replying user into the pre-trained text generation model to obtain a target reply text sequence of the replying user to the target dialogue text output by the pre-trained text generation model.

[0137] For example, in a new dialogue task, the questioning user of the target dialogue question is a patient M, the content of the dialogue question is "when can you hang your number", and the dialogue object of the questioning user is a doctor N. The doctor is the replying user. The target dialogue text "when can you hang your number", the identity information "patient M" of the questioning user, and the identity information "doctor N" of the replying user are input into the pre-trained text generation model to obtain a reply text in line with the language expression habit of the doctor N, i.e., a target reply text sequence of the doctor N to the target dialogue text.

[0138] ​In the embodiment, historical data is acquired, the historical data including a plurality of historical dialogues, identity information of a first dialogue user in the historical dialogues, identity information of a second dialogue user in the historical dialogues, a historical real reply sequence corresponding to each of the historical dialogues, and a real conversion rate of the historical real reply sequence; a pre-constructed text generation model is acquired, the pre-constructed text generation model being trained by using the plurality of historical dialogues, the identity information of the first dialogue user in the historical dialogues, and the identity information of the second dialogue user in the historical dialogues, a predicted reply text sequence corresponding to each of the historical dialogues being output by the pre-constructed text generation model during training, and a predicted conversion rate of the predicted reply text sequence, the predicted reply text sequence including predicted reply texts output by the pre-constructed text generation model in sequence; a first loss value between the predicted reply text sequence and the historical real reply sequence is calculated by using a first loss function, a second loss value between the predicted conversion rate and the real conversion rate is calculated by using a second loss function, parameters of the pre-constructed text generation model are continuously adjusted according to the first loss value and the second loss value, and a pre-trained text generation model is obtained; target dialogue text, identity information of a questioning user in the target dialogue text, and identity information of a reply user are acquired; the target dialogue text, the identity information of the questioning user in the target dialogue text, and the identity information of the reply user are input into the pre-trained text generation model, and a target reply text sequence of the reply user for the target dialogue text output by the pre-trained text generation model is obtained. By training the text generation model by concatenating the identity information of the first dialogue user, the identity information of the second dialogue user, and the historical dialogues in the historical dialogues, the pre-trained text generation model can be personalized according to different users, thereby achieving the purpose of accurately generating personalized dialogue content.

[0139] As Figure 4 shown in FIG. 1, which is a structural schematic diagram of an electronic device for implementing a dialogue generation method according to an embodiment of the present application.

[0140] The electronic device 1 can include a processor 10, a memory 11, and a bus, and can further include a computer program, such as a dialogue generation program, stored in the memory 11 and executable on the processor 10.

[0141] The memory 11 includes at least one type of readable storage medium, such as a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., an SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 11 can include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 can be used to store application software and various data installed in the electronic device 1, such as the code of the dialogue generation program, etc., and can also be used to temporarily store data that has been output or will be output.

[0142] The processor 10 can be composed of an integrated circuit in some embodiments, such as a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, combinations of various control chips, etc. The processor 10 is the control core of the electronic device, which connects various components of the electronic device through various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules (such as the dialogue generation program, etc.) stored in the memory 11 and calling data stored in the memory 11.

[0143] The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize the connection and communication between the memory 11 and at least one processor 10, etc.

[0144] Figure 4 Only the electronic device with components is shown, and those skilled in the art can understand that, Figure 4 The structure shown does not constitute a limitation on the electronic device 1, and can include fewer or more components than shown, or combine certain components, or different component arrangements.

[0145] For example, although not shown, the electronic device 1 can also include a power source (such as a battery) to power the various components. Preferably, the power source can be logically connected to the at least one processor 10 through a power management device, so that the power management device can implement functions such as charge management, discharge management, and power consumption management. The power source can also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The electronic device 1 can also include various sensors, Bluetooth modules, Wi-Fi modules, and the like, which are not described here.

[0146] Further, the electronic device 1 can also include a network interface, which can optionally include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, and the like), which is typically used to establish a communication connection between the electronic device 1 and other electronic devices.

[0147] Optionally, the electronic device 1 can also include a user interface, which can be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally a standard wired interface, a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch, and the like. The display can also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visualized user interface.

[0148] It should be understood that the embodiments are only for illustration and are not limited in the scope of the patent application by this structure.

[0149] The conversation generation program stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, which, when executed in the processor 10, can implement:

[0150] obtaining historical data, the historical data including multiple historical conversations, identity information of a first conversation user in the historical conversations, identity information of a second conversation user in the historical conversations, a historical real reply sequence corresponding to each of the historical conversations, and a real conversion rate of the historical real reply sequence;

[0151] obtain a pre-constructed text generation model, train the pre-constructed text generation model by using a plurality of the historical dialogues, identity information of a first dialogue user in the historical dialogues, identity information of a second dialogue user in the historical dialogues, obtain a predicted reply text sequence corresponding to each of the historical dialogues output by the pre-constructed text generation model during training, and a predicted conversion rate of the predicted reply text sequence, wherein the predicted reply text sequence comprises predicted reply texts output by the pre-constructed text generation model successively;

[0152] calculate a first loss value between the predicted reply text sequence and the historical real reply sequence by using a first loss function, calculate a second loss value between the predicted conversion rate and the real conversion rate by using a second loss function, constantly adjust parameters of the pre-constructed text generation model according to the first loss value and the second loss value, and obtain a pre-trained text generation model;

[0153] obtain target dialogue text, identity information of a questioning user in the target dialogue text, and identity information of a reply user;

[0154] input the target dialogue text, the identity information of the questioning user in the target dialogue text, and the identity information of the reply user into the pre-trained text generation model, and obtain a target reply text sequence of the reply user for the target dialogue text output by the pre-trained text generation model.

[0155] Further, the modules / units of the electronic device 1 are implemented in the form of software function units and sold or used as independent products, which can be stored in a computer readable storage medium. The computer readable storage medium can be volatile or non-volatile. For example, the computer readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a U disk, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM).

[0156] The application further provides a computer readable storage medium, which stores a computer program, and the computer program can realize the following when executed by a processor of an electronic device:

[0157] obtain historical data, wherein the historical data comprises a plurality of historical dialogues, identity information of a first dialogue user in the historical dialogues, identity information of a second dialogue user in the historical dialogues, a historical real reply sequence corresponding to each of the historical dialogues, and a real conversion rate of the historical real reply sequence;

[0158] obtain a pre-constructed text generation model, train the pre-constructed text generation model by using a plurality of the historical dialogues, identity information of a first dialogue user in the historical dialogues, identity information of a second dialogue user in the historical dialogues, obtain a predicted reply text sequence corresponding to each of the historical dialogues output by the pre-constructed text generation model during the training, and a predicted conversion rate of the predicted reply text sequence, the predicted reply text sequence comprising predicted reply texts output by the pre-constructed text generation model successively;

[0159] calculate a first loss value between the predicted reply text sequence and the historical real reply sequence by a first loss function, calculate a second loss value between the predicted conversion rate and the real conversion rate by a second loss function, constantly adjust parameters of the pre-constructed text generation model according to the first loss value and the second loss value, and obtain a pre-trained text generation model;

[0160] obtain a target dialogue text, identity information of a questioning user in the target dialogue text, and identity information of a reply user;

[0161] input the target dialogue text, the identity information of the questioning user in the target dialogue text, and the identity information of the reply user into the pre-trained text generation model, and obtain a target reply text sequence of the reply user for the target dialogue text output by the pre-trained text generation model.

[0162] In addition, each functional module in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional modules.

[0163] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0164] Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and therefore all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any additional reference signs in the claims should not be considered as limiting the claims to which they relate.

[0165] The blockchain referred to in the present application is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism and encryption algorithm. The blockchain is essentially a decentralized database, and is a series of data blocks associated using cryptographic methods, each data block containing information of a batch of network transactions, for verifying the validity (anti-fake) of the information and generating the next block. The blockchain can include a blockchain underlying platform, a platform product service layer and an application service layer.

[0166] Embodiments of the present application can acquire and process relevant data based on holographic projection technology. Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0167] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the system claims can also be implemented by one unit or device through software or hardware. The second word is used to indicate the name, and does not indicate any specific order.

[0168] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A dialogue generation method, characterized by, The method comprises: obtaining historical data, the historical data comprising a plurality of historical conversations, identity information of a first conversation user in the historical conversations, identity information of a second conversation user in the historical conversations, a historical real reply sequence corresponding to each of the historical conversations, and a real conversion rate of the historical real reply sequence; obtaining a pre-constructed text generation model, training the pre-constructed text generation model using the plurality of historical conversations, the identity information of the first conversation user in the historical conversations, and the identity information of the second conversation user in the historical conversations, obtaining a predicted reply text sequence corresponding to each of the historical conversations output by the pre-constructed text generation model during training, and a predicted conversion rate of the predicted reply text sequence, the predicted reply text sequence comprising predicted reply texts output by the pre-constructed text generation model in sequence; calculating a first loss value between the predicted reply text sequence and the historical real reply sequence by a first loss function, calculating a second loss value between the predicted conversion rate and the real conversion rate by a second loss function, constructing an objective function according to the first loss value and the second loss value, and constantly adjusting parameters of the pre-constructed text generation model according to a value of the objective function to obtain a pre-trained text generation model; obtaining a target conversation text, identity information of a questioning user in the target conversation text, and identity information of a replying user; inputting the target conversation text, the identity information of the questioning user in the target conversation text, and the identity information of the replying user into the pre-trained text generation model to obtain a target reply text sequence of the replying user for the target conversation text output by the pre-trained text generation model.

2. The dialogue generation method of claim 1, wherein, The second loss function comprises: wherein N is the total number of the plurality of historical conversations, is a true conversion rate of a historical true reply sequence corresponding to the historical conversation i, is a predicted conversion rate of a predicted reply text sequence corresponding to the historical conversation i.

3. The dialogue generation method of claim 1, wherein, The pre-constructed text generation model is GPT-2.

4. The dialogue generation method according to any one of claims 1 to 3, wherein, After obtaining the historical data, the method further comprises: performing data enhancement on the historical data.

5. The dialogue generation method according to any one of claims 1 to 3, wherein, After obtaining the historical data, the method further comprises: performing data cleaning on the historical data.

6. A dialogue generation apparatus characterized by comprising: The device comprises: a first obtaining module configured to obtain historical data, the historical data comprising a plurality of historical conversations, identity information of a first conversation user in the historical conversations, identity information of a second conversation user in the historical conversations, a historical real reply sequence corresponding to each of the historical conversations, and a real conversion rate of the historical real reply sequence; a training module configured to obtain a pre-constructed text generation model, train the pre-constructed text generation model using the plurality of historical conversations, the identity information of the first conversation user in the historical conversations, and the identity information of the second conversation user in the historical conversations, obtain a predicted reply text sequence corresponding to each of the historical conversations output by the pre-constructed text generation model during training, and a predicted conversion rate of the predicted reply text sequence, the predicted reply text sequence comprising predicted reply texts output by the pre-constructed text generation model in sequence; The parameter adjustment module is configured to calculate a first loss value between the predicted reply text sequence and the historical real reply sequence by using a first loss function, calculate a second loss value between the predicted conversion rate and the real conversion rate by using a second loss function, and continuously adjust parameters of the pre-constructed text generation model according to the first loss value and the second loss value to obtain a pre-trained text generation model. The second obtaining module is configured to obtain target dialogue text, identity information of a questioning user in the target dialogue text, and identity information of a replying user. The dialogue prediction module is configured to input the target dialogue text, the identity information of the questioning user in the target dialogue text, and the identity information of the replying user into the pre-trained text generation model to obtain a target reply text sequence of the replying user for the target dialogue text output by the pre-trained text generation model.

7. The dialogue generation apparatus of claim 6, wherein The second loss function includes: wherein N is the total number of the plurality of historical conversations, is a true conversion rate of a historical true reply sequence corresponding to the historical conversation i, is a predicted conversion rate of a predicted reply text sequence corresponding to the historical conversation i.

8. The dialogue generation apparatus of claim 6, wherein The device further includes: The data enhancement module is configured to perform data enhancement on the historical data after the historical data is obtained.

9. An electronic device, comprising: The electronic device includes: at least one processor; and The memory is in communication with the at least one processor; wherein The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the dialogue generation method according to any one of claims 1 to 5.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the dialogue generation method according to any one of claims 1 to 5.

Images