A dialogue question generation method, device and equipment and storage medium

Abstract

The application provides a dialogue question generation method and device, equipment and a storage medium, wherein the method comprises: obtaining reference documents in a specified scene and historical dialogue data of a dialogue developed for the reference documents; identifying the intention of each round of dialogue contained in the historical dialogue data, and predicting the intention of the next round of dialogue, taking the identified intention and the predicted intention as target intention information; generating an answer to the next round of dialogue based on the historical dialogue data, the reference documents and the target intention information; and generating a question for the next round of dialogue based on the historical dialogue data, the reference documents and the answer to the next round of dialogue. The dialogue question generation method provided by the application can automatically generate a question and an answer for the next round of dialogue, i.e. a question and answer pair for the next round of dialogue. The dialogue question generation method provided by the application has strong generalization ability and stable effect of generating a question and answer pair.

Description

Technical Field

[0001] This invention relates to the field of intelligent dialogue technology, and in particular to a method, apparatus, device, and storage medium for generating dialogue questions. Background Technology

[0002] Dialogue question generation technology aims to predict questions (or questions and answers for the next round of dialogue) based on historical dialogues, according to the needs of a specified scenario. This technology can be used to construct question-and-answer corpora and to guide dialogue in actual conversations. As an emerging and challenging natural language processing task, dialogue question generation has become a new research direction for researchers in related fields.

[0003] Most current dialogue question generation solutions are based on manual methods, meaning that humans predict the questions (or the questions and answers) for the next round of dialogue based on historical conversations. However, manual dialogue question generation solutions are time-consuming, labor-intensive, and require highly skilled personnel; in other words, they are inefficient and costly. Summary of the Invention

[0004] In view of this, the present invention provides a method, apparatus, device, and storage medium for generating dialogue questions, in order to solve the problems of low efficiency and high cost of existing manual dialogue question generation schemes. The technical solution is as follows:

[0005] A method for generating dialogue questions, comprising:

[0006] Obtain reference documents in a specified scenario, as well as historical dialogue data of conversations initiated in response to the reference documents;

[0007] Identify the intent of each round of dialogue contained in the historical dialogue data, predict the intent of the next round of dialogue, and use the identified intent and the predicted intent as target intent information.

[0008] Based on the historical dialogue data, the reference document, and the target intent information, generate the answer for the next round of dialogue;

[0009] Based on the historical dialogue data, the reference document, and the answer to the next round of dialogue, generate the question for the next round of dialogue.

[0010] Optionally, generating the answer for the next round of dialogue based on the historical dialogue data, the reference document, and the target intent information includes:

[0011] Based on the historical dialogue data, the reference document, and the target intent information, candidate answers are extracted from the reference document to obtain a first candidate answer set composed of the extracted candidate answers;

[0012] Based on the intent of the next round of dialogue in the target intent information, the answer to the next round of dialogue is selected from the first candidate answer set.

[0013] Optionally, each candidate answer in the first candidate answer set has a predicted probability, which is the probability that the corresponding candidate answer is the answer for the next round of dialogue;

[0014] The step of selecting answers for the next round of dialogue from the first candidate answer set based on the target intent information includes:

[0015] Delete the candidate answers whose predicted probability is less than a preset probability threshold from the first candidate answer set to obtain the second candidate answer set;

[0016] Select candidate answers from the second set of candidate answers that match the intent of the next round of dialogue, and use them as the answers for the next round of dialogue.

[0017] Optionally, the step of selecting candidate answers from the second candidate answer set that match the intent of the next round of dialogue as answers for the next round of dialogue includes:

[0018] Identify the named entities contained in each candidate answer in the second candidate answer set;

[0019] The candidate answers that match the named entities contained in the second candidate answer set with the intent of the next round of dialogue are selected as the first target answer;

[0020] If there is only one first target answer, then that first target answer is determined as a candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue; if there are multiple first target answers, then the first target answer with the highest predicted probability among the multiple first target answers is determined as a candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue.

[0021] Optionally, the step of selecting candidate answers from the second candidate answer set that match the intent of the next round of dialogue as answers for the next round of dialogue further includes:

[0022] If the second candidate answer set does not contain a candidate answer whose included named entity matches the intent of the next round of dialogue, then based on the pre-constructed relevance information between intent and named entity, the named entity related to the intent of the next round of dialogue is determined, wherein the relevance information between intent and named entity includes several intents and the named entity related to each of the several intents.

[0023] Based on named entities related to the intent of the next round of dialogue, candidate answers that match the intent of the next round of dialogue are selected from the second candidate answer set and used as the answers for the next round of dialogue.

[0024] Optionally, the process of constructing the relevance information between the intent and the named entity includes:

[0025] Acquire multiple dialogue corpora labeled with intent information, where each dialogue corpus includes multiple rounds of dialogue, and each round of dialogue is labeled with intent;

[0026] For each dialogue corpus containing each round of dialogue, named entity recognition is performed on the answers in that round of dialogue, and the identified named entities are used as named entities related to the intent labeled in that round of dialogue;

[0027] Based on the intent of each dialogue annotation contained in each dialogue corpus, and the named entities related to the intent of each dialogue annotation contained in each dialogue corpus, the relevance information between the intent and the named entities is constructed.

[0028] Optionally, the step of filtering candidate answers from the second candidate answer set that match the intent of the next round of dialogue based on named entities related to the intent of the next round of dialogue, and using these as answers for the next round of dialogue, includes:

[0029] From the second candidate answers, select candidate answers that contain named entities related to the intent of the next round of dialogue, and use them as the second target answer;

[0030] If there is only one second target answer, then that second target answer is determined as a candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue;

[0031] If there are multiple second target answers, the second target answer with the highest predicted probability among the multiple second target answers is determined as the candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue.

[0032] Optionally, identifying the intent of each round of dialogue contained in the historical dialogue data and predicting the intent of the next round of dialogue includes:

[0033] Based on each round of dialogue contained in the historical dialogue data, the intent of each round of dialogue contained in the historical dialogue data is identified using a pre-trained intent recognition model.

[0034] Based on the vector representation of the last round of dialogue contained in the historical dialogue data, the intent of the next round of dialogue is predicted using the intent recognition model, wherein the vector representation of the last round of dialogue is a vector containing dialogue information of each round of dialogue determined by the intent recognition model during the intent recognition process.

[0035] The intent recognition model is trained using training dialogue data labeled with real intent information. The real intent information labeled in the training dialogue data includes the real intent of each round of dialogue and the real intent of the next round of dialogue contained in the training dialogue data.

[0036] Optionally, generating the answer for the next round of dialogue based on the historical dialogue data, the reference document, and the target intent information includes:

[0037] Based on the historical dialogue data, the reference document, the target intent information, and the pre-trained answer extraction model, the answer to the next round of dialogue is generated.

[0038] The answer extraction model is trained using training samples labeled with real answers to the next round of dialogue. The training samples labeled with real answers to the next round of dialogue include training dialogue data, reference documents corresponding to the training dialogue data, and intent information that is identified and predicted based on the intent recognition model.

[0039] Optionally, generating questions for the next round of dialogue based on the historical dialogue data, the reference document, and the answer to the next round of dialogue includes:

[0040] Based on the historical dialogue data, the reference document, and the answer to the next round of dialogue, a question for the next round of dialogue is generated using a pre-trained question generation model.

[0041] The question generation model is trained using training samples labeled with real questions for the next round of dialogue. These training samples include training dialogue data, reference documents corresponding to the training dialogue data, and answers for the next round of dialogue determined by the answer extraction model based on the training dialogue data.

[0042] A dialogue question generation device includes: a data acquisition module, an intent recognition and prediction module, an answer generation module, and a question generation module.

[0043] The data acquisition module is used to acquire reference documents in a specified scenario, as well as historical dialogue data of dialogues initiated in response to the reference documents.

[0044] The intent recognition and prediction module is used to identify the intent of each round of dialogue contained in the historical dialogue data, and predict the intent of the next round of dialogue, using the identified intent and the predicted intent as target intent information.

[0045] The answer generation module is used to generate an answer for the next round of dialogue based on the historical dialogue data, the reference document, and the target intent information.

[0046] The question generation module is used to generate questions for the next round of dialogue based on the historical dialogue data, the reference document, and the answer to the next round of dialogue.

[0047] A dialogue question generation device includes: a memory and a processor;

[0048] The memory is used to store programs;

[0049] The processor is configured to execute the program to implement each step of the dialogue question generation method described above.

[0050] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the dialogue question generation method described in any of the preceding claims.

[0051] The present invention provides a method, apparatus, device, and storage medium for generating dialogue questions. First, it acquires a reference document in a specified scenario and historical dialogue data related to the dialogues unfolding around the reference document. Then, it identifies the intent of each round of dialogue contained in the historical dialogue data and predicts the intent of the next round of dialogue. Next, based on the historical dialogue data, the reference document, and the identified and predicted intent, it generates the answer to the next round of dialogue. Finally, based on the historical dialogue data, the reference document, and the answer to the next round of dialogue, it generates the question for the next round of dialogue. The dialogue question generation method provided by this invention can automatically predict the answer to the next round of dialogue using the intent of each round of dialogue contained in the historical dialogue data and the intent of the next round of dialogue. Furthermore, it can automatically predict the question for the next round of dialogue using the predicted answer, thereby obtaining a question-and-answer pair for the next round of dialogue. Compared to manual dialogue question generation schemes, the dialogue question generation method provided by this invention has higher efficiency and lower cost, and the question-and-answer pair generation effect is better. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the hardware architecture involved in this invention;

[0054] Figure 2 A flowchart illustrating the dialogue question generation method provided in an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of the process of identifying the intent of each round of dialogue contained in historical dialogue data and predicting the intent of the next round of dialogue based on the intent recognition model provided in an embodiment of the present invention.

[0056] Figure 4 This is a flowchart illustrating the process of generating answers for the next round of dialogue based on historical dialogue data, reference documents, and target intent information, as provided in an embodiment of the present invention.

[0057] Figure 5 This is a schematic diagram of the structure of the dialogue question generation device provided in an embodiment of the present invention;

[0058] Figure 6 This is a schematic diagram of the structure of a dialogue question generation device provided in an embodiment of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Considering the low efficiency and high cost of existing manual dialogue question generation schemes, the inventors of this case attempted to propose an automatic dialogue question generation scheme to improve the efficiency and reduce the cost of dialogue question generation. To this end, research was conducted, and the initial idea was:

[0061] A rule-based dialogue question generation method is adopted, which relies on resources such as grammatical rules, shallow semantic parsing, and lexical resources to mine question-answer pairs for the next round of dialogue from historical dialogues and with reference to the dialogue context (content in the reference document).

[0062] While rule-based dialogue question generation methods can improve the efficiency and reduce the cost of dialogue question generation to some extent, they have many drawbacks, such as high customization, poor generalization ability, inability to extract question-answer pairs outside the rules, and unstable results in generating question-answer pairs.

[0063] Given the numerous shortcomings of rule-based dialogue question generation methods, the inventors of this case conducted further research and came up with a deep learning-based dialogue question generation method. This method, given the answer to the next round of dialogue, uses a deep learning model to generate the question for the next round of dialogue based on historical dialogues, reference documents, and the answer to the next round of dialogue.

[0064] However, the above-mentioned dialogue question generation method generates questions based on the known answers to the next round of dialogue. It cannot automatically generate questions and answers for the next round of dialogue based on historical dialogues. In addition, since the deep learning-based dialogue question generation method requires model training, the data used for model training needs to be labeled with answers. The labeling of answers is very labor-intensive and requires training professional labelers to perform the labeling.

[0065] In view of the shortcomings of deep learning-based dialogue question generation methods, the inventors of this case continued their research and finally proposed a dialogue question generation method with better performance. This method does not require knowing the answer to the next round of dialogue (no answer annotation is required). It can automatically generate the corresponding questions and answers for the next round by processing the reference document and the historical dialogue data of the dialogues that unfolded on the reference document. The method has strong generalization ability and the effect of generating question-answer pairs is relatively stable.

[0066] Before introducing the dialogue question generation method provided by this invention, the hardware architecture involved in this invention will be described first.

[0067] In one possible implementation, such as Figure 1 As shown, the hardware architecture involved in this invention may include: electronic device 101 and server 102.

[0068] For example, electronic device 101 can be any electronic product that can interact with a user through one or more means such as a keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as mobile phone, laptop, tablet computer, PDA, personal computer, wearable device, smart TV, PAD, etc.

[0069] It should be noted that, Figure 1 This is just one example; there can be many types of electronic devices, not limited to... Figure 1 The laptop in the middle.

[0070] For example, server 102 can be a single server, a server cluster consisting of multiple servers, or a cloud computing server center. Server 102 may include processors, memory, and network interfaces, etc.

[0071] For example, electronic device 101 can establish a connection and communicate with server 102 through a wireless communication network; for example, electronic device 101 can establish a connection and communicate with server 102 through a wired network.

[0072] Electronic device 101 can send a reference document and historical dialogue data of the dialogues that take place on the reference document to server 102. The server generates questions and answers for the next round of dialogue according to the dialogue question generation method provided by the present invention, and sends the generated questions and answers to electronic device 101.

[0073] In another possible implementation, the hardware architecture involved in this invention may include an electronic device. The electronic device is one with strong data processing capabilities.

[0074] For example, an electronic device can be any electronic product that can interact with a user through one or more means such as a keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as a PC, mobile phone, laptop, tablet computer, PDA, personal computer, etc.

[0075] The electronic device can generate questions and answers for the next round of dialogue according to the dialogue question generation method provided by the present invention, and output the generated questions and answers.

[0076] Those skilled in the art should understand that the above-described electronic devices and servers are merely examples, and other existing or future electronic devices or servers that are applicable to this invention should also be included within the scope of protection of this invention, and are hereby incorporated by reference.

[0077] The dialogue question generation method provided by the present invention will be described in the following embodiments.

[0078] First Embodiment

[0079] Please see Figure 2 The diagram illustrates a flowchart of a dialogue question generation method provided in an embodiment of the present invention, which may include:

[0080] Step S201: Obtain the reference document in the specified scenario, as well as the historical dialogue data of the dialogues that unfolded around the reference document.

[0081] The specified scenario can be any scenario that requires the generation of dialogue questions, such as a financial scenario. The historical dialogue data can include a single round of dialogue or multiple rounds of dialogue. Each round of dialogue in the historical dialogue data is a dialogue that revolves around the content in the reference document.

[0082] For example, if the specified scenario is a financial scenario and the reference document can be a financial product introduction document, then the historical dialogue data is the historical dialogue data of the conversations that take place in relation to the content of the financial product introduction document.

[0083] Step S202: Identify the intent of each round of dialogue contained in the historical dialogue data, predict the intent of the next round of dialogue, and use the identified intent and the predicted intent as target intent information.

[0084] For example, if the historical dialogue data includes three rounds of dialogue, then the purpose of step S202 is to identify the intention of the first round of dialogue, the intention of the second round of dialogue, and the intention of the third round of dialogue, and to predict the intention of the fourth round of dialogue. For example, the intention of the dialogue can be time, place, conceptual interpretation, etc.

[0085] Intent identification is performed on each round of dialogue contained in the historical dialogue data to obtain the intent of each round of dialogue contained in the historical dialogue data. In the process of identifying each round of dialogue contained in the historical dialogue data, dialogue information of each round of dialogue contained in the historical dialogue data can be obtained, and the intent of the next round of dialogue can be predicted based on the dialogue information of each round of dialogue contained in the historical dialogue data.

[0086] Step S203: Based on historical dialogue data, reference documents, and target intent information, generate the answer for the next round of dialogue.

[0087] After obtaining the target intent information through step S202, the answer to the next round of dialogue can be predicted based on historical dialogue data and reference documents, combined with the target intent information. Specifically, candidate answers can first be extracted from the reference documents based on historical dialogue data, reference documents, and target intent information to obtain several candidate answers, and then the answer to the next round of dialogue can be selected from these candidate answers.

[0088] Step S204: Based on historical dialogue data, reference documents, and the answers to the next round of dialogue, generate questions for the next round of dialogue.

[0089] After obtaining the answer to the next round of dialogue through step S203, the question for the next round of dialogue can be predicted based on historical dialogue data and reference documents, combined with the answer to the next round of dialogue. Thus, the question-and-answer pair for the next round of dialogue is obtained.

[0090] The dialogue question generation method provided in this invention first acquires a reference document in a specified scenario and historical dialogue data of dialogues unfolding around the reference document. Then, it identifies the intent of each round of dialogue contained in the historical dialogue data and predicts the intent of the next round of dialogue. Next, based on the historical dialogue data, the reference document, and the identified and predicted intent, it generates the answer to the next round of dialogue. Finally, based on the historical dialogue data, the reference document, and the answer to the next round of dialogue, it generates the question for the next round of dialogue. This method can automatically predict the answer to the next round of dialogue using the intent of each round of dialogue contained in the historical dialogue data and the intent of the next round of dialogue. Furthermore, it can automatically predict the question for the next round of dialogue using the predicted answer, thereby obtaining a question-and-answer pair for the next round of dialogue. Compared to manual dialogue question generation schemes, the method provided in this invention has higher efficiency and lower cost, and the question-and-answer pair generation effect is better.

[0091] Furthermore, compared to question generation methods with answers, the dialogue question generation method provided in this embodiment of the invention has lower cost, easier implementation, and greater usability. Compared to rule-based dialogue question generation methods, the dialogue question generation method provided in this embodiment of the invention has stronger generalization ability and more stable question-answer pair generation results.

[0092] Second Embodiment

[0093] This embodiment describes the specific implementation process of "step S202: identify the intent of each round of dialogue contained in the historical dialogue data, predict the intent of the next round of dialogue, and use the identified intent and the predicted intent as target intent information" in the above embodiment.

[0094] The above embodiments mention that intent identification can be performed on each round of dialogue contained in the historical dialogue data to obtain the intent of each round of dialogue contained in the historical dialogue data, and the intent of the next round of dialogue can be predicted based on the dialogue information of each round of dialogue obtained in the process of intent identification on each round of dialogue contained in the historical dialogue data.

[0095] Optionally, the above intent recognition and prediction process can be implemented based on a model. Of course, this embodiment is not limited to this, that is, this embodiment does not limit the specific implementation form of the above intent recognition and prediction process. Next, taking the implementation of the intent recognition and prediction process based on a model as an example, the specific implementation process of step S202 will be introduced.

[0096] Please see Figure 3 This illustrates a flowchart illustrating how an intent recognition model identifies the intent of each round of dialogue contained in historical dialogue data and predicts the intent of the next round of dialogue. The flowchart may include:

[0097] Step S301: Based on each round of dialogue contained in the historical dialogue data, use the pre-trained intent recognition model to identify the intent of each round of dialogue contained in the historical dialogue data.

[0098] Specifically, the historical dialogue data can first be segmented to separate each round of dialogue, and then separators can be inserted between adjacent dialogues. <sep>(For example, for historical dialogue data containing three rounds of dialogue, insert a separator between the first and second rounds of dialogue.) <sep>Insert a separator between the second and third rounds of dialogue. <sep>Then, the historical dialogue data with inserted separators is input into the pre-trained intent recognition model for intent recognition, so as to obtain the intent recognized by the intent recognition model for each round of dialogue input.

[0099] Step S302: Based on the vector representation of the last round of dialogue data contained in the historical dialogue data, use the intent recognition model to predict the intent of the next round of dialogue.

[0100] The vector representation of the last round of dialogue data is a vector containing dialogue information from each round of dialogue, determined by the intent recognition model during the intent recognition process. The intent recognition model predicts the intent of the next round of dialogue based on the vector representation of the last round of dialogue data.

[0101] The intent recognition model in this embodiment is trained using training dialogue data annotated with real intent information. This real intent information includes the real intent of each round of dialogue and the real intent of the next round of dialogue contained in the training dialogue data. For example, the intent recognition model in this embodiment can be, but is not limited to, the sequence labeling model BERT-CRF.

[0102] Specifically, the training process for an intent recognition model may include:

[0103] Step a1: Obtain training dialogue data labeled with real intent information from the first training set.

[0104] The first training set includes multiple training dialogue data labeled with real intent information. Each training dialogue data labeled with real intent information in the first training set can be constructed based on the dialogue data in the corpus.

[0105] It should be noted that the corpus includes multiple question-and-answer dialogues labeled with intent information for a specified scenario, as well as reference documents corresponding to each question-and-answer dialogue. Each question-and-answer dialogue includes multiple rounds of dialogue, each round of dialogue includes a question and an answer, and each round of dialogue is labeled with the intent of that round of dialogue.

[0106] For a question-and-answer dialogue corpus consisting of N rounds of dialogue, one or more training dialogue data labeled with real intent information can be constructed according to the following scheme: take the m rounds of dialogue data before the nth round of dialogue (m is an integer greater than or equal to 1 and less than or equal to n-1) as the constructed training dialogue data, and take the intent labeled in each round of dialogue data before the nth round of dialogue (n is an integer greater than 1 and less than or equal to N) and the intent labeled in the nth round of dialogue as the annotation information of the constructed training dialogue data.

[0107] For example, for a question-and-answer dialogue corpus consisting of four rounds of dialogue, the first round of dialogue can be used as training dialogue data 1, and the intentions labeled in the first round of dialogue and the second round of dialogue can be used as annotation information for training dialogue data 1. Alternatively, the first and second rounds of dialogue can be used as training dialogue data 2, and the intentions labeled in the first, second, and third rounds of dialogue can be used as annotation information for training dialogue data 2. Finally, the first through third rounds of dialogue can be used as training dialogue data 3, and the intentions labeled in the first, second, and third rounds of dialogue can be used as annotation information for training dialogue data 2. The intentions of the four rounds of dialogue are used as the annotation information for training dialogue data 3. In addition, the second round of dialogue can be used as the constructed training dialogue data 4, and the intentions of the second round of dialogue and the third round of dialogue can be used as the annotation information for training dialogue data 4. The third round of dialogue can also be used as the constructed training dialogue data 5, and the intentions of the third round of dialogue and the fourth round of dialogue can be used as the annotation information for training dialogue data 5. The second and third rounds of dialogue can also be used as the constructed training dialogue data 6, and the intentions of the second round of dialogue, the third round of dialogue, and the fourth round of dialogue can be used as the annotation information for training dialogue data 6.

[0108] For multiple question-and-answer dialogues in the corpus that are annotated with intent information, a large amount of training dialogue data with annotated intent information can be constructed in the manner described above.

[0109] Step a2: Input the acquired training dialogue data into the intent recognition model to obtain the intent recognition and prediction results output by the intent recognition model.

[0110] After the training dialogue data is input into the intent recognition model, the model identifies the intent of each round of dialogue contained in the training dialogue data and predicts the intent of the next round of dialogue, thus outputting the intent recognition and prediction results. It should be noted that when inputting the training dialogue data into the intent recognition model, the training dialogue data also needs to be segmented to insert separators between adjacent rounds of dialogue. <sep>Then, the training dialogue data with inserted separators is input into the intent recognition model.

[0111] It should be noted that the intent recognition model has two tasks: first, to label each round of dialogue with an intent label; and second, to predict the intent of the next round of dialogue using the vector representation of the last round of dialogue. Through multi-task learning, it can make full use of the contextual relationships between each round of dialogue and better capture the potential succession relationships of the intents of each round of dialogue.

[0112] Step a3: Based on the intent recognition and prediction results output by the intent recognition model and the real intent information labeled in the input training dialogue data, determine the prediction loss of the intent recognition model.

[0113] Optionally, the cross-entropy loss of the intent recognition model can be determined based on the intent recognition and prediction results output by the intent recognition model and the real intent information labeled in the input training dialogue data. The method for determining the cross-entropy loss is existing technology and will not be described in detail in this embodiment.

[0114] Step a4: Update the parameters of the intent recognition model based on the prediction loss of the intent recognition model.

[0115] The intent recognition model is trained iteratively multiple times according to steps a1 to a4 above until the training termination condition is met (e.g., the model converges, or the preset number of iterations is reached).

[0116] After training, historical dialogue data can be input into the trained intent recognition model. The intent recognition model identifies the intent of each round of dialogue contained in the historical dialogue data and predicts the intent of the next round of dialogue, outputting intent recognition and prediction results based on the historical dialogue data.

[0117] Third Embodiment

[0118] This embodiment describes the specific implementation process of "step S203: generating the answer to the next round of dialogue based on historical dialogue data, reference documents, and target intent information" in the above embodiment.

[0119] Please see Figure 4 This diagram illustrates the process of generating answers for the next round of dialogue based on historical dialogue data, reference documents, and target intent information, and may include:

[0120] Step S401: Based on historical dialogue data, reference documents, and target intent information, extract candidate answers from the reference documents to obtain a first candidate answer set composed of the extracted candidate answers.

[0121] Specifically, the process of extracting candidate answers from reference documents based on historical dialogue data, reference documents, and target intent information can include:

[0122] Step S4011: Based on historical dialogue data, reference documents, and target intent information, predict the first probability and second probability corresponding to each word in the reference document.

[0123] The first probability is the probability that the corresponding word is the starting word of the answer, and the second probability is the probability that the corresponding word is the ending word of the answer.

[0124] Step S4012: Based on the first probability and the second probability corresponding to each word in the reference document, determine the starting word and the ending word of the answer from the reference document.

[0125] After predicting the first and second probabilities for each word in the reference document, the starting and ending words of the answer can be determined from the reference document based on the predicted probabilities.

[0126] Step S4013: Extract candidate answers from the reference document based on the answer start word and answer end word determined from the reference document.

[0127] It should be noted that each extracted candidate answer is a text segment from the start word of an answer to the end word of an answer. That is, a candidate answer consists of an answer start word, the answer end word adjacent to the answer start word, and the part in between.

[0128] For each extracted candidate answer, its predicted probability can be further determined. This predicted probability can be based on the first probability corresponding to the candidate answer's start word and the second probability corresponding to its end word. For example, the first probability of the candidate answer's start word and the second probability corresponding to its end word can be summed, and the sum becomes the predicted probability. Alternatively, the first probability of the candidate answer's start word and the second probability corresponding to its end word can be multiplied, and the result becomes the predicted probability. It should be noted that the predicted probability of a candidate answer is the probability that it will be the answer in the next round of dialogue.

[0129] Step S402: Based on the intent of the next round of dialogue in the target intent information, select the answer for the next round of dialogue from the first candidate answer set.

[0130] Specifically, the process of selecting answers for the next round of dialogue from the first set of candidate answers, based on the target intent information, can include:

[0131] Step S4021: Delete the candidate answers whose predicted probability is less than the preset probability threshold in the first candidate answer set to obtain the second candidate answer set.

[0132] The purpose of step S4021 is to remove candidate answers from the first candidate answer set that are unlikely to be answers in the next round of dialogue, based on a preset probability threshold.

[0133] Step S4022: Select candidate answers from the second candidate answer set that match the intent of the next round of dialogue, and use them as the answers for the next round of dialogue.

[0134] Specifically, the process of selecting candidate answers from the second set of candidate answers that match the intent of the next round of dialogue, and using them as answers for the next round of dialogue, includes:

[0135] Step b1: Identify the named entities contained in each candidate answer in the second candidate answer set.

[0136] Existing named entity recognition methods can be used to identify the named entities contained in each candidate answer in the second candidate answer set.

[0137] Step b2: Select candidate answers from the second candidate answer set whose named entities match the intent of the next round of dialogue, and use them as the first target answer.

[0138] For example, if the intent of the next round of dialogue is time, and a candidate answer in the second candidate answer set contains the named entity "year", then that candidate answer can be determined as the candidate answer that matches the intent of the next round of dialogue.

[0139] Step b3-a: If there is only one first target answer, then the first target answer is determined as a candidate answer that matches the intention of the next round of dialogue, and is used as the answer for the next round of dialogue.

[0140] Step b3-b: If there are multiple first target answers, the first target answer with the highest predicted probability among the multiple first target answers is determined as the candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue.

[0141] For example, if the intent of the next round of dialogue is time, and there are 3 candidate answers in the second candidate answer set whose named entities match the intent of time in the next round of dialogue, then the candidate answer with the highest predicted probability among these 3 candidate answers will be determined as the answer to the next round of dialogue.

[0142] In some cases, the second candidate answer set may not contain the first target answer (i.e., the second candidate answer set does not contain a candidate answer whose named entity matches the intent of the next round of dialogue). To address this situation, this embodiment proposes the following processing strategy:

[0143] Step b4: Based on the pre-built relevance information between intents and named entities, determine the named entities related to the intent of the next round of dialogue.

[0144] The relevance information between intents and named entities includes several intents and the named entities associated with each of the several intents.

[0145] Specifically, the process of constructing the relevance information between intent and named entities includes:

[0146] Step c1: Obtain multiple question-and-answer dialogues labeled with intent information from the corpus.

[0147] Each question-and-answer dialogue consists of multiple rounds of conversation, with each round marked with an intent.

[0148] Step c2: For each round of dialogue contained in each acquired question-and-answer dialogue corpus, perform named entity recognition on the answers in that round of dialogue, and use the recognized named entities as named entities related to the intent labeled in that round of dialogue.

[0149] Named entities related to several intentions can be obtained through step c2.

[0150] Step c3: Based on the intent of each round of dialogue annotation contained in each question-and-answer dialogue corpus, and the named entities related to the intent of each round of dialogue annotation contained in each question-and-answer dialogue corpus, construct the relevance information between intent and named entities.

[0151] For example, there are 3 question-and-answer dialogue corpora. The first question-and-answer dialogue corpus includes 3 rounds of dialogue, where the intent of the first round of dialogue is 'a', and the named entity identified from the answers of the first round of dialogue is 'e'. 1a The intent of the second round of dialogue is b, and the named entity identified from the answers in the second round of dialogue is e. 1b The intent of the third round of dialogue is c, and the named entity identified from the answers in the third round of dialogue is e. 1c The second question-and-answer dialogue corpus includes two rounds of dialogue. The intent of the first round of dialogue is b, and the named entity identified from the answers of the first round of dialogue is e. 2b The intent of the second round of dialogue is 'a', and the named entity identified from the answers of the first round of dialogue is 'e'. 2a The third question-and-answer dialogue corpus includes three rounds of dialogue. The intent of the first round of dialogue is b, and the named entity identified from the answers of the first round of dialogue is e. 3b The intent of the second round of dialogue is d, and the named entity identified from the answers in the second round of dialogue is e. 3d The intent of the third round of dialogue is 'a', and the named entity identified from the answers in the third round of dialogue is 'e'. 3a Statistical analysis shows that there are e named entities related to intent a. 1a e 2a e 3a The named entities associated with intent b are e 1b e 2b e 3b The named entities related to intent c are e 1c The named entities associated with intent d are e 3d .

[0152] For example, the relevance information between intents and named entities is shown below:

[0153] meaning Figure 1 <![CDATA[e 11 ,and 12 …]]> meaning Figure 2 <![CDATA[e 21 、…]]> meaning Figure 3 <![CDATA[e 31 、…]]> … …

[0154] Optionally, when there are many named entities related to a certain intent, named entities with low relevance to that intent can be removed (for example, in multiple dialogue corpora, there are 100 rounds of dialogue labeled with intent a, and named entity e is only identified from the answer of one round of dialogue, indicating that named entity e has low relevance to intent a). Of course, all named entities related to each intent can also be retained.

[0155] Step b5: Based on named entities related to the intent of the next round of dialogue, select candidate answers from the second candidate answer set that match the intent of the next round of dialogue, and use them as answers for the next round of dialogue.

[0156] Specifically, from the second candidate answers, candidate answers containing named entities related to the intent of the next round of dialogue can be selected as the second target answer. If there is only one second target answer, then that second target answer is determined as the candidate answer that matches the intent of the next round of dialogue and is used as the answer for the next round of dialogue. If there are multiple second target answers, then the second target answer with the highest predicted probability among the multiple second target answers is determined as the candidate answer that matches the intent of the next round of dialogue and is used as the answer for the next round of dialogue.

[0157] In one possible implementation, the answer to the next round of dialogue can be generated based on historical dialogue data, reference documents, target intent information, and a pre-trained answer extraction model. Specifically, firstly, based on historical dialogue data, reference documents, and target intent information, the pre-trained answer extraction model extracts candidate answers from the reference documents to obtain a first candidate answer set. Then, based on the intent of the next round of dialogue, the answer to the next round of dialogue is selected from the first candidate answer set.

[0158] Specifically, when extracting candidate answers from a reference document using a pre-trained answer extraction model based on historical dialogue data, reference documents, and target intent information, the historical dialogue data, reference documents, and target intent information are input into the answer extraction model. The answer extraction model outputs a first probability and a second probability for each word in the reference document, and then extracts candidate answers from the reference document based on these first and second probabilities. It should be noted that when inputting the historical dialogue data, reference documents, and target intent information into the answer extraction model, these three parts of the input data are separated by a delimiter. <sep>Separate.

[0159] In this embodiment, the answer extraction model is trained using training samples labeled with the real answers to the next round of dialogue. These training samples include training dialogue data, reference documents corresponding to the training dialogue data, and intent information identified and predicted based on an intent recognition model. For example, the answer extraction model can be, but is not limited to, a BERT-based answer extraction model.

[0160] Specifically, the training process for the answer extraction model can include:

[0161] Step d1: Obtain training samples labeled with the real answers to the next round of dialogue from the second training set.

[0162] The second training set includes multiple training samples labeled with real answers to the next round of dialogue. As mentioned above, the training samples include training dialogue data, reference documents corresponding to the training dialogue data, and intent information that is used to identify and predict the training dialogue data based on the intent recognition model.

[0163] The training samples here are also constructed based on the aforementioned corpus. For example, a question-and-answer dialogue in the corpus includes three rounds of dialogue. The first round of dialogue can be used as training dialogue data 1, and the reference document corresponding to the question-and-answer corpus can be used as the reference document corresponding to training dialogue data 1. Based on the intent recognition model, the intent of training dialogue data 1 is recognized and predicted. In this way, training sample 1 can be obtained, which includes training dialogue data 1, the reference document corresponding to training dialogue data 1, and the intent information of training dialogue data 1. The answer in the second round of dialogue in the corpus can be used as the annotation information of training sample 1. The first round of dialogue and the second round of dialogue can be used as training dialogue data 2, and the reference document corresponding to the question-and-answer dialogue corpus can be used as the reference document corresponding to training dialogue data 2. Based on the intent recognition model, the intent of training dialogue data 2 is recognized and predicted. In this way, training sample 2 can be obtained, which includes training dialogue data 2, the reference document corresponding to training dialogue data 2, and the intent information of training dialogue data 2. The answer in the third round of dialogue in the corpus can be used as the annotation information of training sample 2.

[0164] Step d2: Input the obtained training samples into the answer extraction model to obtain the first and second probabilities of each word in the reference document corresponding to the training dialogue data contained in the training samples.

[0165] The first probability is the probability that the corresponding word in the reference document is the starting word of the answer, and the second probability is the probability that the corresponding word is the ending word of the answer.

[0166] Step d3: Based on the first and second probabilities of each word in the reference document corresponding to the training dialogue data contained in the training samples, extract candidate answers from the reference document corresponding to the training dialogue data contained in the training samples to obtain a candidate answer set composed of the extracted candidate answers.

[0167] Specifically, based on the first and second probabilities of each word in the reference document corresponding to the training dialogue data included in the training samples, the start word and end word of the answer are determined from the reference document corresponding to the training dialogue data included in the training samples. Based on the determined start word and end word of the answer, candidate answers are extracted from the reference document corresponding to the training dialogue data included in the training samples.

[0168] Step d4: Based on the intent information of the training dialogue data contained in the training samples, select the answer for the next round of dialogue from the candidate answer set.

[0169] The specific implementation process of step d4 is similar to that of step S402 above, and will not be described in detail here.

[0170] Step d5: Based on the answers to the next round of dialogue selected from the candidate answer set and the true answers to the next round of dialogue labeled in the training samples, determine the prediction loss of the answer extraction model.

[0171] Optionally, the cross-entropy loss of the answer extraction model is determined based on the answers to the next round of dialogue selected from the candidate answer set and the real answers to the next round of dialogue labeled in the training samples. The method for determining the cross-entropy loss is existing technology and will not be described in detail in this embodiment.

[0172] Step d6: Update the parameters of the answer extraction model based on the prediction loss of the answer extraction model.

[0173] The answer extraction model is trained iteratively multiple times according to steps d1 to d6 above until the training termination condition is met (e.g., the model converges, or the preset number of iterations is reached).

[0174] Fourth embodiment

[0175] This embodiment describes the specific implementation process of "step S204: generating questions for the next round of dialogue based on historical dialogue data, reference documents, and the answers to the next round of dialogue" in the above embodiment.

[0176] In one possible implementation, questions for the next round of dialogue can be generated using a pre-trained question generation model, based on historical dialogue data, reference documents, and answers to the next round of dialogue.

[0177] Specifically, historical dialogue data, reference documents, and the answer to the next round of dialogue are input into a pre-trained question generation model to obtain the question for the next round of dialogue. It should be noted that when inputting the historical dialogue data, reference documents, and the answer to the next round of dialogue into the question generation model, these three parts of the input data are separated by a delimiter. <sep>Separate.

[0178] The question generation model is trained using training samples labeled with real questions for the next round of dialogue. These training samples include training dialogue data, corresponding reference documents, and answers for the next round of dialogue determined by the answer extraction model based on the training dialogue data. For example, the question generation model in this embodiment may, but is not limited to, using the T5 generation model.

[0179] Specifically, the training process of the problem generation model includes:

[0180] Step e1: Obtain training samples labeled with real questions for the next round of dialogue from the third training set.

[0181] The third training set includes multiple training samples labeled with real questions for the next round of dialogue. These training samples are also constructed based on the aforementioned corpus. For example, a question-and-answer dialogue in the corpus includes three rounds of dialogue. The first round of dialogue can be used as training dialogue data 1, and the reference document corresponding to that question-and-answer dialogue can be used as the reference document corresponding to training dialogue data 1. Based on the answer extraction model, the answer for the next round of dialogue is determined for training dialogue data 1. This yields a training set including training dialogue data 1, the reference document corresponding to training dialogue data 1, and the answer for the next round of dialogue for training dialogue data 1. Training sample 1 uses the question from the second round of dialogue in the corpus as its annotation information. The first and second rounds of dialogue can be used as training dialogue data 2. The reference document corresponding to the question-and-answer dialogue corpus is used as the reference document corresponding to training dialogue data 2. Based on the answer extraction model, the answer to the next round of dialogue is determined for training dialogue data 2. In this way, training sample 2 can be obtained, which includes training dialogue data 2, the reference document corresponding to training dialogue data 2, and the answer to the next round of dialogue of training dialogue data 2. The question from the third round of dialogue in the question-and-answer dialogue corpus is used as the annotation information for training sample 2.

[0182] It should be noted that the training dialogue data in the first, second, and third training sets can be the same or different. If the training dialogue data in the three training sets are the same, after constructing a training dialogue data, the reference document corresponding to the training dialogue data, the real answer of the next round of dialogue of the training dialogue data, and the real question of the next round of dialogue of the training dialogue can be further constructed.

[0183] Step e2: Input the acquired training samples into the question generation model to obtain the question for the next round of dialogue generated by the question generation model.

[0184] The training samples consist of three parts separated by a delimiter. <sep>The input is separated into two parts and fed into the question generation model. The question generation model then outputs its predicted question for the next round of dialogue.

[0185] Step e3: Determine the prediction loss of the question generation model based on the question generated by the question generation model for the next round of dialogue and the question for the next round of dialogue labeled in the training samples.

[0186] Optionally, the cross-entropy loss of the question generation model can be determined based on the question of the next round of dialogue generated by the question generation model and the question of the next round of dialogue labeled by the training samples. The method for determining the cross-entropy loss is existing technology and will not be described in detail in this embodiment.

[0187] Step e4: Update the parameters of the problem generation model based on the prediction loss of the problem generation model.

[0188] The problem generation model is trained iteratively multiple times according to steps e1 to e4 above until the training termination condition is met (e.g., the model converges, or the preset number of iterations is reached).

[0189] Fifth embodiment

[0190] This invention also provides a dialogue question generation device. The dialogue question generation device provided in this invention will be described below. The dialogue question generation device described below can be referred to in correspondence with the dialogue question generation method described above.

[0191] Please see Figure 5 The diagram shows a schematic of the structure of a dialogue question generation device provided in an embodiment of the present invention. The dialogue question generation device may include: a data acquisition module 501, an intent recognition and prediction module 502, an answer generation module 503, and a question generation module 504.

[0192] The data acquisition module 501 is used to acquire reference documents in a specified scenario, as well as historical dialogue data of dialogues initiated in relation to the reference documents.

[0193] The intent recognition and prediction module 502 is used to recognize the intent of each round of dialogue contained in the historical dialogue data, predict the intent of the next round of dialogue, and use the recognized intent and the predicted intent as target intent information.

[0194] The answer generation module 503 is used to generate an answer for the next round of dialogue based on the historical dialogue data, the reference document, and the target intent information.

[0195] The question generation module 504 is used to generate questions for the next round of dialogue based on the historical dialogue data, the reference document, and the answer to the next round of dialogue.

[0196] Optionally, the answer generation module 503 includes: an answer extraction module and an answer filtering module.

[0197] The answer extraction module is used to extract candidate answers from the reference document based on the historical dialogue data, the reference document, and the target intent information, so as to obtain a first candidate answer set composed of the extracted candidate answers.

[0198] The answer filtering module is used to filter out answers for the next round of dialogue from the first candidate answer set based on the intent of the next round of dialogue in the target intent information.

[0199] Optionally, each candidate answer in the first candidate answer set has a predicted probability, which is the probability that the corresponding candidate answer is the answer for the next round of dialogue;

[0200] When the answer filtering module filters answers for the next round of dialogue from the first candidate answer set based on the target intent information regarding the intent of the next round of dialogue, it is specifically used for:

[0201] Delete the candidate answers whose predicted probability is less than a preset probability threshold from the first candidate answer set to obtain the second candidate answer set;

[0202] Select candidate answers from the second set of candidate answers that match the intent of the next round of dialogue, and use them as the answers for the next round of dialogue.

[0203] Optionally, when the answer filtering module filters candidate answers from the second candidate answer set that match the intent of the next round of dialogue as answers for the next round of dialogue, it is specifically used for:

[0204] Identify the named entities contained in each candidate answer in the second candidate answer set;

[0205] The candidate answers that match the named entities contained in the second candidate answer set with the intent of the next round of dialogue are selected as the first target answer;

[0206] If there is only one first target answer, then that first target answer is determined as a candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue;

[0207] If there are multiple first target answers, the first target answer with the highest predicted probability among the multiple first target answers is determined as the candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue.

[0208] Optionally, when the answer filtering module filters candidate answers from the second candidate answer set that match the intent of the next round of dialogue as answers for the next round of dialogue, it is also used to:

[0209] If the second candidate answer set does not contain a candidate answer whose included named entity matches the intent of the next round of dialogue, then based on the pre-constructed relevance information between intent and named entity, the named entity related to the intent of the next round of dialogue is determined, wherein the relevance information between intent and named entity includes several intents and the named entity related to each of the several intents.

[0210] Based on named entities related to the intent of the next round of dialogue, candidate answers that match the intent of the next round of dialogue are selected from the second candidate answer set and used as the answers for the next round of dialogue.

[0211] Optionally, the dialogue question generation apparatus provided in this embodiment of the invention may further include: a relevance information construction module. The relevance information construction module is used to construct relevance information between intents and named entities.

[0212] The relevance information building module is specifically used to construct relevance information between intents and named entities when:

[0213] Acquire multiple dialogue corpora labeled with intent information, where each dialogue corpus includes multiple rounds of dialogue, and each round of dialogue is labeled with intent;

[0214] For each dialogue corpus containing each round of dialogue, named entity recognition is performed on the answers in that round of dialogue, and the identified named entities are used as named entities related to the intent labeled in that round of dialogue;

[0215] Based on the intent of each dialogue annotation contained in each dialogue corpus, and the named entities related to the intent of each dialogue annotation contained in each dialogue corpus, the relevance information between the intent and the named entities is constructed.

[0216] Optionally, when the answer filtering module filters candidate answers that match the intent of the next round of dialogue from the second candidate answer set based on named entities related to the intent of the next round of dialogue, and uses them as answers for the next round of dialogue, it is specifically used for:

[0217] From the second candidate answers, select candidate answers that contain named entities related to the intent of the next round of dialogue, and use them as the second target answer;

[0218] If there is only one second target answer, then that second target answer is determined as a candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue;

[0219] If there are multiple second target answers, the second target answer with the highest predicted probability among the multiple second target answers is determined as the candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue.

[0220] Optionally, when identifying the intent of each round of dialogue contained in the historical dialogue data and predicting the intent of the next round of dialogue, the intent recognition and prediction module 502 is specifically used for:

[0221] Based on each round of dialogue contained in the historical dialogue data, the intent of each round of dialogue contained in the historical dialogue data is identified using a pre-trained intent recognition model.

[0222] Based on the vector representation of the last round of dialogue contained in the historical dialogue data, the intent of the next round of dialogue is predicted using the intent recognition model, wherein the vector representation of the last round of dialogue is a vector containing dialogue information of each round of dialogue determined by the intent recognition model during the intent recognition process.

[0223] The intent recognition model is trained using training dialogue data labeled with real intent information. The real intent information labeled in the training dialogue data includes the real intent of each round of dialogue and the real intent of the next round of dialogue contained in the training dialogue data.

[0224] Optionally, when generating the answer generation module 503 generates the answer for the next round of dialogue based on the historical dialogue data, the reference document, and the target intent information, it is specifically used for:

[0225] Based on the historical dialogue data, the reference document, the target intent information, and the pre-trained answer extraction model, the answer to the next round of dialogue is generated.

[0226] The answer extraction model is trained using training samples labeled with real answers to the next round of dialogue. The training samples labeled with real answers to the next round of dialogue include training dialogue data, reference documents corresponding to the training dialogue data, and intent information that is identified and predicted based on the intent recognition model.

[0227] Optionally, when generating questions for the next round of dialogue based on the historical dialogue data, the reference document, and the answer to the next round of dialogue, the question generation module 504 is specifically used for:

[0228] Based on the historical dialogue data, the reference document, and the answer to the next round of dialogue, a question for the next round of dialogue is generated using a pre-trained question generation model.

[0229] The question generation model is trained using training samples labeled with real questions for the next round of dialogue. These training samples include training dialogue data, reference documents corresponding to the training dialogue data, and answers for the next round of dialogue determined by the answer extraction model based on the training dialogue data.

[0230] The dialogue question generation device provided in this embodiment of the invention first acquires a reference document in a specified scenario and historical dialogue data of dialogues unfolding around the reference document. Then, it identifies the intent of each round of dialogue contained in the historical dialogue data and predicts the intent of the next round of dialogue. Next, based on the historical dialogue data, the reference document, and the identified and predicted intent, it generates the answer to the next round of dialogue. Finally, based on the historical dialogue data, the reference document, and the answer to the next round of dialogue, it generates the question for the next round of dialogue. The dialogue question generation device provided in this embodiment of the invention can automatically predict the answer to the next round of dialogue using the intent of each round of dialogue contained in the historical dialogue data and the intent of the next round of dialogue. Furthermore, it can automatically predict the question for the next round of dialogue using the predicted answer, thereby obtaining a question-and-answer pair for the next round of dialogue. The dialogue question generation device provided in this embodiment of the invention has high efficiency and low cost, and the question-and-answer pair generation effect is good. In addition, the dialogue question generation device provided in this embodiment of the invention has strong generalization ability and stable question-and-answer pair generation effect.

[0231] Sixth Embodiment

[0232] This invention also provides a dialogue question generation device; please refer to [link to relevant documentation]. Figure 6 The diagram shows the structure of the dialogue question generation device, which may include: a processor 601, a communication interface 602, a memory 603, and a communication bus 604.

[0233] In this embodiment of the invention, the number of processor 601, communication interface 602, memory 603 and communication bus 606 is at least one, and processor 601, communication interface 602 and memory 603 communicate with each other through communication bus 604.

[0234] The processor 601 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0235] The memory 603 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;

[0236] The memory stores a program, which the processor can call. The program is used for:

[0237] Obtain reference documents in a specified scenario, as well as historical dialogue data of conversations initiated in response to the reference documents;

[0238] Identify the intent of each round of dialogue contained in the historical dialogue data, predict the intent of the next round of dialogue, and use the identified intent and the predicted intent as target intent information.

[0239] Based on the historical dialogue data, the reference document, and the target intent information, generate the answer for the next round of dialogue;

[0240] Based on the historical dialogue data, the reference document, and the answer to the next round of dialogue, generate the question for the next round of dialogue.

[0241] Optionally, the refined and extended functions of the program can be found in the description above.

[0242] Seventh Embodiment

[0243] This invention also provides a readable storage medium that stores a program suitable for execution by a processor, the program being used for:

[0244] Obtain reference documents in a specified scenario, as well as historical dialogue data of conversations initiated in response to the reference documents;

[0245] Identify the intent of each round of dialogue contained in the historical dialogue data, predict the intent of the next round of dialogue, and use the identified intent and the predicted intent as target intent information.

[0246] Based on the historical dialogue data, the reference document, and the target intent information, generate the answer for the next round of dialogue;

[0247] Based on the historical dialogue data, the reference document, and the answer to the next round of dialogue, generate the question for the next round of dialogue.

[0248] Optionally, the refined and extended functions of the program can be found in the description above.

[0249] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0250] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0251] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.< / sep> < / sep> < / sep> < / sep> < / sep> < / sep> < / sep>

Claims

1. A method for generating dialogue questions, characterized in that, include: Obtain reference documents in a specified scenario, as well as historical dialogue data of conversations initiated in response to the reference documents; Separators are inserted between adjacent dialogues in the historical dialogue data. The historical dialogue data with the inserted separators is then input into a pre-trained intent recognition model. The intent recognition model performs two tasks: first, it labels each round of dialogue with an intent to identify the intent of each round of dialogue contained in the historical dialogue data; second, it predicts the intent of the next round of dialogue using the vector representation of the last round of dialogue. The identified intent and the predicted intent are used as target intent information; The historical dialogue data, the reference document, and the target intent information are input into the answer extraction model to generate the answer for the next round of dialogue. The three parts of input data are separated by a delimiter. The answer for the next round of dialogue is selected from the candidate answer set. The candidate answer generation process is as follows: the answer extraction model outputs the first probability and the second probability corresponding to each word in the reference document. The first probability is the probability that the corresponding word is the starting word of the answer, and the second probability is the probability that the corresponding word is the ending word of the answer. Based on the first probability and the second probability corresponding to each word in the reference document, the starting word and the ending word of the answer are determined from the reference document. Based on the starting word and the ending word of the answer determined from the reference document, candidate answers are extracted from the reference document. Based on the historical dialogue data, the reference document, and the answer to the next round of dialogue, generate the question for the next round of dialogue.

2. The dialogue question generation method according to claim 1, characterized in that, The process of inputting the historical dialogue data, the reference document, and the target intent information into the answer extraction model to generate the answer for the next round of dialogue includes: Based on the historical dialogue data, the reference document, and the target intent information, a candidate answer is extracted from the reference document using an answer extraction model to obtain a first candidate answer set composed of the extracted candidate answers; Based on the intent of the next round of dialogue in the target intent information, the answer to the next round of dialogue is selected from the first candidate answer set.

3. The dialogue question generation method according to claim 2, characterized in that, Each candidate answer in the first candidate answer set has a predicted probability, which is the probability that the corresponding candidate answer is the answer for the next round of dialogue; The step of selecting answers for the next round of dialogue from the first candidate answer set based on the target intent information includes: Delete the candidate answers whose predicted probability is less than a preset probability threshold from the first candidate answer set to obtain the second candidate answer set; Select candidate answers from the second set of candidate answers that match the intent of the next round of dialogue, and use them as the answers for the next round of dialogue.

4. The dialogue question generation method according to claim 3, characterized in that, The step of selecting candidate answers from the second candidate answer set that match the intent of the next round of dialogue as the answers for the next round of dialogue includes: Identify the named entities contained in each candidate answer in the second candidate answer set; The candidate answers that match the named entities contained in the second candidate answer set with the intent of the next round of dialogue are selected as the first target answer; If there is only one first target answer, then that first target answer is determined as a candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue; if there are multiple first target answers, then the first target answer with the highest predicted probability among the multiple first target answers is determined as a candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue.

5. The dialogue question generation method according to claim 3, characterized in that, The step of selecting candidate answers from the second candidate answer set that match the intent of the next round of dialogue as answers for the next round of dialogue also includes: If there is no candidate answer in the second candidate answer set whose included named entity matches the intent of the next round of dialogue, then based on the pre-constructed relevance information between intent and named entity, the named entity related to the intent of the next round of dialogue is determined, wherein the relevance information between intent and named entity includes several intents and the named entity related to each of the several intents. Based on named entities related to the intent of the next round of dialogue, candidate answers that match the intent of the next round of dialogue are selected from the second candidate answer set and used as the answers for the next round of dialogue.

6. The dialogue question generation method according to claim 5, characterized in that, The process of constructing the relevance information between the intent and the named entity includes: Acquire multiple dialogue corpora labeled with intent information, where each dialogue corpus includes multiple rounds of dialogue, and each round of dialogue is labeled with intent; For each dialogue corpus containing each round of dialogue, named entity recognition is performed on the answers in that round of dialogue, and the identified named entities are used as named entities related to the intent labeled in that round of dialogue; Based on the intent of each dialogue annotation contained in each dialogue corpus, and the named entities related to the intent of each dialogue annotation contained in each dialogue corpus, the relevance information between the intent and the named entities is constructed.

7. The dialogue question generation method according to claim 5, characterized in that, The step of filtering candidate answers from the second candidate answer set that match the intent of the next round of dialogue, based on named entities related to the intent of the next round of dialogue, as answers for the next round of dialogue, includes: From the second set of candidate answers, select candidate answers that contain named entities related to the intent of the next round of dialogue, and use them as the second target answer; If there is only one second target answer, then that second target answer is determined as a candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue; If there are multiple second target answers, the second target answer with the highest predicted probability among the multiple second target answers is determined as the candidate answer that matches the intent of the next round of dialogue, and is used as the answer for the next round of dialogue.

8. The dialogue question generation method according to any one of claims 1 to 7, characterized in that, The vector representation of the last round of dialogue is a vector containing dialogue information from each round of dialogue, determined by the intent recognition model during the intent recognition process. The intent recognition model is trained using training dialogue data labeled with real intent information. The real intent information labeled in the training dialogue data includes the real intent of each round of dialogue contained in the training dialogue data and the real intent of the next round of dialogue.

9. The dialogue question generation method according to claim 8, characterized in that, The answer extraction model is trained using training samples labeled with real answers to the next round of dialogue. The training samples labeled with real answers to the next round of dialogue include training dialogue data, reference documents corresponding to the training dialogue data, and intent information that is identified and predicted based on the intent recognition model.

10. The dialogue question generation method according to claim 9, characterized in that, The process of generating questions for the next round of dialogue based on the historical dialogue data, the reference document, and the answers to the next round of dialogue includes: Based on the historical dialogue data, the reference document, and the answer to the next round of dialogue, a question for the next round of dialogue is generated using a pre-trained question generation model. The question generation model is trained using training samples labeled with real questions for the next round of dialogue. These training samples include training dialogue data, reference documents corresponding to the training dialogue data, and answers for the next round of dialogue determined by the answer extraction model based on the training dialogue data.

11. A dialogue question generation device, characterized in that, include: Data acquisition module, intent recognition and prediction module, answer generation module, and question generation module The data acquisition module is used to acquire reference documents in a specified scenario, as well as historical dialogue data of dialogues initiated in response to the reference documents. The intent recognition and prediction module is used to insert separators between adjacent dialogues contained in the historical dialogue data, and input the historical dialogue data with inserted separators into a pre-trained intent recognition model. The intent recognition model performs two tasks: first, it labels each round of dialogue with intent to identify the intent of each round of dialogue contained in the historical dialogue data; second, it predicts the intent of the next round of dialogue through the vector representation of the last round of dialogue. The identified intent and the predicted intent are used as target intent information; The answer generation module is used to input the historical dialogue data, the reference document, and the target intent information into the answer extraction model to generate the answer for the next round of dialogue. The three parts of input data—the historical dialogue data, the reference document, and the target intent information—are separated by a delimiter. The answer for the next round of dialogue is selected from the candidate answer set. The candidate answer generation process is as follows: the answer extraction model outputs a first probability and a second probability for each word in the reference document. The first probability is the probability that the corresponding word is the starting word of the answer, and the second probability is the probability that the corresponding word is the ending word of the answer. Based on the first probability and the second probability for each word in the reference document, the starting word and the ending word of the answer are determined from the reference document. Based on the starting word and the ending word of the answer determined from the reference document, candidate answers are extracted from the reference document. The question generation module is used to generate questions for the next round of dialogue based on the historical dialogue data, the reference document, and the answer to the next round of dialogue.

12. A dialogue question generation device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement each step of the dialogue question generation method as described in any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements each step of the dialogue question generation method as described in any one of claims 1 to 10.

Images