A method, device, equipment, medium and product for training a question and answer model

Abstract

The application provides a training method and device of a question and answer model, equipment, a medium and a product. The method comprises the following steps: obtaining a first dialogue record of a target knowledge field; wherein the first dialogue record comprises a plurality of rounds of dialogue between a user role and a customer service role for a target question of the target knowledge field, and a target reply of the customer service role to the target question in the first dialogue record comprises target knowledge information of the target knowledge field; based on the first dialogue record, an N-round sample generation process is performed to obtain training data; and the target question and answer model is trained by using the training data. The method can improve the follow-up ability of the question and answer model in the multi-round dialogue and enhance the question and answer performance of the question and answer model in the target knowledge field.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and in particular, to a training method and device of a question and answer model, an electronic device, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] With the continuous development of computer technology, language models that can be applied to different scenarios have emerged. For example, a language model can be applied to a question and answer scenario to answer user questions through dialogue. A language model applied to a question and answer scenario can also be referred to as a question and answer model.

[0003] Considering that a question and answer scenario is usually related to a specific knowledge field, in the related art, a pre-trained question and answer model is usually fine-tuned using dialogue records of the specific knowledge field to enhance the reasoning ability of the question and answer model in the specific knowledge field. However, the question and answer model fine-tuned by the above method performs poorly in multi-turn dialogue, especially when the user's question is not clearly described, it is difficult to effectively ask follow-up questions. SUMMARY

[0004] The present application provides a training method of a question and answer model. The method can improve the follow-up question ability of the question and answer model in multi-turn dialogue and enhance the question and answer performance of the question and answer model in the target knowledge field. The present application also provides a device, an electronic device, a computer readable storage medium, and a computer program product corresponding to the above method.

[0005] In a first aspect, the present application provides a training method of a question and answer model, which comprises:

[0006] obtaining a first dialogue record of a target knowledge field; wherein the first dialogue record comprises a multi-turn dialogue of a user role and a customer service role for a target question of the target knowledge field, and a target reply of the customer service role to the target question in the first dialogue record comprises target knowledge information of the target knowledge field;

[0007] based on the first dialogue record, performing an N-turn sample generation process to obtain training data;

[0008] training a target question and answer model using the training data;

[0009] wherein the n-turn sample generation process comprises: using the target question and answer model, based on the knowledge information of the target knowledge field and the dialogue context corresponding to the n-turn sample generation process, answering an n-th question to obtain an n-th reply; 1≤n≤N;

[0010] when the n is 1, the nth question is the first question sent by the user role in the first dialogue record; when the n is not 1, the nth question is a question generated by the user role model based on the (n-1)th pursuit, the (n-1)th pursuit is generated by the customer service role model in the (n-1)th sample generation process;

[0011] when the n is 1, the dialogue context corresponding to the nth sample generation process is the nth question; when the n is not 1, the dialogue context corresponding to the nth sample generation process includes a dialogue sequence composed of n-1 questions and n-1 pursuits in the previous n-1 sample generation processes and the nth question;

[0012] the condition for ending the execution of the sample generation process is that the nth reply includes at least part of the target knowledge information.

[0013] In a second aspect, the present application provides a training device of a question and answer model, which comprises:

[0014] an acquisition module, configured to acquire a first dialogue record of a target knowledge field; wherein the first dialogue record includes a plurality of rounds of dialogue of a user role and a customer service role for a target question of the target knowledge field, and a target reply of the customer service role to the target question in the first dialogue record includes target knowledge information of the target knowledge field;

[0015] a generation module, configured to execute N rounds of sample generation processes based on the first dialogue record to obtain training data;

[0016] a training module, configured to train a target question and answer model by using the training data;

[0017] wherein the nth sample generation process includes: using the target question and answer model to answer the nth question based on knowledge information of the target knowledge field and a dialogue context corresponding to the nth sample generation process to obtain the nth reply; 1≤n≤N;

[0018] when the n is 1, the nth question is the first question sent by the user role in the first dialogue record; when the n is not 1, the nth question is a question generated by the user role model based on the (n-1)th pursuit, the (n-1)th pursuit is generated by the customer service role model in the (n-1)th sample generation process;

[0019] When the n is 1, the dialogue context corresponding to the n-th sample generation process is the n-th question; when the n is not 1, the dialogue context corresponding to the n-th sample generation process comprises a dialogue sequence composed of n-1 questions and n-1 follow-up questions in the previous n-1 sample generation processes and the n-th question.

[0020] The condition for ending the execution of the sample generation process is that the N-th reply includes at least part of the target knowledge information.

[0021] In a third aspect, the present application provides an electronic device, comprising a processor and a memory. The processor and the memory are in communication with each other. The processor is configured to execute instructions stored in the memory, so that the electronic device executes the training method of the question and answer model according to the first aspect or any implementation manner of the first aspect.

[0022] In a fourth aspect, the present application provides a computer readable storage medium, which stores instructions. The instructions instruct an electronic device to execute the training method of the question and answer model according to the first aspect or any implementation manner of the first aspect.

[0023] In a fifth aspect, the present application provides a computer program product comprising instructions, which, when executed on an electronic device, cause the electronic device to execute the training method of the question and answer model according to the first aspect or any implementation manner of the first aspect.

[0024] On the basis of the implementation manners of the above aspects, the present application can be further combined to provide more implementation manners.

[0025] As can be seen from the above technical solutions, the present application has the following advantages:

[0026] The present application provides a training method of a question and answer model. The method first acquires a first dialogue record of a target knowledge field. The first dialogue record comprises a multi-turn dialogue of a user role and a customer service role for a target question of the target knowledge field. The target reply of the customer service role to the target question in the first dialogue record comprises target knowledge information of the target knowledge field. Then, based on the first dialogue record, an N-turn sample generation process is executed to obtain training data. The target question and answer model is trained by using the training data.

[0027] The n-th round of sample generation process includes: using the target question and answer model, based on the knowledge information of the target knowledge field and the dialogue context corresponding to the n-th round of sample generation process, answering the n-th question to obtain the n-th reply, 1≤n≤N. When n is 1, the n-th question is the first question sent by the user role in the first dialogue record; when n is not 1, the n-th question is the question generated by the user role model based on the n-1-th follow-up question, and the n-1-th follow-up question is generated by the customer service role model in the n-1-th round of sample generation process. When n is 1, the dialogue context corresponding to the n-th round of sample generation process is the n-th question; when n is not 1, the dialogue context corresponding to the n-th round of sample generation process includes: a dialogue sequence composed of n-1 questions and n-1 follow-up questions generated in the previous n-1 rounds of sample generation process and the n-th question. The condition for ending the execution of the sample generation process is that the N-th reply includes at least part of the target knowledge information.

[0028] In the method, for the question and answer scene of multi-round dialogue in the target knowledge field, by obtaining the first dialogue record generated in the actual question and answer process, on the basis of the first dialogue record, using the model playing the user role (i.e. the question proposer) and the model playing the customer service role (i.e. the question answerer), the process of "follow-up question" and "re-question" in the multi-round dialogue is simulated, and diversified training data is constructed. In this way, the model is fine-tuned using the training data, the follow-up question ability of the question and answer model in the multi-round dialogue is improved, and the question and answer performance of the question and answer model in the target knowledge field is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical method of the embodiments of the present application, the drawings required to be used in the embodiments will be simply introduced as follows.

[0030] Figure 1 A flowchart of a question and answer model training method provided by the embodiments of the present application;

[0031] Figure 2 A schematic diagram of a sample directed graph provided by the embodiments of the present application;

[0032] Figure 3 A structural schematic diagram of a question and answer model training device provided by the embodiments of the present application;

[0033] Figure 4 A structural schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0034] The terms "first", "second", etc. in the embodiments of the present application are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features.

[0035] Firstly, some technical terms and application scenarios involved in the embodiments of the present application are introduced.

[0036] The language model can be understood as a natural language processing model based on deep learning technology. The language model usually has the ability to understand, process and generate natural language. The language model can be applied to different scenarios, for example, the language model can be applied to question answering, text classification, content generation, etc.

[0037] When the language model is applied to the question answering scenario, the language model can also be called a question answering model. The question answering model can answer user questions in the form of dialogue as a virtual assistant, intelligent customer service, etc. Generally, the question answering scenario can be related to a specific knowledge field, for example, the specific knowledge field can be the information technology (IT) field. The user can send a question related to the IT field, and the question answering model answers the question and returns knowledge information related to the IT field to the user.

[0038] In the related art, the reasoning ability of the language model in a specific knowledge field is improved by model fine-tuning of the language model. Specifically, the training process of the language model usually includes a pre-training stage and a post-training stage. In the pre-training stage, the language model is trained by self-supervised learning on a data set. In the post-training stage, the model is fine-tuned by supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) to improve the performance of the language model in a specific knowledge field and optimize the output quality of the language model.

[0039] In the question answering scenario, the pre-trained question answering model is fine-tuned by the dialogue records of a specific knowledge field in the post-training stage to improve the question answering ability of the question answering model in the specific knowledge field. However, the question answering model fine-tuned by the above method performs poorly in multi-turn dialogue, especially when the user provides incomplete information or unclear problem description, it is difficult to effectively ask follow-up questions.

[0040] Therefore, the application provides a training method of a question and answer model. The method first acquires a first dialogue record of a target knowledge field. The first dialogue record includes a plurality of rounds of dialogue between a user role and a customer service role for a target question of the target knowledge field. A target reply of the customer service role to the target question in the first dialogue record includes target knowledge information of the target knowledge field. Then, based on the first dialogue record, an N-round sample generation process is performed to obtain training data. The target question and answer model is trained using the training data.

[0041] The n-th round sample generation process includes: using the target question and answer model, based on the knowledge information of the target knowledge field and a dialogue context corresponding to the n-th round sample generation process, answering the n-th question to obtain an n-th reply, 1≤n≤N. When n is 1, the n-th question is the first question sent by the user role in the first dialogue record. When n is not 1, the n-th question is a question generated by the user role model based on the (n-1)-th follow-up question, and the (n-1)-th follow-up question is generated by the customer service role model in the (n-1)-th round sample generation process. When n is 1, the dialogue context corresponding to the n-th round sample generation process is the n-th question. When n is not 1, the dialogue context corresponding to the n-th round sample generation process includes a dialogue sequence composed of the first n-1 questions and n-1 follow-up questions generated in the first n-1 rounds of sample generation processes and the n-th question. The condition for ending the sample generation process is that the N-th reply includes at least part of the target knowledge information.

[0042] In the method, for the question and answer scene of the plurality of rounds of dialogue in the target knowledge field, by acquiring the first dialogue record generated in the actual question and answer process, based on the first dialogue record, using the model playing the user role (i.e. the question proposer) and the model playing the customer service role (i.e. the question answerer), the process of “follow-up question” and “re-asking” in the plurality of rounds of dialogue is simulated to construct diversified training data. In this way, the model is fine-tuned using the training data to improve the follow-up question capability of the question and answer model in the plurality of rounds of dialogue and enhance the question and answer performance of the question and answer model in the target knowledge field.

[0043] To facilitate understanding of the technical solutions provided in the embodiments of the application, the following will be described with reference to the drawings. Referring to Figure 1 The method specifically includes:

[0044] S101: Acquire a first dialogue record of a target knowledge field.

[0045] The target knowledge field can be understood as any knowledge field related to professional technology. In actual business scenarios, the target knowledge field can be related to a specific business. For example, in a product sales scenario, the target knowledge field can be related to product sales business. For another example, in an IT operation scenario, the target knowledge field can be related to IT operation business.

[0046] In the embodiment of the present application, since it is necessary to enhance the reasoning capability of the question and answer model in the target knowledge field and the multi-round dialogue, the first dialogue record can include a multi-round dialogue in which the user role and the customer service role ask and answer the target question in the target knowledge field, and the target reply of the customer service role to the target question in the first dialogue record includes the target knowledge information in the target knowledge field.

[0047] In other words, the user role can be understood as a questioner, and the user role asks the target question in the target knowledge field. The customer service role can be understood as an answerer, and the customer service role answers the target question in the target knowledge field by recommending the target knowledge information in the target knowledge field to the user role. That is, in the first dialogue record, the user role and the customer service role have a multi-round dialogue, the user role asks the target question, and the customer service role answers the target question and takes the target knowledge information used to answer the target question as the final target reply.

[0048] For example, the user role can be a user, the customer service role can be a customer service personnel, and the target knowledge field can be an IT operation field. The first dialogue record can be as follows:

[0049] “User: Not working

[0050] Customer service personnel: What is not working?

[0051] User: word is not working

[0052] Customer service personnel: Your license has expired. You can reapply from [Apply for license](http: / / how-to-apply)

[0053] In the above first dialogue record as an example, the target question is “word is not working”, the target reply is “Your license has expired. You can reapply from [Apply for license](http: / / how-to-apply)”, and the target knowledge information is “[Apply for license](http: / / how-to-apply)”.

[0054] The embodiments of the present application do not limit the manner of obtaining the first dialogue record. In some embodiments, the historical dialogue record in the question and answer system of the target knowledge field can be used as the first dialogue record, wherein the question and answer system of the target knowledge field can be used for the user role and the customer service role to interact with each other for relevant problems in the target knowledge field. In other embodiments, the dialogue record can also be presented in the form of a work order, that is, completing a problem solving is equivalent to completing a work order, in which case the historical work order related to the target knowledge field in the work order system can be used as the first dialogue record.

[0055] Since subsequent training data needs to be generated based on the first dialogue record, in order to ensure the quality of model fine-tuning, in the embodiments of the present application, the quality of the dialogue of the first dialogue record is required. In some possible implementations, the first dialogue record can be a dialogue record that the user role indicates satisfaction. For example, after the user role and the customer service role complete a multi-turn dialogue, the user role can feed back (such as satisfaction or dissatisfaction) to the dialogue record, and the dialogue record that the user role indicates satisfaction is determined as the first dialogue record, ensuring that the quality of the first dialogue record meets the model fine-tuning requirements. In other possible implementations, the first dialogue record can also be a dialogue record that passes the quality check. For example, a quality checker checks the quality of the dialogue record (such as passing the quality check or failing the quality check), and the dialogue record that passes the quality check is determined as the first dialogue record, ensuring that the quality of the first dialogue record meets the model fine-tuning requirements.

[0056] S102: Based on the first dialogue record, performing an N-turn sample generation process to obtain training data.

[0057] In the embodiments of the present application, the sample generation process can be understood as a process of constructing a new multi-turn dialogue based on the first dialogue record. Wherein, one turn sample generation process can include three steps of "determining a question", "determining a reply" and "determining a follow-up question".

[0058] Specifically, taking the n-th turn sample generation process as an example for illustration, 1≤n≤N, that is, the n-th turn sample generation process can be understood as any one of the N-turn sample generation processes. The n-th turn sample generation process can include: using the target question and answer model to answer the n-th question based on the knowledge information of the target knowledge field and the dialogue context corresponding to the n-th turn sample generation process, to obtain the n-th reply.

[0059] The target question answering model can be understood as a pre-trained language model that requires model fine-tuning. That is, in the embodiments of the present application, the model fine-tuning is performed on the target question answering model, and the model fine-tuned target question answering model is used for question answering. For example, the model fine-tuned target question answering model can perform question answering in the target knowledge field as a virtual assistant, intelligent customer service, etc., to realize intelligent question answering.

[0060] In each round of sample generation process, the step of "determining a reply" is performed by the target question answering model. In the embodiments of the present application, the retrieval-augmented generation (RAG) technology is used to assist the target question answering model to generate a reply. The RAG technology includes two stages: an information retrieval stage and a reply generation stage. In the information retrieval stage, a recall model is called to retrieve knowledge information in the target knowledge field from the knowledge base of the target knowledge field. In the reply generation stage, the target question answering model answers the nth question by means of the knowledge information obtained in the information retrieval stage to generate the nth reply.

[0061] When n is 1, the nth question is the first question sent by the user role in the first dialogue record. That is, in the first round of sample generation process, the first question is the first question sent by the user role in the first dialogue record, and the construction of the multi-round dialogue is started from the first question sent by the user role.

[0062] It should be noted that the first question sent by the user role is not necessarily the same as the target question. In some embodiments, the first question sent by the user role in the first dialogue record proposes the target question, and the first question is the target question. In other embodiments, the first question sent by the user role in the first dialogue record fails to clearly describe the question, and the target question is proposed in the subsequent question of the user role, and the first question is not the target question. For example, in the foregoing example in the IT operation and maintenance field, the first question sent by the user role is "not working", and the target question is "word not working", which are different.

[0063] When n is not 1, the nth question is a question generated by the user role model based on the (n-1)th follow-up question, and the (n-1)th follow-up question is generated by the customer service role model in the (n-1)th round of sample generation process.

[0064] That is, from the second round of sample generation process, the step of "determining a question" is performed by the user role model. In each round of sample generation process including the first round of sample generation process, the step of "determining a follow-up question" is performed by the customer service role model.

[0065] The user role model can be understood as a language model representing a user role, and the customer service role model can be understood as a language model representing a customer service role. The user role model and the customer service role model can be self-developed language models.

[0066] That is, the user role model can simulate and play the role of the user, and perform the step of "determining the question" from the perspective of the user, and the customer service role can simulate and play the role of the customer service personnel, and perform the step of "determining the follow-up question" from the perspective of the customer service personnel.

[0067] In the embodiments of the present application, the condition for ending the execution of the sample generation process is that the Nth reply includes at least part of the target knowledge information. In other words, the sample generation process is executed for N rounds, and when the Nth round of the sample generation process meets the condition for ending the execution of the sample generation process, i.e., the Nth reply includes at least part of the target knowledge information, the sample generation process is ended, and the N+1th round of the sample generation process is no longer executed.

[0068] For example, when the target knowledge information includes only one knowledge article, the condition for ending the execution of the sample generation process is that the Nth reply includes the knowledge article of the target knowledge information. For another example, when the target knowledge information includes multiple knowledge articles, the condition for ending the execution of the sample generation process is that the Nth reply includes more than a proportion threshold of the number of knowledge articles of the target knowledge information.

[0069] That is, if the Nth reply generated by the target question and answer model in the Nth round of the sample generation process includes at least part of the target knowledge information, it indicates that the Nth reply is the target reply in the first dialogue record, and the target question and answer model generates the correct reply to the target question. Based on the first dialogue record, a new dialogue record starting from the first question sent by the user role in the first dialogue record and ending with the Nth reply equivalent to the target reply is constructed, and a high-quality multi-round dialogue is automatically constructed.

[0070] The following is an example of a total of 2 rounds of sample generation processes. In the first round of the sample generation process, the first question is the first question sent by the user role in the first dialogue record, the target question and answer model answers the first question based on the knowledge information of the target knowledge field and the dialogue context corresponding to the first round of the sample generation process, and obtains the first reply. The first reply does not meet the condition for ending the execution of the sample generation process, the customer service role model asks the first question, and obtains the first follow-up question.

[0071] In the second round of the sample generation process, the user role model proposes a second question based on the first question, the target question answering model answers the second question based on the knowledge information of the target knowledge field and the dialogue context corresponding to the second round of the sample generation process, obtains a second reply, and the second reply meets the condition of ending the execution of the sample generation process, but in order to construct training data with the same path prefix, the customer service role model asks the second question and obtains the second question.

[0072] In the embodiments of the present application, the target question answering model answers the nth question in combination with the dialogue context corresponding to the nth round of the sample generation process. The dialogue context corresponding to the nth round of the sample generation process can be understood as a generated dialogue sequence including questions and questions. That is, the target question answering model answers the nth question through the generated dialogue sequence to obtain the nth reply.

[0073] Specifically, when n is 1, the dialogue context corresponding to the nth round of the sample generation process is the nth question. Since no other dialogue content is generated in the first round of the sample generation process, the dialogue context corresponding to the first round of the sample generation process is the first question itself.

[0074] When n is not 1, the dialogue context corresponding to the nth round of the sample generation process includes a dialogue sequence composed of n-1 questions and n-1 questions in the previous n-1 rounds of sample generation processes and the nth question.

[0075] Since the n-1 replies generated in the previous n-1 rounds of sample generation do not meet the condition of ending the execution of the sample generation process, that is, the n-1 replies generated in the previous n-1 rounds of sample generation are not correct replies to the target question, in the previous n-1 rounds of sample generation, the target question answering model cannot give a correct reply based on the existing dialogue sequence, and should ask questions. Therefore, the n-1 replies are not included in the dialogue context corresponding to the nth round of the sample generation process.

[0076] The process of generating replies by the target question answering model, generating questions by the user role model, and generating questions by the customer service role model will be described below.

[0077] The target question answering model can generate replies based on prompt learning. In a generative task (such as a text generation task, a question answering task, or a dialogue task), a prompt can be used to guide the language model to perform a specific output. By configuring the prompt, the language model can understand the background and requirements of the task, and without retraining the language model, the language model can handle different types of natural language processing tasks, increasing the scalability and flexibility of the language model.

[0078] Specifically, a first prompt word is generated, the first prompt word is sent to the target question and answer model, and an nth reply returned by the target question and answer model is received.

[0079] The first prompt word includes knowledge information of the target knowledge field, a dialogue context corresponding to the nth sample generation process, and prompt information for indicating an answer to the nth question. Since the first prompt word includes the above information, the target question and answer model can generate an nth reply to the nth question in the nth sample generation process based on the prompting capability of the first prompt word, in combination with the knowledge information of the target knowledge field and the dialogue context corresponding to the nth sample generation process.

[0080] Similarly, when n is not 1, the user role model can generate a question based on the prompt learning manner. Specifically, a third prompt word is generated, the third prompt word is sent to the user role model, and an nth question returned by the user role model is received.

[0081] The third prompt word includes prompt information for indicating a simulated user role, a dialogue sequence composed of n-1 questions and n-1 follow-up questions generated in the previous n-1 sample generation processes, the first dialogue record, and prompt information for indicating a generated question.

[0082] Since the user role model needs to reask questions based on the existing dialogue sequence, the user role model needs to know the complete and correct first dialogue record to ensure that the nth question generated is consistent with the first dialogue record and avoid generating questions inconsistent with the first dialogue record.

[0083] Since the third prompt word includes the above information, the user role model can simulate the role of the user based on the prompting capability of the third prompt word, in combination with the existing dialogue sequence, to generate an nth question consistent with the first dialogue record in the nth sample generation process.

[0084] Similarly, when n is not 1, the customer service role model can generate a follow-up question based on the prompt learning manner. Specifically, a second prompt word is generated, the second prompt word is sent to the customer service role model, and an n-1 follow-up question returned by the customer service role model is received.

[0085] The second prompt word includes prompt information for indicating a simulated customer service role, a dialogue sequence composed of n-1 questions and n-2 follow-up questions generated in the previous n-1 sample generation processes, and prompt information for indicating a generated follow-up question.

[0086] Since the customer role model has not generated the n-1th follow-up question in the n-1th round of sample generation process, the existing dialogue sequence at this time only includes n-1 questions and n-2 follow-up questions. Since the second prompt word includes the above information, the customer role model can simulate the role of the second object based on the prompting ability of the second prompt word, combine the existing dialogue sequence, and ask the n-1th question in the n-1th round of sample generation process. The n-1th follow-up question is obtained.

[0087] Taking the previous example in the IT operation field as an example, in the first round of sample generation process, the first question is "not working", and the target question and answer model replies to the first question. Since the description of the first question is not clear, the first reply generated by the target question and answer model does not include target knowledge information, and the sample generation process continues. The customer role model asks the first question, at this time, the second prompt word can be as follows:

[0088] "You are responsible for playing the role of a customer service personnel, and based on the following dialogue record, you should reasonably ask questions about the user's problem to help the user locate and solve the problem as soon as possible.

[0089] Dialogue record:

[0090] User: not working

[0091] Send the second prompt word to the customer role model, and the customer role model outputs the first follow-up question "What does not work for you?", and completes the first round of sample generation process.

[0092] In the second round of sample generation process, the user role model re-describes the problem based on the first follow-up question, at this time, the third prompt word can be as follows:

[0093] "The first dialogue record is:

[0094] User: not working

[0095] Customer service personnel: What does not work?

[0096] User: word does not work

[0097] Customer service personnel: Your license has expired. You can reapply from [apply for license] (http: / / how-to-apply)

[0098] You are responsible for playing the role of a user, and based on the first dialogue record, you should give an answer to the follow-up question of the customer service personnel in the dialogue record below, which is based on the above first dialogue record and the follow-up question of the customer service personnel.

[0099] Dialogue record:

[0100] User: not working

[0101] Customer service personnel: What is not working?

[0102] The third prompt word is sent to the user role model, and the user role model outputs the second question "word not working". The target question answering model replies to the second question. Since the second question clearly describes the target question, the target question answering model can generate target knowledge information (i.e., "[application permit](http: / / how-to-apply)") and meet the condition for ending the sample generation process. At this time, in order to construct training data with the same path prefix, the customer service role model can also ask the second question and obtain the second follow-up question.

[0103] After completing the N-turn sample generation process, the Nth reply meets the condition for ending the sample generation process, and the process of constructing the multi-turn dialogue is ended. In the embodiments of the present application, the constructed multi-turn dialogue is converted into a sample directed graph, and the association relationship between "questions", "replies" and "follow-up questions" in the constructed multi-turn dialogue is represented by the sample directed graph.

[0104] Specifically, according to the N questions, N follow-up questions and N replies in the N-turn sample generation process, a sample directed graph is constructed, and according to the sample directed graph, positive samples and negative samples are determined.

[0105] The sample directed graph is composed of a first vertex representing a question, a second vertex representing a reply, a third vertex representing a follow-up question, and a plurality of directed edges. In other words, "questions", "replies" and "follow-up questions" in the multi-turn dialogue are represented as different types of vertices in the sample directed graph.

[0106] The directed edges in the sample directed graph include an edge from the first vertex representing the mth question to the second vertex representing the mth reply, an edge from the first vertex representing the mth question to the third vertex representing the mth follow-up question, and an edge from the third vertex representing the mth follow-up question to the first vertex representing the m+1th question, 1≤m≤N.

[0107] In the N-turn sample generation process, the mth reply is an answer generated by the target question answering model for the mth question, so there is an association relationship between the first vertex representing the mth question and the second vertex representing the mth reply. In the N-turn sample generation process, the mth follow-up question is a follow-up question generated by the customer service role model for the mth question, so there is an association relationship between the first vertex representing the mth question and the third vertex representing the mth follow-up question. In the N

[0108] In the sample generation process, the m+1th question is a re-question generated by the user role model for the mth follow-up question, therefore, there is an association relationship between the third vertex representing the mth follow-up question and the first vertex representing the m+1th question.

[0109] Referring to Figure 2 FIG. 1 shows a schematic diagram of a sample directed graph, in which Qi represents a first vertex, Ai represents a second vertex, ai represents a third vertex, and i represents the ith round of sample generation process. From Figure 2 As can be seen from FIG. 1, there are edges from Q1 to A1, edges from Q1 to a1, edges from a1 to Q2, and the like in the sample directed graph.

[0110] Since there are multiple first vertices representing multiple replies generated by the target question and answer model in the sample directed graph, and there are incorrect replies (i.e., the first N-1 replies) and correct replies (i.e., the Nth reply) in the multiple replies, from the sample directed graph, it can be determined that the positive sample representing the correct dialogue for the target question and the negative sample representing the incorrect dialogue for the target question.

[0111] In some possible implementation manners, from the sample directed graph, a first path between the first vertex representing the 1st question and the second vertex representing the Nth reply is determined, and then the first path and a sub-path in the first path between the first vertex representing the 1st question and any third vertex representing a follow-up question are determined as the positive sample.

[0112] That is, the first path can be understood as a path in the sample directed graph between the first vertex representing the first question and the second vertex representing the last reply. For example, in Figure 2 In FIG. 1, the first path is “Q1-a1-Q2-a2-Q3-……-QN-AN”.

[0113] It can be found that, in the first path, except for the second vertex representing the Nth reply, the rest of the vertices are the first vertex representing the question and the third vertex representing the follow-up question. The Nth reply satisfies the condition of ending the sample generation process, that is, only when the target question and answer model generates the Nth reply, it indicates that the target question and answer model has correctly replied to the target question. That is, the target of model fine-tuning of the target question and answer model is that before the Nth question appears in the multi-round dialogue, the target question and answer model should ask follow-up questions based on the question, rather than directly reply. Therefore, the first path represents a correct multi-round dialogue, and belongs to the positive sample.

[0114] Further, since the first path represents a correct multi-turn dialogue, a sub-path from the first vertex to any third vertex representing a follow-up question in the first path can represent a follow-up question that the target question answering model should generate for the existing dialogue sequence, and thus also represents a correct dialogue, belonging to the positive sample. For example, in the first path, the sub-path from the first vertex to any third vertex representing a follow-up question can be "Q1-a1", "Q1-a1-Q2-a2", and the like. Figure 2

[0115] For the negative sample, from the sample directed graph, a path from the first vertex representing the first question to the second vertex representing the kth reply, and a path from the first vertex representing the first question to the third vertex representing the Nth follow-up question, are determined as the negative sample. Wherein, 1≤k<N.

[0116] That is, the kth reply is any reply in the first N-1 replies, and the target of the model fine-tuning of the target question answering model is that the target question answering model should follow up based on the question for the first N-1 questions, instead of directly replying. Therefore, the path from the first vertex representing the first question to the second vertex representing the kth reply represents an incorrect multi-turn dialogue, belonging to the negative sample.

[0117] The path from the first vertex representing the first question to the third vertex representing the Nth follow-up question is a path from the first vertex representing the first question to the third vertex representing the Nth follow-up question in the sample directed graph. Since the Nth reply is a correct reply, the target of the model fine-tuning of the target question answering model is that the target question answering model should reply for the Nth question, instead of following up. Therefore, the path from the first vertex representing the first question to the third vertex representing the Nth follow-up question represents an incorrect multi-turn dialogue, belonging to the negative sample.

[0118] In this way, from the sample directed graph, the paths belonging to the positive sample and the paths belonging to the negative sample are screened out, and the positive and negative samples are determined simultaneously for the constructed multi-turn dialogue, improving the model fine-tuning efficiency. Moreover, there are positive samples and negative samples with the same path prefix, for example, the positive sample is "Q1-a1-Q2-a2", and the negative sample is "Q1-a1-Q2-A2", having the same path prefix "Q1-a1-Q2", facilitating subsequent targeted adjustment in the model fine-tuning process.

[0119] S103: training the target question answering model using the training data.

[0120] ​In the post-training stage of the target question answering model, the training data can be divided into the form of "input" and "output". Among them, "input" and "output" can also be understood as a question and answer pair, and the "input" part can be understood as a prompt word input into the target question answering model, and the "output" part can be understood as the content that the target question answering model is expected or not expected to generate.

[0121] Specifically, for the positive sample, the "input" part can be referred to as the positive sample input, and the "output" part can be referred to as the positive sample output, that is, when the positive sample input is sent to the target question answering model, the positive sample output is expected to be returned by the target question answering model. For the negative sample, the "input" part can be referred to as the negative sample input, and the "output" part can be referred to as the negative sample output, that is, when the negative sample input is sent to the target question answering model, the negative sample output is not expected to be returned by the target question answering model.

[0122] In the embodiments of the present application, for each positive sample, the following steps are performed: determining the positive sample input according to the path composed of the vertices except the last vertex in the positive sample and the knowledge information of the target knowledge field, and determining the last vertex in the positive sample as the positive sample output, and constructing the positive sample question and answer pair with the positive sample input and the positive sample output.

[0123] For each negative sample, the following steps are performed: determining the negative sample input according to the path composed of the vertices except the last vertex in the negative sample and the knowledge information of the target knowledge field, and determining the last vertex in the negative sample as the negative sample output, and constructing the negative sample question and answer pair with the negative sample input and the negative sample output.

[0124] The last vertex in the positive sample and the negative sample is related to the target question answering model. In the positive sample, the last vertex represents the correct reply that the target question answering model should generate for the target question, or represents the follow-up question that the target question answering model should generate for other questions that are not clearly described. In the negative sample, the last vertex represents the reply that the target question answering model should not generate for other questions that are not clearly described, or represents the follow-up question that the target question answering model should not generate for the target question. Therefore, according to the content of the vertices except the last vertex in the positive sample, the positive sample input is determined, and the content of the last vertex in the positive sample is determined as the positive sample output, and according to the content of the vertices except the last vertex in the negative sample, the negative sample input is determined, and the content of the last vertex in the negative sample is determined as the negative sample output.

[0125] It can be understood that since the positive sample input and the negative sample input are equivalent to the prompt word input into the target question answering model, it is necessary to fill the relevant content in the positive sample and the negative sample into the prompt word template, and then form the positive sample input and the negative sample input.

[0126] Taking the positive sample as "Q1-a1-Q2-a2" and the negative sample as "Q1-a1-Q2-A2" as an example, the positive sample input and the negative sample input can be as follows:

[0127] "As a Q&A expert, your job is to combine the following knowledge information to answer or ask questions to the user, and finally return information that can answer the user's question.

[0128] Knowledge information: {knowledge information of the target knowledge field}

[0129] Dialogue record:

[0130] User: {Q1}

[0131] Intelligent customer service: {a1}

[0132] User: {Q2}"

[0133] Since the positive sample and the negative sample have the same path prefix, the positive sample input and the negative sample input are the same, the positive sample output is "a2", and the negative sample output is "A2".

[0134] After constructing the positive sample question and answer pair and the negative sample question and answer pair, the target question and answer model is trained using the positive sample question and answer pair and the negative sample question and answer pair. In some embodiments, the target question and answer model is subjected to SFT using the positive sample question and answer pair to improve the turn-by-turn dialogue capability of the target question and answer model. In other embodiments, the target question and answer model is subjected to RLHF using the positive sample question and answer pair and the negative sample question and answer pair, for example, using the Kahneman-Tversky optimization (KTO) method for RLHF, using the direct preference optimization (DPO) method for RLHF, etc.

[0135] Further, the embodiments of the present application also support data augmentation of training data in different ways to strengthen the effect of model fine-tuning and improve the robustness of the target question and answer model after model fine-tuning.

[0136] In some possible implementations, the content corresponding to the first vertex and the second vertex in the positive sample is replaced with semantically similar content to generate a new positive sample, and the content corresponding to the first vertex and the second vertex in the negative sample is replaced with semantically similar content to generate a new negative sample.

[0137] That is, the content representing the question and the content representing the reply in the training data are replaced with semantically similar expressions to enrich the number of positive samples and negative samples. Moreover, by adjusting the expression of the positive sample and the negative sample, the question and answer capability of the target question and answer model under similar different expressions is enhanced.

[0138] In some other possible implementations, the knowledge information in the target knowledge domain may include multiple knowledge articles. At least one of the following modification operations is performed: randomly adding at least one knowledge article to the knowledge information in the target knowledge domain; adjusting the order of multiple knowledge articles in the knowledge information in the target knowledge domain; replacing the content representing the titles of multiple knowledge articles in the knowledge information in the target knowledge domain with semantically similar content; and replacing the content representing the article content of multiple knowledge articles in the knowledge information in the target knowledge domain with semantically similar content. Then, using the knowledge information in the target knowledge domain after the modification operations, new positive sample inputs and new negative sample inputs are determined. New positive sample question-answer pairs are constructed using the new positive sample inputs and positive sample outputs, and new negative sample question-answer pairs are constructed using the new negative sample inputs and negative sample outputs.

[0139] In other words, the content related to "knowledge information in the target knowledge domain" in both positive and negative sample inputs is adjusted. This is achieved by adding new knowledge articles, shuffling the order of multiple knowledge articles, or replacing the titles and content of existing knowledge articles, thereby increasing the number of positive and negative samples. Furthermore, by adjusting the content related to "knowledge information in the target knowledge domain," the ability of the target question-answering model to combine different knowledge information for question answering is enhanced.

[0140] This method targets multi-turn dialogue question-and-answer scenarios within the target knowledge domain. It acquires the first dialogue records generated during actual question-and-answer sessions. Based on these records, it utilizes models assuming the user role (questioner) and the customer service role (question answerer) to simulate the processes of "follow-up questions" and "re-questioning" in multi-turn dialogues, constructing diverse training data. This training data is then used for model fine-tuning, improving the question-and-answer model's ability to follow up in multi-turn dialogues and enhancing its performance in the target knowledge domain.

[0141] The above text combined Figure 1 and Figure 2 The training method of the question-answering model provided in the embodiments of this application has been described in detail. The apparatus and equipment provided in the embodiments of this application will be described below with reference to the accompanying drawings.

[0142] See Figure 3 The diagram shows the structure of a training device for the question-answering model. The device 30 includes:

[0143] The acquisition module 301 is configured to acquire a first dialogue record of a target knowledge field; wherein the first dialogue record comprises a multi-round dialogue of a user role and a customer service role for a target question of the target knowledge field, and a target reply of the customer service role for the target question in the first dialogue record comprises target knowledge information of the target knowledge field;

[0144] The generation module 302 is configured to perform an N-round sample generation process based on the first dialogue record to obtain training data;

[0145] The training module 303 is configured to train a target question and answer model by using the training data;

[0146] The n-th round sample generation process comprises: using the target question and answer model, based on the knowledge information of the target knowledge field and a dialogue context corresponding to the n-th round sample generation process, answering an n-th question to obtain an n-th reply; 1≤n≤N;

[0147] When the n is 1, the n-th question is a first question sent by the user role in the first dialogue record; when the n is not 1, the n-th question is a question generated by the user role model based on an n-1-th follow-up question generated by the customer service role model in an n-1-th round sample generation process;

[0148] When the n is 1, the dialogue context corresponding to the n-th round sample generation process is the n-th question; when the n is not 1, the dialogue context corresponding to the n-th round sample generation process comprises a dialogue sequence composed of n-1 questions and n-1 follow-up questions in the first n-1 rounds of sample generation processes and the n-th question;

[0149] The condition for ending the execution of the sample generation process is that the N-th reply comprises at least part of the target knowledge information.

[0150] In some possible implementation manners, the generation module 302 is specifically configured to:

[0151] According to N questions, N follow-up questions and N replies in the N-round sample generation process, a sample directed graph is constructed; wherein the sample directed graph is composed of a first vertex representing a question, a second vertex representing a reply, a third vertex representing a follow-up question and a plurality of directed edges, the directed edges comprise an edge from the first vertex representing an m-th question to the second vertex representing an m-th reply, an edge from the first vertex representing the m-th question to the third vertex representing an m-th follow-up question and an edge from the third vertex representing the m-th follow-up question to the first vertex representing an m+1-th question, 1≤m≤N;

[0152] According to the sample directed graph, positive samples and negative samples are determined; the positive samples represent correct dialogues for the target question, and the negative samples represent incorrect dialogues for the target question.

[0153] In some possible implementation manners, the generation module 302 is specifically configured to:

[0154] From the sample directed graph, a first path between a first vertex representing a first question and a second vertex representing an Nth reply is determined;

[0155] The first path and a sub-path in the first path between the first vertex representing the first question and a third vertex representing an inquiry are determined as a positive sample;

[0156] From the sample directed graph, a path between the first vertex representing the first question and the second vertex representing the kth reply and a path between the first vertex representing the first question and the third vertex representing the Nth inquiry are determined as a negative sample; 1≤k

[0157] In some possible implementation manners, the generation module 302 is further configured to:

[0158] Contents corresponding to the first vertex and the second vertex in the positive sample are replaced with contents that are semantically similar, to generate a new positive sample; and

[0159] Contents corresponding to the first vertex and the second vertex in the negative sample are replaced with contents that are semantically similar, to generate a new negative sample.

[0160] In some possible implementation manners, the training module 303 is specifically configured to:

[0161] For each positive sample, the following steps are performed: a positive sample input is determined according to a path composed of vertices other than a last vertex in the positive sample and knowledge information of the target knowledge field, and the last vertex in the positive sample is determined as a positive sample output; and a positive sample question-answer pair is constructed from the positive sample input and the positive sample output;

[0162] For each negative sample, the following steps are performed: a negative sample input is determined according to a path composed of vertices other than a last vertex in the negative sample and knowledge information of the target knowledge field, and the last vertex in the negative sample is determined as a negative sample output; and a negative sample question-answer pair is constructed from the negative sample input and the negative sample output;

[0163] The target question-answer model is trained by using the positive sample question-answer pair and the negative sample question-answer pair.

[0164] In some possible implementation manners, the knowledge information of the target knowledge field includes a plurality of knowledge articles; and the generation module 302 is further configured to:

[0165] perform at least one modification operation, including: randomly adding at least one knowledge article in the knowledge information of the target knowledge field, adjusting the arrangement order of the plurality of knowledge articles in the knowledge information of the target knowledge field, replacing the content representing the title of the plurality of knowledge articles in the knowledge information of the target knowledge field with semantically similar content, and replacing the content representing the article content of the plurality of knowledge articles in the knowledge information of the target knowledge field with semantically similar content;

[0166] determine new positive sample inputs and new negative sample inputs by using the knowledge information of the target knowledge field after performing the modification operation;

[0167] construct a new positive sample question and answer pair by using the new positive sample inputs and the positive sample outputs, and construct a new negative sample question and answer pair by using the new negative sample inputs and the negative sample outputs.

[0168] In some possible implementation manners, the generation module 302 is specifically configured to:

[0169] generate a first prompt word; wherein the first prompt word includes the knowledge information of the target knowledge field, the dialogue context corresponding to the n-th round of sample generation process, and prompt information for indicating that the n-th question is answered;

[0170] send the first prompt word to the target question and answer model, and receive the n-th reply returned by the target question and answer model.

[0171] In some possible implementation manners, the generation module 302 is specifically configured to:

[0172] generate a second prompt word; wherein the second prompt word includes prompt information for indicating a simulated customer service role, a dialogue sequence composed of n-1 questions and n-2 follow-up questions generated in the previous n-1 rounds of sample generation process, and prompt information for indicating that a follow-up question is generated;

[0173] send the second prompt word to the customer service role model, and receive the n-1-th follow-up question returned by the customer service role model.

[0174] In some possible implementation manners, the generation module 302 is specifically configured to:

[0175] generating a third prompt word; wherein the third prompt word comprises: prompt information for indicating a simulation user role, a dialogue sequence composed of n-1 questions and n-1 follow-up questions generated in the first n-1 rounds of sample generation process, the first dialogue record, and prompt information for indicating generation of a question;

[0176] sending the third prompt word to the user role model, and receiving an n th question returned by the user role model.

[0177] The training apparatus 30 of the question and answer model according to the embodiments of the present application can correspond to performing the methods described in the embodiments of the present application, and the above and other operations and / or functions of each module / unit of the training apparatus 30 of the question and answer model are respectively implemented by the corresponding modules / units Figure 1 The corresponding flow of each method in the illustrated embodiments, for the sake of brevity, will not be repeated here.

[0178] The embodiments of the present application also provide an electronic device. The electronic device is specifically used to implement the functions of the training apparatus 30 of the question and answer model in the embodiments as Figure 3 described above.

[0179] Figure 4 A structural schematic diagram of an electronic device 400 is provided, as Figure 4 shown, the electronic device 400 includes a bus 401, a processor 402, a communication interface 403, and a memory 404. The processor 402, the memory 404, and the communication interface 403 communicate through the bus 401.

[0180] The bus 401 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 4 only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0181] The processor 402 can be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0182] The communication interface 403 is used for external communication. For example, the communication interface 403 can be used for communication with a terminal.

[0183] The memory 404 can include volatile memory, such as random access memory (RAM), and non-volatile memory, such as read-only memory (ROM), floppy disks, hard disks, or solid state drives.

[0184] The memory 404 stores executable code, which the processor 402 executes to perform the aforementioned method for training the question-answer model.

[0185] Specifically, in the case of the embodiment shown, and Figure 3 In the case of the embodiment described, and Figure 3 In the case of the embodiment described, and Figure 3 The software or program code required for the functions of the modules / units in the foregoing method for training the question-answer model can be partially or entirely stored in the memory 404. The processor 402 executes the program code corresponding to each unit stored in the memory 404 to perform the aforementioned method for training the question-answer model.

[0186] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium can be any available medium or data storage device that can be accessed by a computing device, such as a data center. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk), etc. The computer-readable storage medium includes instructions that instruct the computing device to perform the aforementioned method for training the question-answer model of the question-answer model training apparatus 30.

[0187] The embodiments of the present application also provide a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, the processes or functions described in the embodiments of the present application are generated in whole or in part.

[0188] The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer or data center to another website, computer or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode.

[0189] The computer program product is executed by a computer, and the computer executes any of the methods for training the question and answer model. The computer program product can be a software installation package, and when any of the methods for training the question and answer model is needed, the computer program product can be downloaded and executed on the computer.

[0190] The flowcharts and block diagrams in the drawings illustrate the possible architectures, functionality, and operations of systems, methods, and computer program products in accordance with various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or in the reverse order, depending on the functionality involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by dedicated hardware-based systems that perform the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0191] The units involved in the embodiments of the present application described in the embodiments can be implemented in the form of software, or can be implemented in the form of hardware. Among them, the name of the unit / module does not constitute a limitation of the unit itself in some cases.

[0192] The functions described above in the embodiments of the present application can be executed at least in part by one or more hardware logic components. For example, non-limiting examples of exemplary types of hardware logic components that can be used include: field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-chips (SOCs), complex programmable logic devices (CPLDs), and so on.

[0193] In the context of the embodiments of the present application, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0194] It should be noted that the various embodiments described in the specification are progressive, and each embodiment focuses on the differences from other embodiments. The same and similar parts between embodiments can be referred to each other.

[0195] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between the associated objects, which means that there can be three kinds of relationships, for example, "A and / or B" can represent: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0196] It is also to be noted that, as used in the specification and the appended claims, the singular forms "a," "an" and "the" include plural referents unless otherwise indicated. Furthermore, to the extent that the terms "including," "includes," "having," "has," "with," or "contains" are used in either the detailed description and the claims, such terms are intended to be inclusive in a manner similar to the term "comprising" as an open transition term without precluding any additional or other elements.

[0197] The embodiments disclosed herein can each be implemented as a method, apparatus, or article of manufacture using programming instructions. The embodiments disclosed herein can be implemented using software, firmware, hardware, or a combination thereof. The embodiments disclosed herein can be implemented in a computer system that includes one or more processors that are configured with instructions that, once implemented in hardware, cause the computer system to carry out the steps described herein. The instructions can be stored on a computer readable medium, such as a floppy disk, a hard disk, a CD-ROM, a DVD, a memory, a solid state drive, or a magnetic tape. The instructions can also be downloaded from the Internet. The instructions can be implemented in a plurality of programming languages.

[0198] The above description of disclosed embodiments is intended to be illustrative and not restrictive. Many embodiments will be apparent to those of skill in the art upon reading and understanding the above description, and it is therefore contemplated to be within the scope of the application to claim any such embodiments. Changes to the described embodiments can be made without departing from the spirit and scope of the application. Accordingly, the application is not to be limited by the above description, but is only limited by the scope of the appended claims.

Claims

1. A training method for a question-answering model, characterized in that, The method includes: Obtain the first dialogue record of the target knowledge domain; wherein, the first dialogue record includes a multi-round dialogue between a user role and a customer service role regarding a target question in the target knowledge domain, and the target response of the customer service role to the target question in the first dialogue record includes target knowledge information of the target knowledge domain; Based on the first dialogue record, N rounds of sample generation process are performed to obtain training data; The target question-answering model is trained using the training data. The nth round of sample generation process includes: using the target question-answering model, based on the knowledge information of the target knowledge domain and the dialogue context corresponding to the nth round of sample generation process, answering the nth question to obtain the nth response; 1≤n≤N; When n is 1, the nth question is the first question sent by the user role in the first dialogue record; when n is not 1, the nth question is a question generated by the user role model based on the (n-1)th follow-up question, which is generated by the customer service role model during the (n-1)th round of sample generation. When n is 1, the dialogue context corresponding to the nth round of sample generation is the nth question; when n is not 1, the dialogue context corresponding to the nth round of sample generation includes: a dialogue sequence consisting of n-1 questions and n-1 follow-up questions from the previous n-1 rounds of sample generation and the nth question. The condition for ending the sample generation process is that the Nth response includes at least part of the target knowledge information.

2. The method according to claim 1, characterized in that, The step of performing N rounds of sample generation based on the first dialogue record to obtain training data includes: Based on the N questions, N follow-up questions, and N responses generated in the N rounds of sample generation, a directed sample graph is constructed. This directed graph consists of a first vertex representing the question, a second vertex representing the response, a third vertex representing the follow-up question, and multiple directed edges. These directed edges include: an edge from the first vertex representing the m-th question to the second vertex representing the m-th response, an edge from the first vertex representing the m-th question to the third vertex representing the m-th follow-up question, and an edge from the third vertex representing the m-th follow-up question to the first vertex representing the (m+1)-th question, where 1 ≤ m ≤ N. Based on the directed graph of the samples, positive samples and negative samples are determined; wherein, the positive samples represent correct dialogues for the target question, and the negative samples represent incorrect dialogues for the target question.

3. The method according to claim 2, characterized in that, The step of determining positive and negative samples based on the directed graph of the samples includes: From the directed graph of the samples, determine a first path from the first vertex representing the first question to the second vertex representing the Nth response; The first path and the sub-paths within the first path from the first vertex representing the first question to any third vertex representing the follow-up question are determined as positive samples; From the directed graph of the samples, the paths from the first vertex representing the first question to the second vertex representing the k-th response, and the paths from the first vertex representing the first question to the third vertex representing the N-th follow-up question, are determined as negative samples; where 1≤k<N.

4. The method according to claim 3, characterized in that, The method further includes: Replace the content corresponding to the first vertex and the second vertex in the positive sample with semantically similar content to generate a new positive sample; and, The content corresponding to the first vertex and the second vertex in the negative sample is replaced with semantically similar content to generate a new negative sample.

5. The method according to claim 3, characterized in that, The step of training the target question-answering model using the training data includes: For each positive sample, perform the following steps: determine the positive sample input based on the path formed by the vertices in the positive sample excluding the last vertex and the knowledge information of the target knowledge domain, and determine the last vertex in the positive sample as the positive sample output; construct a positive sample question-answer pair by combining the positive sample input and the positive sample output. For each negative sample, the following steps are performed: based on the path formed by the vertices in the negative sample excluding the last vertex and the knowledge information of the target knowledge domain, the negative sample input is determined, and the last vertex in the negative sample is determined as the negative sample output; the negative sample input and the negative sample output are used to construct a negative sample question-answer pair; The target question-answering model is trained using the positive sample question-answer pairs and the negative sample question-answer pairs.

6. The method according to claim 5, characterized in that, The knowledge information in the target knowledge domain includes multiple knowledge articles; Before training the target question-answering model using the positive sample question-answer pairs and the negative sample question-answer pairs, the method further includes: Perform at least one of the following modification operations: randomly add at least one knowledge article to the knowledge information of the target knowledge domain; adjust the order of the multiple knowledge articles in the knowledge information of the target knowledge domain; replace the content representing the titles of the multiple knowledge articles in the knowledge information of the target knowledge domain with semantically similar content; and replace the content representing the article content of the multiple knowledge articles in the knowledge information of the target knowledge domain with semantically similar content. Using the knowledge information of the target knowledge domain after performing the modification operation, new positive sample inputs and new negative sample inputs are determined; The new positive sample input and the positive sample output are used to construct a new positive sample question-answer pair, and the new negative sample input and the negative sample output are used to construct a new negative sample question-answer pair.

7. The method according to any one of claims 1 to 6, characterized in that, The step of using the target question-answering model, based on knowledge information from the target knowledge domain and the dialogue context corresponding to the nth round of sample generation, to answer the nth question and obtain the nth response includes: Generate a first prompt word; wherein the first prompt word includes: knowledge information of the target knowledge domain, dialogue context corresponding to the nth round of sample generation process, and prompt information for indicating the answer to the nth question; Send the first prompt word to the target question-answering model and receive the nth reply returned by the target question-answering model.

8. The method according to any one of claims 1 to 6, characterized in that, Since n is not 1, the (n-1)th follow-up question is generated in the following way: Generate a second prompt word; wherein the second prompt word includes: prompt information for instructing the simulated customer service role, a dialogue sequence consisting of n-1 questions and n-2 follow-up questions generated during the first n-1 rounds of sample generation, and prompt information for instructing the generation of follow-up questions; Send the second prompt to the customer service role model and receive the (n-1)th follow-up question returned by the customer service role model.

9. The method according to any one of claims 1 to 6, characterized in that, Since n is not 1, the nth question is generated in the following way: Generate a third prompt word; wherein the third prompt word includes: prompt information for indicating the simulated user role, a dialogue sequence consisting of n-1 questions and n-1 follow-up questions generated during the first n-1 rounds of sample generation, the first dialogue record, and prompt information for indicating the generation of questions; The third prompt word is sent to the user role model, and the nth question returned by the user role model is received.

10. A training device for a question-answering model, characterized in that, The device includes: The acquisition module is used to acquire a first dialogue record of the target knowledge domain; wherein, the first dialogue record includes a multi-round dialogue between a user role and a customer service role regarding a target question in the target knowledge domain, and the target response of the customer service role to the target question in the first dialogue record includes target knowledge information of the target knowledge domain; The generation module is used to perform N rounds of sample generation process based on the first dialogue record to obtain training data; The training module is used to train the target question-answering model using the training data; The nth round of sample generation process includes: using the target question-answering model, based on the knowledge information of the target knowledge domain and the dialogue context corresponding to the nth round of sample generation process, answering the nth question to obtain the nth response; 1≤n≤N; When n is 1, the nth question is the first question sent by the user role in the first dialogue record; when n is not 1, the nth question is a question generated by the user role model based on the (n-1)th follow-up question, which is generated by the customer service role model during the (n-1)th round of sample generation. When n is 1, the dialogue context corresponding to the nth round of sample generation is the nth question; when n is not 1, the dialogue context corresponding to the nth round of sample generation includes: a dialogue sequence consisting of n-1 questions and n-1 follow-up questions from the previous n-1 rounds of sample generation and the nth question. The condition for ending the sample generation process is that the Nth response includes at least part of the target knowledge information.

11. An electronic device, characterized in that, The electronic device includes a processor and a memory; The processor is configured to execute instructions stored in the memory, causing the electronic device to perform the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, Includes instructions that instruct an electronic device to perform the method as described in any one of claims 1 to 9.

13. A computer program product, characterized in that, The computer program product includes computer-readable instructions for implementing the method of any one of claims 1 to 9.

Images