Post-training method and device of question and answer large model, electronic equipment and storage medium

Abstract

The invention provides a post-training method and device for a question and answer large model, electronic equipment and a storage medium, and relates to the field of artificial intelligence. The method comprises the steps of firstly obtaining a plurality of reasoning data, and constructing a reasoning data set based on all reasoning data; and finally, performing post-training processing on the question and answer large model by utilizing the reasoning data set. Each piece of reasoning data is obtained by logically splitting an original question and an original answer, so that the reasoning data set can be used for guiding question and answer large model learning to perform step-by-step reasoning on the input question. According to the method, the large question and answer model can learn step-by-step reasoning on the question, so that the logical reasoning ability of the model is fundamentally improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a post-training method, device, electronic device and storage medium for a large question-answering model. Background Art

[0002] In the prior art, post-training is often used to make the model's answers more accurate. For example, typical methods are the RLHF (Reinforcement Learning from Human Feedback) method and the DPO (Direct Preference Optimization) method. However, the essence of the RLHF method and the DPO method is to enable the model to answer questions according to human preferences, that is, to make the model's answers close to human preferences, but these two methods do not substantially enhance the model's reasoning ability.

[0003] Therefore, how to increase the model's reasoning ability to make the model's answers more accurate is an urgent problem that needs to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a post-training method, device, electronic device and storage medium for a large question-answering model to improve the problems existing in the prior art.

[0005] The embodiments of the present invention can be implemented as follows: In a first aspect, the present invention provides a post-training method for a large question-answering model, comprising: Acquire some inference data, where the inference data is obtained by logically splitting the original question and its original answer; Based on all the inference data, construct an inference data set; The question-answering large model is post-trained using the reasoning data set; the reasoning data set is used to guide the question-answering large model to learn to perform step-by-step reasoning on input questions.

[0006] Optionally, the inference data set includes a first subset, a second subset, a third subset, and a fourth subset; and the step of constructing the inference data set based on all the inference data includes: For each piece of the inference data, based on the inference data, at least one first sample is obtained by constructing a rule using a progressive inference formula; the first subset includes all first samples corresponding to each piece of the inference data; For each piece of the inference data, based on the inference data, at least one second sample is obtained by using a progressive decomposition construction rule, wherein the second subset includes all second samples corresponding to each piece of the inference data; For each piece of the inference data, based on the inference data, at least one third sample is obtained by using an intermediate interception construction rule, wherein the third subset includes all third samples corresponding to each piece of the inference data; For each piece of the inference data, at least one fourth sample is obtained based on the inference data by using a step-by-step reasoning construction rule, and the fourth subset includes all fourth samples corresponding to each piece of the inference data.

[0007] Optionally, the inference data includes an original question, N sub-questions with progressive logical levels obtained by splitting the original question, and a sub-answer to each sub-question; the step of obtaining at least one first sample by constructing a rule with progressive inference based on the inference data includes: The first guiding text, the original question in the inference data, and the first sub-question are concatenated to obtain a first first sample, and the label of the first first sample is set to be the first sub-answer in the inference data; the first guiding text is used to instruct the question-answering model to infer the answer to the last sub-question in the input content based on the input content; The first guide text, the original question in the reasoning data, the nth sub-question, and each sub-question and its sub-answer before the nth sub-question are concatenated to obtain the nth first sample, and the label of the nth first sample is set to the nth sub-answer in the reasoning data. .

[0008] Optionally, the inference data includes an original question, N sub-questions with progressive logical levels obtained by splitting the original question, and a sub-answer to each sub-question; the step of obtaining at least one second sample based on the inference data by using a progressive decomposition construction rule includes: The second guide text, the original question in the inference data, and the first sub-question are concatenated to obtain a first second sample, and the label of the first second sample is set to the second sub-answer in the inference data; the second guide text is used to instruct the question-answering model to decompose the next sub-question of the last sub-question in the input content from the original question of the input content based on the input content; The second guide text, the original question in the reasoning data, the mth sub-question and each of the previous sub-questions are concatenated to obtain the mth second sample, and the label of the mth second sample is set to the m+1th sub-question in the reasoning data. .

[0009] Optionally, the inference data includes an original question, N sub-questions with progressive logical levels obtained by splitting the original question, and a sub-answer to each sub-question; the step of obtaining at least one third sample by using an intermediate interception construction rule based on the inference data includes: The third guiding text, each other sub-question and other sub-answers in the inference data except the first sub-question and the first sub-answer are concatenated to obtain a first third sample, and the label of the first third sample is set to the first sub-question and the first sub-answer in the inference data; the third guiding text is used to instruct the question-answering large model to infer the missing intermediate sub-questions and their intermediate sub-answers in the input content based on the input content; The third guide text, each other sub-question and other sub-answer in the reasoning data except the m-th sub-question and the m-th sub-answer are concatenated to obtain the m-th third sample, and the label of the m-th third sample is set to the m-th sub-question and the m-th sub-answer in the reasoning data. .

[0010] Optionally, the inference data includes an original question, N sub-questions with progressive logical levels obtained by splitting the original question, and a sub-answer to each sub-question; the step of obtaining at least one third sample by using an intermediate interception construction rule based on the inference data includes: The fourth guide text and the original question in the reasoning data are concatenated to obtain a first fourth sample, and a label of the first fourth sample is set to each sub-question and its sub-answer in the reasoning data; the fourth guide text is used to instruct the question-answering model to infer subsequent sub-questions and sub-answers step by step based on the input content; The fourth guide text, the original question in the reasoning data, each sub-question before the nth sub-question and its sub-answer are concatenated to obtain the nth fourth sample, and the label of the nth fourth sample is set to each sub-question after the n-1th sub-question in the reasoning data and its sub-answer, .

[0011] Optionally, the inference data set includes a first subset, a second subset, a third subset, and a fourth subset; The step of performing post-training processing on the question-answering large model using the inference data set includes: Performing post-training processing on the question-answering large model using the first subset, the second subset, the third subset, and the fourth subset respectively; Among them, the first subset is used to guide the question-answering big model to learn to answer the input question step by step, the second subset is used to guide the question-answering big model to learn to logically decompose the input question, the third subset is used to guide the question-answering big model to learn to complete the missing intermediate process based on step-by-step reasoning, and the fourth subset is used to guide the question-answering big model to learn to logically decompose the input question and then answer it step by step.

[0012] In a second aspect, the present invention further provides a post-training device for a large question-answering model, comprising: A data acquisition module, used to acquire a number of inference data, wherein the inference data is obtained by logically splitting the original question and its original answer; A construction module, used for constructing an inference data set based on all the inference data; A post-training module is used to perform post-training processing on the question-answering large model using the reasoning data set; the reasoning data set is used to guide the question-answering large model to learn to perform step-by-step reasoning on input questions.

[0013] In a third aspect, the present invention further provides an electronic device, comprising: a memory and a processor, wherein the memory stores a software program, and when the electronic device is running, the processor executes the software program to implement the post-training method of the question-answering large model as described in the first aspect.

[0014] In a fourth aspect, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the post-training method of the question-answering large model described in the first aspect is implemented.

[0015] Compared with the prior art, the embodiments of the present invention provide a post-training method, device, electronic device and storage medium for a large question-answering model. First, a number of inference data are obtained, and then an inference data set is constructed based on all the inference data; finally, the large question-answering model is post-trained using the inference data set; since each piece of inference data is obtained by logically splitting the original question and its original answer, the inference data set can be used to guide the large question-answering model to learn to perform step-by-step reasoning on input questions. The present invention enables the large question-answering model to learn to perform step-by-step reasoning on questions, thereby fundamentally improving the logical reasoning ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 One of the flow charts of a post-training method of a large question-answering model provided in an embodiment of the present invention.

[0018] Figure 2 The second flowchart of a post-training method for a large question-answering model provided in an embodiment of the present invention.

[0019] Figure 3A schematic structural diagram of a post-training device for a large question-answering model provided in an embodiment of the present invention.

[0020] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0022] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0024] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.

[0025] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0026] In the prior art, there are two ways to make the model's answers more accurate: The first is to enhance the reasoning ability of the model through post-training to make the model's answers more accurate. Typical methods are the RLHF method and the DPO method. Among them, the implementation principle of the RLHF method is: first, it is necessary to feed data to the model after fine-tuning the instructions, and after obtaining the answer to the data, manually label the preference data, such as the labeled data (x, ylose, ywin); then use the labeled preference data to train a reward model modelaward; in order to ensure that the distribution of the trained model is not too different from the distribution of the original model, a reference model modelrefer is also required; therefore, three models are needed in the training process. The DPO method is to fine-tune the model directly on the preference data set, without the need for a reward model and a reference model, which is a simplified way of the RLHF method.

[0027] The second method is to increase the inference time to make the model answer more accurately. The most common method is to link the model to an external knowledge base or search engine so that the model can answer questions using the acquired external knowledge as a reference. To link to an external knowledge base, the BGE (Bidirectional Generative Embedding) module is needed to segment and vectorize the content of the external knowledge base in advance so that the model can efficiently retrieve and use this knowledge.

[0028] For the first method mentioned above, it essentially just allows the model to answer questions based on human preferences, and does not substantially enhance the model's reasoning ability. In addition, the standard RLHF method requires three models at the same time during training, which has higher requirements for hardware.

[0029] The second method mentioned above only provides more references for the model, and does not substantially increase the reasoning ability of the model. In addition, the reasoning process needs to be connected to the bge module, which increases the extra reasoning time.

[0030] Therefore, how to make the model's answers more accurate by increasing the model's reasoning ability is an urgent problem that needs to be solved.

[0031] Based on the discovery of the above technical problems, the inventors have proposed the following technical solutions to solve or improve the above problems through creative work. It should be noted that the defects in the solutions in the above prior art are the results obtained by the inventors after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed in the embodiments of the present application for the above problems below should all be the contributions made by the inventors to the present application in the process of invention and creation, and should not be understood as technical contents known to those skilled in the art.

[0032] An embodiment of the present invention provides a post-training method for a large question-answering model, which can obtain inference data obtained by logically splitting an original question and its original answer, and use a number of such inference data to construct an inference data set, and then use the inference data set to post-train the large question-answering model, so that the large question-answering model gradually learns to perform step-by-step reasoning on the question during the post-training process, thereby fundamentally improving the logical reasoning ability of the model.

[0033] Please refer to Figure 1 , Figure 1 The present invention provides a flowchart of a post-training method for a large question-answering model. The execution subject of the method may be, but is not limited to, a computing device such as a smart phone, a personal notebook, a personal computer, a server, etc. The post-training method for the large question-answering model includes the following steps: S101. Obtain some inference data.

[0034] In this embodiment, each piece of reasoning data may be obtained by logically splitting an original question and its original answer.

[0035] S102: construct an inference data set based on all the inference data.

[0036] S103: Post-training the question-answering large model using the inference data set.

[0037] In this embodiment, the reasoning data set is used to guide the question-answering large model to learn to perform step-by-step reasoning on input questions. Therefore, in the post-training stage, the question-answering large model can gradually learn to perform step-by-step logical reasoning on input questions.

[0038] The post-training method of the question-answering big model adopted in the embodiment of the present invention first obtains a number of inference data, then constructs an inference data set based on all the inference data; finally, the question-answering big model is post-trained using the inference data set; since each inference data is obtained by logically splitting the original question and its original answer, the inference data set can be used to guide the question-answering big model to learn to perform step-by-step reasoning on the input question. The present invention enables the question-answering big model to learn to perform step-by-step reasoning on questions, thereby fundamentally improving the logical reasoning ability of the model.

[0039] Optionally, you can collect several original questions and their original answers, and then logically split each original question into multiple sub-questions and their sub-answers by manual splitting or using an additional logical splitting model. If a logical splitting model is used, it is necessary to manually verify the sub-questions and their sub-answers to ensure that there are no logical errors.

[0040] Therefore, each piece of the reasoning data may include an original question, N sub-questions with progressive logical levels obtained by splitting the original question, and sub-answers to each sub-question, wherein the Nth sub-answer in the reasoning data is the original answer to the original question.

[0041] In the optional example, suppose the original question is: What is the result of 1+3×9+9÷2? The original answer is 32.5. By splitting it, we can get the following 4 sub-questions and their sub-answers: The first sub-question Q1: What is the result of 3×9? The first sub-answer A1: 27; Second sub-question Q2: What is the result of 9÷2? Second sub-answer A2: 4.5; The third sub-question Q3: What is the result of 1+27? The third sub-answer A3: 28; The fourth sub-question Q4: What is the result of 28+4.5? The fourth sub-answer A4: 32.5.

[0042] This example is only an example of splitting a simple logical reasoning problem and is not limited here.

[0043] In an optional implementation, if the process of logical reasoning that the question-answering model learns to perform is split, then the question-answering model can learn from the shallower to the deeper: progressive problem decomposition, progressive sub-problem reasoning, and complete step-by-step logical reasoning. Moreover, in some scenarios, the logical problem that needs to be answered by the question-answering model may be: throwing out a complete question and some sub-problems of the complete question, and the intermediate missing questions and their answers need to be derived. Therefore, the reasoning data set constructed in the present invention may include four subsets, namely the first subset, the second subset, the third subset, and the fourth subset, and these four subsets are constructed in four different ways.

[0044] See also Figure 2 , that is, the process of "constructing an inference data set based on all the inference data" in the above step S102 may include the following sub-steps S1021~S1024.

[0045] S1021. For each piece of the inference data, based on the inference data, use a progressive inference formula to construct a rule to obtain at least one first sample.

[0046] In this embodiment, the first subset includes all first samples corresponding to each piece of the inference data, that is, each piece of inference data corresponds to at least one first sample in the first subset.

[0047] The first subset is obtained by the first construction method, and the first construction method adopts a progressive reasoning construction rule. Specifically, for each reasoning sample, the process of "obtaining at least one first sample based on the reasoning data by using a progressive reasoning construction rule" in step S1021 may include the following sub-steps S10211~S10212: S10211, concatenating the first guiding text, the original question in the reasoning data, and the first sub-question to obtain a first first sample, and setting a label of the first first sample to be the first sub-answer in the reasoning data; S10212, concatenating the first guiding text, the original question in the reasoning data, the nth sub-question, and each sub-question and its sub-answer before the nth sub-question to obtain an nth first sample, and setting the label of the nth first sample to the nth sub-answer in the reasoning data, .

[0048] In this embodiment, the preset first guiding text can be used to instruct the question-answering model to infer the answer to the last sub-question in the input content based on the input content. The actual text content of the first guiding text is not limited here.

[0049] S1022. For each piece of the inference data, based on the inference data, obtain at least one second sample using a progressive decomposition construction rule.

[0050] In this embodiment, the second subset includes all second samples corresponding to each piece of the inference data, that is, each piece of inference data corresponds to at least one second sample in the second subset.

[0051] The second subset is obtained by a second construction method, and the second construction method adopts a progressive disassembly construction rule. Specifically, for each inference sample, the process of "obtaining at least one second sample based on the inference data by using a progressive disassembly construction rule" in step S1022 may include the following sub-steps S10221~S10222: S10221, concatenating the second guiding text, the original question in the reasoning data, and the first sub-question to obtain a first second sample, and setting a label of the first second sample to be the second sub-answer in the reasoning data; S10222, concatenating the second guide text, the original question in the reasoning data, the mth sub-question and each of the sub-questions before it to obtain an mth second sample, and setting the label of the mth second sample to the m+1th sub-question in the reasoning data, .

[0052] In this embodiment, the preset second guide text can be used to instruct the question-answering model to decompose the next sub-question of the last sub-question in the input content from the original question of the input content based on the input content. The actual text content of the second guide text is not limited here.

[0053] S1023. For each piece of the inference data, based on the inference data, obtain at least one third sample using an intermediate interception construction rule.

[0054] In this embodiment, the third subset includes all third samples corresponding to each piece of the inference data, that is, each piece of inference data corresponds to at least one third sample in the third subset.

[0055] The third subset is obtained by a third construction method, and the third construction method adopts an intermediate interception construction rule. Specifically, for each inference sample, the process of "obtaining at least one third sample based on the inference data by using an intermediate interception construction rule" in step S1023 may include the following sub-steps S10231~S10232: S10231, concatenating the third guiding text, each other sub-question and other sub-answer in the reasoning data except the first sub-question and the first sub-answer to obtain a first third sample, and setting the label of the first third sample to the first sub-question and the first sub-answer in the reasoning data; S10232, concatenating the third guiding text, each other sub-question and other sub-answer in the reasoning data except the m-th sub-question and the m-th sub-answer to obtain an m-th third sample, and setting the label of the m-th third sample to the m-th sub-question and the m-th sub-answer in the reasoning data, .

[0056] In this embodiment, the third guiding text can be used to instruct the question-answering model to infer the missing intermediate sub-questions and their intermediate sub-answers in the input content based on the input content. The actual text content of the third guiding text is not limited here.

[0057] S1024. For each piece of the inference data, based on the inference data, use a step-by-step reasoning construction rule to obtain at least one fourth sample.

[0058] In this embodiment, the fourth subset includes all fourth samples corresponding to each piece of the inference data, that is, each piece of inference data corresponds to at least one fourth sample in the fourth subset.

[0059] The fourth subset is obtained by the fourth construction method, which adopts the step-by-step reasoning construction rule. Specifically, for each reasoning sample, the process of "obtaining at least one fourth sample based on the reasoning data by using the step-by-step reasoning construction rule" in step S1024 may include the following sub-steps S10241~S10242: S10241, concatenating the fourth guiding text and the original question in the reasoning data to obtain a first fourth sample, and setting a label of the first fourth sample to each sub-question and its sub-answer in the reasoning data; S10242, concatenating the fourth guiding text, the original question in the reasoning data, each sub-question before the nth sub-question and its sub-answer to obtain an nth fourth sample, and setting the label of the nth fourth sample to each sub-question after the n-1th sub-question in the reasoning data and its sub-answer, .

[0060] In this embodiment, the preset fourth guiding text can be used to instruct the question-answering model to infer subsequent sub-questions and sub-answers step by step based on the input content. The actual text content of the fourth guiding text is not limited here.

[0061] In the optional example, assume that a piece of reasoning data includes the original question, 5 sub-questions (Q1~Q5) and 5 sub-answers (A1~A5) corresponding to the 5 sub-questions. Then the reasoning data is processed using the four construction methods respectively, and the samples obtained are as follows: (1) Using the first construction method, we can obtain the five first samples shown in Table 1 below, where prompt1 represents the first guide text: Table 1

[0062] (2) Using the second construction method, we can obtain the four second samples shown in Table 2 below, where prompt2 represents the second guide text: Table 2

[0063] (3) Using the third construction method, we can obtain the four third samples shown in Table 3 below, where prompt3 represents the third guide text: Table 3

[0064] (4) Using the fourth construction method, we can obtain the five fourth samples shown in Table 4 below, where prompt4 represents the fourth guide text: Table 4

[0065] The above examples are only examples, and the embodiment of the present invention does not limit the number of sub-questions in a piece of reasoning data.

[0066] Based on the four subsets in the inference data set, the sub-steps of step S103 above may include S1031 or S1032: S1031. Perform post-training processing on the question-answering large model by using the first subset, the second subset, the third subset and the fourth subset respectively.

[0067] S1032. Use the second subset, the first subset, the fourth subset and the third subset in sequence to perform post-training processing on the question-answering large model.

[0068] Combining the contents of the four tables in the above examples, it can be seen that the first subset can be used to guide the question and answer big model to learn to answer the input question step by step, the second subset can be used to guide the question and answer big model to learn to logically decompose the input question, the third subset can be used to guide the question and answer big model to learn to complete the missing intermediate process based on step-by-step reasoning, and the fourth subset can be used to guide the question and answer big model to logically decompose the input question and then answer it step by step.

[0069] Therefore, the four subsets can be used in any order to post-train the large question-answering model. The second subset, the first subset, the fourth subset, and the third subset can also be used in sequence to post-train the large question-answering model, so that the large question-answering model can gradually learn: logical decomposition, step-by-step answers, step-by-step answers after logical decomposition, and completion of missing intermediate processes based on step-by-step reasoning.

[0070] Optionally, the post-training process may include supervised fine-tuning (SFT), instruction tuning, etc. The specific process of post-training is an existing technology and will not be described in detail here.

[0071] It should be noted that the execution order of each step in the above method embodiment is not limited to that shown in the drawings, and the execution order of each step shall be based on the actual application situation.

[0072] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: The present invention constructs four subsets through four construction methods to perform post-training processing on the large question-answering model, thereby substantially enhancing the model's reasoning ability; In the present invention, the first subset is used to guide the question-answering model to learn to answer the input question step by step, so that the question-answering model learns to answer the latest split sub-question based on the original question and the previously split sub-questions and their sub-answers; In the present invention, the second subset can be used to guide the question-answering large model to learn to logically decompose the input question, so that the question-answering large model learns to continue to decompose the next sub-question based on the original question and the decomposed sub-questions; In the present invention, the third subset can be used to guide the question-answering large model to learn to complete the missing intermediate process based on step-by-step reasoning, so that the question-answering large model can learn step-by-step reasoning to answer reasoning problems with missing intermediate steps; In the present invention, the fourth subset can be used to guide the question-answering model to learn to logically decompose the input question and then answer it step by step, so that the question-answering model can learn to completely perform step-by-step logical reasoning on the input question.

[0073] In order to execute the corresponding steps in the above method embodiment and each possible implementation method, an implementation method of a post-training device for a large question-answering model is given below.

[0074] See also Figure 3 , Figure 3 The schematic diagram of the structure of the post-training device of the question-answering large model provided by the embodiment of the present invention is shown. The post-training device 200 of the question-answering large model includes: a data acquisition module 210, a construction module 220 and a post-training module 230.

[0075] The data acquisition module 210 is used to acquire some inference data, where the inference data is obtained by logically splitting the original question and its original answer; A construction module 220, configured to construct an inference data set based on all the inference data; The post-training module 230 is used to perform post-training processing on the question-answering large model using the reasoning data set; the reasoning data set is used to guide the question-answering large model to learn to perform step-by-step reasoning on input questions.

[0076] Optionally, the inference data set includes a first subset, a second subset, a third subset, and a fourth subset. In the process of constructing an inference data set based on all the inference data, the construction module 220 can be specifically used to: for each piece of the inference data, based on the inference data, obtain at least one first sample using a progressive inference construction rule; the first subset includes all first samples corresponding to each piece of the inference data; for each piece of the inference data, based on the inference data, obtain at least one second sample using a progressive decomposition construction rule, and the second subset includes all second samples corresponding to each piece of the inference data; for each piece of the inference data, based on the inference data, obtain at least one third sample using an intermediate interception construction rule, and the third subset includes all third samples corresponding to each piece of the inference data; for each piece of the inference data, based on the inference data, obtain at least one fourth sample using a step-by-step inference construction rule, and the fourth subset includes all fourth samples corresponding to each piece of the inference data.

[0077] Optionally, the inference data includes the original question, N sub-questions with progressive logical levels obtained by splitting the original question, and sub-answers to each sub-question. In the process of obtaining at least one first sample based on the inference data and using progressive reasoning construction rules, the construction module 220 can be specifically used to: concatenate the first guide text, the original question in the inference data, and the first sub-question to obtain the first first sample, and set the label of the first first sample to the first sub-answer in the inference data; the first guide text is used to instruct the question-answering model to infer the answer to the last sub-question in the input content based on the input content; concatenate the first guide text, the original question in the inference data, the nth sub-question, and each sub-question and its sub-answer before the nth sub-question to obtain the nth first sample, and set the label of the nth first sample to the nth sub-answer in the inference data. .

[0078] Optionally, the reasoning data includes the original question, N sub-questions with progressive logical levels obtained by splitting the original question, and sub-answers to each sub-question. In the process of obtaining at least one second sample based on the reasoning data using progressive decomposition construction rules, the construction module 220 can be specifically used to: concatenate the second guide text, the original question in the reasoning data, and the first sub-question to obtain the first second sample, and set the label of the first second sample to the second sub-answer in the reasoning data; the second guide text is used to instruct the question-and-answer large model to decompose the next sub-question of the last sub-question in the input content from the original question of the input content based on the input content; concatenate the second guide text, the original question in the reasoning data, the mth sub-question and each sub-question before it to obtain the mth second sample, and set the label of the mth second sample to the m+1th sub-question in the reasoning data. .

[0079] Optionally, the inference data includes the original question, N sub-questions with progressive logical levels obtained by splitting the original question, and sub-answers of each sub-question. In the process of obtaining at least one third sample based on the inference data using the intermediate truncation construction rule, the construction module 220 can be specifically used to: splice the third guide text, each other sub-question and other sub-answer in the inference data except the first sub-question and the first sub-answer to obtain the first third sample, and set the label of the first third sample to the first sub-question and the first sub-answer in the inference data; the third guide text is used to instruct the question-answering large model to infer the missing intermediate sub-questions and their intermediate sub-answers in the input content based on the input content; splice the third guide text, each other sub-question and other sub-answer in the inference data except the mth sub-question and the mth sub-answer to obtain the mth third sample, and set the label of the mth third sample to the mth sub-question and the mth sub-answer in the inference data. .

[0080] Optionally, the inference data includes the original question, N sub-questions with progressive logical levels obtained by splitting the original question, and sub-answers of each sub-question. In the process of obtaining at least one third sample based on the inference data using the intermediate truncation construction rule, the construction module 220 can be specifically used to: concatenate the fourth guide text and the original question in the inference data to obtain the first fourth sample, and set the label of the first fourth sample to each sub-question and its sub-answer in the inference data; the fourth guide text is used to instruct the question-answering model to infer subsequent sub-questions and sub-answers step by step based on the input content; concatenate the fourth guide text, the original question in the inference data, each sub-question before the nth sub-question and its sub-answer to obtain the nth fourth sample, and set the label of the nth fourth sample to each sub-question and its sub-answer after the n-1th sub-question in the inference data, .

[0081] Optionally, the reasoning data set includes a first subset, a second subset, a third subset, and a fourth subset. In the process of using the reasoning data set to post-train the question-answering large model, the post-training module 230 can be specifically used to: use the first subset, the second subset, the third subset, and the fourth subset to post-train the question-answering large model; wherein the first subset is used to guide the question-answering large model to learn to answer the input question step by step, the second subset is used to guide the question-answering large model to learn to logically decompose the input question, the third subset is used to guide the question-answering large model to learn to complete the missing intermediate process based on step-by-step reasoning, and the fourth subset is used to guide the question-answering large model to learn to logically decompose the input question and then answer it step by step.

[0082] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the post-training device 200 of the large question-answering model described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0083] See also Figure 4 , Figure 4 The electronic device 300 includes a processor 310 , a memory 320 , and a bus 330 , wherein the processor 310 is connected to the memory 320 via the bus 330 .

[0084] The memory 320 can be used to store software programs, for example, software programs corresponding to the post-training device 200 of the question-answering large model provided in the embodiment of the present invention. The processor 310 executes various functional applications and data processing to implement the post-training method of the question-answering large model provided in the embodiment of the present invention by running the software programs stored in the memory 320.

[0085] Among them, the memory 320 can be but is not limited to: RAM (Random Access Memory), ROM (Read Only Memory), FLASH (Flash Memory), PROM (Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electric Erasable Programmable Read-Only Memory), etc.

[0086] The processor 310 may be an integrated circuit chip with signal processing capability. The processor 310 may be a general-purpose processor, including: CPU (Central Processing Unit), NP (Network Processor), SoC (System on Chip), etc.; it may also be: DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0087] Understandably, Figure 4 The structure shown is for illustration only. The electronic device 300 may also include Figure 4 More or fewer components as shown, or with Figure 4 Different configurations are shown. Figure 4 Each component shown in the figure can be implemented by hardware, software or a combination thereof.

[0088] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the post-training method of the large question-answering model disclosed in the above embodiment is implemented. The computer-readable storage medium can be, but is not limited to, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a PROM, an EPROM, an EEPROM, a FLASH disk, or an optical disk.

[0089] In summary, the embodiments of the present invention provide a post-training method, device, electronic device and storage medium for a large question-answering model. First, a number of inference data are obtained, and then an inference data set is constructed based on all the inference data; finally, the large question-answering model is post-trained using the inference data set; since each piece of inference data is obtained by logically splitting the original question and its original answer, the inference data set can be used to guide the large question-answering model to learn to perform step-by-step reasoning on input questions. The present invention enables the large question-answering model to learn to perform step-by-step reasoning on questions, thereby fundamentally improving the logical reasoning ability of the model.

[0090] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A post-training method for a large question-answering model, characterized in that: include: Acquire some inference data, where the inference data is obtained by logically splitting the original question and its original answer; For each piece of the inference data, based on the inference data, at least one first sample is obtained by using a progressive inference formula construction rule; the first subset includes all the first samples corresponding to each piece of the inference data; For each piece of the inference data, based on the inference data, at least one second sample is obtained by using a progressive decomposition construction rule; the second subset includes all the second samples corresponding to each piece of the inference data; For each of the inference data, based on the inference data, at least one third sample is obtained by using an intermediate interception construction rule; the third subset includes all the third samples corresponding to each of the inference data; For each of the inference data, based on the inference data, at least one fourth sample is obtained by using a step-by-step reasoning construction rule; the fourth subset includes all the fourth samples corresponding to each of the inference data; Post-training the question-answering model using the inference data set; The reasoning data set is used to guide the question-answering big model to learn to perform step-by-step reasoning on the input questions, and the reasoning data set includes the first subset, the second subset, the third subset and the fourth subset; the first subset is used to guide the question-answering big model to learn to answer the input questions step by step, the second subset is used to guide the question-answering big model to learn to logically decompose the input questions, the third subset is used to guide the question-answering big model to learn to complete the missing intermediate processes based on step-by-step reasoning, and the fourth subset is used to guide the question-answering big model to learn to logically decompose the input questions and then answer them step by step.

2. The method according to claim 1, characterized in that The reasoning data includes the original question, N sub-questions with progressive logical levels obtained by splitting the original question, and the sub-answers of each sub-question; The step of obtaining at least one first sample by constructing a rule based on the inference data using a progressive inference formula includes: The first guiding text, the original question in the inference data, and the first sub-question are concatenated to obtain a first first sample, and the label of the first first sample is set to be the first sub-answer in the inference data; the first guiding text is used to instruct the question-answering model to infer the answer to the last sub-question in the input content based on the input content; The first guide text, the original question in the reasoning data, the nth sub-question, and each sub-question and its sub-answer before the nth sub-question are concatenated to obtain the nth first sample, and the label of the nth first sample is set to the nth sub-answer in the reasoning data. .

3. The method according to claim 1, characterized in that The reasoning data includes the original question, N sub-questions with progressive logical levels obtained by splitting the original question, and the sub-answers of each sub-question; The step of obtaining at least one second sample based on the inference data by using a progressive decomposition construction rule comprises: The second guide text, the original question in the inference data, and the first sub-question are concatenated to obtain a first second sample, and the label of the first second sample is set to the second sub-answer in the inference data; the second guide text is used to instruct the question-answering model to decompose the next sub-question of the last sub-question in the input content from the original question of the input content based on the input content; The second guide text, the original question in the reasoning data, the mth sub-question and each of the previous sub-questions are concatenated to obtain the mth second sample, and the label of the mth second sample is set to the m+1th sub-question in the reasoning data. .

4. The method according to claim 1, characterized in that The reasoning data includes the original question, N sub-questions with progressive logical levels obtained by splitting the original question, and the sub-answers of each sub-question; The step of obtaining at least one third sample based on the inference data by using an intermediate interception construction rule comprises: The third guiding text, each other sub-question and other sub-answers in the inference data except the first sub-question and the first sub-answer are concatenated to obtain a first third sample, and the label of the first third sample is set to the first sub-question and the first sub-answer in the inference data; the third guiding text is used to instruct the question-answering large model to infer the missing intermediate sub-questions and their intermediate sub-answers in the input content based on the input content; The third guide text, each other sub-question and other sub-answer in the reasoning data except the m-th sub-question and the m-th sub-answer are concatenated to obtain the m-th third sample, and the label of the m-th third sample is set to the m-th sub-question and the m-th sub-answer in the reasoning data. .

5. The method according to claim 1, characterized in that The reasoning data includes the original question, N sub-questions with progressive logical levels obtained by splitting the original question, and the sub-answers of each sub-question; The step of obtaining at least one third sample based on the inference data by using an intermediate interception construction rule comprises: The fourth guide text and the original question in the reasoning data are concatenated to obtain a first fourth sample, and a label of the first fourth sample is set to each sub-question and its sub-answer in the reasoning data; the fourth guide text is used to instruct the question-answering model to infer subsequent sub-questions and sub-answers step by step based on the input content; The fourth guide text, the original question in the reasoning data, each sub-question before the nth sub-question and its sub-answer are concatenated to obtain the nth fourth sample, and the label of the nth fourth sample is set to each sub-question after the n-1th sub-question in the reasoning data and its sub-answer, .

6. The method according to claim 1, characterized in that The step of post-training the question-answering large model using the inference data set includes: The first subset, the second subset, the third subset and the fourth subset are respectively used to perform post-training processing on the question-answering large model.

7. A post-training device for a large question-answering model, characterized in that: include: A data acquisition module, used to acquire a number of inference data, wherein the inference data is obtained by logically splitting the original question and its original answer; Construction modules for: For each piece of the inference data, based on the inference data, at least one first sample is obtained by using a progressive inference formula construction rule; the first subset includes all the first samples corresponding to each piece of the inference data; For each piece of the inference data, based on the inference data, at least one second sample is obtained by using a progressive decomposition construction rule; the second subset includes all the second samples corresponding to each piece of the inference data; For each of the inference data, based on the inference data, at least one third sample is obtained by using an intermediate interception construction rule; the third subset includes all the third samples corresponding to each of the inference data; For each of the reasoning data, based on the reasoning data, at least one fourth sample is obtained by using a step-by-step reasoning construction rule; the fourth subset includes all the fourth samples corresponding to each of the reasoning data; A post-training module, used to perform post-training processing on the question-answering large model using an inference data set; The reasoning data set is used to guide the question-answering big model to learn to perform step-by-step reasoning on the input questions, and the reasoning data set includes the first subset, the second subset, the third subset and the fourth subset; the first subset is used to guide the question-answering big model to learn to answer the input questions step by step, the second subset is used to guide the question-answering big model to learn to logically decompose the input questions, the third subset is used to guide the question-answering big model to learn to complete the missing intermediate processes based on step-by-step reasoning, and the fourth subset is used to guide the question-answering big model to learn to logically decompose the input questions and then answer them step by step.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores a software program, and when the electronic device is running, the processor executes the software program to implement the post-training method of the question-answering large model as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the post-training method of the question-answering large model described in any one of claims 1-6.

Images