Training method for problem-solving model, method for problem-solution, and device

The method enhances the accuracy of large language models by refining the training process with sample questions, answer templates, and supervised fine-tuning, addressing the low accuracy issue in existing LLMs.

JP2024156990AActive Publication Date: 2024-11-06BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2024135383
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-12-20
Filing Date
2024-08-14
Publication Date
2024-11-06
Estimated Expiration
2044-08-14

AI Technical Summary

Technical Problem

Large Language Models (LLMs) suffer from low accuracy in answering problems due to the inconsistency and inaccuracy of pre-training data, leading to suboptimal performance in generating answers.

Method used

A method involving pre-training a problem-solving model by using a combination of a large-scale language model and a step planning model, utilizing sample questions, answer capture templates, and data evaluation templates to refine the training process, including supervised fine-tuning to enhance accuracy.

Benefits of technology

Improves the accuracy of answer generation by ensuring high-quality pre-training data and fine-tuning based on question types, resulting in a more precise problem-solving model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024156990000001_ABST
    Figure 2024156990000001_ABST
Patent Text Reader

Abstract

To provide a training method for a problem-solving model, a device and a method for problem solution.SOLUTION: A training method for a problem-solving model comprises the steps of: acquiring a first sample problem; inputting the first sample problem and a solution step capture template into a large-scale language model to acquire a first sample solution step; inputting the first sample problem, the first sample solution step and a reply solution capture template into the large-scale language model to acquire the first sample reply; pre-training, on the basis of the first sample problem and the first sample solution step, a step planning model; pre-training, on the basis of the first sample problem, the first sample solution step, and the first sample solution reply, a large-scale language model; and acquiring, on the basis of the pre-trained step planning model and the large-scale language model, the problem-solving model.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to the field of computer technology, in particular to the field of manual intelligence technology such as large models, natural language processing, deep learning, etc. A method, an apparatus, an electronic device, and a readable storage medium for training a problem-solving model and solving a problem are provided. [Background technology]

[0002] A large language model (LLM) is a deep learning model obtained by training using a large amount of text data, and can generate natural language text or understand the meaning of natural language text. A large language model has a certain inference ability, and can solve input questions and obtain answers to the questions. However, the large language model in the conventional technology has a problem that the accuracy of the obtained answers is low when solving questions by inference. Summary of the Invention [Means for solving the problem]

[0003] According to a first aspect of the present disclosure, a method for training a problem-answering model is provided, comprising: obtaining a first sample problem; inputting the first sample problem and an answer step capture template into a large-scale language model and obtaining a first sample answer step output by the large-scale language model; inputting the first sample problem, the first sample answer step and an answer capture template into the large-scale language model and obtaining a first sample answer output by the large-scale language model; pre-training a step plan model based on the first sample problem and the first sample answer step; pre-training the large-scale language model based on the first sample problem, the first sample answer step and the first sample answer; and obtaining a problem-answering model based on the pre-trained step plan model and the large-scale language model.

[0004] According to a second aspect of the present disclosure, a problem-answering method is provided, comprising the steps of: obtaining a problem to be answered; inputting the problem to be answered into a step planning model in a problem-answering model and obtaining answer steps output by the step planning model; and inputting the problem to be answered and the answer steps into a large-scale language model in the problem-answering model and obtaining an answer output by the large-scale language model.

[0005] According to a third aspect of the present disclosure, there is provided a training apparatus for a problem-answering model, comprising: a first obtaining unit for obtaining a first sample problem; a first processing unit for inputting the first sample problem and an answer step capture template into a large-scale language model and obtaining a first sample answer step output by the large-scale language model; a second processing unit for inputting the first sample problem, the first sample answer step and an answer capture template into the large-scale language model and obtaining a first sample answer output by the large-scale language model; a first pre-training unit for pre-training a step planning model based on the first sample problem and the first sample answer step; a second pre-training unit for pre-training the large-scale language model based on the first sample problem, the first sample answer step and the first sample answer; and a construction unit for obtaining a problem-answering model based on the pre-trained step planning model and the large-scale language model.

[0006] According to a fourth aspect of the present disclosure, a problem solving device is provided, comprising: a second acquisition unit for acquiring a problem to be answered; a first answering unit for inputting the problem to be answered into a step planning model in a problem answering model and acquiring answer steps output by the step planning model; and a second answering unit for inputting the problem to be answered and the answer steps into a large-scale language model in the problem answering model and acquiring an answer output by the large-scale language model.

[0007] According to a fifth aspect of the present disclosure, there is provided an electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions executable by the at least one processor that, when executed by the at least one processor, cause the at least one processor to perform the method.

[0008] According to a sixth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium having computer instructions stored thereon, the computer instructions causing a computer to perform the method.

[0009] According to a seventh aspect of the present disclosure there is provided a computer program product, comprising a computer program which, when executed by a processor, implements the method.

[0010] It should be understood that the contents described herein are not intended to identify key or important features of the embodiments of the present disclosure, nor should they be used to limit the scope of the present disclosure. Other features of the present disclosure can be readily understood through the following specification. [Brief description of the drawings]

[0011] The drawings are for a better understanding of the present application and are not intended to limit the present application. [Figure 1] FIG. 1 is a schematic diagram according to a first embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram according to a second embodiment of the present disclosure. [Diagram 3] FIG. 11 is a schematic diagram according to a third embodiment of the present disclosure. [Figure 4] FIG. 11 is a schematic diagram according to a fourth embodiment of the present disclosure. [Diagram 5] FIG. 13 is a schematic diagram according to a fifth embodiment of the present disclosure. [Figure 6] FIG. 2 is a block diagram of an electronic device for implementing a method for training a problem-solving model or a method for solving a problem according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, exemplary embodiments of the present application will be described based on the drawings. For ease of understanding, various details of the embodiments of the present application are included, and they should be regarded as merely examples. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of simplicity, the following description omits the description of well-known functions and structures.

[0013] 1 is a schematic diagram according to a first embodiment of the present disclosure. As shown in FIG. 1, the training method for the problem-solving model of this embodiment specifically includes the following steps: S101, obtain the first sample question; S102, inputting the first sample question and the answer step capture template into a large-scale language model, and obtaining a first sample answer step output by the large-scale language model; S103, inputting the first sample question, the first sample answer step and an answer capture template into the large-scale language model, and obtaining a first sample answer output by the large-scale language model; S104, pre-training a step planning model according to the first sample problem and the first sample answer step; S105, pre-training the large-scale language model based on the first sample problem, the first sample answer, and the first sample answer; S106, obtaining a problem-solving model based on the pre-trained step planning model and the large-scale language model.

[0014] According to one aspect, the training method for the problem-answering model in this embodiment uses a large-scale language model to obtain a first sample answer step and a first sample answer respectively based on a first sample problem, a preset answer step capture template, and an answer capture template, thereby realizing the purpose of obtaining pre-training data based on the large-scale language model and reducing the cost of obtaining pre-training data; according to another aspect, pre-training is performed on the large-scale language model in combination with the first sample answer step corresponding to the first sample problem in the pre-training data, so that the large-scale language model can generate an answer corresponding to the problem based on the connection between the problem and the answer step, thereby improving the pre-training effect of the large-scale language model, and enabling the large-scale language model in the problem-answering model to generate more accurate answers.

[0015] In this embodiment, when executing S101, questions extracted from the Internet can be used as first sample questions, and this embodiment does not limit the number of first sample questions obtained, which may be one or multiple.

[0016] In this embodiment, when S101 is performed to obtain a first sample question, an answer corresponding to the first sample question can also be obtained.

[0017] In this embodiment, S101 is executed to obtain a first sample question, and then S102 is executed to input the first sample question and the answer step capture template into a large-scale language model to obtain a first sample answer step output by the large-scale language model, and the obtained first sample answer step in this embodiment is used to represent the answer logic when answering the first sample question.

[0018] In this embodiment, the answer step capture template is preset and used to input into the large-scale language model as a prompt, and the large-scale language model combines the answer step capture template to output a corresponding first sample answer step based on the input first sample question.

[0019] For example, the answer step capture template of this embodiment assumes that you are a reasoning master and is given a problem, you can extract the answer steps corresponding to the problem. Without providing too much detail, it may be that you should pay attention to the core logic. Below are some examples of the task: (Problem): The children are playing games because they are bored. Who does "they" in the sentence refer to? A, children; B, game; (Answer steps): 1, Analyze the sentence and find the name corresponding to the possible pronoun; 2, Consider the syntactic structure and semantic logic respectively and determine the possible pronoun; 3, Integrate the syntactic structure and semantic logic to give the final answer.

[0020] When executing S102 in this embodiment, the first sample question and the above-mentioned answer step capture template are input together into a large-scale language model, and the large-scale language model combines the answer step capture template to output a first sample answer step corresponding to the first sample question.

[0021] It should be understood that when the present embodiment executes S101 to obtain multiple first sample questions, and the present embodiment executes S102, first sample answer steps corresponding to each first sample question can be obtained respectively based on the answer step capture template.

[0022] For example, if the first sample problem is "Forest rangers need to move from the east to the west of the forest with pythons, squirrels, and pine cones, and each time they take one of each; if there are no forest rangers, the pythons will eat the squirrels, and the squirrels will eat the pine cones; how to ensure their safety?", when the embodiment executes S102, the first sample problem and the above-mentioned answer step capture template are input together into the large-scale language model, and the output result of the large-scale language model is "1. Analyze the mutually exclusive relationships between each object, and determine the combinations of objects that cannot co-exist; 2. Select a movement strategy that can achieve the goal without violating the mutually exclusive relationships, and list the specific operation steps", and the output result is the first sample answer step corresponding to the first sample problem.

[0023] In this embodiment, S102 is performed to obtain a first sample answer step corresponding to the first sample question, and then S103 is performed to input the first sample question, the first sample answer step and the answer capture template into the large-scale language model to obtain a first sample answer output by the large-scale language model, and the first sample answer in this embodiment includes an answer corresponding to the answer step of each step.

[0024] In this embodiment, the answer capture template is preset and used to input into the large-scale language model as a prompt, and the large-scale language model combines the answer capture template and outputs a first sample answer corresponding to the first sample question based on the input first sample question and first sample answer steps.

[0025] For example, the answer capture template of this embodiment asks you to be a reasoning master and answer the problem according to the logic of the answer steps, given the answer steps of the problem;Specifically, the requirements are as follows:1, organize the logic of the answer according to the core logic of the given answer steps;2, do not use the title where the related answer steps appear, organize the answer in a reasonable structured manner, and pay attention not to omit important logic when answering, and answer in detail;3, if there is a serious error in the answer steps, answer according to your own answer steps based on your own understanding.The following are some examples of the task:(Question): Children are playing games because they are bored. Who does "they" in the sentence refer to? A. Children, B. Games;(Answer steps): 1. Analyze the sentence and find names that correspond to possible pronouns; 2. Consider the syntactic structure and semantic logic respectively and determine the possible pronouns; 3. Integrate the syntactic structure and semantic logic to give the final answer:(Answer): In the sentence "The children are playing games because they are bored", the word "they" is used to replace certain or some nouns mentioned above to avoid redundancy; In the context of this sentence, there are two nouns, children and games; 1. The first First, we should consider syntactic structure; "because they are bored" is a subordinate sentence, which is used to explain the cause of "the children are playing games", and "they" is likely to replace the subject in the main sentence, i.e., "children"; second, we should consider semantic logic; generally, games themselves are not boring, but people feel bored, so "they" should refer to "children"; by integrating syntactic structure and semantic logic, we can determine that "they" in this sentence refers to "children", so the answer is A, children.

[0026] When executing S103 in this embodiment, the first sample question, the first sample answer step, and the above-mentioned answer capture template are input together into a large-scale language model, and the large-scale language model combines the answer capture template to output a first sample answer corresponding to the first sample answer step.

[0027] It should be understood that when the present embodiment executes S101 to obtain multiple first sample questions, and the present embodiment executes S103, first sample answers corresponding to each first sample question can be obtained respectively based on the answer capture template.

[0028] For example, if the first sample question is “Forest rangers need to move from the east to the west of the forest with pythons, squirrels, and pine cones, and each time they need to take one of each; if there are no forest rangers, the pythons will eat the squirrels, and the squirrels will eat the pine cones; how to ensure their safety?” and the first sample answer steps are “1, analyze the mutually exclusive relationships between the objects, and determine the combinations of objects that cannot co-exist; 2, select a movement strategy that can achieve the goal without violating the mutually exclusive relationships, and list the specific operation steps”, when the embodiment executes S103, the first sample question, the first sample answer steps, and the answer capture template are generated in a unified manner. Together, these are input into a large-scale language model, and the output result of the large-scale language model may be "first, analyze the mutual exclusive relationship between the python, the squirrel, and the pine cone: the python and the squirrel cannot remain together alone, and the squirrel and the pine cone cannot remain together alone"; therefore, in order to ensure that everything reaches the western forest safely: 1, first, the forest ranger takes the squirrel to the western forest; 2, then the forest ranger comes back and takes the python; 3, because the python and the squirrel cannot remain together alone, the forest ranger brings the squirrel back; next, the forest ranger takes the pine cone to the western forest; finally, the forest ranger comes back and takes the squirrel to the western forest.

[0029] In this embodiment, after performing S103 to obtain the first sample answer, the triplet consisting of the first sample question, the first sample answer step corresponding to the first sample question, and the first sample answer can be directly used as pre-training data, and the pre-training data can be used to pre-train the step planning model and the large-scale language model.

[0030] However, the accuracy of the ternary sets obtained by the large-scale language model cannot be guaranteed, and if the accuracy of the ternary sets used as pre-training data is low, it will reduce the pre-training effect of the step planning model and the large-scale language model, and further, the accuracy of the answer step given answer output by the finally obtained problem-answering model may be low.

[0031] Therefore, this embodiment may further include a step of inputting the first sample question, the first sample answer step, the first sample answer, and the data evaluation template into a large-scale language model and obtaining a data evaluation result output by the large-scale language model, and the data evaluation result obtained in this embodiment may be one of evaluation pass and evaluation fail, or may be an evaluation score, and if it is determined that the obtained data evaluation result meets the predetermined requirement (it is determined that the data evaluation result is evaluation pass, or the evaluation score exceeds the predetermined score threshold), the first sample question, the first sample answer step, and the first sample answer are pre-training data.

[0032] In other words, in this embodiment, data evaluation is performed on a three-tuple constructed using a large-scale language model using a preset data evaluation template, and only the three-tuple that passes the data evaluation is used as pre-training data, thereby improving the accuracy of the acquired pre-training data, and further improving the accuracy of pre-training performed on the step planning model and the large-scale language model using the acquired pre-training data.

[0033] In this embodiment, the data evaluation template is preset and is used to input a prompt into the large-scale language model, and the large-scale language model combines the data evaluation template and outputs a data evaluation result based on the input first sample question, first sample answer step, and first sample answer.

[0034] For example, the data evaluation template in this embodiment is: You are a reasoning master. Given a given problem, answer steps and answer, you can obtain a data evaluation result corresponding to the given problem, answer steps and answer. The data evaluation result may be whether the evaluation is passed or not, and may be an evaluation score. The following are some examples of the task: (Problem): The children are playing games because they are bored. Who does "they" in the sentence refer to? A, children, B, game; (Answer steps): 1, Analyze the sentence and find the name corresponding to the possible pronoun; 2, Consider the syntactic structure and semantic logic respectively and determine the possible pronoun; 3, Integrate the syntactic structure and semantic logic to give the final answer; (Answer): In the sentence "The children are playing games because they are bored", the word "they" is used to replace certain or some nouns mentioned above to avoid duplication; In the context of this sentence, there are two nouns, children and games; 1, The first thing to consider is the structure It is the sentence structure; "because they are bored" is a subordinate sentence, which is used to explain the cause of "the children are playing games", and "they" is likely to replace the subject in the main sentence, i.e., "children"; 2. The second thing to consider is semantic logic; generally, games themselves are not boring, but people feel bored, and "they" should refer to "children"; by integrating the syntactic structure and semantic logic, it can be determined that "they" in this sentence refers to "children", so the answer is A, children; (Data evaluation result): The evaluation is passed.

[0035] If it is determined that the data evaluation result obtained in this embodiment does not meet the preset requirements, the first sample question, the first sample answer step, and the first sample answer to be evaluated this time are discarded.

[0036] In this embodiment, when it is determined that the data evaluation result obtained by executing S103 satisfies the preset requirements, the first sample problem, the first sample answer step, and the first sample answer are used as pre-training data. This embodiment further includes a step of inputting the first sample problem into a data generation model and obtaining a candidate answer step and / or a candidate answer output by the data generation model, where the data generation model in this embodiment is obtained by pre-training, and is capable of outputting an answer step and / or an answer corresponding to the problem based on the input problem. In the case where it is determined that the candidate answer step is similar to the first sample answer step (the similarity between the two is equal to or greater than a preset similarity threshold) and / or the candidate answer is similar to the first sample answer (the similarity between the two is equal to or greater than a preset similarity threshold), the first sample problem, the first sample answer step, and the first sample answer are used as pre-training data.

[0037] That is, the present embodiment can further use a large-scale language model to initially filter the obtained triplet, and then use a pre-trained data generation model to finally filter the triplet, thereby further improving the accuracy of the obtained pre-trained data.

[0038] In this embodiment, when executing S103, if it is determined that the candidate answer step is not similar to the first sample answer step, and / or the candidate answer is not similar to the first sample answer, the first sample question, the first sample answer step, and the first sample answer used this time can be discarded, and the answer step and answer corresponding to the first sample question can also be obtained using a manually labeled scheme.

[0039] In addition, when executing S103, this embodiment can also set a common rule to discard any ternary pair that does not satisfy the common rule. For example, the common rule can include a preset number of steps, and further, if it is determined that the number of steps in the first sample answer step is less than the preset number of steps, it is determined that the ternary pair does not satisfy the common rule and is therefore discarded.

[0040] In this embodiment, after executing S103 to obtain the first sample answer, S104 and S105 can be executed respectively, thereby realizing pre-training for the step planning model and the large-scale language model, and this embodiment is not limited to the execution order of S104 and S105.

[0041] In this embodiment, when executing S104 to perform pre-training on the step planning model based on the first sample problem and the first sample answer step, the first sample problem is first input into the step planning model, a first predicted answer step is obtained by the step planning model, a first loss function value is obtained based on the first sample answer step and the first predicted answer step, and the parameters of the step planning model are adjusted based on the obtained first loss function value to obtain the step planning model after pre-training.

[0042] In this embodiment, when executing S105 to pre-train the large-scale language model based on the first sample question, the first sample answer step, and the first sample answer, the first sample question and the first sample answer step are first input into the large-scale language model, a first predicted answer is obtained by the large-scale language model, a second loss function value is obtained based on the first sample answer and the first predicted answer, and the parameters of the large-scale language model are adjusted based on the obtained second loss function value to obtain a large-scale language model after pre-training.

[0043] In this embodiment, S104 is executed to perform pre-training on the step planning model, S105 is executed to perform pre-training on the large-scale language model, and then S106 is executed to obtain a problem-solving model based on the pre-trained step planning model and the large-scale language model.

[0044] When executing S106, this embodiment can directly obtain the step planning model and the large-scale language model after pre-training to construct a problem-answering model. After the constructed problem-answering model obtains an input problem, it first inputs the obtained input problem into the step planning model to obtain the answer steps output by the step planning model, and then inputs the obtained input problem and answer steps into the large-scale language model to obtain the answer output by the large-scale language model.

[0045] In this embodiment, when performing S106 to obtain a problem-answering model based on the pre-trained step planning model and the large-scale language model, the method may further include the steps of obtaining a second sample problem and determining a problem type of the second sample problem, and obtaining an answer step corresponding to the determined problem type as a second sample answer step of the second sample problem, where the answer steps corresponding to different problem types in this embodiment are obtained by a pre-labeling method, performing supervised fine-tuning (SFT) on the pre-trained step planning model based on the second sample problem and the second sample answer step, and obtaining a problem-answering model based on the pre-trained large-scale language model and the step planning model obtained by supervised fine-tuning.

[0046] That is, this embodiment further improves the training effect of the step planning model by performing supervised fine-tuning on the step planning model, and further obtains a problem answering model based on the step planning model obtained by supervised fine-tuning. This embodiment can obtain answer steps based on the problem type of the sample problem, and if the problem answering models are for the same type of problem, it can output answer steps that are similar in number or logic.

[0047] In this embodiment, when performing S106 to obtain a problem-answering model based on the pre-trained step planning model and the large-scale language model, the embodiment may further include the steps of obtaining a second sample problem and determining a problem type of the second sample problem, obtaining an answer step corresponding to the determined problem type as a second sample answer step of the second sample problem, determining an answer step type of the second sample answer step and obtaining an answer corresponding to the determined answer step type as a second sample answer of the second sample problem, performing supervised fine-tuning (SFT) on the pre-trained large-scale language model based on the second sample problem, the second sample answer step, and the second sample answer, and obtaining a problem-answering model based on the pre-trained step planning model and the large-scale language model obtained by supervised fine-tuning.

[0048] That is, the present embodiment further improves the training effect of the large-scale language model by performing supervised fine-tuning on the large-scale language model, and further obtains a problem answer model based on the large-scale language model obtained by supervised fine-tuning. The present embodiment can obtain an answer based on the answer type of the sample answer step, and if the problem answer model is for the same type of answer step, it can output an answer that is similar in number or logic.

[0049] It should be understood that this embodiment can further perform supervised fine-tuning on the pre-trained step planning model and the large-scale language model at the same time when executing S106, and further obtain a problem-solving model based on the step planning model and the large-scale language model obtained by supervised fine-tuning.

[0050] The training process of the problem answering model in this embodiment is taken as an example to explain. A first sample problem is obtained. In this embodiment, a first sample problem in text format can be obtained. Furthermore, after performing voice recognition on the obtained problem in voice format, the recognition result can also be the first sample problem. The obtained first sample problem and the answer step capture template in text format are input into a large-scale language model to obtain a first sample answer step in text format output by the large-scale language model. The first sample problem, the first sample answer step, and the answer capture template in text format are input into a large-scale language model to obtain a first sample answer in text format output by the large-scale language model. According to the first sample problem and the first sample answer step in text format, a step planning model is pre-trained. According to the first sample problem, the first sample answer step, and the first sample answer in text format, a large-scale language model is pre-trained. According to the step planning model and the large-scale language model obtained after pre-training, a problem answering model is obtained.

[0051] 2 is a schematic diagram according to a second embodiment of the present disclosure. FIG. 2 shows a configuration diagram of a problem-solving model of the embodiment. The problem-solving model of the embodiment includes a step planning model and a large-scale language model, the step planning model is used to output answer steps based on an input problem, the large-scale language model is used to output an answer corresponding to the input problem based on the input problem and the answer steps output by the step planning model, the output answer may include multiple sub-answers, and each sub-answer corresponds to one answer step.

[0052] 3 is a schematic diagram according to the third embodiment of the present disclosure. As shown in FIG. 3, the problem solving method of this embodiment specifically includes the following steps: S301, obtain the questions to be answered; S302, inputting the problem to be solved into a step planning model in a problem solving model, and obtaining the answer steps output by the step planning model; S303, the problem to be solved and the solving steps are input into a large-scale language model in the problem-solving model, and an answer output by the large-scale language model is obtained.

[0053] In other words, in this embodiment, by combining a pre-trained problem-answering model to first obtain the answer steps of the problem to be answered, and then obtaining an answer based on the obtained answer steps, the accuracy of the answer corresponding to the obtained problem to be answered can be improved, and the performance of the large-scale language model in the problem-answering model when answering complex inference problems can be improved.

[0054] The process of answering a problem in this embodiment will be taken as an example to explain how to obtain a problem to be answered. In this embodiment, the problem to be answered in text format input from the input side can be obtained. Furthermore, after performing voice recognition on the problem in audio format input from the input side, the recognition result can also be used as the problem to be answered. The problem to be answered is input into a step planning model in the problem answering model, the answer steps in text format output by the step planning model are obtained, the problem to be answered and the answer steps in text format are input into a large-scale language model in the problem answering model, the answer in text format output by the large-scale language model is obtained as the answer result corresponding to the problem to be answered, and the obtained answer is returned to the input side and displayed to the user.

[0055] 4 is a schematic diagram according to a fourth embodiment of the present disclosure. As shown in FIG. 4, a training device 400 for a problem-solving model in this embodiment includes a first acquisition unit 401, a first processing unit 402, a second processing unit 403, a first pre-training unit 404, a second pre-training unit 405, and a construction unit 406; The first acquisition unit 401 is used to acquire a first sample question; The first processing unit 402 is used for inputting the first sample problem and the answer step capture template into a large-scale language model, and obtaining a first sample answer step output by the large-scale language model; a second processing unit 403 is used for inputting the first sample question, the first sample answer step and the answer capture template into the large-scale language model, and obtaining a first sample answer output by the large-scale language model; A first pre-training unit 404 is used for pre-training a step planning model according to the first sample problem and the first sample solution step; A second pre-training unit 405 is used to pre-train the large-scale language model based on the first sample problem, the first sample answer step, and the first sample answer; The construction unit 406 is used to obtain a problem-solving model based on the pre-trained step planning model and the large-scale language model.

[0056] The first acquisition unit 401 can take questions extracted from the Internet as first sample questions, and this embodiment does not limit the number of first sample questions acquired, which may be one or multiple.

[0057] When the first obtaining unit 401 obtains a first sample question, it can also obtain an answer corresponding to the first sample question.

[0058] In this embodiment, after the first acquisition unit 401 acquires the first sample question, the first processing unit 402 inputs the first sample question and the answer step capture template into the large-scale language model to acquire the first sample answer step output by the large-scale language model; and the first sample answer step acquired by the first processing unit 402 is used to represent the answer logic when answering the first sample question.

[0059] In this embodiment, the answer step capture template is preset and used to input into the large-scale language model as a prompt, and the large-scale language model combines the answer step capture template to output a corresponding first sample answer step based on the input first sample question.

[0060] The first processing unit 402 jointly inputs the first sample problem and the above-mentioned answer step capture template into the large-scale language model, and the large-scale language model combines the answer step capture template to output a first sample answer step corresponding to the first sample problem.

[0061] It should be understood that when multiple first sample questions are acquired, the first processing unit 402 can acquire first sample answer steps corresponding to each first sample question respectively based on the answer step capture template.

[0062] After the first processing unit 402 in this embodiment obtains the first sample answer step corresponding to the first sample problem, the second processing unit 403 inputs the first sample problem, the first sample answer step and the answer capture template into the large-scale language model, and obtains the first sample answer output by the large-scale language model, and the first sample answer obtained by the second processing unit 403 includes an answer corresponding to the answer step of each step.

[0063] In this embodiment, the answer capture template is preset and used to input into the large-scale language model as a prompt, and the large-scale language model combines the answer capture template and outputs a first sample answer corresponding to the first sample question based on the input first sample question and first sample answer steps.

[0064] The second processing unit 403 inputs the first sample question, the first sample answer step, and the above-mentioned answer capture template together into a large-scale language model, and the large-scale language model combines the answer capture template to output a first sample answer corresponding to the first sample answer step.

[0065] It should be understood that when multiple first sample questions are acquired, the second processing unit 403 can acquire first sample answers corresponding to each first sample question respectively based on the answer capture template.

[0066] After obtaining the first sample answer, the second processing unit 403 can directly take the triplet consisting of the first sample question, the first sample answer step corresponding to the first sample question, and the first sample answer as pre-training data, and further use the pre-training data to pre-train the step planning model and the large-scale language model.

[0067] However, the accuracy of the ternary sets obtained by the large-scale language model cannot be guaranteed, and if the accuracy of the ternary sets used as pre-training data is low, it will reduce the pre-training effect of the step planning model and the large-scale language model, and further, the accuracy of the answer step given answer output by the finally obtained problem-answering model may be low.

[0068] Therefore, after obtaining the first sample answer, the second processing unit 403 may further include the following content: input the first sample question, the first sample answer step, the first sample answer, and the data evaluation template into the large-scale language model, obtain the data evaluation result output by the large-scale language model, and if it is determined that the obtained data evaluation result meets the predetermined requirements, the first sample question, the first sample answer step, and the first sample answer are pre-training data.

[0069] In other words, the second processing unit 403 performs data evaluation on the ternary pairs constructed using the large-scale language model according to a preset data evaluation template, and only the ternary pairs that pass the data evaluation are used as pre-training data, thereby improving the accuracy of the acquired pre-training data, and further improving the accuracy of pre-training performed on the step planning model and the large-scale language model using the acquired pre-training data.

[0070] In this embodiment, the data evaluation template is preset and is used to input a prompt into the large-scale language model, and the large-scale language model combines the data evaluation template and outputs a data evaluation result based on the input first sample question, first sample answer step, and first sample answer.

[0071] If it is determined that the data evaluation result obtained in this embodiment does not meet the preset requirements, the first sample question, the first sample answer step, and the first sample answer to be evaluated this time are discarded.

[0072] When the second processing unit 403 determines that the obtained data evaluation result meets the predetermined requirements, when the first sample question, the first sample answer step, and the first sample answer are taken as pre-training data, the second processing unit 403 may further include the following content: input the first sample question into a data generation model, obtain a candidate answer step and / or a candidate answer output by the data generation model, and when it is determined that the candidate answer step is similar to the first sample answer step and / or the candidate answer is similar to the first sample answer, take the first sample question, the first sample answer step, and the first sample answer as pre-training data.

[0073] That is, the second processing unit 403 can further use a large-scale language model to initially filter the obtained triplet, and then use a pre-trained data generation model to finally filter the triplet, further improving the accuracy of the obtained pre-trained data.

[0074] If it is determined that the candidate answer step is not similar to the first sample answer step, and / or the candidate answer is not similar to the first sample answer, the second processing unit 403 can discard the first sample question, the first sample answer step, and the first sample answer used this time, and can also use a manually labeled scheme to obtain the answer step and answer corresponding to the first sample question.

[0075] The second processing unit 403 can also establish a common rule and discard triplets that do not satisfy the common rule.

[0076] In this embodiment, after the second processing unit 403 obtains the first sample answer, the first pre-training unit 404 and the second pre-training unit 405 can respectively perform pre-training on the step planning model and the large-scale language model.

[0077] When the first pre-training unit 404 pre-trains the step planning model based on the first sample problem and the first sample answer step, it can first input the first sample problem into the step planning model, obtain a first predicted answer step output by the step planning model, obtain a first loss function value based on the first sample answer step and the first predicted answer step, and adjust parameters of the step planning model based on the obtained first loss function value to obtain a step planning model after pre-training.

[0078] When the second pre-training unit 405 pre-trains the large-scale language model based on the first sample problem, the first sample answer step, and the first sample answer, it can first input the first sample problem and the first sample answer step into the large-scale language model, obtain a first predicted answer output by the large-scale language model, obtain a second loss function value based on the first sample answer and the first predicted answer, and adjust parameters of the large-scale language model based on the obtained second loss function value to obtain a large-scale language model after pre-training.

[0079] In this embodiment, after the first pre-training unit 404 and the second pre-training unit 405 complete pre-training, the construction unit 406 obtains a problem-solving model based on the pre-trained step planning model and the large-scale language model.

[0080] The construction unit 406 can directly obtain the pre-trained step planning model and the large-scale language model to construct a problem-answering model. After the constructed problem-answering model obtains an input problem, it first inputs the obtained input problem into the step planning model to obtain the answer steps output by the step planning model, and then inputs the obtained input problem and answer steps into the large-scale language model to obtain the answer output by the large-scale language model.

[0081] When the construction unit 406 obtains a problem-answering model based on the pre-trained step planning model and the large-scale language model, it may further include the following content: obtain a second sample problem, determine a problem type of the second sample problem, obtain an answer step corresponding to the determined problem type as a second sample answer step of the second sample problem, perform supervised fine-tuning (SFT) on the pre-trained step planning model based on the second sample problem and the second sample answer step, and obtain a problem-answering model based on the pre-trained large-scale language model and the step planning model obtained by supervised fine-tuning.

[0082] That is, the construction unit 406 further improves the training effect of the step planning model by performing supervised fine-tuning on the step planning model, and further obtains a problem answering model based on the step planning model obtained by supervised fine-tuning. This embodiment can obtain answer steps based on the problem type of the sample problem, and if the problem answering models are for the same type of problem, they can output answer steps that are similar in number or logic.

[0083] When the construction unit 406 obtains a problem-answering model based on the pre-trained step planning model and the large-scale language model, the construction unit 406 may further include the following content: obtaining a second sample problem, determining a problem type of the second sample problem, obtaining an answer step corresponding to the determined problem type as a second sample answer step of the second sample problem, determining an answer step type of the second sample answer step, obtaining an answer corresponding to the determined answer step type as a second sample answer of the second sample problem, performing supervised fine-tuning (SFT) on the pre-trained large-scale language model based on the second sample problem, the second sample answer step, and the second sample answer, and obtaining a problem-answering model based on the pre-trained step planning model and the large-scale language model obtained by supervised fine-tuning.

[0084] That is, the construction unit 406 further improves the training effect of the large-scale language model by performing supervised fine-tuning on the large-scale language model, and further obtains a problem answer model based on the large-scale language model obtained by supervised fine-tuning. This embodiment can obtain an answer based on the answer type of the sample answer step, and if the problem answer model is for the same type of answer step, it can output an answer that is similar in number or logic.

[0085] It should be understood that the construction unit 406 can further perform supervised fine-tuning on the pre-trained step planning model and the large-scale language model simultaneously, and further obtain a problem-solving model based on the step planning model and the large-scale language model obtained by supervised fine-tuning.

[0086] 5 is a schematic diagram according to a fifth embodiment of the present disclosure. As shown in FIG. 5, a problem solving device 500 of this embodiment includes a second acquisition unit 501, a first solving unit 502, and a second solving unit 503; The second acquisition unit 501 is used for acquiring questions to be answered; The first solution unit 502 is used for inputting the problem to be solved into a step planning model in the problem solving model, and obtaining the solution steps output by the step planning model; A second answering unit 503 is used to input the problem to be answered and the answering steps into a large-scale language model in the problem-answering model, and obtain an answer output by the large-scale language model.

[0087] In the technical solution disclosed herein, the collection, storage, application and other processes of relevant user personal information all comply with relevant laws and regulations and do not violate public order and morals.

[0088] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0089] 6 shows a schematic block diagram of an electronic device 600 for training a problem-solving model or solving a problem according to an embodiment of the present disclosure. The electronic device is intended to represent various types of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various types of mobile devices, such as personal digital assistants, mobile phones, smartphones, wearable devices, and other similar computing devices. The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the present disclosure described herein and / or as claimed.

[0090] 6, the device 600 includes a computing unit 601, which can perform various appropriate operations and processes based on a computer program stored in a read-only memory (ROM) 602 or loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 can also store various programs and data required for the device 600 to operate. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0091] Several components in device 600 are connected to an I / O interface 605, including input units 606 such as a keyboard, mouse, etc., output units 607 such as various types of displays, speakers, etc., storage units 608 such as a disk, optical disk, etc., and communication units 609 such as a network card, modem, wireless communication transceiver, etc. The communication units 609 enable device 600 to exchange information / data with other devices via a computer network, such as the Internet, and / or various telecommunication networks.

[0092] The computing unit 601 is a general-purpose and / or dedicated processing component with various processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, computing units that execute various machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 executes the various methods and processes described above, such as the method for training a problem-solving model or the problem-solving method. For example, in some embodiments, the method for training a problem-solving model or the problem-solving method can be implemented as a computer software program tangibly contained in a machine-readable medium, such as the storage unit 608.

[0093] In some embodiments, a part or all of the computer program is loaded and / or installed in the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, it can perform one or more steps of the above problem-answering model training method or problem-solving method. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the problem-answering model training method or problem-solving method via any other suitable manner (e.g., by firmware).

[0094] Various implementations of the systems and techniques described herein may be realized in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system on a chip systems (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include being implemented in one or more computer programs that may be executed and / or interpreted by a programmable system that includes at least one programmable processor, which may be a special purpose or general purpose programmable processor, and that may receive data and instructions from and transmit data and instructions to a storage system, at least one input device, and at least one output device.

[0095] The program codes for implementing the methods of the present disclosure can be created using any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable vehicle positioning or positioning model training device such that the functions / operations specified in the flowcharts and / or block diagrams are implemented when the program codes are executed by the processor or controller. The program codes can be executed entirely on the machine, partially on the machine, partially on the machine as a separate software package and partially on a remote machine, or entirely on a remote machine or server.

[0096] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use with or in combination with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium includes, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

[0097] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other types of devices can also be used to provide interaction with a user, for example, feedback provided to the user can be any form of sensing feedback (e.g., visual feedback, auditory feedback, or haptic feedback) and can receive input from the user in any form (including acoustic input, speech input, and haptic input).

[0098] The systems and techniques described herein may be implemented in a computing system including a back-end component (e.g., a data server), or a computing system including a middleware component (e.g., an application server), or a computing system including a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user interacts with an implementation of the systems and techniques described herein), or in a computing system including any combination of such back-end, middleware, and front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0099] The computer system may include a client and a server. The client and the server are generally remote from each other and usually interact with each other via a communication network. The relationship between the client and the server is generated by a computer program that runs on a corresponding computer and has a client-server relationship with each other. The server may be a cloud server, also called cloud computing or cloud host, which is a host product in a cloud computing service system and solves the defects of high management difficulty and low business scalability that exist in traditional physical hosts and VPS services (abbreviated as "Virtual Private Server", or "VPS"). The server may be a server of a distributed system or a server that combines blockchain.

[0100] It should be understood that steps can be rearranged, added, or deleted using various types of flows shown above. For example, each step described in the present disclosure can be performed in parallel, sequentially, or in a different order, but is not limited herein as long as the technical solution disclosed in the present disclosure can achieve the desired results.

[0101] The above specific implementation methods do not constitute a limitation of the protection scope of the present disclosure. Those skilled in the art can make various modifications, combinations, subcombinations, and substitutions based on design requirements and other factors. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method for training a problem-solving model, comprising: obtaining a first sample question; inputting the first sample problem and an answer step capture template into a large-scale language model and obtaining a first sample answer step output by the large-scale language model; inputting the first sample question, the first sample answer step and an answer capture template into the large-scale language model and obtaining a first sample answer output by the large-scale language model; pre-training a step planning model based on the first sample problem and the first sample solution; pre-training the large-scale language model based on the first sample questions, the first sample answers, and the first sample answers; Obtaining a problem-solving model based on the pre-trained step planning model and the pre-trained large-scale language model. How to train a problem-solving model.

2. inputting the first sample problem, the first sample answer, the first sample answer, and a data evaluation template into the large-scale language model, and obtaining a data evaluation result output by the large-scale language model; If it is determined that the data evaluation result satisfies a preset requirement, the first sample question, the first sample answer, and the first sample answer are set as pre-training data. A method for training a problem-solving model according to claim 1.

3. The first sample problem, the first sample answer step, and the step of using the first sample answer as pre-training data include: inputting the first sample problem into a data generation model and obtaining candidate answer steps and / or candidate answers output by the data generation model; if it is determined that the candidate answer step is similar to the first sample answer step and / or the candidate answer is similar to the first sample answer, selecting the first sample question, the first sample answer step, and the first sample answer as the pre-training data. A method for training a problem-solving model according to claim 2.

4. The step of obtaining a problem-solving model based on the pre-trained step planning model and a large-scale language model includes: obtaining a second sample question and determining a question type of the second sample question; obtaining an answer step corresponding to the question type as a second sample answer step of the second sample question; performing supervised fine-tuning on a pre-trained step planning model based on the second sample problem and the second sample solution step; Obtaining the problem-solving model based on a pre-trained large-scale language model and a step planning model obtained by supervised fine-tuning; A method for training a problem-solving model according to claim 1.

5. The step of obtaining a problem-solving model based on the pre-trained step planning model and a large-scale language model includes: obtaining a second sample question and determining a question type of the second sample question; obtaining an answer step corresponding to the question type as a second sample answer step of the second sample question; determining an answer step type of the second sample answer step, and obtaining an answer corresponding to the answer step type as a second sample answer for the second sample question; performing supervised fine-tuning on a pre-trained large-scale language model based on the second sample problems, the second sample answers, and the second sample answers; Obtaining the problem-solving model based on a pre-trained step planning model and a large-scale language model obtained by supervised fine-tuning. A method for training a problem-solving model according to claim 1.

6. The step of obtaining a problem-solving model based on the pre-trained step planning model and a large-scale language model includes: obtaining a second sample question and determining a question type of the second sample question; obtaining an answer step corresponding to the question type as a second sample answer step of the second sample question; determining an answer step type of the second sample answer step, and obtaining an answer corresponding to the answer step type as a second sample answer for the second sample question; performing supervised fine-tuning on a pre-trained step planning model based on the second sample problem and the second sample solution step; performing supervised fine-tuning on a pre-trained large-scale language model based on the second sample problems, the second sample answers, and the second sample answers; Obtaining the problem-solving model based on the step planning model obtained by supervised fine-tuning and a large-scale language model; A method for training a problem-solving model according to claim 1.

7. The step of pre-training a step planning model based on the first sample problem and the first sample solution step includes: inputting the first sample problem into the step planning model and obtaining a first predicted answer step output by the step planning model; obtaining a first loss function value based on the first sample solution step and the first predicted solution step; and adjusting parameters of the step planning model based on the first loss function value to obtain a pre-trained step planning model. A method for training a problem-solving model according to claim 1.

8. The step of pre-training the large-scale language model based on the first sample questions, the first sample answers, and the first sample answers includes: inputting the first sample question and the first sample answer into the large-scale language model and obtaining a first predicted answer output by the large-scale language model; obtaining a second loss function value based on the first sample response and the first predicted response; and adjusting parameters of the large-scale language model based on the second loss function value to obtain a pre-trained large-scale language model. A method for training a problem-solving model according to claim 1.

9. A method for solving a problem, comprising: obtaining a question to be answered; inputting the problem to be solved into a step planning model in a problem solving model, and obtaining answer steps output by the step planning model; inputting the problem to be answered and the answering step into a large-scale language model in the problem-answering model, and obtaining an answer output by the large-scale language model; The problem-solving model is trained according to the method of any one of claims 1 to 8. How to answer questions.

10. A problem-solving model training device, comprising: a first acquisition unit for acquiring a first sample question; a first processing unit for inputting the first sample problem and an answer step capture template into a large-scale language model and obtaining a first sample answer step output by the large-scale language model; a second processing unit for inputting the first sample question, the first sample answer step and an answer capture template into the large-scale language model and obtaining a first sample answer output by the large-scale language model; a first pre-training unit for pre-training a step planning model based on the first sample problem and the first sample answer step; a second pre-training unit for pre-training the large-scale language model based on the first sample problem, the first sample answer step, and the first sample answer; A construction unit for obtaining a problem-solving model based on the pre-trained step planning model and the pre-trained large-scale language model; A training device for problem-solving models.

11. The second processing unit further comprises: inputting the first sample problem, the first sample answer step, the first sample answer, and a data evaluation template into the large-scale language model, and obtaining a data evaluation result output by the large-scale language model; If it is determined that the data evaluation result satisfies a preset requirement, the first sample question, the first sample answer, and the first sample answer are used as pre-training data. A training device for a problem-solving model according to claim 10.

12. When the first sample problem, the first sample answer step, and the first sample answer are used as pre-training data, the second processing unit specifically: inputting the first sample problem into a data generation model and obtaining candidate answer steps and / or candidate answers output by the data generation model; if it is determined that the candidate answer step is similar to the first sample answer step and / or the candidate answer is similar to the first sample answer, the first sample question, the first sample answer step, and the first sample answer are the pre-training data. A training device for a problem-solving model according to claim 11.

13. When the construction unit obtains a problem-solving model based on the pre-trained step planning model and the large-scale language model, specifically, Obtaining a second sample question and determining a question type of the second sample question; obtaining an answer step corresponding to the question type as a second sample answer step of the second sample question; performing supervised fine-tuning on a pre-trained step planning model based on the second sample problem and the second sample answer steps; Obtaining the problem-solving model based on a pre-trained large-scale language model and a step planning model obtained by supervised fine-tuning; A training device for a problem-solving model according to claim 10.

14. When the construction unit obtains a problem-solving model based on the pre-trained step planning model and the large-scale language model, specifically, obtaining a second sample question and determining a question type of the second sample question; obtaining an answer step corresponding to the question type as a second sample answer step of the second sample question; determining an answer step type of the second sample answer step, and obtaining an answer corresponding to the answer step type as a second sample answer for the second sample question; performing supervised fine-tuning on a pre-trained large-scale language model based on the second sample problems, the second sample answers, and the second sample answers; Obtaining the problem-solving model based on a pre-trained step planning model and a large-scale language model obtained by supervised fine-tuning; A training device for a problem-solving model according to claim 10.

15. When the construction unit obtains a problem-solving model based on the pre-trained step planning model and the large-scale language model, specifically, obtaining a second sample question and determining a question type of the second sample question; obtaining an answer step corresponding to the question type as a second sample answer step of the second sample question; determining an answer step type of the second sample answer step, and obtaining an answer corresponding to the answer step type as a second sample answer for the second sample question; performing supervised fine-tuning on a pre-trained step planning model based on the second sample problem and the second sample answer steps; performing supervised fine-tuning on a pre-trained large-scale language model based on the second sample problems, the second sample answers, and the second sample answers; Obtaining the problem-solving model based on the step planning model and the large-scale language model obtained by supervised fine-tuning; A training device for a problem-solving model according to claim 10.

16. When the first pre-training unit pre-trains a step planning model according to the first sample problem and the first sample answer step, specifically, inputting the first sample problem into the step planning model and obtaining a first predicted answer step output by the step planning model; obtaining a first loss function value based on the first sample solution step and the first predicted solution step; adjusting parameters of the step planning model based on the first loss function value to obtain a pre-trained step planning model; A training device for a problem-solving model according to claim 10.

17. When the second pre-training unit pre-trains the large-scale language model according to the first sample problem, the first sample answer step, and the first sample answer, specifically, inputting the first sample problem and the first sample answer into the large-scale language model and obtaining a first predicted answer output by the large-scale language model; obtaining a second loss function value based on the first sample response and the first predicted response; Adjusting parameters of the large-scale language model based on the second loss function value to obtain a pre-trained large-scale language model. A training device for a problem-solving model according to claim 10.

18. A problem solving device, comprising: a second acquisition unit for acquiring questions to be answered; a first solving unit for inputting the problem to be solved into a step planning model in a problem solving model and obtaining answer steps output by the step planning model; a second answering unit for inputting the problem to be answered and the answering step into a large-scale language model in the problem answering model and obtaining an answer output by the large-scale language model; The problem-solving model is trained on the basis of an apparatus according to any one of claims 10 to 17. Problem solving device.

19. An electronic device, At least one processor; a memory in communication with the at least one processor; The memory stores instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a method according to any one of claims 1 to 8. electronic equipment.

20. A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising: The computer instructions cause the computer to carry out a method according to any one of claims 1 to 8. A non-transitory computer-readable storage medium.

21. A computer program comprising: The computer program, when executed by a processor, implements the method according to any one of claims 1 to 8. Computer program.

Citation Information

Patent Citations

  • Generation method, model training method, equipment and storage medium

    CN116821308A

  • Construction method and application of question and answer interaction model with cognitive reasoning ability

    CN116991996A

  • Question answering method and language model training method, apparatus, device, and storage medium

    JP2021089739A