Model processing method and device, equipment and storage medium

Abstract

The invention discloses a model processing method and device, equipment and a storage medium, and the method comprises the steps: obtaining progressive thinking chain data which comprises a sample problem and N reasoning steps; constructing N rounds of dialogue samples step by step by adopting progressive thinking chain data, wherein each round of dialogue sample comprises input information and output information; the input information in the nth round of dialogue sample is used for indicating that the nth reasoning step is predicted based on the sample problem and the first (n-1) reasoning steps; the output information in the nth round of dialogue sample at least comprises an nth reasoning step obtained from the progressive thinking chain data; and calling a to-be-optimized language model to perform prediction processing according to the input information of each round of dialogue sample in sequence, and optimizing model parameters of the language model based on the difference between the prediction result of each round of dialogue sample and the output information in the corresponding dialogue sample so as to improve the logical reasoning ability of the language model.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, specifically to the field of artificial intelligence technology, and in particular to a model processing method, apparatus, device and storage medium. Background Art

[0002] Currently, to enable language models to have logical reasoning capabilities for reasoning problems, the following method is typically used to optimize language models: sample problems and the corresponding actual reasoning process are obtained. The sample problems are used to generate task description instructions, which instruct the sample problem to be reasoned and output the reasoning process. The language model is then called to reason on the sample problem according to the task description instructions. Based on the discrepancy between the reasoning process output by the language model and the actual reasoning process, the language model is optimized. This shows that existing model optimization methods are relatively simple, and the resulting language models have poor logical reasoning capabilities. Summary of the Invention

[0003] The embodiments of the present application provide a model processing method, apparatus, device, and storage medium that can enhance the logical reasoning capability of a language model.

[0004] In one aspect, an embodiment of the present application provides a model processing method, the method comprising:

[0005] Acquire progressive thinking chain data, the progressive thinking chain data including: a sample question and N reasoning steps, the N reasoning steps being used to infer a standard answer corresponding to the sample question, where N is a positive integer;

[0006] N rounds of dialogue samples are constructed step by step using the progressive thought chain data, each round of dialogue samples including input information and output information; the input information in the n-th round of dialogue sample is used to indicate: predicting the n-th reasoning step based on the sample question and the first n-1 reasoning steps; the output information in the n-th round of dialogue sample at least includes: the n-th reasoning step obtained from the progressive thought chain data, n∈[1,N];

[0007] Calling the language model to be optimized to perform prediction processing based on the input information of each round of dialogue samples in turn to obtain the prediction results of each round of dialogue samples;

[0008] Optimizing model parameters of the language model based on the difference between the prediction result of each round of dialogue samples and the output information in the corresponding dialogue samples.

[0009] On the other hand, an embodiment of the present application provides a model processing device, the device comprising:

[0010] an acquisition unit, configured to acquire progressive thinking chain data, wherein the progressive thinking chain data includes: a sample question and N reasoning steps, wherein the N reasoning steps are used to infer a standard answer corresponding to the sample question, where N is a positive integer;

[0011] a processing unit configured to construct N rounds of dialogue samples step by step using the progressive thought chain data, each round of dialogue samples including input information and output information; the input information in the n-th round of dialogue sample being used to indicate: predicting the n-th reasoning step based on the sample question and the first n-1 reasoning steps; and the output information in the n-th round of dialogue sample at least including: the n-th reasoning step obtained from the progressive thought chain data, where n∈[1,N];

[0012] The processing unit is further configured to call the language model to be optimized to perform prediction processing based on the input information of each round of dialogue samples in sequence, thereby obtaining a prediction result for each round of dialogue samples;

[0013] The processing unit is further configured to optimize the model parameters of the language model based on the difference between the prediction result of each round of dialogue samples and the output information in the corresponding dialogue samples.

[0014] In another aspect, an embodiment of the present application provides a computer device, the computer device including an input interface and an output interface, and the computer device further including:

[0015] processors and computer storage media;

[0016] The processor is suitable for implementing one or more instructions, the computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by the processor and executing the above-mentioned model processing method.

[0017] On the other hand, an embodiment of the present application provides a computer storage medium, which stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the above-mentioned model processing method.

[0018] On the other hand, an embodiment of the present application provides a computer program product, which includes one or more instructions; when the one or more instructions in the computer program product are executed by a processor, the above-mentioned model processing method is implemented.

[0019] The embodiment of the present application can obtain progressive thinking chain data, which includes a sample question and N reasoning steps for inferring the standard answer corresponding to the sample question; and use the progressive thinking chain data to construct N rounds of dialogue samples step by step, each round of dialogue samples includes input information and output information, and the input information in the nth round of dialogue sample is used to indicate: predicting the nth reasoning step based on the sample question and the first n-1 reasoning steps, and the output information in the nth round of dialogue sample at least includes: the nth reasoning step obtained from the progressive thinking chain data. Then, the language model to be optimized is called to perform prediction processing according to the input information of each round of dialogue samples in turn, and the model parameters of the language model are optimized based on the difference between the prediction results of each round of dialogue samples and the output information in the corresponding dialogue samples. It can be seen that this method can enable the language model to perform multiple rounds of predictions, and each round adds a new reasoning step prediction work, which can better guide the language model to perform step-by-step reasoning on sample problems and improve the logical reasoning ability of the language model; and as the number of prediction rounds n increases, the context information referred to by the language model during prediction (such as the first n-1 reasoning steps) also increases accordingly, which can enable the language model to enrich the logical knowledge between the learned reasoning steps in each round, thereby further improving the logical reasoning ability of the language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 This is a flow chart of a model processing solution jointly executed by a terminal and a server provided in an embodiment of the present application;

[0022] Figure 2 This is a flow chart of a model processing method provided in an embodiment of the present application;

[0023] Figure 3 This is a schematic diagram of a weight distribution provided in an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of another weight distribution provided in an embodiment of the present application;

[0025] Figure 5 This is a structural diagram of a model processing device provided in an embodiment of the present application;

[0026] Figure 6 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application.

[0028] The embodiment of the present application is based on AI (Artificial Intelligence) technology and proposes a model processing solution based on a progressive thinking chain to enhance the logical reasoning ability of a large language model (LLM). Among them, AI technology refers to the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, AI is a comprehensive technology in computer science; it mainly produces a new intelligent machine that can respond in a similar way to human intelligence by understanding the essence of intelligence, so that the intelligent machine has multiple functions such as perception, reasoning and decision-making. Accordingly, AI technology is an interdisciplinary subject, which may include but is not limited to machine learning (ML) / deep learning, etc. Among them, machine learning is the core of AI and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Specifically, machine learning is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Deep learning, on the other hand, is a machine learning technique that utilizes deep neural network systems. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, and supervised learning.

[0029] Large language models, also known as large language models or language models, are deep learning models with a large number of parameters that are trained to understand and generate natural language text. These models are based on the Transformer architecture (a model structure that uses an attention mechanism for data processing) and are pre-trained on large amounts of text data to learn to capture the complex patterns and structure of language. These pre-trained models can then be fine-tuned (supervised fine-tuning, SFT) on task-specific datasets to solve various natural language processing (NLP) tasks, such as question reasoning, text classification, sentiment analysis, text generation, and summarization. Supervised fine-tuning, which involves fine-tuning model parameters using supervised learning techniques, can be used to improve the performance of pre-trained models on specific tasks. The basic steps are as follows: 1. Pre-training the model: Training on a large amount of text data. 2. Task-related data: Collecting labeled data relevant to the specific task. 3. Fine-tuning: Using the collected labeled data to fine-tune the model parameters of the pre-trained model. 4. Evaluation and application: Evaluating the fine-tuned model and applying it to real-world problems. It can be seen that the advantage of supervised fine-tuning is that it utilizes the general knowledge of the pre-trained model and adapts the corresponding model to specific tasks by fine-tuning the model parameters, which can reduce data labeling costs and training time.

[0030] When using large language models to solve natural language processing tasks, prompts play a key role. In the fields of programming and natural language processing, prompts usually refer to a piece of input text that guides the large language model to generate the expected output. Specifically, a prompt can be a question, a statement, or a task description, which provides contextual information and generation goals for the large language model. The large language model generates relevant answers or output text based on the prompts. For example, given the instruction "Translate the following English sentence into Chinese: "Hello, how are you?"", the large language model may generate "Hello, how are you?" as the output. It can be seen that when using pre-trained large language models to solve practical problems, designing appropriate prompts is crucial because it directly affects the quality and relevance of the large language model's generation. Sometimes, it may be necessary to try multiple instruction forms to find the instruction that can guide the large language model to generate the best output.

[0031] Progressive chaining of thought, also known as chain-of-thought (CoT), is an improved prompting strategy used to improve the performance of large language models in complex reasoning tasks, such as arithmetic reasoning, commonsense reasoning, and symbolic reasoning. Unlike in-context learning (ICL), chaining does not simply construct prompts using input-output pairs. Instead, it incorporates intermediate reasoning steps that introduce hints to the final output of the large language model. This allows the large language model to gradually reason about the question based on relevant hints and output the inferred answer. Thus, chaining is a form of discrete prompt learning; compared to in-context learning, chaining incorporates intermediate derivation hints, which enhances the logical reasoning ability of the large language model. In-context learning, rather than training the large language model, it appends input-output pairs constructed based on examples to the front of the current question input during the model inference phase (i.e., the model application phase). This allows the large language model to process these texts at once and output the answer to the current question, thus completing the task.

[0032] Specifically, the general principle of the model processing solution based on the progressive thinking chain proposed in the embodiment of the present application is as follows: (1) Construct progressive thinking chain data (i.e., construct data containing the display reasoning process). Among them, the constructed progressive thinking chain data may at least include: sample questions and N reasoning steps for inferring the standard answer corresponding to the sample questions. (2) Based on the form of multi-round dialogue, supervised instruction fine-tuning is performed on the progressive thinking chain data to obtain N rounds of dialogue samples. Among them: ① Supervised instruction fine-tuning of the large language model is a fine-tuning method for specific instructions or tasks; similar to the traditional supervised fine-tuning method, it relies on labeled data, but in this case, the label is usually related to the specific instruction or task. This enables the large language model to better understand and execute the given instruction, thereby achieving better performance on the specific task. Through supervised instruction fine-tuning, the large language model can better understand and execute the given instruction, thereby achieving higher performance on the specific task; this is because the pre-trained large language model has learned rich language knowledge, and supervised instruction fine-tuning enables it to apply this knowledge to specific instructions or tasks. It is understandable that since the pre-trained large language model has been trained on a large amount of data, supervised instruction fine-tuning usually requires less labeled data and computing resources. ② Each round of dialogue samples obtained through supervised instruction fine-tuning includes input information and output information, and the input information in the n-th round of dialogue sample is used to indicate: predicting the n-th reasoning step based on the sample question and the first n-1 reasoning steps, and the output information in the n-th round of dialogue sample at least includes: the n-th reasoning step obtained from the progressive thinking chain data, n∈[1,N]. (3) The large language model to be optimized is called to perform prediction processing based on the input information of each round of dialogue samples in turn, and the model parameters of the large language model are optimized based on the difference between the prediction results of each round of dialogue samples and the output information in the corresponding dialogue samples.

[0033] In a specific implementation, the aforementioned model processing solution based on the progressive thinking chain can be executed by a computer device, which can be a terminal or a server. The terminal can be a smartphone, a computer (such as a tablet, laptop, desktop computer), a smart wearable device (such as a smartwatch, smart glasses), an intelligent voice interaction device, a smart home appliance (such as a smart TV), an in-vehicle terminal, or an aircraft. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Furthermore, the terminal and server can be located within or outside the blockchain network, without limitation. Furthermore, the terminal and server can also upload any internally stored data to the blockchain network for storage, thereby preventing tampering with the internally stored data and improving data security.

[0034] Alternatively, the above-mentioned model processing scheme based on the progressive thinking chain can be jointly executed by the terminal and the server. For example, the terminal can be responsible for constructing the progressive thinking chain data, and sending the constructed progressive thinking chain data to the server. The server can fine-tune the progressive thinking chain data in a supervised manner based on the form of multiple rounds of dialogue to obtain N rounds of dialogue samples, and call the large language model to be optimized to perform prediction processing based on the input information of each round of dialogue samples in turn, thereby optimizing the model parameters of the large language model based on the difference between the prediction results of each round of dialogue samples and the output information in the corresponding dialogue samples. For another example, the terminal can be responsible for constructing the progressive thinking chain data, and fine-tune the progressive thinking chain data in a supervised manner based on the form of multiple rounds of dialogue to obtain N rounds of dialogue samples, and then send the N rounds of dialogue samples to the server. The server can call the large language model to be optimized to perform prediction processing based on the input information of each round of dialogue samples in turn, and optimize the model parameters of the large language model based on the difference between the prediction results of each round of dialogue samples and the output information in the corresponding dialogue samples, such as Figure 1 It is understandable that the examples here only exemplify two specific implementations of the model processing solution jointly executed by the terminal and the server, and are not exhaustive.

[0035] Based on the above description, it can be seen that the model processing solution based on the progressive thinking chain proposed in the embodiment of the present application can better guide the large language model to perform step-by-step reasoning on sample problems in the process of optimizing the large language model by calling the large language model for multiple rounds of predictions, and adding a new reasoning step prediction work in each round, thereby improving the logical reasoning ability of the large language model; and as the number of prediction rounds n increases, the context information referred to by the large language model during prediction (such as the first n-1 reasoning steps) also increases accordingly, so that the large language model can enrich the logical knowledge between the learned reasoning steps in each round, thereby further improving the logical reasoning ability of the large language model.

[0036] Based on the above description of the model processing solution based on the progressive chain of thought, the present embodiment proposes a model processing method for a language model (i.e., a large language model). The model processing method can be executed by the aforementioned computer device (terminal or server), or by both the terminal and the server. For ease of explanation, the present embodiment uses a computer device executing the model processing method as an example.

[0037] See Figure 2 As shown, the model processing method may include the following steps S201-S204:

[0038] S201, obtaining progressive thinking chain data.

[0039] In the example of the present application, the progressive thinking chain data includes: a sample problem and N reasoning steps. Among them, the sample problem can be any problem that needs to be solved by reasoning, such as a mathematical problem (such as a mathematical arithmetic problem, a mathematical word problem, etc.), a logical thinking problem, a common sense reasoning problem, and so on. The sample problem may have a standard answer (that is, a preset answer for the sample problem), and the standard answer can be obtained by performing N steps of reasoning on the sample problem, where N is a positive integer. It can be seen that the N reasoning steps in the progressive thinking chain data refer to: N steps for inferring the standard answer corresponding to the sample problem; and these N reasoning steps are executed step by step based on the reasoning logic. Optionally, since the progressive thinking chain data is used to optimize the language model, in order to make the language model have the ability to reason step by step and the ability to generate predicted answers after the language model is optimized, the progressive thinking chain data can also include: the standard answer inferred based on the N reasoning steps, so that the language model can be supervised and fine-tuned based on the standard answer in the future.

[0040] In one specific implementation, a computer device may use an open source dataset to obtain progressive thought chain data. The so-called open source dataset refers to an open dataset in which each piece of data is formatted in the displayed progressive thought chain format. In this specific implementation, step S201 may be implemented by obtaining the open source dataset and selecting data from the open source dataset as the progressive thought chain data.

[0041] Among them, the obtained open source datasets may include but are not limited to the GSM8K dataset (which may be referred to as the first open source dataset) and the Strategy QA dataset (which may be referred to as the second open source dataset). The GSM8K dataset consists of 8.5K (K represents thousand) elementary school mathematics word problems (7.5K training problems and 1K test problems). Each problem requires 2 to 8 reasoning steps to solve, and the reasoning process of each problem mainly involves a series of elementary calculations using basic arithmetic operations (addition, subtraction, multiplication, and division). The Strategy QA dataset was created through a crowdsourcing channel to elicit creative and diverse yes / no questions that require implicit reasoning steps; the questions in the Strategy QA dataset use strategies to infer the reasoning steps, and in order to guide and evaluate the question answering process, each question in the Strategy QA dataset is annotated with the reasoning steps used to answer it, as well as encyclopedia paragraphs that provide evidence for the answers to each step, thereby forming data in a progressive thinking chain format. Exemplarily, a piece of data in the Strategy QA dataset can be as follows:

[0042] Problem: A buys 3, 12 packs of soda for a party. Including A, 6 people are at the party. Half of the people at the party have 3 sodas each, 2 of the people have 4, and 1 person has 5. How many sodas are left over when the party is over?

[0043] Solution: A buys 3, 12 packs of soda, for 3×12=<<3×12=36>>36sodas (User A bought 3×12 bottles of soda, so 3×12=36, a total of 36 bottles of soda);

[0044] 6 people attend the party, so half of them is 6 / 2=<<6 / 2=3>>3people (There are 6 people attending the party, so half of them is 6 / 2=3, 3 people);

[0045] Each of those people drinks 3 sodas, so they drink 3×3=<<3×3=9>>9 sodas (each of the 3 people drank 3 bottles of soda, so the 3 people drank 3×3=9, 9 bottles of soda);

[0046] Two people drink 4 sodas, which means they drink 2×4=<<2×4=8>>8 sodas (each of the two people drank 4 bottles of soda, which means the two people drank 2×4=8, 8 bottles of soda);

[0047] With One people drinks 5, so the total drank to 5+9+8+3=<<5+9+8+3=25>>25 sodas (With 1 person drinking 5 bottles, so the total drank 5+9+8+3=25, 25 bottles of soda);

[0048] As A started off with 36 sodas, that means there are 36-25=<<36-25=11>>11 sodas left (since user A started with 36 bottles of soda, this means there are 36-25=11, 11 bottles of soda left).

[0049] Final Answer: 11.

[0050] In another specific implementation, a computer device can use model distillation to obtain progressive thought chain data. Model distillation refers to a method based on black-box distillation, using a text generation model (a generative language model) as a teacher model to distill progressive thought chain data. Specifically, a large number of elementary and junior high school mathematics problems and common sense reasoning problems can be collected as sample problems. By constructing prompts (instructions), the text generation model can be fed with the problem-solving process containing progressive reasoning steps to obtain progressive thought chain data.

[0051] In this specific implementation, step S201 may be implemented as follows: the computer device may obtain prompt text, a case study, and a sample question. The case study includes a sample question and the inference data output after reasoning about the sample question. The inference data includes at least one inference step corresponding to the sample question and the inferred answer, and the format of the inference data may sequentially display the sample question, at least one inference step, and the answer. The prompt text prompts: perform step-by-step reasoning on the question in the input field and output inference data, and the format of the output inference data is consistent with the format of the inference data in the case study. The computer device may then use the prompt text, the case study, and the sample question to generate a task description instruction. The task description instruction includes an input field, and the sample question is located in the input field. For example, the computer device may populate the input field with the sample question and then concatenate the prompt text, the case study, and the input field to generate the task description instruction. After generating the task description instruction, the computer device may invoke a text generation model to perform task processing according to the task description instruction to output inference data. The inference data output by the text generation model is used as progressive thinking chain data.

[0052] It should be noted that: ① The step-by-step reasoning mentioned above can be understood as reasoning step by step, and each step of reasoning is performed based on the semantics of the above information (such as the question to be reasoned, and the information inferred before the current step). ② The embodiment of the present application does not limit the specific form of the prompt text mentioned above. For example, the specific form of the prompt text can be the following first form: "The following is a conversation between a human and a machine assistant. The human gives a question, and the machine assistant thinks step by step to get the correct answer to the question. Please refer to the reasoning data in the case to convert the input question into a reference format. Note: The reasoning steps of the machine assistant are not limited. Please think step by step and generate a logically clear and correct reasoning process and corresponding reasoning data. Please pay attention to maintain consistency between the format of the generated reasoning data and the format of the reasoning data in the case." Alternatively, the specific form of the prompt text can be the following second form: "One or more cases and input questions are given below. Please perform step-by-step reasoning on the input question, and please refer to the format of the reasoning data in the case to generate and output the reasoning data of the corresponding input question", etc. ③ The embodiment of the present application does not limit the number of cases mentioned above and the specific form of each case. For example, the number of cases can be 1, 2, 3 or more. For another example, the specific form of a case can be an input-output pair. In this case, the case can include an input field and an output field, where the input field includes a question and the output field includes inference data. Alternatively, the specific form of the case can be a question-and-answer format or a dialogue format, etc. ④ The embodiments of this application do not limit the combination format of the individual data in the inference data in the case. For example, the combination format can be a dialogue format, a direct splicing format, etc.

[0053] Based on the above description, the specific form of the prompt text is the first form, the number of cases is 2, the specific form of each case is the form of an input-output pair, and the combination form of each data in the reasoning data in the output field is the form of a dialogue. The task description instructions generated by the embodiment of the present application using the prompt text, cases and sample questions can be exemplarily shown as follows:

[0054] The following is a conversation between a human and a machine assistant. The human asks a question, and the machine assistant thinks through the steps to arrive at the correct answer. Please refer to the example to convert the input question into a reference format. Note: The machine assistant can use any number of reasoning steps. Please think through the steps and generate a logically clear and correct reasoning process. Please ensure that the generated format is consistent with the reference example.

[0055] For example:

[0056] Example 1:

[0057] Input: A dictionary is 40 mm thick and a math book is 5 mm thick. How many math books are as thick as a dictionary?

[0058] Output: {""Human"":""A dictionary is 40 mm thick, and a math book is 5 mm thick. How many math books are as thick as a dictionary? "","Machine Assistant"":{""Step 1"":""We need to find a number such that when multiplied by the thickness of the math book, we get the thickness of the dictionary, i.e., q×5 mm = 40 mm. "","Step 2"":""Divide 40 mm by 5 mm, and we get q = 40 / 5 = 8. "","Step 3"":""So, we need 8 math books to be as thick as a dictionary. The answer is 8. ""}}

[0059] Example 2:

[0060] Input: Xiao Ming had 3 oranges yesterday and bought 2 more oranges today. How many oranges does he have now?

[0061] Output: {""Human"":""Xiaoming had 3 oranges yesterday and bought 2 more oranges today. How many oranges does he have now? "",""Machine Assistant"":{""Step 1"":""We need to add the number of oranges he had yesterday and the number of oranges he bought today, which is 3 oranges + 2 oranges. "",""Step 2"":""After adding them together, we get 3 + 2 = 5 oranges. "",""Step 3"":""So, Xiaoming now has a total of 5 oranges. The answer is 5. ""}}

[0062] Input question: (sample question)

[0063] Output: ".

[0064] Based on the above description, it should be noted that: the above is only an illustrative description of the specific implementation of obtaining a progressive thinking chain data. When the step S201 is actually executed, multiple progressive thinking chain data can also be obtained, and the subsequent steps S202-S204 are executed based on each progressive thinking chain data respectively to optimize the language model multiple times. In this case, multiple progressive thinking chain data can all be obtained from open source data sets, or all can be obtained based on the model distillation method, or can be obtained by combining the open source data set and the model distillation method, without limitation. It is worth emphasizing that when any method embodiment proposed in the embodiment of the present application is applied to a specific product or technology, any data is collected with the permission or consent of the user / webpage, and the collection, use and processing of relevant data comply with the relevant laws, regulations and standards of the relevant region.

[0065] S202, using progressive thinking chain data to construct N rounds of dialogue samples step by step.

[0066] The so-called step-by-step construction can be understood as follows: as the number of dialogue rounds increases, the reasoning steps with the number of steps matching the current round are selected from the progressive thought chain data to construct the dialogue sample for the current round. N rounds of dialogue samples can be used to guide the language model to perform N rounds of prediction processing; the dialogue samples of each round are arranged in order from smallest to largest round, and to enable the language model to distinguish dialogue samples from different rounds, a separator (such as [Seperator]) can be inserted between adjacent dialogue samples for identification. In addition, each dialogue sample includes input information and output information. For the nth round of dialogue sample:

[0067] The input information in the n-th round of dialogue sample is used to indicate: predicting the n-th reasoning step based on the sample question and the first n-1 reasoning steps; it can be understood that, n∈[1,N], when n=1, the n-th round of dialogue sample is the first round of dialogue sample. Since in this case, the first n-1 (i.e., the first 0) reasoning steps are empty, in this case, the input information in the first round of dialogue sample can actually be used to indicate: predicting the first reasoning step based on the sample question.

[0068] The output information in the n-th round of dialogue sample includes at least: the n-th reasoning step obtained from the progressive thinking chain data. In a specific implementation, the output information in the n-th round of dialogue sample may also include: the first n-1 reasoning steps obtained from the progressive thinking chain data; that is, the output information of the n-th round of dialogue sample in this case includes: the first n reasoning steps obtained from the progressive thinking chain data. By adopting the first n reasoning steps in the progressive thinking chain data to construct the output information in the n-th round of dialogue sample, the language model can be guided based on the output information to predict and output the first n reasoning steps when making the n-th round prediction, so as to ensure logical coherence, thereby improving the prediction accuracy and further improving the accuracy of model optimization. In another specific implementation, the output information in the n-th round of dialogue sample may also not include: the first n-1 reasoning steps obtained from the progressive thinking chain data; by adopting the n-th reasoning step in the progressive thinking chain data to construct the output information in the n-th round of dialogue sample, the language model can be guided based on the output information to predict the next n-th reasoning step when making the n-th round prediction, thereby improving the prediction efficiency and thus improving the model optimization efficiency.

[0069] Optionally, if the progressive thinking chain data also includes a standard answer, then when n=N, the nth round of dialogue sample is the Nth round of dialogue sample. In this case, the output information in the nth round of dialogue sample may also include the standard answer. This can guide the language model to not only predict and output the Nth reasoning step (or the first N reasoning steps) during the Nth round of prediction processing, but also output the predicted answer generated based on the predicted reasoning steps. Of course, it is understandable that in other embodiments, even if the progressive thinking chain data also includes a standard answer, the output information in the Nth round of dialogue sample may not include the standard answer; in this case, after the language model performs the Nth round of prediction processing, a predicted answer can be generated based on the predicted reasoning steps and the generated predicted answer can be output.

[0070] Based on the above description, the following example illustrates an N-round conversation sample:

[0071] Assume that the progressive thinking chain data obtained in step S201 is as follows:

[0072] "[Human]: Solve the following system of linear equations in two variables:

[0073] 2x+3y=12

[0074] 4x-y=10

[0075] [Machine Assistant]: OK, we can solve this system of equations by following these steps:

[0076] Step 1: We can first multiply the second equation by 3 to eliminate y.

[0077] This results in the new system of equations:

[0078] 2x+3y=12

[0079] 12x-3y=30

[0080] Step 2: We then add the first equation to the new second equation, eliminating y.

[0081] This will give: 14x = 42

[0082] Step 3: Solving this equation, we get x = 42 / 14 = 3.

[0083] Step 4: Substituting x=3 into the original first equation, we get 2×3+3y=12, that is, 3y=12-6=6, so y=6 / 3=2.

[0084] Therefore, the solution of this system of linear equations is x=3, y=2".

[0085] It can be understood that in the above example of progressive thought chain data, the sample question is "[Human]: Solve the following linear equation system: 2x + 3y = 12, 4x - y = 10"; the standard answer is "Therefore, the solution to this linear equation system is x = 3, y = 2." Since the number of reasoning steps N = 4, four rounds of dialogue samples can be constructed step by step using this progressive thought chain data. The specific forms of these four rounds of dialogue samples can be any of the following:

[0086] (1) If the output information in the n-th round of dialogue sample includes the first n reasoning steps obtained from the progressive thinking chain data, and the output information in the fourth round of dialogue sample also includes the standard answer, then the four-round dialogue sample can be as follows:

[0087] "[Human]: Think step by step and solve the following system of linear equations: 2x + 3y = 12, 4x - y = 10. Step 1: [Machine Assistant]: We can first multiply the second equation by 3 to eliminate y. This will give us a new system of equations: 2x + 3y = 12, 12x - 3y = 30.

[0088] [Seperator]

[0089] Human: Think step by step and solve the following system of linear equations: 2x + 3y = 12, 4x - y = 10. Step 1: We can first multiply the second equation by 3 to eliminate y. This will give us the new system of equations: 2x + 3y = 12, 12x - 3y = 30. Step 2: Machine: We can first multiply the second equation by 3 to eliminate y. This will give us the new system of equations: 2x + 3y = 12, 12x - 3y = 30. Then, we add the first equation to the new second equation and eliminate y. This will give us: 14x = 42.

[0090] [Seperator]

[0091] Human: Think step by step and solve the following system of linear equations: 2x + 3y = 12, 4x - y = 10. Step 1: We can first multiply the second equation by 3 to eliminate y. This will give us the new system of equations: 2x + 3y = 12, 12x - 3y = 30. Step 2: Then we add the first equation to the new second equation and eliminate y. This will give us: 14x = 42. Step 3: Machine Assistant: We can first multiply the second equation by 3 to eliminate y. This will give us the new system of equations: 2x + 3y = 12, 12x - 3y = 30. Then we add the first equation to the new second equation and eliminate y. This will give us: 14x = 42. Solving this equation, we get x = 42 / 14 = 3.

[0092] [Seperator]

[0093] Human: Think step by step and solve the following system of linear equations: 2x + 3y = 12, 4x - y = 10. Step 1: We can first multiply the second equation by 3 to eliminate y. This will give us a new system of equations: 2x + 3y = 12, 12x - 3y = 30. Step 2: Then we add the first equation to the new second equation and eliminate y. This will give us: 14x = 42. Step 3: Solving this equation, we get x = 42 / 14 = 3. Step 4: Machine Assistant: We can first multiply the second equation by 3 to eliminate y. This will give us a new system of equations: 2x + 3y = 12, 12x - 3y = 30. Then we add the first equation to the new second equation and eliminate y. This gives: 14x = 42; solving this equation, we get x = 42 / 14 = 3; substituting x = 3 into the original first equation, we get 2×3 + 3y = 12, or 3y = 12 - 6 = 6, so y = 6 / 3 = 2. Therefore, the solution to this system of two linear equations is x = 3, y = 2."

[0094] In the above example, [Seperator] is a separator used to separate two adjacent rounds of dialogue samples; a paragraph of text constitutes a round of dialogue sample, and the characters before "

Machine Assistant

Machine Assistant

Human

Machine Assistant

[0095] (2) If the output information in the n-th round of dialogue sample includes the n-th reasoning step obtained from the progressive thinking chain data, and the output information in the 4-th round of dialogue sample also includes the standard answer, then the 4-round dialogue sample can be as follows:

[0096] "[Human]: Think step by step and solve the following system of linear equations: 2x + 3y = 12, 4x - y = 10. Step 1: [Machine Assistant]: We can first multiply the second equation by 3 to eliminate y. This will give us a new system of equations: 2x + 3y = 12, 12x - 3y = 30.

[0097] [Seperator]

[0098] Human: Think step by step and solve the following system of linear equations: 2x + 3y = 12, 4x - y = 10. Step 1: We can first multiply the second equation by 3 to eliminate y. This will give us the new system of equations: 2x + 3y = 12, 12x - 3y = 30. Step 2: Machine: Then we add the first equation to the new second equation and eliminate y. This gives us: 14x = 42.

[0099] [Seperator]

[0100] Human: Think step by step and solve the following system of linear equations: 2x + 3y = 12, 4x - y = 10. Step 1: We can first multiply the second equation by 3 to eliminate y. This will give us a new system of equations: 2x + 3y = 12, 12x - 3y = 30. Step 2: Then we add the first equation to the new second equation and eliminate y. This gives us: 14x = 42. Step 3: Machine Assistant: For this equation, we get x = 42 / 14 = 3.

[0101] [Seperator]

[0102] [Human]: Think step by step and solve the following system of linear equations with two variables: 2x + 3y = 12, 4x - y = 10; Step 1: We can first multiply the second equation by 3 to eliminate y. This will give a new system of equations: 2x + 3y = 12, 12x - 3y = 30; Step 2: Then we add the first equation and the new second equation and eliminate y. This will give: 14x = 42; Step 3: Solving this equation, we get x = 42 / 14 = 3. Step 4: [Machine Assistant]: Substituting x = 3 into the original first equation, we get 2 × 3 + 3y = 12, that is, 3y = 12 - 6 = 6, so y = 6 / 3 = 2. Therefore, the solution to this system of linear equations with two variables is x = 3, y = 2".

[0103] It is worth emphasizing that the above examples are merely illustrative of the form of each round of dialogue samples and do not limit the form of each round of dialogue samples.

[0104] S203: Call the language model to be optimized to perform prediction processing according to the input information of each round of dialogue samples in turn to obtain the prediction results of each round of dialogue samples.

[0105] In a specific implementation, a computer device may input N rounds of dialogue samples into a language model to be optimized, causing the language model to perform prediction processing based on the input information of each round of dialogue samples, in ascending order of rounds, to obtain a prediction result for each round of dialogue samples. Since the input information in the n-th round of dialogue samples indicates the prediction of the n-th reasoning step based on the sample question and the first n-1 reasoning steps, the language model, when performing prediction processing based on the input information in the n-th round of dialogue samples, may refer to the sample question and the first n-1 predicted reasoning steps to predict the n-th reasoning step, thereby generating a prediction result for the n-th round of dialogue samples based on the predicted n-th reasoning step.

[0106] Furthermore, based on the description of step S202, when the output information in the n-th round of dialogue sample also includes the first n-1 reasoning steps obtained from the progressive thinking chain data, the prediction result for the n-th round of dialogue sample includes the predicted n reasoning steps (i.e., the predicted n-th reasoning step and the predicted first n-th reasoning steps); when the output information in the n-th round of dialogue sample does not include the first n-1 reasoning steps obtained from the progressive thinking chain data, the prediction result for the n-th round of dialogue sample includes the predicted n-th reasoning step. Furthermore, when n = N, if the output information in the N-th round of dialogue sample includes the standard answer, the prediction result for the N-th round of dialogue sample also includes the predicted answer generated based on the predicted reasoning steps.

[0107] S204: Optimize the model parameters of the language model based on the difference between the prediction result of each round of dialogue sample and the output information in the corresponding dialogue sample.

[0108] In one specific implementation, the computer device may substitute the difference between the prediction result of each round of dialogue samples and the output information in the corresponding dialogue samples into the model loss function to perform a loss value calculation to obtain a model loss value, thereby optimizing the model parameters of the language model in a direction that reduces the model loss value. The model loss function can be set according to actual needs or empirical values, and there is no limitation on this. The so-called direction of reducing the model loss value refers to: a model optimization direction with the goal of minimizing the model loss value; performing model optimization in this direction ensures that the model loss value generated by the language model after each optimization is less than the model loss value generated by the language model before optimization. For example, if the model loss value calculated this time is 0.85, then after optimizing the model parameters of the language model in a direction that reduces the model loss value, the model loss value generated by the optimized language model should be less than 0.85.

[0109] In another specific implementation, the computer device may determine the loss value corresponding to each round of dialogue samples based on the difference between the prediction result of each round of dialogue samples and the output information in the corresponding dialogue samples. Specifically, for the nth round of dialogue samples, the computer device may use a preset loss function (a loss function preset based on actual needs or empirical values) to perform a loss value calculation based on the difference between the prediction result of the nth round of dialogue samples and the output information in the nth round of dialogue samples to obtain the loss value corresponding to the nth round of dialogue samples. The loss value and the difference are positively correlated, that is, the greater the difference, the greater the loss value. After obtaining the loss values ​​corresponding to the Nth round of dialogue samples, the computer device may integrate (sum or average) the loss values ​​corresponding to the Nth round of dialogue samples to obtain the model loss value, thereby optimizing the model parameters of the language model in the direction of reducing the model loss value.

[0110] In another specific implementation, the computer device can determine the loss value corresponding to each round of dialogue samples based on the difference between the prediction results of each round of dialogue samples and the output information in the corresponding dialogue samples; and weight the N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples. Then, the model parameters of the language model can be optimized based on the loss weight of each round of dialogue samples and the corresponding loss value. Specifically, the loss weight of each round of dialogue samples and the corresponding loss value can be weighted and summed to obtain the model loss value of the language model, thereby optimizing the model parameters of the language model in the direction of reducing the model loss value. The specific implementation method of weighting N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples can be found in the following description:

[0111] (1) When the output information in the n-th round of dialogue sample also includes: the first n-1 reasoning steps obtained from the progressive thinking chain data, since the prediction result of the n-th round of dialogue sample includes: the predicted n reasoning steps, in order to ensure that the language model can output a nearly complete answer, the computer device can be designed to have a loss weight ladder for different rounds, that is, the loss weight of the dialogue sample in the round corresponding to the incomplete answer in the previous step is low, and the loss weight of the dialogue sample in the subsequent rounds gradually increases with the completeness of the answer (that is, the completeness of the output reasoning steps). In this case, the loss weight of the n-th round of dialogue sample can be positively correlated with the value of n, that is, the larger the value of n, the larger the loss weight of the n-th round of dialogue sample; and when n=N, the loss weight of the n-th round of dialogue sample (that is, the N-th round of dialogue sample) is the baseline value (such as the value 1).

[0112] Based on this, a specific implementation method for weighting N rounds of dialogue samples to obtain the loss weight for each round of dialogue samples can be: for the nth round of dialogue samples, based on the total number of rounds N of the N rounds of dialogue samples, cosine attenuation processing is performed on the round number n corresponding to the nth round of dialogue samples to obtain the cosine attenuation value corresponding to the nth round of dialogue samples. The cosine attenuation processing refers to attenuating a value (such as the round number n) using a cosine-related method. Its attenuation graph is similar to the cosine function, and the cosine attenuation value obtained by the cosine attenuation processing is negatively correlated with the value of n. Then, the difference between the baseline value and the cosine attenuation value corresponding to the nth round of dialogue samples is determined, and the determined difference is mapped to a preset numerical range to obtain the loss weight for the nth round of dialogue samples. The preset numerical range mentioned here refers to an interval consisting of zero and the baseline value, such as [0, 1]. For example, based on this implementation, taking the baseline value equal to 1 as an example, the loss weight for the nth round of dialogue samples can be calculated using the following formula:

[0113] Loss_sample_mask_curstep=[(1-math.cos(step×3.14159 / total_step)) / 2]

[0114] Among them, loss_sample_mask_curstep represents the loss weight of the n-th round dialogue sample, step represents the round number n corresponding to the n-th round dialogue sample, and total_step represents the total round number N of N-round dialogue samples; math.cos(step×3.14159 / total_step) represents the cosine attenuation value corresponding to the n-th round dialogue sample, and its value range is [-1, 1]. When step = total_step, the cosine attenuation value is -1; 1-math.cos(step×3.14159 / total_step) represents the determined difference, and (1-math.cos(step×3.14159 / total_step)) / 2 represents mapping the determined difference to the interval [0, 1].

[0115] For example, let N = 4 (i.e. the progressive thinking chain data includes sample questions and 4 reasoning steps), then see Figure 3It can be seen that after the computer device uses the progressive thought chain data to step-by-step construct four rounds of dialogue samples, the above formula can be used to calculate the loss weights of the first round of dialogue samples to be 0.146, the loss weight of the second round of dialogue samples to be 0.500, the loss weight of the third round of dialogue samples to be 0.854, and the loss weight of the fourth round of dialogue samples to be 1. This shows that constructing a loss weight ladder using the above formula can make the loss weights corresponding to dialogue samples in different rounds change nonlinearly, with the loss weights of dialogue samples in the early rounds being low and the loss weights of dialogue samples in the later rounds gradually increasing with the completeness of the answer, thus ensuring that the model can provide answers that tend to be complete and correct.

[0116] It should be understood that the above is merely an example of a specific implementation of weight assignment and is not exhaustive. For example, in other embodiments, when assigning weights to N rounds of dialogue samples to obtain the loss weight for each round, the computer device may also determine the ratio between the baseline value and the total number of rounds N of the N rounds of dialogue samples. For the nth round of dialogue samples, the ratio is multiplied by the number of rounds N corresponding to the nth round of dialogue samples to obtain the loss weight for the nth round of dialogue samples. For example, assuming N = 10 and the baseline value is 1, then the ratio between the baseline value and N is 0.1. The loss weights for the 10 rounds of dialogue samples are, in order: 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, and 1.0. Alternatively, when assigning weights to N rounds of dialogue samples to obtain the loss weight for each round of dialogue samples, the computer device may also perform a multiplication process on the preset value based on the number of rounds of each dialogue sample to obtain an incremental result for each round of dialogue samples; and perform a normalization process on the incremental result for each round of dialogue samples based on the maximum incremental result (i.e., dividing the incremental result for each round of dialogue samples by the maximum incremental result) to obtain the loss weight for each round of dialogue samples. For example, assuming N = 4 and the preset value is 0.1, the incremental results for the four rounds of dialogue samples are, in order: 0.1, 0.2, 0.3, and 0.4; then, the incremental results for each round of dialogue samples are normalized using the maximum incremental result (i.e., 0.4), i.e., dividing the incremental result for each round of dialogue samples by the maximum incremental result, to obtain the loss weights for the four rounds of dialogue samples, in order: 0.25, 0.05, 0.75, and 1.0. It can be seen from this that by constructing the loss weight ladder in a multiplier manner, the loss weights corresponding to dialogue samples in different rounds can change linearly.

[0117] (2) When the output information in the n-th round of dialogue samples does not include the first n-1 reasoning steps obtained from the progressive thought chain data, since the prediction result of the n-th round of dialogue samples includes the predicted n-th reasoning step, the language model only predicts one step in each round. Therefore, the impact of each round of prediction results on the language model can be the same, which can ensure that the language model learns the logical relationship in each round of dialogue samples equally. In this case, the loss weight of each round of dialogue samples can be the same.

[0118] Based on this, the specific implementation method of weighting N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples can be: the computer device can obtain a preset weight value as the loss weight of each round of dialogue samples; or the ratio between the benchmark value and the total number of rounds N of N rounds of dialogue samples is used as the loss weight of each round of dialogue samples. For example, if N = 4 (i.e., the progressive thinking chain data includes sample questions and 4 reasoning steps), and the benchmark value is 1, then see Figure 4 It can be seen that after the computer device uses the progressive thinking chain data to step-by-step construct four rounds of dialogue samples, it can calculate that the ratio between the baseline value and N is 0.25. Therefore, the loss weight corresponding to each round of dialogue samples can be set to 0.25.

[0119] Furthermore, after optimizing the model parameters of the language model through step S204, the speech model can be directly put into use; alternatively, it can be determined whether the language model has reached convergence. If the speech model has reached convergence, the converged speech model can be put into use; if it has not reached convergence, the above steps S201-S204 can be repeated to optimize the model parameters of the language model again until the language model reaches convergence, thereby putting the converged speech model into use. Wherein, the language model reaching convergence means: the number of training times of the language model reaches a preset number; alternatively, the language model reaching convergence means: the model parameters of the language model no longer change, or the amplitude of the change in the model parameters is less than a threshold; alternatively, the language model reaching convergence means: the model loss value of the language model no longer decreases, or the amplitude of the decrease in the model loss value is less than a threshold, etc.

[0120] Furthermore, after the language model is put into use, the computer device can obtain the target problem to be solved and generate a target instruction text based on the target problem. The target instruction text is used to instruct: to perform step-by-step reasoning on the target problem and generate a predicted answer; then, the language model can be called to perform prediction processing based on the target instruction text to obtain the predicted answer corresponding to the target problem. Optionally, the target instruction text can also be used to indicate the individual reasoning steps generated during the output reasoning process; in this case, when the language model is called to perform prediction processing based on the target instruction text, the individual reasoning steps used to infer the corresponding predicted answer can also be obtained, thereby making the language model's reasoning process visual and improving the credibility of the predicted answer.

[0121] The embodiment of the present application can obtain progressive thinking chain data, which includes a sample question and N reasoning steps for inferring the standard answer corresponding to the sample question; and use the progressive thinking chain data to construct N rounds of dialogue samples step by step, each round of dialogue samples includes input information and output information, and the input information in the nth round of dialogue sample is used to indicate: predicting the nth reasoning step based on the sample question and the first n-1 reasoning steps, and the output information in the nth round of dialogue sample at least includes: the nth reasoning step obtained from the progressive thinking chain data. Then, the language model to be optimized is called to perform prediction processing according to the input information of each round of dialogue samples in turn, and the model parameters of the language model are optimized based on the difference between the prediction results of each round of dialogue samples and the output information in the corresponding dialogue samples. It can be seen that this method can enable the language model to perform multiple rounds of predictions, and each round adds a new reasoning step prediction work, which can better guide the language model to perform step-by-step reasoning on sample problems and improve the logical reasoning ability of the language model; and as the number of prediction rounds n increases, the context information referred to by the language model during prediction (such as the first n-1 reasoning steps) also increases accordingly, which can enable the language model to enrich the logical knowledge between the learned reasoning steps in each round, thereby further improving the logical reasoning ability of the language model.

[0122] Based on the description of the above method embodiment, the present application embodiment further discloses a model processing device; the model processing device can be a computer program (including one or more instructions) running on a computer device, and the model processing device can be used to execute each step in the above method flow. Figure 5 , the model processing device can run the following units:

[0123] An acquisition unit 501 is configured to acquire progressive thinking chain data, wherein the progressive thinking chain data includes: a sample question and N reasoning steps, wherein the N reasoning steps are used to infer a standard answer corresponding to the sample question, where N is a positive integer;

[0124] Processing unit 502 is configured to construct N rounds of dialogue samples step by step using the progressive thought chain data, where each round of dialogue samples includes input information and output information; the input information in the n-th round of dialogue sample is used to indicate: predicting the n-th reasoning step based on the sample question and the first n-1 reasoning steps; the output information in the n-th round of dialogue sample at least includes: the n-th reasoning step obtained from the progressive thought chain data, where n∈[1,N];

[0125] The processing unit 502 is further configured to call the language model to be optimized to perform prediction processing based on the input information of each round of dialogue samples in sequence, thereby obtaining a prediction result for each round of dialogue samples;

[0126] The processing unit 502 is further configured to optimize the model parameters of the language model based on the difference between the prediction result of each round of dialogue samples and the output information in the corresponding dialogue samples.

[0127] In one embodiment, when the processing unit 502 is configured to optimize the model parameters of the language model based on the difference between the prediction result of each round of dialogue sample and the output information in the corresponding dialogue sample, it can be specifically configured to:

[0128] Determining a loss value corresponding to each round of dialogue sample based on a difference between a prediction result of each round of dialogue sample and output information in the corresponding dialogue sample;

[0129] Performing weight distribution on the N rounds of dialogue samples to obtain a loss weight for each round of dialogue samples;

[0130] Optimizing model parameters of the language model based on the loss weight and corresponding loss value of each round of dialogue samples.

[0131] In another embodiment, when the output information in the n-th round of dialogue sample also includes: the first n-1 reasoning steps obtained from the progressive thinking chain data, the prediction result of the n-th round of dialogue sample includes: the predicted n reasoning steps; wherein the loss weight of the n-th round of dialogue sample is positively correlated with the value of n; and when n=N, the loss weight of the n-th round of dialogue sample is a baseline value.

[0132] Accordingly, when the processing unit 502 is used to perform weight distribution on the N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples, it can be specifically used to:

[0133] For the n-th round of dialogue sample, based on the total number of rounds N of the N-round dialogue samples, perform cosine attenuation processing on the number of rounds n corresponding to the n-round dialogue sample to obtain a cosine attenuation value corresponding to the n-round dialogue sample; wherein the cosine attenuation value is negatively correlated with the value of n;

[0134] Determine a difference between a baseline value and a cosine attenuation value corresponding to the n-th round of dialogue sample, and map the determined difference to a preset numerical range to obtain a loss weight for the n-th round of dialogue sample; wherein the preset numerical range is an interval consisting of a zero value and a baseline value.

[0135] In another embodiment, when the output information in the n-th round of dialogue sample also includes: the first n-1 reasoning steps obtained from the progressive thinking chain data, the prediction result of the n-th round of dialogue sample includes: the predicted n reasoning steps; wherein the loss weight of the n-th round of dialogue sample is positively correlated with the value of n; and when n=N, the loss weight of the n-th round of dialogue sample is a baseline value.

[0136] Accordingly, when the processing unit 502 is used to perform weight distribution on the N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples, it can be specifically used to:

[0137] Determine a ratio between a benchmark value and a total number N of rounds of the N-round dialogue sample;

[0138] For the n-th round dialogue sample, the ratio is multiplied according to the round number n corresponding to the n-th round dialogue sample to obtain the loss weight of the n-th round dialogue sample.

[0139] In another embodiment, the prediction result of the n-th round of dialogue samples includes: the predicted n-th reasoning step; wherein the loss weight of each round of dialogue samples is the same.

[0140] Accordingly, when the processing unit 502 is used to perform weight distribution on the N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples, it can be specifically used to:

[0141] The ratio between the benchmark value and the total number of rounds N of the N rounds of dialogue samples is used as the loss weight of each round of dialogue samples;

[0142] Alternatively, obtain a preset weight value as the loss weight for each round of dialogue samples.

[0143] In another embodiment, when the processing unit 502 is configured to optimize the model parameters of the language model based on the loss weight and the corresponding loss value of each round of dialogue samples, it may be specifically configured to:

[0144] Performing weighted summation on the loss weight and corresponding loss value of each round of dialogue samples to obtain a model loss value of the language model;

[0145] Optimize the model parameters of the language model in a direction of reducing the model loss value.

[0146] In another embodiment, the progressive thinking chain data also includes the standard answer; when n=N, the output information in the n-th round of dialogue sample also includes the standard answer; the prediction result of the n-th round of dialogue sample includes: the predicted reasoning steps and the predicted answer generated based on the predicted reasoning steps.

[0147] Accordingly, after optimizing the model parameters of the language model, the processing unit 502 may further be configured to:

[0148] Obtaining a target problem to be solved, and generating a target instruction text based on the target problem, wherein the target instruction text is used to instruct: performing step-by-step reasoning on the target problem and generating a predicted answer;

[0149] The language model is called to perform prediction processing based on the target instruction text to obtain a predicted answer corresponding to the target question.

[0150] In another embodiment, when the acquisition unit 501 is used to acquire the progressive thinking chain data, it can be specifically used to:

[0151] Obtain prompt text, a case, and a sample question; the case includes a sample question and inference data output after reasoning on the sample question, the inference data including at least one inference step corresponding to the sample question and the inferred answer; the prompt text is used to prompt: perform step-by-step reasoning on the question in the input field and output inference data, and keep the format of the output inference data consistent with the format of the inference data in the case;

[0152] Generate a task description instruction using the prompt text, the case, and the sample question; wherein the task description instruction includes the input field, and the sample question is located in the input field;

[0153] The text generation model is called to perform task processing according to the task description instruction to output an inference data; and the inference data output by the text generation model is used as progressive thinking chain data.

[0154] According to another embodiment of the present application, Figure 5 The various units in the model processing device shown can be separately or all merged into one or several other units to constitute, or a certain unit (or units) therein can also be split into multiple smaller units in function to constitute, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, other units can also be included based on the model processing device. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0155] According to another embodiment of the present application, a computer program (including one or more instructions) capable of executing the steps involved in the above method embodiment can be constructed by running the computer program (including one or more instructions) on a general computing device such as a computer including processing elements and storage elements such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM). Figure 5 The model processing device shown in the embodiment of the present application is used to implement the model processing method of the embodiment of the present application. The computer program can be recorded on a computer-readable storage medium, for example, and loaded into the above-mentioned computing device through the computer-readable storage medium and run therein.

[0156] It is worth noting that, in the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can include a part of the overall module or unit of the module or unit function.

[0157] The embodiment of the present application can obtain progressive thinking chain data, which includes a sample question and N reasoning steps for inferring the standard answer corresponding to the sample question; and use the progressive thinking chain data to construct N rounds of dialogue samples step by step, each round of dialogue samples includes input information and output information, and the input information in the nth round of dialogue sample is used to indicate: predicting the nth reasoning step based on the sample question and the first n-1 reasoning steps, and the output information in the nth round of dialogue sample at least includes: the nth reasoning step obtained from the progressive thinking chain data. Then, the language model to be optimized is called to perform prediction processing according to the input information of each round of dialogue samples in turn, and the model parameters of the language model are optimized based on the difference between the prediction results of each round of dialogue samples and the output information in the corresponding dialogue samples. It can be seen that this method can enable the language model to perform multiple rounds of predictions, and each round adds a new reasoning step prediction work, which can better guide the language model to perform step-by-step reasoning on sample problems and improve the logical reasoning ability of the language model; and as the number of prediction rounds n increases, the context information referred to by the language model during prediction (such as the first n-1 reasoning steps) also increases accordingly, which can enable the language model to enrich the logical knowledge between the learned reasoning steps in each round, thereby further improving the logical reasoning ability of the language model.

[0158] Based on the description of the above method embodiment and apparatus embodiment, the present application embodiment also provides a computer device. Figure 6, the computer device at least includes a processor 601, an input interface 602, an output interface 603 and a computer storage medium 604. Among them, the processor 601, input interface 602, output interface 603 and computer storage medium 604 in the computer device can be connected via a bus or other means. The computer storage medium 604 can be stored in the memory of the computer device, and the computer storage medium 604 is used to store a computer program, and the computer program includes one or more instructions. The processor 601 is used to execute one or more instructions in the computer program stored in the computer storage medium 604. The processor 601 (or CPU (Central Processing Unit)) is the computing core and control core of the computer device, which is suitable for implementing one or more instructions, and is specifically suitable for loading and executing one or more instructions to realize the corresponding method flow or corresponding function.

[0159] In one embodiment, the processor 601 described in the embodiment of the present application can be used to perform a series of optimization processes on the language model, specifically including: an acquisition unit, used to obtain progressive thinking chain data, the progressive thinking chain data including: a sample question and N reasoning steps, the N reasoning steps being used to infer the standard answer corresponding to the sample question, where N is a positive integer; a processing unit, used to use the progressive thinking chain data to construct N rounds of dialogue samples step by step, each round of dialogue samples including input information and output information; the input information in the nth round of dialogue sample is used to indicate: predicting the nth reasoning step based on the sample question and the first n-1 reasoning steps; the output information in the nth round of dialogue sample at least includes: the nth reasoning step obtained from the progressive thinking chain data, n∈[1, N]; the processing unit is also used to call the language model to be optimized to perform prediction processing according to the input information of each round of dialogue sample in turn to obtain the prediction result of each round of dialogue sample; the processing unit is also used to optimize the model parameters of the language model based on the difference between the prediction result of each round of dialogue sample and the output information in the corresponding dialogue sample, and so on.

[0160] The embodiment of the present application also provides a computer storage medium (Memory), which is a memory device in a computer device for storing computer programs and data. It is understandable that the computer storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer storage medium provides a storage space, which stores the operating system of the computer device. In addition, a computer program is also stored in the storage space, which includes one or more instructions suitable for being loaded and executed by the processor 601, and these instructions can be one or more program codes. It should be noted that the computer storage medium here can be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage; optionally, it can also be at least one computer storage medium located away from the aforementioned processor.

[0161] In one embodiment, a processor may load and execute one or more instructions stored in a computer storage medium to implement the corresponding steps in the above-mentioned embodiment of the model processing method. In a specific implementation, the processor may load and execute the following steps:

[0162] Acquire progressive thinking chain data, the progressive thinking chain data including: a sample question and N reasoning steps, the N reasoning steps being used to infer a standard answer corresponding to the sample question, where N is a positive integer;

[0163] N rounds of dialogue samples are constructed step by step using the progressive thought chain data, each round of dialogue samples including input information and output information; the input information in the n-th round of dialogue sample is used to indicate: predicting the n-th reasoning step based on the sample question and the first n-1 reasoning steps; the output information in the n-th round of dialogue sample at least includes: the n-th reasoning step obtained from the progressive thought chain data, n∈[1,N];

[0164] Calling the language model to be optimized to perform prediction processing based on the input information of each round of dialogue samples in turn to obtain the prediction results of each round of dialogue samples;

[0165] Optimizing model parameters of the language model based on the difference between the prediction result of each round of dialogue samples and the output information in the corresponding dialogue samples.

[0166] In one embodiment, when optimizing the model parameters of the language model based on the difference between the prediction result of each round of dialogue sample and the output information in the corresponding dialogue sample, the one or more instructions may be loaded and specifically executed by the processor:

[0167] Determining a loss value corresponding to each round of dialogue sample based on a difference between a prediction result of each round of dialogue sample and output information in the corresponding dialogue sample;

[0168] Performing weight distribution on the N rounds of dialogue samples to obtain a loss weight for each round of dialogue samples;

[0169] Optimizing model parameters of the language model based on the loss weight and corresponding loss value of each round of dialogue samples.

[0170] In another embodiment, when the output information in the n-th round of dialogue sample also includes: the first n-1 reasoning steps obtained from the progressive thinking chain data, the prediction result of the n-th round of dialogue sample includes: the predicted n reasoning steps; wherein the loss weight of the n-th round of dialogue sample is positively correlated with the value of n; and when n=N, the loss weight of the n-th round of dialogue sample is a baseline value.

[0171] Accordingly, when weights are assigned to the N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples, the one or more instructions may be loaded and specifically executed by the processor:

[0172] For the n-th round of dialogue sample, based on the total number of rounds N of the N-round dialogue samples, perform cosine attenuation processing on the number of rounds n corresponding to the n-round dialogue sample to obtain a cosine attenuation value corresponding to the n-round dialogue sample; wherein the cosine attenuation value is negatively correlated with the value of n;

[0173] Determine a difference between a baseline value and a cosine attenuation value corresponding to the n-th round of dialogue sample, and map the determined difference to a preset numerical range to obtain a loss weight for the n-th round of dialogue sample; wherein the preset numerical range is an interval consisting of a zero value and a baseline value.

[0174] In another embodiment, when the output information in the n-th round of dialogue sample also includes: the first n-1 reasoning steps obtained from the progressive thinking chain data, the prediction result of the n-th round of dialogue sample includes: the predicted n reasoning steps; wherein the loss weight of the n-th round of dialogue sample is positively correlated with the value of n; and when n=N, the loss weight of the n-th round of dialogue sample is a baseline value.

[0175] Accordingly, when weights are assigned to the N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples, the one or more instructions may be loaded and specifically executed by the processor:

[0176] Determine a ratio between a benchmark value and a total number N of rounds of the N-round dialogue sample;

[0177] For the n-th round dialogue sample, the ratio is multiplied according to the round number n corresponding to the n-th round dialogue sample to obtain the loss weight of the n-th round dialogue sample.

[0178] In another embodiment, the prediction result of the n-th round of dialogue samples includes: the predicted n-th reasoning step; wherein the loss weight of each round of dialogue samples is the same.

[0179] Accordingly, when weights are assigned to the N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples, the one or more instructions may be loaded and specifically executed by the processor:

[0180] The ratio between the benchmark value and the total number of rounds N of the N rounds of dialogue samples is used as the loss weight of each round of dialogue samples;

[0181] Alternatively, obtain a preset weight value as the loss weight for each round of dialogue samples.

[0182] In another embodiment, when optimizing the model parameters of the language model based on the loss weight and the corresponding loss value of each round of dialogue samples, the one or more instructions may be loaded and specifically executed by the processor:

[0183] Performing weighted summation on the loss weight and corresponding loss value of each round of dialogue samples to obtain a model loss value of the language model;

[0184] Optimize the model parameters of the language model in a direction of reducing the model loss value.

[0185] In another embodiment, the progressive thinking chain data also includes the standard answer; when n=N, the output information in the n-th round of dialogue sample also includes the standard answer; the prediction result of the n-th round of dialogue sample includes: the predicted reasoning steps and the predicted answer generated based on the predicted reasoning steps.

[0186] Accordingly, after optimizing the model parameters of the language model, the one or more instructions may be loaded and specifically executed by the processor:

[0187] Obtaining a target problem to be solved, and generating a target instruction text based on the target problem, wherein the target instruction text is used to instruct: performing step-by-step reasoning on the target problem and generating a predicted answer;

[0188] The language model is called to perform prediction processing based on the target instruction text to obtain a predicted answer corresponding to the target question.

[0189] In another embodiment, when acquiring the progressive thinking chain data, the one or more instructions may be loaded and specifically executed by the processor:

[0190] Obtain prompt text, a case, and a sample question; the case includes a sample question and inference data output after reasoning on the sample question, the inference data including at least one inference step corresponding to the sample question and the inferred answer; the prompt text is used to prompt: perform step-by-step reasoning on the question in the input field and output inference data, and keep the format of the output inference data consistent with the format of the inference data in the case;

[0191] Generate a task description instruction using the prompt text, the case, and the sample question; wherein the task description instruction includes the input field, and the sample question is located in the input field;

[0192] The text generation model is called to perform task processing according to the task description instruction to output an inference data; and the inference data output by the text generation model is used as progressive thinking chain data.

[0193] The embodiment of the present application can obtain progressive thinking chain data, which includes a sample question and N reasoning steps for inferring the standard answer corresponding to the sample question; and use the progressive thinking chain data to construct N rounds of dialogue samples step by step, each round of dialogue samples includes input information and output information, and the input information in the nth round of dialogue sample is used to indicate: predicting the nth reasoning step based on the sample question and the first n-1 reasoning steps, and the output information in the nth round of dialogue sample at least includes: the nth reasoning step obtained from the progressive thinking chain data. Then, the language model to be optimized is called to perform prediction processing according to the input information of each round of dialogue samples in turn, and the model parameters of the language model are optimized based on the difference between the prediction results of each round of dialogue samples and the output information in the corresponding dialogue samples. It can be seen that this method can enable the language model to perform multiple rounds of predictions, and each round adds a new reasoning step prediction work, which can better guide the language model to perform step-by-step reasoning on sample problems and improve the logical reasoning ability of the language model; and as the number of prediction rounds n increases, the context information referred to by the language model during prediction (such as the first n-1 reasoning steps) also increases accordingly, which can enable the language model to enrich the logical knowledge between the learned reasoning steps in each round, thereby further improving the logical reasoning ability of the language model.

[0194] It should be noted that, according to one aspect of the present application, a computer program product or computer program is also provided, which includes one or more instructions, and the one or more instructions are stored in a computer storage medium. The processor of the computer device reads the one or more instructions from the computer storage medium, and the processor executes the one or more instructions, so that the computer device performs the methods provided in various optional ways in the above-mentioned method embodiments. It should be understood that what is disclosed above is only a preferred embodiment of the present application, and it is certainly not used to limit the scope of the rights of the present application. Therefore, equivalent changes made in accordance with the claims of the present application are still within the scope covered by the present application.

Claims

1. A model processing method, characterized in that: include: Acquire progressive thinking chain data, the progressive thinking chain data including: a sample question and N reasoning steps, the N reasoning steps being used to infer a standard answer corresponding to the sample question, where N is a positive integer; N rounds of dialogue samples are constructed step by step using the progressive thought chain data, each round of dialogue samples including input information and output information; the input information in the n-th round of dialogue sample is used to indicate: predicting the n-th reasoning step based on the sample question and the first n-1 reasoning steps; the output information in the n-th round of dialogue sample at least includes: the n-th reasoning step obtained from the progressive thought chain data, n∈[1,N]; Calling the language model to be optimized to perform prediction processing based on the input information of each round of dialogue samples in turn to obtain the prediction results of each round of dialogue samples; Optimizing model parameters of the language model based on the difference between the prediction result of each round of dialogue samples and the output information in the corresponding dialogue samples.

2. The method according to claim 1, wherein Optimizing the model parameters of the language model based on the difference between the prediction result of each round of dialogue sample and the output information in the corresponding dialogue sample includes: Determining a loss value corresponding to each round of dialogue sample based on a difference between a prediction result of each round of dialogue sample and output information in the corresponding dialogue sample; Performing weight distribution on the N rounds of dialogue samples to obtain a loss weight for each round of dialogue samples; Optimizing model parameters of the language model based on the loss weight and corresponding loss value of each round of dialogue samples.

3. The method according to claim 2, wherein When the output information in the n-th round of dialogue sample further includes: the first n-1 reasoning steps obtained from the progressive thinking chain data, the prediction result of the n-th round of dialogue sample includes: the predicted n reasoning steps; The loss weight of the n-th round of dialogue sample is positively correlated with the value of n; and when n=N, the loss weight of the n-th round of dialogue sample is a benchmark value.

4. The method according to claim 3, wherein The weighting of the N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples includes: For the n-th round of dialogue sample, based on the total number of rounds N of the N-round dialogue samples, perform cosine attenuation processing on the number of rounds n corresponding to the n-round dialogue sample to obtain a cosine attenuation value corresponding to the n-round dialogue sample; wherein the cosine attenuation value is negatively correlated with the value of n; Determine a difference between a baseline value and a cosine attenuation value corresponding to the n-th round of dialogue sample, and map the determined difference to a preset numerical range to obtain a loss weight for the n-th round of dialogue sample; wherein the preset numerical range is an interval consisting of a zero value and a baseline value.

5. The method according to claim 3, wherein The weighting of the N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples includes: Determine a ratio between a benchmark value and a total number N of rounds of the N-round dialogue sample; For the n-th round dialogue sample, the ratio is multiplied according to the round number n corresponding to the n-th round dialogue sample to obtain the loss weight of the n-th round dialogue sample.

6. The method according to claim 2, wherein The prediction result of the n-th round of dialogue samples includes: the predicted n-th reasoning step; wherein the loss weight of each round of dialogue samples is the same.

7. The method according to claim 6, wherein The weighting of the N rounds of dialogue samples to obtain the loss weight of each round of dialogue samples includes: The ratio between the benchmark value and the total number of rounds N of the N rounds of dialogue samples is used as the loss weight of each round of dialogue samples; Alternatively, obtain a preset weight value as the loss weight for each round of dialogue samples.

8. The method according to any one of claims 2 to 7, wherein: Optimizing the model parameters of the language model based on the loss weight and the corresponding loss value of each round of dialogue samples includes: Performing weighted summation on the loss weight and corresponding loss value of each round of dialogue samples to obtain a model loss value of the language model; Optimize the model parameters of the language model in a direction of reducing the model loss value.

9. The method according to claim 1, wherein The progressive thinking chain data also includes the standard answer; When n=N, the output information in the n-th round of dialogue sample also includes the standard answer; the prediction result of the n-th round of dialogue sample includes: the predicted reasoning steps and the predicted answer generated based on the predicted reasoning steps.

10. The method according to claim 9, wherein After optimizing the model parameters of the language model, the method further includes: Obtaining a target problem to be solved, and generating a target instruction text based on the target problem, wherein the target instruction text is used to instruct: performing step-by-step reasoning on the target problem and generating a predicted answer; The language model is called to perform prediction processing based on the target instruction text to obtain a predicted answer corresponding to the target question.

11. The method according to claim 1, wherein The step of obtaining the progressive thinking chain data includes: Obtain prompt text, a case, and a sample question; the case includes a sample question and inference data output after reasoning on the sample question, the inference data including at least one inference step corresponding to the sample question and the inferred answer; the prompt text is used to prompt: perform step-by-step reasoning on the question in the input field and output inference data, and keep the format of the output inference data consistent with the format of the inference data in the case; Generate a task description instruction using the prompt text, the case, and the sample question; wherein the task description instruction includes the input field, and the sample question is located in the input field; The text generation model is called to perform task processing according to the task description instruction to output an inference data; and the inference data output by the text generation model is used as progressive thinking chain data.

12. A model processing device, characterized in that: include: an acquisition unit, configured to acquire progressive thinking chain data, wherein the progressive thinking chain data includes: a sample question and N reasoning steps, wherein the N reasoning steps are used to infer a standard answer corresponding to the sample question, where N is a positive integer; a processing unit configured to construct N rounds of dialogue samples step by step using the progressive thought chain data, each round of dialogue samples including input information and output information; the input information in the n-th round of dialogue sample being used to indicate: predicting the n-th reasoning step based on the sample question and the first n-1 reasoning steps; and the output information in the n-th round of dialogue sample at least including: the n-th reasoning step obtained from the progressive thought chain data, where n∈[1,N]; The processing unit is further configured to call the language model to be optimized to perform prediction processing based on the input information of each round of dialogue samples in sequence, thereby obtaining a prediction result for each round of dialogue samples; The processing unit is further configured to optimize the model parameters of the language model based on the difference between the prediction result of each round of dialogue samples and the output information in the corresponding dialogue samples.

13. A computer device comprising an input interface and an output interface, characterized in that: Also includes: processors and computer storage media; The processor is suitable for implementing one or more instructions, the computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by the processor and executing the model processing method according to any one of claims 1 to 11.

14. A computer storage medium, characterized in that The computer storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the model processing method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product includes one or more instructions; when the one or more instructions in the computer program are executed by a processor, the model processing method according to any one of claims 1 to 11 is implemented.

Images