Training of task processing model and task processing method and device

Abstract

The application discloses training and task processing method and equipment of task processing model, relates to the field of artificial intelligence technology, target sample data includes historical task and first task processing result; the historical task is input to the task processing model, and the second task processing result is output, and the overthinking score of the task processing model in the task processing process is determined; if the overthinking score is greater than the preset threshold, the position information when the first task processing result first appears in the second task processing result is determined, the second task processing result is divided, and the basic processing result and the evolution processing result of the historical task are obtained; the task processing model is fine-tuned and trained based on the preprocessed evolution processing result, the basic processing result and the sample data set constructed by using the target sample data, the problem of overthinking of the model in complex task processing is avoided, the task processing efficiency and the accuracy are improved, and the task processing effectiveness and the overall performance are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a task processing model training method and device and a task processing method and device. BACKGROUND

[0002] Current solutions to overthinking include: optimizing models and data, mainly RL (Reinforcement learning) design of length-sensitive rewards to limit output, and using variable-length CoT (Chain of Thought) data and auxiliary constraints through SFT (Supervised Fine-Tuning) to shorten the reasoning path; optimizing reasoning methods by changing the generation strategy to reduce the length of reasoning. The disadvantages are as follows: limiting the ability of the pre-training model to constantly explore multiple strategies when processing complex tasks, resulting in reduced accuracy, and often failing to occur, resulting in the pre-training model processing tasks inefficiently and inaccurately.

[0003] Therefore, how to avoid the problem of overthinking in the model in complex task processing, improve the task processing efficiency and accuracy of the model, and improve the task processing effectiveness and overall performance of the model is a problem to be solved by those skilled in the art. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a task processing model training method and device and a task processing method and device, which can avoid the problem of overthinking in the model in complex task processing, improve the task processing efficiency and accuracy of the model, and improve the task processing effectiveness and overall performance of the model. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a task processing model training method, comprising:

[0006] Obtaining target sample data; the target sample data includes historical tasks and corresponding first task processing results;

[0007] Inputting the historical tasks into a task processing model to output second task processing results, and determining an overthinking score of the task processing model in the task processing process based on the second task processing results; the task processing model is a model constructed based on a pre-training model;

[0008] If the overthinking score is greater than a preset threshold, determining position information of the first time when the first task processing result appears from the second task processing result, dividing the second task processing result based on the position information to obtain a basic processing result and an evolution processing result of the historical task;

[0009] The evolution processing result is preprocessed, and a target task processing model is fine-tuned and trained based on the preprocessed evolution processing result, a basic processing result, and a sample data set constructed by using target sample data, to obtain the target task processing model.

[0010] In a second aspect, the present application discloses a task processing method, comprising:

[0011] Obtaining a task to be processed;

[0012] Processing the task to be processed by using the target task processing model to obtain a corresponding task processing result; wherein the target task processing model is a model trained by using the training method of the task processing model.

[0013] In a third aspect, the present application discloses an electronic device, comprising:

[0014] A memory for storing a computer program;

[0015] A processor for executing the computer program to implement the steps of the foregoing method.

[0016] It can be seen that the application provides a task processing model training method, including obtaining target sample data; the target sample data includes a historical task and a corresponding first task processing result; inputting the historical task into a task processing model to output a second task processing result, and determining an overthinking score of the task processing model in a task processing process based on the second task processing result; the task processing model is a model constructed based on a pre-training model; if the overthinking score is greater than a preset threshold, determining position information when the first task processing result first appears from the second task processing result, dividing the second task processing result based on the position information to obtain a basic processing result and an evolution processing result of the historical task; preprocessing the evolution processing result, and fine-tuning the task processing model based on the preprocessed evolution processing result, the basic processing result, and a sample data set constructed by using the target sample data to obtain a target task processing model. The application obtains target sample data, inputs the historical task into a task processing model to output a second task processing result, determines an overthinking score of the task processing model in a task processing process based on the second task processing result, improves the accuracy of the output of the task processing model, can comprehensively evaluate the processing efficiency of the model, if the overthinking score is greater than a preset threshold, determines position information when the first task processing result first appears from the second task processing result, divides the second task processing result to obtain a basic processing result and an evolution processing result of the historical task, ensures that the model can distinguish the clear boundary between necessary reasoning and excessive calculation, pre-processes the evolution processing result, can strengthen the ability of the model to stop reasoning independently, reduces the phenomenon of excessive reasoning in the original task processing process, fine-tunes the task processing model based on the preprocessed evolution processing result, the basic processing result, and a sample data set constructed by using the target sample data to obtain a target task processing model, so that the model can learn to distinguish between effective reasoning and redundant reasoning, avoids the problem of overthinking of the pre-training model in complex task processing, and maintains the ability of the pre-training model to solve complex problems. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 A flowchart of a task processing model training method disclosed by the application;

[0019] Figure 2 A flowchart of a task processing method disclosed by the application;

[0020] Figure 3 A specific flowchart for implementing model training and code generation is provided in the present application.

[0021] Figure 4 A training device structure diagram of a task processing model disclosed in the present application is shown. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0023] Current solutions to overthinking include optimizing the model and data, mainly RL design of length-sensitive rewards to limit output, and using variable-length CoT data and auxiliary constraints with SFT to shorten the reasoning path; optimizing the reasoning method by changing the generation strategy to reduce the length of reasoning. The disadvantages are as follows: limiting the ability of the pre-training model to constantly explore multiple strategies when processing complex tasks, resulting in reduced accuracy, often with failure, resulting in low efficiency and low accuracy of the pre-training model when processing tasks. Therefore, how to avoid the problem of overthinking of the model in complex task processing, improve the efficiency and accuracy of the model in task processing, and improve the effectiveness and overall performance of the model in task processing are problems that need to be solved by those skilled in the art.

[0024] Referring to Figure 1 The embodiments of the present application disclose a task processing model training method, which can specifically include:

[0025] Step S11: obtaining target sample data; the target sample data includes historical tasks and corresponding first task processing results.

[0026] Step S12: inputting the historical tasks into the task processing model to output second task processing results, and determining the overthinking score of the task processing model in the task processing process based on the second task processing results; the task processing model is a model constructed based on a pre-training model.

[0027] In order to solve the problem of overthinking of the task processing model, the present application first identifies the model or data in the mode that has overthinking phenomenon, therefore, the overthinking score is introduced, which is used as an overthinking measurement index. The overthinking score is obtained by integrating two complementary indexes, including a task processing efficiency ratio and an overthinking efficiency ratio, through a composite scoring mechanism.

[0028] The task processing model can derive the correct answer relatively early in the reasoning process, but will continue to generate additional solution attempts. To quantify this inefficiency, the application proposes a method for training the task processing model.

[0029] Taking the task processing model as an example, LCoT (Long Chain-of-Thought) is a large language model that can generate longer reasoning processes, including OpenAI-o3 (an upgraded version of OpenAI's reasoning model) and DeepSeek-R1 series of models. It simulates human step-by-step thinking to solve complex problems. Compared to the basic CoT, the long chain-of-thought model can handle more complex and step-by-step reasoning tasks, maintain longer logical coherence, and its typical feature is that it may use a multi-round self-validation mechanism.

[0030] In this embodiment, the task processing efficiency ratio of the task processing model in the task processing process is calculated based on the second task processing result; the overthinking efficiency ratio of the task processing model in the task processing process is calculated using the second task processing result; and the task processing efficiency ratio and the overthinking efficiency ratio are weighted and summed to obtain the overthinking score.

[0031] Among them, the task processing efficiency ratio of the task processing model in the task processing process is calculated based on the second task processing result, including: obtaining the task processing information of the task processing model in the task processing process; determining the number of task processing steps required when the first task processing result first appears from the task processing information; determining the total number of task processing steps of the task processing information; and taking the ratio between the number of task processing steps and the total number of task processing steps as the task processing efficiency ratio of the task processing model in the task processing process.

[0032] Specifically, the formula for calculating the task processing efficiency ratio can be expressed as:

[0033] ;

[0034] Among them, is the task processing efficiency ratio, FS is the number of task processing steps required when the first task processing result first appears, and TS is the total number of task processing steps of the task processing information.

[0035] With the task processing model as an example of the LCoT large language model, the task processing efficiency ratio can be understood as the ratio between the number of reasoning steps required to arrive at the first correct answer within a thinking segment and the total number of steps in the entire thinking segment. The quantification of reasoning steps is diverse, which can be paragraphs, sentences, or character lengths. This ratio directly measures the efficiency of reasoning. The closer the ratio is to 1, the more the model spends most of the reasoning steps before arriving at the correct answer, reflecting a focused and efficient reasoning process. Conversely, the closer the ratio is to 0, the more the model continues to reason after arriving at the correct answer, which may indicate overthinking or redundant calculations. This step-based calculation enables the present application to evaluate the structural efficiency of the reasoning process independently of the specific implementation of token (token) quantity.

[0036] Through analysis of the trajectory of the long thought chain large language model on complex logical reasoning problems, the overthinking efficiency ratio can be expressed as:

[0037] ;

[0038] ;

[0039] wherein, is the overthinking efficiency ratio, is the pre-arranged overthinking token term dictionary, TT is the total number of tokens in the thinking segment, II() is the indicator function, is the ith vocabulary in the thinking segment.

[0040] When belongs to , it takes the value 1, otherwise 0. This ratio quantifies the language features of overthinking in the reasoning trajectory. The higher the value of , the more there is rethinking and verification language, which is usually related to redundant reasoning patterns. In the Chinese language environment, the vocabulary includes: but, etc., but, or, maybe, perhaps, re, let me again, let us again; in the English language environment, including: Wait, Hmm, but, perhaps, alternatively, again, Or.

[0041] wherein the overthinking score is obtained by weighted sum of the task processing efficiency ratio and the overthinking efficiency ratio, comprising: determining a first weight parameter corresponding to the overthinking efficiency ratio according to the business requirement; performing direction consistency conversion on the task processing efficiency ratio to obtain a converted task processing efficiency ratio; calculating a second weight parameter corresponding to the converted task processing efficiency ratio based on the first weight parameter; calculating a first product between the first weight parameter and the overthinking efficiency ratio; calculating a second product between the second weight parameter and the converted task processing efficiency ratio; and taking a sum between the first product and the second product as the overthinking score.

[0042] In this embodiment, in order to comprehensively evaluate the overthinking, an overthinking score is introduced, which can be expressed as:

[0043]

[0044] wherein score is the overthinking score, is a first weight parameter corresponding to the overthinking efficiency ratio, is a converted task processing efficiency ratio, is a second weight parameter corresponding to the converted task processing efficiency ratio.

[0045] In this application, is converted to to ensure direction consistency, because the higher the value, the higher the efficiency (the less overthinking), while the higher the value, the more obvious the overthinking mode. is used to balance the contribution of each component to the final score.

[0046] For example, the value of is set to 0.1, which can significantly reduce the calculation resources and delay if the correct answer can be obtained earlier, therefore, is the main component, with a weight of 90%. This priority is consistent with the goal of this application to minimize unnecessary calculations while maintaining the quality of reasoning.

[0047] Although provides valuable signals about overthinking, it is more sensitive to changes in prompt expressions, corpus characteristics, and specific model output patterns. In order to assign a lower weight (10%) to, which can reduce the amplification of noise that may be caused by these style fluctuations.

[0048] In addition, when evaluating the long chain reasoning ability of large language models, it is not enough to simply evaluate based on the above indicators. It is necessary to combine the accuracy on a specific evaluation set for a comprehensive evaluation. The accuracy measures the likelihood of the model producing a correct result.

[0049] Step S13: If the overthinking score is greater than the preset threshold, the position information at which the first task processing result first appears is determined from the second task processing result, the second task processing result is divided based on the position information, and the basic processing result and the evolution processing result of the historical task are obtained.

[0050] For the task processing result determined by the measurement index of overthinking through a long thinking chain, the first step is to determine the position information at which the correct answer is first obtained in the task processing result, and the starting range of the verification part of the correct answer after the correct answer is first obtained. By confirming the position information at which the correct answer is first obtained in the task processing result, the basic processing result and the evolution processing result are divided.

[0051] There are two ways to divide the basic processing result and the evolution processing result: the first is to use a third-party pre-training model to complete the division of the basic processing result and the evolution processing result through a prompt word engineering. The second is to use a light model to complete the division.

[0052] The third-party pre-training model includes but is not limited to qwen3-32B, gpt-oss-120b; the light model includes but is not limited to gpt-oss-20b, qwen3-4B.

[0053] In this embodiment, if the overthinking score is greater than the preset threshold, the third-party pre-training model is used to identify the second task processing result, so as to determine the position information at which the first task processing result first appears from the second task processing result, and add a mark at the position corresponding to the position information; the second task processing result is divided based on the mark at the position corresponding to the position information, and the basic processing result and the evolution processing result of the historical task are obtained.

[0054] The second task processing result is divided based on the mark at the position corresponding to the position information, and the basic processing result and the evolution processing result of the historical task are obtained, including: recording the number of marks, and determining whether the number of marks is greater than a mark number threshold; if the number of marks is greater than the mark number threshold, the position information with the most votes is used as target position information by using a voting method; the target mark corresponding to the target position information is determined, the second task processing result is divided based on the target mark, and the basic processing result and the evolution processing result of the historical task are obtained; if the number of marks is not greater than the mark number threshold, the process of identifying the second task processing result by using the third-party pre-training model is repeated until the number of marks is greater than the mark number threshold.

[0055] In this embodiment, the first division process of the basic processing result and the evolution processing result by means of the third-party pre-training model is as follows: the second task processing result output by the task processing model is cut according to a fixed character length, for example, 2000, and by means of the third-party pre-training model, on the basis of historical tasks and corresponding first task processing results and fixed lengths, the third-party pre-training model is used to complete the judgment of whether the first task processing result appears in the second task processing result in the first fixed length, and the position information of the first appearance of the first task processing result, and a <1st_answer> label is added at the end of the first task processing result, and if not, the next fixed length task processing result is judged until the first appearance of the first task processing result.

[0056] To ensure correctness, the judgment result of the third-party pre-training model can be sampled multiple times, and the position information with the most votes is taken as the target position information. The reason for dividing according to the fixed length is that long thinking chain data is generally longer due to the existence of multi-round evolution scheme, and the semantic analysis and instruction following ability of the pre-training model will decrease when processing long text, so it is necessary to cut according to the fixed length to ensure the stability of the ability of the third-party pre-training model. The part of using the third-party pre-training model for division requires an example as follows:

[0057] As an AI assistant, your task is to accurately identify the first occurrence of the answer in the given [reasoning analysis] and mark it as required.

[0058] Processing rules:

[0059] 1. Strict first occurrence principle:

[0060] - Scan from the beginning of the text sentence by sentence to find the first answer that directly answers the [original question]

[0061] - Must be the first explicitly given answer in context, not a setup or reasoning process

[0062] - Ignore all subsequent repeated answers or explanations

[0063] - Mathematical equivalence, i.e. same: different formats (such as ` ` and `\\sqrt{6}` ) or equivalent expressions (such as `3 / 2` and `1.5` ) are considered the same answer

[0064] - Substance over form: ignore formatting (such as boxed, color, etc.), only focus on the mathematical content itself

[0065] - Appear earlier than any subsequent equivalent expression

[0066] - The first occurrence of the answer must be preceded by the string: Final Answer

[0067] - The first occurrence of the answer must be in plain text or code style, not LaTeX math formula

[0068] 2. Tagging requirements:

[0069] - Insert the <1st_answer> tag immediately after the end of the first answer sentence in the confirmation

[0070] - Only mark the truly first occurrence of the answer, even if there are more detailed explanations later

[0071] - Do not mark restatements of the question, reasoning processes, or intermediate conclusions

[0072] 3. Output requirements:

[0073] - Only output the complete sentence with the <1st_answer> tag

[0074] - Do not output any other content, including explanations or subsequent answers

[0075] 4. Key points to note:

[0076] - Must start checking from the beginning of the text to ensure it is the truly first occurrence

[0077] - Do not be distracted by subsequent more detailed explanations

[0078] - Must ignore format differences, judge based on mathematical content consistency

[0079] - Look for answers in plain text or code style, the first occurrence of the answer must not be a LaTeX math formula

[0080] The <1st_answer> is the basis for the processing result, and the <1st_answer> is the evolution of the processing result.

[0081] In this application, the basic processing result refers to the first solution at the beginning of the reasoning process. After understanding the problem, it will be solved step by step. This constitutes the basis of the reasoning process and provides guidance for subsequent evolving solutions. The evolution of the processing result is to reflect on, improve, supplement or summarize the basic processing result, and may also propose new solutions.

[0082] In the embodiment, if the overthinking score is greater than the preset threshold, the marked data is obtained from the third-party pre-training model; the marked data is data obtained by adding a mark to the third-party pre-training model at a position corresponding to the position information; the lightweight model is trained by using the marked data to obtain a trained lightweight model; and the trained lightweight model is used to divide the second task processing result to obtain the basic processing result and the evolution processing result of the historical task.

[0083] In the embodiment, the second division process of the basic processing result and the evolution processing result by using the lightweight model is as follows:

[0084] Considering that the data volume of the sample data set constructed by using the target sample data is large, and in order to ensure the correctness of data labeling, the parameters of the third-party pre-training model are large, the time cost and algorithm cost of labeling are too large. The lightweight model is trained by using the data obtained by adding a mark to the third-party pre-training model at the position information. On the premise of ensuring the accuracy of the test set, the labeling of the large-volume supervised fine-tuning data set is completed by using the lightweight model.

[0085] In addition to the division of the basic processing result and the evolution processing result, that is, the recognition of the position information when the first task processing result appears for the first time, the verification process in the evolution processing result also needs to be divided and labeled. That is, each time the verification process of the first task processing result in the evolution processing result is recorded as a verify process, and one of the manifestations of the process thinking is that the verify process is too much. The labeling process can still be completed by using the third-party pre-training model. That is, the lightweight model can also be trained by using the marked data in the third-party pre-training model, and the trained lightweight model is used to divide the basic processing result and the evolution processing result.

[0086] Step S14: pre-processing the evolution processing result, and fine-tuning training the task processing model based on the pre-processed evolution processing result, the basic processing result, and the sample data set constructed by using the target sample data to obtain a target task processing model.

[0087] In the embodiment, the evolution processing result is deleted except the content of the first appearance of the first task processing result to obtain a deleted evolution processing result; the deleted evolution processing result is adaptively deleted according to a fixed length to obtain a pre-processed evolution processing result; the task processing model is first fine-tuned by using the sample data set constructed by using the target sample data to obtain a trained task processing model; and the trained task processing model is second fine-tuned based on the pre-processed evolution processing result and the basic processing result to obtain a target task processing model.

[0088] To strengthen the ability of the model to stop reasoning autonomously, the repetition of the solution is a natural termination information, that is, when the model gets the same answer twice, it is a strong indication that reasoning should be ended. Therefore, the application proposes a content deletion operation, which retains the basic processing result and retains the content of the first task processing result first appearing in the evolution processing result. That is, delete the content of the evolution processing result except the content of the first task processing result first appearing, which not only retains the verified part in the original reasoning process to improve accuracy, but also reduces the over-reasoning phenomenon of the original reasoning process.

[0089] To make the deletion operation more natural, the patent proposes to add clear natural language prompts after the reasoning stopping point, that is, the first appearance of the first task processing result in the evolution processing result, to further strengthen the self-regulation ability, for example: "I have got the same correct answer many times, it is time to end thinking", similar prompt words are placed at the boundary of the retained content and the deleted content as the critical point of stopping thinking, which keeps the fluency of reasoning while clearly indicating when to stop.

[0090] The application also proposes an adaptive deletion operation. The disadvantage of the direct deletion scheme is that for less difficult questions, one evolution processing result is enough for the verification confirmation of the basic processing result, but for more difficult questions, longer reasoning and more verification confirmations are needed to ensure accuracy. If a uniform fixed length truncation is used for all texts, it may cause the trained model to have a problem of accuracy decline when processing difficult tasks. Therefore, a gradual adaptive strategy can be considered to tailor the reasoning length for each length. This scheme will perform incremental analysis on each reasoning step to determine a customized termination point instead of a fixed termination point. This scheme is completed in an iterative process. First, the scheme will completely retain the basic processing result, and then gradually increase the correct verification steps in the evolution processing result. Each time a correct verification step is added, the overthinking score of the existing reasoning process is calculated. If the overthinking threshold has been exceeded, reasoning stops. If necessary, clear natural language prompts are added to further strengthen the self-regulation ability. If the overthinking threshold has not been exceeded, the next correct verification step in the evolution processing result is added until the original reasoning process is exhausted. Through this scheme, complex problems can retain more steps, while simple problems can stop earlier.

[0091] Among them, the trained task processing model is subjected to second fine-tuning training, including: calculating the loss value of the basic processing result by using the loss function; and using the loss value and the preprocessed evolution processing result to perform second fine-tuning training on the trained task processing model.

[0092] To inhibit the overthinking problem of the task processing model, the patent starts from the fine-tuning training link, that is, to reform the structure of the sample data set. This is mainly because the long thought chain reasoning type large model usually obtains the long thought chain ability through fine-tuning training, and the data used in fine-tuning training generally comes from data distillation. Therefore, it is necessary to solve the overthinking problem from the source stage, and it is also necessary to start from the reconstruction of distilled data. The purpose of the present application is to reduce the overthinking problem of the long thought chain, while maintaining the ability of the long thought chain model to solve complex problems. Therefore, the fine-tuning training scheme is proposed: the first fine-tuning training, that is, using the sample data set constructed by the target sample data to fine-tune the training, that is, using the inference data with overthinking characteristics to fine-tune the model, to enable the model to learn the ability to solve complex problems. The training cost of this process is large and long. The second fine-tuning training is based on the preprocessed evolution processing result and the basic processing result on the basis of the first fine-tuning training, so as to enable the model to complete the problem solving ability through a more simplified thinking process, and the training cost is smaller. The full amount of data can be down-sampled and trained.

[0093] To learn the ability to complete problem solving through a more simplified thinking process on the basis of the existing model's ability to solve complex problems, because it is essentially a format problem, the training cost of this stage is smaller, and the full amount of data can be down-sampled and trained.

[0094] In addition, in addition to the content deletion and adaptive deletion proposed in the present application, an automatic selection module can also be set, and the automatic selection module can be provided to the user end. The user can automatically match the optimal preprocessing method according to the data set scale and task priority; the task priority includes efficiency priority and accuracy priority; for example, the large-scale data set automatically matches the adaptive deletion. An overthinking early warning visualization tool can also be provided, for example, the overthinking score and inference efficiency ratio can be visualized, and the overthinking early warning visualization tool can be used to display different visualization methods to the operation and maintenance personnel, such as table form, chart form, etc., to facilitate the operation and maintenance personnel to monitor the pre-training model state and quickly locate the high-redundancy inference task type, and provide direction for subsequent optimization.

[0095] In this embodiment, target sample data is obtained; the target sample data includes historical tasks and corresponding first task processing results; the historical tasks are input into a task processing model to output second task processing results, and an overthinking score of the task processing model in a task processing process is determined based on the second task processing results; the task processing model is a model constructed based on a pre-trained model; if the overthinking score is greater than a preset threshold, position information when the first task processing result first appears is determined from the second task processing results, the second task processing results are divided based on the position information to obtain a basic processing result and an evolved processing result of the historical tasks; the evolved processing result is preprocessed, and the task processing model is fine-tuned based on the preprocessed evolved processing result, the basic processing result, and a sample data set constructed using the target sample data to obtain a target task processing model. The application obtains target sample data, inputs historical tasks into a task processing model to output second task processing results, determines an overthinking score of the task processing model in a task processing process based on the second task processing results, improves the accuracy of the output of the task processing model, can comprehensively evaluate the processing efficiency of the model, if the overthinking score is greater than a preset threshold, determines position information when the first task processing result first appears from the second task processing results, divides the second task processing results to obtain a basic processing result and an evolved processing result of the historical tasks, ensures that the model can distinguish the clear boundary between necessary reasoning and excessive calculation, preprocesses the evolved processing result to strengthen the ability of the model to stop reasoning independently, reduces the phenomenon of excessive reasoning in the original task processing process, fine-tunes the task processing model based on the preprocessed evolved processing result, the basic processing result, and a sample data set constructed using the target sample data to obtain a target task processing model, so that the model can learn to distinguish between effective reasoning and redundant reasoning, avoid the problem of overthinking of the pre-trained model in complex task processing, and maintain the ability of the pre-trained model to solve complex problems.

[0096] Referring to Figure 2 The embodiment of the application discloses a task processing method, which can specifically include:

[0097] Step S21: obtaining a task to be processed.

[0098] Step S22: processing the task to be processed by using a target task processing model to obtain a corresponding task processing result; wherein the target task processing model is a model trained by using the training method of the task processing model as described above.

[0099] In this embodiment, different processing paths in the target task processing model are utilized, or multiple target task processing models are utilized to perform parallel processing on the to-be-processed task, to obtain initial task processing results and determine corresponding task processing amounts; when the task processing amount is greater than the target task processing amount determined based on the service demand in advance, the task processing results are filtered from the initial task processing results in a voting manner.

[0100] For the same question, the thought chain length of the correct answer is often shorter than that of the incorrect answer. Therefore, in the evaluation process, considering the answer with the shortest thought chain can improve the evaluation accuracy and efficiency. In other words, although more complex problems usually require more thinking tokens, for a single problem example, a shorter thinking track is more likely to be correct, which not only guarantees the correctness of the problem, but also improves the reasoning efficiency.

[0101] The majority voting method is based on the basic assumption that the errors of different models are independent and the answer with the majority is more likely to be correct. The method generates multiple candidate answers for the same input through multiple models (or different reasoning paths of the same model), and selects the answer with the highest frequency as the final output.

[0102] Therefore, based on the majority voting method, the present application proposes a short strategy, which is as follows: for a specific problem, batch reasoning is performed, that is, k times of parallel reasoning operations (using a random sampling strategy) are performed on the same problem to generate k results. Once m (m≤k) shortest thinking tracks are completed, the calculation of all generation processes is terminated. Then, majority voting is performed on the m shortest answers. If the number of votes is the same, the answer with the shortest thinking chain is selected as the final result. Since the calculation amount of the thinking track is large, after the m shortest tracks are completed, all generation processes are terminated, which not only saves calculation resources, but also significantly shortens the actual time consumption by virtue of parallel decoding. After the m decoding thinking processes are completed, all other generation processes are terminated. Therefore, the total reasoning time is the time spent by the mth shortest generation process. Therefore, when m=1, it is the most efficient choice. When m=3, a good balance between efficiency and performance can be achieved.

[0103] In addition, the model training method and the task processing method of the present application can also be applied to the code generation technology scene to realize the specific process of model training and code generation as shown in Figure 3 The specific steps are as follows:

[0104] (1) selecting target sample data from the constructed sample data set; the target sample data includes historical tasks and corresponding first task processing results;

[0105] (2) input the historical task into the code generation model to output a second task processing result, and determine an overthinking score of the code generation model in the task processing process based on the second task processing result; the code generation model is a model constructed based on a pre-trained model;

[0106] (3) if the overthinking score is greater than a preset threshold, determining position information at which the first task processing result first appears from the second task processing result, dividing the second task processing result based on the position information to obtain a basic processing result and an evolution processing result of the historical task;

[0107] (4) preprocessing the evolution processing result, and obtaining a target code generation model based on the preprocessed evolution processing result, the basic processing result, and fine-tuning training of the code generation model using a sample data set;

[0108] (5) obtaining a to-be-processed task;

[0109] (6) processing the to-be-processed task using the target code generation model to generate a corresponding target code.

[0110] The application proposes an overthinking score for measuring the severity of overthinking of a model, which can comprehensively evaluate the task processing efficiency of the model, and can assist in identifying overthinking data in the SFT fine-tuning data, more accurately understand the reasoning effectiveness of the model, provide a more targeted direction for further optimizing the model, and evaluate more comprehensively and meticulously. In terms of data, a truncation strategy is used to add a mark at the position at which the first task processing result first appears, suppress the appearance frequency of the expansion solution after the basic processing result, thereby alleviating the overthinking characteristics presented in the data, and ensuring that the model can learn to distinguish the clear boundary between necessary reasoning and excessive calculation; through evolution processing result preprocessing and fine-tuning training, the model is helped to learn appropriate stopping reasoning and how to simplify the reasoning process, laying a foundation for timely stopping redundant thinking and shortening the reasoning path; using the short strategy based on the majority voting strategy, a better balance between efficiency and performance can be achieved.

[0111] Referring to Figure 4 The embodiment of the application discloses a task processing model training device, which can specifically include:

[0112] The data acquisition module 11 is configured to acquire target sample data; the target sample data includes historical tasks and corresponding first task processing results;

[0113] The overthinking score determination module 12 is configured to input the historical task into the task processing model to output a second task processing result, and determine an overthinking score of the task processing model in the task processing process based on the second task processing result; the task processing model is a model constructed based on a pre-trained model;

[0114] The division module 13 is configured to, if the overthinking score is greater than a preset threshold, determine position information at which the first task processing result first appears from the second task processing result, divide the second task processing result based on the position information, and obtain a basic processing result and an evolved processing result of the historical task.

[0115] The fine-tuning training module 14 is configured to pre-process the evolved processing result, and fine-tune the task processing model based on the pre-processed evolved processing result, the basic processing result, and a sample data set constructed by using target sample data, to obtain a target task processing model.

[0116] In some embodiments, the overthinking score determination module 12 can specifically include:

[0117] The task processing efficiency ratio calculation module is configured to calculate a task processing efficiency ratio of the task processing model in the task processing process based on the second task processing result.

[0118] The overthinking efficiency ratio calculation module is configured to calculate an overthinking efficiency ratio of the task processing model in the task processing process by using the second task processing result.

[0119] The weighted summation module is configured to perform weighted summation on the task processing efficiency ratio and the overthinking efficiency ratio to obtain the overthinking score.

[0120] In some embodiments, the task processing efficiency ratio calculation module can specifically include:

[0121] The task processing information acquisition module is configured to acquire task processing information of the task processing model in the task processing process.

[0122] The task processing step number determination module is configured to determine, from the task processing information, a number of task processing steps required when the first task processing result first appears.

[0123] The task processing total step determination module is configured to determine a total number of task processing steps of the task processing information.

[0124] The ratio calculation module is configured to take a ratio between the number of task processing steps and the total number of task processing steps as the task processing efficiency ratio of the task processing model in the task processing process.

[0125] In some embodiments, the weighted summation module can specifically include:

[0126] The first weight parameter determination module is used to determine the first weight parameter corresponding to the overthinking efficiency ratio based on business needs.

[0127] The direction consistency conversion module is used to perform direction consistency conversion on the task processing efficiency ratio to obtain the converted task processing efficiency ratio.

[0128] The second weight parameter calculation module is used to calculate the second weight parameter corresponding to the task processing efficiency ratio after conversion based on the first weight parameter.

[0129] The first product calculation module is used to calculate the first product between the first weight parameter and the overthinking efficiency ratio.

[0130] The second product calculation module is used to calculate the second product between the second weight parameter and the transformed task processing efficiency ratio;

[0131] The summation module is used to sum the first and second products as the overthinking score.

[0132] In some specific embodiments, the partitioning module 13 may specifically include:

[0133] A tagging module is added to identify the processing results of the second task using a third-party pre-trained model, so as to determine the location information when the processing result of the first task first appears in the processing results of the second task, and add a tag at the location corresponding to the location information.

[0134] The second task processing result division module is used to divide the second task processing results based on the markers at the corresponding locations of the location information, so as to obtain the basic processing results and evolution processing results of the historical tasks.

[0135] In some specific embodiments, the second task processing result partitioning module may specifically include:

[0136] The marking count determination module is used to record the marking count and determine whether the marking count is greater than the marking count threshold.

[0137] The target location information determination module is used to select the location information with the most votes as the target location information if the number of markings exceeds the marking number threshold.

[0138] The first partitioning module is used to determine the target markers corresponding to the target location information, and to partition the processing results of the second task based on the target markers to obtain the basic processing results and evolution processing results of the historical tasks.

[0139] The repeating execution module is configured to repeatedly execute the process of identifying the second task processing result by using the third-party pre-trained model until the number of labels is greater than the label number threshold, if the number of labels is not greater than the label number threshold.

[0140] In some embodiments, the second task processing result division module can specifically include:

[0141] The labeled data acquisition module is configured to acquire labeled data from the third-party pre-trained model, wherein the labeled data is data obtained by adding labels to the third-party pre-trained model at positions corresponding to the position information.

[0142] The lightweight model training module is configured to train the lightweight model by using the labeled data to obtain a trained lightweight model.

[0143] The second division module is configured to divide the second task processing result by using the trained lightweight model to obtain the basic processing result and the evolution processing result of the historical task.

[0144] In some embodiments, the fine-tuning training module 14 can specifically include:

[0145] The other content deletion module is configured to delete other content in the evolution processing result except the content of the first appearance of the first task processing result to obtain a deleted evolution processing result.

[0146] The adaptive deletion module is configured to adaptively delete the deleted evolution processing result according to a fixed length to obtain a preprocessed evolution processing result.

[0147] In some embodiments, the fine-tuning training module 14 can specifically include:

[0148] The first fine-tuning training module is configured to perform first fine-tuning training on the task processing model by using a sample data set constructed by using the target sample data to obtain a trained task processing model.

[0149] The second fine-tuning training module is configured to perform second fine-tuning training on the trained task processing model based on the preprocessed evolution processing result and the basic processing result to obtain a target task processing model.

[0150] In some embodiments, the second fine-tuning training module can specifically include:

[0151] The loss value calculation module is configured to calculate a loss value of the basic processing result by using a loss function.

[0152] The specific training module is configured to perform second fine-tuning training on the trained task processing model by using the loss value and the preprocessed evolution processing result.

[0153] The features of the embodiments of the training apparatus of the task processing model can be found in the related descriptions of the embodiments of the training method of the task processing model, which will not be repeated here.

[0154] Embodiments of the present application also provide an electronic device, comprising a memory and a processor, the memory storing a computer program, and the processor being configured to execute the computer program to perform the steps in any of the above method embodiments.

[0155] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed.

[0156] In an example embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0157] Embodiments of the present application also provide a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps in any of the above method embodiments.

[0158] Embodiments of the present application also provide another computer program product, which comprises a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in any of the above method embodiments.

[0159] The skilled person can further realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms in the above description. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0160] The training and task processing method and device of the task processing model provided in the present application are described in detail. The principles and implementation modes of the present application are described by applying specific examples, and the above description of the examples is only applicable to help understand the method of the present application and its core idea. It should be pointed out that, for ordinary skilled persons in the technical field, some improvements and modifications can be made to the present application without departing from the principles of the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

Claims

1. A training method for a task processing model, characterized in that, The method comprises: obtaining target sample data; the target sample data comprises historical tasks and corresponding first task processing results; the target sample data is text; inputting the historical tasks into a task processing model to output second task processing results, and determining an overthinking score of the task processing model in a task processing process based on the second task processing results; the task processing model is a model constructed based on a pre-trained model; if the overthinking score is greater than a preset threshold, determining position information at which the first task processing result first appears in the second task processing results, dividing the second task processing results based on the position information to obtain a basic processing result and an evolution processing result of the historical tasks; preprocessing the evolution processing result, and fine-tuning the task processing model based on the preprocessed evolution processing result, the basic processing result, and a sample data set constructed using the target sample data to obtain a target task processing model; determining position information at which the first task processing result first appears in the second task processing results based on a third-party pre-trained model, and adding a mark at the position corresponding to the position information; dividing the second task processing results based on the mark at the position corresponding to the position information to obtain a basic processing result and an evolution processing result of the historical tasks. 2.The method of Claim 1, wherein, The method further comprises: calculating a task processing efficiency ratio of the task processing model in the task processing process based on the second task processing results; calculating an overthinking efficiency ratio of the task processing model in the task processing process using the second task processing results; performing weighted summation on the task processing efficiency ratio and the overthinking efficiency ratio to obtain the overthinking score. 3.The method of Claim 2, wherein, The method further comprises: obtaining task processing information of the task processing model in the task processing process; determining the number of task processing steps required when the first task processing result first appears in the task processing information; determining the total number of task processing steps of the task processing information; taking the ratio between the number of task processing steps and the total number of task processing steps as the task processing efficiency ratio of the task processing model in the task processing process. 4.The method of Claim 2, wherein, The method further comprises: determining a first weight parameter corresponding to the overthinking efficiency ratio according to business requirements; performing direction consistency conversion on the task processing efficiency ratio to obtain a converted task processing efficiency ratio; calculating a second weight parameter corresponding to the converted task processing efficiency ratio based on the first weight parameter; and calculating a first product between the first weight parameter and the overthinking efficiency ratio; calculating a second product between the second weight parameter and the converted task processing efficiency ratio; taking a sum between the first product and the second product as an overthinking score. 5.The method of Claim 1, wherein, The dividing the second task processing result based on the mark at the position corresponding to the position information comprises: recording a mark number, and determining whether the mark number is greater than a mark number threshold; if the mark number is greater than the mark number threshold, using a voting method to take position information with the most votes as target position information; determining a target mark corresponding to the target position information, and dividing the second task processing result based on the target mark to obtain a basic processing result and an evolution processing result of a historical task; if the mark number is not greater than the mark number threshold, repeating the process of identifying the second task processing result using the third-party pre-trained model until the mark number is greater than the mark number threshold. 6.The method of Claim 1, wherein, The dividing the second task processing result based on the mark at the position corresponding to the position information comprises: obtaining mark data from the third-party pre-trained model; the mark data is data obtained after the third-party pre-trained model adds a mark at the position corresponding to the position information; training the lightweight model using the mark data to obtain a trained lightweight model; dividing the second task processing result using the trained lightweight model to obtain a basic processing result and an evolution processing result of a historical task. 7.The method of Claim 1, wherein, The preprocessing the evolution processing result comprises: deleting other content in the evolution processing result except content of the first occurrence of the first task processing result to obtain a deleted evolution processing result; performing adaptive deletion on the deleted evolution processing result according to a fixed length to obtain a preprocessed evolution processing result. 8.The method of Claim 1, wherein, The fine-tuning training the task processing model based on the preprocessed evolution processing result, the basic processing result, and a sample data set constructed using the target sample data to obtain a target task processing model comprises: performing first fine-tuning training on the task processing model using a sample data set constructed using the target sample data to obtain a trained task processing model; performing second fine-tuning training on the trained task processing model based on the preprocessed evolution processing result and the basic processing result to obtain a target task processing model. 9.The method of Claim 8, wherein, The second fine-tuning training the trained task processing model based on the preprocessed evolution processing result and the basic processing result comprises: calculating a loss value of the basic processing result using a loss function; performing second fine-tuning training on the trained task processing model using the loss value and the preprocessed evolution processing result.

10. A task processing method characterized by comprising: comprises: obtaining a task to be processed; The target task processing model is trained by using the training method of the task processing model according to any one of claims 1 to 9.

11. The task processing method according to claim 10, wherein The processing of the to-be-processed task by using the target task processing model to obtain a corresponding task processing result comprises: The to-be-processed task is processed by using different processing paths in the target task processing model, or by using multiple target task processing models to perform parallel processing on the to-be-processed task to obtain an initial task processing result, and a corresponding task processing amount is determined; When the task processing amount is greater than a target task processing amount determined in advance based on a business requirement, a task processing result is selected from the initial task processing result in a voting manner.

12. An electronic device, comprising: The computer program product comprises: a memory for storing a computer program; a processor for executing the computer program to implement the training method of the task processing model according to any one of claims 1 to 9, or the task processing method according to any one of claims 10 to 11.

Images