Task processing model training and task processing method and device

Abstract

The invention discloses a task processing model training and task processing method and device, and relates to the technical field of artificial intelligence. Target sample data comprises a historical task and a first task processing result; inputting the historical task into the task processing model, outputting a second task processing result, and determining an excessive thinking score of the task processing model in the task processing process; if the excessive thinking score is greater than a preset threshold value, determining position information when the first task processing result appears for the first time from the second task processing results, and dividing the second task processing results to obtain a basic processing result and an evolution processing result of the historical task; fine tuning training is performed on the task processing model based on the preprocessed evolution processing result, the basic processing result and the sample data set constructed by using the target sample data, the problem of excessive thinking of the model in complex task processing is avoided, the task processing efficiency and accuracy are improved, and the task processing effectiveness and overall performance are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a task processing model training method and device and a task processing method. BACKGROUND

[0002] Current solutions to overthinking include: optimizing models and data, mainly RL (Reinforcement learning) design of length-sensitive rewards to limit output, and using variable-length CoT (Chain of Thought) data and auxiliary constraints through SFT (Supervised Fine-Tuning) to shorten the reasoning path; optimizing reasoning methods by changing the generation strategy to reduce the length of reasoning. The disadvantages are as follows: limiting the ability of the pre-training model to constantly explore multiple strategies when processing complex tasks, resulting in reduced accuracy, and often failing, resulting in low efficiency and low accuracy of the pre-training model when processing tasks.

[0003] Therefore, how to avoid the problem of overthinking in complex task processing of the model, improve the task processing efficiency and accuracy of the model, and improve the task processing effectiveness and overall performance of the model is a problem to be solved by those skilled in the art. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a task processing model training method and device and a task processing method, which can avoid the problem of overthinking in complex task processing of the model, improve the task processing efficiency and accuracy of the model, and improve the task processing effectiveness and overall performance of the model. The specific scheme is as follows: In a first aspect, the present application discloses a task processing model training method, comprising: obtaining target sample data; the target sample data includes historical tasks and corresponding first task processing results; inputting the historical tasks into a task processing model to output second task processing results, and determining an overthinking score of the task processing model in the task processing process based on the second task processing results; the task processing model is a model constructed based on a pre-training model; if the overthinking score is greater than a preset threshold, determining position information of the first time when the first task processing result appears from the second task processing result, dividing the second task processing result based on the position information to obtain a basic processing result and an evolution processing result of the historical task; preprocessing the evolution processing result, and fine-tuning the task processing model based on the preprocessed evolution processing result, the basic processing result, and a sample data set constructed using the target sample data to obtain a target task processing model.

[0005] In a second aspect, the present application discloses a task processing method, comprising: obtaining a to-be-processed task; processing the to-be-processed task by using a target task processing model to obtain a corresponding task processing result, wherein the target task processing model is a model trained by using the training method of the task processing model.

[0006] In a third aspect, the present application discloses an electronic device, comprising: a memory for storing a computer program; a processor for executing the computer program to implement the steps of the foregoing method.

[0007] It can be seen that the application provides a task processing model training method, including obtaining target sample data; the target sample data includes a historical task and a corresponding first task processing result; inputting the historical task into a task processing model to output a second task processing result, and determining an overthinking score of the task processing model in a task processing process based on the second task processing result; the task processing model is a model constructed based on a pre-trained model; if the overthinking score is greater than a preset threshold, determining position information when the first task processing result first appears from the second task processing result, dividing the second task processing result based on the position information to obtain a basic processing result and an evolution processing result of the historical task; preprocessing the evolution processing result, and fine-tuning the task processing model based on the preprocessed evolution processing result, the basic processing result, and a sample data set constructed by using the target sample data to obtain a target task processing model. The application obtains target sample data, inputs the historical task into a task processing model to output a second task processing result, determines an overthinking score of the task processing model in a task processing process based on the second task processing result, improves the accuracy of the output of the task processing model, can comprehensively evaluate the processing efficiency of the model, if the overthinking score is greater than a preset threshold, determines position information when the first task processing result first appears from the second task processing result, divides the second task processing result to obtain a basic processing result and an evolution processing result of the historical task, ensures that the model can distinguish the clear boundary between necessary reasoning and excessive calculation, pre-processes the evolution processing result, can strengthen the ability of the model to stop reasoning independently, reduces the phenomenon of excessive reasoning in the original task processing process, fine-tunes the task processing model based on the preprocessed evolution processing result, the basic processing result, and a sample data set constructed by using the target sample data to obtain a target task processing model, so that the model can learn to distinguish effective reasoning and redundant reasoning, avoids the problem of overthinking of the pre-trained model in complex task processing, and maintains the ability of the pre-trained model to solve complex problems. BRIEF DESCRIPTION OF DRAWINGS

[0008] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0009] Figure 1 A flowchart of a task processing model training method disclosed by the application; Figure 2 A flowchart of a task processing method disclosed by the application; Figure 3 A specific flowchart for implementing model training and code generation is provided for the present application. Figure 4 A training device structure diagram of a task processing model disclosed by the present application is shown. DETAILED DESCRIPTION

[0010] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0011] Current solutions to overthinking include: optimizing the model and data, mainly RL design of length-sensitive rewards to limit output, and using variable-length CoT data and auxiliary constraints with SFT to shorten the reasoning path; optimizing the reasoning method by changing the generation strategy to reduce the length of reasoning. The disadvantages are as follows: limiting the ability of the pre-training model to constantly explore multiple strategies when processing complex tasks, resulting in reduced accuracy, often with failure, resulting in low efficiency and low accuracy of the pre-training model when processing tasks. Therefore, how to avoid the problem of overthinking of the model in complex task processing, improve the efficiency and accuracy of the model in task processing, and improve the effectiveness and overall performance of the model in task processing are problems that need to be solved by those skilled in the art.

[0012] Referring to Figure 1 The embodiments of the present application disclose a task processing model training method, which can specifically include: Step S11: obtaining target sample data; the target sample data includes historical tasks and corresponding first task processing results.

[0013] Step S12: inputting the historical task into the task processing model to output a second task processing result, and determining an overthinking score of the task processing model in the task processing process based on the second task processing result; the task processing model is a model constructed based on a pre-training model.

[0014] In order to solve the problem of overthinking of the task processing model, the present application first identifies the model or data in the mode that has overthinking phenomenon, therefore, the overthinking score is introduced, which is used as an overthinking measurement index. The overthinking score is two complementary indexes integrated through a composite scoring mechanism, including a task processing efficiency ratio and an overthinking efficiency ratio.

[0015] Task processing models can arrive at the correct answer relatively early in the reasoning process, but they continue to generate additional solution attempts. To quantify this inefficiency, this application proposes a method for training task processing models.

[0016] Taking the task processing model as an example, the LCoT (Long Chain-of-Thought) large language model refers to a large language model that can generate a relatively long reasoning process, including the OpenAI-o3 (an upgraded version of the OpenAI reasoning model) and DeepSeek-R1 series of models. It solves complex problems by simulating the step-by-step thinking process of humans. Compared with the basic CoT, the long chain-of-thought model can handle more complex reasoning tasks with more steps and maintain longer logical coherence. Its typical feature is that it may use a multi-round self-verification mechanism.

[0017] In this embodiment, the task processing efficiency ratio of the task processing model in the task processing process is calculated based on the second task processing result; the overthinking efficiency ratio of the task processing model in the task processing process is calculated using the second task processing result; and the task processing efficiency ratio and the overthinking efficiency ratio are weighted and summed to obtain the overthinking score.

[0018] The calculation of the task processing efficiency ratio of the task processing model in the task processing process based on the second task processing result includes: obtaining the task processing information of the task processing model in the task processing process; determining the number of task processing steps required when the first task processing result first appears from the task processing information; determining the total number of task processing steps in the task processing information; and taking the ratio between the number of task processing steps and the total number of task processing steps as the task processing efficiency ratio of the task processing model in the task processing process.

[0019] Specifically, the formula for calculating the task processing efficiency ratio can be expressed as: ; in, The task processing efficiency ratio is defined as follows: FS is the number of task processing steps required when the first task processing result is obtained for the first time, and TS is the total number of task processing steps for the task processing information.

[0020] Taking the task processing model as the LCoT large language model as an example, the task processing efficiency ratio can be understood as the ratio between the number of reasoning steps required to obtain the first correct answer within a thinking segment and the total number of steps in the entire thinking segment. The quantification of reasoning steps is diverse and can be in terms of paragraphs, sentences, or character lengths. This ratio directly measures the reasoning efficiency. The closer the ratio is to 1, it indicates that the model spends most of the reasoning steps before obtaining the correct answer, reflecting a focused and efficient reasoning process. On the contrary, the closer the ratio is to 0, it indicates that the model still conducts a large amount of reasoning after obtaining the correct answer, which may mean there is overthinking or redundant calculation. This step-based calculation enables this application to evaluate the structural efficiency of the reasoning process independently of the specific number of tokens (marks) implemented.

[0021] By analyzing the trajectory of the long thinking chain large language model on complex logical reasoning problems, the overthinking efficiency ratio can be expressed as: ; ; where, is the overthinking efficiency ratio, is the pre-organized dictionary of overthinking marked terms, TT is the total number of tokens in the thinking segment, II() is the indicator function, is the i-th vocabulary in the thinking segment.

[0022] When belongs to it takes the value of 1, otherwise 0. This ratio quantifies the overthinking language features in the reasoning trajectory. The higher the value of , it indicates that there is more reconsideration and verification language, which is usually associated with redundant reasoning patterns. In the Chinese language environment, the vocabulary includes: but, etc., however, or, may, perhaps, re, let me again, let us again; in the English language environment, it includes: Wait, Hmm, but, perhaps, alternatively, again, Or.

[0023] Among them, the overthinking score is obtained by weighted summation of the task processing efficiency ratio and the overthinking efficiency ratio, including: determining the first weight parameter corresponding to the overthinking efficiency ratio according to business requirements; performing direction consistency conversion on the task processing efficiency ratio to obtain the converted task processing efficiency ratio; calculating the second weight parameter corresponding to the converted task processing efficiency ratio based on the first weight parameter; calculating the first product between the first weight parameter and the overthinking efficiency ratio; calculating the second product between the second weight parameter and the converted task processing efficiency ratio; taking the sum between the first product and the second product as the overthinking score.

[0024] In this embodiment, in order to comprehensively evaluate overthinking, an overthinking score is introduced, which can be represented as: ; The score is the overthinking score. The first weighted parameter corresponding to the overthinking efficiency ratio. The ratio of the converted task processing efficiency. The second weight parameter corresponds to the ratio of the converted task processing efficiency.

[0025] This application will Convert to To ensure consistency of direction, because A higher value indicates higher efficiency (less overthinking), while The higher the value, the more pronounced the overthinking pattern. This is used to balance the contribution of each component to the final score.

[0026] For example, Setting the value to 0.1 allows for an earlier correct answer, significantly reducing computational resources and latency. It is a major component, with a weight of 90%. This priority is consistent with the objective of this application to minimize unnecessary computation while maintaining the quality of reasoning.

[0027] Although It provides valuable signals about overthinking, but it is more sensitive to changes in cueing, corpus features, and specific model output patterns. Assigning a lower weight (10%) can mitigate the noise amplification that these style fluctuations may cause.

[0028] Furthermore, when it is necessary to use volume to evaluate the long thought chain reasoning ability of a large language model, it is not possible to evaluate it solely based on the above indicators. It is necessary to combine it with the accuracy on a specific evaluation set for a comprehensive evaluation. Accuracy measures the likelihood that the model will produce correct results.

[0029] Step S13: If the overthinking score is greater than the preset threshold, determine the location information of the first occurrence of the first task processing result from the second task processing result, divide the second task processing result based on the location information, and obtain the basic processing result and evolution processing result of the historical task.

[0030] For task processing results that exhibit overthinking as determined by metrics such as long thought chains, the first step is to determine the location where the correct answer is first obtained in the task processing results, and the starting range of each subsequent verification of the correct answer. By confirming the location where the correct answer is first obtained in the task processing results, the basic processing results and the evolved processing results can be distinguished.

[0031] There are two ways to distinguish between basic processing results and evolved processing results: The first is to use a third-party pre-trained model to complete the distinction through prompt word engineering. The second is to use a lightweight model to complete the distinction.

[0032] Among them, third-party pre-trained models include, but are not limited to, qwen3-32B and gpt-oss-120b; lightweight models include, but are not limited to, gpt-oss-20b and qwen3-4B.

[0033] In this embodiment, if the overthinking score is greater than a preset threshold, a third-party pre-trained model is used to identify the processing result of the second task so as to determine the location information when the processing result of the first task first appears in the processing result of the second task and add a mark at the location corresponding to the location information; the processing result of the second task is divided based on the mark at the location corresponding to the location information to obtain the basic processing result and the evolution processing result of the historical task.

[0034] The process involves dividing the second task processing results based on the markers at the corresponding locations to obtain the basic and evolved processing results of the historical tasks. This includes: recording the number of markers and determining whether the number of markers exceeds a marker count threshold; if the number of markers exceeds the threshold, using a voting method to select the location information with the most votes as the target location information; determining the target marker corresponding to the target location information; dividing the second task processing results based on the target marker to obtain the basic and evolved processing results of the historical tasks; and if the number of markers does not exceed the marker count threshold, repeating the process of using a third-party pre-trained model to identify the second task processing results until the number of markers exceeds the marker count threshold.

[0035] In this embodiment, the first process of dividing the basic processing result and the evolved processing result using a third-party pre-trained model is as follows: The second task processing result output by the task processing model is segmented according to a fixed character length, such as 2000. Using the third-party pre-trained model, based on historical tasks and the corresponding first task processing result and the fixed length, starting from the first fixed-length task processing result, the third-party pre-trained model determines whether the first task processing result has appeared in the second task processing result within the fixed length, and the position information of the first occurrence of the first task processing result. A <1st_answer> marker is added to the end of the first task processing result. If not, the next fixed-length task processing result is determined until the first task processing result appears.

[0036] To ensure accuracy, the judgment results of the third-party pre-trained model can be sampled multiple times, and the position information with the most votes can be used as the target position information through a voting method. The reason for dividing the data into fixed-length segments is that long thought chain data is generally quite long due to the existence of multi-round evolution schemes, and the semantic analysis and instruction following capabilities of pre-trained models decrease when processing long texts. Therefore, it is necessary to perform fixed-length segmentation to ensure the stability of the capabilities of the third-party pre-trained model. Examples of the requirements for segmentation using the third-party pre-trained model are as follows: As an AI assistant, your task is to accurately identify the first occurrence of the answer in a given [reasoning analysis] and mark it as required.

[0037] Processing rules: 1. Strict first-appearance principle: - Scan the text sentence by sentence from the beginning to find the first answer that directly addresses the [original question]. - The answer must be explicitly given for the first time in the context, not as a preamble or reasoning process. - Ignore all subsequent repeated answers or explanations - Mathematical equivalence means the same: different formats (such as ` ` and `\\sqrt{6}`) or equivalent expressions (such as `3 / 2` and `1.5`) are considered the same answer. - Substance over form: Ignore formatting (such as boxed text, colors, etc.) and focus solely on the mathematical content itself. - Predates any subsequent equivalent expression - The first occurrence of the answer must be before the string: Final Answer. - The answer first appears in plain text or code style, not a LaTeX mathematical formula. 2. Marking requirements: - Insert the <1st_answer> tag immediately at the end of the first confirmed answer sentence. - Only mark the answer that appears for the first time, even if there is a more detailed explanation later. - Do not mark restates, reasoning processes, or intermediate conclusions. 3. Output requirements: - Only output the complete sentence with the <1st_answer> tag. - Do not output any other content, including explanations or follow-up answers. 4. Important Notes: - You must start checking from the beginning of the text to ensure it is a genuine first appearance. - Don't let more detailed explanations later mislead your judgment. - Formatting differences must be ignored; consistency of mathematical content should be the criterion for judgment. - Look for answers written in plain text or code style; the first answer that appears should definitely not be a LaTeX mathematical formula. The result before <1st_answer> is taken as the basic processing result, and the result after <1st_answer> is taken as the evolution processing result.

[0038] In this application, the basic processing result refers to the first solution at the start of the reasoning process. After understanding the problem, it is solved step by step. This forms the basis of the reasoning process and guides subsequent evolving solutions. The evolving processing result refers to reflections, improvements, supplements, or summaries of the basic processing result, and may also propose new solutions.

[0039] In this embodiment, if the overthinking score is greater than a preset threshold, labeled data is obtained from a third-party pre-trained model. The labeled data is the data obtained by adding labels to the positions corresponding to the location information in the third-party pre-trained model. The lightweight model is trained using the labeled data to obtain the trained lightweight model. The trained lightweight model is used to divide the processing results of the second task to obtain the basic processing results and evolution processing results of the historical task.

[0040] In this embodiment, the second process for dividing the basic processing results and the evolution processing results using a lightweight model is as follows: Considering the large size of the sample dataset constructed using the target sample data, and the large parameters of the third-party pre-trained model used to ensure the accuracy of data annotation, the time and algorithm costs of annotation are excessive. Therefore, a lightweight model is trained on the data obtained after adding labels to the location information of the third-party pre-trained model. While ensuring the accuracy of the test set, this lightweight model is used to complete the annotation of a large-scale supervised fine-tuning dataset.

[0041] In addition to distinguishing between basic and evolved processing results—that is, identifying the location information when the first task processing result first appears—it is also necessary to label and delineate the verification process within the evolved processing results. Specifically, each verification process of the first task processing result within the evolved processing results is recorded as a verify process. A characteristic of this process is the excessive number of verify processes. This labeling process can still be assisted by a third-party pre-trained model. In other words, the labeled data from a third-party pre-trained model can be used to train a lightweight model, and the trained lightweight model can then be used to distinguish between the basic and evolved processing results.

[0042] Step S14: Preprocess the evolution processing results, and fine-tune the task processing model based on the preprocessed evolution processing results, the basic processing results, and the sample dataset constructed using the target sample data to obtain the target task processing model.

[0043] In this embodiment, all content except for the content that appears for the first time in the evolution processing result is deleted to obtain the deleted evolution processing result; the deleted evolution processing result is adaptively deleted according to a fixed length to obtain the preprocessed evolution processing result; the task processing model is first fine-tuned and trained using the sample dataset constructed using the target sample data to obtain the trained task processing model; the trained task processing model is then fine-tuned and trained based on the preprocessed evolution processing result and the basic processing result to obtain the target task processing model.

[0044] To enhance the model's ability to autonomously stop inference, the repetition of solutions serves as a natural termination signal; that is, when the model obtains the same answer twice, it is a strong indication that inference should end. Therefore, this application proposes a content deletion operation that retains the basic processing results and the content in the evolved processing results that first appears as the first task processing result. In other words, it deletes all content in the evolved processing results except for the content that first appears as the first task processing result, thus preserving the verification portion of the original inference process to improve accuracy while reducing over-inference in the original inference process.

[0045] To make the deletion operation more natural, this patent proposes to add clear natural language prompts at the reasoning stopping point, that is, after the first occurrence of the first task processing result in the evolution processing results, to further enhance the self-regulation ability. For example, prompts such as "I have gotten the same correct answer many times, it is time to stop thinking" are placed at the boundary between the content to be retained and the content to be deleted, as a critical point for stopping thinking, which maintains the fluency of reasoning while clearly indicating when to stop.

[0046] This application also proposes an adaptive deletion operation. The disadvantage of direct deletion is that for easier problems, a single evolutionary processing result is sufficient to verify the basic processing result. However, for more difficult problems, longer inference and more verification are needed to ensure accuracy. If a uniform fixed-length truncation is applied to all texts, the trained model may experience a decrease in accuracy when handling more difficult tasks. Therefore, a stepwise adaptive strategy can be considered, tailoring an appropriate inference length for each length. This scheme performs incremental analysis on each inference step to determine a customized termination point, rather than a fixed one. This scheme is completed in an iterative process. First, the scheme retains the basic processing result completely. Then, it gradually adds correct verification steps from the evolutionary processing result. For each correct verification step added, the overthinking score of the existing inference process is calculated. If the overthinking threshold is exceeded, the inference stops, and if necessary, clear natural language prompts are added to further enhance the self-regulation ability. If the overthinking threshold is not exceeded, the correct verification steps from the next evolutionary processing result are added, until the original inference process is exhausted. This approach allows complex problems to retain more steps, while simple problems can be stopped earlier.

[0047] The second fine-tuning training of the trained task processing model includes: calculating the loss value of the basic processing result using the loss function; and using the loss value and the preprocessed evolution processing result to perform the second fine-tuning training of the trained task processing model.

[0048] To mitigate the overthinking problem in task processing models, this patent proposes to address it from the fine-tuning training stage, specifically by modifying the structure of the sample dataset. This is primarily because the long-thinking capabilities of large, long-chain reasoning models are typically acquired through fine-tuning training. The data used in fine-tuning training generally comes from data distillation; therefore, it is necessary to address the overthinking problem at its source and to modify the distilled data. The purpose of this application is to reduce the overthinking problem of long-chain reasoning models while maintaining their ability to solve complex problems. Therefore, a fine-tuning training scheme is proposed: First, fine-tuning training utilizes a sample dataset constructed from the target sample data. This involves using reasoning data exhibiting overthinking characteristics to fine-tune the model, enabling it to learn the ability to solve complex problems. This process is costly and time-consuming. Second, fine-tuning training, building upon the first fine-tuning training, further fine-tunes the model based on preprocessed evolutionary processing results and basic processing results. This allows the model to complete problem-solving through a more streamlined thinking process, reducing training costs. Training can be performed after downsampling the entire dataset.

[0049] To build upon the existing model's ability to solve complex problems, we aim to learn the ability to complete problem-solving through a more streamlined thought process. Since this is essentially a formatting issue, the training cost at this stage is relatively low, and training can be performed after downsampling the entire dataset.

[0050] In addition to the content deletion and adaptive deletion proposed in this application, an automatic selection module can be set up and provided to the user. Users can automatically match the optimal preprocessing method based on dataset size and task priority; task priority includes efficiency priority and accuracy priority; for example, automatic adaptive deletion can be matched for large-scale datasets. An overthinking warning visualization tool can also be provided. For example, overthinking scores and inference efficiency ratios can be visualized. This tool can present the data to operations personnel in different visualization formats, such as tables and charts, facilitating monitoring of the pre-trained model's status, quickly identifying the types of tasks with high redundancy inference, and providing direction for subsequent optimization.

[0051] In this embodiment, target sample data is acquired; the target sample data includes historical tasks and corresponding first task processing results; the historical tasks are input into the task processing model to output second task processing results, and the overthinking score of the task processing model in the task processing process is determined based on the second task processing results; the task processing model is a model built based on a pre-trained model; if the overthinking score is greater than a preset threshold, the position information of the first occurrence of the first task processing result is determined from the second task processing results, and the second task processing results are divided based on the position information to obtain the basic processing results and evolved processing results of the historical tasks; the evolved processing results are preprocessed, and the task processing model is fine-tuned and trained based on the preprocessed evolved processing results, the basic processing results, and the sample dataset built using the target sample data to obtain the target task processing model. This application acquires target sample data, inputs historical tasks into a task processing model to output a second task processing result, determines the overthinking score of the task processing model during task processing based on the second task processing result, improves the accuracy of the task processing model's output, and can comprehensively evaluate the model's processing efficiency. If the overthinking score is greater than a preset threshold, the position information of the first occurrence of the first task processing result is determined from the second task processing result, and the second task processing result is divided to obtain the basic processing result and the evolved processing result of the historical task. This ensures that the model can distinguish a clear boundary between necessary reasoning and overcomputation. Preprocessing the evolved processing result can enhance the model's ability to autonomously stop reasoning and reduce the phenomenon of overthinking in the original task processing process. Adaptive deletion can retain more steps for complex problems and stop simple problems earlier. Based on the preprocessed evolved processing result, the basic processing result, and the sample dataset constructed using the target sample data, the task processing model is fine-tuned and trained to obtain the target task processing model. This enables the model to learn to distinguish between effective reasoning and redundant reasoning, avoids the overthinking problem in the pre-trained model in complex task processing, and maintains the pre-trained model's ability to solve complex problems.

[0052] See Figure 2 As shown in the figure, an embodiment of the present invention discloses a task processing method, which may specifically include: Step S21: Obtain the tasks to be processed.

[0053] Step S22: Use the target task processing model to process the task to be processed and obtain the corresponding task processing result; wherein, the target task processing model is a model trained using the task processing model training method described above.

[0054] In this embodiment, different processing paths in the target task processing model, or multiple target task processing models, are used to process the task to be processed in parallel to obtain the initial task processing result and determine the corresponding task processing volume. When the task processing volume is greater than the target task processing volume determined in advance based on business needs, the task processing result is selected from the initial task processing result by voting.

[0055] For the same question, the correct answer often has a shorter thought process than the incorrect answer. Therefore, considering the answer with the shortest thought process during evaluation can improve both accuracy and efficiency. In other words, while more complex questions usually require more thought tokens, for a single question example, a shorter thought path is more likely to be correct, ensuring both the correctness of the question and improving reasoning efficiency.

[0056] The majority voting method is based on the fundamental assumption that "errors in different models are independent and answers with the most consistent majority are more likely to be correct". It generates multiple candidate answers for the same input through multiple models (or different inference paths of the same model) and selects the answer with the highest frequency as the final output.

[0057] Therefore, this application proposes a shortest-chain selection strategy based on the majority voting method, as follows: For a specific problem, batch reasoning is performed, that is, k parallel reasoning operations (using a random sampling strategy) are performed on the same problem to generate k results. Once m (m≤k) shortest thought trajectories are completed, the computation of all generation processes is terminated. Subsequently, a majority vote is conducted among these m shortest answers. If there is a tie, the answer with the shortest thought chain is selected as the final result. Since the computation of thought trajectories is relatively large, terminating all generation processes after the completion of m shortest trajectories not only saves computational resources, but also significantly shortens the actual time consumption due to the parallel decoding method. This method terminates all other generation processes after the first m decoding thought processes are completed, so the total reasoning time is the time spent on the mth shortest generation process. Therefore, when m=1, it is the most efficient choice. When m=3, a good balance between efficiency and performance can be achieved.

[0058] Furthermore, the model training method and task processing method of this application can also be applied to code generation technology scenarios to realize the specific process of model training and code generation, such as... Figure 3 As shown, the specific steps are as follows: (1) Filter target sample data from the constructed sample dataset; the target sample data includes historical tasks and the corresponding first task processing results; (2) Input the historical task into the code generation model to output the second task processing result, and determine the overthinking score of the code generation model in the task processing process based on the second task processing result; the code generation model is a model built based on the pre-trained model; (3) If the overthinking score is greater than the preset threshold, the location information of the first task processing result is determined from the second task processing result, and the second task processing result is divided based on the location information to obtain the basic processing result and evolution processing result of the historical task. (4) Preprocess the evolution results, and fine-tune the code generation model based on the preprocessed evolution results, the basic processing results, and the sample dataset to obtain the target code generation model; (5) Obtain tasks to be processed; (6) Use the target code generation model to process the task to be processed in order to generate the corresponding target code.

[0059] This application proposes an overthinking score to measure the severity of overthinking in a model. This score comprehensively evaluates the model's task processing efficiency and helps identify overthinking data in SFT fine-tuning data, providing a more accurate understanding of the model's inference effectiveness and offering more targeted directions for further model optimization. The evaluation is more comprehensive and detailed. Regarding the data, a truncation strategy is employed, marking the location where the first task processing result appears to suppress the frequency of extended solutions following the basic processing result. This mitigates the overthinking characteristics presented in the data, ensuring the model can learn to distinguish the clear boundary between necessary inference and overcomputation. Through evolutionary processing result preprocessing and fine-tuning training, the model learns how to appropriately stop inference and how to simplify the inference process, laying the foundation for timely cessation of redundant thinking and shortening inference paths. Utilizing a short-passing strategy based on a majority voting strategy achieves a good balance between efficiency and performance.

[0060] See Figure 4 As shown in the figure, an embodiment of the present invention discloses a training device for a task processing model, which may specifically include: Data acquisition module 11 is used to acquire target sample data; the target sample data includes historical tasks and the corresponding first task processing results; The overthinking score determination module 12 is used to input historical tasks into the task processing model to output the second task processing result, and to determine the overthinking score of the task processing model in the task processing process based on the second task processing result; the task processing model is a model built based on a pre-trained model. The segmentation module 13 is used to determine the location information of the first occurrence of the first task processing result from the second task processing result if the overthinking score is greater than a preset threshold, and to segment the second task processing result based on the location information to obtain the basic processing result and evolution processing result of the historical task. The fine-tuning training module 14 is used to preprocess the evolution processing results and fine-tune the task processing model based on the preprocessed evolution processing results, the basic processing results, and the sample dataset constructed using the target sample data to obtain the target task processing model.

[0061] In some specific embodiments, the overthinking score determination module 12 may specifically include: The task processing efficiency ratio calculation module is used to calculate the task processing efficiency ratio of the task processing model in the task processing process based on the second task processing result. The overthinking efficiency ratio calculation module is used to calculate the overthinking efficiency ratio of the task processing model during the task processing process using the results of the second task processing. The weighted summation module performs a weighted summation of the task processing efficiency ratio and the overthinking efficiency ratio to obtain the overthinking score.

[0062] In some specific embodiments, the task processing efficiency is higher than that of the calculation module, which may specifically include: The task processing information acquisition module is used to acquire task processing information of the task processing model during the task processing process; The task processing step number determination module is used to determine the number of task processing steps required when the first task processing result first appears from the task processing information; The task processing total steps determination module is used to determine the total number of task processing steps in the task processing information. The ratio calculation module is used to calculate the ratio between the number of task processing steps and the total number of task processing steps, which is used as the task processing efficiency ratio of the task processing model in the task processing process.

[0063] In some specific embodiments, the weighted summation module may specifically include: The first weight parameter determination module is used to determine the first weight parameter corresponding to the overthinking efficiency ratio based on business needs. The direction consistency conversion module is used to perform direction consistency conversion on the task processing efficiency ratio to obtain the converted task processing efficiency ratio. The second weight parameter calculation module is used to calculate the second weight parameter corresponding to the task processing efficiency ratio after conversion based on the first weight parameter. The first product calculation module is used to calculate the first product between the first weight parameter and the overthinking efficiency ratio. The second product calculation module is used to calculate the second product between the second weight parameter and the transformed task processing efficiency ratio; The summation module is used to sum the first and second products as the overthinking score.

[0064] In some specific embodiments, the partitioning module 13 may specifically include: A tagging module is added to identify the processing results of the second task using a third-party pre-trained model, so as to determine the location information when the processing result of the first task first appears in the processing results of the second task, and add a tag at the location corresponding to the location information. The second task processing result division module is used to divide the second task processing results based on the markers at the corresponding locations of the location information, so as to obtain the basic processing results and evolution processing results of the historical tasks.

[0065] In some specific embodiments, the second task processing result partitioning module may specifically include: The marking count determination module is used to record the marking count and determine whether the marking count is greater than the marking count threshold. The target location information determination module is used to select the location information with the most votes as the target location information if the number of markings exceeds the marking number threshold. The first partitioning module is used to determine the target markers corresponding to the target location information, and to partition the processing results of the second task based on the target markers to obtain the basic processing results and evolution processing results of the historical tasks. The repetitive execution module is used to repeatedly execute the process of recognizing the results of the second task using a third-party pre-trained model if the number of markings is not greater than the number of markings threshold, until the number of markings is greater than the number of markings threshold.

[0066] In some specific embodiments, the second task processing result partitioning module may specifically include: The labeled data acquisition module is used to obtain labeled data from third-party pre-trained models; the labeled data is the data obtained by the third-party pre-trained models after adding labels at the positions corresponding to the location information. The lightweight model training module is used to train a lightweight model using labeled data to obtain the trained lightweight model. The second partitioning module is used to partition the processing results of the second task using the trained lightweight model, so as to obtain the basic processing results and the evolution processing results of the historical task.

[0067] In some specific embodiments, the fine-tuning training module 14 may specifically include: The Other Content Deletion Module is used to delete all content in the evolution processing result except for the content that appears for the first time in the first task processing result, and obtain the evolution processing result after deletion; The adaptive deletion module is used to adaptively delete the evolution processing results after deletion according to a fixed length, so as to obtain the preprocessed evolution processing results.

[0068] In some specific embodiments, the fine-tuning training module 14 may specifically include: The first fine-tuning training module is used to perform the first fine-tuning training on the task processing model using the sample dataset constructed from the target sample data, so as to obtain the trained task processing model. The second fine-tuning training module is used to perform a second fine-tuning training on the trained task processing model based on the preprocessed evolution processing results and the basic processing results, so as to obtain the target task processing model.

[0069] In some specific embodiments, the second fine-tuning training module may specifically include: The loss value calculation module is used to calculate the loss value of the basic processing results using a loss function; The specific training module is used to perform a second fine-tuning of the trained task processing model using the loss value and the preprocessed evolution results.

[0070] The description of the features in the embodiment corresponding to the training device of the task processing model can be found in the relevant description of the embodiment corresponding to the training method of the task processing model, and will not be repeated here.

[0071] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0072] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when it is run.

[0073] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0074] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0075] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0076] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0077] The training method and apparatus for a task processing model provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A training method for a task processing model, characterized in that, include: Obtain target sample data; The target sample data includes historical tasks and the corresponding first task processing results; The historical task is input into the task processing model to output a second task processing result, and the overthinking score of the task processing model in the task processing process is determined based on the second task processing result. The task processing model is a model built based on a pre-trained model; If the overthinking score is greater than a preset threshold, the location information of the first occurrence of the first task processing result is determined from the second task processing result, and the second task processing result is divided based on the location information to obtain the basic processing result and the evolution processing result of the historical task. The evolution processing results are preprocessed, and the task processing model is fine-tuned and trained based on the preprocessed evolution processing results, the basic processing results, and the sample dataset constructed using the target sample data to obtain the target task processing model.

2. The training method for the task processing model according to claim 1, characterized in that, The determination of the overthinking score of the task processing model based on the second task processing result includes: Calculate the task processing efficiency ratio of the task processing model in the task processing process based on the results of the second task processing; The overthinking efficiency ratio of the task processing model during the task processing process is calculated using the results of the second task processing. The overthinking score is obtained by weighted summing of the task processing efficiency ratio and the overthinking efficiency ratio.

3. The training method for the task processing model according to claim 2, characterized in that, The calculation of the task processing efficiency ratio of the task processing model based on the second task processing result includes: Obtain task processing information from the task processing model during the task processing process; Determine the number of task processing steps required when the first task processing result first appears from the task processing information; Determine the total number of task processing steps for the task processing information; The ratio between the number of task processing steps and the total number of task processing steps is used as the task processing efficiency ratio of the task processing model in the task processing process.

4. The training method for the task processing model according to claim 2, characterized in that, The weighted summation of the task processing efficiency ratio and the overthinking efficiency ratio to obtain the overthinking score includes: Determine the first weighting parameter corresponding to the efficiency ratio of overthinking based on business needs; The task processing efficiency ratio is transformed to obtain the transformed task processing efficiency ratio by performing a directional consistency transformation. The second weight parameter corresponds to the task processing efficiency ratio calculated and converted based on the first weight parameter; Calculate the first product between the first weighting parameter and the overthinking efficiency ratio; Calculate the second product between the second weight parameter and the transformed task processing efficiency ratio; The sum of the first product and the second product is taken as the overthinking score.

5. The training method for the task processing model according to claim 1, characterized in that, The step of determining the location information of the first occurrence of the first task processing result from the second task processing result, and dividing the second task processing result based on the location information to obtain the basic processing result and the evolution processing result of the historical task includes: A third-party pre-trained model is used to identify the processing result of the second task, so as to determine the location information when the processing result of the first task first appears in the processing result of the second task, and a mark is added at the location corresponding to the location information; The processing results of the second task are divided based on the markers at the locations corresponding to the location information to obtain the basic processing results and evolution processing results of the historical tasks.

6. The training method for the task processing model according to claim 5, characterized in that, The second task processing result is divided based on the marker at the location corresponding to the location information to obtain the basic processing result and the evolved processing result of the historical task, including: Record the number of markings and determine whether the number of markings exceeds a marking count threshold. If the number of markings exceeds the marking threshold, the location information with the most votes will be used as the target location information using a voting method. Determine the target marker corresponding to the target location information, and divide the second task processing result based on the target marker to obtain the basic processing result and evolution processing result of the historical task; If the number of markings is not greater than the marking threshold, the process of using a third-party pre-trained model to identify the results of the second task is repeated until the number of markings is greater than the marking threshold.

7. The training method for the task processing model according to claim 5, characterized in that, The second task processing result is divided based on the marker at the location corresponding to the location information to obtain the basic processing result and the evolved processing result of the historical task, including: Labeled data is obtained from the third-party pre-trained model; the labeled data is the data obtained by the third-party pre-trained model after adding labels at the positions corresponding to the location information; The lightweight model is trained using the labeled data to obtain the trained lightweight model; The trained lightweight model is used to divide the processing results of the second task, resulting in the basic processing results and the evolution processing results of the historical task.

8. The training method for the task processing model according to claim 1, characterized in that, The preprocessing of the evolution results includes: Delete all content in the evolution processing result except for the content that appears for the first time in the first task processing result, to obtain the evolution processing result after deletion; Adaptive deletion is performed on the deleted evolution results according to a fixed length to obtain the preprocessed evolution results.

9. The training method for the task processing model according to claim 1, characterized in that, The process of fine-tuning and training the task processing model based on the preprocessed evolutionary processing results, the basic processing results, and the sample dataset constructed using the target sample data to obtain the target task processing model includes: The task processing model is first fine-tuned and trained using a sample dataset constructed from the target sample data to obtain the trained task processing model. Based on the preprocessed evolution processing results and the basic processing results, the trained task processing model is subjected to a second fine-tuning training to obtain the target task processing model.

10. The training method for the task processing model according to claim 9, characterized in that, The second fine-tuning training of the trained task processing model based on the preprocessed evolution processing results and the basic processing results includes: The loss value of the basic processing result is calculated using a loss function; The trained task processing model is then fine-tuned using the loss value and the preprocessed evolution results.

11. A task processing method, characterized in that, include: Get tasks to be processed; The task to be processed is processed using a target task processing model to obtain the corresponding task processing result; wherein, the target task processing model is a model trained using the training method of the task processing model as described in any one of claims 1 to 10.

12. The task processing method according to claim 11, characterized in that, The process of using the target task processing model to process the task to be processed and obtaining the corresponding task processing result includes: By utilizing different processing paths in the target task processing model, or by utilizing multiple target task processing models, the task to be processed is processed in parallel to obtain the initial task processing result and determine the corresponding task processing volume. When the task processing volume exceeds the target task processing volume determined in advance based on business needs, the task processing results are filtered from the initial task processing results by means of voting.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the method as described in any one of claims 1 to 12.

Images