Task processing model training method and device, computer device, and storage medium

Abstract

This application relates to the field of natural language processing, and provides a training method, apparatus, computer device, and storage medium for a task processing model. The method involves adjusting the difficulty of the previous task using a task generation model to determine the difficulty of the current task, and then adjusting the previous inference task to match the difficulty of the current task. The current inference task is sent to the task processing model for inference, resulting in multiple inference answers. The task generation model then evaluates the quality of these answers, obtaining the actual quality evaluation value and inference reward value for the current inference task. Based on the actual quality evaluation value and inference reward value, the model parameters are updated, and iterative training is performed using the current task difficulty as the new previous task difficulty and the current inference task as the new previous inference task. Employing self-generated tasks for model training reduces the model's reliance on manually labeled data and promotes the gradual improvement of the model's generalization ability.

Description

Technical Field

[0001] This application relates to the field of natural language processing, and more particularly to a training method, apparatus, computer device, and storage medium for a task processing model. Background Technology

[0002] With the widespread application of Large Language Models (LLMs) in Natural Language Processing (NLP), the reasoning and self-learning capabilities of language models have become key research directions. Traditional language model training methods rely on large amounts of manually labeled training data and task labels, and optimize model performance through supervised learning or reinforcement learning.

[0003] However, the biggest bottleneck of this approach lies in its high demand for a large amount of manually labeled data, especially in open-domain inference tasks where manually labeled data is difficult to obtain or has significant limitations. Therefore, in recent years, self-supervised learning has become an important new research direction.

[0004] Self-supervised learning methods are trained using labels generated by the model itself. While this reduces the reliance on manually labeled data, it still requires manual design of corresponding training tasks and optimization objectives. This still leads to the problem of dependence on human experience, resulting in limited model feature representation, insufficient generalization ability, and difficulty in adapting to diverse real-world application scenarios.

[0005] In view of this, in order to solve the above problems, a training method for task processing models is proposed. Summary of the Invention

[0006] This application provides a method, apparatus, computer device, and storage medium for training a task processing model, in order to solve the problem of dependence on manual tasks during the training of the task processing model.

[0007] In a first aspect, embodiments of this application provide a method for training a task processing model, comprising: The difficulty of the previous round of tasks is adjusted by the task generation model, the difficulty of the current round of tasks is determined, and the previous round of reasoning tasks is adjusted to the current round of reasoning tasks that match the difficulty of the current round of tasks. The current round of reasoning task is sent to the task processing model for reasoning to obtain multiple reasoning answers for the current round of reasoning task. The task generation model is then used to evaluate the quality of the multiple reasoning answers to obtain the actual quality evaluation value and reasoning reward value for the current round of reasoning task. The parameters of the task processing model and the task generation model are updated based on the actual quality assessment value and the inference reward value. The current round of task difficulty is used as the new previous round of task difficulty, and the current round of reasoning task is used as the new previous round of reasoning task, and iterative training is performed.

[0008] Secondly, embodiments of this application provide a training apparatus for a task processing model, comprising: The task generation module is used to adjust the difficulty of the previous round of tasks through the task generation model, determine the difficulty of the current round of tasks, and adjust the previous round of inference tasks to the current round of inference tasks that match the difficulty of the current round of tasks. The task evaluation module is used to send the current round of reasoning task to the task processing model for reasoning, obtain multiple reasoning answers for the current round of reasoning task, and evaluate the answer quality of the multiple reasoning answers through the task generation model to obtain the actual quality evaluation value and reasoning reward value of the current round of reasoning task. The model training module is used to update the parameters of the task processing model and the task generation model based on the actual quality assessment value and the inference reward value. The current round of task difficulty is used as the new previous round of task difficulty, and the current round of reasoning task is used as the new previous round of reasoning task, and iterative training is performed.

[0009] Optionally, the task generation module is used for: Using the task generation model, the difficulty of the previous round of tasks is adjusted based on the quality difference between the historical quality assessment value and the expected quality assessment value of the previous round of inference tasks, thereby determining the difficulty of the current round of tasks.

[0010] Optionally, the task generation module is used for: When the quality difference of the previous inference task indicates that the historical quality assessment value is higher than the expected quality assessment value, a task difficulty increase is generated based on the quality difference and a preset difficulty adjustment coefficient, and the difficulty of the previous task is adjusted upward based on the task difficulty increase to determine the difficulty of the current task. When the quality difference indicates that the historical quality assessment value is lower than the expected quality assessment value, a task difficulty reduction amount is generated based on the quality difference and the difficulty adjustment coefficient, and the difficulty of the previous round of tasks is adjusted down based on the task difficulty reduction amount to determine the difficulty of the current round of tasks. When the quality difference indicates that the historical quality assessment value is equal to the expected quality assessment value, the difficulty of the previous round of tasks is adjusted to zero, and the difficulty of the previous round of tasks is used as the difficulty of the current round of tasks.

[0011] Optionally, the task generation module is used for: Based on the reasoning ability parameters after the previous round of parameter tuning, the knowledge reasoning level, applicable task types, and task difficulty range of the task processing model are determined; wherein, the lower limit of the task difficulty range is higher than the expected task difficulty corresponding to the knowledge reasoning level. Retrieve at least one proposition-related knowledge point that matches the knowledge reasoning level, the applicable task type, and the task difficulty range, and adjust the previous round reasoning task to the current round reasoning task based on at least one proposition-related knowledge point and the current round task difficulty.

[0012] Optionally, the inference capability parameters after the previous round of parameter tuning include multiple inference parameter items, each of which describes a performance index of the task processing model when processing the previous round of inference task; the task generation module is used for: Each of the inference parameter terms is normalized to obtain a preprocessed parameter term for each inference parameter term; The reasoning ability is evaluated based on each of the preprocessing parameter items and their respective preset evaluation weights to obtain the reasoning ability evaluation value of the task processing model. Determine the reasoning ability assessment range in which the reasoning ability assessment value falls, and obtain the knowledge reasoning level, the task difficulty range, and the applicable task type associated with the reasoning ability assessment range.

[0013] Optionally, the task generation module is used for: Based on the difficulty of the current round of tasks, determine the content of the tasks to be modified in the previous round of reasoning tasks; Based on at least one of the proposition-related knowledge points and the set of logical reasoning relationships associated with each of the proposition-related knowledge points, the text of the task to be modified is modified to generate the current round of reasoning task.

[0014] Optionally, the task evaluation module is used for: The current round of reasoning task is sent to the task processing model, and at least one key problem-solving information in the current round of reasoning task is extracted; For each key problem-solving information, at least one historical knowledge point associated with the key problem-solving information in the knowledge point retrieval set is recorded as a reference problem-solving knowledge point; when no associated historical knowledge point is found in the knowledge point retrieval set, the key problem-solving information is converted into a newly added reference problem-solving knowledge point; the knowledge point retrieval set is composed of at least one of the historical knowledge points already learned by the task processing model and the historical knowledge points already included in the knowledge graph; Based on the recorded reference problem-solving knowledge points and the set of logical reasoning relationships associated with each reference problem-solving knowledge point, multiple problem-solving reasoning paths are obtained; For each of the problem-solving reasoning paths, problem-solving derivation steps are sequentially executed on at least one of the key problem-solving information according to the problem-solving reasoning path to obtain the reasoning answer.

[0015] Optionally, after obtaining multiple problem-solving reasoning paths, before sending the next round of reasoning tasks to the task processing model for reasoning, the device further includes a graph update module, which is used to: For each of the problem-solving reasoning paths, when the problem-solving reasoning path contains newly added reference problem-solving knowledge points, the knowledge graph is updated based on the newly added reference problem-solving knowledge points and the set of logical reasoning relationships associated with the newly added reference problem-solving knowledge points in the problem-solving reasoning path.

[0016] Optionally, after updating the knowledge graph, the graph updating module is further configured to: Starting from the newly added reference problem-solving knowledge points, the system traverses along the edges associated with the newly added reference problem-solving knowledge points in the knowledge graph to extract the set of logical reasoning relationships of the newly added reference problem-solving knowledge points. The set of logical reasoning relations is validated by logical rules. If at least one logical reasoning relation is found to conflict with a logical rule, the knowledge graph is then corrected a second time.

[0017] Optionally, after updating the knowledge graph, the graph update module is further used to: When the reasoning reward value is not lower than the preset reasoning reward value threshold, in the knowledge graph, multiple problem-solving reasoning paths are marked as important reasoning paths, and the reference problem-solving knowledge points involved in the multiple problem-solving reasoning paths and the edges associated with each reference problem-solving knowledge point are marked as important knowledge elements. When the reasoning reward value is lower than the reasoning reward value threshold, in the knowledge graph, multiple problem-solving reasoning paths are marked as reasoning paths to be verified, and the reference problem-solving knowledge points involved in the multiple problem-solving reasoning paths and the edges associated with each reference problem-solving knowledge point are marked as knowledge elements to be verified.

[0018] Optionally, when the knowledge point retrieval set contains all historical knowledge points already learned by the task processing model, the task evaluation module retrieves the key problem-solving information using the following method: Extract the semantic features of the question stem from the key problem-solving information; Semantic matching is performed between the semantic features of the question stem and the semantic features of each historical knowledge point already learned in the task processing model. Each historical knowledge point whose semantic feature similarity exceeds a preset feature similarity threshold is recorded as the reference problem-solving knowledge point.

[0019] Optionally, when the knowledge point retrieval set contains historical knowledge points already included in the knowledge graph, the task evaluation module retrieves the key problem-solving information using the following method: The key problem-solving information is matched with the knowledge graph to obtain at least one candidate knowledge point that is successfully matched. For each candidate knowledge point, starting from the candidate knowledge point, traverse along the edges associated with the candidate knowledge point in the knowledge graph, and record each historical knowledge point encountered as the reference solution knowledge point.

[0020] Optionally, the task evaluation module is used for: From the multiple reasoning answers, one reasoning answer is selected as a reference answer. Using the task generation model, the response quality of the remaining reasoning answers is evaluated based on the reference answer, and a response quality evaluation value is obtained for each of the other reasoning answers. The task generation model obtains the actual quality assessment value based on the quality assessment values ​​of each response and the total number of the multiple inference answers. The inference reward value is obtained by performing a profit and loss feedback based on the actual quality assessment value.

[0021] Thirdly, embodiments of this application also provide a computer device, including a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor performs the steps of any of the above-described task processing model training methods.

[0022] Fourthly, embodiments of this application also provide a computer-readable storage medium including program code, which, when the program product is run on a computer device, is used to cause the computer device to perform the steps of the training method of any of the above-described task processing models.

[0023] Fifthly, embodiments of this application also provide a computer program product, including computer instructions, which are executed by a processor to perform the steps of any of the above-described task processing model training methods.

[0024] In this embodiment, the difficulty of the previous task is adjusted by the task generation model to determine the difficulty of the current task, and the previous inference task is adjusted to match the difficulty of the current inference task. By adjusting the task difficulty, the newly generated inference task is made closer to the current level of the task processing model. This avoids training stagnation due to overly simple tasks and prevents a decline in the model's inference ability due to excessively difficult tasks. It helps the model gradually improve its inference ability, maintaining a balance between challenge and competence, thereby better mastering knowledge.

[0025] Next, the current round of reasoning task is sent to the task processing model for reasoning, resulting in multiple reasoning answers for the current round of reasoning task. The task generation model then evaluates the quality of these multiple reasoning answers to obtain the actual quality evaluation value and reasoning reward value for the current round of reasoning task.

[0026] Finally, based on the actual quality assessment value and the inference reward value, the parameters of the task processing model and the task generation model are updated, and the difficulty of the current round of tasks is used as the new difficulty of the previous round of tasks, and the inference task of the current round of tasks is used as the new inference task of the previous round of tasks, and iterative training is carried out.

[0027] By employing dynamically generated inference tasks for model training, the model's reliance on manually labeled data is reduced, mitigating the training costs and limitations associated with manual labeling and driving the model's generalization ability to improve gradually. The task processing model trained in this way can transfer its learned underlying processing logic and inference methods to new task scenarios, even those entirely new types of tasks never encountered during training. This achieves effective cross-task generalization, significantly reducing the operational costs associated with relabeling data and updating model parameters due to changes in task scenarios. Furthermore, it broadens the applicability of the model in actual deployments and enhances its long-term suitability in dynamic and open environments. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1A A logical diagram illustrating the training of a task processing model for educational tutoring scenarios; Figure 1B A logical diagram illustrating the training process for a task-oriented model in code generation and repair scenarios; Figure 1CA logical diagram illustrating the training of a task processing model for a financial risk assessment scenario; Figure 1D A logical diagram illustrating the training of a task processing model for intelligent question answering scenarios; Figure 2 This is an optional schematic diagram of an application scenario in the embodiments of this application; Figure 3A A flowchart illustrating the training task processing model provided in this application embodiment; Figure 3B A logical schematic diagram of the training task processing model provided in the embodiments of this application; Figure 3C A schematic diagram of the process for generating the current round of inference task provided in an embodiment of this application; Figure 3D A flowchart illustrating the reasoning and analysis process for the task processing model provided in the embodiments of this application; Figure 3E A logical diagram illustrating the reasoning analysis performed on the task processing model provided in the embodiments of this application; Figure 3F A logical diagram illustrating the retrieval of knowledge points related to key problem-solving information using a knowledge graph, as provided in an embodiment of this application. Figure 3G This is a schematic diagram of a knowledge graph provided for an embodiment of this application; Figure 4 This is a logical diagram illustrating the application of this application's embodiments in an intelligent question-answering scenario; Figure 5 A schematic diagram of the structure of a training device for a task processing model provided in an embodiment of this application; Figure 6 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application; Figure 7 This is a schematic diagram of the hardware structure of another computer device that applies an embodiment of this application. Detailed Implementation

[0030] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0031] For ease of understanding, the terms used in the embodiments of this invention are explained below.

[0032] 1. Self-supervised learning: This is an unsupervised machine learning approach that does not rely on manually labeled data. Instead, it automatically constructs supervisory signals directly from massive amounts of unlabeled data, utilizing the data's structure, context, or inherent properties. The model learns the general feature representations and potential patterns of the data by learning these signals, and trains and optimizes based on its own output feedback, continuously adjusting model parameters and gradually improving its feature extraction and inference capabilities.

[0033] 2. Reinforcement Learning: This is a machine learning method that optimizes decision-making strategies through interaction with the environment and based on reward and punishment feedback. As the model continuously attempts to perform actions, it adjusts its behavior based on positive or negative feedback from the environment, gradually learning the optimal strategy that maximizes long-term gains.

[0034] 3. Knowledge Graph: A knowledge network that describes knowledge points and their relationships in the form of a graph. It is usually composed of nodes and directed edges. Nodes represent knowledge points (such as concepts, facts, etc.), and edges represent logical reasoning relationships between knowledge points. By organizing scattered knowledge points into a structured and reasonable knowledge system, it provides traceable, scalable, and associative reasoning support for the model.

[0035] 4. Reasoning Path: In a reasoning task, the model uses extracted reference problem-solving knowledge points and corresponding logical reasoning relationships to form a complete reasoning chain by sequentially connecting knowledge points and logical rules in the order from premises to the solution objective. It clearly reflects the process of gradually deriving conclusions from known information.

[0036] 5. Historical reasoning path: This is the path through which the task processing model performs logical reasoning by considering the relationships between nodes and edges in the knowledge graph when answering a task. This helps the model to utilize existing knowledge more efficiently to answer the task.

[0037] The design concept of the embodiments of this application is briefly introduced below: With the widespread application of LLM in NLP, the reasoning and self-learning capabilities of language models have become key research directions. Traditional language model training methods rely on large amounts of manually labeled training data and task labels, and optimize model performance through supervised learning or reinforcement learning.

[0038] However, the biggest bottleneck of this method lies in its high demand for a large amount of manually labeled data, especially in open-domain reasoning tasks where manually labeled data is difficult to obtain or has significant limitations. Therefore, in recent years, self-supervised learning has become an important new research direction.

[0039] Self-supervised learning methods are trained using labels generated by the model itself. While this reduces the reliance on manually labeled data, it still requires manual design of corresponding training tasks and optimization objectives. This still leads to the problem of dependence on human experience, resulting in limited model feature representation, insufficient generalization ability, and difficulty in adapting to diverse real-world application scenarios.

[0040] In view of this, in order to solve the above problems, a training method for task processing models is proposed.

[0041] First, the difficulty of the previous round of tasks is adjusted by the task generation model to determine the difficulty of the current round of tasks, and the previous round of reasoning tasks is adjusted to match the difficulty of the current round of tasks.

[0042] Next, the current round of reasoning task is sent to the task processing model for reasoning, resulting in multiple reasoning answers for the current round of reasoning task. The task generation model then evaluates the quality of these multiple reasoning answers to obtain the actual quality evaluation value and reasoning reward value for the current round of reasoning task.

[0043] Finally, based on the actual quality assessment value and the inference reward value, the parameters of the task processing model and the task generation model are updated, and the difficulty of the current round of tasks is used as the new difficulty of the previous round of tasks, and the inference task of the current round of tasks is used as the new inference task of the previous round of tasks, and iterative training is carried out.

[0044] By adjusting the difficulty of the tasks, the newly generated reasoning tasks are made closer to the current level of the task processing model. This avoids stagnation in training due to overly simple tasks, and also prevents a decline in the model's reasoning ability due to overly difficult tasks. This helps the model to gradually improve its reasoning ability, maintain a balance between challenge and competence, and thus better master knowledge.

[0045] By employing dynamically generated inference tasks for model training, the model's reliance on manually labeled data is reduced, mitigating the training costs and limitations associated with manual labeling and driving the model's generalization ability to improve gradually. The task processing model trained in this way can transfer its learned underlying processing logic and inference methods to new task scenarios, even those entirely new types of tasks never encountered during training. This achieves effective cross-task generalization, significantly reducing the operational costs associated with relabeling data and updating model parameters due to changes in task scenarios. Furthermore, it broadens the applicability of the model in actual deployments and enhances its long-term suitability in dynamic and open environments.

[0046] The preferred embodiments of this application are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit this application. Furthermore, the embodiments and features in the embodiments of this application can be combined with each other without conflict.

[0047] The training method for task processing models provided in this application can be used to train models for scenarios such as educational tutoring, operation and maintenance monitoring, financial risk assessment, and copywriting generation. This method feeds back the quality differences in the historical performance of the task processing model into the generation process of new tasks. Based on the model's knowledge mastery and weaknesses, the task difficulty is adaptively adjusted, and then a dynamically generated model is used for training, reducing the model's reliance on manually labeled data.

[0048] Combination Figure 1A In an educational tutoring scenario, the task generation model generates an initial reasoning task: "Given that one angle of an isosceles triangle is 45°, calculate the angles of the other two angles." The task processing model, through reasoning analysis, obtains multiple possible inference answers (such as 67.5° and 67.5°, 45° and 90°, etc.). Next, the task generation model evaluates the quality of these inference answers, obtaining an actual quality assessment value (such as the correctness of the answer and the standardization of the solution steps) and a reasoning reward value (such as a reasoning reward of 0.7 for high answer accuracy). Based on these two values, the parameters of both models are updated.

[0049] Next, the task generation model adjusts the difficulty of the first-round task based on the quality difference between the actual and expected quality assessment values, determines the difficulty of the second-round task, and generates a second-round reasoning task matching the difficulty of the second-round task: "Given that the vertex angle of an isosceles triangle is 45°, what is the angle between the altitude on one of its legs and the base?". The above training steps are repeated for the second-round reasoning task until the model's overall score (such as accuracy and reasoning completeness) reaches a preset threshold when faced with a series of geometric variations. At this point, the trained task processing model is output.

[0050] During the model application phase, the user-input question, "In triangle ABC, AB=AC, ∠A=30°, what is the angle between the altitude CD on side AB and the base BC?" is sent to the task processing model. Based on its learned geometric properties, trigonometric functions, and graphical decomposition reasoning ability, the model outputs the answer, "The angle is 75°."

[0051] Combination Figure 1BIn the scenario of automated code generation and repair, the task generation model generates the first round of inference task: "Write a function to determine whether a positive integer is a prime number." The task processing model, through inference analysis, obtains multiple code files (such as a naive implementation based on trial division, and a implementation based on...). (Optimization of the regularity). Next, the task generation model evaluates the quality of the responses based on these code files, obtaining actual quality evaluation values ​​(such as code correctness and time complexity) and inference reward values ​​(such as the completeness of boundary condition handling), and updates the parameters of both models based on these two values.

[0052] Next, the task generation model adjusts the difficulty of the first-round task based on the quality difference between the actual and expected quality assessment values, determines the difficulty of the second-round task, and generates a second-round inference task that matches the difficulty of the second-round task: "Optimize the above function so that it can efficiently calculate all prime factors of a number." The above training steps are repeated for the second-round inference task until the code generated by the model passes automated test set validation in terms of functionality, robustness, and processing efficiency. At this point, the trained task processing model is output.

[0053] During the model application phase, the user-input question "Please write a prime factorizer that can handle large integer arrays" is sent to the task processing model. Based on a comprehensive reasoning of algorithm optimization, parallel computing, and system resource management, the model outputs the corresponding answer: "A prime factorizer combined with a multi-threaded pool can be used. The core code is as follows: [Generated Code]".

[0054] Combination Figure 1C In a financial risk assessment scenario, the task generation model generates the first round of reasoning tasks: "Based on a company's financial data from the past year (revenue growth rate of 10%, debt ratio of 60%, and price-to-earnings ratio of 15), please assess its short-term solvency risk level." The task processing model, through reasoning analysis, obtains multiple inference answers (such as "medium risk, mainly based on the current ratio being close to the warning line," and "low risk, due to ample cash flow from operating activities"). Next, the task generation model evaluates the quality of these inference answers, obtaining an actual quality assessment value (such as the accuracy of indicator citations and the rigor of risk logic) and a reasoning reward value (such as the degree of consideration for non-financial factors), and updates the parameters of both models based on these two values.

[0055] Next, the task generation model adjusts the difficulty of the first-round task based on the quality difference between the actual and expected quality assessment values, determines the difficulty of the second-round task, and generates a second-round inference task that matches the difficulty of the second-round task: "If the industry to which the aforementioned company belongs suddenly issues regulatory policies, and its largest customer accounts for 40% of its revenue, please reassess its three-year operating risk and provide hedging recommendations based on the aforementioned financial data." The above training steps are repeated for the second-round inference task until the model's output conclusions pass validation in terms of backtesting accuracy, comprehensive risk coverage, and feasibility of recommendations when comprehensively processing risk assessment cases containing multi-dimensional and complex information such as industry dynamics and company management. At this point, the trained task processing model is output.

[0056] During the model application phase, the user-input question, "Please analyze the investment risks and opportunities for new energy vehicle companies under multiple pressures such as soaring raw material prices, subsidy reductions, and competitors launching new solid-state battery technologies," is sent to the task processing model. Based on a comprehensive inference of industry trends, technological iterations, and market demand, the model outputs the corresponding answer: "In the short term, there is a risk of compressed profit margins; it is recommended to focus on companies with outstanding supply chain management capabilities. In the medium to long term, technology leaders will gain a differentiated advantage. Specifically, [an analysis report containing risk weights, opportunity points, and monitoring indicators will be generated]."

[0057] Combination Figure 1D In the intelligent question-answering scenario, the task generation model generates the first round of reasoning tasks: "The current live stream topic is 'Outdoor Camping Gear Selection,' and a viewer asked, 'With a budget under 500 yuan, please recommend a tent suitable for beginners.' Please generate recommendation scripts that match the host's style." The task processing model, through reasoning analysis, outputs multiple live stream scripts (such as "Recommend XX brand automatic quick-opening tent, waterproof rating 3000mm, weight 3.2kg, recent promotional price 459 yuan," and "You can consider YY double aluminum pole tent, weight 2.8kg, but slightly more complicated to set up, price 438 yuan"). The task generation model evaluates the quality of the responses based on these live stream scripts, calculating the actual quality evaluation value (such as product parameter accuracy, price consistency, and naturalness of expression) and the reasoning reward value.

[0058] After updating the parameters of the two models based on these two values, the system adjusts the difficulty of the first-round task according to the quality difference between the actual and expected quality assessment values, determines the difficulty of the second-round task, and generates a second-round inference task that matches the difficulty of the second-round task: "An audience member asked, 'How does this tent perform in heavy rain?' Please generate a supplementary answer that is both professional and easy to understand, combining the principles of tent waterproofing technology with real-world usage scenarios." This training process is repeated for the second-round inference task until the model can generate answers that conform to the style of live streaming, are accurate and comprehensive, and can proactively uncover the user's potential needs in real-time interaction, addressing various product inquiries, technology comparisons, and usage scenario-related questions. The trained task processing model is then output.

[0059] During the application phase, when a user asks, "I want to buy a pair of hiking shoes that can handle the rainy season in western Sichuan, with a budget of around 800 yuan, please recommend some," the model, based on a comprehensive inference from an outdoor equipment knowledge base, real-time weather data, and brand reputation information, outputs the answer: "I recommend the XX waterproof hiking shoes, which feature highly waterproof fabric and are currently discounted to 769 yuan. These are outdoor hiking shoes designed for rainy environments, with ABC anti-slip technology in the sole, suitable for the terrain of western Sichuan you mentioned. If you are looking for a lightweight experience, you can also consider..." The model also simultaneously generates a product link and a comparison chart of waterproof performance.

[0060] Figure 2 One application scenario is shown, which includes two terminal devices 210 and a server 230. The terminal devices 210 establish a communication connection with the server 230 through a wired network or a wireless network.

[0061] Among them, terminal devices 210 include, but are not limited to: mobile phones, computers (such as tablets, laptops, desktop computers, etc.), smart home appliances, smart voice interaction devices (such as smartwatches, smart speakers, etc.), vehicle terminals, aircraft, etc.

[0062] The server 230 in this application embodiment can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. This application does not impose any restrictions on these services.

[0063] Users access the intelligent question-and-answer service platform through terminal device 210, which displays an intelligent question-and-answer interaction page on screen 220. Users enter questions in the input boxes on the intelligent question-and-answer interaction page, and terminal device 210 sends the questions to server 230. Server 230 calls the task processing model to answer the question and then pushes the answer to the intelligent question-and-answer interaction page through a pre-established communication channel for the user to view.

[0064] The model used is trained using the method provided in this solution. This model can adaptively adjust the task difficulty and type according to its own mastery and weaknesses, reducing reliance on manually labeled data and achieving targeted reinforcement learning. In the process of actually handling various tasks, even when faced with completely new task types that were never encountered during the training phase, the model can maintain reliable inference performance and reasonable output results, significantly improving the generalization and robustness of task processing. It avoids problems such as excessive output deviation and unreasonable results due to unfamiliar task scenarios, thereby providing stable, efficient and accurate intelligent services for intelligent question answering services and continuously providing users with a stable and reliable interactive experience.

[0065] To adapt to the operational needs of different environments, the task processing model offers flexible deployment methods. It can be deployed locally on server 230 to make full use of existing hardware resources, or it can be deployed to other servers according to resource planning and business needs to achieve the optimal match between deployment location and hardware resources.

[0066] Next, combined Figures 3A-3B The schematic diagram shown illustrates the iterative training of the model using the task processing model training method provided in the embodiments of this application.

[0067] This method is applied to a model training system that includes a task processing model and a task generation model. The system adopts a self-evolution mechanism, using the task processing model to solve reasoning tasks and the task generation model to evaluate the performance of the task processing model. No human intervention is required. Based on self-feedback and reinforcement learning, the system continuously optimizes its task generation and reasoning capabilities, forming a virtuous cycle of continuous self-improvement. It has the ability to improve itself in the long term, enhancing the model's generalization and transfer capabilities in different task types, knowledge domains, and difficulty scenarios. It can flexibly transfer learned reasoning logic and knowledge to unknown reasoning scenarios, continuously improving the overall reasoning performance.

[0068] S301: Adjust the difficulty of the previous round of tasks through the task generation model, determine the difficulty of the current round of tasks, and adjust the previous round of inference tasks to the current round of inference tasks that match the difficulty of the current round of tasks.

[0069] Self-supervised learning methods in related technologies are trained using labels generated by the model itself. Although this reduces the reliance on manually labeled data, it still requires manual design of corresponding training tasks and optimization objectives. This still leads to the problem of relying on human experience, resulting in limited model feature expression, insufficient generalization ability, and difficulty in adapting to diverse real-world application scenarios.

[0070] To address this issue, this application employs a task self-generation mechanism. The task generation model automatically generates a current round of reasoning tasks that are suitable for the knowledge mastery level of the task processing model and have a certain level of challenge, based on the difficulty of the current round of tasks. The generated tasks are then used to train the model, further reducing the model's dependence on manually labeled data.

[0071] If the current round is the first round, then the reasoning task for the current round is generated in the following way: Reasoning ability parameters Used to describe the reasoning ability and learning status of a task processing model, including reasoning parameters such as the current answer accuracy, reasoning depth, stability, and historical completion status of the task processing model.

[0072] First, the task generation model initializes the task generator based on the reasoning ability parameters, and determines the knowledge reasoning level, applicable task types, and task difficulty range of the task processing model.

[0073] Applicable task types include, but are not limited to, common sense reasoning tasks, text understanding tasks, semantic parsing tasks, intent understanding tasks, mathematical reasoning tasks, logical reasoning tasks, causal inference tasks, conditional judgment tasks, rule matching tasks, multi-hop reasoning tasks, fact-checking tasks, relation extraction tasks, question-and-answer generation tasks, summary and induction tasks, classification and judgment tasks, and comparative analysis tasks.

[0074] The lower limit of the task difficulty range is higher than the expected task difficulty corresponding to the knowledge reasoning level. This allows for the selection of challenging knowledge points from the knowledge graph or initial reasoning task set that match the model's current capabilities as proposition materials for new tasks. The lower limit of the task difficulty range should be slightly higher than the edge of the model's capabilities to avoid generating tasks that are too simple or too complex. This encourages the model to adaptively increase difficulty during training, gradually improving its comprehensive ability in knowledge reasoning and task solving. As a result, the overall learning effect and reasoning level of the model are steadily optimized, enabling it to gradually cope with increasingly complex reasoning tasks.

[0075] Next, in the preset initial reasoning task set, an initial reasoning task that matches the knowledge reasoning level, applicable task type and task difficulty range of the task processing model is retrieved, and the initial reasoning task is used as the reasoning task for the current round.

[0076] Specifically, a multi-dimensional matching retrieval is performed in the initial inference task set. By comparing the current knowledge reasoning level, applicable task types, and task difficulty range of the task processing model with the corresponding attributes labeled for each task in the task set, matching initial inference tasks are finally selected and used as the current round of inference tasks. This process ensures that the current round of inference tasks not only matches the model's current capability stage but also adapts to the task types it excels at, while meeting the preset difficulty requirements, thus providing a scientific and effective starting point for subsequent iterative training.

[0077] If the current round is not the first round, then the reasoning task for the current round is generated in the following way: First, the difficulty of the previous task is adjusted based on the quality difference between the historical quality assessment value and the expected quality assessment value of the previous inference task, using the task generation model, to determine the difficulty of the current task.

[0078] Specifically, after the task processing model completes the previous round of reasoning, the task generation model evaluates the quality of the task processing model's responses, obtaining historical quality assessment values. Furthermore, the expected quality assessment value of the task processing model is also obtained. This assessment value describes the ideal quality level the task processing model should achieve when responding to the previous round of reasoning, reflecting the model's preset response standards and expected results when completing this type of reasoning task. It can serve as a benchmark for measuring the model's actual response performance and judging whether the model's capabilities meet the standards.

[0079] When the quality difference of the previous inference task indicates that the historical quality assessment value is higher than the expected quality assessment value, the task difficulty increase is generated based on the quality difference and the preset difficulty adjustment coefficient. Based on the task difficulty increase, the difficulty of the previous task is adjusted upward to determine the difficulty of the current task.

[0080] When the quality difference indicates that the historical quality assessment value is lower than the expected quality assessment value, the task difficulty reduction amount is generated based on the quality difference and the difficulty adjustment coefficient. Based on the task difficulty reduction amount, the difficulty of the previous round of tasks is adjusted down to determine the difficulty of the current round of tasks.

[0081] When the quality difference indicates that the historical quality assessment value equals the expected quality assessment value, the difficulty of the previous round of tasks is adjusted to zero, and the difficulty of the previous round of tasks is used as the difficulty of the current round of tasks.

[0082] The following formula can be used to calculate the task difficulty adjustment: Formula 1; In Formula 1, S is the historical quality assessment value of the previous round of reasoning task. It is the expected quality assessment value of the previous round of reasoning task. The difference lies in the quality of the previous reasoning task. It is a difficulty adjustment factor. This is the amount of time the task difficulty is adjusted.

[0083] The difficulty of the current round of tasks can be calculated using the following formula: Formula 2; In Formula 2, It's the same difficulty as the previous round of tasks. This is the amount of time / money that the task difficulty is adjusted. This refers to the difficulty of the current round of tasks.

[0084] The task generation model determines the reasoning capability boundary of the task processing model based on the quality difference between the historical quality assessment value and the expected quality assessment value of the previous reasoning task. Based on this, it dynamically adjusts the difficulty level of subsequent tasks. This ensures that the task difficulty always matches the reasoning level of the task processing model, keeping the model within a learnable yet challenging range. This avoids both inefficiency caused by overly simple tasks and overfitting caused by overly difficult tasks. It encourages the model to adaptively increase the difficulty during training, gradually improving its comprehensive ability of knowledge reasoning and task solving. This steadily optimizes the overall learning effect and reasoning level of the model, enabling it to gradually cope with increasingly complex reasoning tasks.

[0085] Next, based on the reasoning ability parameters after the previous round of parameter tuning and the previous round of reasoning task, combined with the difficulty of the current round's task, the current round's reasoning task is generated. Specifically, the following formula can be used to generate the current round's reasoning task: Formula 3; In Formula 3, It's the reasoning task from the previous round. These are the reasoning ability parameters after the previous round of parameter tuning. This is the current round of reasoning task.

[0086] Optional, such as Figure 3C As shown, the specific implementation details for generating the current round of reasoning tasks are as follows: S3011: Based on the reasoning ability parameters after the previous round of parameter tuning, determine the knowledge reasoning level, applicable task types, and task difficulty range of the task processing model; wherein, the lower limit of the task difficulty range is higher than the expected task difficulty corresponding to the knowledge reasoning level.

[0087] The inference capability parameters after the previous round of parameter tuning include multiple inference parameter items. Each inference parameter item is used to describe a performance index of the task processing model when processing the previous round of inference task.

[0088] The dimensions and value ranges of different inference parameters often vary significantly (e.g., some parameters have single-digit values, while others have values ​​in the hundreds or thousands). If the original inference parameters are used directly for inference calculations, parameters with large value ranges will excessively dominate the inference process, while parameters with small value ranges will be ignored, thus interfering with the accuracy of the inference logic. Furthermore, the high discreteness of the original inference parameters increases the computational complexity of the model and affects inference efficiency. Therefore, normalization is necessary to standardize and regulate the parameters.

[0089] First, preprocessing methods such as Z-score standardization and min-max normalization are used to normalize each inference parameter, resulting in preprocessed parameter terms for each inference parameter. Normalization eliminates the influence of differences in dimensions and value ranges among different parameter terms, ensuring that each parameter plays a balanced role in the inference process, avoiding a single parameter dominating the inference result, and improving the rationality and accuracy of the inference logic. Simultaneously, the normalized parameter values ​​are more concentrated and regular, which reduces the computational difficulty of the model, reduces redundant computational overhead, accelerates inference speed, and improves model processing efficiency.

[0090] Next, a weighted summation method is used to evaluate the reasoning ability based on each preprocessing parameter and its preset evaluation weight, thereby obtaining the reasoning ability evaluation value of the task processing model. The reasoning ability evaluation range in which the reasoning ability evaluation value falls is determined, and the knowledge reasoning level, task difficulty range, and applicable task type associated with the reasoning ability evaluation range are obtained.

[0091] By employing a weighted summation method to assess reasoning ability, the impact of different dimensions of reasoning parameters on the overall ability is comprehensively considered, making the assessment results more objective, comprehensive, and in line with the actual level. Based on the assessment results, the current reasoning ability and knowledge mastery of the model can be accurately determined, enabling a quantitative judgment of the model's ability status and providing a reliable basis for subsequent task difficulty adjustment, knowledge point supplementation, and reasoning strategy optimization.

[0092] The task generation model can also constrain and correct the difficulty of the current round of tasks based on the current difficulty range of the task processing model, so that the difficulty value is limited to a reasonable range and avoids the situation of being too high or too low. This ensures that the newly generated inference tasks can effectively test the model's inference level and provide the model with a suitable training gradient, thereby steadily improving the model's knowledge application and problem-solving ability.

[0093] S3012: Retrieve at least one proposition-related knowledge point that matches the knowledge reasoning level, applicable task type, and task difficulty range, and adjust the previous round reasoning task to the current round reasoning task based on at least one proposition-related knowledge point and the current round task difficulty.

[0094] The knowledge reasoning level, applicable task type, and task difficulty range of the task processing model are used as retrieval constraints. The applicable ability level, corresponding task type, and recommended difficulty level of each historical knowledge point are matched, retrieved, and filtered in the knowledge graph. Knowledge points that simultaneously meet the model's knowledge reasoning level, applicable task type, and task difficulty range are selected. Finally, at least one proposition-related knowledge point that is suitable for the current state of the model is retrieved.

[0095] Next, based on the difficulty of the current round of tasks, determine the content of the tasks to be modified in the previous round of reasoning tasks.

[0096] Specifically, based on the difficulty of the current round of tasks, a comprehensive analysis and targeted adjustment are made to the core content of the previous round of reasoning tasks, including the task description, problem statement, preconditions, constraints, and solution objectives. Through structured decomposition and optimized configuration of task-related elements, the specific aspects requiring modification and the extent of modification are determined to ensure an effective match between the modified task difficulty and the model's capabilities.

[0097] Then, based on at least one proposition-related knowledge point and the set of logical reasoning relationships associated with each of the at least one proposition-related knowledge point, the text to be modified is modified to generate the current round of reasoning task.

[0098] For example, the model's current reasoning ability is assessed as medium, and the difficulty of the current round's task is medium-high. The previous round's reasoning task was "Given a triangle with two interior angles of 30° and 60°, find the third interior angle." Based on the difficulty of the current round's task, the difficulty of the previous round's reasoning task is increased, and the task content is modified as follows: The complexity of the preconditions is increased; the single triangle is changed to a right triangle with one acute angle of 30° and the side opposite that angle is 5cm; the solution objective is expanded from only finding the interior angle to finding the hypotenuse length, the measure of the other acute angle, and the area of ​​the triangle; constraints are added, such as keeping angle calculations as integers and unifying the unit of length to centimeters. The final current round's reasoning task is: "In a right triangle, the side opposite the 30° acute angle is 5cm. Find the measure of the other acute angle, the length of the hypotenuse, and the area of ​​the triangle. The angle results should be kept as integers, and the units of the results should be unified as centimeters and square centimeters."

[0099] By modifying the previous inference task based on the difficulty of the current task, the complexity and assessment dimensions of the task can be flexibly controlled. This ensures that the task difficulty is highly compatible with the model's current inference ability and knowledge level, avoiding both overly easy tasks that prevent effective model training and overly difficult tasks that negatively impact model learning efficiency. Furthermore, targeted adjustments to the previous inference task allow for the full reuse of existing task structures and knowledge points, reducing the cost of building new tasks from scratch and improving task generation efficiency and standardization. By reasonably adjusting preconditions, solution objectives, and constraints, the model's weaknesses can be accurately identified, strengthening its understanding and application of relevant knowledge points. This makes training and evaluation more targeted, steadily improving the model's inference stability and problem-solving ability.

[0100] The automatically generated reasoning tasks undergo multi-dimensional quality verification, including task solvability, discriminative power, and difficulty matching. Solvability verification checks whether the task conditions are complete, logically consistent, and whether there are contradictions or unsolvable scenarios, ensuring the model can arrive at a reasonable answer based on existing knowledge and reasoning paths. Discriminative power verification determines whether the task effectively reflects different ability levels of the model, effectively identifying and distinguishing between knowledge mastery and reasoning proficiency. Difficulty matching verification compares the actual difficulty of the task with the preset difficulty range to confirm its suitability for the model's current reasoning ability and knowledge level. Through multi-level and comprehensive checks and screenings, the final generated tasks are ensured to be reasonable, effective, and of moderate difficulty, enabling stable use for model training and capability evaluation.

[0101] S302: Send the current round of reasoning task to the task processing model for reasoning, obtain multiple reasoning answers for the current round of reasoning task, and evaluate the quality of the multiple reasoning answers through the task generation model to obtain the actual quality evaluation value and reasoning reward value of the current round of reasoning task.

[0102] Combination Figures 3D-3E The specific process of model reasoning analysis is explained.

[0103] S3021: Send the current round of reasoning task to the task processing model and extract at least one key problem-solving information from the current round of reasoning task.

[0104] The original text of the current inference task (such as natural language questions, inference propositions, scene descriptions, etc.) is input into the model's processing module. First, the text is cleaned to remove irrelevant and redundant information, special symbols, invalid spaces and ambiguous expressions. At the same time, word segmentation, part-of-speech tagging and sentence structure splitting are completed to transform the unstructured original text into task text that can be recognized by the model, laying the foundation for subsequent semantic understanding.

[0105] The model, built on LLM technology, performs semantic parsing on preprocessed task text. Leveraging its context-aware capabilities, it uncovers the underlying semantic logic and core meaning of the text, accurately identifies key segments relevant to problem-solving, distinguishes between known and unsolved information, and avoids semantic ambiguity, ensuring a precise understanding of the task's intent. Fine-tuning and optimization of the task processing model can further enhance its semantic understanding accuracy, adapting it to knowledge reasoning scenarios across different domains.

[0106] The task processing model extracts at least one key problem-solving information from the task based on the semantic understanding results output by LLM.

[0107] Specifically, by using LLM technology to identify guiding statements such as "known," "given," and "premise" in the task text, the known facts, premise data, background knowledge, initial states, and other preconditions provided by the task are extracted, structured, and the starting basis for the reasoning process is clarified. For example, in mathematical reasoning tasks, the numerical values, formulas, and known relationships given in the question are extracted; in common sense reasoning tasks, the objective facts and background information explicitly given in the text are extracted.

[0108] Leveraging the logical recognition capabilities of LLM, the model extracts implicit or explicit constraints from the task, namely the rules, limitations, logical boundaries, and prohibitions that must be followed during the solution process. Examples include time constraints, numerical range constraints, and logical relationship constraints. Simultaneously, through LLM's contextual analysis, it completes the implicit constraint information, preventing inference biases caused by omitted constraints.

[0109] By using LLM to identify target-guiding words such as "seek," "need to determine," "verify," and "derive" in the text, the ultimate solution objective of the task can be clarified, namely the result to be achieved in the reasoning process, the conclusion to be drawn, the proposition to be verified, or the core problem to be solved. The solution objective can be quantified or clarified to avoid deviation from the reasoning direction.

[0110] If the current inference task is complex, it is broken down into several interconnected and progressively more complex subtasks. For each subtask, at least one key problem-solving piece of information is extracted using the model. Based on this extracted information, the relevant reference problem-solving knowledge points needed to solve the subtask are identified.

[0111] Before executing step 3022, the extracted key problem-solving information, such as preconditions, constraints, and solution objectives, is cross-validated to determine whether the extracted information is complete and whether there are any contradictions. Erroneous information is removed and missing information is supplemented.

[0112] S3022: For each key problem-solving information, record at least one historical knowledge point associated with the key problem-solving information in the knowledge point retrieval set as a reference problem-solving knowledge point; when no associated historical knowledge point is found in the knowledge point retrieval set, the key problem-solving information is converted into a newly added reference problem-solving knowledge point; the knowledge point retrieval set is composed of at least one of the historical knowledge points learned by the task processing model and the historical knowledge points included in the knowledge graph.

[0113] For simple reasoning tasks, the model can quickly solve problems by relying on its acquired knowledge. For complex reasoning tasks, the model can combine its existing knowledge system with the knowledge points in the knowledge graph to perform comprehensive reasoning and step-by-step solutions.

[0114] When the knowledge point retrieval set contains all historical knowledge points already learned by the task processing model, the following method is used when retrieving key problem-solving information: Extract the semantic features of the question stem from the key problem-solving information, and perform semantic matching between the semantic features of the question stem and the semantic features of each historical knowledge point already learned in the task processing model. Record each historical knowledge point whose semantic feature similarity exceeds the preset feature similarity threshold as a reference problem-solving knowledge point.

[0115] By using semantic features for association matching, we can accurately locate historical knowledge points that are highly relevant to the current round of reasoning tasks. This effectively avoids interference from irrelevant knowledge in the reasoning process, ensuring that the selected knowledge points have sufficient reference value, thereby improving problem-solving efficiency and result accuracy. At the same time, by reusing the knowledge already learned by the model through semantic matching, there is no need for repeated retrieval or relearning, which reduces the model's computational overhead and improves the ability to respond quickly to similar and related reasoning tasks.

[0116] Combination Figure 3F When the knowledge point retrieval set contains historical knowledge points already included in the knowledge graph, the following method is used when retrieving key problem-solving information: The key problem-solving information is matched with the knowledge graph to obtain at least one candidate knowledge point that is successfully matched. Then, for each candidate knowledge point, a depth-first traversal or a breadth-first traversal is performed along the edges associated with the candidate knowledge point in the knowledge graph, and each historical knowledge point encountered is recorded as a reference problem-solving knowledge point.

[0117] By matching key problem-solving information with the knowledge graph, candidate knowledge points relevant to the task can be quickly located, avoiding starting the problem from scratch and providing a reliable knowledge foundation for subsequent reasoning. Starting with candidate knowledge points and traversing along the edges to expand related historical knowledge points, the implicit connections and reasoning chains within the knowledge graph can be fully explored. This effectively supplements background knowledge and reasoning basis that a single matching result cannot cover, making the reference problem-solving knowledge points more comprehensive and complete, avoiding biased or incomplete reasoning due to insufficient knowledge. Simultaneously, traversing and retrieving based on the structured relationships within the knowledge graph significantly improves the systematic and logical nature of knowledge acquisition, reduces interference from irrelevant information, improves the efficiency of model reasoning, and enhances the rigor and rationality of the problem-solving process, making the final output more convincing and accurate.

[0118] S3023: Based on the recorded reference problem-solving knowledge points and the set of logical reasoning relationships associated with each reference problem-solving knowledge point, obtain multiple problem-solving reasoning paths.

[0119] After obtaining the reference problem-solving knowledge points and their corresponding logical reasoning relationships, and taking the solution objective as the ultimate goal and the preconditions as the starting point of reasoning, the reference problem-solving knowledge points are structurally organized and logically connected. Based on the causal, subordinate, deductive, and equivalence relationships between the knowledge points, the scattered reference problem-solving knowledge points are gradually connected in a logical reasoning order, forming multiple problem-solving reasoning paths with complete deductive chains.

[0120] Furthermore, the logical coherence, condition completeness, and target orientation of each path are verified, and paths with logical breaks, contradictory relationships, or deviations from the solution target are eliminated, ultimately yielding multiple reasonable and usable problem-solving reasoning paths.

[0121] For example, taking the current reasoning task "A triangle has two interior angles of 30° and 60°, find the third interior angle" as an example, the input conditions "The sum of the interior angles of a triangle is 180°" and "The two interior angles are known to be 30° and 60°" are extracted from the task, and the solution objective is determined to be "the measure of the third interior angle". Through knowledge matching and graph traversal, the following reference solution knowledge points are obtained: 1. The sum of the interior angles of a triangle equals 180°; 2. The third angle can be obtained by subtracting the sum of the two known angles; 3. The rules of angle subtraction. The model is linked according to the set of logical reasoning relationships. Reference solution knowledge point 1 leads to the conclusion that the sum of the interior angles is fixed. Combined with reference solution knowledge point 2, the calculation method is determined. Then, reference solution knowledge point 3 is used to complete the numerical calculation, ultimately forming a complete problem-solving reasoning path.

[0122] The knowledge graph pre-contains historical reasoning paths formed by the model when handling similar or identical reasoning tasks. During the current task processing, the structural paradigms and reasoning logic of these historical paths can be directly referenced. The extracted reference problem-solving knowledge points, along with their associated logical reasoning relationships, are then systematically combined and logically connected to quickly generate multiple standardized, reliable, and task-specific problem-solving reasoning paths. The knowledge graph endows the model with cross-task knowledge transfer capabilities, enhancing its adaptability to various tasks. Especially when facing unfamiliar tasks and entirely new reasoning scenarios, the knowledge graph enables stronger generalization performance, quickly constructing reasonable reasoning paths and effectively improving accuracy and robustness in unknown scenarios.

[0123] For example, when solving complex mathematical reasoning tasks, the model can deduce the optimal reasoning path based on the historical reasoning paths for this type of task or similar tasks recorded in the knowledge graph, combined with the retrieved reference problem-solving knowledge points and the set of logical reasoning relationships associated with each reference problem-solving knowledge point.

[0124] By introducing historical reasoning paths, the model can fully reuse its accumulated historical knowledge and reasoning experience, avoid repeatedly performing redundant reasoning calculations, significantly improve reasoning efficiency and the rationality of path generation, and further enhance the practicality and rigor of the model in solving reasoning tasks.

[0125] S3024: For each problem-solving reasoning path, according to the problem-solving reasoning path, execute the problem-solving derivation steps sequentially on at least one key problem-solving information to obtain the reasoning answer.

[0126] For each generated problem-solving reasoning path, guided by the logical order within the path, at least one key problem-solving piece of information is substituted, and the corresponding problem-solving derivation steps are executed sequentially to obtain the reasoned answer. Each derivation step strictly follows the logical reasoning relationship between the reference problem-solving knowledge points in the path, ensuring the rigor and coherence of the derivation process.

[0127] For example, taking the current reasoning task "Given that two interior angles of a triangle are 30° and 60°, find the type of the triangle and the measure of the third interior angle", suppose one of the generated reasoning paths is "The sum of the interior angles of the triangle is 180°" to "Subtract the known two angles from the sum of the interior angles to get the third interior angle", and then "Determine the type of the triangle based on the three interior angle measures".

[0128] First, the key information extracted for solving the problem includes the premise (two interior angles are 30° and 60°), constraints (the sum of the interior angles of a triangle is 180°, and the triangle type determination rule), and the objective (the measure of the third interior angle, and the triangle type). Next, following this reasoning path, the derivation steps are executed sequentially: First, based on the reference knowledge point that "the sum of the interior angles of a triangle is 180°," the derivation benchmark is determined; second, the angle calculation derivation is performed, subtracting the known 30° and 60° from 180° to obtain a third interior angle of 90°; third, combining the reference knowledge point of "the determination rule for right triangles (a triangle with one interior angle of 90° is a right triangle)," the triangle is deduced to be a right triangle. Finally, integrating the results of the two derivations, the complete answer is that the measure of the third interior angle is 90°, and the triangle is a right triangle.

[0129] By extracting key problem-solving information, the known conditions and solution objectives of the task can be accurately identified, avoiding interference from irrelevant information. Then, based on the key problem-solving information and the knowledge point retrieval set of the knowledge graph, a standardized and orderly problem-solving reasoning path is generated. This allows for the full reuse of historical knowledge and logical relationships, making the reasoning process more structured and targeted. By sequentially executing problem-solving derivation steps on at least one key piece of information according to the logical reasoning relationship of each problem-solving reasoning path, the derivation process is ensured to be systematic and logically closed, avoiding logical confusion, omissions, or deviations from the solution objective, significantly improving the accuracy and reliability of the task analysis results. Simultaneously, the step-by-step derivation method clearly presents the complete derivation process from key information to the final result, making the reasoning process traceable and verifiable, facilitating subsequent verification and optimization of the derivation logic.

[0130] After obtaining multiple reasoning answers, the analysis results of each task are verified in multiple dimensions, including consistency verification between analysis results, logical verification of the reasoning process and knowledge rules, and constraint verification of whether they meet the task constraints and preset boundaries. The rationality, accuracy, and compliance of each analysis result are comprehensively judged. Through multi-layer verification, abnormal results with logical contradictions, inconsistencies, or violations of constraints are filtered out. Finally, the reliable result with the highest credibility, the strongest stability, and the best fit for the task requirements is selected as the final output of the model.

[0131] From multiple inference answers, one inference answer is selected as the reference parsing result (i.e., pseudo-label). Through self-consistency judgment, the inference answer with the highest consistency is selected from multiple inference answers and used as the reference answer for the current round of inference task, which is used to train and evaluate the model's performance.

[0132] Using a task generation model, based on the reference answer, the quality of each remaining reasoning answer is evaluated, yielding a quality evaluation value for each. Then, using the task generation model again, a task evaluation is performed based on the obtained quality evaluation values ​​and the total number of reasoning answers, resulting in the actual quality evaluation value. Specifically, the actual quality evaluation value of the task processing model in the current round can be calculated using the following formula: Formula 4; In Formula 4, n is the total number of possible solutions. It is the i-th reasoning answer. It is a reference answer selected from multiple inference options. This is the actual quality assessment value.

[0133] Using the model's own output as the reference answer provides a rapid, standardized benchmark in scenarios lacking manually labeled data, effectively reducing data labeling costs and time. Evaluation based on the reference answer more accurately reflects the model's current capabilities, precisely demonstrating the difference between the given answer and the model's optimal reasoning result, thus enabling quantitative assessment of solution quality. Furthermore, the reference answer can be dynamically updated as the model's capabilities evolve, continuously adapting to different training and evaluation stages, making quality assessment more timely and targeted, and providing stable and reliable feedback for model iteration and optimization.

[0134] After completing each inference task, the task generation model provides feedback on profit and loss based on the actual quality assessment value of the task processing model. When the actual quality assessment value is not lower than the preset quality assessment value threshold, profit feedback is provided based on the actual quality assessment value, generating an inference reward value; when the actual quality assessment value is lower than the quality assessment value threshold, loss feedback is provided based on the actual quality assessment value, generating an inference reward value.

[0135] The reasoning reward value can be calculated using the following formula: Formula 5; In Formula 5, This is the actual quality assessment value. R is the preset weight value, and R is the inference reward value.

[0136] This application can also use the degree of difference in task content between the current round of reasoning and the previous round of reasoning, or the degree of difference in knowledge points, logical relationships, and structure of the knowledge graph before and after the update, as the model novelty of this model. Model novelty is used to describe the learning innovation and adaptability of the model when facing new tasks and new knowledge. The more significant the difference in task content and the richer the knowledge graph update, the higher the model novelty; conversely, the lower the model novelty, the more similar the task content and the less significant the knowledge graph update.

[0137] Based on actual quality assessment values ​​and model novelty Profit and loss feedback is implemented. When the model has high novelty and the actual quality assessment value is excellent, profit feedback is implemented to generate inference reward value. When the model has high novelty but the actual quality assessment value is poor, loss feedback is implemented based on the actual quality assessment value to generate inference reward value.

[0138] The reasoning reward value can be calculated using the following formula: Formula 6; In Formula 6, This is the actual quality assessment value. It is the novelty of the model. , R is the preset weight value, and R is the inference reward value.

[0139] Feedback is provided based on the actual quality assessment values ​​of the task processing model, providing a quantitative and intuitive evaluation guide for the model's reasoning performance. When the model answers accurately and performs well in a task, a reasoning reward value is generated to positively incentivize effective reasoning behavior and strengthen the model's learning and reuse of high-quality problem-solving paths and knowledge points. When the model answers with significant deviations and performs poorly, a reasoning reward value is generated to negatively penalize unreasonable reasoning behavior, prompting the model to proactively correct logical flaws and knowledge errors. This mechanism allows the model to clearly perceive the differences in its own answer quality, enabling precise gradient-based iterative optimization, effectively improving model learning efficiency, and steadily enhancing its knowledge reasoning ability and task answering stability.

[0140] S303: Based on the actual quality assessment value and the inference reward value, update the parameters of the task processing model and the task generation model, take the current round task difficulty as the new previous round task difficulty, and take the current round inference task as the new previous round inference task, and perform iterative training.

[0141] The system first calibrates and corrects the actual quality assessment value based on the inference reward value, obtaining a calibrated actual quality assessment value. Then, it compares this calibrated actual quality assessment value with the expected quality assessment value to calculate the loss value generated by the task processing model when handling the current inference task. Based on this loss value, the system updates the parameters of both the task processing model and the task generation model. By iteratively optimizing the model's internal weights and inference-related parameters, the gap between the actual output and the expected standard is narrowed, thereby improving the problem-solving accuracy of the task processing model and the task generation quality of the task generation model.

[0142] Knowledge graphs, as a related technology, are constructed by analyzing entities, concepts, and the relationships between them. By introducing knowledge graphs, the model's ability to understand and reason about information is further enhanced. However, the construction of knowledge graphs relies on external information sources and manual annotation tools, and they do not yet have the ability to generate knowledge on their own, making it difficult to dynamically adapt to new tasks and added information.

[0143] To address this issue, this application introduces a knowledge graph construction and dynamic update mechanism in addition to the self-evolution mechanism. The knowledge graph helps the model quickly acquire task-related background knowledge during reasoning, enhancing the depth and accuracy of reasoning. As the task progresses, the model will continuously expand and optimize the knowledge graph to adapt to increasingly complex reasoning tasks.

[0144] Knowledge graphs are represented by a graph structure. Nodes in the graph represent knowledge points (such as concepts, facts, principles, formulas, etc.), while edges represent the logical reasoning relationships between knowledge points (such as causal relationships, definitional relationships, etc.).

[0145] In this application, the knowledge graph is constructed based on the model's accumulated experience during the reasoning process. Each time the model solves a task, it extracts knowledge points from the task and adds these points to the knowledge graph according to the reasoning path during the solution process. As the model continuously processes new tasks, it continuously expands and updates the knowledge graph, making the knowledge within the graph more comprehensive and context-sensitive.

[0146] After forming multiple problem-solving reasoning paths, before sending the next round of reasoning tasks to the task processing model, the system, for each problem-solving reasoning path, checks if the path contains newly added reference problem-solving knowledge points. At that time, the knowledge graph is updated based on the newly added reference problem-solving knowledge points and the set of logical reasoning relationships associated with the newly added reference problem-solving knowledge points in the problem-solving reasoning path.

[0147] As the model accumulates more tasks and knowledge, the knowledge graph will continuously expand and update. Whenever a new reference solution is extracted, the nodes in the knowledge graph will be updated. If a new reference solution is not yet included in the knowledge graph, it will be added as a new node. If a new knowledge point is related to an existing historical knowledge point, an edge will be added to the graph.

[0148] The update formula for knowledge graphs is: ,in, It is the updated knowledge graph. This is the knowledge graph before the update. , 'r' represents the newly added knowledge points extracted in the current round of reasoning task, and 'r' represents the logical reasoning relationship between the newly added knowledge points.

[0149] For example, regarding the reference problem-solving knowledge points and Based on the reasoning process of two knowledge points in the problem-solving reasoning path, if a logical reasoning relationship is determined between the two knowledge points, then an edge r is added between the two knowledge points to represent the type of relationship between them (such as definitional relationship, causal relationship, dependency relationship, etc.), forming an edge r like this: Figure 3G The knowledge graph shown.

[0150] By dynamically updating the knowledge graph, it not only carries the knowledge points learned during the model training phase, but also continuously absorbs new knowledge points, supplements logical connections, and improves reasoning paths through the model's responses and inference results in various reasoning tasks, achieving dynamic expansion and iterative enhancement of the knowledge system. As the number of task processing sessions increases, the knowledge graph becomes more comprehensive, its logical relationships more complete, and its reasoning basis more sufficient, providing more comprehensive and reliable knowledge support for the model to conduct complex reasoning, accurate retrieval, and autonomous learning, significantly improving the model's long-term reasoning ability and knowledge application level.

[0151] As the number of tasks processed increases, some existing knowledge points in the knowledge graph may logically conflict with newly added reasoning paths. To ensure the overall accuracy, consistency, and reliability of the knowledge graph and its reasoning, this application introduces a self-correction mechanism for the knowledge graph.

[0152] After updating the knowledge graph, the system uses the newly added reference problem-solving knowledge points as the starting point for traversal, and traverses along the edges associated with these knowledge points in the knowledge graph to extract the set of logical reasoning relationships. The set of logical reasoning relationships is then validated against logical rules. If at least one logical reasoning relationship is found to conflict with a logical rule, the knowledge graph undergoes a secondary correction.

[0153] In addition to using logical rules for verification, confidence assessment can also be performed on newly added reference problem-solving knowledge points. By combining multi-dimensional information such as the frequency of edges between historical knowledge points and new knowledge points in the knowledge graph, the actual usage effect of this type of knowledge in previous rounds of reasoning tasks, reasoning success rate, and reuse value, a confidence score is calculated and assigned to the new knowledge point.

[0154] If it is determined that new knowledge has a logical conflict or inconsistency with the content in the existing knowledge graph, the knowledge will not be directly added to the database. Instead, it will be marked as pending verification, or the model will be triggered to re-perform the reasoning and verification process for the conflicting content. Only after it is confirmed that there are no errors will a decision be made on whether to include it in the knowledge graph.

[0155] By verifying and correcting logically conflicting knowledge points and reasoning relationships within the knowledge graph, knowledge contradictions are eliminated, knowledge expression is standardized, and the internal logic of the knowledge graph is ensured to be rigorous and consistent, providing a reliable knowledge foundation for model reasoning. Simultaneously, reasonable corrections can improve the storage and retrieval efficiency of the knowledge graph, avoiding redundant information from interfering with the model's reasoning judgments, making the model more accurate and efficient in handling various reasoning tasks. A continuously maintained and optimized knowledge graph can maintain high quality and high availability in the long term, further supporting the model to achieve more complex knowledge associations and deep reasoning, providing a solid guarantee for the stable operation and long-term self-improvement of the overall system.

[0156] In some reasoning tasks, the model may encounter new domain knowledge that does not fully match the coverage of the current knowledge graph. In such cases, the model will automatically align and integrate the new knowledge with the existing knowledge points and logical relationships in the graph, further expanding the knowledge domain and scope of application of the knowledge graph.

[0157] Meanwhile, to avoid the knowledge graph becoming too large and cluttered due to continuous expansion, the model uses information filtering when expanding the graph, retaining only knowledge points that are highly relevant to the current task scenario and reasoning process and have practical reasoning value. This ensures that while expanding knowledge, the knowledge graph remains compact and efficient.

[0158] Specifically, each newly added knowledge point undergoes a relevance assessment and is filtered based on a preset threshold, retaining only those scoring above the threshold for inclusion in the knowledge graph. In the actual calculation process, the degree of relevance of knowledge points is comprehensively judged from the following dimensions: First, task semantic relevance, assessing the degree of matching between the knowledge point and the keywords, core concepts, related knowledge points, and constraints in the current reasoning task; a higher matching degree indicates higher relevance. Second, reasoning path relevance, determining whether the knowledge point is in the key reasoning path of the current problem-solving process or whether it can directly support the derivation of an intermediate conclusion. Third, graph structure distance, judging based on the topological distance between the knowledge point and the key knowledge points of the current task in the knowledge graph; closer node distances generally indicate stronger relevance. Fourth, historical validity, referring to the frequency of use and reasoning contribution of the knowledge point in similar historical tasks; higher historical usage frequency and higher reasoning success rate result in higher priority. Fifth, confidence and conflict status, prioritizing the retention of knowledge points with high confidence and no significant conflict with existing reasoning results; knowledge points with significant conflicts or low reliability are temporarily excluded.

[0159] After updating the knowledge graph, when the reasoning reward value is not lower than the preset reasoning reward value threshold, multiple problem-solving reasoning paths are marked as important reasoning paths in the knowledge graph, and each reference problem-solving knowledge point involved in multiple problem-solving reasoning paths and the edge associated with each reference problem-solving knowledge point are marked as important knowledge elements, thereby strengthening their priority and reuse value in the knowledge graph and facilitating quick access for similar tasks in the future.

[0160] When the reasoning reward value is lower than the reasoning reward value threshold, multiple problem-solving reasoning paths are marked as reasoning paths to be verified in the knowledge graph. Each reference problem-solving knowledge point involved in the multiple problem-solving reasoning paths and the edge associated with each reference problem-solving knowledge point are marked as knowledge elements to be verified, or the confidence score of the relevant knowledge points is reduced. This indicates that the knowledge point has potential problems such as insufficient effectiveness, limited applicability, or logical deviation, and more reasoning data and scenario information need to be added for further verification and optimization.

[0161] This approach not only identifies high-value, reliable knowledge, but also promptly locates and isolates suspicious or invalid knowledge points, preventing erroneous knowledge from interfering with model reasoning. While continuously optimizing the quality of the knowledge graph, it significantly improves the stability and accuracy of model reasoning, providing crucial support for the model to achieve autonomous iteration and self-improvement.

[0162] Next, combined Figure 4 Taking the intelligent question-answering scenario as an example, this paper introduces the application process of this method in the corresponding scenario.

[0163] In the intelligent question-answering scenario, based on the reasoning ability parameters of the task processing model (temperature=0.7, top_p=0.9), the first question is generated: "Given that the perimeter of a rectangle is 20 centimeters, and both its length and width are integers, find all possible combinations of length and width, and calculate the area of ​​the rectangle for each combination."

[0164] The task processing model, aided by a knowledge graph (containing knowledge points such as rectangles, perimeter formulas, area formulas, integer constraints, and enumeration methods), answers the question and generates multiple answers: Answer 1: "Let the length be a and the width be b, then 2(a+b)=20 => a+b=10. Enumerate the following under the condition that a>b>0 and a and b are integers: (a, b) = (9, 1), (8, 2), (7, 3), (6, 4), (5, 5). The corresponding area S = a..." The values ​​for b are 9, 16, 21, 24, and 25 square centimeters, respectively. Answer 2: "Half perimeter a+b = 10. Possible integer pairs: (1, 9), (2, 8), (3, 7), (4, 6), (5, 5). Note the length-width order, take the pair with length ≥ width: (9, 1), (8, 2), (7, 3), (6, 4), (5, 5). Area: 9, 16, 21, 24, 25." Answer 3: "Given a + b = 10, and a and b are positive integers, a takes values ​​from 1 to 9, and b = 10 - a. We need to satisfy a ≥ b, i.e., a ≥ 10 - a => a ≥ 5. Therefore, a = 5, 6, 7, 8, 9. The corresponding combinations are (5, 5), (6, 4), (7, 3), (8, 2), (9, 1). The area is 25, 24, 21, 16, 9." The task generation model uses self-consistency judgment to vote for the most consistent answer C from multiple answers, which is then used as the reference answer for the question. Based on the reference answer, the quality of each answer is evaluated (out of 1.0), resulting in an actual quality evaluation value of 0.9 for the task processing model's response to the first question. Since the actual quality evaluation value exceeds the threshold (set at 0.88), it indicates that the current model has a good grasp of enumeration application problems under integer constraints, and a reasoning reward value of 0.02 is generated.

[0165] The system calibrates and corrects the actual quality assessment value based on the inference reward value, obtaining a calibrated actual quality assessment value of 0.8996. This calibrated actual quality assessment value is then compared with the expected quality assessment value (0.88) to calculate the loss value of 0.000384 (calculated using mean squared error) generated by the task processing model when processing the current inference task. This loss value is then used to update the parameters of both the task processing model and the task generation model.

[0166] After answering the first question, the new knowledge points "semi-perimeter" and "ordered integer pair enumeration" are extracted from the multiple problem-solving reasoning paths generated when the model answers the first question. Based on the set of logical reasoning relationships between the new knowledge points in the problem-solving reasoning paths, "semi-perimeter" and "ordered integer pair enumeration" are taken as new nodes, and "perimeter problem - [often convertible to] -> semi-perimeter", "integer constraint - [common method] -> ordered integer pair enumeration", and "rectangle area - [affected by] -> ordered length and width pairs [influenced]" are taken as new edges and added to the knowledge graph.

[0167] After answering the first question, based on the quality difference (0.02) between the actual quality assessment value (0.9) and the expected quality assessment value (0.88), and combined with the preset difficulty adjustment coefficient (0.15), the difficulty of the first question was increased from beginner to intermediate.

[0168] Then, based on the adjusted reasoning ability parameters, the knowledge reasoning level of the task processing model is determined to be intermediate, the applicable task type is "algebraic equations and integer constraint combination problem", and the task difficulty range is "intermediate". Using these three points as search conditions, at least one matching proposition-related knowledge point, "maximum value problem" or "inequality", is retrieved from the knowledge graph. Based on at least one proposition-related knowledge point and the current round's task difficulty (intermediate), the content of the previous round's reasoning task (fixed perimeter integer side rectangle problem) is adjusted to generate a second question: "In a rectangle with a fixed perimeter of 20 centimeters and integer sides, when is the area largest? When is the area smallest? Please prove your conclusion and discuss the trend of shape change of the rectangle."

[0169] Repeat the above operation for the second question, iteratively train the task processing model and the task generation model until the training loss value is below 0.0001 for 10 consecutive rounds, and the accuracy reaches 98% on the new knowledge graph subgraph containing "inequalities" and "maximum and minimum values", and the trained task processing model is obtained.

[0170] During the model application phase, the user enters the question "A right triangle has two legs that are consecutive positive integers, and the hypotenuse is 5. Find the perimeter and area of ​​this triangle." into the input box of the AI ​​math tutoring tool interface. The server calls the task processing model, which answers the question based on the current knowledge graph (including nodes and relationships such as Pythagorean theorem, integers, equations, and enumerations) and the latest reasoning ability parameters, outputs multiple answers, marks the answer with the highest confidence as the final answer, and returns the information "The three sides of this right triangle are 3, 4, and 5. The perimeter is 12 centimeters, and the area is 6 square centimeters" along with the core steps to the AI ​​math tutoring tool interface for the user to see.

[0171] Based on the same inventive concept as the above-described method embodiments, this application also provides a training device for a task processing model. For example... Figure 5 As shown, the training device 500 for the task processing model includes: The task generation module 501 is used to adjust the difficulty of the previous round of tasks through the task generation model, determine the difficulty of the current round of tasks, and adjust the previous round of reasoning tasks to the current round of reasoning tasks that match the difficulty of the current round of tasks. The task evaluation module 502 is used to send the current round of reasoning task to the task processing model for reasoning, obtain multiple reasoning answers for the current round of reasoning task, and evaluate the answer quality of multiple reasoning answers through the task generation model to obtain the actual quality evaluation value and reasoning reward value of the current round of reasoning task. The model training module 503 is used to update the parameters of the task processing model and the task generation model based on the actual quality assessment value and the inference reward value. The current round of task difficulty is used as the new previous round of task difficulty, and the current round of reasoning task is used as the new previous round of reasoning task for iterative training.

[0172] Optionally, the task generation module 501 is used for: By using a task generation model, the difficulty of the previous round of tasks is adjusted based on the quality difference between the historical quality assessment value and the expected quality assessment value of the previous round of inference tasks, and the difficulty of the current round of tasks is determined.

[0173] Optionally, the task generation module 501 is used for: When the quality difference of the previous inference task indicates that the historical quality assessment value is higher than the expected quality assessment value, the task difficulty increase is generated based on the quality difference and the preset difficulty adjustment coefficient. Based on the task difficulty increase, the difficulty of the previous task is adjusted upward to determine the difficulty of the current task. When the quality difference indicates that the historical quality assessment value is lower than the expected quality assessment value, the task difficulty reduction amount is generated based on the quality difference and the difficulty adjustment coefficient. Based on the task difficulty reduction amount, the difficulty of the previous round of tasks is adjusted down to determine the difficulty of the current round of tasks. When the quality difference indicates that the historical quality assessment value equals the expected quality assessment value, the difficulty of the previous round of tasks is adjusted to zero, and the difficulty of the previous round of tasks is used as the difficulty of the current round of tasks.

[0174] Optionally, the task generation module 501 is used for: Based on the reasoning ability parameters after the previous round of parameter tuning, the knowledge reasoning level, applicable task types, and task difficulty range of the task processing model are determined; among them, the lower limit of the task difficulty range is higher than the expected task difficulty corresponding to the knowledge reasoning level. Retrieve at least one proposition-related knowledge point that matches the knowledge reasoning level, applicable task type, and task difficulty range. Based on at least one proposition-related knowledge point and the current task difficulty, adjust the previous reasoning task to the current reasoning task.

[0175] Optionally, the inference capability parameters after the previous round of parameter tuning include multiple inference parameter items, each of which describes a performance metric of the task processing model when processing the previous round of inference task; the task generation module 501 is used for: Each inference parameter is normalized to obtain the preprocessed parameter for each inference parameter. The reasoning ability of the task processing model is evaluated based on each preprocessing parameter and its preset evaluation weight. Determine the reasoning ability assessment value within the reasoning ability assessment range, and obtain the knowledge reasoning level, task difficulty range, and applicable task type associated with the reasoning ability assessment range.

[0176] Optionally, the task generation module 501 is used for: Based on the difficulty of the current round of tasks, determine the content of the tasks to be modified in the previous round of reasoning tasks; Based on at least one proposition-related knowledge point and the set of logical reasoning relationships associated with each of the at least one proposition-related knowledge point, the text of the task to be modified is modified to generate the current round of reasoning task.

[0177] Optionally, the task evaluation module 502 is used for: Send the current round of reasoning task to the task processing model and extract at least one key problem-solving information from the current round of reasoning task; For each key problem-solving information, at least one historical knowledge point associated with the key problem-solving information in the knowledge point retrieval set is recorded as a reference problem-solving knowledge point; when no associated historical knowledge point is found in the knowledge point retrieval set, the key problem-solving information is converted into a newly added reference problem-solving knowledge point; the knowledge point retrieval set consists of at least one of the historical knowledge points learned by the task processing model and the historical knowledge points included in the knowledge graph. Based on the recorded reference problem-solving knowledge points and the set of logical reasoning relationships associated with each reference problem-solving knowledge point, multiple problem-solving reasoning paths are obtained; For each problem-solving reasoning path, the problem-solving derivation steps are executed sequentially for at least one key problem-solving information according to the problem-solving reasoning path to obtain the reasoned answer.

[0178] Optionally, after obtaining multiple problem-solving reasoning paths, before sending the next round of reasoning tasks to the task processing model for reasoning, the training device 500 of the task processing model also includes a graph update module 504, which is used for: For each problem-solving reasoning path, when the problem-solving reasoning path contains newly added reference problem-solving knowledge points, the knowledge graph is updated based on the newly added reference problem-solving knowledge points and the set of logical reasoning relationships associated with the newly added reference problem-solving knowledge points in the problem-solving reasoning path.

[0179] Optionally, after updating the knowledge graph, the graph update module 504 is also used for: Starting from the newly added reference problem-solving knowledge points, the system traverses along the edges that the newly added reference problem-solving knowledge points are associated with in the knowledge graph, and extracts the set of logical reasoning relationships of the newly added reference problem-solving knowledge points. The set of logical reasoning relations is validated by logical rules. When it is determined that there is at least one logical reasoning relation that conflicts with the logical rules, the knowledge graph is corrected a second time.

[0180] Optionally, after updating the knowledge graph, the graph update module 504 is also used for: When the reasoning reward value is not lower than the preset reasoning reward value threshold, in the knowledge graph, multiple problem-solving reasoning paths are marked as important reasoning paths, and each reference problem-solving knowledge point involved in multiple problem-solving reasoning paths and the edge associated with each reference problem-solving knowledge point are marked as important knowledge elements. When the reasoning reward value is lower than the reasoning reward value threshold, multiple problem-solving reasoning paths are marked as reasoning paths to be verified in the knowledge graph, and each reference problem-solving knowledge point involved in the multiple problem-solving reasoning paths and the edge associated with each reference problem-solving knowledge point are marked as knowledge elements to be verified.

[0181] Optionally, when the knowledge point retrieval set contains all historical knowledge points already learned by the task processing model, the task evaluation module 502 uses the following method when retrieving key problem-solving information: Extract semantic features from key problem-solving information; Semantic matching is performed between the semantic features of the question stem and the semantic features of each historical knowledge point already learned in the task processing model. Each historical knowledge point whose semantic feature similarity exceeds the preset feature similarity threshold is recorded as a reference problem-solving knowledge point.

[0182] Optionally, when the knowledge point retrieval set contains historical knowledge points already included in the knowledge graph, the task evaluation module 502 uses the following method when retrieving key problem-solving information: The key problem-solving information is matched with the knowledge graph to obtain at least one candidate knowledge point that is successfully matched. For each candidate knowledge point, starting from the candidate knowledge point, traverse along the edges associated with the candidate knowledge point in the knowledge graph, and record each historical knowledge point encountered as a reference solution knowledge point.

[0183] Optionally, the task evaluation module 502 is used for: From multiple inference answers, one inference answer is selected as the reference answer. Using the task generation model, the response quality of each of the remaining inference answers is evaluated based on the reference answer, and the response quality evaluation value of each other inference answer is obtained. The task generation model obtains the actual quality assessment value based on the quality assessment values ​​of each response and the total number of multiple inference answers. The actual quality assessment value is then used to provide feedback on gains and losses, resulting in an inference reward value.

[0184] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0185] Having introduced the training method and apparatus for the task processing model according to exemplary embodiments of this application, we will now introduce a computer device according to another exemplary embodiment of this application.

[0186] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0187] Based on the same inventive concept as the above-described method embodiments, this application also provides a computer device. In one embodiment, the computer device may be a server, such as... Figure 2 The server 230 is shown. In this embodiment, the computer device 600 has the following structure: Figure 6 As shown, it may include at least a memory 601, a communication module 603, and at least one processor 602.

[0188] The memory 601 is used to store computer programs executed by the processor 602. The memory 601 may mainly include a program storage area and a data storage area. The program storage area may store the operating system and programs required to run instant messaging functions, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.

[0189] Memory 601 may be volatile memory, such as random-access memory (RAM); memory 601 may also be non-volatile memory, such as read-only memory, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or memory 601 may be any other medium capable of carrying or storing a desired computer program having the form of instructions or data structures and accessible by a computer, but is not limited thereto. Memory 601 may be a combination of the above-described memories.

[0190] Processor 602 may include one or more central processing units (CPUs) or digital processing units, etc. Processor 602 is used to implement the training method of the above-mentioned task processing model when calling the computer program stored in memory 601.

[0191] The communication module 603 is used to communicate with terminal devices and other servers.

[0192] This application embodiment does not limit the specific connection medium between the memory 601, communication module 603, and processor 602 described above. This application embodiment... Figure 6 The memory 601 and the processor 602 are connected via a bus 604, and the bus 604 is in Figure 6 The diagram uses thick lines to describe the connections between other components; these are for illustrative purposes only and should not be considered limiting. The 604 bus can be divided into address bus, data bus, control bus, etc. For ease of description, Figure 6 It is described using only a thick line, but does not indicate that there is only one bus or one type of bus.

[0193] The memory 601 stores a computer storage medium, which stores computer-executable instructions. These instructions are used to implement the training method of the task processing model according to embodiments of this application. The processor 602 is used to execute the aforementioned training method of the task processing model, such as... Figure 3A As shown.

[0194] In another embodiment, the computer device can also be other computer devices, such as... Figure 2 The terminal device 210 is shown. In this embodiment, the structure of the computer device can be as follows: Figure 7As shown, it includes components such as a communication component 710, a memory 720, a display unit 730, a camera 740, a sensor 750, an audio circuit 760, a Bluetooth module 770, and a processor 780.

[0195] The communication component 710 is used to communicate with the server. In some embodiments, it may include a Wireless Fidelity (WiFi) module, which is a short-range wireless transmission technology. Electronic devices can use the WiFi module to help objects send and receive information.

[0196] The memory 720 can be used to store software programs and data. The processor 780 executes various functions of the terminal device 210 and performs data processing by running the software programs or data stored in the memory 720. The memory 720 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The memory 720 stores an operating system that enables the terminal device 210 to run. In this application, the memory 720 may store the operating system and various application programs, and may also store computer programs that execute the training method of the task processing model of the embodiments of this application.

[0197] The display unit 730 can also be used to display information input by the object or information provided to the object, as well as various menus of the terminal device 210, in a graphical user interface (GUI). Specifically, the display unit 730 may include a display screen 732 disposed on the front of the terminal device 210. The display screen 732 may be configured as a liquid crystal display, a light-emitting diode, or the like. The display unit 730 can be used to display the question-and-answer interactive interface, model training interface, etc., as described in the embodiments of this application.

[0198] The display unit 730 can also be used to receive input digital or character information and generate signal inputs related to object settings and function control of the terminal device 210. Specifically, the display unit 730 may include a touch screen 731 disposed on the front of the terminal device 210, which can collect touch operations on or near the object, such as clicking a button, dragging a scroll bar, etc.

[0199] The touchscreen 731 can be placed on top of the display screen 732, or the touchscreen 731 and the display screen 732 can be integrated to realize the input and output functions of the terminal device 210. After integration, it can be referred to as a touch display screen. In this application, the display unit 730 can display the application and the corresponding operation steps.

[0200] Camera 740 can be used to capture still images, and objects can publish images captured by camera 740 through an application. There can be one or multiple cameras 740. An object generates an optical image through a lens, which is projected onto a photosensitive element. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then transmitted to processor 780 to be converted into a digital image signal.

[0201] The terminal device may also include at least one sensor 750, such as an accelerometer 751, a proximity sensor 752, a fingerprint sensor 753, and a temperature sensor 754. The terminal device may also be equipped with other sensors such as a gyroscope, barometer, hygrometer, thermometer, infrared sensor, light sensor, and motion sensor.

[0202] Audio circuitry 760, speaker 761, and microphone 762 provide an audio interface between the device and terminal device 210. Audio circuitry 760 converts received audio data into electrical signals, which are then transmitted to speaker 761, where they are converted into sound signals for output. Terminal device 210 may also be equipped with volume buttons for adjusting the volume of the sound signal. On the other hand, microphone 762 converts collected sound signals into electrical signals, which are received by audio circuitry 760, converted into audio data, and then output to communication component 710 for transmission to, for example, another terminal device 210, or to memory 720 for further processing.

[0203] The Bluetooth module 770 is used to interact with other Bluetooth devices that also have a Bluetooth module via the Bluetooth protocol. For example, a terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smartwatch) that also has a Bluetooth module through the Bluetooth module 770, thereby exchanging data.

[0204] The processor 780 is the control center of the terminal device, connecting various parts of the terminal through various interfaces and lines. It executes various functions and processes data by running or executing software programs stored in the memory 720 and calling data stored in the memory 720. In some embodiments, the processor 780 may include one or more processing units; the processor 780 may also integrate an application processor and a baseband processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the baseband processor mainly handles wireless communication. It is understood that the baseband processor may not be integrated into the processor 780. In this application, the processor 780 can run the operating system, applications, user interface display and touch response, and the task processing model training method of this application embodiment. Furthermore, the processor 780 is coupled to the display unit 730.

[0205] In some possible implementations, various aspects of the task processing model training method provided in this application can also be implemented in the form of a program product, which includes a computer program. When the program product is run on a computer device, the computer program causes the computer device to perform the steps in the task processing model training method according to the various exemplary embodiments of this application described above. For example, the computer device can perform actions such as... Figure 3A The steps are shown in the figure.

[0206] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0207] The program product of the embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include a computer program, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium that contains or stores a program that may be used by or in conjunction with a command execution system, apparatus, or device.

[0208] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a readable computer program. This propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting a program for use by or in conjunction with a command execution system, apparatus, or device.

[0209] Computer programs contained on readable media may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0210] Computer programs for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The computer program can execute entirely on the user's computer device, partially on the user's computer device, as a standalone software package, partially on the user's computer device and partially on a remote computer device, or entirely on a remote computer device. In cases involving remote computer devices, the remote computer device can be connected to the user's computer device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer device (e.g., via the Internet using an Internet service provider).

[0211] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0212] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0213] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing a computer-usable computer program.

[0214] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0215] These computer program commands may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the commands stored in the computer-readable storage medium produce an article of manufacture including command means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0216] These computer program commands may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the commands executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0217] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0218] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A training method for a task processing model, characterized in that, include: The difficulty of the current round of tasks is determined by adjusting the difficulty of the previous round of tasks through the task generation model; Based on the reasoning ability parameters after the previous round of parameter tuning, the knowledge reasoning level, applicable task types, and task difficulty range of the task processing model are determined; wherein, the lower limit of the task difficulty range is higher than the expected task difficulty corresponding to the knowledge reasoning level. Retrieve at least one proposition-related knowledge point that matches the knowledge reasoning level, the applicable task type, and the task difficulty range, and adjust the previous round reasoning task to the current round reasoning task based on at least one proposition-related knowledge point and the current round task difficulty. The current round of reasoning task is sent to the task processing model for reasoning to obtain multiple reasoning answers for the current round of reasoning task. The task generation model is then used to evaluate the quality of the multiple reasoning answers to obtain the actual quality evaluation value and reasoning reward value for the current round of reasoning task. The parameters of the task processing model and the task generation model are updated based on the actual quality assessment value and the inference reward value. The current round of task difficulty is used as the new previous round of task difficulty, and the current round of reasoning task is used as the new previous round of reasoning task, and iterative training is performed.

2. The method of claim 1, wherein, The step of adjusting the difficulty of the previous round of tasks and determining the difficulty of the current round of tasks through the task generation model includes: Using the task generation model, the difficulty of the previous round of tasks is adjusted based on the quality difference between the historical quality assessment value and the expected quality assessment value of the previous round of inference tasks, thereby determining the difficulty of the current round of tasks.

3. The method of claim 2, wherein, The step of adjusting the difficulty of the previous task based on the quality difference between the historical quality assessment value and the expected quality assessment value of the previous inference task using the task generation model, and determining the difficulty of the current task, includes: When the quality difference of the previous inference task indicates that the historical quality assessment value is higher than the expected quality assessment value, a task difficulty increase is generated based on the quality difference and a preset difficulty adjustment coefficient, and the difficulty of the previous task is adjusted upward based on the task difficulty increase to determine the difficulty of the current task. When the quality difference indicates that the historical quality assessment value is lower than the expected quality assessment value, a task difficulty reduction amount is generated based on the quality difference and the difficulty adjustment coefficient, and the difficulty of the previous round of tasks is adjusted down based on the task difficulty reduction amount to determine the difficulty of the current round of tasks. When the quality difference indicates that the historical quality assessment value is equal to the expected quality assessment value, the difficulty of the previous round of tasks is adjusted to zero, and the difficulty of the previous round of tasks is used as the difficulty of the current round of tasks.

4. The method of claim 1, wherein, The inference capability parameters after the previous round of parameter tuning include multiple inference parameter items, each of which is used to describe a performance index of the task processing model when processing the previous round of inference task; The process of determining the knowledge reasoning level, applicable task types, and task difficulty range of the task processing model based on the reasoning ability parameters after the previous round of parameter tuning includes: Each of the inference parameter terms is normalized to obtain a preprocessed parameter term for each inference parameter term; The reasoning ability is evaluated based on each of the preprocessing parameter items and their respective preset evaluation weights to obtain the reasoning ability evaluation value of the task processing model. Determine the reasoning ability assessment range in which the reasoning ability assessment value falls, and obtain the knowledge reasoning level, the task difficulty range, and the applicable task type associated with the reasoning ability assessment range.

5. The method of claim 1, wherein, The step of adjusting the previous round of reasoning task to the current round of reasoning task based on at least one of the proposition-related knowledge points and in conjunction with the difficulty of the current round of task includes: Based on the difficulty of the current round of tasks, determine the content of the tasks to be modified in the previous round of reasoning tasks; Based on at least one of the proposition-related knowledge points and the set of logical reasoning relationships associated with each of the proposition-related knowledge points, the text of the task to be modified is modified to generate the current round of reasoning task.

6. The method of claim 1, wherein, The step of sending the current round of reasoning task to the task processing model for reasoning to obtain multiple reasoning answers for the current round of reasoning task includes: The current round of reasoning task is sent to the task processing model, and at least one key problem-solving information in the current round of reasoning task is extracted; For each key problem-solving information, at least one historical knowledge point associated with the key problem-solving information in the knowledge point retrieval set is recorded as a reference problem-solving knowledge point; when no associated historical knowledge point is found in the knowledge point retrieval set, the key problem-solving information is converted into a newly added reference problem-solving knowledge point; the knowledge point retrieval set is composed of at least one of the historical knowledge points already learned by the task processing model and the historical knowledge points already included in the knowledge graph; Based on the recorded reference problem-solving knowledge points and the set of logical reasoning relationships associated with each reference problem-solving knowledge point, multiple problem-solving reasoning paths are obtained; For each of the problem-solving reasoning paths, problem-solving derivation steps are sequentially executed on at least one of the key problem-solving information according to the problem-solving reasoning path to obtain the reasoning answer.

7. The method of claim 6, wherein, After obtaining multiple problem-solving reasoning paths, and before sending the next round of reasoning tasks to the task processing model for reasoning, the process also includes: For each of the problem-solving reasoning paths, when the problem-solving reasoning path contains newly added reference problem-solving knowledge points, the knowledge graph is updated based on the newly added reference problem-solving knowledge points and the set of logical reasoning relationships associated with the newly added reference problem-solving knowledge points in the problem-solving reasoning path.

8. The method of claim 7, wherein, After updating the knowledge graph, the following is also included: Starting from the newly added reference problem-solving knowledge points, the system traverses along the edges associated with the newly added reference problem-solving knowledge points in the knowledge graph to extract the set of logical reasoning relationships of the newly added reference problem-solving knowledge points. The set of logical reasoning relations is validated by logical rules. If at least one logical reasoning relation is found to conflict with a logical rule, the knowledge graph is then corrected a second time.

9. The method of claim 7, wherein, After updating the knowledge graph, the following is also included: When the reasoning reward value is not lower than the preset reasoning reward value threshold, in the knowledge graph, multiple problem-solving reasoning paths are marked as important reasoning paths, and the reference problem-solving knowledge points involved in the multiple problem-solving reasoning paths and the edges associated with each reference problem-solving knowledge point are marked as important knowledge elements. When the reasoning reward value is lower than the reasoning reward value threshold, in the knowledge graph, multiple problem-solving reasoning paths are marked as reasoning paths to be verified, and the reference problem-solving knowledge points involved in the multiple problem-solving reasoning paths and the edges associated with each reference problem-solving knowledge point are marked as knowledge elements to be verified.

10. The method of claim 6, wherein, When the knowledge point retrieval set contains all the historical knowledge points already learned by the task processing model, the following method is used when retrieving the key problem-solving information: Extract the semantic features of the question stem from the key problem-solving information; Semantic matching is performed between the semantic features of the question stem and the semantic features of each historical knowledge point already learned in the task processing model. Each historical knowledge point whose semantic feature similarity exceeds a preset feature similarity threshold is recorded as the reference problem-solving knowledge point.

11. The method as described in claim 6, characterized in that, When the knowledge point retrieval set contains historical knowledge points already included in the knowledge graph, the following method is used when retrieving the key problem-solving information: The key problem-solving information is matched with the knowledge graph to obtain at least one candidate knowledge point that is successfully matched. For each candidate knowledge point, starting from the candidate knowledge point, traverse along the edges associated with the candidate knowledge point in the knowledge graph, and record each historical knowledge point encountered as the reference solution knowledge point.

12. The method as described in claim 1, characterized in that, The step of evaluating the quality of the multiple reasoning answers using the task generation model to obtain the actual quality evaluation value and reasoning reward value for the current round of reasoning task includes: From the multiple reasoning answers, one reasoning answer is selected as a reference answer. Using the task generation model, the response quality of the remaining reasoning answers is evaluated based on the reference answer, and a response quality evaluation value is obtained for each of the other reasoning answers. The task generation model obtains the actual quality assessment value based on the quality assessment values ​​of each response and the total number of the multiple inference answers. The inference reward value is obtained by performing a profit and loss feedback based on the actual quality assessment value.

13. A training device for a task processing model, characterized in that, include: The task generation module is used to adjust the difficulty of the previous round of tasks and determine the difficulty of the current round of tasks through the task generation model. Based on the reasoning ability parameters after the previous round of parameter tuning, the knowledge reasoning level, applicable task types, and task difficulty range of the task processing model are determined; wherein, the lower limit of the task difficulty range is higher than the expected task difficulty corresponding to the knowledge reasoning level. Retrieve at least one proposition-related knowledge point that matches the knowledge reasoning level, the applicable task type, and the task difficulty range, and adjust the previous round reasoning task to the current round reasoning task based on at least one proposition-related knowledge point and the current round task difficulty. The task evaluation module is used to send the current round of reasoning task to the task processing model for reasoning, obtain multiple reasoning answers for the current round of reasoning task, and evaluate the quality of the multiple reasoning answers through the task generation model to obtain the actual quality evaluation value and reasoning reward value of the current round of reasoning task. The model training module is used to update the parameters of the task processing model and the task generation model based on the actual quality assessment value and the inference reward value. The current round of task difficulty is used as the new previous round of task difficulty, and the current round of reasoning task is used as the new previous round of reasoning task, and iterative training is performed.

14. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, It stores a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method according to any one of claims 1-12.

16. A computer program product, characterized in that, The computer program product includes a computer program stored on a computer-readable storage medium, the computer program including program instructions that, when executed by a computer device, cause the computer device to perform the steps of the method according to any one of claims 1-12.

Images