Task execution method and device based on model scheduling and storage medium

Abstract

The invention relates to a task execution method and device based on model scheduling and a storage medium, and the method comprises the steps: training a task scheduling strategy model, and obtaining a trained task scheduling strategy model; after a task is received, determining a task type of the task through the trained task scheduling strategy model; selecting a calling tool or a large language model; when the tool is called, obtaining an execution record of the task; when the large language model is called, the large language model decomposes the task to obtain a plurality of sub-tasks, and tools corresponding to the sub-tasks are called to obtain execution records of the task; and performing online reinforcement learning updating on the trained task scheduling strategy model according to an execution record of the task. According to the method, a task scheduling strategy model can be utilized, a tool or a large language model can be reasonably scheduled, and the task scheduling strategy model is optimized online according to task execution records; and the task accuracy and the balance of the calculation cost are improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a task execution method, apparatus and storage medium based on model scheduling. Background Technology

[0002] With the development of artificial intelligence technology, large-scale pre-trained language models have achieved remarkable results in the field of natural language processing. These large models possess powerful generalization, understanding, and reasoning capabilities, enabling them to handle complex tasks across various open domains. However, directly applying a single large model to real-world scenarios also faces significant limitations: on the one hand, large models incur enormous computational overhead for inference, making it difficult to meet the demands of high concurrency and low latency applications; on the other hand, large models lack expertise in specific domain details and specialized knowledge, making it difficult to handle problems in specialized domains in a timely and efficient manner.

[0003] Existing technologies employ a method of using a large model to plan the workflow and a small model to execute standardized subtasks. The accuracy of this approach depends on the large model; any deviation in the workflow planning by the large model will affect the execution of the small models, thus impacting the overall task accuracy. Furthermore, the large model has a high computational cost, resulting in a poor balance between task accuracy and computational cost.

[0004] There is currently no effective solution to the problem of poor balance between task accuracy and computational cost in related technologies. Summary of the Invention

[0005] This embodiment provides a task execution method, apparatus, and storage medium based on model scheduling to address the problem of poor balance between task accuracy and computational cost in related technologies.

[0006] Firstly, this embodiment provides a task execution method based on model scheduling, including:

[0007] The task scheduling strategy model is trained to obtain the trained task scheduling strategy model;

[0008] Upon receiving a task, the task type is determined using the trained task scheduling strategy model; based on the task type, either a calling tool or a large language model is selected.

[0009] When the tool is invoked, the tool executes the task and obtains the execution record of the task;

[0010] When the large language model is invoked, the large language model decomposes the task into several sub-tasks. Through the trained task scheduling strategy model, the tool corresponding to the sub-task is invoked, and the tool executes the corresponding sub-task to obtain the execution record of the task.

[0011] Based on the execution record of the task, a reward signal is calculated; based on the reward signal, the trained task scheduling strategy model is updated through online reinforcement learning.

[0012] In some embodiments, the task scheduling strategy model is trained to obtain a trained task scheduling strategy model, including:

[0013] The task scheduling strategy model is initialized and trained using historical task data, expert strategy samples, or simulated tasks to obtain an initial task scheduling strategy model.

[0014] In the constructed task simulation environment, the initial task scheduling strategy model is trained using a reinforcement learning algorithm to obtain the trained task scheduling strategy model.

[0015] In some embodiments, upon receiving a task, the task type is determined using the trained task scheduling strategy model; based on the task type, a tool or a large language model is selected for invocation, including:

[0016] Upon receiving a task, the trained task scheduling strategy model is used to perform semantic understanding and task structure analysis on the task to determine the task type; based on the task type, a calling tool or a large language model is selected.

[0017] In some embodiments, when the tool is invoked, the tool executes the task and obtains an execution record of the task, including:

[0018] When the tool is invoked, the tool executes the task and the execution result of the tool is obtained;

[0019] Based on the execution result of the tool, determine whether to call a new tool; if no new tool is called, obtain the execution record of the task.

[0020] In some embodiments, when the large language model is invoked, the large language model decomposes the task into several sub-tasks. Using the trained task scheduling strategy model, the tool corresponding to each sub-task is invoked, and the tool executes the corresponding sub-task, resulting in an execution record of the task, including:

[0021] When the large language model is invoked, the large language model decomposes the task into several subtasks. Through the trained task scheduling strategy model, the tool corresponding to the subtask is invoked, and the tool executes the corresponding subtask to obtain the execution record of the subtask. Based on the execution record of the subtask, the execution record of the task is obtained.

[0022] Secondly, this embodiment provides a task execution device based on model scheduling, including a task scheduling strategy model construction module, a task execution module, and a task scheduling strategy model optimization module, wherein:

[0023] The task scheduling strategy model construction module is used to train the task scheduling strategy model to obtain the trained task scheduling strategy model.

[0024] The task execution module is used to, upon receiving a task, determine the task type using the trained task scheduling strategy model; select either a tool or a large language model to invoke based on the task type; when the tool is invoked, the tool executes the task and obtains the task's execution record; when the large language model is invoked, the large language model decomposes the task into several subtasks, and invokes the corresponding tool for each subtask using the trained task scheduling strategy model, the tool executes the corresponding subtask, and obtains the task's execution record.

[0025] The task scheduling strategy model optimization module is used to calculate a reward signal based on the task execution record; and to update the trained task scheduling strategy model through online reinforcement learning based on the reward signal.

[0026] In some embodiments, the task scheduling strategy model building module includes an initialization training module and an advanced training module, wherein:

[0027] The initialization training module is used to perform initialization training on the task scheduling strategy model using historical task data, expert strategy samples, or simulation tasks to obtain an initial task scheduling strategy model.

[0028] The advanced training module is used to train the initial task scheduling strategy model in the constructed task simulation environment using a reinforcement learning algorithm, so as to obtain the trained task scheduling strategy model.

[0029] Thirdly, this embodiment provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the model-based scheduling task execution method described in any of the first aspects above.

[0030] Fourthly, this embodiment provides an electronic device including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the model-based scheduling task execution method described in the first aspect above.

[0031] Fifthly, this embodiment provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the model-based scheduling task execution method described in the first aspect above.

[0032] Compared with related technologies, the model-based task execution method provided in this embodiment trains a task scheduling strategy model to obtain a trained task scheduling strategy model. Upon receiving a task, the trained task scheduling strategy model determines the task type. Based on the task type, it selects to invoke either a tool or a large language model. When an tool is invoked, it executes the task, obtaining the task execution record. When a large language model is invoked, it decomposes the task into several subtasks. Using the trained task scheduling strategy model, it invokes the corresponding tool for each subtask, which executes the corresponding subtask, obtaining the task execution record. A reward signal is calculated based on the task execution record. Based on the reward signal, the trained task scheduling strategy model is updated through online reinforcement learning. This method can utilize the task scheduling strategy model to rationally schedule tools or large language models and optimize the task scheduling strategy model online based on task execution records, thus improving the balance between task accuracy and computational cost.

[0033] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0034] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0035] Figure 1 This is a hardware structure block diagram of the terminal for the task execution method based on model scheduling in this embodiment;

[0036] Figure 2 This is a flowchart of the task execution method based on model scheduling in this embodiment;

[0037] Figure 3 This is a flowchart of the training method for the task scheduling strategy model in this embodiment;

[0038] Figure 4 This is a structural block diagram of the model-based scheduling task execution device in this embodiment;

[0039] Figure 5 This is a structural block diagram of the task scheduling strategy model construction module in this embodiment. Detailed Implementation

[0040] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0041] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0042] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the task execution method based on model scheduling in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0043] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the model-based scheduling task execution method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thus implementing the aforementioned method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0044] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0045] This embodiment provides a task execution method based on model scheduling. Figure 2 This is a flowchart of the task execution method based on model scheduling in this embodiment, as follows: Figure 2 As shown, the process includes the following steps:

[0046] Step S201: Train the task scheduling strategy model to obtain the trained task scheduling strategy model.

[0047] This task scheduling strategy model can be a lightweight language model with approximately 1 billion to 3 billion parameters. Examples include Qwen-1.5B and LLaMA-1B. This model is responsible for understanding the user task, decomposing the task, and selecting the optimal tool. The tool can be a lightweight machine learning model, a large language model, an application programming interface (API), or a database. The lightweight machine learning model can be a support vector machine (SVM) or a decision tree. This task scheduling strategy model possesses natural language understanding and generation capabilities. Its input can be the current user task text, historical tool call trajectories, and functional descriptions of each tool. Its output can be structured action decision instructions, such as tool call formats, tool selection markers, or direct response text. For example, the input of this task scheduling strategy model at time t can be represented as:

[0048] ;

[0049] in This represents the original task text submitted by the current user, which can be a question or requirement description in natural language. It represents the historical tool call trajectory, that is, the record of tool calls that the system has made in the current task context. The call record includes information such as the type of tool called, the order of calls, and input / output. This represents a candidate toolset used for matching and selection by task scheduling strategy models. This represents system status information, which may include system resource load conditions, such as the current call latency, response time, or cost estimates of each tool. This system status information guides the task scheduling strategy model in balancing scheduling efficiency and cost. The task scheduling strategy model takes as input... The output at that time can be represented as:

[0050] ;

[0051] in Indicates a certain number is This tool can be used for lightweight machine learning models, large language models, application programming interfaces (APIs), or databases.

[0052] Specifically, the task scheduling strategy model is trained to obtain the trained task scheduling strategy model. This process consists of two stages: an initialization training stage and a high-order training stage. The initialization training stage utilizes historical task data, expert policy samples, or simulated tasks to guide the task scheduling strategy model to learn existing optimal scheduling behaviors through behavioral cloning or imitation learning, thereby acquiring preliminary task allocation capabilities—this is the initial task scheduling strategy model. The high-order training stage employs reinforcement learning algorithms, utilizing the current task state, historical decision trajectories, and tool call paths to further train the initial task scheduling strategy model, resulting in the trained task scheduling strategy model. This trained task scheduling strategy model is capable of accurately scheduling complex tasks.

[0053] Step S202: After receiving the task, determine the task type using the trained task scheduling strategy model; and select the calling tool or large language model based on the task type.

[0054] Specifically, a unified collaborative task environment is first constructed. Using a proxy orchestration framework such as LangChain, a large language model and multiple tools are integrated into the same environment, registered in a unified scheduling interface for the task scheduling strategy model to call. Each tool has a natural language or structured capability description, such as function labels, applicable input types, and resource consumption estimates, which the task scheduling strategy model uses to understand its applicable scenarios and functional scope. This collaborative task environment allows for flexible invocation of the large language model and multiple tools. Upon receiving a task, the trained task scheduling strategy model performs semantic understanding and task structure analysis to determine the task type. For example, the task could be finding the cause of a loose cigarette pack seal. Based on the task type, either a tool or the large language model is selected for invocation. When a tool's function description matches the task's semantic analysis, the task type is considered simple, and the corresponding tool is invoked. When no tool's function description matches the task's semantic analysis, the task type is considered complex, and the large language model is invoked.

[0055] Step S203: When a tool is invoked, the tool executes the task and obtains the task execution record; when a large language model is invoked, the large language model decomposes the task into several sub-tasks, and through the trained task scheduling strategy model, invokes the tool corresponding to the sub-task, the tool executes the corresponding sub-task, and obtains the task execution record.

[0056] Specifically, when a tool is invoked, it executes a task and obtains an execution record. This execution record includes the task invocation path and the final task execution result. The task invocation path includes status information, action information, and result information. The status information is the task context information at the current moment, including the original task text, completed tool invocation records, descriptions of each tool's capabilities, and the current system load. The action information is the output of the task scheduling strategy model, i.e., the scheduling behavior, which involves invoking a tool or a large language model, or choosing to directly answer the current task. The result information is the execution result obtained after invoking the tool or large language model to execute the task or sub-task, which can be natural language text, structured answers, search results, or simulation data. When the large language model is invoked, it decomposes the task. For example, if the task is to optimize CNC machining paths to save energy, the large language model decomposes it into four sub-tasks: equipment energy consumption model invocation, parameter combination generation, multi-round path simulation, and result sorting and filtering. The task scheduling strategy model invokes the corresponding tools for each sub-task based on the semantics and current context of the sub-tasks. Each tool executes its corresponding sub-task, obtaining the execution record for each sub-task. The execution record of the task is obtained based on the execution records of each subtask.

[0057] Step S204: Calculate the reward signal based on the task execution record; update the trained task scheduling strategy model through online reinforcement learning based on the reward signal.

[0058] Specifically, a reward signal is calculated based on the task execution record. This reward signal includes a task correctness reward and a task invocation efficiency reward. Based on this reward signal, the trained task scheduling strategy model is updated through online reinforcement learning. For example, in a "die-casting equipment thermal management optimization" task, the task scheduling strategy model achieves lower cost and more accurate control by invoking a strategy path combining thermal balance simulation and approximate modeling. This strategy path receives a high reward signal and is reinforced by the task scheduling strategy model; in subsequent similar tasks, this path will be prioritized, resulting in adaptive evolution.

[0059] In this embodiment, a trained task scheduling strategy model is obtained by training the model. Upon receiving a task, the task type is determined using the trained model. Based on the task type, either a tool or a large language model is selected for invocation. When a tool is invoked, it executes the task, generating an execution record. When a large language model is invoked, it decomposes the task into several subtasks. Using the trained model, the corresponding tool for each subtask is invoked, and the tool executes the subtask, generating an execution record. A reward signal is calculated based on the execution record. The trained task scheduling strategy model is then updated using online reinforcement learning based on the reward signal. This approach leverages the task scheduling strategy model to rationally schedule tools or a large language model and optimizes the model online based on task execution records, improving the balance between task accuracy and computational cost.

[0060] In some of these embodiments, Figure 3 This is a flowchart of the training method for the task scheduling strategy model in this embodiment, as follows: Figure 3 As shown, the process includes the following steps:

[0061] Step S301: Initialize and train the task scheduling strategy model using historical task data, expert strategy samples, or simulation tasks to obtain the initial task scheduling strategy model.

[0062] Specifically, by utilizing historical task data, expert policy samples, or simulated tasks, and through behavioral cloning or imitation learning, the task scheduling strategy model is guided to learn existing optimal scheduling behaviors, thereby acquiring preliminary task allocation capabilities—that is, the initial task scheduling strategy model. This process can also employ random initialization combined with offline reinforcement learning algorithms to explore and generate basic policies in a simulation environment, thus obtaining the initial task scheduling strategy model. This initial task scheduling strategy model also possesses the ability to initialize complex tasks, that is, to hand over complex tasks to a large language model for task decomposition.

[0063] Step S302: In the constructed task simulation environment, the initial task scheduling strategy model is trained using a reinforcement learning algorithm to obtain the trained task scheduling strategy model.

[0064] Specifically, in the constructed task simulation environment, the initial task scheduling strategy model is further trained using a reinforcement learning algorithm based on the execution records of historical tasks, resulting in a trained task scheduling strategy model. This reinforcement learning algorithm can be methods such as proximal policy optimization or group-relative policy optimization. The execution records of the historical tasks can be represented as follows:

[0065] ;

[0066] in This represents the state at step t, i.e., the task context information at the current moment, including the original task text, completed tool call records, descriptions of the capabilities of each tool, and the current system load; This represents the output of the task scheduling strategy model at step t, i.e., in state t. The scheduling behavior can be to invoke a tool or a large language model, or to choose to directly answer the current task. This represents the result of step t, which is the execution result obtained after calling the tool or large language model to perform the task or sub-task. It can be natural language text, structured answers, search results, or simulation data, etc.; t represents the total number of steps in the task execution process to call the tool or large language model. , , The combination of these three elements represents the task invocation path; y represents the final task execution result, i.e., the final output upon task completion. The combined reward function is designed as follows:

[0067] ;

[0068] in The reward for correct task execution is represented as:

[0069] ;

[0070] The task invocation efficiency reward is represented as:

[0071] ;

[0072] Where m represents the total number of calls and c represents the smoothing parameter. This combined reward function makes the task scheduling strategy model favor policies that provide correct answers and have fewer total calls. A proximate policy optimization algorithm is used for policy updates, and the pruning objective function is expressed as:

[0073] ;

[0074] in The strategy probability ratio measures the change in probability between the current strategy and the old strategy on the same call selection, and is expressed as:

[0075] ;

[0076] in This represents the state at step t, i.e., the task context information at the current moment, including the original task text, completed tool call records, descriptions of the capabilities of each tool, and the current system load; This represents the output of the task scheduling strategy model at step t, i.e., in state t. The scheduling behavior can be to invoke a tool or a large language model, or to choose to directly answer the current task. This represents the probability that the old policy will choose a certain scheduling behavior in a certain state; This represents the probability that the new policy will choose the same scheduling behavior under the same conditions. When this value is much greater than 1, it means that the new policy is more inclined to choose this scheduling behavior, and vice versa. The advantage function is used to evaluate the current scheduling behavior. A value greater than 0 indicates good scheduling behavior, while a value less than 0 indicates poor scheduling behavior. Indicates will Cut in Within the range, for example, setting It is 0.2. Restricted to Within the specified interval. This near-end policy optimization algorithm can prevent policy updates from being too rapid, thus ensuring a stable and reliable training process.

[0077] In some embodiments, after receiving a task, the task type is determined by a trained task scheduling strategy model; and a tool or large language model is selected for invocation based on the task type. This includes: after receiving a task, performing semantic understanding and task structure analysis on the task by a trained task scheduling strategy model to determine the task type; and selecting a tool or large language model for invocation based on the task type.

[0078] Specifically, upon receiving a task, the trained task scheduling strategy model performs semantic understanding and task structure analysis to determine the task type. Based on the task type, it selects either a tool or a large language model to invoke. When the task type is simple, the corresponding tool is selected based on its functional description; when the task type is complex, the large language model is invoked.

[0079] In some embodiments, when a tool is invoked, the tool executes a task and obtains a task execution record, including: when a tool is invoked, the tool executes a task and obtains the tool's execution result; based on the tool's execution result, determining whether to invoke a new tool; and when no new tool is invoked, obtaining the task execution record.

[0080] Specifically, when a tool is invoked, it executes a task and yields its execution result. The trained task scheduling strategy model then evaluates the tool's execution result to determine whether a new tool needs to be invoked. If no new tool is needed, the task is considered complete, and the task execution record is obtained. Otherwise, the model continues invoking the corresponding tool until no new tool is needed, at which point the task is considered complete, and the task execution record is obtained. For example, if the task is to find the cause of a poor seal, the task scheduling strategy model invokes a temperature deviation detector and a pressure rule checker. If these tools return results indicating "insufficient to explain the poor seal weld," the model, based on historical reward experience, determines that ordinary tools have limited recognition capabilities in this scenario. Therefore, it upgrades to invoke a higher-level tool, such as a heat seal texture analysis model, to identify defects in the seal line texture. This higher-level tool provides a high-confidence output stating that "temperature fluctuations in the right-side heating block cause uneven distribution of hot melt adhesive." The task scheduling strategy model then determines whether other tools need to be invoked. Since the tool's execution result is clear and the task structure is simple, the model chooses to directly terminate the decision chain and output a final diagnostic suggestion. This yields the task execution record.

[0081] In some embodiments, when a large language model is invoked, the large language model decomposes the task into several subtasks. A trained task scheduling strategy model is then used to invoke the corresponding tool for each subtask. The tool executes the corresponding subtask, and the execution record of the task is obtained. This includes: when the large language model is invoked, the large language model decomposes the task into several subtasks. A trained task scheduling strategy model is then used to invoke the corresponding tool for each subtask. The tool executes the corresponding subtask, and the execution record of the subtask is obtained. Based on the execution record of the subtask, the execution record of the task is obtained.

[0082] Specifically, when the large language model is invoked, it decomposes the task into several sub-tasks. For example, the task of optimizing CNC machining paths to save energy is decomposed by the large language model into four sub-tasks: invoking the equipment energy consumption model, generating parameter combinations, performing multi-round path simulations, and sorting and filtering the results. The task scheduling strategy model invokes the corresponding tools based on the sub-tasks. The tools execute the corresponding sub-tasks, obtaining the execution records of the sub-tasks. Based on the execution records of the sub-tasks, the execution record of the entire task can be obtained.

[0083] This embodiment also provides a task execution device based on model scheduling, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0084] Figure 4 This is a structural block diagram of the model-based scheduling task execution device in this embodiment, as shown below. Figure 4 As shown, the model-based scheduling task execution device 40 includes: a task scheduling strategy model construction module 401, a task execution module 402, and a task scheduling strategy model optimization module 403, wherein:

[0085] The task scheduling strategy model construction module 401 is used to train the task scheduling strategy model to obtain the trained task scheduling strategy model.

[0086] The task execution module 402 is used to determine the task type after receiving a task by using a trained task scheduling strategy model; select either to call a tool or a large language model based on the task type; when calling a tool, the tool executes the task and obtains the task execution record; when calling a large language model, the large language model decomposes the task into several subtasks, calls the corresponding tool for each subtask by using the trained task scheduling strategy model, and the tool executes the corresponding subtask and obtains the task execution record.

[0087] The task scheduling strategy model optimization module 403 is used to calculate the reward signal based on the task execution record; and to update the trained task scheduling strategy model through online reinforcement learning based on the reward signal.

[0088] Figure 5 This is a structural block diagram of the task scheduling strategy model construction module in this embodiment, as shown below. Figure 5As shown, the task scheduling strategy model construction module 401 includes an initialization training module 501 and an advanced training module 502. The initialization training module 501 is used to initialize and train the task scheduling strategy model using historical task data, expert strategy samples, or simulated tasks to obtain an initial task scheduling strategy model. The advanced training module 502 is used to train the initial task scheduling strategy model using a reinforcement learning algorithm in a constructed task simulation environment to obtain a trained task scheduling strategy model.

[0089] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0090] This embodiment also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described model-based task execution methods.

[0091] This embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program and the processor being configured to run the computer program to perform the steps in any of the above method embodiments.

[0092] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0093] Furthermore, in conjunction with the model-based scheduling task execution method provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the model-based scheduling task execution methods described in the above embodiments.

[0094] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0095] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0096] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0097] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0098] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A method for model-based scheduling of task execution, characterized in that, include: The task scheduling strategy model is trained to obtain the trained task scheduling strategy model; Upon receiving a task, the task type is determined using the trained task scheduling strategy model; based on the task type, either a calling tool or a large language model is selected. When the tool is invoked, the tool executes the task and obtains the execution record of the task; When the large language model is invoked, the large language model decomposes the task into several sub-tasks. Through the trained task scheduling strategy model, the tool corresponding to the sub-task is invoked, and the tool executes the corresponding sub-task to obtain the execution record of the task. Based on the execution record of the task, a reward signal is calculated; based on the reward signal, the trained task scheduling strategy model is updated through online reinforcement learning.

2. The model-based scheduled task execution method of claim 1, wherein, The task scheduling strategy model is trained to obtain the trained task scheduling strategy model, including: The task scheduling strategy model is initialized and trained using historical task data, expert strategy samples, or simulated tasks to obtain an initial task scheduling strategy model. In the constructed task simulation environment, the initial task scheduling strategy model is trained using a reinforcement learning algorithm to obtain the trained task scheduling strategy model.

3. The model-based scheduled task execution method of claim 1, wherein, Upon receiving a task, the task type is determined using the trained task scheduling strategy model. Based on the task type, select the appropriate tool or large language model to invoke, including: Upon receiving a task, the trained task scheduling strategy model is used to perform semantic understanding and task structure analysis on the task to determine the task type; based on the task type, a calling tool or a large language model is selected.

4. The model-based schedule-driven task execution method of claim 1, wherein, When the tool is invoked, it executes the task and obtains the task execution record, including: When the tool is invoked, the tool executes the task and the execution result of the tool is obtained; Based on the execution result of the tool, determine whether to call a new tool; if no new tool is called, obtain the execution record of the task.

5. The model-based scheduled task execution method of claim 1, wherein, When the large language model is invoked, it decomposes the task into several sub-tasks. Using the trained task scheduling strategy model, the tool corresponding to each sub-task is invoked, and the tool executes the corresponding sub-task, resulting in an execution record of the task, including: When the large language model is invoked, the large language model decomposes the task into several subtasks. Through the trained task scheduling strategy model, the tool corresponding to the subtask is invoked, and the tool executes the corresponding subtask to obtain the execution record of the subtask. Based on the execution record of the subtask, the execution record of the task is obtained.

6. A model-based scheduling task execution apparatus, characterized by comprising: It includes a task scheduling strategy model construction module, a task execution module, and a task scheduling strategy model optimization module, among which: The task scheduling strategy model construction module is used to train the task scheduling strategy model to obtain the trained task scheduling strategy model. The task execution module is configured to, after receiving a task, determine a task type of the task by using the trained task scheduling strategy model; select a tool or a large language model according to the task type; when the tool is called, the tool executes the task to obtain an execution record of the task; when the large language model is called, the large language model decomposes the task to obtain a plurality of sub-tasks, and calls a tool corresponding to the sub-tasks by using the trained task scheduling strategy model, so that the tool executes the corresponding sub-tasks to obtain the execution record of the task. The task scheduling strategy model optimization module is configured to calculate a reward signal according to the execution record of the task, and perform online reinforcement learning update on the trained task scheduling strategy model according to the reward signal.

7. The model-based schedule-driven task execution apparatus according to claim 6, wherein, The task scheduling strategy model construction module includes an initialization training module and an advanced training module. The initialization training module is configured to perform initialization training on the task scheduling strategy model by using historical task data, expert strategy samples or simulation tasks to obtain an initial task scheduling strategy model. The advanced training module is configured to train the initial task scheduling strategy model by using a reinforcement learning algorithm in a constructed task simulation environment to obtain a trained task scheduling strategy model.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the model-based scheduling task execution method in any one of claims 1 to 5. 9.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores the computer program, and the processor is configured to run the computer program to execute the model-based scheduling task execution method in any one of claims 1 to 5.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the model-based scheduling task execution method in any one of claims 1 to 5.

Images