Task scheduling method and device, computer equipment, storage medium and program product

Abstract

The invention relates to a task scheduling method and device, computer equipment, a storage medium and a program product. The method comprises the steps of determining priorities of tasks to be executed in at least two task queues according to a preset time period, and updating the at least two task queues according to the priorities; according to the updated task sequence of the at least two task queues, the resource demand of each to-be-executed task and the resource specification of each node in the node pool corresponding to the at least two task queues, allocating each to-be-executed task to a matched node for processing; wherein the type of the node pool comprises a reasoning type and a training type, and each task queue corresponds to one type of node pool. By adopting the method, the priority of the to-be-executed tasks in the task queue can be periodically updated, the situation that the to-be-executed tasks with different priorities cannot be scheduled is avoided, meanwhile, the to-be-executed tasks are allocated to more appropriate nodes to be processed, the task processing efficiency can be improved, and the task fine scheduling effect can be optimized.

Description

Technical Field

[0001] The present application relates to the technical field of resource scheduling, and in particular to a task scheduling method, apparatus, computer equipment, storage medium and program product. Background Art

[0002] When processing AI / ML (Artificial Intelligence / Machine Learning) type batch workloads, heterogeneous resource scheduling is often involved. Even for the same type of resources, there may be differences in parameters such as model and architecture. Since different resources, such as device hardware, are applicable to different scenarios, it is difficult to perform refined management.

[0003] Currently, Kubernetes clusters support setting task priorities. If the scheduling requirements are met, high-priority tasks will be scheduled earlier. However, this scheduling method cannot meet the needs of AI / ML workload scenarios. Usually, online reasoning tasks of AI / ML occupy fewer resources, while offline training tasks occupy more resources and often occupy the resources of the entire machine. When the traversal strategy is used for task scheduling, if resources are not met, the task at the head of the queue cannot be scheduled, and the scheduler will try the next task. However, this scheduling method may result in a situation where high-priority tasks cannot be scheduled. When the FIFO task scheduling method is used, the tasks at the back of the queue will be blocked, resulting in the inability to schedule low-priority tasks. Summary of the invention

[0004] Based on this, it is necessary to provide a task scheduling method, device, computer equipment, storage medium and program product that can optimize the task scheduling effect in real time in response to the above technical problems.

[0005] In a first aspect, the present application provides a task scheduling method, comprising:

[0006] Determine the priorities of the tasks to be executed in at least two task queues according to a preset time period, and update the at least two task queues according to the priorities;

[0007] According to the updated task order of at least two task queues, the resource requirements of each task to be executed, and the resource specifications of each node in the node pool corresponding to at least two task queues, each task to be executed is assigned to a matching node for processing; wherein the types of node pools include inference type and training type, and each task queue corresponds to a type of node pool.

[0008] In one embodiment, before determining the priorities of the tasks to be executed in at least two task queues according to a preset time period, the method further includes:

[0009] Creating a target cluster; wherein the target cluster includes at least one node pool of inference type and at least one node pool of training type;

[0010] Create at least two task queues; wherein the at least two task queues include at least one online reasoning task queue of artificial intelligence / machine learning AI / ML and at least one offline training task queue of AI / ML, the online reasoning task queue of AI / ML corresponds to a node pool of reasoning type, and the offline training task queue of AI / ML corresponds to a node pool of training type.

[0011] In one embodiment, according to the updated task order of at least two task queues, the resource requirements of each task to be executed, and the resource specifications of each node in the node pool corresponding to the at least two task queues, each task to be executed is assigned to a matching node for processing, including:

[0012] For each updated task queue, determine the target task from the tasks to be executed in the task queue according to the task order of the task queue;

[0013] Determine whether there is a node in the node pool corresponding to the task queue whose resource specifications match the resource requirements of the target task;

[0014] If yes, the target task is assigned to the matching node for processing, and the operation of determining the target task from the tasks to be executed in the task queue is returned;

[0015] If not, the target task or at least one task to be executed after the target task is assigned to a matching node for processing according to the priority of the target task and the priority of at least one task to be executed after the target task.

[0016] In one embodiment, according to the priority of the target task and the priority of at least one task to be executed after the target task, the target task or at least one task to be executed after the target task is assigned to a matching node for processing, including:

[0017] comparing the priority of the target task with the priority of at least one task to be executed after the target task;

[0018] If there is a pending task whose priority is not lower than the priority of the target task, the pending task will be used as the new target task;

[0019] If there is no pending task whose priority is not lower than the priority of the target task, the target task is assigned to the matching node for processing after the resource specification of the node in the node pool corresponding to the waiting task queue matches the resource requirement of the target task, and the operation of determining the target task from the pending tasks in the task queue is returned.

[0020] In one embodiment, determining the priorities of tasks to be executed in at least two task queues according to a preset time period includes:

[0021] For each task queue, according to a preset time period, the task score of each task to be executed in the task queue is determined according to the resource requirements, expected waiting time and expected execution time of each task to be executed;

[0022] Update the priority of each task to be executed according to the task score of each task to be executed.

[0023] In one embodiment, the method further comprises:

[0024] In response to the task creation instruction, obtaining a task description file;

[0025] Create corresponding tasks to be executed according to the task description file;

[0026] According to the priority of the task to be executed, the task to be executed is added to the task queue of the corresponding type.

[0027] In a second aspect, the present application also provides a task scheduling device, comprising:

[0028] A queue updating module, used to determine the priorities of tasks to be executed in at least two task queues according to a preset time period, and update at least two task queues according to the priorities;

[0029] The task allocation module is used to allocate each task to be executed to a matching node for processing according to the resource requirements of each task to be executed and the resource specifications of each node in the node pool corresponding to at least two task queues; wherein the types of node pools include inference type and training type, and each task queue corresponds to a type of node pool.

[0030] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of the first aspect when executing the computer program.

[0031] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the method of the first aspect when the computer program is executed by a processor.

[0032] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which implements the steps of the method of the first aspect when executed by a processor.

[0033] The above-mentioned task scheduling method, device, computer equipment, storage medium and program product determine the priority of the tasks to be executed in at least two task queues according to a preset time period, and update at least two task queues according to the priority; according to the task order of the at least two task queues after the update, the resource requirements of each task to be executed, and the resource specifications of each node in the node pool corresponding to the at least two task queues, each task to be executed is assigned to a matching node for processing; wherein the types of node pools include reasoning types and training types, and each task queue corresponds to a type of node pool. The present application periodically updates the priority of the tasks to be executed in the task queues to avoid the problem that high-priority tasks to be executed cannot be scheduled, resulting in the inability to schedule low-priority tasks. At the same time, combined with the resource requirements of the tasks to be executed and the resource specifications of the nodes, the tasks to be executed can be assigned to more suitable nodes for processing, thereby improving the efficiency of task execution, and then improving the refined scheduling effect of the task queue as a whole. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 An application environment diagram of a task scheduling method in one embodiment;

[0036] Figure 2 A flowchart of a task scheduling method in one embodiment;

[0037] Figure 3 A schematic diagram of a process for creating a target cluster and a task queue in one embodiment;

[0038] Figure 4 A schematic diagram of a flow chart for allocating tasks to be performed in one embodiment;

[0039] Figure 5 A schematic diagram of a process for determining the priority of a task to be executed in one embodiment;

[0040] Figure 6 A schematic diagram of a process of adding a task to be executed to a task queue in one embodiment;

[0041] Figure 7 is a flowchart of a task scheduling method in another embodiment;

[0042] Figure 8 is a structural block diagram of a task scheduling device in one embodiment;

[0043] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] The task scheduling method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Specifically, the server 104 determines the priority of the tasks to be executed in at least two task queues according to a preset time period, and updates at least two task queues according to the priority; according to the task order of the at least two task queues after the update, the resource requirements of each task to be executed, and the resource specifications of each node in the node pool corresponding to the at least two task queues, each task to be executed is assigned to a matching node for processing; wherein the types of node pools include inference types and training types, and each task queue corresponds to a type of node pool. The server 104 can be an independent physical server, or it can be a server cluster or distributed system composed of multiple physical servers, or it can be a cloud server that provides cloud computing services.

[0046] In an exemplary embodiment, Figure 2 As shown, a task scheduling method is provided, which is applied to Figure 1 The following is an example of a server in the example, including:

[0047] S201, determining priorities of tasks to be executed in at least two task queues according to a preset time period, and updating at least two task queues according to the priorities.

[0048] Wherein, the length of the preset time period can be an empirical value, such as 3 seconds or 3 minutes, and the present embodiment does not limit this. By periodically determining the priority of the tasks to be performed in the task queue, and then updating the task queue according to the priority, the queue situation of the task queue can be perceived in real time, and the tasks to be performed in the task queue can be managed in a refined manner to avoid the problem of task queue blocking.

[0049] In the application scenario of batch workloads, there are generally multiple task queues, that is, at least two task queues. Different types of tasks to be executed can be added to different task queues, and elastic quotas for task queue resource scheduling are supported to achieve resource borrowing and sharing between different task queues. Exemplarily, for each task queue, the minimum and maximum values ​​of the elastic quota are set. Task queues with sufficient resources can provide unoccupied resources to other task queues for scheduling within the range allowed by the elastic quota (their own schedulable resources are not less than the minimum value of the elastic quota, and not higher than the maximum value of the elastic quota). Compared with fixed quotas, this method can effectively improve overall resource utilization.

[0050] At the beginning of each new time period, the priority of the tasks to be executed in the task queue is re-determined, and then the order of the tasks to be executed in the task queue is rearranged according to the priority of the tasks to be executed, so as to update the task queue and obtain an updated task queue. In the updated task queue, the higher the priority of the tasks to be executed, the higher the order, that is, the higher the priority of the tasks to be executed, the earlier they are taken out for execution.

[0051] Optionally, the priority of the task to be executed is determined according to its importance. Exemplarily, a priority algorithm is used to calculate the priority of the task to be executed in the current cycle according to the task category, task scheduling time, resource application amount, and priority when entering the queue of the task to be executed.

[0052] It is understandable that there may be different pending tasks with the same priority. For pending tasks with the same priority, the pending tasks can be arranged according to the time when they enter the task queue.

[0053] S202, allocating each task to be executed to a matching node for processing according to the updated task order of at least two task queues, the resource requirements of each task to be executed, and the resource specifications of each node in the node pool corresponding to the at least two task queues.

[0054] Multiple node pools are maintained in the Kubernetes cluster, each of which includes multiple computing nodes for executing tasks to be executed assigned from task queues. The types of node pools include reasoning type and training type, and each task queue corresponds to a type of node pool. Exemplarily, a task queue composed of reasoning tasks corresponds to a node pool of the reasoning type, and a task queue composed of training tasks corresponds to a node pool of the training type.

[0055] For each task queue, according to the task order updated in the current cycle, the tasks to be executed are sequentially assigned to the matching nodes in the node pool corresponding to the task queue for processing. Specifically, according to the task order of the updated task queue, the tasks to be executed are taken from the task queue for assignment, and the tasks to be executed with the highest priority are generally taken, that is, the tasks to be executed with the highest order.

[0056] According to the resource requirements of the tasks to be executed in the task queue and the resource specifications of each node in the node pool corresponding to the task queue, determine whether the current resources meet the execution conditions of the tasks to be executed, and then allocate the tasks to be executed to the corresponding nodes for processing if the execution conditions are met. Among them, the resource requirements of the tasks to be executed are used to characterize the number of resources required to execute the tasks to be executed, and the resource specifications of the nodes are used to characterize the number of resources that the nodes can schedule. The resources may include but are not limited to GPU, RDMA and FPGA.

[0057] Optionally, when the current resources do not meet the execution conditions of the tasks to be executed, the tasks to be executed are taken out of the task queue and allocated again according to the task order of the updated task queue, so as to avoid the situation where the highest priority task to be executed cannot be scheduled, affecting the subsequent tasks to be executed and causing the task queue to be blocked.

[0058] In the above-mentioned task scheduling method, the priority of the tasks to be executed in at least two task queues is determined according to a preset time period, and at least two task queues are updated according to the priority; according to the task order of the at least two task queues after the update, the resource requirements of each task to be executed, and the resource specifications of each node in the node pool corresponding to the at least two task queues, each task to be executed is assigned to a matching node for processing; wherein the types of node pools include reasoning types and training types, and each task queue corresponds to a type of node pool. This embodiment periodically updates the priority of the tasks to be executed in the task queues to avoid the problem that high-priority tasks to be executed cannot be scheduled, resulting in the inability to schedule low-priority tasks. At the same time, combined with the resource requirements of the tasks to be executed and the resource specifications of the nodes, the tasks to be executed can be assigned to more suitable nodes for processing, thereby improving the efficiency of task execution, and then improving the refined scheduling effect of the task queue as a whole.

[0059] In an exemplary embodiment, Figure 3 As shown, before S201, the task scheduling method further includes:

[0060] S301, create a target cluster.

[0061] The target cluster is a Kubernetes cluster, including at least one inference-type node pool and at least one training-type node pool.

[0062] Based on the node model, usage cost, availability, architecture and other information, the node specifications of the node are set to facilitate the division of each node into the corresponding node pool and complete the group management of each node in the target cluster. Exemplarily, the nodes in the node pool can be GPU nodes, and the node specifications of the GPU nodes are set, such as model, video memory, manufacturer and other information.

[0063] Specifically, in the application scenario of batch processing workloads of AI / ML, each node is divided into an inference type node pool or a training type node pool, so that each node in the inference type node pool is used to process online inference tasks of AI / ML, and each node in the training type node pool is used to process offline training tasks of AI / ML.

[0064] S302: Create at least two task queues.

[0065] Based on actual application requirements, create multiple different types of task queues to cache different types of tasks to be executed.

[0066] Create at least one AI / ML online reasoning task queue and at least one AI / ML offline training task queue. The AI / ML online reasoning task queue corresponds to the reasoning type node pool, and the AI / ML offline training task queue corresponds to the training type node pool.

[0067] Exemplarily, create an inference type node pool named "infer", where the GPU node model is "T4", the video memory is "10GiB", and the performance is "1T fpops". Create a training type node pool named "training", where the GPU node model is "A100", the video memory is "40GiB", and the performance is "10T fpops". Different node pools can realize resource borrowing and sharing within the scope allowed by the elastic quota; then create an AI / ML online inference task queue Q1 and an AI / ML offline training task queue Q2, the task queue Q1 corresponds to the node pool "infer", and the task queue Q2 corresponds to the node pool "training"; further, the tasks to be executed in the task queue Q1 can be assigned to the matching nodes in the node pool "infer" for processing, and the tasks to be executed in the task queue Q2 can be assigned to the matching nodes in the node pool "training" for processing.

[0068] In this embodiment, before allocating the pending tasks in the task queue to the matching nodes for processing, a target cluster is first created, and at least two different types of task queues are created to associate the task queue with the node pool in the target cluster, so as to facilitate the subsequent allocation of different types of pending tasks to different nodes for processing, thereby improving resource utilization and thus improving task processing efficiency.

[0069] In an exemplary embodiment, Figure 4 As shown, the above S202 includes:

[0070] S401 , for each updated task queue, determine a target task from the tasks to be executed in the task queue according to the task order of the task queue.

[0071] At the beginning of each new time period, the task queue is rearranged based on the priority of the tasks to be executed in the current period, and then the target task is taken out from the task queue according to the task order of the task queue.

[0072] Specifically, the first task to be executed in the task queue is taken out as the target task.

[0073] S402, determining whether there is a node in the node pool corresponding to the task queue whose resource specification matches the resource requirement of the target task.

[0074] The task identifier of the task to be executed is associated with the resource requirement and stored in the task queue. The resource requirement of the target task can be directly queried from the task queue.

[0075] According to the queue type of the task queue, at least one corresponding node pool is determined, and then the resource requirements of the target task are compared with the resource specifications of each node in the determined node pool to determine whether the resource specifications of any node match the resource requirements of the target task.

[0076] It can be understood that if there is a node whose resource specification matches the resource requirement of the target task, it is determined that the node has the resource capability to process the target task, and the target task can be assigned to the node; if there is no node whose resource specification matches the resource requirement of the target task, it is determined that the target cluster does not currently have the resource capability to process the target task, and the target task can be assigned to the corresponding node for processing after the resource specification of the resource matches the resource requirement of the target task.

[0077] If yes, execute S403; if no, execute S404.

[0078] S403, assigning the target task to a matching node for processing, and returning to execute the operation of determining the target task from the tasks to be executed in the task queue.

[0079] After the target task is assigned to the matching node, a new target task is determined from the tasks to be executed in the task queue according to the task order of the task queue. For example, the task to be executed that is ranked first at this time is taken out as the new target task, and the step of assigning the target task to the matching node for processing is repeated according to the resource requirements of the target task and the resource specifications of each node.

[0080] S404: According to the priority of the target task and the priority of at least one task to be executed after the target task, assign the target task or at least one task to be executed after the target task to a matching node for processing.

[0081] Since there is no node whose resource specifications match the resource requirements of the target task, different allocation strategies can be determined according to the priority of the target task and the priorities of other tasks to be executed after the target task.

[0082] For example, after waiting for the resource specifications of the node to match the resource requirements of the target task, the target task is assigned to the matching node for processing, and then a new target task is determined from other tasks to be executed for assignment.

[0083] For example, first determine the task with the highest priority from other tasks to be executed after the target task. If there is a node whose resource specifications match the resource requirements of the task to be executed, assign the task to be executed to the matching node for processing, and then return to the step of determining whether there is a node whose resource specifications match the resource requirements of the target task.

[0084] Optionally, compare the priority of the target task with the priority of at least one pending task after the target task; if there is a pending task whose priority is not lower than the priority of the target task, use the pending task as the new target task; if there is no pending task whose priority is not lower than the priority of the target task, wait until there is a node in the node pool corresponding to the task queue whose resource specifications match the resource requirements of the target task, then assign the target task to the matching node for processing, and return to execute the operation of determining the target task from the pending tasks in the task queue.

[0085] It should be noted that there may be tasks with the same priority. If the target task cannot be assigned to a node for processing temporarily, and the priority of the task to be executed after the target task is the same as the priority of the target task, the task to be executed after the target task can be used as the new target task. If the target task cannot be assigned to a node for processing temporarily, and the priority of the task to be executed after the target task is lower than the priority of the target task, the task to be executed after the target task can be assigned after the target task is assigned to a node.

[0086] In the present embodiment, according to the task order of the task queue, the tasks to be executed in the task queue are assigned to the corresponding nodes for processing, and considering the priority of each task to be executed, the tasks to be executed with high priority are preferentially processed, and at the same time, in combination with the resource requirements of the tasks to be executed and the resource specifications of the nodes, the tasks to be executed are assigned to nodes with corresponding computing capabilities for processing, which is conducive to improving the task processing efficiency and optimizing the scheduling effect of the tasks to be executed. In the case where there is no node with corresponding node capacity temporarily, in the present embodiment, the corresponding allocation method can also be selected according to the priority of the tasks to be executed after the target task and the target task, so as to avoid the tasks to be executed from waiting for a long time, thereby improving the efficiency of the task queue to be dequeued in real time.

[0087] In an exemplary embodiment, Figure 5 As shown, the above S201 includes:

[0088] S501 , for each task queue, according to a preset time period, and based on the resource requirements, expected waiting time, and expected execution time of each task to be executed in the task queue, determine the task score of each task to be executed.

[0089] At the beginning of each new time period, the priority of each task to be executed is re-determined according to the resource requirements, expected waiting time and expected execution time of each task to be executed in the task queue. Specifically, the task score of each task to be executed is calculated according to the resource requirements, expected waiting time and expected execution time of each task to be executed, and then the priority of each task to be executed is measured by the task score.

[0090] It should be noted that the processing requirements of the task to be executed may include waiting for a certain period of time before processing, so the expected waiting time of the task to be executed is the waiting time required to process the task to be executed. The expected execution time of the task to be executed is the processing time required to process the task to be executed. The above expected waiting time and expected execution time can be used as task information of the task to be executed, and stored in the task queue in association with the task identifier of the task to be executed.

[0091] Optionally, for each task to be executed, the quantifiable resource requirement, expected waiting time and expected execution time of the task to be executed are substituted into the corresponding function to calculate the task score of the task to be executed.

[0092] S502: Update the priority of each task to be executed according to the task score of each task to be executed.

[0093] Furthermore, the task scores of the tasks to be executed are compared to determine the priority of the tasks to be executed in the current cycle.

[0094] Optionally, when adding the to-be-executed tasks to the task queue, an initial priority may be preset, and then each time a new time period is entered, the priority of the to-be-executed tasks is updated to form a new task queue.

[0095] In this embodiment, the priority of each task to be executed is re-determined in a digitized manner within each time period, which is helpful to avoid the problem of task queue blocking. Tasks to be executed with high priority can be processed first, thereby realizing refined scheduling of the task queue.

[0096] In an exemplary embodiment, Figure 6 As shown, the above task scheduling method further includes:

[0097] S601, in response to a task creation instruction, obtaining a task description file.

[0098] The task creation instruction is used to create a corresponding task to be executed. The task creation instruction may be an instruction input by a user on a terminal device, or an instruction executed when a script file is automatically run. This embodiment does not limit this.

[0099] In response to the task creation instruction, a task description file of the task to be executed is acquired from a preset location.

[0100] S602: Create a corresponding task to be executed according to the task description file.

[0101] Furthermore, according to the task description file, task information of the task to be executed is extracted, including but not limited to resource requirements of the task to be executed, so as to create the task to be executed.

[0102] Optionally, when creating a task to be executed according to the task description file, an initial priority of the task to be executed is determined.

[0103] S603: Add the task to be executed to a task queue of a corresponding type according to the priority of the task to be executed.

[0104] According to the task type of the task to be executed, the task queue corresponding to the task to be executed can be determined, and the task to be executed can be added to the task queue of the corresponding type, thereby ensuring the refined management of the task to be executed and improving the scheduling efficiency. For example, when the task to be executed is an online reasoning task of AI / ML, the corresponding type of task queue is the online reasoning task queue of AI / ML.

[0105] When adding each task to be executed to the corresponding task queue, the initial priority of each task to be executed needs to be considered. The initial priority of the task to be executed can be the priority preset when the task to be executed is created. According to the priority of the current task to be executed and the priorities of other tasks to be executed that already exist in the task queue, the current task to be executed is added to the corresponding position in the task queue, so that each task to be executed in the task queue is arranged according to the priority.

[0106] In this embodiment, a task to be executed is created and added to the corresponding task queue according to the initial priority of the task to be executed. This can achieve simultaneous entry and exit of the task to be executed, avoid conflicts in the priorities of the tasks to be executed, and is conducive to optimizing the precise management of the task queue.

[0107] In an exemplary embodiment, Figure 7 As shown, a task scheduling method is provided, including:

[0108] S701, create a target cluster.

[0109] The target cluster includes at least one inference type node pool and at least one training type node pool.

[0110] S702: Create at least two task queues.

[0111] Among them, at least two task queues include at least one online reasoning task queue of artificial intelligence / machine learning AI / ML and at least one offline training task queue of AI / ML, the online reasoning task queue of AI / ML corresponds to a node pool of reasoning type, and the offline training task queue of AI / ML corresponds to a node pool of training type.

[0112] S703, in response to the task creation instruction, obtaining a task description file, and creating a corresponding task to be executed according to the task description file, and adding the task to be executed to a task queue of a corresponding type according to the priority of the task to be executed.

[0113] S704, according to a preset time period, determine the task score of each task to be executed according to the resource requirements, expected waiting time and expected execution time of each task to be executed in the task queue, and update the priority of each task to be executed according to the task score of each task to be executed, so as to update at least two task queues.

[0114] S705, allocating each task to be executed to a matching node for processing according to the updated task order of the at least two task queues, the resource requirements of each task to be executed, and the resource specifications of each node in the node pool corresponding to the at least two task queues.

[0115] The types of node pools include inference type and training type, and each task queue corresponds to a type of node pool.

[0116] Specifically, for each updated task queue, determine the target task from the tasks to be executed in the task queue according to the task order of the task queue; determine whether there is a node in the node pool corresponding to the task queue whose resource specifications match the resource requirements of the target task; if so, assign the target task to the matching node for processing, and return to execute the operation of determining the target task from the tasks to be executed in the task queue; if not, compare the priority of the target task with the priority of at least one task to be executed after the target task; if there is a task to be executed whose priority is not lower than the priority of the target task, use the task to be executed as the new target task; if there is no task to be executed whose priority is not lower than the priority of the target task, wait until there is a node in the node pool corresponding to the task queue whose resource specifications match the resource requirements of the target task, assign the target task to the matching node for processing, and return to execute the operation of determining the target task from the tasks to be executed in the task queue.

[0117] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0118] Based on the same inventive concept, the embodiment of the present application also provides a task scheduling device for implementing the task scheduling method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more task scheduling device embodiments provided below can refer to the limitations on the task scheduling method above, and will not be repeated here.

[0119] In an exemplary embodiment, Figure 8 As shown, a task scheduling device is provided, comprising:

[0120] The queue updating module 10 is used to determine the priorities of the tasks to be executed in at least two task queues according to a preset time period, and update the at least two task queues according to the priorities.

[0121] The task allocation module 20 is used to allocate each task to be executed to a matching node for processing according to the resource requirements of each task to be executed and the resource specifications of each node in the node pool corresponding to at least two task queues; wherein the types of node pools include inference type and training type, and each task queue corresponds to a type of node pool.

[0122] In an exemplary embodiment, the task scheduling device further includes:

[0123] The cluster creation module is used to create a target cluster; wherein the target cluster includes at least one node pool of reasoning type and at least one node pool of training type.

[0124] A queue creation module is used to create at least two task queues; wherein the at least two task queues include at least one online reasoning task queue for artificial intelligence / machine learning AI / ML and at least one offline training task queue for AI / ML, the online reasoning task queue for AI / ML corresponds to a node pool of reasoning type, and the offline training task queue for AI / ML corresponds to a node pool of training type.

[0125] In an exemplary embodiment, the queue updating module 10 includes:

[0126] The target task determination unit is used to determine the target task from the tasks to be executed in the task queue according to the task sequence of the task queue for each updated task queue.

[0127] The resource condition judgment unit is used to judge whether there is a node in the node pool corresponding to the task queue whose resource specification matches the resource requirement of the target task.

[0128] The task allocation unit is used to allocate the target task to the matching node for processing when there is a match, and return to execute the operation of determining the target task from the tasks to be executed in the task queue; and in the case of mismatch, allocate the target task or at least one task to be executed after the target task to the matching node for processing according to the priority of the target task and the priority of at least one task to be executed after the target task.

[0129] In an exemplary embodiment, the task allocation unit includes:

[0130] The priority comparison subunit is used to compare the priority of the target task with the priority of at least one task to be executed after the target task.

[0131] The task allocation subunit is used to, when there is a task to be executed whose priority is not lower than the priority of the target task, use the task to be executed as the new target task; and when there is no task to be executed whose priority is not lower than the priority of the target task, wait for the resource specification of the node in the node pool corresponding to the task queue to match the resource requirement of the target task, and then allocate the target task to the matching node for processing, and return to execute the operation of determining the target task from the tasks to be executed in the task queue.

[0132] In an exemplary embodiment, the queue updating module 10 further includes:

[0133] The task score determination unit is used to determine the task score of each task to be executed in each task queue according to the resource requirements, expected waiting time and expected execution time of each task to be executed in the task queue according to a preset time period.

[0134] The priority updating unit is used to update the priority of each task to be executed according to the task score of each task to be executed.

[0135] In an exemplary embodiment, the task scheduling device further includes:

[0136] The file acquisition module is used to acquire a task description file in response to a task creation instruction.

[0137] The task creation module is used to create corresponding tasks to be executed according to the task description file.

[0138] The task adding module is used to add the tasks to be executed to the task queue of the corresponding type according to the priority of the tasks to be executed.

[0139] Each module in the above task scheduling device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0140] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig. 9As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as task queues. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a task scheduling method is implemented.

[0141] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0142] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned task scheduling method when executing the computer program.

[0143] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned task scheduling method are implemented.

[0144] In one embodiment, a computer program product is provided, including a computer program, which implements the steps of the above-mentioned task scheduling method when executed by a processor.

[0145] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0146] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0147] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A task scheduling method, characterized in that: The method comprises: Determine the priorities of the tasks to be executed in at least two task queues according to a preset time period, and update the at least two task queues according to the priorities; According to the updated task order of at least two task queues, the resource requirements of each task to be executed, and the resource specifications of each node in the node pool corresponding to the at least two task queues, each task to be executed is assigned to a matching node for processing; wherein the types of the node pool include inference type and training type, and each task queue corresponds to a type of node pool.

2. The method according to claim 1, characterized in that Before determining the priorities of the tasks to be executed in at least two task queues according to the preset time period, the method further includes: Creating a target cluster; wherein the target cluster includes at least one node pool of reasoning type and at least one node pool of training type; Create at least two task queues; wherein the at least two task queues include at least one online reasoning task queue of artificial intelligence / machine learning AI / ML and at least one offline training task queue of AI / ML, the online reasoning task queue of AI / ML corresponds to the node pool of the reasoning type, and the offline training task queue of AI / ML corresponds to the node pool of the training type.

3. The method according to claim 1, characterized in that The method allocates each task to be executed to a matching node for processing according to the updated task order of the at least two task queues, the resource requirements of each task to be executed, and the resource specifications of each node in the node pool corresponding to the at least two task queues, including: For each updated task queue, determining a target task from the tasks to be executed in the task queue according to the task order of the task queue; Determine whether there is a node in the node pool corresponding to the task queue whose resource specification matches the resource requirement of the target task; If yes, the target task is assigned to a matching node for processing, and the operation of determining the target task from the tasks to be executed in the task queue is returned; If not, the target task or at least one task to be executed after the target task is assigned to a matching node for processing according to the priority of the target task and the priority of at least one task to be executed after the target task.

4. The method according to claim 3, characterized in that The step of allocating the target task or at least one task to be executed after the target task to a matching node for processing according to the priority of the target task and the priority of at least one task to be executed after the target task includes: comparing the priority of the target task with the priority of at least one task to be executed after the target task; If there is a pending task whose priority is not lower than the priority of the target task, the pending task is used as the new target task; If there is no task to be executed whose priority is not lower than the priority of the target task, wait until there is a node in the node pool corresponding to the task queue whose resource specifications match the resource requirements of the target task, then assign the target task to the matching node for processing, and return to execute the operation of determining the target task from the tasks to be executed in the task queue.

5. The method according to claim 1, characterized in that Determining the priorities of the tasks to be executed in at least two task queues according to a preset time period includes: For each task queue, according to a preset time period, the task score of each task to be executed in the task queue is determined according to the resource requirements, expected waiting time and expected execution time of each task to be executed; Update the priority of each task to be executed according to the task score of each task to be executed.

6. The method according to claim 1, characterized in that The method further comprises: In response to the task creation instruction, obtaining a task description file; According to the task description file, create a corresponding task to be executed; According to the priority of the task to be executed, the task to be executed is added to a task queue of a corresponding type.

7. A task scheduling device, characterized in that: The device comprises: A queue updating module, used to determine the priorities of tasks to be executed in at least two task queues according to a preset time period, and update the at least two task queues according to the priorities; The task allocation module is used to allocate each task to be executed to a matching node for processing according to the resource requirements of each task to be executed and the resource specifications of each node in the node pool corresponding to at least two task queues; wherein the types of the node pool include inference type and training type, and each task queue corresponds to a type of node pool.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images