Task scheduling method and system, electronic device, medium and product

Abstract

The present disclosure provides a task scheduling method, system, electronic device, medium and product. The task scheduling method comprises: when a training or inference task of a large model is received, determining whether resources that can be provided by a computing cluster of the large model for the task meet a resource request of the task according to resource usage of the computing cluster, wherein the resources are computing resources and / or storage resources; in the case that the resources that can be provided by the computing cluster for the task do not meet the resource request of the task, counting a starvation time of the task; in the case that the starvation time of the task reaches a preset starvation threshold, setting a state of the task to a starvation state, so that the task can obtain a resource quota preferentially; and after the resources that can be obtained by the task meet the resource request, setting the state of the task to a waiting scheduling state, so that the task can be scheduled according to a priority of the task.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and more specifically, to a task scheduling method, system, electronic device, medium, and product. Background Technology

[0002] In large-scale deep learning computing cluster environments, task scheduling for training and inference of large models, as well as the efficient utilization of resources, become critical issues. When the computing cluster is under high load for extended periods, training and inference tasks of large models may experience long wait times due to resource contention, leading to prolonged training time or degraded inference performance.

[0003] For scheduling training and inference tasks of large models, the following scheduling methods are commonly used in this field.

[0004] Preemptive scheduling: allows higher-priority tasks to preempt resources from lower-priority tasks that are currently executing. This ensures that urgent or high-priority tasks can acquire resources more quickly.

[0005] Task fragmentation: Large tasks are broken down into smaller task units, and these smaller task units are scheduled separately. This can improve resource utilization and prevent a large task from occupying resources for an extended period of time.

[0006] Queue scheduling: Queues are used to organize tasks and schedule them according to the priority of tasks in the queue. This ensures that high-priority tasks are executed earlier.

[0007] While the aforementioned scheduling methods can reduce training time and improve inference performance to some extent, they all have some shortcomings that cannot meet the needs of large-scale model training and inference scenarios. For example, preemptive scheduling may cause resource jitter due to task preemption, reducing the overall throughput of the cluster; task sharding may increase task management and communication overhead and is not suitable for all types of tasks; queue scheduling may cause low-priority tasks to wait for a long time, introducing unfairness. Summary of the Invention

[0008] In view of this, this disclosure provides a task scheduling method, system, electronic device, medium, and product, which introduces a starvation-based scheduling strategy to improve the execution efficiency of training and inference tasks of large models and the utilization of computing / storage resources of computing clusters.

[0009] In one aspect, this disclosure provides a task scheduling method, including:

[0010] When a training or inference task for a large model is received, the resource usage of the computing cluster of the large model is used to determine whether the resources that the computing cluster can provide for the task meet the resource request of the task, wherein the resources are computing resources and / or storage resources.

[0011] If the resources that the computing cluster can provide for the task are insufficient to meet the resource request of the task, a starvation time is set for the task.

[0012] When the starvation time of the task reaches a preset starvation threshold, the task's state is set to a starved state, so that the task can preferentially obtain resource quotas; and

[0013] Once the resources available to the task satisfy the resource request, the task's status is set to a waiting-to-schedule state, enabling the task to be scheduled according to its priority.

[0014] According to an embodiment, the method further includes:

[0015] If it is determined that the computing cluster can provide resources to satisfy the resource request of the task, the task's state is set to a waiting-to-schedule state, enabling the task to be scheduled according to its priority; and

[0016] If the hunger time of the task has not reached the hunger threshold, the hunger time of the task continues to be timed.

[0017] According to an embodiment, when the task is in a starved state, the priority of the task is increased, and when the task's state is changed from a starved state to a waiting-to-schedule state, the task's original priority is restored.

[0018] According to an embodiment, when the task is in a starved state, tasks in a waiting-to-be-scheduled state are scheduled according to priority.

[0019] On the other hand, this disclosure also provides a task scheduling system, including:

[0020] The starvation handling scheduling engine is configured as follows:

[0021] When a training or inference task for a large model is received, the resource usage of the computing cluster of the large model is used to determine whether the resources that the computing cluster can provide for the task meet the resource request of the task, wherein the resources are computing resources and / or storage resources.

[0022] If the resources that the computing cluster can provide for the task are insufficient to meet the resource request of the task, a starvation time is set for the task.

[0023] When the starvation time of the task reaches a preset starvation threshold, the task's state is set to a starved state, so that the task can preferentially obtain resource quotas; and

[0024] Once the resources available to the task satisfy the resource request, the task's status is set to a waiting-to-schedule state, enabling the task to be scheduled according to its priority.

[0025] The application layer is used to receive configuration parameters set by the user and pass the configuration parameters to the control module. The configuration parameters include the starvation threshold and the priority of the task.

[0026] The control module is configured to manage the configuration parameters and provide the configuration parameters to the starvation processing scheduling engine; and

[0027] The monitoring system is configured to monitor the resource usage of the large-scale computing cluster.

[0028] According to an embodiment, the hunger handling scheduling engine is further configured as follows:

[0029] If it is determined that the computing cluster can provide resources to satisfy the resource request of the task, the task's state is set to a waiting-to-schedule state, enabling the task to be scheduled according to its priority; and

[0030] If the hunger time of the task has not reached the hunger threshold, the hunger time of the task continues to be timed.

[0031] According to an embodiment, the hunger handling scheduling engine is further configured as follows:

[0032] When the task is in a starved state, its priority is increased, and when the task's state is changed from a starved state to a waiting-to-schedule state, its original priority is restored.

[0033] In another aspect, this disclosure also provides an electronic device, characterized in that the electronic device comprises:

[0034] At least one processor; and

[0035] The memory communicatively connected to the at least one processor, wherein,

[0036] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the task scheduling method of this disclosure.

[0037] In another aspect, this disclosure also provides a non-transitory computer-readable storage medium, characterized in that the non-transitory computer-readable storage medium stores computer instructions for causing a computer to execute the task scheduling method of this disclosure.

[0038] The task scheduling method, system, electronic device, medium, and product disclosed herein have at least the following advantages compared to related technologies. By timing the starvation period for tasks whose resource requests are not met by the computing / storage resources available to the computing cluster, and designating them as starved tasks (with their status set to starved) when their starvation period reaches a starvation threshold, and setting specific scheduling rules for starved tasks to prioritize their resource allocation, the scheduling problem when resources are insufficient can be clearly addressed. This avoids resource contention caused by prematurely processing resource-intensive tasks, and ensures that starved tasks receive sufficient resources at appropriate times, effectively reducing task waiting time, improving task processing efficiency, increasing the training / inference efficiency of large models, and enhancing the utilization rate of computing / storage resources in the computing cluster. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 An exemplary flowchart of a task scheduling method according to an embodiment of the present disclosure is shown;

[0041] Figure 2 Another exemplary flowchart of a task scheduling method according to an embodiment of the present disclosure is shown;

[0042] Figure 3 An exemplary structural diagram of a task scheduling system according to an embodiment of the present disclosure is shown;

[0043] Figure 4 Another exemplary structural diagram of a task scheduling system according to an embodiment of the present disclosure is shown;

[0044] Figure 5 An exemplary structural schematic diagram of an apparatus capable of implementing the methods according to embodiments of the present disclosure is shown. Detailed Implementation

[0045] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0046] The following specific examples illustrate the implementation of this disclosure. Those skilled in the art can easily understand other advantages and effects of this disclosure from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. This disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this disclosure. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0047] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this disclosure, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0048] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this disclosure. The drawings only show the components related to this disclosure and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0049] Furthermore, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the aspects can be practiced without these specific details.

[0050] First refer to Figure 1 and Figure 2 This application describes a task scheduling method according to embodiments thereof. Figure 1 This is an exemplary flowchart illustrating a task scheduling method according to an embodiment of the present disclosure. Figure 2 This is another exemplary flowchart illustrating a task scheduling method according to an embodiment of the present disclosure. Figure 1 and Figure 2 As shown, the task scheduling method according to an embodiment of this disclosure includes the following operations.

[0051] First of all, Figure 1 In step S101, upon receiving a training or inference task for a large model, the system determines whether the resources the computing cluster can provide for the task meet the task's resource request (resource requirement) based on the resource usage of the large model's computing cluster. These resources include computing resources and / or storage resources. This step can, for example, be performed by the starvation scheduling engine of a task scheduling system, as described in detail later.

[0052] Specifically, such as Figure 2 As shown, in step S201, a training or inference task for a large model is received. The large model can be, for example, a large language model based on Transformer or any other large model with training / inference tasks, and its application domain can be, for example, any application domain such as image processing, speech recognition, natural language processing, or computer vision. The training or inference task for the large model can be, for example, the computation of a single layer of the large model's network, or the computation of one or more operators. The received task can be added, for example, to a scheduling queue for managing unscheduled tasks. The scheduling queue can include multiple unscheduled tasks, and the task's state can be, for example, a starved state or a waiting-for-scheduling state, as described later.

[0053] Next, in step S202, based on the resource usage of the large model's computing cluster, it is determined whether the resources the computing cluster can provide for the task meet the task's resource request. The large model's computing cluster can be, for example, a distributed cluster for training or inference of the large model, including multiple computing nodes (e.g., multiple CPUs or GPUs), storage nodes (e.g., storage servers), etc. Resources can be, for example, computing resources and / or storage resources. The task's resource request can be, for example, an estimate of the task's demand for computing and / or storage resources based on the task's computational complexity and data size, which can be obtained from the task information upon receiving the task or estimated after receiving the task. Upon receiving a large model training or inference task, for example, the resource request processing module of the starvation scheduling engine determines, based on the usage (or remaining status) of the large model's computing cluster's computing and / or storage resources, whether the resources the computing cluster can provide for the received task meet the task's resource request.

[0054] Next, in Figure 1 In step S102, if the resources that the computing cluster can provide for the task are insufficient to meet the task's resource requests, a starvation timer is set for the task. This step can be performed, for example, by a starvation handling scheduling engine.

[0055] Specifically, such as Figure 2As shown, when the determination in step S202 is "No," that is, when it is determined that the resources that the computing cluster can provide for the task do not meet the resource request of the task, the process proceeds to step S203, where a starvation timer is started for the task. This step can, for example, be executed by the starvation timing module of the starvation processing scheduling engine of the task scheduling system through a timer under the control of the timer management module of the task scheduling system's control module. When the determination in step S202 is "Yes," that is, when it is determined that the resources that the computing cluster can provide for the task meet the resource request of the task, the process proceeds to step S208, which will be described later.

[0056] With the hunger time of the task being timed, next, in Figure 1 In step S103, when the task's starvation time reaches a preset starvation threshold, the task's state is set to a starved state (or the task is set to a starved task) so that the task can preferentially obtain resource quotas (idle quotas). This step can be performed, for example, by a starvation handling scheduling engine.

[0057] Specifically, such as Figure 2As shown, in step S203, when the task's starvation time is timed, in step S204, it is determined whether the task's starvation time has reached a preset starvation threshold. The starvation threshold can be a fixed time period, such as 24 hours, set by the user through the application layer configuration interface or configuration API of the task scheduling system, for example. The configuration interface or configuration API transmits configuration parameters such as the starvation threshold to the configuration management module of the task scheduling system's control module. The configuration management module manages the configuration parameters and provides them to the timer management module of the control module. The timer management module determines whether the task's starvation time has reached the starvation threshold based on the timed starvation time and the starvation threshold. If the determination in step S204 is "no," meaning the task's starvation time has not reached the starvation threshold, the process returns to step S203 to continue timing the task's starvation. If the determination in step S204 is "yes," meaning the task's starvation time has reached the starvation threshold, the process proceeds to step S205, where the task is set as a starved task, that is, the task's state is set to a starved state. This step can be performed, for example, by the task state management module of the starvation processing scheduling engine. When a task is in a starved state, i.e., when the task is a hungry task, in step S206, the hungry task is given priority in obtaining resource quotas. Prioritizing resource quotas may include preempting the current remaining resource quotas (idle quotas) of the computing cluster and new resource quotas generated when used resources are released, preventing hungry tasks with high resource demands from waiting too long for resources due to resource quotas being allocated to other tasks (newly received tasks or tasks in a waiting-to-be-scheduled state). This step can be performed, for example, by the hungry task scheduling module of the hunger processing scheduling engine. For instance, the hungry task scheduling module can mark resource quotas as used by hungry tasks or allow hungry tasks to lock resource quotas by creating records in data structures such as resource mapping tables that associate resource quotas with hungry tasks, thereby giving hungry tasks priority in obtaining resource quotas. When a task is in a starved state, its priority can be increased so that hungry tasks can obtain resource quotas first. It can be considered that hungry tasks have a higher priority than other tasks (newly received tasks or tasks in a waiting-to-be-scheduled state) and do not need to obtain all requested resources at once, but can accumulate resource quotas to obtain sufficient resources to satisfy the task's resource requests. If the scheduling queue includes multiple hungry tasks, resource quotas can be allocated to the hungry tasks according to their priorities, so that higher-priority hungry tasks can obtain sufficient resources earlier.

[0058] Next, in Figure 1In step S104, after the resources available to the task satisfy its resource request, the task's state is set to a waiting-to-schedule state, allowing tasks to be scheduled according to their priority. This step can be performed, for example, by a starvation-based scheduling engine.

[0059] Specifically, such as Figure 2 As shown, after step S206, in step S207, it is determined whether the task has obtained sufficient resources, that is, whether the resources that the task can obtain meet the resource request. The resources that the task can obtain at this time are the accumulation of the priority resource quota. This step can be executed, for example, by the resource request processing module of the starvation processing scheduling engine. If the determination in step S207 is "no," that is, if it is determined that the resources that the task can obtain do not meet the resource request, it returns to step S206 to continue accumulating the resource quota. If the determination in step S207 is "yes," that is, if it is determined that the resources that the task can obtain meet the resource request, it proceeds to step S208, and in step S208, the task's state is set to the waiting-for-scheduling state. Furthermore, if the determination in step S202 is "yes," that is, if it is determined that the computing cluster can provide resources that meet the task's resource request, it also proceeds to step S208, and in step S208, the task's state is set to the waiting-for-scheduling state. Step S208 can be executed, for example, by the task state management module of the starvation processing scheduling engine. When a task is in a waiting-to-schedule state, in step S209, the task is scheduled according to its priority. Task scheduling in a non-starved state can be performed, for example, through a regular task scheduling module. When a task is in a waiting-to-schedule state, the task priority used is its original priority, which is the priority set by the user as a configuration parameter when submitting the task, for example, through the application-layer configuration interface or configuration API, based on the task's importance and urgency. Priority can be represented numerically, for example, ranging from 0 to 99, with larger numbers indicating higher priority. If a task has experienced a starved state and its priority was increased during that state, its original priority can be restored in step S208 when the task's state is changed from a starved state to a waiting-to-schedule state.

[0060] In some embodiments, when a task is starved or in a starved state, it does not actually consume computing and storage resources of the computing cluster. In this case, to improve resource utilization and training / inference efficiency, other tasks in a waiting-to-schedule state (i.e., those whose resource requests can be met) can be scheduled according to priority. Furthermore, when executing a task that has experienced starvation (i.e., has high computing / storage resource requirements), if computing / storage resources become available, these resources can be allocated to other tasks to ensure efficient resource utilization.

[0061] In this embodiment of the disclosure, a task scheduling system is also disclosed. Figure 3 This is an exemplary structural diagram of a task scheduling system according to an embodiment of the present disclosure. The system includes: an application layer 301, a control module 302, a starvation handling scheduling engine 303, and a monitoring system 304.

[0062] The starvation handling scheduling engine 303 is configured to: upon receiving a training or inference task for a large model, determine whether the resources that the computing cluster can provide for the task meet the task's resource request based on the resource usage of the large model's computing cluster, where the resources are computing resources and / or storage resources; if the resources that the computing cluster can provide for the task do not meet the task's resource request, start a starvation timer for the task; if the task's starvation time reaches a preset starvation threshold, set the task's state to a starved state so that the task can obtain resource quotas first; and after the resources that the task can obtain meet its resource request, set the task's state to a waiting-to-schedule state so that tasks can be scheduled according to their priority.

[0063] Application layer 301 is configured to receive configuration parameters set by the user and pass these parameters to control module 302. The configuration parameters include a starvation threshold and task priority.

[0064] The control module 302 is configured to manage configuration parameters and provide the configuration parameters to the starvation processing scheduling engine 303.

[0065] Monitoring system 304 is configured to monitor the resource usage of the large model's computing cluster.

[0066] In some embodiments, the starvation handling scheduling engine 303 is further configured to: set the state of the task to a waiting scheduling state when it is determined that the computing cluster can provide resources to the task to meet the resource request of the task, so that the task can be scheduled according to the priority of the task; and continue to count the starvation time of the task when the starvation time of the task has not reached the starvation threshold.

[0067] In some embodiments, the starvation handling scheduling engine 303 is further configured to: increase the priority of a task when the task is in a starved state, and restore the task to its original priority when the task's state is changed from a starved state to a waiting-to-schedule state.

[0068] In some embodiments, such as Figure 4As shown, the application layer 301 may include a configuration interface, a configuration API, and a security and access control module. The configuration interface and configuration API are used to receive configuration parameters such as user-set starvation thresholds and task priorities, and then pass these parameters to the configuration management module of the control module 302. The security and access control module can be used, for example, to manage user login and operation permissions.

[0069] The control module 302 may include a configuration management module, a logging module, a security and access control module, and a timer management module. The configuration management module receives and manages user-set configuration parameters from the application layer 301, and transmits these parameters to the timer management module and the hunger task scheduling module of the hunger processing scheduling engine 303 at appropriate intervals. The logging module records key events such as task scheduling and hunger timing. The security and access control module manages permissions for task scheduling and task state changes, ensuring system security. The timer management module controls the hunger timing module of the hunger processing scheduling engine 303 to time the hunger duration of tasks and determines whether the hunger time has reached the hunger threshold.

[0070] The starvation handling scheduling engine 303 may include a resource request processing module, a starvation timing module, a task status management module, and a starvation task scheduling module. The resource request processing module determines whether the computing cluster can provide sufficient resources to meet the resource requests of the received tasks. The starvation timing module uses a timer to time the starvation period of a task. The task status management module sets the task status to either a starved state or a waiting-to-be-scheduled state. The starvation task scheduling module schedules starved tasks, ensuring that starved tasks have priority in obtaining the remaining resource quotas of the computing cluster. When there are starved tasks with different priorities, they are scheduled according to their priority.

[0071] The monitoring system 304 may include resource release monitoring and resource allocation monitoring. Resource release monitoring is used to monitor the resource release status of the computing cluster, such as which tasks released how much resource, and how much resource remains. Resource allocation monitoring is used to monitor the resource allocation status of the computing cluster, such as which tasks were allocated how much resource, and how much resource is currently being used.

[0072] The aforementioned components, units, or modules interact through clearly defined interfaces to form a collaborative system for effectively implementing the task scheduling method according to this disclosure.

[0073] In this disclosure, an electronic device is also disclosed, characterized in that the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a task scheduling method according to this disclosure.

[0074] In this disclosure, a non-transitory computer-readable storage medium is also disclosed, which stores computer instructions for causing a computer to execute a task scheduling method according to this disclosure.

[0075] In this disclosure, a computer program product is also disclosed, comprising a computing program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform a task scheduling method according to this disclosure.

[0076] In this embodiment of the disclosure, a computer program is also disclosed, which includes program instructions that, when executed by a computer, cause the computer to perform a task scheduling method according to the present disclosure.

[0077] Figure 5 An exemplary structural schematic diagram of a device 1000 capable of implementing the methods according to embodiments of the present disclosure is shown. In some embodiments, more or fewer devices may be included than those shown. In some embodiments, implementation may be carried out using a single or multiple devices. In some embodiments, implementation may be carried out using cloud-based or distributed devices.

[0078] like Figure 5 As shown, device 1000 includes a processor 1001, which can perform various appropriate operations and processes based on programs and / or data stored in read-only memory (ROM) 1002 or programs and / or data loaded from storage portion 1008 into random access memory (RAM) 1003. Processor 1001 may be a multi-core processor or may contain multiple processors. In some embodiments, processor 1001 may include a general-purpose main processor and one or more special coprocessors, such as a central processing unit (CPU), graphics processing unit (GPU), neural network processor (NPU), digital signal processor (DSP), etc. Various programs and data required for the operation of device 1000 are also stored in RAM 1003. Processor 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. Input / output (I / O) interface 1005 is also connected to bus 1004.

[0079] The processor and memory described above are used together to execute a program stored in the memory. When the program is executed by a computer, it can implement the methods, steps, or functions described in the above embodiments.

[0080] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, touchscreen, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. A removable medium 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 1010 as needed so that computer programs read from it can be installed into storage section 1008 as needed. Figure 5 The diagram only shows a portion of the components and does not imply that the device 1000 only includes... Figure 5 The components shown.

[0081] The systems, devices, modules, or units described in the above embodiments can be implemented by a computer or its associated components. The computer may be, for example, a mobile terminal, smartphone, personal computer, laptop computer, in-vehicle human-machine interface device, personal digital assistant, media player, navigation device, game console, tablet computer, wearable device, smart TV, Internet of Things system, smart home, industrial computer, server, or a combination thereof.

[0082] Although not shown, in embodiments of this disclosure, a computer-readable storage medium is provided having a computer program / instructions stored thereon that, when executed by a processor, implements a task scheduling method according to this disclosure.

[0083] Storage media in embodiments of this disclosure include articles that are permanent and non-permanent, removable and non-removable, capable of storing information by any method or technology. Examples of storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0084] Although not shown, embodiments of this disclosure also provide a computer program product, including: a computer program / instructions that, when executed by a processor, implement a task scheduling method according to this disclosure.

[0085] The methods, programs, systems, apparatuses, etc., of the embodiments of this disclosure can be executed or implemented in one or more networked computers, or practiced in a distributed computing environment. In the embodiments of this specification, in these distributed computing environments, tasks can be performed by remote processing devices connected via a communication network.

[0086] Those skilled in the art will understand that the embodiments described in this specification can be provided as methods, systems, or computer program products. Therefore, those skilled in the art will realize that the functional modules / units or controllers and related method steps described in the above embodiments can be implemented in software, hardware, or a combination of both.

[0087] Unless explicitly stated otherwise, the actions or steps of the methods or procedures described in the embodiments of this disclosure do not necessarily have to be performed in a specific order and can still achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0088] This document describes several embodiments of the present disclosure; however, for the sake of brevity, the descriptions of the embodiments are not exhaustive, and identical or similar features or portions between the embodiments may be omitted. In this document, "one embodiment," "some embodiments," "example," "specific example," or "some examples" refers to at least one embodiment or example applicable to the present disclosure, but not all embodiments. The above terms do not necessarily refer to the same embodiment or example. Without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples.

[0089] The exemplary systems and methods of this disclosure have been specifically shown and described with reference to the foregoing embodiments, and are merely examples of the best mode for implementing the systems and methods. Those skilled in the art will understand that various changes can be made to the embodiments of the systems and methods described herein without departing from the spirit and scope of this disclosure as defined in the appended claims when implementing the systems and / or methods.

Claims

1. A task scheduling method, characterized in that, include: When a training or inference task for a large model is received, the resource usage of the computing cluster of the large model is used to determine whether the resources that the computing cluster can provide for the task meet the resource request of the task, wherein the resources are computing resources and / or storage resources. If the resources that the computing cluster can provide for the task are insufficient to meet the resource request of the task, a starvation time is set for the task. When the starvation time of the task reaches a preset starvation threshold, the status of the task is set to a starvation state so that the task can obtain resource quotas first. as well as Once the resources available to the task satisfy the resource request, the task's status is set to a waiting-to-schedule state, enabling the task to be scheduled according to its priority.

2. The method according to claim 1, characterized in that, The method further includes: If it is determined that the computing cluster can provide resources to satisfy the resource request of the task, the task's state is set to a waiting-to-schedule state, enabling the task to be scheduled according to its priority; and If the hunger time of the task has not reached the hunger threshold, the hunger time of the task continues to be timed.

3. The method according to claim 1, characterized in that, When the task is in a starved state, its priority is increased, and when the task's state is changed from a starved state to a waiting-to-schedule state, its original priority is restored.

4. The method according to claim 1 or 2, characterized in that, When the task is starved or when the task is in a starved state, other tasks in a waiting-to-schedule state are scheduled according to priority.

5. A task scheduling system, characterized in that, include: The starvation handling scheduling engine is configured as follows: When a training or inference task for a large model is received, the resource usage of the computing cluster of the large model is used to determine whether the resources that the computing cluster can provide for the task meet the resource request of the task, wherein the resources are computing resources and / or storage resources. If the resources that the computing cluster can provide for the task are insufficient to meet the resource request of the task, a starvation time is set for the task. When the starvation time of the task reaches a preset starvation threshold, the status of the task is set to a starvation state so that the task can obtain resource quotas first. as well as Once the resources available to the task satisfy the resource request, the task's status is set to a waiting-to-schedule state, enabling the task to be scheduled according to its priority. The application layer is configured to receive configuration parameters set by the user and pass the configuration parameters to the control module. The configuration parameters include the starvation threshold and the priority of the task. The control module is configured to manage the configuration parameters and provide the configuration parameters to the starvation processing scheduling engine. as well as The monitoring system is configured to monitor the resource usage of the large-scale computing cluster.

6. The system according to claim 5, characterized in that, The starvation handling scheduling engine is also configured to: If it is determined that the computing cluster can provide resources to satisfy the resource request of the task, the task's state is set to a waiting-to-schedule state, enabling the task to be scheduled according to its priority; and If the hunger time of the task has not reached the hunger threshold, the hunger time of the task continues to be timed.

7. The system according to claim 5 or 6, characterized in that, The starvation handling scheduling engine is also configured to: When the task is in a starved state, its priority is increased, and when the task's state is changed from a starved state to a waiting-to-schedule state, its original priority is restored.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and The memory communicatively connected to the at least one processor, wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the task scheduling method according to any one of claims 1 to 4.

9. A non-transitory computer-readable storage medium, characterized in that, The non-transitory computer-readable storage medium stores computer instructions for causing a computer to perform the task scheduling method according to any one of claims 1 to 4.

10. A computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the task scheduling method according to any one of claims 1 to 4.

Images