Scheduling system, scheduling method of scheduling system, electronic equipment and storage medium

Abstract

The invention discloses a scheduling system, a scheduling method of the scheduling system, electronic equipment and a storage medium, and belongs to the technical field of cloud computing. The scheduling system comprises a scheduling control module which is used for responding to a received to-be-scheduled task and determining a target arrangement computing node from arrangement computing nodes contained in a container arrangement cluster according to the computing power specification requirement of the to-be-scheduled task; the resource management module is used for deploying a target container group on the target arrangement computing node and registering the target container group as a scheduling computing node in the job scheduling cluster; wherein the computing power resource quantity allocated to the target container group is matched with the computing power specification requirement; and the job scheduling module is used for distributing the to-be-scheduled task to the target container group for execution, the computing power resource quantity of the scheduling computing node corresponding to the target container group is dynamically applied and divided based on the demand and is not all the system resource quantity of the scheduling computing node, and the problem that the utilization rate of the computing power resource of the job scheduling cluster is low is effectively solved.

Description

Technical Field

[0001] This application belongs to the field of cloud computing technology, and in particular relates to a scheduling system, a scheduling method for the scheduling system, an electronic device, and a storage medium. Background Technology

[0002] In job scheduling systems, some clusters, such as the SimpleLinux Utility for Resource Management (Slurm), do not allow a node to run multiple tasks (or jobs) simultaneously by default. Resources on a node cannot be shared by multiple tasks (i.e., overcommitting or sharing is not supported). In this case, a large number of node resources (especially computing resources) will be idle. Especially during periods of light load, the system's computing resources are "monopolized" by a few tasks, resulting in low resource utilization. Summary of the Invention

[0003] This application aims to solve at least one of the technical problems existing in related technologies. To this end, this application proposes a scheduling system, a scheduling method for the scheduling system, an electronic device, and a storage medium, thereby improving the utilization rate of system computing power and other resources.

[0004] In a first aspect, this application provides a scheduling system, which includes a container orchestration cluster and a job scheduling cluster, wherein the job scheduling cluster is deployed and runs on the container orchestration cluster; the scheduling system further includes: The scheduling control module is used to respond to the received task to be scheduled and determine the target orchestration computing node from the orchestration computing nodes contained in the container orchestration cluster according to the computing power specifications of the task to be scheduled. The resource management module is used to deploy target container groups on target orchestration computing nodes and register the target container groups as scheduling computing nodes in the job scheduling cluster; wherein, the amount of computing resources allocated to the target container groups matches the computing power specification requirements; The job scheduling module is used to distribute tasks to be scheduled to target container groups for execution.

[0005] The scheduling system of this application includes a container orchestration cluster and a job scheduling cluster. The job scheduling cluster is deployed and run on the container orchestration cluster. During task scheduling, the scheduling control module responds to the received tasks to be scheduled and determines the target orchestration computing node from the orchestration computing nodes included in the container orchestration cluster according to the computing power specifications of the tasks to be scheduled. The resource management module deploys the target container group on the target orchestration computing node and registers the target container group as a scheduling computing node in the job scheduling cluster. The computing power resources allocated to the target container group match the computing power specifications. The job scheduling module distributes the tasks to be scheduled to the target container group for execution. That is, although the created scheduling computing node is exclusively used by the tasks to be scheduled, the computing power resources corresponding to the scheduling computing node are dynamically applied for and allocated based on demand, and are not the total system resources of the orchestration computing nodes in the container orchestration cluster. This effectively solves the problem of low computing power resource utilization in job scheduling clusters (such as the Slurm native cluster).

[0006] According to one embodiment of this application, the resource management module is used to divide a target virtual computing power slice that meets the computing power specification requirements from the physical computing power card on the target orchestration computing node using a container resource isolation mechanism or a device virtualization plugin; and bind the target virtual computing power slice as a computing power resource to the target container group for computing power resource sharing and reuse.

[0007] According to one embodiment of this application, a scheduling control module is used to select candidate orchestration computing nodes that meet the computing power specifications of the task to be scheduled from the orchestration computing nodes included in the container orchestration cluster. Access the node dynamic score queue and obtain the node score of each candidate orchestration compute node from the node dynamic score queue; the node score is calculated based on multiple resource monitoring parameters of the candidate orchestration compute node; The candidate orchestration computation node with the highest node score is determined as the target orchestration computation node.

[0008] According to one embodiment of this application, the scheduling control module is used to maintain a dynamic score queue for nodes in the following manner: Periodically obtain resource monitoring parameters for each orchestration computing node from the monitoring component; the resource monitoring parameters include at least one performance parameter and a resource remaining status parameter; Based on the resource monitoring parameters of each orchestration computing node, determine the parameter scores of each resource monitoring parameter of the orchestration computing node. The parameter scores and parameter weights of each resource monitoring parameter are weighted and summed to obtain the node score of the orchestration calculation node; The node scores of each orchestration calculation node are stored in the node dynamic score queue in descending order of node scores.

[0009] According to one embodiment of this application, the resource management module is used to set the node name of the scheduling computing node corresponding to the target container group to the task name of the task to be scheduled when registering the target container group as a scheduling computing node in the job scheduling cluster; The job scheduling module is used to find the target container group from the job scheduling cluster based on the task name of the task to be scheduled, and distribute the task to be scheduled to the target container group for execution.

[0010] According to one embodiment of this application, the job scheduling module includes a container process manager; Once the scheduled tasks have been completed, the target container group sends a task completion semaphore to the container process manager, causing the container process manager to exit the main process. When the main process exits, the target container group is automatically destroyed and releases its computing resources. When the resource management module detects the destruction semaphore of the target container group, it deregisters the scheduling computing node corresponding to the target container group from the job scheduling cluster.

[0011] According to one embodiment of this application, the scheduling control module is used to obtain custom resources submitted by the user to the scheduling control module for tasks to be scheduled; When custom resources include task priorities, the tasks to be scheduled are inserted into the task scheduling queue of the job scheduling cluster according to the task priorities. If the custom resource does not include task priority, insert the task to be scheduled into the tail of the task scheduling queue; Read the currently scheduled tasks from the head of the task scheduling queue.

[0012] Secondly, this application provides a scheduling method for a scheduling system, the scheduling system including a container orchestration cluster and a job scheduling cluster; the job scheduling cluster is deployed and runs on the container orchestration cluster; the scheduling system also includes a scheduling control module, a resource management module, and a job scheduling module; the scheduling method includes: The scheduling control module responds to the received task to be scheduled and determines the target orchestration computing node from the orchestration computing nodes contained in the container orchestration cluster according to the computing power specifications of the task to be scheduled. The target container group is deployed on the target orchestration computing node through the resource management module, and the target container group is registered as a scheduling computing node in the job scheduling cluster; wherein the amount of computing resources allocated to the target container group matches the computing power specification requirements. The job scheduling module distributes the tasks to be scheduled to the target container group for execution.

[0013] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a scheduling method of the scheduling system provided in the first aspect above.

[0014] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements a scheduling method of the scheduling system provided in the first aspect above.

[0015] Fifthly, this application provides a chip including a processor and a communication interface coupled to the processor, the processor being used to run programs or instructions to implement the scheduling method of the scheduling system provided in the first aspect.

[0016] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements a scheduling method of the scheduling system provided in the first aspect above.

[0017] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects: The scheduling system comprises a container orchestration cluster and a job scheduling cluster. The job scheduling cluster is deployed and runs within the container orchestration cluster. During task scheduling, the scheduling control module responds to received tasks and determines the target orchestration compute node from among the orchestration compute nodes in the container orchestration cluster based on the compute requirements of the tasks. The resource management module deploys the target container group on the target orchestration compute node and registers the target container group as a scheduling compute node in the job scheduling cluster. The compute resources allocated to the target container group match the compute requirements. The job scheduling module distributes the tasks to be scheduled to the target container group for execution. Although the created scheduling compute node is exclusively used by the task to be scheduled, the compute resources corresponding to the scheduling compute node are dynamically requested and allocated based on demand, and do not represent all the system resources of the orchestration compute nodes in the container orchestration cluster. This effectively solves the problem of low compute resource utilization in job scheduling clusters (such as the Slurm native cluster).

[0018] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0019] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a schematic diagram of the scheduling system provided in the embodiments of this application; Figure 2 This is one of the schematic diagrams of the scheduling system obtained after deploying a Slurm cluster on a Kubernetes cluster, provided in an embodiment of this application; Figure 3 This is a second schematic diagram of the scheduling system comprising a Kubernetes cluster and a Slurm cluster provided in the embodiments of this application; Figure 4 This is a schematic diagram of the process for constructing a dynamic score queue for nodes provided in an embodiment of this application; Figure 5 This is a schematic diagram of the interaction between various parts of the scheduling system provided in the embodiments of this application; Figure 6 This is a flowchart illustrating the scheduling method of the scheduling system provided in this application embodiment; Figure 7 This is a schematic diagram of the structure of the scheduling device of the scheduling system provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0021] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0022] Slurm, as a widely adopted cluster management and job scheduling system in the field of high-performance computing, is designed to meet the needs of large-scale computing scenarios. However, it suffers from drawbacks such as low resource utilization, poor heterogeneous computing power partitioning and scheduling capabilities, poor scalability, and high operation and maintenance costs. The reasons are as follows: 1. Regarding poor scalability, as the complexity of the job increases, the types of resource devices required also increase, such as heterogeneous computing power cards from different manufacturers, such as GPUs (Graphics Processing Units) / NPUs (Neural Processing Units). To enable Slurm to support the scheduling of such devices, developers need to perform secondary development and recompile the source code based on Slurm's General Resource Scheduling Plugin (Gres), which greatly increases the workload of developers and the operation and maintenance costs, resulting in poor scalability.

[0023] 2. Regarding low resource utilization, Slurm, by default, does not allow a single node to run multiple jobs simultaneously. Resources on a node are not shared by multiple jobs or tasks (i.e., over-allocation), even if the node only uses 50% of its CPU (Central Processing Unit) and memory. Although setting OverSubscribe to "YES" in the slurm.conf configuration file allows a node to run multiple jobs, the actual resource utilization of the node often falls far short of the requested resource threshold, resulting in a large number of idle node resources. This is especially true during periods of light load, when resources are "monopolized" by a few jobs, leading to a decrease in overall utilization. Traditional Slurm clusters and Kubernetes clusters are deployed and maintained independently, unable to share hardware resources, and cannot make idle hardware resources available for use by either party at any time.

[0024] 3. Regarding limited scheduling capabilities, Slurm lacks fine-grained awareness of GPU / NPU memory and SM (Streaming Multiprocessor Unit) utilization, and cannot automatically avoid high loads (e.g., cards with memory usage >90% will still be assigned tasks). Whenever a task needs to use heterogeneous computing power from different vendors, the gres.conf file on the Slurm node often needs to be configured in advance. Furthermore, whenever the application changes to a new computing power specification or partitioning method, the gres.conf file must be reset, making it extremely inflexible.

[0025] To address at least one of the aforementioned deficiencies, this application provides a scheduling system, a scheduling method for the scheduling system, an electronic device, and a storage medium.

[0026] The scheduling system, scheduling method of the scheduling system, electronic device and storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0027] This application provides a scheduling system that includes a container orchestration cluster and a job scheduling cluster.

[0028] A container orchestration cluster refers to a cluster environment that uses container orchestration tools (such as Kubernetes, Docker Swarm, etc.) to manage and coordinate the operation, scaling, load balancing, and fault recovery of multiple containers.

[0029] In practical applications, container orchestration clusters can specifically be Kubernetes clusters, Docker Swarm clusters, etc.

[0030] A job scheduling cluster is a cluster that uses a scheduling system to uniformly manage and allocate computing resources, enabling efficient scheduling, execution, and monitoring of various tasks or jobs. Job scheduling clusters can handle a large number of concurrent computing tasks, ensuring efficient resource utilization and managing the order, priority, and dependencies of job execution.

[0031] In practical applications, the job scheduling cluster can be a slurm cluster, or a cluster that shares at least one of the same defects as the slurm cluster (poor splitting scheduling capability, poor scalability, high operation and maintenance costs, etc.).

[0032] See Figure 1 This application provides a schematic diagram of the composition of a job scheduling system, including a scheduling system comprising a container orchestration cluster and a job scheduling cluster; the job scheduling cluster is deployed and runs on the container orchestration cluster; Nodes in a container orchestration cluster are called orchestration nodes, which can be divided into orchestration compute nodes and orchestration management nodes. The orchestration management node is a key part responsible for the control and management of the entire cluster. It is responsible for coordinating all the work in the container orchestration cluster and ensuring the healthy and stable operation of the cluster.

[0033] The orchestration compute nodes are responsible for running applications, managing the network, handling resource allocation, and ensuring the high availability and resilience of the system.

[0034] Each component in the orchestration node exists as a container group, and each scheduling node in the job scheduling cluster also exists as a container group in the orchestration node.

[0035] The nodes in a job scheduling cluster are called scheduling nodes, which can be divided into scheduling computing nodes and scheduling management nodes.

[0036] The scheduling management node is responsible for managing the resources of the entire job scheduling cluster, scheduling jobs, and interacting with users. Typically, a job scheduling cluster has only one scheduling management node, but in some configurations, multiple scheduling management nodes can be used for redundancy.

[0037] The scheduling management node is deployed on the orchestration management node and exists in the form of a container group.

[0038] Scheduling compute nodes are the nodes in a job scheduling cluster that actually execute jobs. They are responsible for running user-submitted tasks. Scheduling compute nodes can be physical servers or virtual machines, and are typically configured with strong computing power and large amounts of memory.

[0039] The scheduling compute nodes are deployed on the orchestration compute nodes and also exist in the form of container groups.

[0040] The scheduling system also includes a scheduling control module, a resource management module, and a job scheduling module, all of which are located in the orchestration management node.

[0041] The scheduling control module is responsible for task scheduling, which includes a scheduling controller that exists in the form of a container group (specifically, slurm-scheduler).

[0042] The scheduling controller receives a task to be scheduled, determines the target orchestration compute node from the orchestration compute nodes in the container orchestration cluster according to the computing power specifications of the task to be scheduled, and submits the scheduling result indicating the target orchestration compute node to the deployment controller.

[0043] The resource management module may specifically include a deployment controller (specifically, a slurm-operator) and a synchronizer, which exist in the form of a container group.

[0044] The deployment controller is used to deploy target container groups on target orchestration compute nodes based on the custom resources of the tasks to be scheduled; the synchronizer is used to register the target container groups as scheduling compute nodes in the job scheduling cluster; wherein, the amount of computing resources allocated to the target container groups matches the computing power specification requirements.

[0045] The job scheduling module may specifically include scheduling management nodes that exist in the form of container groups.

[0046] Once the tasks to be scheduled have been synchronized to the job scheduling cluster by the synchronizer, the scheduling management node will schedule the tasks to be scheduled into the target container group for execution.

[0047] The following explanations will use Kubernetes clusters for container orchestration and Slurm clusters for job scheduling as examples.

[0048] In a Kubernetes cluster, the orchestration compute node is called a Kubernetes compute node; the orchestration management node is called a Kubernetes management node; the container group in a Kubernetes cluster is called a pod; the scheduling compute node in a Slurm cluster is called a Slurm compute node; and the scheduling management node in a Slurm cluster is called a Slurm management node.

[0049] See Figure 2 This application provides one of the schematic diagrams of the scheduling system obtained after deploying a Slurm cluster on a Kubernetes cluster.

[0050] The Kubernetes management node is a crucial part responsible for the control and management of the entire cluster. It coordinates all work within the Kubernetes cluster, ensuring its healthy and stable operation.

[0051] Each component in the Kubernetes management node exists as a container group (pod). The Kubernetes management node includes the Slurm management node, deployment controller, scheduling controller, and synchronizer of the job scheduling cluster, which exist as container groups (pods).

[0052] The deployment controller is an automatic deployment controller used to automate and manage the deployment and management of the Slurm job scheduling system within a Kubernetes cluster. Specifically, in the case of a Slurm cluster, this deployment controller is slurm-operator.

[0053] The slurm-operator typically requires the deployment of multiple components, such as the slurm controller (Slurmctld), slurm compute nodes, the scheduling controller slurm-scheduler, and a database. The slurm-operator automates the deployment process of these components, ensuring they function correctly within the Kubernetes cluster.

[0054] slurm-operator allows for automatic management and expansion of slurm clusters, supports dynamic scaling of compute nodes, and simplifies node management.

[0055] With slurm-operator, users can fully automate the lifecycle management of their slurm clusters, ensuring cluster stability and high availability. slurm-operator can work with other resources in the Kubernetes cluster, such as running jobs and tasks scheduled by the slurm cluster on the Kubernetes cluster. This allows Kubernetes cluster users to easily combine High Performance Computing (HPC) job scheduling with cloud-native technologies.

[0056] The scheduling controller is responsible for scheduling and allocating computing resources to ensure efficient execution of job tasks. It is an open-source, high-performance cluster job scheduler widely used in high-performance computing environments. In the case of a Slurm cluster, this scheduling controller is specifically slurm-scheduler.

[0057] slurm-scheduler is a scheduling manager developed for slurm clusters running on Kubernetes clusters. It manages and coordinates the scheduling of jobs or tasks, and is responsible for determining which jobs or tasks will run on which compute nodes in the cluster based on user requests and the availability of cluster resources.

[0058] The slurm-scheduler primarily receives user-submitted tasks to be scheduled, queues them based on cluster resource availability, and then schedules them. It ensures that jobs are scheduled according to certain priorities and resource requirements (such as CPU, memory, GPU, etc.).

[0059] During subsequent scheduling, slurm-operator will deploy the required target container group pod as a pod on the Kubernetes compute node, and the target container group pod will be registered as a slurm compute node of the slurm cluster.

[0060] The scheduling controller is used to read the task to be scheduled at the head of the task scheduling queue and obtain the specification requirements in the custom resources of the task to be scheduled. It determines a target Kubernetes compute node (corresponding to the aforementioned target orchestration computation) from the Kubernetes compute nodes that meet the specification requirements and submits the scheduling result for the target Kubernetes compute node to the deployment controller slurm-operator. The deployment controller, slurm-operator, is used to deploy a target container group pod on the target Kubernetes compute node based on the custom resources of the task to be scheduled and the scheduling result of the target Kubernetes compute node. It also registers the target container group pod as a slurm compute node and adds it to the slurm cluster through a synchronizer, and synchronizes the task to be scheduled to the job scheduling cluster. The system resources allocated to the target container group pod include computing resources, storage resources, etc. The amount of computing resources and storage resources are the resource amounts required by the specifications. For example, if the resource amounts required by the specifications are 8-core CPU and 32GB of memory, then the computing resources of the target container group pod are 8-core CPU and the storage resources are 32GB of memory.

[0061] The Slurm management node is used to schedule tasks to the target container group pod for execution when the tasks to be scheduled have been synchronized to the Slurm cluster by the synchronizer.

[0062] See Figure 3 This application provides a second schematic diagram of the composition of a scheduling system that includes a Kubernetes cluster and a Slurm cluster.

[0063] A Kubernetes cluster consists of Kubernetes compute nodes and Kubernetes management nodes.

[0064] Both Kubernetes compute nodes and Kubernetes management nodes have their own node hardware, such as a 32-core CPU, 256GB of memory, and other devices.

[0065] Because compute nodes have high computing power requirements, Kubernetes compute nodes can also be equipped with physical devices such as GPUs / NPUs (high-performance heterogeneous computing devices), which can be mounted on pods on Kubernetes compute nodes.

[0066] In this embodiment of the application, after deploying a Kubernetes cluster, a custom resource (CustomResourceDefinition, or SlurmCluster CR for short) of SlurmCluster (i.e. Slurm cluster) can be created on the Kubernetes cluster. The deployment controller slurm-operator is generated on the Kubernetes management node in the form of a pod, and then the slurmCluster CR is submitted to the Kubernetes cluster.

[0067] When custom resources are submitted in the job scheduling cluster, slurm-operator deploys a slurm management pod (corresponding to the slurm management node), a scheduler slurm-scheduler, and a synchronizer slurm-syncer on the Kubernetes management node based on the custom resources. All three also exist as pods on the Kubernetes management node. The pod corresponding to the slurm management node is also called the slurm management pod.

[0068] Slurm manages pods including container processes, requested or limited resource devices. Container processes include the process manager supervisor and the management terminal slurmctld. Requested or limited resource devices include, for example, an 8-core CPU, 32GB of memory, and other devices.

[0069] In addition, the Kubernetes management node can also include the monitoring center Prometheus (in the form of pods), various monitoring collectors (promlets), and HAMI (Heterogeneous AI Computing Virtualization Middleware) components (which can be used to partition computing resources).

[0070] The monitoring collector promlet can provide monitoring parameters for various resources on the Kubernetes management node it resides in. These parameters can include remaining parameters for resources such as CPU, memory, GPU / NPU, and disk, as well as performance parameters such as resource utilization, network performance, and disk performance.

[0071] Prometheus periodically pulls resource monitoring parameters from the promiselet and persists them to Kubernetes Persistent Volume Claims (PVCs).

[0072] Kubernetes compute nodes include regular Kubernetes pods, monitoring and data acquisition promlets that exist as pods, and GPU / NPU device plugins.

[0073] A typical Kubernetes pod includes a container process and requested or limited resource devices. Requested or limited resource devices generally include an 8-core CPU, 32GB of memory, and a full CPU / NPU card or partitioned CPU / NPU. The full CPU / NPU card or partitioned CPU / NPU can be mounted to the physical CPU / NPU provided by the Kubernetes compute node. Inference scripts within the container process can use this full CPU / NPU card or partitioned CPU / NPU.

[0074] In a scheduling system, a job is a computational unit requested by a user or application; it represents a workload that needs to be executed. In a job scheduling system, jobs are typically submitted to the scheduling system and managed and executed as a complete unit of work.

[0075] An assignment typically consists of multiple tasks.

[0076] A task is the smallest unit of execution in a job; it may be a step in executing a program, script, or computation. Tasks typically require specific computing resources (such as CPU, memory, etc.). Tasks awaiting scheduling are those that have not yet been executed.

[0077] In many job scheduling systems, tasks of a job may be executed in parallel to speed up computation.

[0078] After a user submits a custom resource (SlurmTask CR) for a task to be scheduled (SlurmTask) on a Kubernetes cluster, if the task has a priority, the slurm-operator will be triggered to insert it into a certain position in the task scheduling queue according to the priority configured in the task to be scheduled; if the task has no priority, the slurm-operator will be triggered to directly insert the task to be scheduled into the tail of the queue.

[0079] The slurm cluster's scheduling controller, slurm-scheduler, automatically reads the tasks at the head of the queue and schedules them in order, obtaining the custom resources for each task. This custom resource defines the rules and environment configurations required for the task. The specifications include the number of CPU cores, memory size, number of GPUs / NPUs, GPU / NPU memory size (supporting computing power partitioning), etc. The rules can be used as conditions to filter candidate Kubernetes compute nodes that meet the specifications from each Kubernetes compute node.

[0080] Then, slurm-scheduler selects a target Kubernetes compute node from among the candidate Kubernetes compute nodes. This target Kubernetes compute node is used to deploy the target container group pod that will execute the scheduled task.

[0081] It is understood that each candidate Kubernetes compute node is capable of creating the target container group pod. Therefore, theoretically, any candidate Kubernetes compute node can be selected as the target Kubernetes compute node. However, in order to improve task execution efficiency, this application embodiment calculates the node score of each candidate Kubernetes compute node based on the resource monitoring parameters of each candidate Kubernetes compute node.

[0082] slurm-operator periodically pulls resource monitoring parameters from Prometheus for each Kubernetes node and performs subsequent node score calculations.

[0083] The higher the node score, the more suitable the Kubernetes compute node is in terms of scheduling priority compared to other nodes. In other words, the node is more suitable to run a certain pod. The Kubernetes compute node with the highest node score can be used as the target Kubernetes compute node.

[0084] The scheduling controller slurm-scheduler determines the target Kubernetes compute node and submits the scheduling result for the target Kubernetes compute node to the slurm-operator. The scheduling result may include the node identifier of the target Kubernetes compute node. In other words, the scheduling result is used to instruct the creation of the target container group pod on the target Kubernetes compute node to execute the scheduled task.

[0085] The deployment controller slurm-operator can deploy a target container group pod (also known as a "slurm compute pod") on the target Kubernetes compute node based on the custom resources of the task to be scheduled and the scheduling result of the scheduling controller slurm-scheduler.

[0086] The Slurm compute pod also includes container processes and requested or restricted resource devices. The container processes include a process manager (supervisor), an execution endpoint (slurmd), and Slurm tasks (pytorch). (A Slurm task PyTorch refers to the process of running a PyTorch deep learning task in a Slurm cluster environment.) The Slurm task PyTorch can use requested or restricted resource devices, which may include a 16-core CPU, 128GB of memory, and a full CPU / NPU card or a partitioned CPU / NPU. The full CPU / NPU card or partitioned CPU / NPU can be mounted on the physical CPU / NPU provided by the Kubernetes compute node.

[0087] It is worth noting that the amount of system resources (CPU, memory, GPU, etc.) occupied by the target container group pod is the amount of resources required by the specification. That is, the amount of computing power and storage resources allocated to the target container group pod is the amount required by the specification of the task to be scheduled.

[0088] After the slurm-syncer detects that the target container group pod is running normally, it communicates with the slurm management pod (containing the slurmctld process) on the main Kubernetes management node to add the target container group pod as a slurm compute node to the slurm cluster and synchronize the tasks to be scheduled to the slurm cluster (specifically synchronized to the management slurmtld).

[0089] After the slurm management pod (specifically the slurmtld) obtains the task to be scheduled, it will schedule the task to the target container group pod created above, and execute it by the slurmd in the target container group pod. The detailed process will be described in the following section.

[0090] The scheduling system in this application includes a container orchestration cluster and a job scheduling cluster. The job scheduling cluster is deployed and run on the container orchestration cluster. During task scheduling, the scheduling control module responds to the received task to be scheduled and determines the target orchestration computing node from the orchestration computing nodes included in the container orchestration cluster according to the computing power specifications of the task to be scheduled. The resource management module deploys the target container group on the target orchestration computing node and registers the target container group as a scheduling computing node in the job scheduling cluster. The computing power resources allocated to the target container group match the computing power specifications. The job scheduling module distributes the task to be scheduled to the target container group for execution. That is, although the created scheduling computing node is exclusively used by the task to be scheduled, the computing power resources corresponding to the scheduling computing node are dynamically applied for and allocated based on demand, and are not all the system resources of the orchestration computing nodes in the container orchestration cluster. This effectively solves the problem of low computing power resource utilization in job scheduling clusters (such as the Slurm native cluster).

[0091] In some embodiments, the scheduling management node, deployment controller, scheduling controller, and synchronizer of the job scheduling cluster are deployed in the following manner: Create custom resources for job scheduling clusters on an existing container orchestration cluster, and create deployment controllers on the orchestration management node as container groups; When submitting custom resources to the job scheduling cluster, the deployment controller deploys container groups corresponding to the scheduling management node, scheduling controller, and synchronizer on the Kubernetes management node based on the custom resources.

[0092] As illustrated in the foregoing embodiments, the Kubernetes cluster has a custom deployment controller slurm-operator and a scheduling controller slurm-scheduler. After the deployment controller slurm-operator is deployed, it can automatically deploy the pods corresponding to the Slurm management node, scheduling controller, and synchronizer, thus avoiding the problem of Slurm's limited ability to schedule heterogeneous computing resources and reducing the workload of users' secondary development.

[0093] In some embodiments, the resource management module is also used to obtain custom resources for tasks to be scheduled submitted by the user to the scheduling control module; When custom resources include task priorities, the tasks to be scheduled are inserted into the task scheduling queue of the job scheduling cluster according to the task priorities. If the custom resource does not include task priority, insert the task to be scheduled into the tail of the job scheduling cluster. The scheduling control module is used to read the currently scheduled tasks from the head of the task scheduling queue.

[0094] The foregoing embodiments have already explained that after a user submits a custom resource (SlurmTask CR) for a task to be scheduled (SlurmTask) on a Kubernetes cluster, if the task has a priority, the scheduling controller slurm-operator will be triggered to insert it into a certain position in the task scheduling queue according to the priority in the task configuration; if the task has no priority, the slurm-operator will be triggered to directly insert the task to be scheduled into the tail of the queue, which will not be elaborated further here.

[0095] The scheduling controller can read the currently scheduled tasks from the head of the task scheduling queue in real time.

[0096] In some embodiments, the scheduling control module is used to select candidate orchestration computing nodes that meet the computing power specifications of the task to be scheduled from the orchestration computing nodes included in the container orchestration cluster. Access the node dynamic score queue and obtain the node score of each candidate orchestration compute node from the node dynamic score queue; the node score is calculated based on multiple resource monitoring parameters of the candidate orchestration compute node; The candidate orchestration computation node with the highest node score is determined as the target orchestration computation node.

[0097] See below Figure 4 This application provides a flowchart illustrating the process of constructing a dynamic score queue for nodes, involving the interaction between the deployment controller slurm-operator (which exists in the form of a pod), the monitoring center Prometheus, and various monitoring collectors' promises, including: 1. Prometheus periodically (e.g., every 5 seconds) automatically pulls the resource monitoring parameters provided by the Promlets of each monitoring collector deployed on each Kubernetes node and persists them to the Kubernetes PVC.

[0098] 2. slurm-operator periodically pulls monitoring parameters for monitored resources from Prometheus.

[0099] 3. Calculate and update the node score of each Kubernetes compute node based on the resource monitoring parameters, and store the node scores of each Kubernetes compute node in the node dynamic score queue in descending order of node score. The node dynamic score queue stores node 2 with a score of a, node 1 with a score of b, node 3 with a score of c, etc.

[0100] In some embodiments, the scheduling control module is used to maintain the node dynamic score queue based on the following method: Periodically obtain resource monitoring parameters for each orchestration computing node from the monitoring component; the resource monitoring parameters include at least one performance parameter and a resource remaining status parameter; Based on the resource monitoring parameters of each orchestration computing node, determine the parameter scores of each resource monitoring parameter of the orchestration computing node. The parameter scores and parameter weights of each resource monitoring parameter are weighted and summed to obtain the node score of the orchestration calculation node; The node scores of each orchestration calculation node are stored in the node dynamic score queue in descending order of node scores.

[0101] As illustrated in the foregoing embodiments, the scheduling controller acquires the performance parameters and resource utilization of each orchestration computing node in real time or periodically.

[0102] Performance parameters include various resource monitoring parameters, which may include parameters on the remaining status of resources such as CPU, memory, GPU / NPU, and disk, as well as performance parameters such as resource utilization, network performance, and disk performance.

[0103] Resource utilization can specifically include the idle rate of resources such as CPU, memory, and GPU / NPU. The higher the idle rate, the higher the score, as follows: For example, the CPU idle rate score cpu_score is represented by the following expression (1) or (2): cpu_score=(1-node_cpu_usage / total_node_cpu)*100(1), cpu_score=(1-pod_cpu_request / total_node_cpu)*100(2), Among them, node_cpu_usage represents the current CPU usage of the Kubernetes compute node, pod_cpu_request represents the CPU request or allocation of the Kubernetes compute node, node_cpu_usage and pod_cpu_request are theoretically the same, and total_node_cpu represents the total amount of CPUs owned by the Kubernetes compute node.

[0104] For example, the memory free time score (memory_score) is represented by the following expression (3) or (4): memory_score=(1-node_memory_usage / total_node_memory)*100(3), memory_score=(1-pod_memory_request / total_node_memory) * 100 (4), Among them, node_memory_usage represents the current memory usage of the Kubernetes compute node, pod_memory_request represents the memory request or allocation of the Kubernetes compute node, node_memory_usage and pod_memory_request are theoretically the same, and total_node_memory represents the total amount of memory owned by the Kubernetes compute node.

[0105] For example, the GPU idle rate score gpu_score is represented by the following expression (5) or (6): gpu_score=(1-node_gpu_usage / total_node_gpu)*100(5); gpu_score=(1-pod_gpu_request / total_node_gpu)*100(6); Among them, node_gpu_usage represents the current GPU usage of the Kubernetes compute node, pod_gpu_request represents the number of GPU requests or requests of the Kubernetes compute node, node_gpu_usage and pod_gpu_request are theoretically the same, and total_node_gpu represents the total number of GPUs owned by the Kubernetes compute node.

[0106] Network performance can be specifically determined by sub-items such as network bandwidth (maximum data transmission rate), network latency, network utilization, and packet loss rate of the network interface. Therefore, the network performance score can be the weighted sum of the network bandwidth sub-item score, network latency sub-item score, network utilization sub-item score, and packet loss rate sub-item score, respectively, and the sub-item weights. The sub-item weights can be set to 0.25 by default (the sum of the sub-item weights is 1, which can be set according to actual needs). The network performance can be specifically represented by the following expression (7): network_score=network_bandwidth_score*0.25+network_latency_score*0.25+network_throughput_score*0.25+network_loss_score*0.25 (7); Among them, network_score represents the network performance score, network_bandwidth_score represents the network bandwidth sub-score, network_latency_score represents the network latency sub-score, network_throughput_score represents the network utilization sub-score, and network_loss_score represents the packet loss rate sub-score.

[0107] network_bandwidth represents network bandwidth, which is the maximum data transfer rate of a network interface and the theoretical limit of network throughput. A reference threshold of 10Gbps (1250MB / s) is used.

[0108] Example of data collection (promQL): node_network_speed_bytes; It can be determined by the following expression (8): network_bandwidth_score=[min(network_bandwidth / 10000,1)]*100(8), As can be seen from formula (8), network_bandwidth_score has an upper limit, which is 100.

[0109] network_latency represents network latency, which is the round-trip time (RTT) of a data packet from source to destination. It directly affects response speed, with a reference threshold of 100ms.

[0110] Data acquisition example (promQL): Measurements must be taken using a dedicated probe or the ping command, such as: ping_latency_seconds{instance=~"$node"} It can be determined by the following expression (9): network_latency_score=[1-min(network_latency / 100,1)]*100(9), As can be seen from formula (9), the larger the network latency (network_latency) is, the smaller the network latency sub-item score (network_latency_score) is.

[0111] network_throughput represents network utilization and helps avoid congestion. A reference threshold of 80% of the theoretical bandwidth is used.

[0112] Data collection example (promQL): rate(node_network_receive_bytes_total[5m])+rate(node_network_transmit_bytes_total[5m]); It can be determined by the following expression (10): network_throughput_score=[1-min(network_throughput / throughput_max,1)]*100(10), Among them, throughput_max represents the maximum network utilization.

[0113] `network_packet_loss` represents the packet loss rate, which is the percentage of data packets lost during transmission and reflects network reliability. The reference threshold is 5% (0.05). Higher packet loss rates result in lower scores, with a score of 0 for rates exceeding 5%.

[0114] Data acquisition example (promQL): Measurements must be taken using a dedicated probe or the ping command, such as: ping_network_packet_loss{instance=~"$node"}; It can be characterized by the following expression (11): network_packet_loss_score=[1-min(network_packet_loss / 0.05,1)]*100(11).

[0115] Disk performance can be determined by a combination of sub-items such as disk utilization, number of read / write operations, and read / write latency. Therefore, the disk performance score can be the weighted sum of the scores for disk utilization, number of read / write operations, and read / write latency, respectively, and their respective weights. The weights of the sub-items can be set to 1 / 3 by default (but can be set according to actual needs; the sum of the weights of the sub-items is 1). Network performance can be characterized by the following expression (12): disk_score=disk_util_score+disk_iops_score+disk_latency_score(12), Among them, disk_util_score represents the score of the disk utilization sub-item; disk_iops_score represents the score of the number of read and write operations sub-item; and disk_latency_score represents the score of the read and write latency sub-item.

[0116] disk_util represents disk utilization, indicating the percentage of time spent on disk I / O, reflecting disk activity levels. Higher utilization results in a lower score. A reference threshold of 90% is generally considered an excessive load.

[0117] Example of disk_util data collection (promQL): node_disk_io_time_seconds_total; disk_util_score can be determined by the following formula (13): disk_util_score=[1-min(disk_util / 90,1)]*100(13).

[0118] disk_iops represents the number of read and write operations per second, measuring random read and write processing capability. The reference threshold (iops_max) is set according to the disk type (e.g., SSD: 20000, HDD: 200). When the value reaches or exceeds the theoretical maximum value, the score is 1.

[0119] Data collection example (promQL): rate(node_disk_reads_completed_total[5m])+rate(node_disk_writes_completed_total[5m]); disk_iops_score can be determined by the following formula (14): disk_iops_score=min(disk_iops / iops_max,1)(14).

[0120] disk_latency represents read / write latency, which is the average time (ms) of each I / O operation and directly affects response speed. The reference threshold latency_max is set according to the disk type (e.g., SSD: 20ms, HDD: 50ms).

[0121] Data collection example (promQL): (rate(node_disk_io_time_seconds_total[5m]) / rate(node_disk_reads_completed_total[5m]+node_disk_writes_completed_total[5m]))*1000; disk_latency_score can be determined by the following expression (15): disk_latency_score=1-min(disk_latency / latency_max,1)(15).

[0122] After obtaining the parameter scores of each resource monitoring parameter, the preset parameter weights for each resource monitoring parameter can be obtained. The parameter scores and parameter weights of each resource monitoring parameter are then weighted and summed to obtain the node score of the Kubernetes compute node, which can be represented by the following expression (16): Node score = Resource remaining parameters (CPU, memory, GPU / NPU) * Usage weight + Resource utilization rate * Resource utilization rate weight + Network performance * Network performance weight + Disk performance * Disk performance weight (16). Usage weights, resource utilization weights, network performance weights, and disk performance weights can be set according to actual conditions, and there are no restrictions on them.

[0123] The node score of each Kubernetes compute node can be determined based on the above expressions (1)-(16), and the node scores can be stored in the node dynamic score queue in order of score from high to low or from low to high.

[0124] In some embodiments, the resource management module is used to utilize container resource isolation mechanisms or device virtualization plugins to divide target virtual computing power slices that meet computing power specifications from the physical computing power cards on the target orchestration computing nodes; and to bind the target virtual computing power slices as computing power resources to the target container group for sharing and reuse of computing power resources.

[0125] The resource management module is used to divide the target virtual computing power slices that meet the computing power specifications from the physical computing power cards on the target orchestration computing nodes using container resource isolation mechanisms or device virtualization plugins (such as the HAMI plugin). That is, it divides the computing power resource quantity required by the computing power specifications and the storage resource quantity required by the storage specifications. The computing power resource quantity refers to the amount of computing power resources, and the storage resource quantity refers to the amount of storage resources.

[0126] Physical computing cards can be, for example, CPUs, GPUs, NPUs, etc.

[0127] The target virtual computing power slice can perfectly meet the computing power rule requirements. The target virtual computing power slice can be bound to the target container group as a computing power resource. Although the virtual computing power slice of the target container group is subsequently exclusively occupied by the task to be scheduled, it does not affect the allocation of other virtual computing power slices on the target orchestration computing node to other tasks.

[0128] In some embodiments, the resource management module is used to set the node name of the scheduling computing node corresponding to the target container group to the task name of the task to be scheduled when registering the target container group as a scheduling computing node in the job scheduling cluster. The job scheduling module is used to find the target container group from the job scheduling cluster based on the task name of the task to be scheduled, and distribute the task to be scheduled to the target container group for execution.

[0129] As described in the foregoing embodiments, a target container group pod will be created in the target orchestration compute node to execute the task to be scheduled. When registering the target container group as a scheduling compute node in the job scheduling cluster, the name of the scheduling compute node corresponding to the target container group pod can be directly named as the task name of the task to be scheduled.

[0130] The slurm management pod corresponding to the scheduling management node includes the management terminal slurmctld, which is used for scheduling management.

[0131] When the management end performs node scheduling, slurmctld can directly find the scheduling compute node corresponding to the target container group pod in the job scheduling cluster based on the task name of the task to be scheduled. The node name of the scheduling compute node corresponding to the target container group pod is the same as the task name, which can reduce the matching time of slurm's native scheduling and speed up the task process.

[0132] Specifically, the task to be scheduled is executed by the slurmd execution terminal in the scheduling compute node corresponding to the target container group pod.

[0133] In some embodiments, the job scheduling module includes a container process manager; Once the scheduled tasks have been completed, the target container group sends a task completion semaphore to the container process manager, causing the container process manager to exit the main process. When the main process exits, the target container group is automatically destroyed and releases its computing resources. When the resource management module detects the destruction semaphore of the target container group, it deregisters the scheduling computing node corresponding to the target container group from the job scheduling cluster.

[0134] The task status can be monitored through the Sidecar container or a specific process. After the scheduled task is completed, slurm-syncer can quickly detect it through the Sidecar container or a specific process, and release relevant system resources and maintain slurm cluster information. The specific process is as follows: after the slurm task in the target container group is completed, it notifies the supervisor to exit the main process through the completion semaphore. The target container group will be automatically destroyed and resources will be released. The life cycle of the target container group ends.

[0135] After receiving the slurm-pod destruction semaphore, slurm-syncer notifies slurmctld to update the slurm cluster information and deregister the slurm compute node corresponding to the target container group, thereby releasing the occupied resources in a timely manner.

[0136] See Figure 5 This application provides an embodiment of the interaction diagram of each part of the scheduling system, including the following process: Step 501: The user submits custom resources for the task to be scheduled to the deployment controller slurm-operator of the Kubernetes cluster; Step 502: The pod-based deployment controller slurm-operator inserts the tasks to be scheduled into the job scheduling queue according to task priority. The job scheduling queue includes task 1 with priority a, task 3 with priority b, task 2 with priority c, and so on. Step 503: The pod-based scheduling controller slurm-scheduler retrieves the highest-priority scheduled task (i.e., task 1) from the job scheduling queue. Step 504: The pod-based scheduling controller slurm-scheduler filters out all candidate Kubernetes compute nodes that meet the specification requirements of the custom resources to be scheduled, and accesses the node dynamic score queue to select the candidate Kubernetes compute node with the highest node score as the target Kubernetes compute node. The node dynamic score queue includes the node scores of each Kubernetes compute node, which are arranged in descending order as node 2 with score a, node 1 with score b, node 3 with score c, and so on. Step 505: The pod-based scheduling controller slurm-scheduler submits the scheduling results for the target Kubernetes compute node to the deployment controller slurm-operator; Step 506: The pod-based deployment controller slurm-operator deploys a target slurm compute pod (the aforementioned target container group) on the target Kubernetes compute node. Step 507, the pod-type synchronizer slurm-syncer automatically creates a target slurm computing pod upon detecting that one has been created; Step 508: The pod-based synchronizer slurm-syncer adds the target slurm compute pod to the slurm cluster as a slurm compute node and synchronizes the tasks to be scheduled to the slurm cluster. Step 509: The pod-based scheduling management node (Slurm-managed pod) searches for the target scheduling compute pod in the Slurm cluster based on the task name of the task to be scheduled, and schedules the task to be scheduled to the target scheduling compute pod. Step 510: Schedule the task to be scheduled to the execution end slurmd of the target scheduling computing pod to execute the task to be scheduled (i.e., task 1).

[0137] The detailed execution process of each step is described in the aforementioned embodiments and will not be repeated here.

[0138] See Figure 6 This application provides a flowchart illustrating a scheduling method for a scheduling system. The scheduling system includes a container orchestration cluster and a job scheduling cluster. The job scheduling cluster is deployed and runs on the container orchestration cluster. The scheduling system also includes a scheduling control module, a resource management module, and a job scheduling module.

[0139] The scheduling method includes the following steps: Step 610: In response to the received task to be scheduled, the scheduling control module determines the target orchestration computing node from the orchestration computing nodes contained in the container orchestration cluster according to the computing power specifications of the task to be scheduled. Step 620: Deploy the target container group on the target orchestration computing node through the resource management module, and register the target container group as a scheduling computing node in the job scheduling cluster; wherein, the amount of computing resources allocated to the target container group matches the computing power specification requirements; Step 630: Distribute the tasks to be scheduled to the target container group for execution through the job scheduling module.

[0140] The detailed execution process of steps 610 to 630 is described in the aforementioned embodiments and will not be repeated here.

[0141] In some embodiments, deploying a target container group on a target orchestration compute node via a resource management module includes: By utilizing container resource isolation mechanisms or device virtualization plugins through the resource management module, target virtual computing power slices that meet the computing power specifications are divided from the physical computing power cards on the target orchestration computing nodes; the target virtual computing power slices are then bound as computing power resources to the target container group for sharing and reuse of computing power resources.

[0142] In some embodiments, determining the target orchestration compute node from among the orchestration compute nodes included in the container orchestration cluster includes: Candidate orchestration compute nodes that meet the computing power specifications of the task to be scheduled are selected from the orchestration compute nodes contained in the container orchestration cluster. Access the node dynamic score queue and obtain the node score of each candidate orchestration compute node from the node dynamic score queue; the node score is calculated based on multiple resource monitoring parameters of the candidate orchestration compute node; The candidate orchestration computation node with the highest node score is determined as the target orchestration computation node.

[0143] In some embodiments, the scheduling control module maintains the node dynamic score queue in the following manner: Periodically obtain resource monitoring parameters for each orchestration computing node from the monitoring component; the resource monitoring parameters include at least one performance parameter and a resource remaining status parameter; Based on the resource monitoring parameters of each orchestration computing node, determine the parameter scores of each resource monitoring parameter of the orchestration computing node. The parameter scores and parameter weights of each resource monitoring parameter are weighted and summed to obtain the node score of the orchestration calculation node; The node scores of each orchestration calculation node are stored in the node dynamic score queue in descending order of node scores.

[0144] In some embodiments, registering the target container group as a scheduling compute node in a job scheduling cluster includes: When registering a target container group as a scheduling compute node in a job scheduling cluster through the resource management module, the node name of the scheduling compute node corresponding to the target container group is set to the task name of the task to be scheduled. The job scheduling module distributes the tasks to be scheduled to the target container group for execution, including: The job scheduling module searches for the target container group from the job scheduling cluster based on the task name of the task to be scheduled, and then distributes the task to be scheduled to the target container group for execution.

[0145] In some embodiments, the job scheduling module includes a container process manager; The method also includes: Once the scheduled task has been completed, the target container group sends a task completion semaphore to the container process manager, causing the container process manager to exit the main process. The target container group can be automatically destroyed and its computing resources released when the main process exits. When the resource management module detects the destruction semaphore of the target container group, it deregisters the scheduling computing node corresponding to the target container group from the job scheduling cluster.

[0146] In some embodiments, the method further includes: The resource management module is also used to obtain custom resources submitted by users to the scheduling control module for tasks to be scheduled; When custom resources include task priorities, the tasks to be scheduled are inserted into the task scheduling queue of the job scheduling cluster according to the task priorities. If the custom resource does not include task priority, insert the task to be scheduled into the tail of the task scheduling queue; The scheduling control module responds to the received tasks to be scheduled, including The scheduling control module reads the current tasks to be scheduled from the head of the task scheduling queue.

[0147] For a detailed description of the execution process of the scheduling method, please refer to the execution process of each model in the aforementioned scheduling system; it will not be repeated here.

[0148] The scheduling method of the scheduling system provided in this application can be executed by the scheduling device of the scheduling system. This application uses the example of the scheduling device of the scheduling system executing the scheduling method to illustrate the scheduling device of the scheduling system provided in this application.

[0149] This application also provides a scheduling device for a scheduling system. The scheduling system includes a container orchestration cluster and a job scheduling cluster. The job scheduling cluster is deployed and runs on the container orchestration cluster. The scheduling system also includes a scheduling control module, a resource management module, and a job scheduling module.

[0150] like Figure 7 As shown, the scheduling device includes: a first processing module 710, a second processing module 720 and a third processing module 730.

[0151] The first processing module 710 is used to respond to the received task to be scheduled through the scheduling control module, and determine the target orchestration computing node from the orchestration computing nodes contained in the container orchestration cluster according to the computing power specifications of the task to be scheduled. The second processing module 720 is used to deploy the target container group on the target orchestration computing node through the resource management module, and register the target container group as a scheduling computing node in the job scheduling cluster; wherein the amount of computing resources allocated to the target container group matches the computing power specification requirements. The third processing module 730 is used to distribute the tasks to be scheduled to the target container group for execution through the job scheduling module.

[0152] The detailed execution process of each module of the scheduling device can be found in the aforementioned scheduling method, and will not be repeated here.

[0153] The scheduling device of the scheduling system in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM or self-service machine, etc. The embodiments of this application do not specifically limit it.

[0154] The scheduling device of the scheduling system in this application embodiment can be a device with an operating system. The operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems. This application embodiment does not specifically limit the specific operating system.

[0155] In some embodiments, such as Figure 8 As shown, this application embodiment also provides an electronic device 800, including a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801. When the program is executed by the processor 801, it implements the various processes of the scheduling method embodiment of the above-mentioned scheduling system and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0156] Processor 801 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 801 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0157] The memory 802 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium capable of carrying or storing computer programs and capable of being read by a computer, without limitation herein.

[0158] The memory 802 is used to store computer programs that execute the embodiments of this application, and the execution is controlled by the processor 801. The processor 801 is used to execute the computer programs stored in the memory 802 to implement the steps shown in the foregoing method embodiments.

[0159] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0160] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the scheduling method embodiment of the above-described scheduling system and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0161] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0162] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the scheduling method of the above-described scheduling system.

[0163] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0164] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the scheduling method embodiment of the above-described scheduling system, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0165] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0166] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0167] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0168] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0169] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0170] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A scheduling system, characterized in that, The scheduling system includes a container orchestration cluster and a job scheduling cluster, wherein the job scheduling cluster is deployed and runs on the container orchestration cluster; The scheduling system also includes: The scheduling control module is used to respond to the received task to be scheduled and determine the target orchestration computing node from the orchestration computing nodes included in the container orchestration cluster according to the computing power specifications of the task to be scheduled. The resource management module is used to deploy target container groups on the target orchestration computing nodes and register the target container groups as scheduling computing nodes in the job scheduling cluster; wherein the amount of computing resources allocated to the target container groups matches the computing power specification requirements; The job scheduling module is used to distribute the tasks to be scheduled to the target container group for execution.

2. The scheduling system according to claim 1, characterized in that, The resource management module is used to divide the target virtual computing power slices that meet the computing power specifications from the physical computing power cards on the target orchestration computing node by using container resource isolation mechanisms or device virtualization plugins. The target virtual computing power slice is bound to the target container group as a computing power resource to enable the sharing and reuse of computing power resources.

3. The scheduling system according to claim 1, characterized in that, The scheduling control module is used to select candidate orchestration computing nodes that meet the computing power specifications of the task to be scheduled from the orchestration computing nodes included in the container orchestration cluster. Access the node dynamic score queue and obtain the node score of each candidate orchestration calculation node from the node dynamic score queue; The node score is calculated based on multiple resource monitoring parameters of the candidate orchestration calculation node; The candidate orchestration computing node with the highest node score is determined as the target orchestration computing node.

4. The scheduling system according to claim 3, characterized in that, The scheduling control module is used to maintain the node dynamic score queue based on the following method: The system periodically retrieves resource monitoring parameters for each orchestration computing node from the monitoring component; these resource monitoring parameters include at least one performance parameter and a resource remaining status parameter. Based on the resource monitoring parameters of each orchestration computing node, the parameter scores of each resource monitoring parameter of the orchestration computing node are determined respectively. The node score of the orchestration calculation node is obtained by weighted summation of the parameter scores and parameter weights of each resource monitoring parameter. The node scores of each orchestration calculation node are stored in the node dynamic score queue in descending order of node scores.

5. The scheduling system according to claim 1, characterized in that, The resource management module is used to set the node name of the scheduling computing node corresponding to the target container group to the task name of the task to be scheduled when registering the target container group as a scheduling computing node in the job scheduling cluster. The job scheduling module is used to find the target container group from the job scheduling cluster according to the task name of the task to be scheduled, and distribute the task to be scheduled to the target container group for execution.

6. The scheduling system according to claim 5, characterized in that, The job scheduling module includes a container process manager; When the scheduled task is completed, the target container group sends a task completion semaphore to the container process manager so that the container process manager exits the main process. When the main process exits, the target container group is automatically destroyed and releases its computing resources. When the resource management module detects the destruction semaphore of the target container group, it deregisters the scheduling computing node corresponding to the target container group from the job scheduling cluster.

7. The scheduling system according to any one of claims 1-6, characterized in that, The resource management module is also used to obtain the custom resources of the task to be scheduled submitted by the user to the scheduling control module; If the custom resource includes task priority, the task to be scheduled is inserted into the task scheduling queue of the job scheduling cluster according to the task priority. If the custom resource does not include task priority, the task to be scheduled is inserted at the tail of the task scheduling queue; The scheduling control module is used to read the current tasks to be scheduled from the head of the task scheduling queue.

8. A scheduling method for a scheduling system, characterized in that, The scheduling system includes a container orchestration cluster and a job scheduling cluster; the job scheduling cluster is deployed and runs on the container orchestration cluster. The scheduling system also includes a scheduling control module, a resource management module, and a job scheduling module; The scheduling method includes: In response to the received task to be scheduled, the scheduling control module determines the target orchestration computing node from the orchestration computing nodes included in the container orchestration cluster according to the computing power specifications of the task to be scheduled. The resource management module deploys the target container group on the target orchestration computing node and registers the target container group as a scheduling computing node in the job scheduling cluster; wherein the amount of computing resources allocated to the target container group matches the computing power specification requirements; The job scheduling module distributes the tasks to be scheduled to the target container group for execution.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the scheduling method of the scheduling system as described in claim 8.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the scheduling method of the scheduling system as described in claim 8.

Images