A resource scheduling method and device, electronic equipment and storage medium

Abstract

The application provides a resource scheduling method and device, electronic equipment and storage medium. The method comprises: in response to a partition configuration operation, dividing candidate resource nodes in a resource node cluster according to a partition label corresponding to the partition configuration operation to obtain divided candidate resource partitions; in response to a restriction information configuration operation, establishing a corresponding candidate resource pool for each candidate resource partition according to the configured restriction information; when a first workload is received, allocating a corresponding first resource pool for the first workload according to a resource pool identifier contained in the first workload; wherein the first workload comprises a first task or a plurality of first container groups; and scheduling a first resource node for the first workload according to real-time load information of each candidate resource node in the corresponding candidate resource partition of the first resource pool. The resource allocation strategy can be automatically adjusted according to the real-time load to ensure that the system can still operate efficiently when the load changes.

Description

Technical Field

[0001] This application relates to the field of resource scheduling technology, and more specifically, to a resource scheduling method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of artificial intelligence technology, the demand for computing resources for AI training and inference tasks is surging, especially relying on heterogeneous acceleration resources such as GPUs and TPUs. These resources are characterized by scarcity, high cost, and difficulty in fragmentation, which, unlike easily scheduled CPU resources, places higher demands on the scheduling system.

[0003] Currently, Kubernetes, as the mainstream container orchestration platform, has a mature default scheduler, kube-scheduler, which performs well in general scenarios. However, it has significant limitations in AI training and push scenarios. On the one hand, it uses a single Pod as the scheduling unit, which makes it difficult to adapt to the collaborative scheduling needs of a group of related Pods in an AI task. Moreover, relying solely on resource request status (request / limit) can easily lead to excessive resource reservation and waste. On the other hand, some solutions that support real-time load scheduling are mostly limited to CPU and memory, ignoring the GPU load information that is the core of AI tasks, and thus cannot meet the needs of heterogeneous resource scheduling. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, apparatus, electronic device and storage medium for resource scheduling to overcome the problems in the prior art.

[0005] In a first aspect, embodiments of this application provide a resource scheduling method, the method comprising: In response to the partition configuration operation, the candidate resource nodes in the resource node cluster are divided according to the partition label corresponding to the partition configuration operation to obtain the divided candidate resource partitions. In response to the restriction information configuration operation, a corresponding candidate resource pool is established for each candidate resource partition according to the configured restriction information; wherein, the candidate resource pool represents the resource scheduling range and resource usage quota of the candidate resource; Upon receiving a first workload, a corresponding first resource pool is allocated to the first workload based on the resource pool identifier contained in the first workload; wherein, the first workload includes a first task or multiple first container groups; Based on the real-time load information of each candidate resource node in the candidate resource partition corresponding to the first resource pool, the first resource node is scheduled for the first workload.

[0006] In some technical solutions of this application, the above method obtains the first workload in the following manner: In response to the first task creation operation, construct the first workload; Alternatively, in response to a container group creation operation, a corresponding target container group may be generated; The target container group and the associated container group are used as the first container group to obtain the first workload; wherein, the associated container group is a container group that is associated with the target container group.

[0007] In some technical solutions of this application, the above method also includes: In response to the priority level setting operation, a corresponding priority level is set for the first workload.

[0008] In some technical solutions of this application, the above-mentioned scheduling of the first resource node for the first workload based on the real-time load information of each of the candidate resource nodes in the candidate resource partition corresponding to the first resource pool includes: When the first resource pool can meet the scheduling requirements of the first workload, the candidate resource node with the lowest resource utilization rate is selected as the first resource node from the candidate resource nodes in the candidate resource partition corresponding to the first resource pool.

[0009] In some technical solutions of this application, the above method also includes: If the first resource pool is unable to meet the scheduling requirements of the first workload, the resources in the first resource pool are released according to the first priority level corresponding to the first workload. After the resources are released, the resource node with the lowest resource utilization rate is selected as the first resource node from the candidate resource nodes in the candidate resource partition corresponding to the first resource pool.

[0010] In some technical solutions of this application, the above-mentioned resource release of the first resource pool according to the first priority level corresponding to the first workload includes: Identify the groups of containers in the first resource pool that are lower than the first priority level and are to be released; Resources are released sequentially according to the priority level of the containers to be released, until the first resource pool can meet the scheduling requirements of the first workload.

[0011] In some technical solutions of this application, the above method also includes: Upon receiving a second workload, a corresponding second resource pool is allocated to the second workload based on the resource pool identifier contained in the second workload; wherein, the second workload includes a second task or multiple second container groups; Based on the real-time load information of each candidate resource node in the candidate resource partition corresponding to the second resource pool, the second resource node is scheduled for the second workload.

[0012] Secondly, embodiments of this application provide a resource scheduling apparatus, the apparatus comprising: The first response module is used to respond to the partition configuration operation. Based on the partition label corresponding to the partition configuration operation, it divides the candidate resource nodes in the resource node cluster to obtain the divided candidate resource partitions. The second response module is used to respond to the restriction information configuration operation and, according to the configured restriction information, establish a corresponding candidate resource pool for each candidate resource partition; wherein, the candidate resource pool represents the resource scheduling range and resource usage quota of the candidate resource; The allocation module is configured to, upon receiving a first workload, allocate a corresponding first resource pool to the first workload based on the resource pool identifier contained in the first workload; wherein the first workload includes a first task or multiple first container groups. The scheduling module is used to schedule the first resource node for the first workload based on the real-time load information of each of the candidate resource nodes in the candidate resource partition corresponding to the first resource pool.

[0013] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the resource scheduling method described above.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the resource scheduling method described above.

[0015] The technical solutions provided by the embodiments of this application may include the following beneficial effects: This application's method includes: responding to a partition configuration operation, dividing candidate resource nodes in a resource node cluster according to the partition label corresponding to the partition configuration operation to obtain partitioned candidate resource partitions; responding to a restriction information configuration operation, establishing corresponding candidate resource pools for each candidate resource partition according to the configured restriction information; wherein, the candidate resource pool represents the resource scheduling range and resource usage quota of the candidate resources; upon receiving a first workload, allocating a corresponding first resource pool to the first workload according to the resource pool identifier included in the first workload; wherein, the first workload includes a first task or multiple first container groups; scheduling first resource nodes for the first workload according to the real-time load information of each candidate resource node in the candidate resource partition corresponding to the first resource pool. This method can automatically adjust the resource allocation strategy according to the real-time load, ensuring that the system can still operate efficiently when the load changes.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a resource scheduling method provided in an embodiment of this application is shown. Figure 2 A schematic diagram of the first embodiment provided by the present application is shown; Figure 3 A schematic diagram of the second embodiment provided in this application is shown; Figure 4 This illustration shows a schematic diagram of extracting real-time load information according to an embodiment of this application; Figure 5 A schematic diagram of a third embodiment provided in this application is shown; Figure 6 A schematic diagram of the fourth embodiment provided in this application is shown; Figure 7 A schematic diagram of a resource scheduling apparatus provided in an embodiment of this application is shown; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0020] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0022] With the rapid development of artificial intelligence technology, the demand for computing resources, data management, and deployment flexibility is constantly increasing. Against this backdrop, Kubernetes, as a leader in container orchestration, is gradually becoming the core infrastructure supporting AI applications, thanks to its powerful resource management, automated deployment, and elastic scaling capabilities. AI applications (such as model training) typically require substantial computing resources, especially relying on heterogeneous acceleration resources such as GPUs and TPUs. These resources are scarce, expensive, and not easily sharded, unlike CPUs which are readily scheduled. To meet the complex scheduling needs of AI applications, Kubernetes' scheduling system is evolving from a single scheduler model to a multi-scheduler, multi-scenario architecture. The default scheduler, kube-scheduler, is mature, suitable for most general-purpose scenarios, and supports extending scheduling policies through a plugin mechanism, making it a core scheduling component in current mainstream production environments. Essentially, kube-scheduler is designed for general-purpose applications. For task scheduling in AI training and push environments, which require more reliance on GPU computing resources, kube-scheduler becomes somewhat inadequate.

[0023] General-purpose schedulers in AI training and inference environments typically only support one aspect. For example, the default scheduler only considers resource requests (requests / limits) when scheduling metrics, often leading to excessive resource reservation and waste. Some schedulers can also schedule based on real-time load, but they mostly only consider CPU and memory load, ignoring GPU load, which is insufficient for the scheduling needs of AI environments. Furthermore, in AI training and inference environments, training and inference tasks have varying priorities. With relatively limited GPU resources, ensuring smooth task execution—that is, prioritizing high-priority tasks and allowing low-priority tasks to be preempted as needed—is crucial. While some schedulers implement priority scheduling, the implementation often arbitrarily evicts low-priority tasks. While this satisfies the requirement of prioritizing high-priority tasks, it can cause low-priority tasks to become unstable.

[0024] Based on this, embodiments of this application provide a resource scheduling method, apparatus, electronic device, and storage medium that can automatically adjust resource allocation strategies according to real-time load, ensuring efficient system operation even under varying load conditions. Simultaneously, computing resources are dynamically allocated based on task priority and real-time resource status, enabling rapid response for high-priority tasks while ensuring stable operation of low-priority tasks. The following description uses embodiments to illustrate this.

[0025] Figure 1The diagram illustrates a flowchart of a resource scheduling method provided in an embodiment of this application, wherein the method includes steps S101-S104; specifically: S101. In response to the partition configuration operation, the candidate resource nodes in the resource node cluster are divided according to the partition label corresponding to the partition configuration operation to obtain the divided candidate resource partitions. S102. In response to the restriction information configuration operation, according to the configured restriction information, establish a corresponding candidate resource pool for each candidate resource partition; wherein, the candidate resource pool represents the resource scheduling range and resource usage quota of the candidate resource; S103. Upon receiving the first workload, allocate a corresponding first resource pool to the first workload based on the resource pool identifier contained in the first workload; wherein, the first workload includes a first task or multiple first container groups. S104. Based on the real-time load information of each candidate resource node in the candidate resource partition corresponding to the first resource pool, schedule the first resource node for the first workload.

[0026] The following describes some embodiments of this application in detail. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0027] This application provides a resource scheduling method based on Kubernetes. Kubernetes (K8s) is an open-source container orchestration platform used to automate the deployment, scaling, and management of containers. It aims to provide a platform for container scheduling, orchestration, high availability, and automation across host clusters, allowing users to more easily manage the lifecycle of applications.

[0028] The existing Kubernetes system comprises several core components, primarily: kube-apiserver: As the front-end interface of the entire system, it is responsible for handling requests from users and other components, providing RESTful API services, and serving as the hub for communication between components.

[0029] etcd: A distributed key-value store system used to store all configuration information of a Kubernetes cluster; it is the cluster's database.

[0030] kube-scheduler: Responsible for scheduling cluster resources, scheduling Pods to run on appropriate nodes according to predefined scheduling policies.

[0031] kube-controller-manager: Runs a series of controller processes, such as node controllers, replica controllers, and endpoint controllers, which are responsible for maintaining the state of the cluster and ensuring that the actual state is consistent with the expected state.

[0032] kubelet: An agent program that runs on each node and is responsible for maintaining the lifecycle of containers and ensuring that containers run according to the Pod specifications.

[0033] kube-proxy: Runs on each node and is responsible for maintaining the network rules on the node, enabling communication between Pods and between Pods and the external network.

[0034] In addition, Kubernetes defines a series of abstract concepts, such as Pod (the smallest deployment unit, containing one or more containers), Service (providing a stable network access point for Pods), Deployment (declaratively defining Pods and ReplicaSets for managing stateless applications), StatefulSet (for managing stateful applications, providing stable network identification and storage for Pods), ConfigMap and Secret (for configuration management, storing configuration information and sensitive data required by applications), etc. These concepts together constitute the core functional system of Kubernetes.

[0035] Kubernetes, as a resource node cluster, consists of multiple candidate resource nodes. Kubernetes manages and controls these nodes through its core components (such as kubelet and kube-proxy). Specifically, a Kubernetes cluster comprises multiple nodes that serve as the actual computing resource carriers, running containerized applications (such as Pods).

[0036] In existing technologies, Kubernetes scheduling affects all of its resources. To avoid this impact on all Kubernetes resources during resource scheduling, this application's embodiments first partition the candidate resource nodes in Kubernetes before scheduling, such as... Figure 2 As shown.

[0037] In this embodiment, the partitioning of Kubernetes candidate resource nodes is based on user-defined criteria. Specifically, users can set corresponding partition labels for each candidate resource node according to the specific needs of their business system, and then divide the candidate resource nodes into multiple candidate resource partitions based on these labels. These partition labels can be set according to the attributes of the candidate resource nodes or set by the user. For example, CPU and GPU labels can be set to divide the system into CPU partitions, or CPU1 and CPU2 can be set to divide the system into first and second CPU partitions. Labels can also be set according to the purpose of the nodes, dividing them into Ascend partitions, compute partitions, elastic partitions, etc. An Ascend partition generally refers to a specific area allocated for related computing tasks of the Ascend AI processor in a system using an Ascend processor. A compute partition typically refers to an area specifically used for handling regular computing tasks. An elastic partition is a partitioning method that dynamically adjusts resource allocation based on system load and demand. By operating on partitions, it is possible to avoid the impact of resource scheduling in the current partition on other partitions. For example, resource scheduling in the first partition will not affect the resources in the second partition.

[0038] After partitioning, to avoid resource waste in each partition, this embodiment of the application also sets up a candidate resource pool for each candidate resource partition. Here, the candidate resource pool represents the resource scheduling range and resource usage limit of the candidate resources. That is, when a user uses resources, they can only use them within the restrictions of the corresponding resource pool, avoiding the situation where they occupy all or most of the resources, leaving no resources available for other users.

[0039] When setting up a candidate resource pool for a candidate resource partition, the setting basis in this embodiment is also user-defined. Users set corresponding restriction information for different candidate resource partitions based on the needs of their specific business systems, and construct the candidate resource pool corresponding to that partition based on the restriction information. In specific implementation, the restriction information here can be a specific threshold. For example, the restriction information could be 5c, 16G, 2*8G GPU (5c means the resource pool can use the computing resources of 5 CPU cores in the corresponding partition; 16G means the resource pool can use 16GB of memory resources in the corresponding partition; 8G means the video memory capacity of a single GPU is 8GB; "2*" means the resource pool can use 2 such GPUs in the corresponding partition, i.e., a total of 16GB of GPU resources).

[0040] Create corresponding resource pools on top of the partitions. The resource pools are configured with the quota information of the partitions. For example, a shared resource pool contains quotas of 5c, 16G, and 2*8G GPUs for elastic partitions.

[0041] When partitioning and creating resource pools, you can follow these steps: First, define two third-party resources: Partition and ResourcePool. Partition represents a partition, used to organize multiple nodes into a single partition. ResourcePool is used to divide the resources within a partition. For example... Figure 3 As shown, a task (workload) specifies a resource pool, which references an associated partition. The partition references an associated node based on its nodeLabel. This ensures that the task can only be deployed within the scope of the associated partition.

[0042] The following is an example of a custom resource: apiVersion: gigentech.com / v1beta1 kind: Partition metadata: name: "gpu-partition" spec: nodeLabels: node-type: node-910b The nodeLabels parameter specifies the labels of the nodes.

[0043] apiVersion: gigentech.com / v1beta1 kind: ResourcePool metadata: name: "share-pool" spec: preemption: policy: LowerPriority / Never resources: - name: "gpu-partition" resources: - name: "cpu" Quota: 6 - name: "memory" Quota: 12Gi - name: "gpu-vendor.example / example-gpu" Quota: 4 The resource pool includes a `resources` array, where each element represents the usage quota (limitation information) for a partition. The example above shows the use of 6 CPUs, 12 Gi memory, and 4 GPUs in the `gpu-partition` partition. The resource pool's quota information ensures reasonable allocation within the cluster and prevents malicious resource predation. It also includes a `preemption` attribute, which specifies whether preemption is enabled in the current resource pool; that is, when resources are insufficient, low-priority tasks will be preempted by high-priority tasks.

[0044] After the above configuration, Kubernetes can begin resource scheduling. Resource scheduling begins upon receiving any workload. For ease of description, this application's embodiments distinguish between the first and second workloads. It should be noted that in this application's embodiments, the workload includes a job or multiple container groups (pods), rather than processing a single pod as an object as in the prior art. A pod (container group) is the smallest unit of scheduling, consisting of one or more closely related containers that share network, storage, and other resources, collaboratively completing specific functions. In AI scenarios, a pod can correspond to a specific execution instance in a training or inference task, such as a service node in an inference task or a computing node in a training task. Multiple related pods are organized into the same workload through tags, annotations, etc., to jointly complete complex AI tasks (such as training or inference). A job is an abstract concept used to manage "one-off tasks," mainly targeting tasks that run for a short time and terminate upon completion. In AI scenarios, a job typically corresponds to a complete training task (such as a model training process), the goal of which is to schedule a group of pods to perform computations, and the pods terminate upon completion of the task. Workload can be understood as a task (job) or a group of related pods. That is, when a Workload corresponds to a single training task, it can be regarded as a job. Both jobs and pods are components of a Workload. Workload enables unified scheduling, resource allocation, and priority management, adapting to the needs of integrated AI training and inference scenarios.

[0045] The workload includes specified resource pools and priority levels. When specifying resource pools, a resource pool identifier can be used, meaning a corresponding resource pool identifier is set within the workload. Upon receiving the first workload, the corresponding first resource pool is allocated to it based on the resource pool identifier contained within the first workload. The priority level can be user-defined or determined based on priority level identifiers within the workload. For example, priority levels can include first priority, second priority, and third priority (first priority is higher than second priority, and second priority is higher than third priority), or levels A, B, and C (level A is higher than level B, and level B is higher than level C). Priority serves as the basis for subsequent resource preemption and scheduling order.

[0046] In an optional implementation, the process of generating the first workload may be as follows: in response to a first task creation operation, construct the first workload; or in response to a container group creation operation, generate a corresponding target container group; and use the target container group and associated container group as the first container group to obtain the first workload; wherein the associated container group is a container group that is associated with the target container group. In response to a priority level setting operation, set a corresponding priority level for the first workload.

[0047] After determining the first resource pool corresponding to the first workload, it is necessary to schedule the corresponding first resource node for the first workload to process it. In this embodiment, the determination scope for the first resource node includes all candidate resource nodes in the candidate resource partition corresponding to the first resource pool. By obtaining the real-time load information of each candidate resource node in the candidate resource partition corresponding to the first resource pool, and then comparing all the real-time load information, the first resource node for processing the first workload is determined.

[0048] The real-time load information of the candidate resource nodes is implemented through a custom CRD nodemetric. In this embodiment, a daemonset is started, that is, an exporter application is deployed on each candidate resource node to collect the real-time load information of that candidate resource node. Then, a custom nodemetric controller is used to obtain the real-time load information of the candidate resource nodes and store it in the kube-apiserver. Figure 3 The part with the green line in the middle, or Figure 4 The steps shown are as follows: Step 1: The node load controller obtains the real-time load information of each candidate resource node in the candidate resource partition corresponding to the first resource pool from the kube-apiserver; Step 2: Edit the node and obtain the real-time load information of each candidate resource node in the corresponding candidate resource partition of the first resource pool. The node's metric-exporter will store some metric information over a certain period of time, which the node load controller can directly read without calculating it itself.

[0049] Step 3: The node load controller stores the real-time load information of the nodes into the kube-apiserver.

[0050] The following is an example of the third-party resources that have been introduced: apiVersion: gigentech.com / v1alpha1 kind: NodeMetric metadata: name: node1 spec: nodeMetric: - Duration: 5m usage: p50: resources: CPU: 2671m memory: 10194404Ki GPU: 20194204Ki p90: resources: CPU: 3514m memory: 10214100Ki GPU: 30224130Ki p95: resources: CPU: 3617m memory: 10219260Ki GPU: 10114100Ki p99: resources: CPU: 4194m memory: 10232516Ki GPU: 11214120Ki In one alternative implementation, when scheduling a first resource node for a first workload, two situations may occur: the first resource pool can meet the scheduling requirements of the first workload, and the first resource pool cannot meet the scheduling requirements of the first workload.

[0051] If the first resource pool can meet the scheduling requirements of the first workload, the resource node with the lowest resource utilization rate in the corresponding resource partition of the first resource pool can be selected as the first resource node. Then, the first resource node is used to process the first workload.

[0052] If the first resource pool cannot meet the scheduling requirements of the first workload, resource preemption is necessary. This involves evicting some resources from the first resource pool so that the pool can then meet the scheduling requirements of the first workload. Resource preemption is based on the first priority level corresponding to the first workload. Evicting resources from the first resource pool involves removing resources with a priority level lower than the first priority level. After resource release, among the candidate resource nodes in the corresponding candidate resource partition of the first resource pool, the candidate resource node with the lowest resource utilization is selected as the first resource node. The specific eviction process is as follows: identify the container groups (pods) in the first resource pool with a priority level lower than the first priority level; release resources sequentially according to the priority level of the container groups to be released until the first resource pool can meet the scheduling requirements of the first workload.

[0053] For example, resources below the first priority level include pods of the second priority level and pods of the third priority level (the second priority level is higher than the third priority level). When releasing resources, the third priority level pods are released first. If the first resource pool still cannot meet the scheduling requirements of the first workload after the third priority level pods are released, the second priority level pods are released until the first resource pool can meet the scheduling requirements of the first workload.

[0054] In an optional implementation, in specific implementation, it can be carried out according to... Figure 5 In this way: Firstly, this application's embodiment designs a Workload CRD resource, with the scheduler using Workload as the basic scheduling unit. A workload can be understood as a task (job) or, for AI applications, a group of related Pods. Each workload has a corresponding node load controller (workload-controller). The priority of a workload determines whether it will be preempted for resources. When resources are insufficient, tasks within the same resource pool can preempt resources from each other; higher-priority tasks can preempt resources from lower-priority tasks. This design meets the needs of AI training and inference scenarios. When a low-priority training task occupies resources in the resource pool, and a high-priority inference task is submitted to the pool, this inference task will have priority in using resources. When the resource pool has sufficient resources, it directly allocates resources to the inference task. When the resource pool is insufficient, the inference task will sort all tasks with lower priority and evict them from lowest to highest priority until the high-priority inference task can be deployed.

[0055] For example, step 1: The user creates a pod; Step 2: The workload controller monitors unscheduled pods in the cluster; Step 3: Check if the unscheduled pod has a related ownership, such as deployment, statefulset, etc. Step 4: Create relevant workload resources based on the pod's affiliation. Different pods can be organized into the same workload using tags, annotations, etc., and then saved to kube-apiserver for the scheduler to use.

[0056] The workload definition is roughly as follows: apiVersion: kueue.x-k8s.io / v1beta1 kind: Workload metadata: name: wl-1 namespace: default spec: priority: 0 resourcePool: share-pool pods: - count: 1 name: test-1 template: metadata: {} spec: ... - count: 2 name: test-2 template: metadata: {} spec: ... The specific scheduling process can also be followed according to Figure 6 In this way: Step 1: The user submits the pod; Step 2: The workload controller creates the workload based on the description above; Step 3: kube-apiserver saves the workload.

[0057] Step 4: The scheduler detects newly created workloads and adds them to a queue. The scheduler has only one queue, which contains both newly added workloads awaiting scheduling and evictiond pods awaiting scheduling. The queue is sorted by priority to ensure that high-priority workloads are scheduled first.

[0058] Step 5: The scheduler retrieves the highest priority workload from the queue and schedules it.

[0059] Step 6: Based on the resource pool referenced in the workload.

[0060] Step 7: Iterate through the partitions referenced by the resource pool and obtain partition information. If no partitions are available to try, stop the scheduling process.

[0061] Step 8: Obtain partition information from kube-apiserver. This step can be checked by monitoring the cache to see the number of queries.

[0062] Step 9: Determine if the current partition's quota is sufficient to deploy the pods in the workload. If it is, proceed to step 13; otherwise, continue to the next preemption process. Step 10: Check if the current resource pool supports preemption. If it does not support preemption, skip to Step 7 to traverse the next partition. Otherwise, continue to the next step.

[0063] Step 11: View the low-priority tasks in the resource pool, sort the low-priority tasks, and evict low-priority tasks.

[0064] Step 12: Determine if there are enough available resources to schedule the current pod. If the resource pool has sufficient remaining resources, proceed to the next step. Otherwise, jump to step 7 to traverse the next partition.

[0065] Step 13: Set the pod's nodeselector to the partition's label to ensure that the pod remains in this partition when performing other scheduling operations in the future.

[0066] Step 14: Obtain real-time load information of nodes in the partition, and obtain real-time information of nodes in the partition from kube-apiserver; Step 15: Based on real-time load information and other information such as topology, affinity, and other scheduling rules, select the node with the highest score as the scheduling node. Set the nodeName of the pod to this node, and continue to traverse the other pods in the workload.

[0067] In an optional implementation, the embodiments of this application may also receive a second workload simultaneously or sequentially. The process for processing the second workload is the same as that for the first workload, and will not be described again here. Depending on specific business needs, the second resource pool may be the same as or different from the first resource pool. When the second resource pool is different from the first resource pool, their resource scheduling does not affect each other.

[0068] Figure 7 This invention provides a schematic diagram of a resource scheduling apparatus according to an embodiment of the present application. The apparatus includes: The first response module is used to respond to the partition configuration operation. Based on the partition label corresponding to the partition configuration operation, it divides the candidate resource nodes in the resource node cluster to obtain the divided candidate resource partitions. The second response module is used to respond to the restriction information configuration operation and, according to the configured restriction information, establish a corresponding candidate resource pool for each candidate resource partition; wherein, the candidate resource pool represents the resource scheduling range and resource usage quota of the candidate resource; The allocation module is configured to, upon receiving a first workload, allocate a corresponding first resource pool to the first workload based on the resource pool identifier contained in the first workload; wherein the first workload includes a first task or multiple first container groups. The scheduling module is used to schedule the first resource node for the first workload based on the real-time load information of each of the candidate resource nodes in the candidate resource partition corresponding to the first resource pool.

[0069] The first workload is obtained in the following manner: In response to the first task creation operation, construct the first workload; Alternatively, in response to a container group creation operation, a corresponding target container group may be generated; The target container group and the associated container group are used as the first container group to obtain the first workload; wherein, the associated container group is a container group that is associated with the target container group.

[0070] The third response module is used to respond to the priority level setting operation and set the corresponding priority level for the first workload.

[0071] The step of scheduling a first resource node for the first workload based on the real-time load information of each candidate resource node in the candidate resource partition corresponding to the first resource pool includes: When the first resource pool can meet the scheduling requirements of the first workload, the candidate resource node with the lowest resource utilization rate is selected as the first resource node from the candidate resource nodes in the candidate resource partition corresponding to the first resource pool.

[0072] If the first resource pool is unable to meet the scheduling requirements of the first workload, the resources in the first resource pool are released according to the first priority level corresponding to the first workload. After the resources are released, the resource node with the lowest resource utilization rate is selected as the first resource node from the candidate resource nodes in the candidate resource partition corresponding to the first resource pool.

[0073] The step of releasing resources from the first resource pool according to the first priority level corresponding to the first workload includes: Identify the groups of containers in the first resource pool that are lower than the first priority level and are to be released; Resources are released sequentially according to the priority level of the containers to be released, until the first resource pool can meet the scheduling requirements of the first workload.

[0074] The allocation module is configured to, upon receiving a second workload, allocate a corresponding second resource pool to the second workload based on the resource pool identifier contained in the second workload; wherein the second workload includes a second task or multiple second container groups; The scheduling module is used to schedule the second resource node for the second workload based on the real-time load information of each of the candidate resource nodes in the candidate resource partition corresponding to the second resource pool.

[0075] like Figure 8As shown, this application provides an electronic device for executing the resource scheduling method described in this application. The device includes a memory, a processor, a bus, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the resource scheduling method described above.

[0076] Specifically, the aforementioned memory and processor can be general-purpose memory and processor, without any specific limitations. When the processor runs a computer program stored in the memory, it can execute the resource scheduling method described above.

[0077] Corresponding to the resource scheduling method in this application, this application embodiment also provides a computer-readable storage medium storing a computer program, which is executed by a processor to perform the steps of the resource scheduling method described above.

[0078] Specifically, the storage medium can be a general-purpose storage medium, such as a removable disk or hard disk, and when the computer program on the storage medium is run, it can execute the resource scheduling method described above.

[0079] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0080] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0081] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0082] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0083] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0084] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A resource scheduling method, characterized in that, The method includes: In response to the partition configuration operation, the candidate resource nodes in the resource node cluster are divided according to the partition label corresponding to the partition configuration operation to obtain the divided candidate resource partitions. In response to the restriction information configuration operation, a corresponding candidate resource pool is established for each candidate resource partition according to the configured restriction information; wherein, the candidate resource pool represents the resource scheduling range and resource usage quota of the candidate resource; Upon receiving a first workload, a corresponding first resource pool is allocated to the first workload based on the resource pool identifier contained in the first workload; wherein, the first workload includes a first task or multiple first container groups; Based on the real-time load information of each candidate resource node in the candidate resource partition corresponding to the first resource pool, the first resource node is scheduled for the first workload; The method obtains the first workload in the following manner: In response to the first task creation operation, construct the first workload; Alternatively, in response to a container group creation operation, a corresponding target container group may be generated; The target container group and the associated container group are used as the first container group to obtain the first workload; wherein, the associated container group is a container group that is associated with the target container group; The method further includes: In response to the priority level setting operation, a corresponding priority level is set for the first workload; The step of scheduling a first resource node for the first workload based on the real-time load information of each candidate resource node in the candidate resource partition corresponding to the first resource pool includes: When the first resource pool can meet the scheduling requirements of the first workload, the candidate resource node with the lowest resource utilization rate is selected as the first resource node from the candidate resource nodes in the candidate resource partition corresponding to the first resource pool.

2. The method according to claim 1, characterized in that, The method further includes: If the first resource pool is unable to meet the scheduling requirements of the first workload, the resources in the first resource pool are released according to the first priority level corresponding to the first workload. After the resources are released, the resource node with the lowest resource utilization rate is selected as the first resource node from the candidate resource nodes in the candidate resource partition corresponding to the first resource pool.

3. The method according to claim 2, characterized in that, The step of releasing resources from the first resource pool according to the first priority level corresponding to the first workload includes: Identify the groups of containers in the first resource pool that are lower than the first priority level and are to be released; Resources are released sequentially according to the priority level of the containers to be released, until the first resource pool can meet the scheduling requirements of the first workload.

4. The method according to claim 1, characterized in that, The method further includes: Upon receiving a second workload, a corresponding second resource pool is allocated to the second workload based on the resource pool identifier contained in the second workload; wherein, the second workload includes a second task or multiple second container groups; Based on the real-time load information of each candidate resource node in the candidate resource partition corresponding to the second resource pool, the second resource node is scheduled for the second workload.

5. A resource scheduling apparatus, characterized in that, The device includes: The first response module is used to respond to the partition configuration operation. Based on the partition label corresponding to the partition configuration operation, it divides the candidate resource nodes in the resource node cluster to obtain the divided candidate resource partitions. The second response module is used to respond to the restriction information configuration operation and, according to the configured restriction information, establish a corresponding candidate resource pool for each candidate resource partition; wherein, the candidate resource pool represents the resource scheduling range and resource usage quota of the candidate resource; The allocation module is configured to, upon receiving a first workload, allocate a corresponding first resource pool to the first workload based on the resource pool identifier contained in the first workload; wherein the first workload includes a first task or multiple first container groups. The scheduling module is used to schedule the first resource node for the first workload based on the real-time load information of each of the candidate resource nodes in the candidate resource partition corresponding to the first resource pool. The first workload is obtained in the following manner: In response to the first task creation operation, construct the first workload; Alternatively, in response to a container group creation operation, a corresponding target container group may be generated; The target container group and the associated container group are used as the first container group to obtain the first workload; wherein, the associated container group is a container group that is associated with the target container group; In response to the priority level setting operation, a corresponding priority level is set for the first workload; The step of scheduling a first resource node for the first workload based on the real-time load information of each candidate resource node in the candidate resource partition corresponding to the first resource pool includes: When the first resource pool can meet the scheduling requirements of the first workload, the candidate resource node with the lowest resource utilization rate is selected as the first resource node from the candidate resource nodes in the candidate resource partition corresponding to the first resource pool.

6. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the resource scheduling method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the resource scheduling method as described in any one of claims 1 to 4.

Images