Resource scheduling method and device based on k8s cluster

Abstract

The application relates to a resource scheduling method and device based on a k8s cluster, wherein the resource scheduling method based on the k8s cluster comprises the following steps: obtaining a to-be-scheduled Pod that fails to be scheduled due to insufficient resources in the k8s cluster; filtering all Pods running in the cluster to obtain a first set of Pods with a task priority lower than that of the to-be-scheduled Pod; filtering the first set according to a target GPU model required by the to-be-scheduled Pod to obtain a second set of Pods bound with a target GPU model identifier GPU; determining the recovery scores of the Pods in the second set; determining a to-be-recovered Pod from the second set according to the recovery scores, and marking a node where the to-be-recovered Pod is located as a target node; releasing the to-be-recovered Pod from the target node and scheduling the to-be-scheduled Pod to the target node. The application solves the problem of low resource utilization caused by rough resource scheduling strategies in a large-scale cluster environment, especially in a heterogeneous resource scene such as GPU.

Description

Technical Field

[0001] This application relates to the field of cluster task scheduling technology, and in particular to a resource scheduling method and apparatus based on a Kubernetes cluster. Background Technology

[0002] In scenarios such as artificial intelligence model training, video processing, and high-performance computing, graphics processing units (GPUs) are the most scarce computing resources. In Kubernetes (k8s) clusters, because training tasks are often long-lasting and resource-intensive, if low-priority tasks occupy GPU resources, high-priority tasks will not be able to obtain resources in a timely manner, resulting in resource waste and task delays.

[0003] While Kubernetes natively supports priority-based resource scheduling (preemption) mechanisms, in scenarios with incompressible resources such as GPUs, its default preemption lacks awareness of the actual resource quantity of tasks, the remaining resources of nodes, and the dependencies between task groups. This may lead to: resources being unable to meet the actual resource requirements of high-priority tasks after scheduling; releasing too many Pods, causing node resource fragmentation; and grouped scheduled tasks being split and scheduled before the minimum number of runnable tasks is met, resulting in the overall task being unable to run even though it has occupied resources.

[0004] Currently, no effective solution has been proposed for the problem of low resource utilization caused by coarse resource scheduling strategies in large-scale cluster environments, especially in heterogeneous resource scenarios such as GPUs. Summary of the Invention

[0005] This application provides a resource scheduling method and apparatus based on a Kubernetes cluster, which at least solves the problems of low resource utilization and long waiting time for high-priority tasks caused by coarse scheduling strategies in related technologies.

[0006] In a first aspect, embodiments of this application provide a resource scheduling method based on a Kubernetes cluster, including:

[0007] Retrieve the Pods in the Kubernetes cluster that failed to be scheduled due to insufficient resources;

[0008] All Pods running in the Kubernetes cluster are filtered at the first level to obtain a first set; the task priority of each Pod in the first set is lower than the task priority of the Pod to be scheduled.

[0009] Based on the target GPU model required by the Pod to be scheduled, the first set is filtered in a second level to obtain a second set; each Pod in the second set is bound to a GPU identified by the target GPU model.

[0010] Based on the hardware resource consumption of each Pod in the second set, the load information of the node where each Pod is located, and the task priority of each Pod, the recycling score of each Pod in the second set is determined.

[0011] Based on the recycling score, determine the Pods to be recycled from the second set, and mark the node where the Pods to be recycled are located as the target node;

[0012] Release the Pod to be recycled from the target node and schedule the Pod to be scheduled to the target node.

[0013] Secondly, embodiments of this application provide a resource scheduling device based on a Kubernetes cluster, comprising:

[0014] The resource awareness module is used to identify Pods in the Kubernetes cluster that have failed to be scheduled due to insufficient resources.

[0015] The candidate filtering module is used to perform a first-level filter on all Pods running in the Kubernetes cluster to obtain a first set; the task priority of each Pod in the first set is lower than the task priority of the Pod to be scheduled; the first set is then filtered a second time according to the target GPU model required by the Pod to be scheduled to obtain a second set; each Pod in the second set is bound to a GPU identified by the target GPU model.

[0016] The scoring module is used to determine the recycling score of each Pod in the second set based on the hardware resource consumption of each Pod, the load information of the node where each Pod is located, and the task priority of each Pod.

[0017] The recycling confirmation module is used to determine the Pods to be recycled from the second set based on the recycling score, and mark the node where the Pods to be recycled are located as the target node;

[0018] The scheduling and coordination module is used to release the Pod to be recycled from the target node and schedule the Pod to be scheduled to the target node.

[0019] Compared to related technologies, the resource scheduling method and apparatus based on a Kubernetes cluster provided in this application solves the problem of low resource utilization caused by coarse resource scheduling strategies in related technologies. It achieves accurate and low-cost resource recycling and scheduling, significantly improving the utilization efficiency of heterogeneous resources, especially GPUs, and the task scheduling performance of large-scale clusters. This is achieved by: acquiring Pods that failed to be scheduled due to insufficient resources in the Kubernetes cluster; performing a first-level filter on all Pods running in the Kubernetes cluster based on task priority to obtain a first set; performing a second-level filter on the first set based on the target GPU model to obtain a second set; determining the recycling score of each Pod in the second set based on the hardware resource usage of each Pod, the load information of the node where each Pod resides, and the task priority of each Pod; identifying Pods to be recycled from the second set based on the recycling score and marking the node where the Pods to be recycled resides as the target node; releasing the Pods to be recycled from the target node and scheduling the Pods to be scheduled to the target node. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 A flowchart of the resource scheduling method based on a Kubernetes cluster provided in this application;

[0022] Figure 2 This is a flowchart illustrating the method for determining Pods to be recycled in non-grouped tasks according to this application;

[0023] Figure 3 This is a flowchart illustrating the method for determining Pods to be recycled during grouped tasks in this application;

[0024] Figure 4 A flowchart illustrating a resource scheduling method based on a Kubernetes cluster, as shown in an exemplary embodiment of this application;

[0025] Figure 5 The structural block diagram of the resource scheduling device based on the Kubernetes cluster provided in this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application. Furthermore, it is understood that although the efforts made in such a development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, modifications to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0027] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0028] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application means two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The terms “first,” “second,” “third,” etc., used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0029] In the embodiments of this application, the technical solutions involved are mainly applied to the Kubernetes container orchestration platform. Kubernetes (k8s) is an open-source platform used for automating the deployment, scaling, and management of containerized applications. In a k8s cluster, a Pod is the basic, smallest deployable and manageable unit of computation. In the embodiments of this application, the term "task" refers to a single running instance of an application or computation job. On the k8s platform, a task can be completed collaboratively by one or more Pods (for example, a distributed training task may include a parameter server Pod and multiple worker node Pods). The Pod is the specific carrier and representation of the task in the k8s cluster. The core of the resource scheduling method in this application is to schedule the Pods carrying the task and bind them to cluster nodes with sufficient available resources for execution.

[0030] This embodiment provides a resource scheduling method based on a Kubernetes cluster. Figure 1 The flowchart of the resource scheduling method based on a Kubernetes cluster provided in this application is as follows: Figure 1 As shown, the process includes the following steps:

[0031] Step S110: Obtain the Pods in the Kubernetes cluster that failed to be scheduled due to insufficient resources.

[0032] There are many reasons why Pod scheduling might fail in a Kubernetes cluster, such as node taint intolerance, node affinity mismatch, port conflicts, and storage volume mounting failure. This solution only focuses on Pods that fail to be scheduled due to insufficient computing resources within the cluster.

[0033] Specifically, retrieve the Pods in the Kubernetes cluster that failed to be scheduled due to insufficient resources and identify them as the Pods to be scheduled in this application. In detail, first, send an HTTP GET request with the parameter `fieldSelector=status.phase=Pending` to the Kubernetes API Server to obtain a list of all Pods in the Kubernetes cluster that are in a waiting-to-be-scheduled state; then, iterate through this list and parse the `status.conditions` field of each Pod (this field is a Pod status sub-resource, which can be returned with the API response or via `kubectl get pod`). <name>(View using -ojson); when the field shows type as PodScheduled and status as False, it indicates that the Pod has not yet been scheduled successfully. At the same time, the specific reason for the unsuccessful scheduling will be recorded in the corresponding reason and message fields. Therefore, by parsing the reason field, Pods with a value of "Unschedulable" and a message field containing the string Insufficient (case-insensitive) can be filtered out. In other words, Pods that cannot be scheduled due to insufficient CPU, memory, GPU or custom resources can be filtered out in the above way, and these Pods will be used as the Pods to be scheduled in this application.

[0034] Furthermore, in a Kubernetes cluster, each Pod corresponds to a different task priority, which describes how frequently a Pod is prioritized during scheduling and preemption. In one implementation, this task priority can be obtained through its associated PriorityClass resource. PriorityClass is a cluster-level API object provided by Kubernetes, whose value field defines the priority value; a larger value indicates a higher priority. In other implementations, this priority information can also be recorded in a custom field in the Pod's metadata.annotations.

[0035] Therefore, to avoid initiating unnecessary scheduling evaluations for a large number of low-priority waiting tasks, tasks of a specific priority level can be selectively preempted for scheduling as described in this application.

[0036] In detail, after obtaining the list of Pods to be scheduled, the process first filters out Pods that fail to be scheduled due to insufficient resources and defines them as candidate Pods. Then, the task priority of each candidate Pod is obtained. In one implementation, this can be done by directly reading the Pod's `spec.priority` field (whose value comes from the value of its associated `PriorityClass`) or by querying its associated `PriorityClass` resource. Subsequently, based on the task priorities, candidate Pods with priority values ​​higher than a preset threshold are further filtered out and designated as the first target Pods. Finally, the first target Pod is determined as the Pod to be scheduled that requires preemptive scheduling. It is understood that the preset threshold can be flexibly configured according to cluster management policies and business needs; for example, it can be set to trigger the preemptive scheduling process only for high-priority Pods with a priority value greater than or equal to 1000.

[0037] Through the above steps, by real-time monitoring of the Kubernetes cluster scheduling status and precise analysis of the reasons for Pod scheduling failures, the problem of not being able to accurately identify the core tasks that cause scheduling blockage due to resource bottlenecks in related technologies has been solved. This enables the precise locking of high-priority Pods that truly require resource protection, providing the correct triggering objects and target basis for the subsequent precise and low-impact resource preemption and reclamation process (i.e., the scheduling process).

[0038] Step S120: Perform first-level filtering on all Pods running in the k8s cluster to obtain the first set; the task priority of each Pod in the first set is lower than the task priority of the Pod to be scheduled.

[0039] Specifically, firstly, you can obtain a list of all currently running Pods in the Kubernetes cluster by sending an HTTP GET request with the parameter `fieldSelector=status.phase=Running` to the Kubernetes API Server. This list contains metadata for each Pod (such as namespace, name, priority class name, etc.) and its runtime status.

[0040] Secondly, for each running Pod in the list, its corresponding task priority value can be obtained by reading the spec.priority field of the Pod (the value of which comes from the value of its associated PriorityClass). Then, the first-level filtering is performed, that is, the priority value is compared with the priority value of the Pod to be scheduled determined in step S110, and only Pods with priority values ​​that are strictly less than the priority value of the Pod to be scheduled are retained, forming the first set after filtering.

[0041] The first-level filtering based on task priority described above only retains running Pods with task priorities lower than the Pod to be scheduled, thus preventing invalid preemption between tasks of the same or higher priority.

[0042] Furthermore, to enhance the security and system stability of the preemption operation and avoid accidental damage to core cluster components or newly started tasks, this step may also perform an additional eviction operation on the aforementioned first set. This eviction operation is performed according to at least one of the following preset eviction rules:

[0043] (1) Remove Pods from the system namespace.

[0044] Specifically, examine the `metadata.namespace` field of each Pod in the first set mentioned above, and remove Pods whose values ​​belong to system reserved namespaces (e.g., `kube-system`, `kube-public`, `kube-node-lease`, etc.) from the first set. Pods under these namespaces typically host the core control plane components of the Kubernetes cluster or critical node daemons. Preempting such Pods may cause cluster malfunctions, so they must be protected.

[0045] (2) Remove Pods whose runtime is less than the preset protection period threshold.

[0046] Specifically, the `status.startTime` field of each Pod in the first set is checked to calculate its runtime from startup time to the current time. If the runtime is less than a preset protection period threshold (e.g., the system default is 5 minutes), the Pod is removed from the first set. This runtime-based protection mechanism aims to protect tasks that have just started and are still in the initialization or warm-up phase, avoiding the inability of tasks to complete normally due to frequent creation and eviction, effectively improving system stability and scheduling fairness.

[0047] (3) Remove Pods that are currently in the exit phase.

[0048] Specifically, check the `metadata.deletionTimestamp` field of each Pod in the first set. If this field is not empty, it indicates that the Pod has been marked as pending deletion and is in the process of exiting (commonly known as the Terminating state). Pods in this state are about to release the resources they occupy. Preempting them not only has no practical benefit but may also cause inconsistencies or conflicts in resource states, so they are removed.

[0049] After the priority filtering and optional removal operations described above, the Pods that remain constitute a more refined and secure first set. Each Pod in this first set meets the condition that its task priority is lower than that of the Pod to be scheduled, and does not include Pods under the system namespace, Pods with a runtime shorter than the preset protection period threshold, or Pods that are currently in the exit phase. It can serve as a reliable candidate for subsequent resource matching and scoring processes.

[0050] Through the above steps, a first set is determined by a first-level filtering rule based on task priority. This is further combined with a multi-screening and elimination mechanism that includes system namespace elimination, elimination of runtime less than a preset protection period threshold, and elimination of Pods in the exit phase. This solves the problem in related technologies where preemption operations may cause accidental damage to critical system components, frequent interruption of newly started tasks, or interference with the normal exit process. It achieves accurate, safe, and hierarchical screening of the set of candidate Pods that can be preempted for scheduling, and provides a reliable target range for subsequent refined preemption scheduling decisions based on GPU model and reclamation score.

[0051] Step S130: Based on the target GPU model required by the Pod to be scheduled, perform a second-level filter on the first set to obtain a second set; each Pod in the second set is bound to a GPU with a target GPU model identifier.

[0052] The target GPU model refers to the specific graphics processing unit (GPU) hardware model that the scheduled Pod requests to complete its computing task, such as NVIDIA Tesla V100, NVIDIA A100, or NVIDIA H100. In large-scale heterogeneous computing clusters, different nodes may be equipped with different GPU models, which differ in computing power, memory capacity, and architectural characteristics. The scheduled Pod typically specifies its required GPU model through a node selector or node affinity rules to ensure that the task can be correctly executed on a node with the corresponding hardware capabilities.

[0053] Specifically, the hardware resource requirements of the Pod to be scheduled include the target GPU model, number of GPUs, number of CPU cores, and memory size. These parameters are defined in the `spec.containers[].resources.requests` and `spec.containers[].resources.limits` fields of the Pod specification file. GPU resource requirements can be declared by extending the resource name (e.g., nvidia.com / gpu), while constraints on GPU models are typically recorded in the `spec.nodeSelector` or `spec.affinity.nodeAffinity` fields. For example, a Pod to be scheduled can be limited to nodes equipped with Tesla V100 GPUs by setting the tag key-value pair `nvidia.com / gpu.product:Tesla-V100` in the `nodeSelector`. Correspondingly, the required target GPU model can be accurately obtained by parsing the aforementioned specification fields of the Pod to be scheduled.

[0054] After obtaining the target GPU model, a second-level filtering can be performed on the first set obtained in step S120. Specifically, for each Pod in the first set, the model identifier of the GPU bound to that Pod can be obtained by querying the device allocation record in its metadata.annotations field (e.g., the nvidia.com / gpu.product annotation written by the NVIDIA Device Plugin) or by reading the tag information of its node (e.g., the nvidia.com / gpu.product tag on the node). The Pod is retained only if the GPU model bound to it matches the target GPU model required by the Pod to be scheduled, ensuring that the GPU model bound to the retained Pod is the same as the GPU model required by the Pod to be scheduled. Therefore, a second-level filtering based on the target GPU model is performed on each Pod in the first set to obtain a second set, where each Pod in the second set is a Pod bound to a GPU with the target GPU model identifier.

[0055] Furthermore, to ensure that the nodes where the preempted and released resources are located can be actually used by the Pods to be scheduled, a selection based on node affinity constraints can be performed in addition to the selection based on GPU model.

[0056] In detail, firstly, the node affinity constraint information of the Pod to be scheduled is obtained. This node affinity constraint information is defined in the `spec.affinity.nodeAffinity` field of the Pod to be scheduled, and is used to limit the range of nodes that the Pod can be scheduled to (for example, the node must be located in a specific availability zone, have a specific rack label, or belong to a specific node pool). Then, the set of Pods filtered based on GPU model is designated as the first candidate Pod set. Each Pod in the first candidate Pod set is traversed, its node information is obtained, and it is checked whether the node satisfies the node affinity constraint information of the Pod to be scheduled. If the node where a Pod is located does not satisfy the above constraints, the Pod is removed from the first candidate Pod set. If the node where a Pod is located satisfies the above constraints, it is marked as the second target Pod and retained in the first candidate Pod set. Finally, the retained second target Pods constitute the second set. At this time, each Pod in the second set is bound to the GPU identified by the target GPU model and satisfies the node affinity constraint information required for the Pod to be scheduled.

[0057] Through the above steps, the target GPU model and node affinity constraint information are obtained by parsing the specifications of the Pod to be scheduled. The first set is then subjected to dual filtering based on GPU model matching and node affinity constraint information verification. This solves the problem in related technologies where GPU resources released by preemption may not be usable by the Pod to be scheduled due to model mismatch or node position not meeting the constraints. This enables fine-grained screening of preemptible Pods, ensuring that every GPU resource released by the preemption operation is a valid and usable resource, and providing an accurate candidate range for subsequent recycling scoring and recycling decisions.

[0058] Step S140: Determine the recycling score of each Pod in the second set based on the hardware resource usage of each Pod in the second set, the load information of the node where each Pod is located, and the task priority of each Pod.

[0059] The recycling score is a numerical metric used to quantify the priority of recycling each candidate Pod in the second set. By calculating the recycling score for each candidate Pod, multiple eligible preemptible Pods can be objectively ranked, thus selecting Pods that contribute significantly to meeting the resource requirements of the scheduled Pods while having a minimal impact on running tasks in the cluster as preemption targets, thereby achieving effective resource release with minimal impact.

[0060] Specifically, the calculation of the recycling score is based on information from the following three dimensions:

[0061] (1) Hardware resource usage of Pod.

[0062] The hardware resource usage of a Pod refers to the amount of various hardware resources that the Pod currently requests and uses. This information can be obtained by querying the Pod's `spec.containers[].resources.requests` field, including but not limited to the number of GPUs (obtained through the extended resource name such as `nvidia.com / gpu`), the number of CPU cores (`cpu` field value), and the memory size (`memory` field value). It's understandable that the more hardware resources a Pod uses, the more hardware resources can be released by reclaiming the Pod, the greater its contribution to meeting the hardware resource needs of other Pods to be scheduled, and the higher its reclamation score.

[0063] (2) Load information of each Pod node.

[0064] The load information of each Pod's node refers to the overall resource usage pressure of that node, including GPU utilization, CPU utilization, and memory utilization. This load information can be obtained in real time by calling the Kubernetes Metrics API or the HTTP interface exposed by the cAdvisor (Container Advisor) embedded in the kubelet on the node. It's understandable that if the load on a Pod's node is high, reclaiming the Pod not only releases resources but also alleviates the node's resource pressure, which is beneficial for overall cluster load balancing; conversely, reclaiming Pods on low-load nodes has less overall system benefit. Therefore, node load information can be used as one of the factors for adjusting the reclamation score. In one implementation, the load information can be evaluated using specific parameters based on the actual application scenario. For example, when the actual scenario primarily involves GPU computing tasks, GPU utilization can be used as the load information; when the actual application scenario is more sensitive to CPU resources, CPU utilization can be used; or a weighted average of GPU utilization, CPU utilization, and memory utilization can be used. Furthermore, the acquired load information can be normalized and mapped to a value between 0 and 1 for weighted calculation with other factors in the scoring formula. The revival score is negatively correlated with the quantified value of node load; that is, the higher the node load, the greater the contribution of reviving the Pods on that node to alleviating cluster pressure, and the higher the score; the lower the node load, the less urgency there is to revive the Pods on that node, and the lower the score.

[0065] (3) Task priority of each Pod.

[0066] The task priority of each Pod refers to the scheduling priority value assigned to that Pod in the Kubernetes cluster. This priority value can be obtained by querying the value field of the PriorityClass resource associated with the Pod. It's understood that the lower the task priority of a Pod, the lower the cost of preemption and the smaller the impact on business operations; therefore, it should be given a higher preemption bias in the scoring. In one implementation, the revival score is negatively correlated with the Pod's task priority; that is, the lower the priority value, the higher the revival score.

[0067] After obtaining the above information, a recycling score can be calculated for each Pod in the second set. One possible calculation method is to use a weighted summation formula, which can be expressed as:

[0068] ;

[0069] in, , , These represent the number of GPUs, CPU cores, and memory capacity released by each Pod in the second set, respectively. This indicates the load information of the node where the Pod resides; P represents the task priority value of the candidate Pod (or a normalized value); w1 to w5 are configurable weight coefficients used to adjust the influence of each factor on the score according to the actual cluster management strategy. This formula quantifies the impact of hardware resource release (positively correlated factor) and preemption operation (negatively correlated factor), enabling the reclamation score to reflect the suitability of reclamating each Pod to be evaluated. A higher score indicates that the Pod is more suitable to be selected as a preemption target.

[0070] By taking the steps described above, the hardware resource usage of each Pod in the second set, the load information of the node where each Pod is located, and the task priority of each Pod, and by using a weighted scoring mechanism to quantitatively evaluate each candidate Pod, the problem of lacking objective selection basis for preemption decision-making and difficulty in selecting the best among multiple candidate tasks in related technologies is solved. This achieves quantitative ranking of preemptible Pods and provides decision support for subsequent preemption execution that releases effective resources with minimal impact.

[0071] Step S150: Based on the recycling score, determine the Pods to be recycled from the second set and mark the node where the Pods to be recycled are located as the target node.

[0072] Specifically, after calculating the recycling score for each candidate Pod in the second set in step S140, the Pods in the second set can be sorted in descending order of recycling score. The higher the recycling score, the more effective resources can be released by recycling the Pod, the greater the positive impact on the load of the node, and the lower its own task priority, thus making it more suitable as a preemptive recycling target.

[0073] Based on the ranking results and the hardware resource requirements of the Pods to be scheduled, the Pods to be reclaimed are determined. Specifically, starting with the Pod with the highest reclamation score, the resources occupied by each Pod in the second set are simulated and released sequentially. After each simulation, it is determined whether the current resources of the Kubernetes cluster can meet the scheduling requirements of the Pods to be scheduled. When the requirements are met, one or more candidate Pods participating in the simulation release are identified as Pods to be reclaimed, and the nodes containing these Pods are marked as target nodes.

[0074] Through the above steps, and by using a strategy based on recycling score ranking and resource satisfaction assessment, the problem of lack of quantitative basis for preemption selection in related technologies is solved, and the preemption target is determined to accurately meet the resource needs of the Pods to be scheduled at a relatively low cost.

[0075] Step S160: Release the Pod to be recycled from the target node and schedule the Pod to be scheduled to the target node.

[0076] First, resource release operations are performed on the Pods to be reclaimed as determined in step S150. In one implementation, the Eviction API provided by Kubernetes can be invoked to initiate a graceful eviction request for the aforementioned Pods to be reclaimed. The Eviction API is a standard interface in Kubernetes specifically designed to handle Pod preemptive eviction. By sending a POST request with an Eviction object to the API Server, it can be ensured that the Pod terminates gracefully according to its defined terminationGracePeriodSeconds, rather than being forcibly deleted, minimizing the impact on running services and ensuring the consistency of task status and the smooth reclamation of resources.

[0077] After initiating an eviction request, continuously monitor the status changes of the Pod to be reclaimed. This can be done by querying the Pod's status information from the Kubernetes API Server and checking whether its status.phase field has changed to Succeeded or Failed, thus determining whether the Pod to be reclaimed has been completely removed from the cluster. Once it is confirmed that the Pod to be reclaimed has terminated and the hardware resources (especially GPUs) it previously occupied have been reclaimed by its node, the resource release operation is considered complete.

[0078] Subsequently, a resource change notification or scheduling trigger signal can be sent to the scheduler (kube-scheduler) in the Kubernetes cluster. In one implementation, this can be done by writing a trigger flag to the metadata.annotations field of the Pod to be scheduled (e.g., scheduler.alpha.kubernetes.io / trigger-scheduling: true), or by explicitly notifying the scheduler to re-evaluate the scheduling feasibility of the Pod by calling the scheduler extension interface. After receiving the signal or detecting a change in the cluster resource state, the scheduler will reinstate the Pod in the Pending state into the scheduling loop. Since the resources on the target node have been released at this time, the scheduler can successfully bind the Pod to the target node during resource matching, completing the final scheduling.

[0079] Through the above steps, by calling the Kubernetes standard Eviction API to execute Pod eviction, continuously monitoring the resource release status, and actively triggering the scheduler to reschedule, the problem of poor connection between resource release and scheduling after preemption in related technologies, which caused Pods waiting to be scheduled to still have to wait for a long time, is solved. This achieves close coordination between resource release and task scheduling, ensuring that high-priority Pods waiting to be scheduled can quickly obtain scheduling opportunities after resources are ready.

[0080] Through the steps S110-S160 described above, this application provides a resource scheduling method based on a Kubernetes cluster. This method first accurately identifies high-priority Pods that fail to schedule due to insufficient resources, triggering a preemption and reclamation process. Then, it filters out a first set based on task priority; next, it filters out a second set based on GPU model matching. Furthermore, it quantifies and calculates the reclamation score for each Pod in the second set by comprehensively considering the hardware resource usage of each Pod, the load information of the node where each Pod resides, and a weighted scoring mechanism based on the task priority of each Pod. Finally, based on the reclamation score ranking and the hardware resource requirements of the Pods to be scheduled, it determines the Pods to be reclaimed and their target nodes, and completes resource release and task binding through a coordinated eviction and scheduling process. This complete process solves the problems of low resource utilization and long waiting times for high-priority tasks caused by coarse preemption strategies in related technologies. It achieves accurate perception, selective reclamation, and efficient reuse of heterogeneous resources such as GPUs in a Kubernetes environment, significantly improving cluster resource utilization efficiency and the scheduling performance of high-priority tasks.

[0081] It should be noted that during the preemptive scheduling process described above, the task group type to which the Pod to be scheduled belongs differs, and the logic for judging the scheduling feasibility after resource release also differs. In practical applications, tasks in a Kubernetes cluster can be divided into two types: grouped tasks and ungrouped tasks. To facilitate a more precise understanding of the resource scheduling method in this application, these two task group types will be explained first below.

[0082] In this application, the task grouping types corresponding to the Pods to be scheduled include grouped tasks and ungrouped tasks. Grouped tasks refer to task types that require multiple Pods to collaborate and jointly complete a computational job. Typical grouped tasks include distributed training tasks based on Gang-Scheduling and machine learning tasks with parameter server architectures. These tasks are characterized by dependencies between their subordinate Pods. The entire task can only start and execute normally when all Pods within the group are successfully scheduled and running. If only some Pods are successfully scheduled while the remaining Pods are waiting due to insufficient resources, the scheduled Pods, although occupying resources, cannot advance the actual computational work, resulting in resource waste and potentially causing the overall scheduling failure of the grouped task. Ungrouped tasks refer to task types that can be completed independently by a single Pod. Ungrouped tasks do not depend on the collaboration of other Pods; once a single subordinate Pod is successfully scheduled and running, it can begin executing computational work. The overall progress of the task is directly related to the running status of that Pod.

[0083] In practice, the task group type of the Pod to be scheduled can be determined by parsing the `metadata.annotations` or `metadata.labels` fields. For example, in a Kubernetes cluster, a Pod can be marked as belonging to a grouped task by setting specific annotations (such as `scheduler.alpha.kubernetes.io / task-group-name` or `gang.scheduling.kubernetes.io / name`) in its definition; alternatively, it can be determined by checking whether the Pod is associated with a Gang-Scheduling-related custom resource (such as a PodGroup). If the Pod to be scheduled carries the above identifier or is associated with the corresponding grouped scheduling resource, its corresponding task group type is determined to be a grouped task; otherwise, the task group type of the Pod to be scheduled is determined to be a non-grouped task.

[0084] Due to the fundamental difference in resource requirement fulfillment conditions between grouped and non-grouped tasks, this application designs differentiated methods for determining Pods to be reclaimed for these two task grouping types. The following section combines... Figure 2 and Figure 3 The specific implementation methods for both non-grouped task scenarios and grouped task scenarios are described in detail.

[0085] Specifically, Figure 2 This is a flowchart illustrating the method for determining Pods to be reclaimed in non-grouped tasks, as described in this application. Figure 2 As shown, when the task group type corresponding to the Pod to be scheduled is a non-grouped task, the method for determining the Pod to be reclaimed from the second set based on the reclamation score includes the following steps:

[0086] Step S201: Select the Pod with the highest current recycling score from the second set as the candidate Pod for recycling.

[0087] Step S202: Evaluate the first simulated available resources of the node where the candidate Pod is located after releasing the candidate Pod for recycling.

[0088] In detail, the release of candidate Pods for recycling can be simulated in memory. This involves deducting the amount of hardware resources occupied by the Pod (including the number of GPUs, CPU cores, and memory capacity) from the allocated resources of the node where it resides, and updating the virtual available resource status of the node, which is the simulated available resource. Here, it is the first simulated available resource.

[0089] Step S203: Determine whether the first simulated available resources meet the hardware resource requirements of the Pod to be scheduled. If the first simulated available resources meet the hardware resource requirements of the Pod to be scheduled, proceed to step S204; if the first simulated available resources do not meet the hardware resource requirements of the Pod to be scheduled, proceed to step S205.

[0090] Step S204: The candidate Pod to be recycled is determined as the Pod to be recycled, and the process ends.

[0091] In detail, this situation means that after the candidate Pod is successfully released, the available resources of the node where it is located can meet the hardware resource requirements of the Pod to be scheduled. Subsequently, the Pod to be scheduled can be scheduled to the target node where the Pod to be reclaimed is located.

[0092] Step S205: Remove the candidate Pods for recycling from the second set and add them to the recycling set.

[0093] Step S206: Determine whether the current second set is an empty set. If it is an empty set, proceed to step S207; otherwise, proceed to step S208.

[0094] Step S207: Resource scheduling fails, process ends.

[0095] In detail, this situation occurs when all Pods in the current second set have been evaluated, but even in the most ideal scenario (i.e., all Pods on the same node in the reclaimed set are released), the hardware resource requirements of the Pods to be scheduled are still insufficient. At this point, there are not enough preemptible resources in the cluster, and it is necessary to continue waiting for the cluster resources to be released naturally or for manual scaling.

[0096] Step S208: Select the Pod with the highest current recycling score from the current second set as the candidate Pod.

[0097] Step S209: Evaluate the second simulated available resources of the node where the alternative Pod is located after releasing the alternative Pod.

[0098] In detail, the release of alternative Pods can be simulated in memory, that is, the amount of hardware resources occupied by them (including the number of GPUs, the number of CPU cores and the memory capacity) is deducted from the allocated resources of the node where they are located, and the virtual available resource status of the node is updated, which is the second simulated available resource here.

[0099] Step S210: Determine whether the available resources of the second simulation meet the hardware resource requirements. If they do, proceed to step S211; otherwise, proceed to step S212.

[0100] Step S211: Identify the candidate Pods as Pods to be recycled and end the process.

[0101] In detail, this situation means that when a single candidate Pod for recycling cannot meet the demand, a backup Pod is selected and its release is simulated. After the hardware resources occupied by the backup Pod are simulated to be released, the available resources on its node can meet the hardware resource requirements of the Pod to be scheduled. Subsequently, the Pod to be scheduled can be scheduled to the target node where the backup Pod is located.

[0102] Step S212: Remove the candidate Pod from the current second set and put it into the recycling set.

[0103] Step S213: In the current recycling set, group each Pod in the recycling set according to the node where each Pod is located to obtain the grouping result; for any group in the grouping result, evaluate the third simulated available resources of the node corresponding to the group after releasing all Pods in the group.

[0104] In detail, for each Pod in the reclaimed set, the node where the Pod resides is obtained by querying its `spec.nodeName` field. Pods located on the same node are grouped into the same group, forming multiple Pod groups based on nodes. For any group, all Pods within that group are simulated for release in memory. This involves sequentially deducting the amount of hardware resources (including the number of GPUs, CPU cores, and memory capacity) used by each Pod in the group from the allocated resources of its node, and updating the virtual available resource state of that node. After all Pods in the group have been simulated for release, the virtual available resource state of that node becomes the third simulated available resource at this point.

[0105] Understandably, since non-grouped tasks only need to meet the hardware resource requirements of a single scheduled Pod, and a scheduled Pod can only be scheduled to a single node, it is only necessary to evaluate whether the available resources of the same node meet the requirements after multiple Pods are released cumulatively, without considering the case of cross-node resource accumulation. The above-described node-grouped evaluation method avoids invalid cross-node accumulation judgments of resources released from different nodes, making the evaluation process more accurate and efficient.

[0106] Step S214: Determine whether the available resources of the third simulation meet the hardware resource requirements. If they do, proceed to step S215; otherwise, return to step S206.

[0107] Step S215: Identify all Pods in the group as Pods to be reclaimed and end the process.

[0108] In detail, this situation occurs when releasing a single candidate Pod still cannot meet the hardware resource requirements of the Pod to be scheduled. The candidate Pod is then added to the recycling set and grouped by node. Subsequently, after evaluation, it is found that releasing all Pods within a node group allows the node's accumulated available resources to meet the hardware resource requirements of the Pod to be scheduled. In this case, all Pods within that node group are collectively identified as Pods to be recycled. After these Pods are successfully released, the Pod to be scheduled can then be scheduled to that node.

[0109] Through the above steps, by starting with the Pod with the highest recycling score, performing single-Pod simulated release and resource satisfaction assessment sequentially, and gradually accumulating the recycling set for joint assessment when a single Pod cannot meet the requirements, the problem of how to meet scheduling requirements with the minimum recycling cost in non-grouped task scenarios is solved. This achieves the goal of minimizing the number of preempted Pods while ensuring that the Pods to be scheduled can be successfully scheduled, effectively reducing the overall impact of preemption operations on the tasks running in the cluster.

[0110] Figure 3 This is a flowchart illustrating the method for determining Pods to be reclaimed during grouped tasks in this application, such as... Figure 3 As shown, when the task group type corresponding to the Pod to be scheduled is a grouped task, the method for determining the Pod to be reclaimed from the second set based on the reclamation score includes the following steps:

[0111] Step S301: Determine the Pods in the same group as the Pod to be scheduled, and determine the group task scheduling requirements of the group task.

[0112] The group task scheduling requirement defines the total amount of hardware resources and node affinity constraints that each node must meet when the Pod to be scheduled and its group of Pods are successfully scheduled in the Kubernetes cluster. When the Kubernetes cluster can find nodes that meet the hardware resource requirements and node affinity constraints for the Pod to be scheduled and its group of Pods, ensuring that all Pods in the group task can be successfully scheduled and bound to the corresponding nodes, the current virtual available resources of the Kubernetes cluster are said to satisfy the group task scheduling requirement for that group task. It is understood that in a Kubernetes cluster, a Pod can only be deployed on one node, and one or more Pods belonging to the same group task are allowed to be deployed on the same node.

[0113] In detail, by querying the group task identifier recorded in the metadata.annotations or metadata.labels fields of the Pod to be scheduled, a list of all Pods in the Pending state under the group task can be obtained from the Kubernetes API Server, and these Pods are designated as Pods to be scheduled in the same group. Subsequently, the hardware resource requirements (including the number of GPUs, the number of CPU cores, and the memory capacity) and node affinity constraints of the Pod to be scheduled and each Pod in the same group are summarized to form the group task scheduling requirements for the group task.

[0114] Step S302: Select the Pod with the highest current recycling score from the second set as the candidate recycling Pod, and evaluate the fourth simulated available resource of the node where the candidate recycling Pod is located after releasing the candidate recycling Pod.

[0115] In detail, the release of candidate Pods for recycling can be simulated in memory. This involves deducting the amount of hardware resources occupied by the Pod (including the number of GPUs, CPU cores, and memory capacity) from the allocated resources of the node where it resides, updating the virtual available resource status of the node, and obtaining the fourth simulated available resource.

[0116] Step S303: Determine whether the available resources in the fourth simulation meet the group task scheduling requirements. If the available resources in the fourth simulation meet the group task scheduling requirements, proceed to step S304; if the available resources in the fourth simulation do not meet the group task scheduling requirements, proceed to step S305.

[0117] The process of determining whether the group task scheduling requirement is met includes: in the virtual available resources state of the cluster after simulating the release of candidate reclaimed Pods, attempting to find nodes that meet the hardware resource requirements and node affinity constraints for each Pod to be scheduled and each Pod in the same group to be scheduled. If all Pods can find suitable nodes, the group task scheduling requirement is determined to be met; if any Pod cannot find a suitable node, the requirement is determined not to be met.

[0118] Step S304: The candidate Pod to be recycled is determined as the Pod to be recycled, and the process ends.

[0119] In detail, this situation is as follows: after the candidate Pod is successfully released, the available resources of its node are sufficient to support all Pods in the entire group task (i.e., the Pod to be scheduled and its group of Pods to be scheduled), and the node affinity constraints of each Pod are satisfied. Subsequently, all Pods in the group task can be scheduled to the target node where the candidate Pod is located.

[0120] Step S305: Remove the candidate Pods for recycling from the second set and add them to the recycling set.

[0121] Step S306: Determine whether the current second set is an empty set. If it is an empty set, proceed to step S307; otherwise, proceed to step S308.

[0122] Step S307: Resource scheduling fails, process ends.

[0123] In detail, this situation occurs when all candidate Pods in the current second set have been evaluated, but even in the most ideal scenario (i.e., all Pods in the reclaim set are released), it is still impossible to find nodes that meet the hardware resource requirements and node affinity constraints for all Pods in the entire grouped task. At this point, there are not enough preemptible resources in the cluster to support the grouped task, and it is necessary to continue waiting for the cluster resources to be released naturally or for manual expansion.

[0124] Step S308: Select the Pod with the highest current recycling score from the current second set as the candidate Pod.

[0125] Step S309: Evaluate the fifth simulated available resource on the node where the alternative Pod is located after releasing the alternative Pod.

[0126] In detail, the release of the alternative Pod can be simulated in memory, that is, the amount of hardware resources occupied by it is deducted from the allocated resources of its node, the virtual available resource status of the node is updated, and the fifth simulated available resource is obtained.

[0127] Step S310: Determine whether the available resources in the fifth simulation meet the requirements for grouped task scheduling. If they do, proceed to step S311; otherwise, proceed to step S312.

[0128] Step S311: Identify the candidate Pods as Pods to be recycled and end the process.

[0129] In detail, this scenario refers to the release of alternative Pods when a single candidate Pod cannot meet the scheduling requirements of the grouped task. After the alternative Pod is released, the available resources on its node are sufficient to support all Pods in the entire grouped task, and the node affinity constraints of each Pod are satisfied. In other words, the Pod to be scheduled for this grouped task, along with other Pods in the same group, can be simultaneously scheduled to the target node where the alternative Pod resides for execution.

[0130] Step S312: Remove the candidate Pod from the current second set and put it into the recycling set.

[0131] Step S313: Evaluate the simulated available resource status of the Kubernetes cluster after releasing each Pod in the current reclaimed set.

[0132] In detail, the release of hardware resources occupied by all Pods in the current reclaim set can be simulated in memory. For each Pod in the reclaim set, its `spec.nodeName` field is obtained to determine its node, and its `spec.containers[].resources.requests` field is parsed to obtain the amount of hardware resources it occupies (including the number of GPUs, CPU cores, and memory capacity). Based on the virtual available resource state of each node maintained in memory, the amount of hardware resources occupied by each Pod is deducted from the allocated resources of the corresponding node, and the virtual available resource amount of that node is increased accordingly. After all Pods in the reclaim set have been simulated and released, the resulting virtual available resource state of each node is the simulated available resource state of the Kubernetes cluster.

[0133] Understandably, since the Pods in the reclaimed set may be distributed across different nodes in the Kubernetes cluster, the simulated release of each Pod in the reclaimed set will cause the virtual available resources on multiple nodes to increase simultaneously. This simulated available resource status of the Kubernetes cluster will serve as the basis for subsequent judgments on whether the requirements for grouped task scheduling are met.

[0134] Step S314: Determine whether the simulated available resource status of the Kubernetes cluster meets the group task scheduling requirements. If the simulated available resource status of the Kubernetes cluster meets the group task scheduling requirements, proceed to step S315; if the simulated available resource status of the Kubernetes cluster does not meet the group task scheduling requirements, return to step S306.

[0135] In detail, the simulated available resource state of the aforementioned Kubernetes cluster satisfying the grouped task scheduling requirement means that, under this simulated available resource state, nodes that meet the hardware resource requirements and node affinity constraints can be found for each scheduled Pod and each Pod in the same group, and the nodes that meet the requirements are composed of two or more nodes where each Pod in the reclamation set resides. It can be understood that the situation where releasing a candidate Pod on a single node can satisfy the grouped task scheduling requirement has already been handled in step S310 and proceeds to step S311 to end the process. Therefore, the situation addressed in step S314 specifically refers to: multiple Pods located on different nodes have accumulated in the reclamation set, and only when these nodes jointly release resources can the grouped task scheduling requirement be satisfied at the cluster level.

[0136] Step S315: Select the smallest set of Pods from the recycling set that allows the Kubernetes cluster to meet the group task scheduling requirements, identify the Pods in this smallest set as Pods to be recycled, and end the process.

[0137] In detail, this scenario occurs when releasing a single candidate Pod is insufficient to meet the scheduling requirements of grouped tasks. The candidate Pod is then added to the recycling set, which already contains multiple Pods located on different nodes. Subsequent evaluation reveals that when Pods on several nodes in the recycling set are released together, the cluster's overall simulated available resources can find nodes that meet the needs of all Pods in the grouped tasks. At this point, a subset with the fewest Pods is selected from the recycling set, ensuring that releasing this subset satisfies the grouped task scheduling requirements. Pods from this smallest subset are then identified as the Pods to be recycled.

[0138] One exemplary implementation of selecting the smallest set of Pods from the recycling set is as follows:

[0139] First, the Pods in the reclaimed collection are grouped according to their respective nodes, resulting in the grouping results. Specifically, the spec.nodeName field of each Pod is obtained, and Pods located on the same node are grouped into the same group.

[0140] Secondly, for each node's corresponding group, arrange them in descending order of each Pod's recycling score. Since Pods were already selected in descending order of recycling score when they were added to the recycling set, this order can be directly used to follow the order in which the Pods were added to the recycling set.

[0141] Then, iterate through all node combinations consisting of two or more nodes. For any node combination, select the top few Pods with the highest recycling scores from the groups corresponding to each node (the number of selections increases sequentially from 1). After simulating the release of these Pods, evaluate whether the simulated available resource state of the node combination can jointly meet the grouped task scheduling requirements. When multiple feasible node combinations exist, select the combination with the fewest total Pods; if the total number of Pods is the same, the combination with the highest sum of recycling scores can be selected first.

[0142] Finally, the set of Pods containing the combination with the smallest total number of Pods determined in the above process is taken as the minimum Pod set, and the Pods in it are identified as Pods to be recycled.

[0143] Through steps S301 to S315, in a grouped task scenario, by globally assessing the scheduling requirements of the entire grouped task, an iterative simulation release and evaluation strategy is adopted. The Pods with the highest recycling scores are successively selected from the second set for trial use. Furthermore, when the simulated available resource state of the Kubernetes cluster meets the scheduling requirements of the grouped task, the smallest set of Pods is further selected from the recycling set as the final preemption target. This method ensures that high-priority grouped tasks can be fully scheduled while releasing resources with the fewest possible Pods, effectively reducing the overall impact of preemption operations on running tasks in the cluster and solving the problems of resource waste and task failure caused by incomplete scheduling in grouped task scenarios.

[0144] The embodiments of this application will be described and illustrated below through another example.

[0145] Figure 4 This is a flowchart illustrating a resource scheduling method based on a Kubernetes cluster, as shown in an exemplary embodiment of this application. Figure 4 As shown, the method includes the following steps:

[0146] Step S401, Obtain the Pods to be Scheduled: Obtain the Pods to be scheduled in the Kubernetes cluster that failed to be scheduled due to insufficient resources.

[0147] Step S402, perform first-level filtering: Perform first-level filtering on all Pods running in the Kubernetes cluster to obtain the first set. The first-level filtering includes filtering based on task priority, and further combines at least one of the following: system namespace removal, runtime protection period removal, and exit status removal.

[0148] Step S403: Perform second-level filtering: Based on the target GPU model and node affinity constraint information required by the Pod to be scheduled, perform second-level filtering on the first set to obtain the second set.

[0149] Step S404, calculate the recycling score: Calculate the recycling score of each Pod based on the hardware resource usage of each Pod in the second set, the load information of the node where it is located, and the task priority of each Pod.

[0150] Step S405: Determine the task group type corresponding to the Pod to be scheduled: If the Pod to be scheduled belongs to a non-grouped task, proceed to step S406; if the Pod to be scheduled belongs to a grouped task, proceed to step S407.

[0151] Step S406, determine the Pod to be recycled according to method one: adopt method one, i.e., the above Figure 2 The process shown involves identifying Pods to be reclaimed from the second set. Then, proceed to step S408.

[0152] Step S407, determine the Pod to be recycled according to method two: adopt method two, i.e., the above Figure 3 The method flow shown determines the Pods to be reclaimed from the second set. Then it proceeds to step S408.

[0153] Step S408, Mark the target node: Mark the node where the Pod to be recycled is located as the target node.

[0154] Step S409, Release the Pod to be recycled: Release the Pod to be recycled from the target node.

[0155] Step S410, Schedule the Pod to be scheduled: Schedule the Pod to be scheduled to the target node.

[0156] By distinguishing the task grouping types of the Pods to be scheduled and adopting appropriate recycling strategies for non-grouped and grouped tasks respectively, the problem of the preemption mechanism in related technologies being unable to distinguish task types and causing grouped tasks to fail as a whole due to partial preemption is solved. Differentiated and refined preemption processing for different types of tasks is achieved. While ensuring the success rate of high-priority task scheduling, invalid preemption and resource waste are effectively avoided, and the overall scheduling performance and resource utilization efficiency in large-scale cluster environments are improved.

[0157] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0158] This embodiment also provides a resource scheduling device based on a Kubernetes cluster, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the terms "module," "unit," "subunit," etc., can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0159] Figure 5 The structural block diagram of the resource scheduling device based on the Kubernetes cluster provided in this application is as follows: Figure 5 As shown, the device includes:

[0160] Resource awareness module 10 is used to obtain the scheduled Pods in the Kubernetes cluster that have failed to be scheduled due to insufficient resources;

[0161] The candidate filtering module 20 is used to perform a first-level filter on all Pods running in the Kubernetes cluster to obtain a first set; the task priority of each Pod in the first set is lower than the task priority of the Pod to be scheduled; the first set is then filtered a second time according to the target GPU model required by the Pod to be scheduled to obtain a second set; each Pod in the second set is bound to a GPU with a target GPU model identifier.

[0162] The scoring module 30 is used to determine the recycling score of each Pod in the second set based on the hardware resource consumption of each Pod, the load information of the node where each Pod is located, and the task priority of each Pod.

[0163] The recycling confirmation module 40 is used to determine the Pods to be recycled from the second set based on the recycling score, and mark the node where the Pod to be recycled is located as the target node;

[0164] The scheduling coordination module 50 is used to release the Pods to be recycled from the target node and schedule the Pods to be scheduled to the target node.

[0165] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination. Specific examples in this embodiment can be found in the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0166] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0167] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0168] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.< / name>

Claims

1. A resource scheduling method based on a Kubernetes cluster, characterized in that, include: Retrieve the Pods in the Kubernetes cluster that failed to be scheduled due to insufficient resources; All Pods running in the Kubernetes cluster are filtered at the first level to obtain a first set; the task priority of each Pod in the first set is lower than the task priority of the Pod to be scheduled. Based on the target GPU model required by the Pod to be scheduled, the first set is filtered in a second level to obtain a second set; each Pod in the second set is bound to a GPU identified by the target GPU model. Based on the hardware resource consumption of each Pod in the second set, the load information of the node where each Pod is located, and the task priority of each Pod, the recycling score of each Pod in the second set is determined. Based on the recycling score, determine the Pods to be recycled from the second set, and mark the node where the Pods to be recycled are located as the target node; Release the Pod to be recycled from the target node and schedule the Pod to be scheduled to the target node.

2. The resource scheduling method based on a Kubernetes cluster according to claim 1, characterized in that, The process of obtaining the scheduled Pods in the Kubernetes cluster that failed to be scheduled due to insufficient resources includes: Obtain the list of Pods to be scheduled corresponding to the Kubernetes cluster; Pods that fail to be scheduled due to insufficient resources in the list of Pods to be scheduled are identified as candidate Pods for scheduling. Based on the task priority of the candidate scheduling Pods, the first target Pod with a task priority higher than a preset threshold is selected from the candidate scheduling Pods as the Pod to be scheduled.

3. The resource scheduling method based on a Kubernetes cluster according to claim 1, characterized in that, The first-level filtering of all Pods running in the Kubernetes cluster yields a first set, including: Obtain the set of Pods that are currently running in the Kubernetes cluster; According to the preset removal rules, Pods that meet the removal rules are removed from the Pod set to obtain the first set; wherein, the preset removal rules include at least one of the following rules: remove Pods under the system namespace, remove Pods whose task priority is not lower than the task priority of the Pod to be scheduled, remove Pods whose runtime is less than a preset protection period threshold, and remove Pods that are currently in the exit phase.

4. The resource scheduling method based on a Kubernetes cluster according to claim 1, characterized in that, The second set is obtained by performing a second-level filter on the first set based on the target GPU model required by the Pod to be scheduled, including: Based on the target GPU model and the GPU models bound to each Pod in the first set, a first candidate Pod set matching the target GPU model is determined; Based on the node affinity constraint information of the Pod to be scheduled and the nodes where each Pod in the first candidate Pod is located, a second target Pod that satisfies the node affinity constraint information is searched from the first candidate Pod set; the node affinity constraint information is used to limit the range of nodes that the Pod to be scheduled can be scheduled. The set consisting of the second target Pods is defined as the second set.

5. The resource scheduling method based on a Kubernetes cluster according to claim 1, characterized in that, When the task group type corresponding to the Pod to be scheduled is a non-grouped task, the step of determining the Pod to be reclaimed from the second set based on the reclamation score includes: Select the Pod with the highest current recycling score from the second set as the candidate Pod for recycling; After releasing the candidate Pod for recycling, assess the first simulated available resources of the node where the candidate Pod for recycling resides. When the first simulated available resources meet the hardware resource requirements of the Pod to be scheduled, the candidate Pod to be recycled is determined as the Pod to be recycled.

6. The resource scheduling method based on a Kubernetes cluster according to claim 5, characterized in that, When the first simulated available resources do not meet the hardware resource requirements. Remove the candidate Pods to be recycled from the second set and add them to the recycling set; Select the Pod with the highest current recycling score from the current second set as the candidate Pod; After releasing the candidate Pod, assess the second simulated available resources of the node where the candidate Pod resides; When the second simulated available resources meet the hardware resource requirements, the candidate Pod is determined as the Pod to be recycled; When the available resources in the second simulation do not meet the hardware resource requirements, the alternative Pod is removed from the current second set and placed into the recycling set; In the current recycling set, the Pods in the recycling set are grouped according to the node where each Pod is located, and the grouping results are obtained; For any group in the grouping results, evaluate the third simulated available resources of the node corresponding to the group after releasing all Pods in the group; When the third simulated available resources meet the hardware resource requirements, all Pods in the group are identified as the Pods to be reclaimed. If the available resources in the third simulation do not meet the hardware resource requirements, the step of selecting the Pod with the highest current recycling score from the current second set as the candidate Pod is executed again.

7. The resource scheduling method based on a Kubernetes cluster according to claim 1, characterized in that, When the task group type corresponding to the Pod to be scheduled is a grouped task, the step of determining the Pod to be reclaimed from the second set based on the reclamation score includes: Identify the Pods in the same group as the Pod to be scheduled and determine the group task scheduling requirements of the group task; the group task scheduling requirements are used to limit the total amount of hardware resources and node affinity constraints that each node in the Kubernetes cluster needs to meet when the Pod to be scheduled and the Pods in the same group are successfully scheduled in the Kubernetes cluster. Select the Pod with the highest current recycling score from the second set as the candidate Pod for recycling; After evaluating and releasing the candidate Pod for recycling, assess the fourth simulated available resources of the candidate Pod for recycling; When the fourth simulated available resources meet the group task scheduling requirements, the candidate Pod to be recycled is determined as the Pod to be recycled.

8. The resource scheduling method based on a Kubernetes cluster according to claim 7, characterized in that, When the fourth simulated available resource does not meet the group task scheduling requirements, the method further includes: Remove the candidate Pods to be recycled from the second set and add them to the recycling set; Select the Pod with the highest current recycling score from the current second set as the candidate Pod; After the alternative Pod is released, assess the fifth simulated available resource of the node where the alternative Pod is located; When the fifth simulated available resource meets the group task scheduling requirements, the candidate Pod is determined as the Pod to be reclaimed; When the fifth simulated available resources do not meet the group task scheduling requirements, the alternative Pod is removed from the current second set and placed into the recycling set; The simulated available resource status of the Kubernetes cluster after assessing and releasing each Pod in the currently reclaimed set; When the simulated available resources of the Kubernetes cluster meet the group task scheduling requirements, the smallest set of Pods that makes the Kubernetes cluster meet the group task scheduling requirements is selected from the recycling set, and the Pods in the smallest set of Pods are determined as the Pods to be recycled. If the simulated available resource status of the Kubernetes cluster does not meet the group task scheduling requirements, the step of selecting the Pod with the highest current recycling score from the current second set as the candidate Pod is executed again.

9. The resource scheduling method based on a Kubernetes cluster according to claim 1, characterized in that, The step of releasing the Pod to be recycled from the target node and scheduling the Pod to be scheduled to the target node includes: Call the Kubernetes Eviction API to perform a release operation on the Pod to be reclaimed; After detecting that the Pod to be reclaimed has terminated and the hardware resources occupied by the Pod to be reclaimed have been successfully released, a resource change notification or scheduling trigger signal is sent to the scheduler in the Kubernetes cluster so that the scheduler binds the Pod to be scheduled to the target node.

10. A resource scheduling device based on a Kubernetes cluster, characterized in that, include: The resource awareness module is used to identify Pods in the Kubernetes cluster that have failed to be scheduled due to insufficient resources. The candidate filtering module is used to perform a first-level filter on all Pods running in the Kubernetes cluster to obtain a first set; the task priority of each Pod in the first set is lower than the task priority of the Pod to be scheduled; the first set is then filtered a second time according to the target GPU model required by the Pod to be scheduled to obtain a second set; each Pod in the second set is bound to a GPU identified by the target GPU model. The scoring module is used to determine the recycling score of each Pod in the second set based on the hardware resource consumption of each Pod, the load information of the node where each Pod is located, and the task priority of each Pod. The recycling confirmation module is used to determine the Pods to be recycled from the second set based on the recycling score, and mark the node where the Pods to be recycled are located as the target node; The scheduling and coordination module is used to release the Pod to be recycled from the target node and schedule the Pod to be scheduled to the target node.

Images