A resource scheduling method and related apparatus

Abstract

This application discloses a resource scheduling method and related apparatus. The method includes: obtaining a first resource utilization rate for each node; wherein, a node is used to provide resources to a container group according to a scheduling request from a container group deployed on the node; filtering each node according to a first utilization rate threshold to obtain at least one target node; wherein, the first resource utilization rate of the target node is less than or equal to the first utilization rate threshold; for each target node, applying a scoring mechanism that matches the priority of the target container group, and scoring the target node according to its second resource utilization rate and resource allocation rate; and implementing resource scheduling according to the priority of the target container group and the scores of each target node corresponding to the priority. This achieves load balancing among nodes and improves the operational stability of high-priority tasks.

Description

Technical Field

[0001] This application relates to the field of resource processing technology, and in particular to a resource scheduling method and related apparatus. Background Technology

[0002] Container cloud is a cloud service product that deploys container services on cluster servers using Docker technology; it can be viewed as a lightweight Linux cloud server. Currently, many container cloud platforms provide application runtime platforms through technologies such as Docker and Kubernetes, enabling rapid application deployment, elastic scaling, and dynamic adjustment of application environment resources. Kubernetes is a management system capable of orchestrating and scheduling containers for the automatic deployment, scaling, and management of applications. Kubernetes consists of multiple nodes; a node is the smallest computing hardware unit in Kubernetes, and each node can provide the computing resources, memory resources, and other resources required for application operation.

[0003] Each application's execution can be considered a task. When implementing tasks in different scenarios through a cluster server, cluster scheduling is required to allocate appropriate resources for task execution. Therefore, efficient resource scheduling is a crucial issue in the field of cluster scheduling. A resource scheduling system can allocate suitable physical nodes based on the task's resource requests to allocate corresponding resources for task execution.

[0004] Among related technologies, the first approach is static scheduling, which compares the requested resources with the node's available resources to determine if the node has sufficient resources to accommodate the container group deploying the task. This approach may result in low cluster resource utilization. The second approach is dynamic scheduling. One scenario of dynamic scheduling is to consider only historical resource utilization, which leads to poor service stability during resource scheduling. Another scenario of dynamic scheduling is to reclaim allocated but unused resources for rescheduling. However, the algorithm for calculating reclaimable resources is complex, and when resource usage fluctuates or experiences daily tidal phenomena, it becomes difficult to calculate reclaimable resources over long periods. Summary of the Invention

[0005] This application provides a resource scheduling method and related apparatus to improve the utilization rate of cluster resources and enhance the stability of task operation.

[0006] In a first aspect, one embodiment of this application provides a resource scheduling method, including:

[0007] Obtain the first resource utilization of each node; whereby a node is used to provide resources to a container group based on the scheduling requests of the container group deployed on the node;

[0008] For each target node, a scoring mechanism matching the priority of the target container group is applied, and the target node is scored based on its second resource utilization rate and resource allocation rate; wherein, the target node is obtained by filtering each node according to the first utilization rate threshold, and the first resource utilization rate of the target node is less than or equal to the first utilization rate threshold.

[0009] Resource scheduling is achieved based on the priority of the target container group and the score of each target node corresponding to that priority.

[0010] Secondly, one embodiment of this application provides a resource scheduling apparatus, including:

[0011] The acquisition module is used to acquire the first resource utilization of each node; the node is used to provide resources to the container group according to the scheduling request of the container group deployed on the node;

[0012] The scoring module is used to apply a scoring mechanism that matches the priority of the target container group to each target node, and to score the target node based on the second resource utilization rate and the resource allocation rate of the target node; wherein, the target node is obtained by filtering each node according to the first utilization rate threshold, and the first resource utilization rate of the target node is less than or equal to the first utilization rate threshold.

[0013] The scheduling module is used to schedule resources based on the priority of the target container group and the score of each target node corresponding to the priority.

[0014] Thirdly, one embodiment of this application provides a resource scheduling system, which includes a scheduler and at least one component. The scheduler is used to process data monitored by the at least one component to implement the steps of any of the above methods.

[0015] Fourthly, one embodiment of this application provides a resource scheduling device, including a memory, a processor, a computer program stored in the memory and executable on the processor, and a resource scheduling system of the third aspect, wherein the resource scheduling system is deployed on the processor, and the processor executes the steps of any of the above methods when executing the computer program.

[0016] Fifthly, one embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of any of the above methods.

[0017] Sixthly, one embodiment of this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above methods.

[0018] The embodiments of this application have the following beneficial effects:

[0019] First, nodes are screened, filtering out those with high loads to obtain target nodes whose resource utilization is below a first utilization threshold. Tasks are then prioritized, meaning the corresponding target container groups are divided into multiple priority groups. A score is assigned to each target node based on both its second resource utilization and resource allocation rate. Resource scheduling is then implemented based on the priority of the target container group and the scores of the target nodes corresponding to that priority. Tasks of different priorities are dynamically scheduled with balanced load, improving cluster resource utilization while maximizing load balancing among nodes. This significantly improves the operational stability of high-priority tasks. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A schematic diagram of a resource scheduling framework provided in an embodiment of this application;

[0022] Figure 2 This is a schematic flowchart of a resource scheduling method provided in an embodiment of this application;

[0023] Figure 3 This is a schematic diagram of resource scheduling before one embodiment of the present application;

[0024] Figure 4 A schematic diagram illustrating a resource scheduling process according to an embodiment of this application;

[0025] Figure 5 A schematic diagram illustrating node resource overselling as provided in one embodiment of this application;

[0026] Figure 6 This is a schematic diagram of the structure of a resource scheduling device provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the structure of a resource scheduling device provided in an embodiment of this application. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0029] For ease of understanding, the terms used in the embodiments of this application are explained below:

[0030] (1) Representational state transfer (REST) ​​is an architectural style. The representation layer refers to the form in which resources are presented in a concrete way.

[0031] (2) An application programming interface (API) is a set of rules for defining how applications or devices connect and communicate with each other. It is a mechanism that enables one application or service to access resources in another application or service.

[0032] The number of any elements in the accompanying drawings is for illustrative purposes only and not as a limitation, and any naming is for distinction only and has no limiting meaning.

[0033] Container cloud is a cloud service product that deploys container services on clustered servers using Docker technology. It can be viewed as a lightweight Linux cloud server. With customizable configurations and networking, it can be used as a virtual host, cloud server, or cluster server, suitable for website building, running applications, configuring load balancers, and building service clusters. Customers don't need to handle complex underlying server maintenance; they only need to install the provided images to run containers, and only pay for the container resources they use. Therefore, while meeting functional requirements, using container cloud can maximize cost savings and reduce operating expenses.

[0034] Many container cloud platforms currently provide application runtime platforms through technologies such as Docker and Kubernetes, thereby achieving automated operation and maintenance, rapid application deployment, elastic scaling, and dynamic adjustment of application environment resources, improving R&D and operational efficiency. Kubernetes is a management system capable of orchestrating and scheduling containers for the automatic deployment, scaling, and management of applications. Kubernetes consists of multiple nodes, which are the smallest computing hardware units in Kubernetes. Each node can provide the computing resources, memory resources, and other resources required for application operation. In Kubernetes, one or more containers are encapsulated in a container group (pod) and run on a node. Kubernetes provides a built-in load balancer and scalable automatic resource scheduling capabilities. The scheduler is responsible for collecting and analyzing the resources already occupied by nodes, and based on the analysis results, scheduling resources according to the resource requests of pods, thereby allocating newly created container groups to available nodes for deployment. The pod request resource configuration includes two parts: `request` and `limit`. `limit` specifies the maximum CPU and memory specifications a single pod can use, while `request` guarantees the minimum resources the system can allocate to a single pod. `request` affects scheduling. During scheduling, the scheduling component uses the available resources on each node and the already allocated pod requests to calculate the available resources. If the allocated requests are equal to or close to the available resources, no more pods will be scheduled. Resources can include CPU resources, memory resources, and other resources.

[0035] This shows that Kubernetes uses static scheduling, which means scheduling based on the resources requested by the container. That is, it compares the resources requested by the pod with the available resources on the node to determine if the node has enough resources to accommodate the pod. The biggest advantage of static scheduling is its simplicity, efficiency, and ease of cluster resource management. However, in real-world environments, tasks often have a degree of subjectivity and arbitrariness in choosing container sizes. To ensure task stability, tasks may request far more resources than they actually need, resulting in very low resource utilization for the task containers. If a large proportion of such tasks exist, it can easily lead to low cluster resource utilization.

[0036] To address the issue of low actual utilization of cluster resources, the Kubernetes community primarily employs methods such as pod resource allocation compression and AutoScale elastic scaling. Pod resource allocation compression involves designing a compression ratio when a pod is created. When a pod creation request is received, the container cloud platform automatically modifies the pod's resource request value based on this compression ratio. This compresses the resource requests across the entire cluster, allowing the cluster to create more pods and thus improving actual resource utilization. However, compressing pod resource requests only occurs during pod creation or rebuilding, such as service deployment or upgrades. It cannot be performed while the service is running, and the actual load fluctuations of each pod vary, making determining the compression ratio a challenge. AutoScale elastic scaling primarily employs horizontal scaling (HPA) and vertical scaling (VPA). HPA dynamically increases or decreases the number of service pods based on actual resource usage monitoring data, thereby ensuring that the actual resource utilization of each pod closely approximates its resource request value. VPA also dynamically adjusts the resource request and limit values ​​of a pod based on actual resource usage monitoring data, making the actual resource utilization of each pod close to the pod's resource request value. However, HPA reduces the number of pods when tasks are idle. When task load fluctuates, new pods need to be created and started to scale up, which is time-consuming and unacceptable for tasks. When VPA modifies pod resources, it rebuilds the pod, causing application tasks to be interrupted.

[0037] In summary, the Kubernetes community solutions all have certain problems in solving cluster resource load optimization. A dynamic scheduling scheme needs to be implemented based on specific task scenarios to improve the actual utilization of cluster resources.

[0038] Dynamic scheduling solutions in related technologies fall into two categories. One is to over-sell node resources based on historical node resource utilization. However, pod resource usage can fluctuate, and when pod resource utilization spikes after dynamic scheduling based on historical node utilization, it can lead to resource shortages for other pods, affecting service stability. Another approach involves reclaiming allocated but unused resources from various types of resources on node components and repurposing them for scheduling. However, algorithms for calculating reclaimable resources are complex, and calculating reclaimable resources over long periods becomes difficult when resource usage fluctuates or experiences daily-level fluctuations. Furthermore, differentiating between reclaimable and schedulable resources in scheduling, and using different resources for services of different quality levels, results in two sets of resource views for the scheduler, further complicating the scheduling algorithm.

[0039] Therefore, for scenarios with low cluster resource utilization, this application proposes a simple dynamic scheduling method that is aware of node complexity. Based on the historical load information of nodes, pods are dynamically scheduled to increase the number of pods deployed within a node and improve cluster resource utilization. Furthermore, by defining task priorities, resources for pods corresponding to high-priority tasks are guaranteed, thereby ensuring the stability of high-priority task operation.

[0040] After introducing the design concept of the embodiments of this application, the following is a brief introduction to the application scenarios to which the technical solutions of the embodiments of this application can be applied. It should be noted that the application scenarios described below are only for illustrating the embodiments of this application and are not intended to limit the scope. In specific implementation, the technical solutions provided by the embodiments of this application can be flexibly applied according to actual needs.

[0041] To further illustrate the technical solutions provided in the embodiments of this application, a detailed description is provided below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiments of this application provide method operation steps as shown in the following embodiments or drawings, the method may include more or fewer operation steps based on conventional or non-inventive methods. In steps where there is no logically necessary causal relationship, the execution order of these steps is not limited to the execution order provided in the embodiments of this application.

[0042] refer to Figure 1 The following describes a resource scheduling framework provided by an embodiment of this application.

[0043] Kube-apiserver is a native core component of Kubernetes, providing Kubernetes REST APIs (APIs that conform to REST architectural design principles) and is responsible for storing various types of information in Kubernetes.

[0044] Kubelet is a native node proxy component for Kubernetes, responsible for maintaining the total amount and allocable resources of a node, as well as creating containers.

[0045] Node exporter is a node monitoring and data collection component that collects real-time information on the actual usage of node resources.

[0046] Prometheus is an open-source monitoring component that stores the collected information and performs statistical aggregation on the monitoring data based on query conditions.

[0047] Node-annotator is a self-developed component responsible for pulling monitoring data from Prometheus and periodically synchronizing the node's real load information (such as CPU usage and memory usage) to the node's annotation field. At the same time, it dynamically calculates the node's resource overselling coefficient based on the node's allocated resources and actual utilization.

[0048] Scheduler is the native scheduler for Kubernetes, which schedules resources based on pod request resources and the available resources on a node.

[0049] Extern-scheduler is a self-developed extended scheduler that periodically obtains node load information from the node information annotation field by listening to kube-apiserver, and performs scheduling based on the actual node load.

[0050] When pod resource usage fluctuates or resource tides occur on a node, pods compete for resources, leading to the eviction of some pods. This application addresses this issue by defining different priorities for tasks, thus achieving different levels of service quality guarantees. Priorities describe the order in which tasks satisfy pod resource requests and the order in which pods are evictioned when node resources are scarce. Priorities are defined using Kubernetes' PriorityClass resource, including three levels: Prod (high), Mid (medium), and Low (low). Prod priority tasks have the highest priority resource guarantee; these pods are evicted last during node eviction. They are typically used for high-priority online tasks, where interruption and resource delay response are not allowed. Low priority tasks have the lowest priority resource guarantee; when resources are scarce, requested resources cannot be guaranteed, and these pods are evicted first during node eviction. They are typically used for offline tasks and other tasks that are not sensitive to latency, where execution is interrupted or paused for short periods.

[0051] Combination Figure 1 ,refer to Figure 2The technical solutions provided in the embodiments of this application are described below. This application provides a resource scheduling method applied to a container cloud platform, which includes at least the following steps:

[0052] S201, Obtain the first resource utilization rate of each node.

[0053] The node is used to provide resources to the container group based on the scheduling requests of the container group deployed on the node.

[0054] S202. For each target node, apply a scoring mechanism that matches the priority of the target container group, and score the target node based on the second resource utilization rate and resource allocation rate.

[0055] The target node is obtained by filtering each node according to the first utilization threshold, and the first resource utilization of the target node is less than or equal to the first utilization threshold.

[0056] S203. Resource scheduling is implemented based on the priority of the target container group and the score of each target node corresponding to the priority.

[0057] In the above embodiment, nodes are first screened, filtering out nodes with high loads to obtain target nodes whose first resource utilization is less than a first utilization threshold. Tasks are then prioritized, meaning the corresponding target container groups are divided into multiple priority groups, and each target node is scored by comprehensively considering its second resource utilization and resource allocation rate. Resource scheduling is then implemented based on the priority of the target container group and the scores of the target nodes corresponding to that priority. Tasks of different priorities are dynamically scheduled with balanced load, improving cluster resource utilization while maximizing load balancing among nodes. This significantly improves the operational stability of high-priority tasks.

[0058] Regarding S101, container groups have been deployed on the node. The tasks within these containers utilize resources during execution, which typically include CPU and memory resources. The primary resource utilization includes the CPU and memory utilization determined by the deployed container groups during resource usage. In practical applications, resources may include other resources as well; the CPU and memory resources mentioned here are merely illustrative and do not constitute a specific limitation.

[0059] For example, the Extern-scheduler extended dynamic scheduler listens to changes in Node information, obtains node load information from the Node information annotation field, and performs balanced scheduling of tasks of different priorities based on the node load information.

[0060] Taking a single node as an example, the process of obtaining the first resource utilization rate of that node is explained as follows:

[0061] The Node-annotator primarily synchronizes the actual load information of nodes. It monitors data through the Prometheus monitoring component and calculates the node's average CPU utilization over the past 5 minutes (e.g., 0.33142), average CPU utilization over 1 hour (e.g., 0.33495), and maximum CPU utilization over 24 hours (e.g., 0.33295), as well as average memory utilization over 5 minutes (e.g., 0.3401), average memory utilization over 1 hour (e.g., 0.3461), and maximum memory utilization over 24 hours (e.g., 0.3525). The node-annotator component then updates the annotation field in the node with this data.

[0062] In addition, the weights of each utilization rate are pre-configured. For example, the weight of the 5-minute average CPU utilization is 0.2, the weight of the 1-hour average CPU utilization is 0.3, and the weight of the 24-hour maximum CPU utilization is 0.5. Similarly, the weight of the 5-minute average memory utilization is 0.2, the weight of the 1-hour average memory utilization is 0.3, and the weight of the 24-hour maximum memory utilization is 0.5.

[0063] By summing the average utilization rates and their weights, we get the CPU utilization rate as 0.33142*0.2 + 0.33495*0.3 + 0.33295*0.5 = 0.333244, and the memory utilization rate as 0.3401*0.2 + 0.3461*0.3 + 0.3525*0.5 = 0.3481.

[0064] The CPU utilization and memory utilization of each node are obtained using the above method. The first utilization threshold includes a first CPU utilization threshold and a first memory utilization threshold. The first CPU utilization threshold and the first memory utilization threshold can be used to filter each node. Specifically, nodes with CPU utilization greater than the first CPU utilization threshold and nodes with memory utilization greater than the first memory utilization threshold are deleted. With this design, the CPU utilization of the target nodes after filtering is less than or equal to the first CPU utilization threshold, and the memory utilization of the target nodes after filtering is less than or equal to the first memory utilization threshold.

[0065] Regarding S202, for each target node, a scoring mechanism matching the priority of the target container group is applied, and the target node is scored based on its second resource utilization rate and resource allocation rate.

[0066] The target container group is the group of containers that will be scheduled to the node, and the priority of the target container group is the priority of the tasks it executes. Different tasks have different priorities, and according to a pre-defined priority determination rule, they are divided into high-priority container groups, medium-priority container groups, and low-priority container groups.

[0067] Because the scheduling principles for target container groups with different priorities are different, their matching scoring mechanisms are also different. If the priority of the target container group is high, the priority matching scoring mechanism for the target container group is the first scoring mechanism; if the priority of the target container group is medium or low, the priority matching scoring mechanism for the target container group is the second scoring mechanism.

[0068] In the first scoring mechanism, the second resource utilization rate is negatively correlated with the score; that is, the lower the second resource utilization rate, the higher the score. Similarly, the resource allocation rate is negatively correlated with the score; that is, the lower the resource allocation rate, the lower the score. In the second scoring mechanism, the second resource utilization rate is negatively correlated with the score; that is, the lower the second resource utilization rate, the higher the score. The resource allocation rate is positively correlated with the score; that is, the lower the resource allocation rate, the lower the score. The difference between the two scoring mechanisms is that high-priority target container groups can be scheduled to nodes with low resource allocation rates. However, to reserve resources for high-priority target containers, medium-priority and low-priority containers are scheduled to nodes with high resource allocation rates.

[0069] Due to the different scoring mechanisms, the same target node has two scores: a first target score and a second target score. The first target score is the score for the target node when the target container group has a high priority, while the second target score is the score for the target node when the target container group has a low priority.

[0070] Given a defined scoring mechanism, the scoring process for a target node will be explained using a single target node as an example.

[0071] The process of determining the first target score:

[0072] (1) Apply the first scoring mechanism to score the target node based on the second resource utilization rate of the target node and obtain the first score.

[0073] Here, different weights can be set for CPU utilization and memory utilization, and the weighted average is used to obtain the second resource utilization rate. The higher the second resource utilization rate, the higher the score of the target node. In this step, the first score of the target node is obtained according to this rule.

[0074] (2) The target node is scored based on the resource allocation rate to obtain the second score.

[0075] The lower the resource allocation rate, the higher the target node's score. In this step, the target node's second score is obtained according to this rule.

[0076] (3) Weight the first score and the second score to obtain the first target score of the target node when the priority of the target container group is high.

[0077] Based on the pre-set weights of the first and second scores, the first and second scores are weighted together to obtain the first target score of the target node when the priority of the target container group is high.

[0078] The process of determining the second target score:

[0079] (1) Apply the second scoring mechanism to score the target node based on the second resource utilization rate of the target node, and obtain the third score.

[0080] The higher the resource utilization rate, the higher the target node's score. In this step, the target node's third score is obtained according to this rule.

[0081] (2) The target node is scored based on the resource allocation rate to obtain the fourth score.

[0082] The lower the resource allocation rate, the lower the target node's score. In this step, the target node's fourth score is obtained according to this rule.

[0083] (3) Weight the third score and the fourth score to obtain the second target score of the target node when the target container group has a high priority.

[0084] Based on the pre-set weights of the third and fourth scores, the third and fourth scores are weighted together to obtain the second target score of the target node when the target container group has a high priority.

[0085] Regarding S203, the resource scheduling process is explained according to the different priorities of the target container groups.

[0086] In the first case, the target container group has a high priority. In this case, the target node's score is the first target score.

[0087] A. If there are N target nodes whose second resource utilization rate is less than or equal to the second utilization rate threshold and whose resource allocation rate is less than or equal to the allocation rate threshold, schedule the target container group to the target node with the highest first target score among the N target nodes.

[0088] The lower the second resource utilization rate and the lower the resource allocation rate, the more suitable the target container group is to be scheduled to the target node. Therefore, if N target nodes have a second resource utilization rate less than or equal to the second utilization rate threshold and a resource allocation rate less than or equal to the allocation rate threshold, this design can schedule the target container group to the target node with the highest first target score among the N target nodes.

[0089] B. If the resource allocation rate of each target node is greater than the allocation rate threshold, the target container group will be scheduled to the target node with the second lowest resource utilization rate among all target nodes.

[0090] If the resource allocation rate of each target node is greater than the allocation rate threshold, it indicates that the resource allocation rate of each target node is already high. In this case, scheduling does not need to be based on the first target score. Instead, the target node with the lowest resource utilization rate can be selected from among the target nodes, and the target container group can be scheduled to that target node.

[0091] C. When the second resource utilization rate of each target node is greater than the second utilization rate threshold, delete the low-priority container group in the first reference target node and schedule the target container group to the first reference target node; wherein, the first reference target node is the target node with the highest second target score among the target nodes with low-priority container groups.

[0092] In this example, if the second resource utilization rate of each target node is greater than the second utilization rate threshold, it indicates that the second resource utilization rate of each target node is high. Among the target nodes, those containing low-priority container groups are identified, and the target node with the highest second target score is designated as the first reference target node. The low-priority container groups in the first reference target node are removed, and the target container groups are scheduled to the first reference target node. In this example, reserving resources for high-priority task scheduling ensures that resources for high-priority container groups are prioritized.

[0093] In the second scenario, the target container group has a medium or low priority. In this scenario, the target node's score is the second target score.

[0094] A. If there are M target nodes whose second resource utilization rate is less than or equal to the second utilization rate threshold and whose resource allocation rate is less than or equal to the allocation rate threshold, schedule the target container group to the target node with the highest second target score among the M target nodes; where M is an integer greater than or equal to 1 and less than or equal to the total number of target nodes.

[0095] This example can be found in step A of the first case, and will not be repeated here.

[0096] B. When the second resource utilization rate of each target node is greater than the second utilization rate threshold, for the medium-priority target container group, delete the low-priority container group in the second reference target node, and schedule the medium-priority target container group to the second reference target node; control the low-priority target container group to wait for scheduling, wherein the second reference target node is the target node with the highest second target score among the target nodes with low-priority container groups.

[0097] This example can be found in step B of the first scenario, and will not be repeated here. Additionally, low-priority target container groups have no resources available for allocation and will wait indefinitely.

[0098] In a specific example, see Figure 3 This shows a schematic diagram before resource scheduling. Figure 4 A schematic diagram of a resource scheduling process is shown. After multi-priority load balancing scheduling is completed, as pods on a node experience resource usage fluctuations or resource tides, the node load may increase, leading to resource preemption by pods and affecting pod operational stability. This application uses a priority eviction strategy to migrate pods to nodes with lower loads to balance the node load.

[0099] The Descheduler listens to the Kube-apiserver to obtain node load and pod information. When a node's load is high, it implements an eviction policy, prioritizing the eviction of low-priority task pods, followed by medium-priority task pods, and finally high-priority task pods. Evicted pods are then dynamically scheduled to low-priority nodes by the scheduler, thus reducing the high load on the nodes. Simultaneously, due to the eviction priority order, high-priority task pods are avoided as much as possible, ensuring the stability and resource utilization of high-priority tasks.

[0100] Because the resource scheduling process considers both the relationship between the second resource utilization rate and the second utilization rate threshold, as well as the relationship between the resource allocation rate and the allocation rate threshold, the calculation process of the resource allocation rate will be explained below using a target node as an example.

[0101] The actual total resources of the target node are determined based on the resource overselling coefficient of the target node, and the resource allocation rate is determined based on the actual total resources and the allocated resources.

[0102] The resource overselling coefficient varies depending on the resource type. In this embodiment, the resource overselling coefficient includes the CPU resource overselling coefficient and the memory resource overselling coefficient, and the actual total resource includes the actual total CPU resource and the actual total memory resource.

[0103] For any given node, in scenarios where the actual resource usage of the container group is far lower than the requested resources, a common problem arises: the node's actual load is low, but its allocable resources have already been exhausted by the requested resources from the container group, preventing new container groups from scheduling to that node. By overselling node resources, the allocated but unused resources on the node can be made available for scheduling by new container groups.

[0104] The Node-annotator dynamically calculates the node resource overselling coefficient based on the node's actual historical load information, and Kubelet then uses this coefficient to oversell node resources. A diagram illustrating node resource overselling is shown below. Figure 5 As shown.

[0105] By modifying the Kubelet logic, when Kubelet periodically reports and updates the allocable resources of a node in the Node status, the actual total resources and allocable resources in the Node status are modified based on the node resource over-sale coefficient in the Node information annotation field. The actual total resources are calculated and defined as follows:

[0106] Rr=F*T

[0107] Where r represents the resource type, i.e., CPU or memory, F represents the oversold coefficient of the corresponding resource, and T represents the initial total amount of resources.

[0108] For example, the actual total CPU resources are determined based on the CPU resource overselling factor and the initial total CPU resources of the target node. For instance, if the CPU resource overselling factor is 1.5 and the initial total CPU resources of the target node are 100 cores, then the actual total CPU resources are 100 cores * 1.5 = 150 cores.

[0109] For example, the actual total memory resources are determined based on the memory resource overselling coefficient and the initial total memory resources of the target node. For instance, if the memory resource overselling coefficient is 1.3 and the initial total memory resources of the target node are 1G, then the actual total memory resources are 1.3G.

[0110] The calculation of the allocatable resources of a node is defined as follows:

[0111] Ur = F * TA

[0112] Where r represents the resource type, i.e., CPU or memory, F represents the oversold coefficient of the corresponding resource, T represents the initial total amount of resources, and A represents the amount of resources already allocated.

[0113] As above, for reference Figure 5This diagram illustrates a method for overselling node resources. By virtually increasing the total resources of a node through an overselling coefficient, the available resources of the node are indirectly expanded. This is equivalent to adding allocated but unused resources from the node to the available resources for reuse, thereby improving the actual utilization rate of node resources.

[0114] The first resource allocation rate is determined based on the actual total CPU resources and the allocated CPU resources; the second resource allocation rate is determined based on the actual total memory resources and the allocated memory resources; the first resource allocation rate and the second resource allocation rate are weighted to obtain the resource allocation rate.

[0115] For example, if the actual total CPU resources are 150 cores, and the allocated CPU resources are 30 cores, then the first resource allocation rate is 0.2. If the actual total memory resources are 1.3GB, and the allocated memory resources are 0.3GB, then the second resource allocation rate is 0.23. If the weight of the first resource allocation rate is, for example, 0.4, and the weight of the second resource allocation rate is, for example, 0.6, then the weighted resource allocation rate is 0.2*0.4 + 0.23*0.6 = 0.186.

[0116] During the process, the resource overselling coefficient was determined as follows:

[0117] For the first resource, the allocation rate of the first resource is determined based on the amount of first resource already allocated to the current target node and the initial total amount of first resource; the resource overselling coefficient of the first resource is determined based on the allocation rate of the first resource, the resource utilization rate of the first resource, and the weight corresponding to the resource utilization rate; where the first resource is CPU resource or memory resource.

[0118] For example, taking CPU resources as an example, the allocated CPU resources are 50 cores, the initial total CPU resources are 100 cores, and the CPU resource allocation rate is 0.5. Then, based on the CPU resource allocation rate of 0.5, the CPU resource utilization rate, and the weight corresponding to the resource utilization rate (see the aforementioned embodiment), the CPU resource overselling coefficient is determined.

[0119] Continuing with CPU resources as an example, the resource overselling coefficient can be determined using the following formula:

[0120]

[0121] Where F is the resource overselling coefficient, and a is the current resource allocation rate; Let represent resource utilization rate, where usage_i is the resource utilization rate of the i-th preset time period, weight_i is the weight of the resource utilization rate of the i-th preset time period, and h is the number of preset time periods.

[0122] For example, if 'a' is 0.5, the resource utilization rate for i=1 (5 minutes) is 0.33142, with a corresponding weight of 0.2; the resource utilization rate for i=2 (1 hour) is 0.33495, with a corresponding weight of 0.3; and the resource utilization rate for i=3 (24 hours) is 0.3525, with a corresponding weight of 0.5. In this example, F = 1 + (0.4 - 0.33142 * 0.2 + 0.33495 * 0.3 + 0.33295 * 0.5) * 0.8 = 1.13.

[0123] The above embodiments, based on real historical load information and dynamic overselling of node resources, further divide tasks into multiple priorities. According to node load conditions, an extended scheduler dynamically schedules tasks of different priorities for load balancing, improving cluster resource utilization while achieving load balancing among nodes as much as possible. Simultaneously, due to the priority-based eviction policy, high-priority task pods are minimized from being evicted, thus significantly improving the operational stability of high-priority tasks.

[0124] like Figure 6 As shown, based on the same inventive concept as the above-described resource scheduling method, this application embodiment also provides a resource scheduling device, which is applied to a container cloud platform. The device includes an acquisition module 61, a scoring module 62, and a scheduling module 63.

[0125] The acquisition module 61 is used to acquire the first resource utilization of each node; wherein, the node is used to provide resources to the container group according to the scheduling request of the container group deployed on the node;

[0126] The scoring module 62 is used to apply a scoring mechanism that matches the priority of the target container group to each target node, and to score the target node based on the second resource utilization rate and the resource allocation rate of the target node; wherein, the target node is obtained by filtering each node according to the first utilization rate threshold, and the first resource utilization rate of the target node is less than or equal to the first utilization rate threshold.

[0127] The scheduling module 63 is used to schedule resources based on the priority of the target container group and the score of each target node corresponding to the priority.

[0128] In one possible design, the first resource utilization rate is the CPU utilization and memory utilization rate determined based on the container group deployed on the node during resource usage; the first utilization rate threshold includes a first CPU utilization threshold and a first memory utilization threshold.

[0129] In one possible design, a filtering module is also included for:

[0130] Delete nodes whose CPU utilization is greater than the first CPU utilization threshold.

[0131] Delete nodes whose memory utilization is greater than the first memory utilization threshold.

[0132] In one possible design, if the priority of the target container group is high, then the priority matching scoring mechanism for the target container group is the first scoring mechanism; if the priority of the target container group is medium or low, then the priority matching scoring mechanism for the target container group is the second scoring mechanism.

[0133] In the first scoring mechanism, the second resource utilization rate is negatively correlated with the score, and the resource allocation rate is also negatively correlated with the score. In the second scoring mechanism, the second resource utilization rate is negatively correlated with the score, and the resource allocation rate is positively correlated with the score. The second resource utilization rate is the utilization rate obtained by weighting CPU utilization rate and memory utilization rate.

[0134] In one possible design, the scoring module 62 is specifically used for:

[0135] The first scoring mechanism is applied, and the target node is scored based on the second resource utilization rate to obtain the first score; the target node is scored based on the resource allocation rate to obtain the second score; the first score and the second score are weighted to obtain the first target score of the target node when the priority of the target container group is high.

[0136] The second scoring mechanism is applied to score the target node based on its second resource utilization rate to obtain a third score; the target node is also scored based on its resource allocation rate to obtain a fourth score; the third and fourth scores are weighted to obtain the second target score of the target node when the priority of the target container group is medium or low.

[0137] In one possible design, the scheduling module 63 is specifically used for:

[0138] If there are N target nodes whose second resource utilization is less than or equal to the second utilization threshold and whose resource allocation rate is less than or equal to the allocation rate threshold, the target container group will be scheduled to the target node with the highest first target score among the N target nodes; where N is an integer greater than or equal to 1 and less than or equal to the total number of target nodes;

[0139] If the resource allocation rate of each target node is greater than the allocation rate threshold, the target container group will be scheduled to the target node with the second lowest resource utilization rate among all target nodes.

[0140] If the second resource utilization rate of each target node is greater than the second utilization rate threshold, the low-priority container group in the first reference target node is deleted, and the target container group is scheduled to the first reference target node; wherein, the first reference target node is the target node with the highest first target score among the target nodes with low-priority container groups.

[0141] In one possible design, the scheduling module 63 is specifically used for:

[0142] If there are M target nodes whose second resource utilization is less than or equal to the second utilization threshold and whose resource allocation rate is less than or equal to the allocation rate threshold, the target container group will be scheduled to the target node with the highest second target score among the M target nodes; where M is an integer greater than or equal to 1 and less than or equal to the total number of target nodes.

[0143] If the second resource utilization rate of each target node is greater than the second utilization rate threshold, the low-priority container group in the second reference target node is deleted, and the target container group is scheduled to the second reference target node; wherein, the second reference target node is the target node with the highest second target score among the target nodes with low-priority container groups.

[0144] In one possible design, a determination module is also included to determine the resource allocation rate in the following manner:

[0145] For each target node, the actual total amount of resources for the target node is determined based on the resource overselling coefficient of the target node;

[0146] The resource allocation rate is determined based on the actual total amount of resources and the resources already allocated.

[0147] In one possible design, the resource oversupply coefficient includes the CPU resource oversupply coefficient and the memory resource oversupply coefficient, and the actual total resource includes the actual total CPU resource and the actual total memory resource; the determining module is specifically used for:

[0148] The actual total CPU resources are determined based on the CPU resource over-sale factor and the initial total CPU resources of the target node; the actual total memory resources are determined based on the memory resource over-sale factor and the initial total memory resources of the target node.

[0149] In one possible design, the module is specifically used for:

[0150] The first resource allocation rate is determined based on the actual total CPU resources and the allocated CPU resources; the second resource allocation rate is determined based on the actual total memory resources and the allocated memory resources.

[0151] The resource allocation rate is obtained by weighting the first resource allocation rate and the second resource allocation rate.

[0152] In one possible design, the module is specifically used for:

[0153] For the first resource, the allocation rate of the first resource is determined based on the amount of first resource already allocated to the current target node and the initial total amount of first resource.

[0154] The resource overselling coefficient of the first resource is determined based on the allocation rate of the first resource, the resource utilization rate of the first resource, and the weight corresponding to the resource utilization rate.

[0155] The first resource is either CPU resources or memory resources.

[0156] In one possible design, the determination module is specifically used to determine the resource overselling coefficient using the following formula:

[0157]

[0158] Where F is the resource overselling coefficient, and a is the current resource allocation rate; Let represent resource utilization rate, where usage_i is the resource utilization rate of the i-th preset time period, weight_i is the weight of the resource utilization rate of the i-th preset time period, and h is the number of preset time periods.

[0159] The resource processing apparatus proposed in this application embodiment adopts the same inventive concept as the above-described resource processing method and can achieve the same beneficial effects, which will not be repeated here.

[0160] Based on the same inventive concept as the resource processing method described above, this application also provides a resource scheduling system. This resource scheduling system is deployed as software on a container cloud. The system includes a scheduler and at least one component. The scheduler processes data monitored by the at least one component to implement the steps of the resource scheduling method described above. The actual functions of the scheduler and component can be found in [reference needed]. Figure 1 That will not be elaborated here.

[0161] Based on the same inventive concept as the resource processing method described above, embodiments of this application also provide a resource scheduling device, which can be a cloud server, such as a container cloud. Figure 7 As shown, the resource scheduling device may include a processor 701 and a memory 702, and the resource scheduling system may be deployed on the processor 701.

[0162] The processor 701 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0163] Memory 702, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. Memory 702 in the embodiments of this application may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0164] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned computer storage medium can be any available medium or data storage device that a computer can access, including but not limited to: mobile storage devices, random access memory (RAM), magnetic storage (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs)) and other media capable of storing program code.

[0165] Alternatively, if the integrated units described above in this application are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this application. The aforementioned storage medium includes: mobile storage devices, random access memory (RAM), magnetic memory (e.g., floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical memory (e.g., CDs, DVDs, BDs, HVDs, etc.), and semiconductor memory (e.g., ROMs, EPROMs, EEPROMs, non-volatile memory (NAND flash), solid-state drives (SSDs), etc.) and other media capable of storing program code.

[0166] The above embodiments are only used to provide a detailed description of the technical solutions of this application. However, the description of the above embodiments is only for the purpose of helping to understand the methods of the embodiments of this application and should not be construed as a limitation on the embodiments of this application. Any changes or substitutions that can be easily conceived by those skilled in the art should be covered within the protection scope of the embodiments of this application.

Claims

1. A resource scheduling method, characterized in that, include: Obtain the first resource utilization rate of each node; wherein, the node is used to provide resources to the container group according to the scheduling request of the container group deployed on the node; For each target node, a scoring mechanism matching the priority of the target container group is applied, scoring the target node based on its second resource utilization rate and resource allocation rate. The target node is obtained by filtering each node according to a first utilization threshold, and the first resource utilization rate of the target node is less than or equal to the first utilization threshold. If the priority of the target container group is high, the scoring mechanism matching the priority of the target container group is the first scoring mechanism; if the priority of the target container group is medium or low, the scoring mechanism matching the priority of the target container group is the second scoring mechanism. In the first scoring mechanism, the second resource utilization rate and the resource allocation rate are negatively correlated with the score. In the second scoring mechanism, the second resource utilization rate and the resource allocation rate are positively correlated with the score. The second resource utilization rate is a weighted average of CPU utilization and memory utilization. Resource scheduling is achieved based on the priority of the target container group and the score of each target node corresponding to the priority.

2. The resource scheduling method according to claim 1, characterized in that, The first resource utilization rate is the CPU utilization and memory utilization rate determined based on the container group deployed on the node during resource usage; the first utilization rate threshold includes a first CPU utilization rate threshold and a first memory utilization rate threshold.

3. The resource scheduling method according to claim 2, characterized in that, The step of filtering each node according to the first utilization threshold includes: Delete nodes whose CPU utilization is greater than the first CPU utilization threshold. Delete nodes whose memory utilization rate is greater than the first memory utilization threshold.

4. The resource scheduling method according to claim 1, characterized in that, A scoring mechanism that matches the priority of the target container group is applied to score the target node based on its second resource utilization rate and resource allocation rate, including: Applying the first scoring mechanism, the target node is scored based on the second resource utilization rate to obtain a first score; the target node is scored based on the resource allocation rate to obtain a second score; the first score and the second score are weighted to obtain a first target score for the target node when the priority of the target container group is high. Applying the second scoring mechanism, the target node is scored based on the second resource utilization rate to obtain a third score; the target node is scored based on the resource allocation rate to obtain a fourth score; the third score and the fourth score are weighted to obtain a second target score for the target node when the priority of the target container group is medium or low.

5. The resource scheduling method according to claim 4, characterized in that, If the target container group has a high priority, resource scheduling is implemented based on the priority of the target container group and the scores of each target node corresponding to the priority, including: If there are N target nodes whose second resource utilization rate is less than or equal to the second utilization rate threshold and whose resource allocation rate is less than or equal to the allocation rate threshold, the target container group will be scheduled to the target node with the highest first target score among the N target nodes; where N is an integer greater than or equal to 1 and less than or equal to the total number of target nodes; If the resource allocation rate of each target node is greater than the allocation rate threshold, the target container group will be scheduled to the target node with the lowest resource utilization rate among the target nodes. If the second resource utilization rate of each target node is greater than the second utilization rate threshold, the low-priority container group in the first reference target node is deleted, and the target container group is scheduled to the first reference target node; wherein, the first reference target node is the target node with the highest first target score among the target nodes with low-priority container groups.

6. The resource scheduling method according to claim 4, characterized in that, If the priority of the target container group is medium or low, then the resource scheduling based on the priority of the target container group and the score of each target node includes: If there are M target nodes whose second resource utilization is less than or equal to the second utilization threshold and whose resource allocation rate is less than or equal to the allocation rate threshold, the target container group will be scheduled to the target node with the highest second target score among the M target nodes; where M is an integer greater than or equal to 1 and less than or equal to the total number of target nodes; When the second resource utilization rate of each of the target nodes is greater than the second utilization rate threshold, for the medium-priority target container group, the low-priority container group in the second reference target node is deleted, and the medium-priority target container group is scheduled to the second reference target node; the low-priority target container group is controlled to wait for scheduling, wherein the second reference target node is the target node with the highest second target score among the target nodes with low-priority container groups.

7. The resource scheduling method according to any one of claims 1 to 6, characterized in that, The resource allocation rate is determined as follows: For each target node, the actual total resources of the target node are determined based on the resource overselling coefficient of the target node; The resource allocation rate is determined based on the actual total amount of resources and the allocated resources.

8. The resource scheduling method according to claim 7, characterized in that, The resource overselling coefficient includes the CPU resource overselling coefficient and the memory resource overselling coefficient, and the actual total resource volume includes the actual total CPU resource volume and the actual total memory resource volume. The determination of the actual total resources of the target node based on the resource overselling coefficient of the target node includes: The actual total CPU resources are determined based on the CPU resource overselling coefficient and the initial total CPU resources of the target node; The actual total memory resources are determined based on the memory resource overselling coefficient and the initial total memory resources of the target node.

9. The resource scheduling method according to claim 8, characterized in that, Determining the resource allocation rate based on the actual total resources and the allocated resources includes: A first resource allocation rate is determined based on the actual total CPU resources and the allocated CPU resources; a second resource allocation rate is determined based on the actual total memory resources and the allocated memory resources. The resource allocation rate is obtained by weighting the first resource allocation rate and the second resource allocation rate.

10. A resource scheduling device, characterized in that, include: An acquisition module is used to acquire the first resource utilization rate of each node; wherein, the node is used to provide resources to the container group according to the scheduling request of the container group deployed on the node; A scoring module is used to apply a scoring mechanism matching the priority of a target container group to each target node, scoring the target node based on its second resource utilization rate and resource allocation rate. The target nodes are selected by filtering each node according to a first utilization threshold, and the first resource utilization rate of the target node is less than or equal to the first utilization threshold. If the priority of the target container group is high, the scoring mechanism matching the priority of the target container group is the first scoring mechanism; if the priority of the target container group is medium or low, the scoring mechanism matching the priority of the target container group is the second scoring mechanism. In the first scoring mechanism, the second resource utilization rate and the resource allocation rate are negatively correlated with the score. In the second scoring mechanism, the second resource utilization rate and the resource allocation rate are positively correlated with the score. The second resource utilization rate is a weighted average of CPU utilization rate and memory utilization rate. The scheduling module is used to perform resource scheduling based on the priority of the target container group and the score of each target node corresponding to the priority.

11. A resource scheduling system, characterized in that, The resource scheduling system includes a scheduler and at least one component, wherein the scheduler is used to process data monitored by the at least one component to implement the steps of the method according to any one of claims 1 to 9.

Images