Resource scheduling method, device, electronic device and computer-readable storage medium

Abstract

The present disclosure relates to a resource scheduling method, device, electronic device and computer-readable storage medium. The resource scheduling method includes: obtaining the historical resource utilization rate and the initial configuration resource amount of the target node of the container cluster; determining the resource overselling coefficient of the target node according to the historical resource utilization rate; wherein the resource overselling coefficient is a multiple of the resource amount after the initial configuration resource amount is oversold, and when the historical resource utilization rate is less than the preset resource utilization rate, the resource overselling coefficient is negatively correlated with the historical resource utilization rate; and determining the schedulable resource amount of the target node according to the resource overselling coefficient and the initial configuration resource amount.

Description

Technical Field

[0001] The present disclosure relates to the technical field of container clusters, and in particular to a resource scheduling method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Kubernetes is a new leading distributed solution based on container technology, referred to as K8S. It is an open source container cluster management system from Google. Its design was inspired by a container management system called Borg within Google, and it inherits Google's more than ten years of experience in using container clusters. Kubernetes provides a complete range of functions for containerized applications, such as deployment and operation, resource scheduling, service discovery, and dynamic scaling, which greatly improves the convenience of large-scale container cluster management.

[0003] In terms of cluster management, Kubernetes divides the machines in the cluster into a Master node and a group of worker nodes. Each Node can have multiple Pods, which represent a group of one or more application containers (such as Docker or rkt) and some shared resources of these containers. The Node can aggregate the resource requests of each container of the Pod on it and communicate with the Master. The Master runs a group of processes related to cluster management. These processes implement management capabilities such as resource management and Pod scheduling of the entire cluster, and can allocate resources such as CPU and memory to each container to realize the operation of the container.

[0004] However, Kubernetes uses static scheduling, which refers to container scheduling based on the resources requested by the container. The biggest advantage of static scheduling is that it is simple and efficient, and cluster resource management is convenient. However, in the actual environment, the business is subjective and blind when choosing container specifications. In order to ensure business stability, the business will apply for resources far greater than its actual resource usage, resulting in low resource utilization of the business container. If such a large proportion of business occurs, it is easy to lead to low cluster resource utilization. Summary of the invention

[0005] The present disclosure provides a resource scheduling method, device, electronic device and computer-readable storage medium to at least solve the problem of low cluster resource utilization in the related art, and may not solve any of the above problems.

[0006] According to a first aspect of the present disclosure, a resource scheduling method is provided, the resource scheduling method comprising: obtaining a historical resource utilization rate and an initially configured resource amount of a target node of a container cluster; determining a resource overselling coefficient of the target node according to the historical resource utilization rate; wherein the resource overselling coefficient is a multiple of the resource amount after overselling based on the initially configured resource amount, and when the historical resource utilization rate is less than a preset resource utilization rate, the resource overselling coefficient is negatively correlated with the historical resource utilization rate; determining a schedulable resource amount of the target node according to the resource overselling coefficient and the initially configured resource amount.

[0007] Optionally, the step of determining the resource oversold coefficient of the target node based on the historical resource utilization includes: if the historical resource utilization is less than the preset resource utilization, obtaining the oversold constant of the target node, and exponentially processing the historical resource utilization to obtain an indexed value, and using the difference between the oversold constant and the indexed value as the resource oversold coefficient; wherein the oversold constant is the sum of a predetermined maximum value of the resource oversold coefficient and a minimum value of the indexed value; if the historical resource utilization is greater than or equal to the preset resource utilization, determining the resource oversold coefficient to be 1.

[0008] Optionally, the step of exponentially processing the historical resource utilization to obtain an indexed value includes: determining the product of the historical resource utilization and a first parameter; determining the indexed value using a second parameter as a base and the product as an exponent; wherein the first parameter is a positive value and the second parameter is greater than 1, or the first parameter is a negative value and the second parameter is less than 1.

[0009] Optionally, after the step of determining the schedulable resource amount of the target node based on the resource oversold coefficient and the initial configured resource amount, the resource scheduling method further includes: obtaining the deployed resource amount of the container set corresponding to the target node; if it is determined that the deployed resource amount is greater than or equal to the schedulable resource amount, re-determining the resource oversold coefficient based on the deployed resource amount and the initial configured resource amount, and returning to the step of determining the schedulable resource amount of the target node based on the resource oversold coefficient and the initial configured resource amount.

[0010] Optionally, the step of re-determining the resource overselling coefficient based on the deployed resource amount and the initially configured resource amount includes: determining a ratio of the deployed resource amount to the initially configured resource amount as the updated resource overselling coefficient.

[0011] Optionally, before the step of obtaining the historical resource utilization and the initial configuration resource amount of the target node of the container cluster, the resource scheduling method further includes: in response to intercepting node information, determining the node corresponding to the node information as the target node; after the step of determining the schedulable resource amount of the target node based on the resource oversold coefficient and the initial configuration resource amount, the resource scheduling method further includes: updating the schedulable resource amount in the node information, and reporting the updated node information.

[0012] According to a second aspect of the present disclosure, a resource scheduling device is provided, comprising: an acquisition unit, configured to execute: acquiring a historical resource utilization rate and an initially configured resource amount of a target node of a container cluster; a coefficient determination unit, configured to execute: determining a resource overselling coefficient of the target node according to the historical resource utilization rate; wherein the resource overselling coefficient is a multiple of the resource amount after overselling based on the initially configured resource amount, and when the historical resource utilization rate is less than a preset resource utilization rate, the resource overselling coefficient is negatively correlated with the historical resource utilization rate; and a scheduling determination unit, configured to execute: determining a schedulable resource amount of the target node according to the resource overselling coefficient and the initially configured resource amount.

[0013] Optionally, the coefficient determination unit is further configured to execute: if the historical resource utilization is less than the preset resource utilization, obtain the oversold constant of the target node, and index the historical resource utilization to obtain an indexed value, and use the difference between the oversold constant and the indexed value as the resource oversold coefficient; wherein the oversold constant is the sum of a predetermined maximum value of the resource oversold coefficient and a minimum value of the indexed value; if the historical resource utilization is greater than or equal to the preset resource utilization, determine that the resource oversold coefficient is 1.

[0014] Optionally, the coefficient determination unit is further configured to perform: determining the product of the historical resource utilization and a first parameter; determining the exponential value using a second parameter as a base and the product as an exponent; wherein the first parameter is a positive value and the second parameter is greater than 1, or the first parameter is a negative value and the second parameter is less than 1.

[0015] Optionally, the acquisition unit is further configured to execute: obtain the deployed resource amount of the container set corresponding to the target node; the coefficient determination unit is further configured to execute: if it is determined that the deployed resource amount is greater than or equal to the schedulable resource amount, redetermine the resource oversold coefficient based on the deployed resource amount and the initial configuration resource amount, and run the scheduling determination unit.

[0016] Optionally, the coefficient determination unit is further configured to perform: determining a ratio of the deployed resource amount to the initially configured resource amount as the updated resource overselling coefficient.

[0017] Optionally, the resource scheduling device also includes: an interception unit, configured to execute: in response to intercepting node information, determining the node corresponding to the node information as the target node; an update unit, configured to execute: updating the amount of schedulable resources in the node information and reporting the updated node information.

[0018] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein the computer executable instructions, when executed by the at least one processor, prompt the at least one processor to execute the resource scheduling method according to the present disclosure.

[0019] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor of an electronic device, the electronic device is prompted to execute the resource scheduling method according to the present disclosure.

[0020] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising computer instructions, which implement the resource scheduling method according to the present disclosure when executed by at least one processor.

[0021] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0022] According to the resource scheduling method and resource scheduling device of the embodiments of the present invention, by determining the resource overselling coefficient and the corresponding schedulable resource amount according to the historical resource utilization of the target node, the actual load of the target node can be considered. When the historical resource utilization is low and the target node's previous requested resource amount is artificially high, the target node is appropriately allocated a resource amount that exceeds the actual supply capacity, so that the target node can provide more virtual resources for Pods and containers, which helps to deploy more Pods and containers, thereby improving the resource utilization of the target node and making the resources of the target node be utilized as much as possible; when the historical resource utilization is high and the target node's previous requested resource amount is relatively reasonable, resource overselling is appropriately reduced to prevent excessive resource overselling, which helps to ensure the stability of cluster services while overselling resources.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0025] Figure 1 is a diagram showing the relationship between the resource request amount and the actual resource usage amount in the related art.

[0026] Figure 2 is a flow chart illustrating a resource scheduling method according to an exemplary embodiment of the present disclosure.

[0027] Figure 3 Detailed description of the invention is given in detail.

[0028] Figure 4 FIG. 4 is a schematic diagram showing the amount of schedulable resources according to an exemplary embodiment of the present disclosure.

[0029] Figure 5 is a curve showing a change trend of a CPU overselling coefficient and a memory overselling coefficient according to an exemplary embodiment of the present disclosure.

[0030] Figure 6 is a block diagram illustrating a resource scheduling apparatus according to an exemplary embodiment of the present disclosure.

[0031] Figure 7 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0034] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0035] Containerization technology refers to the technology that effectively divides the resources of a single operating system into isolated groups in order to better balance conflicting resource usage requirements between isolated groups. Unlike virtualization technology, which integrates network resources such as servers and storage devices into resource pools and then divides them, in containerization technology, containerized applications run directly on physical machines, so that physical machines can give full play to their performance and improve hardware utilization efficiency. Also thanks to containerization technology, even if the physical machine goes down, it can be restored through previous backups to avoid irreparable consequences.

[0036] Containerization is the mainstream trend today. There are enough open source tools for reference. Kubernetes is one of them. Kubernetes is a container orchestration tool based on container technology. It provides powerful enough functions to facilitate our development and operation and self-development according to the needs of the scene. This reduces the difficulty of development and operation and saves operation costs.

[0037] Kubernetes divides the machines in the cluster into a Master node and a group of worker nodes. Each Node can have multiple Pods. A Pod represents a group of one or more application containers (such as Docker or rkt) and some shared resources of these containers. The Node can aggregate the resource requests of each container of the Pod on it and communicate with the Master. The Master runs a group of processes related to cluster management. These processes implement management capabilities such as resource management and Pod scheduling of the entire cluster, and can allocate resources such as CPU and memory to each container to realize the operation of the container.

[0038] However, Kubernetes uses static scheduling, which refers to packing scheduling based on the resources requested by the container without considering the actual load of the node. The biggest advantage of static scheduling is that it is simple and efficient, and convenient for cluster resource management. The biggest disadvantage is also obvious, that is, regardless of the actual load of the node, it is very easy to cause the cluster load to be low. In the actual environment, the business has a certain degree of subjectivity and blindness when choosing container specifications, such as Figure 1As shown in the figure, in order to ensure business stability, the business will request resources that are far greater than its actual resource usage (requests>>reality), resulting in low resource utilization of the business container. If such business accounts for a large proportion, it is easy to cause low cluster resource utilization. Figure 1 As shown in the figure, the amount of resources requested by the business is too large, even close to the upper limit of the resources that can be scheduled (limits>requests), which will cause the cluster to be unable to accommodate more business containers according to the Kubernetes static scheduling strategy, further restricting the cluster resource utilization.

[0039] Therefore, the present disclosure provides a resource scheduling method, device, electronic device and computer-readable storage medium. By determining the resource overselling coefficient and the corresponding schedulable resource amount according to the historical resource utilization of the target node, the actual load of the target node can be considered. When the historical resource utilization is low and the target node's previous requested resource amount is artificially high, the target node is appropriately allocated a resource amount that exceeds the actual supply capacity, so that the target node can provide more virtual resources for Pods and containers, which helps to deploy more Pods and containers, thereby improving the resource utilization of the target node and making the resources of the target node be utilized as much as possible; when the historical resource utilization is high and the target node's previous requested resource amount is relatively reasonable, resource overselling is appropriately reduced to prevent excessive resource overselling, which helps to ensure the stability of cluster services while overselling resources.

[0040] Next, we will refer to Figures 2 to 7 A resource scheduling method and a resource scheduling device according to exemplary embodiments of the present disclosure are described in detail.

[0041] Figure 2 is a flow chart showing a resource scheduling method according to an exemplary embodiment of the present disclosure. It should be understood that the resource scheduling method according to the exemplary embodiment of the present disclosure can be implemented in a terminal device such as a smart phone, a tablet computer, a personal computer (PC), or in a device such as a server.

[0042] Reference Figure 2 In step 201, the historical resource utilization and the initial configured resource amount of the target node of the container cluster are obtained.

[0043] The target node is the Node node whose schedulable resources currently need to be adjusted. The schedulable resources refer to the resources available for Pod scheduling. In the static scheduling of related technologies, the schedulable resources are the initial configuration resources of the target node, that is, the schedulable resources before the resources are oversold, which is determined by the resources of the physical machine allocated to the target node. The historical resource utilization reflects the utilization rate of the scheduled resources by each Pod (ultimately, each container) corresponding to the target node. It can be calculated by calculating the ratio of actual resource usage to the scheduled resources (refer to Figure 1 , which is reality / requests). It should be noted that in order to achieve dynamic adjustment of the amount of schedulable resources, the average value of resource utilization in the past period of time (that is, the period corresponding to the past set time) can be obtained at set intervals as the historical resource utilization of the past period of time, that is, the historical resource utilization is dynamically updated, so the amount of schedulable resources can be continuously updated in subsequent steps. As an example, the maximum and minimum values ​​of the resource utilization of the target node in the past 5 minutes can be obtained every 5 minutes, and the average value can be calculated as the historical resource utilization of the past 5 minutes.

[0044] Optionally, for the target node, before step 201, the resource scheduling method according to the exemplary embodiment of the present disclosure may further include: in response to intercepting the node information, determining the node corresponding to the node information as the target node. The node information may specifically be information reflecting the requested resource amount of the node, or more precisely, the requested resource amount of the pod and container corresponding to the node. By intercepting the node information uploaded by the node, it is possible to intercept in time when the node issues a resource request, thereby overselling resources to the corresponding node in a targeted manner. Accordingly, after determining the schedulable resource amount according to the resource overselling in the subsequent steps, the resource scheduling method according to the exemplary embodiment of the present disclosure may further include: updating the schedulable resource amount in the node information, and reporting the updated node information. By reporting the updated node information, the integrity of the data flow can be guaranteed. At the same time, by directly updating the schedulable resource amount in the node information, that is, updating the schedulable resource amount from the initial configuration resource amount to the schedulable resource amount determined in the subsequent steps, the dynamic resource scheduling can be completed with the help of the existing node information, thereby reducing the improvement cost. It should be understood that the initial configuration resource amount can also be separately saved in the node information for continuous updating of the schedulable resource amount.

[0045] As an example, in Kubernetes, the Master node runs a group of processes related to cluster management, including Kube-ApiServer and Kube-Controller-Manager. The former can provide the addition, deletion, modification and query of various resource objects, and the latter can realize the monitoring of cluster status. The resource scheduling method according to the exemplary embodiment of the present disclosure can be implemented based on Kube-Controller-Manager. In addition, refer to Figure 3 , the Node node runs kubelet, which is responsible for the communication between the Master node and the Node node, and can manage Pods and containers. The Node node's kubelet will report the node information to the ApiServer. According to the exemplary embodiment of the present disclosure, this node information can be intercepted and the following operations are performed: the schedulable resource amount of the node before overselling (i.e., the initial configuration resource amount) is written into the annotation field of the node for easy comparison; thereafter, the historical resource utilization is obtained at set intervals to execute the subsequent step 202, and the obtained resource overselling coefficient is written into the annotation field of the node; the obtained historical resource utilization is written into the annotation field of the node for easy observation and coordination of scheduling.

[0046] In step 202, the resource overselling coefficient of the target node is determined based on the historical resource utilization rate. The resource overselling coefficient is a multiple of the resource amount after the initial configuration resource amount is oversold. When the historical resource utilization rate is less than the preset resource utilization rate, the resource overselling coefficient is negatively correlated with the historical resource utilization rate. Figure 3 and Figure 4, the resource oversell coefficient is Resource OversellRate, then the initial configuration resource amount Node Resource is expanded by a corresponding multiple as the schedulable resource amount Node ResourceAllocatable. The historical resource utilization rate can reflect the actual resource utilization of the target node in the past (for example, in the past 5 minutes mentioned above). By making the resource oversell coefficient negatively correlated with the historical resource utilization rate when the historical resource utilization rate is less than the preset resource utilization rate, the actual load of the target node can be considered. When the historical resource utilization rate is low and the target node's previous requested resource amount is artificially high, the target node is appropriately allocated with schedulable resources that exceed the actual supply capacity, so that the target node can provide more virtual resources for Pods and containers, which helps to deploy more Pods and containers, thereby improving the resource utilization rate of the target node and making the resources of the target node as much as possible. When the historical resource utilization rate is high and the target node's previous requested resource amount is relatively reasonable, appropriately reducing resource overselling, that is, reducing the amount of schedulable resources, can achieve reasonable resource overselling, prevent excessive resource overselling, and help to ensure the stability of cluster services while overselling resources.

[0047] Optionally, the preset resource utilization is less than 1, while the historical resource utilization varies between 0 and 1, which indicates that the resource overselling coefficient is negatively correlated with the historical resource utilization only when the historical resource utilization is relatively small. That is to say, when the historical resource utilization is still low, the higher the historical resource utilization, the lower the resource overselling coefficient. After the historical resource utilization reaches the preset resource utilization, considering that the requested resource amount of the target node is relatively reasonable, the resource overselling coefficient can be maintained at 1, and resources are no longer oversold to ensure system stability. It should be noted that the preset resource utilization can be determined based on the actual operating conditions, for example, it can be 55%, and the present disclosure does not limit this.

[0048] Specifically, in the case where the historical resource utilization is less than the preset resource utilization, step 202 can obtain the oversold constant of the target node, and perform indexation processing on the historical resource utilization to obtain the indexed value, and use the difference between the oversold constant and the indexed value as the resource oversold coefficient. In other words, the resource oversold coefficient is the difference between the oversold constant and the indexed value. Among them, the oversold constant is the sum of the maximum value of the predetermined resource oversold coefficient and the minimum value of the indexed value, and the predetermined maximum value of the resource oversold coefficient is determined after analyzing the resource utilization of the current cluster, and the present disclosure does not limit the specific analysis method. Due to the limitation in the previous article that when the historical resource utilization rate is less than the preset resource utilization rate, the resource overselling coefficient is negatively correlated with the historical resource utilization rate, so the indexed value here is positively correlated with the historical resource utilization rate, that is, the indexed value gradually increases with the increase of the historical resource utilization rate, and the indexed value itself has the performance of gradually accelerating the growth rate, so that when the value of the resource overselling coefficient gradually decreases with the increase of the historical resource utilization rate, its decreasing speed gradually accelerates, and the resource overselling coefficient can be quickly reduced after the resource utilization rate increases, thereby preventing excessive overselling of resources and helping to ensure the stability of cluster services.

[0049] Optionally, the step of performing indexation processing on the historical resource utilization to obtain the indexed value may include: determining the product of the historical resource utilization and the first parameter; and determining the indexed value with the second parameter as the base and the product as the exponent. That is, y=b ax , z = cb ax , where x is the historical resource utilization rate, y is the indexed value, z is the resource oversold coefficient, a is the first parameter, b is the second parameter, and c is the oversold constant. a In essence, it constitutes the base of the exponential value, and correspondingly, x constitutes the exponent of the exponential value. Wherein, the first parameter a is a positive value, and the second parameter b is greater than 1 (for example, the natural constant e), or the first parameter a is a negative value, and the second parameter b is less than 1 (for example, the reciprocal of the natural constant e 1 / e), which can fully guarantee b a is greater than 1, so that the indexed value y increases with the increase of the historical resource utilization rate x. In addition, by splitting the base of the indexed value into the first parameter and the second parameter, the base can be unified by selecting a unified second parameter b for all resource scheduling scenarios, and then a reasonable and substantial base b can be configured for the indexed value by determining an appropriate first parameter a. a On the one hand, compared with directly taking values ​​or using scientific notation, b aThe form of can retain more numerical details when the first parameter a takes the same number of digits after the decimal point, which helps to reduce the storage burden of the resource oversold coefficient calculation formula while ensuring the calculation accuracy. On the other hand, it can reduce the differences in the forms of different calculation formulas. Specifically for the determination of each parameter, theoretically, when the historical resource utilization rate x is 0, the resource oversold coefficient z can be maximized. When the historical resource utilization rate x is 0, the indexed value y is stable at 1. The reasonable maximum value of the resource oversold coefficient can be increased by 1 as the oversold constant c, which facilitates the determination of the oversold constant c. After determining the oversold constant c, as mentioned above, after the historical resource utilization rate x reaches the preset resource utilization rate (for example, 55%), the resource oversold coefficient z can be maintained at 1. Based on this, the oversold constant c, x=preset resource utilization rate, and z=1 can be substituted into the formula z=cb ax , thereby determining the actual base b a The value of b determines the speed of change of the exponential value. a , after selecting the second parameter b, a Find the logarithm to get the first parameter a. Specifically, in b a Under certain circumstances, the larger the value of the second parameter b, the smaller the value of b when the first parameter a increases or decreases by the same magnitude. a The larger the increase or decrease, the more numerical details will be retained, but if the value of b is too large, a will need to have enough digits after the decimal point to reflect b. a The change of will also increase the storage burden, and too much is as bad as too little. Therefore, a reasonable value of the second parameter b also helps to retain more numerical details. As an example, the second parameter b is the natural constant e.

[0050] Specifically, the resources to be scheduled may include CPU and memory. Accordingly, in step 201, the historical CPU utilization CPU Utilization, the initially configured CPU resource amount Node CPU, the historical memory utilization MemoryUtilization, and the initially configured memory resource amount Node Memory of the target node may be obtained, and the above formula z=cb may be referred to. ax In step 202, a CPU oversell rate CPU OversellRate is determined according to the historical CPU utilization CPU Utilization, and a memory oversell rate Memory OversellRate is determined according to the historical memory utilization Memory Utilization, for example, using the following formula:

[0051]

[0052]

[0053] The trend curves of CPU oversold coefficient and memory oversold coefficient drawn by the above formula are as follows: Figure 5 shown.

[0054] In step 203, the schedulable resource amount of the target node is determined according to the resource overselling coefficient and the initial configured resource amount.

[0055] Specifically, refer to Figure 3 and Figure 4 As discussed above, by calculating the product of the resource oversold coefficient and the initial configured resource quantity, the schedulable resource quantity of the target node can be obtained. Figure 5 For example, the following formula can be used to calculate the schedulable CPU resource Node CPUAllocatable and the schedulable memory resource Node MemoryAllocatable:

[0056] NodeCPUAllocatable=NodeCPU×CPUOversellrate

[0057] NodeMemoryAllocatable=NodeMemory×MemoryOversellrate

[0058] Further, after step 203, the resource scheduling method according to the exemplary embodiment of the present disclosure also includes: obtaining the amount of deployed resources of the container set corresponding to the target node, if it is determined that the amount of deployed resources is greater than or equal to the amount of schedulable resources, redetermining the resource overselling coefficient based on the amount of deployed resources and the amount of initially configured resources, and returning to step 203. When the resource utilization of the system suddenly increases significantly, the resource overselling coefficient determined in step 202 will show a downward trend. At this time, the amount of deployed resources of the container set deployed in the target node (specifically, the sum of the amount of deployed resources of the deployed Pods) may be greater than or equal to the current amount of schedulable resources of the target node. At this time, by redetermining the resource overselling coefficient in combination with the actual amount of deployed resources and the amount of initially configured resources, the newly determined amount of schedulable resources can meet the operation of the currently deployed containers and ensure service stability.

[0059] Specifically, based on the amount of deployed resources and the amount of initially configured resources, the step of re-determining the resource overselling coefficient includes: determining the ratio of the amount of deployed resources to the amount of initially configured resources as the updated resource overselling coefficient, so that the corresponding updated amount of schedulable resources is exactly equal to the amount of deployed resources of the deployed container set, and there are no extra resources to be allocated to other Pods and containers, and no new Pods and containers can be deployed, which can ensure that the deployed Pods and containers are not evicted, thereby reliably ensuring service stability. Still taking CPU and memory as an example, the updated CPU overselling coefficient and memory overselling coefficient are:

[0060]

[0061]

[0062] Among them, ∑PodCPURequest is the sum of the deployed CPU resources of each deployed Pod, and ∑PodMemoryRequest is the sum of the deployed memory resources of each deployed Pod.

[0063] Figure 6 is a block diagram showing a resource scheduling device according to an exemplary embodiment of the present disclosure. It should be understood that the resource scheduling device according to the exemplary embodiment of the present disclosure can be implemented in a terminal device such as a smart phone, a tablet computer, a personal computer (PC) in the form of software, hardware, or a combination of software and hardware, and can also be implemented in a device such as a server.

[0064] Reference Figure 6 The resource scheduling device 600 includes an acquisition unit 601, a coefficient determination unit 602, and a scheduling determination unit 603.

[0065] The acquisition unit 601 acquires the historical resource utilization and the initial configured resource amount of the target node of the container cluster.

[0066] The target node is the Node node whose schedulable resources currently need to be adjusted. The schedulable resources refer to the resources available for Pod scheduling. In the static scheduling of related technologies, the schedulable resources are the initial configuration resources of the target node, that is, the schedulable resources before the resources are oversold, which is determined by the resources of the physical machine allocated to the target node. The historical resource utilization reflects the utilization rate of the scheduled resources by each Pod (ultimately, each container) corresponding to the target node. It can be calculated by calculating the ratio of actual resource usage to the scheduled resources (refer to Figure 1, which is reality / requests). It should be noted that in order to achieve dynamic adjustment of the amount of schedulable resources, the average value of resource utilization in the past period of time (that is, the period corresponding to the past set time) can be obtained at set intervals as the historical resource utilization of the past period of time, that is, the historical resource utilization is dynamically updated, so the amount of schedulable resources can be continuously updated in subsequent operations. As an example, the maximum and minimum values ​​of the resource utilization of the target node in the past 5 minutes can be obtained every 5 minutes, and the average value can be calculated as the historical resource utilization of the past 5 minutes.

[0067] Optionally, the resource scheduling device 600 according to the exemplary embodiment of the present disclosure may further include an interception unit (not shown in the figure). Before the acquisition unit 601 runs, the interception unit may determine the node corresponding to the node information as the target node in response to intercepting the node information. The node information may specifically be information reflecting the requested resource amount of the node, or more precisely, the requested resource amount of the pod and container corresponding to the node. By intercepting the node information uploaded by the node, it is possible to intercept in time when the node issues a resource request, thereby overselling resources to the corresponding node in a targeted manner. Accordingly, the resource scheduling device 600 according to the exemplary embodiment of the present disclosure may further include an update unit (not shown in the figure). The update unit may update the schedulable resource amount in the node information after the coefficient determination unit 602 and the scheduling determination unit 603 determine the schedulable resource amount, and report the updated node information. By reporting the updated node information, the integrity of the data flow can be guaranteed. At the same time, by directly updating the schedulable resource amount in the node information, that is, updating the schedulable resource amount from the initial configuration resource amount to the schedulable resource amount determined in the subsequent steps, the dynamic resource scheduling can be completed with the help of the existing node information, thereby reducing the improvement cost. It should be understood that the initial configured resource amount may also be separately saved in the node information for continuous updating of the schedulable resource amount.

[0068] As an example, in Kubernetes, the Master node runs a group of processes related to cluster management, including Kube-ApiServer and Kube-Controller-Manager. The former can provide the addition, deletion, modification and query of various resource objects, and the latter can realize the monitoring of cluster status. According to the exemplary embodiment of the present disclosure, the resource scheduling device 600 can be implemented based on Kube-Controller-Manager. In addition, referring to Figure 3, the Node node runs kubelet, which is responsible for the communication between the Master node and the Node node, and can manage Pods and containers. The Node node's kubelet will report the node information to the ApiServer. According to the exemplary embodiment of the present disclosure, this node information can be intercepted and the following operations are performed: the schedulable resource amount of the node before overselling (i.e., the initial configuration resource amount) is written into the annotation field of the node for easy comparison; thereafter, the historical resource utilization is obtained at set intervals to run the coefficient determination unit 602, and the obtained resource overselling coefficient is written into the annotation field of the node; the obtained historical resource utilization is written into the annotation field of the node for easy observation and coordination with scheduling.

[0069] The coefficient determination unit 602 can determine the resource overselling coefficient of the target node according to the historical resource utilization rate. The resource overselling coefficient is a multiple of the resource amount after the initial configuration resource amount is oversold. When the historical resource utilization rate is less than the preset resource utilization rate, the resource overselling coefficient is negatively correlated with the historical resource utilization rate. Figure 3 and Figure 4 , the resource oversell coefficient can be the multiple of the over-allocated resource amount Resource OversellRate, then the initial configuration resource amount Node Resource is expanded by the corresponding multiple as the schedulable resource amount Node ResourceAllocatable. The historical resource utilization rate can reflect the actual resource utilization of the target node in the past (for example, in the past 5 minutes mentioned above). By making the resource oversell coefficient negatively correlated with the historical resource utilization rate when the historical resource utilization rate is less than the preset resource utilization rate, the actual load of the target node can be considered. When the historical resource utilization rate is low and the target node's previous requested resource amount is artificially high, the target node is appropriately allocated with schedulable resources that exceed the actual supply capacity, so that the target node can provide more virtual resources for Pods and containers, which helps to deploy more Pods and containers, thereby improving the resource utilization of the target node and making the resources of the target node as much as possible. When the historical resource utilization rate is high and the target node's previous requested resource amount is relatively reasonable, appropriately reducing resource oversell, that is, reducing the schedulable resource amount, can achieve reasonable resource oversell, prevent excessive resource oversell, and help to ensure the stability of cluster services while overselling resources.

[0070] Optionally, the preset resource utilization is less than 1, while the historical resource utilization varies between 0 and 1, which indicates that the resource overselling coefficient is negatively correlated with the historical resource utilization only when the historical resource utilization is relatively small. That is to say, when the historical resource utilization is still low, the higher the historical resource utilization, the lower the resource overselling coefficient. After the historical resource utilization reaches the preset resource utilization, considering that the requested resource amount of the target node is relatively reasonable, the resource overselling coefficient can be maintained at 1, and resources are no longer oversold to ensure system stability. It should be noted that the preset resource utilization can be determined based on the actual operating conditions, for example, it can be 55%, and the present disclosure does not limit this.

[0071] Specifically, in the case where the historical resource utilization is less than the preset resource utilization, the coefficient determination unit 602 can obtain the oversold constant of the target node, and perform indexation processing on the historical resource utilization to obtain the indexed value, and use the difference between the oversold constant and the indexed value as the resource oversold coefficient. In other words, the resource oversold coefficient is the difference between the oversold constant and the indexed value. Among them, the oversold constant is the sum of the maximum value of the predetermined resource oversold coefficient and the minimum value of the indexed value, and the predetermined maximum value of the resource oversold coefficient is determined after analyzing the resource utilization of the current cluster, and the present disclosure does not limit the specific analysis method. Limited by the setting range of historical resource utilization specified in the previous article, the resource overselling coefficient is negatively correlated with the historical resource utilization, which makes the indexed value here positively correlated with the historical resource utilization, that is, the indexed value gradually increases with the increase of historical resource utilization, and the indexed value itself has the performance of gradually accelerating growth, so that when the value of the resource overselling coefficient gradually decreases with the increase of historical resource utilization, its decreasing speed gradually accelerates, and the resource overselling coefficient can be quickly reduced after the resource utilization increases, thereby preventing excessive overselling of resources and helping to ensure the stability of cluster services.

[0072] Optionally, the coefficient determination unit 602 performs indexation processing on the historical resource utilization to obtain the indexed value, which may include: determining the product of the historical resource utilization and the first parameter; and determining the indexed value with the second parameter as the base and the product as the exponent. That is, satisfying y=b ax , z = cb ax , where x is the historical resource utilization rate, y is the indexed value, z is the resource oversold coefficient, a is the first parameter, b is the second parameter, and c is the oversold constant. a In essence, it constitutes the base of the exponential value, and correspondingly, x constitutes the exponent of the exponential value. Wherein, the first parameter a is a positive value, and the second parameter b is greater than 1 (for example, the natural constant e), or the first parameter a is a negative value, and the second parameter b is less than 1 (for example, the reciprocal of the natural constant e 1 / e), which can fully guarantee b ais greater than 1, so that the indexed value y increases with the increase of the historical resource utilization rate x. In addition, by splitting the base of the indexed value into the first parameter and the second parameter, the base can be unified by selecting a unified second parameter b for all resource scheduling scenarios, and then a reasonable and substantial base b can be configured for the indexed value by determining an appropriate first parameter a. a On the one hand, compared with directly taking values ​​or using scientific notation, b a The form of can retain more numerical details when the first parameter a takes the same number of digits after the decimal point, which helps to reduce the storage burden of the resource oversold coefficient calculation formula while ensuring the calculation accuracy. On the other hand, it can reduce the differences in the forms of different calculation formulas. Specifically for the determination of each parameter, theoretically, when the historical resource utilization rate x is 0, the resource oversold coefficient z can be maximized. When the historical resource utilization rate x is 0, the indexed value y is stable at 1. The reasonable maximum value of the resource oversold coefficient can be increased by 1 as the oversold constant c, which facilitates the determination of the oversold constant c. After determining the oversold constant c, as mentioned above, after the historical resource utilization rate x reaches the preset resource utilization rate (for example, 55%), the resource oversold coefficient z can be maintained at 1. Based on this, the oversold constant c, x=preset resource utilization rate, and z=1 can be substituted into the formula z=cb ax , thereby determining the actual base b a The value of b determines the speed of change of the exponential value. a , after selecting the second parameter b, a Find the logarithm to get the first parameter a. Specifically, in b a Under certain circumstances, the larger the value of the second parameter b, the smaller the value of b when the first parameter a increases or decreases by the same magnitude. a The larger the increase or decrease, the more numerical details will be retained, but if the value of b is too large, a will need to have enough digits after the decimal point to reflect b. a The change of will also increase the storage burden, and too much is as bad as too little. Therefore, a reasonable value of the second parameter b also helps to retain more numerical details. As an example, the second parameter b is the natural constant e.

[0073] The scheduling determination unit 603 may determine the schedulable resource amount of the target node according to the resource overselling coefficient and the initial configured resource amount. Figure 3 and Figure 4 As discussed above, by calculating the product of the resource overselling coefficient and the initial configured resource quantity, the schedulable resource quantity of the target node can be obtained.

[0074] Further, after the scheduling determination unit 603 determines the amount of schedulable resources, the acquisition unit 601 can also obtain the amount of deployed resources of the container set corresponding to the target node, and the coefficient determination unit 602 can also execute: if it is determined that the amount of deployed resources is greater than or equal to the amount of schedulable resources, the resource overselling coefficient is re-determined according to the amount of deployed resources and the amount of initially configured resources, and the scheduling determination unit 603 is run. When the resource utilization rate of the system suddenly increases significantly, the resource overselling coefficient determined by the coefficient determination unit 602 will show a downward trend. At this time, the amount of deployed resources of the container set deployed in the target node (specifically, the sum of the amount of deployed resources of the deployed Pod) may be greater than or equal to the current amount of schedulable resources of the target node. At this time, the coefficient determination unit 602 re-determines the resource overselling coefficient by combining the actual amount of deployed resources and the amount of initially configured resources, so that the newly determined amount of schedulable resources by the scheduling determination unit 603 can meet the operation of the currently deployed containers and ensure service stability.

[0075] Specifically, the coefficient determination unit 602 redetermines the resource overselling coefficient based on the deployed resource amount and the initial configured resource amount, and the operation includes: determining the ratio of the deployed resource amount to the initial configured resource amount as the updated resource overselling coefficient, so that the corresponding updated schedulable resource amount is exactly equal to the deployed resource amount of the deployed container set, so that there are no excess resources to be allocated to other Pods and containers, and new Pods and containers will no longer be deployed, which can ensure that the deployed Pods and containers are not evicted, thereby reliably ensuring service stability.

[0076] Figure 7 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.

[0077] Reference Figure 7 The electronic device 700 includes at least one memory 701 and at least one processor 702. A set of computer executable instructions is stored in the at least one memory 701. When the set of computer executable instructions is executed by the at least one processor 702, a resource scheduling method according to an exemplary embodiment of the present disclosure is executed.

[0078] As an example, the electronic device 700 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instruction set. Here, the electronic device 700 is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above instruction (or instruction set) individually or in combination. The electronic device 700 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0079] In the electronic device 700, the processor 702 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0080] The processor 702 may execute instructions or codes stored in the memory 701, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0081] The memory 701 may be integrated with the processor 702, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 701 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 701 and the processor 702 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 702 can read files stored in the memory.

[0082] In addition, the electronic device 700 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 700 may be connected to each other via a bus and / or a network.

[0083] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided, and when the instructions in the computer-readable storage medium are executed by at least one processor of an electronic device, the electronic device is prompted to execute the resource scheduling method according to the exemplary embodiment of the present disclosure. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0084] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided. The computer program product includes computer instructions. When the computer instructions are executed by at least one processor, the at least one processor is prompted to perform the resource scheduling method according to the exemplary embodiment of the present disclosure.

[0085] According to the resource scheduling method, device, electronic device and computer-readable storage medium of the exemplary embodiments of the present disclosure, by determining the resource overselling coefficient and the corresponding schedulable resource amount according to the historical resource utilization of the target node, the actual load of the target node can be considered. When the historical resource utilization is low and the target node's previous requested resource amount is artificially high, the target node is appropriately allocated a resource amount that exceeds the actual supply capacity, so that the target node can provide more virtual resources for Pods and containers, which helps to deploy more Pods and containers, thereby improving the resource utilization of the target node and making the resources of the target node be utilized as much as possible; when the historical resource utilization is high and the target node's previous requested resource amount is relatively reasonable, resource overselling is appropriately reduced to prevent excessive resource overselling, which helps to ensure the stability of cluster services while overselling resources.

[0086] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0087] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A resource scheduling method, characterized in that: The resource scheduling method comprises: Get the historical resource utilization and initial configuration resource amount of the target node of the container cluster; Determine the resource overselling coefficient of the target node according to the historical resource utilization rate; wherein the resource overselling coefficient is a multiple of the resource amount after the initial configuration resource amount is oversold, and when the historical resource utilization rate is less than the preset resource utilization rate, the resource overselling coefficient is negatively correlated with the historical resource utilization rate; Determining the schedulable resource amount of the target node according to the resource overselling coefficient and the initial configured resource amount; The step of determining the resource overselling coefficient of the target node according to the historical resource utilization rate includes: If the historical resource utilization is less than the preset resource utilization, the oversold constant of the target node is obtained, and the historical resource utilization is indexed to obtain an indexed value, and the difference between the oversold constant and the indexed value is used as the resource oversold coefficient; wherein the oversold constant is the sum of a predetermined maximum value of the resource oversold coefficient and a minimum value of the indexed value.

2. The resource scheduling method according to claim 1, characterized in that: The step of determining the resource overselling coefficient of the target node according to the historical resource utilization rate further includes: If the historical resource utilization is greater than or equal to the preset resource utilization, the resource overselling coefficient is determined to be 1.

3. The resource scheduling method according to claim 1, characterized in that: The step of performing index processing on the historical resource utilization to obtain an index value includes: Determine the product of the historical resource utilization and a first parameter; Determine the exponential value by taking the second parameter as the base and the product as the exponent; The first parameter is a positive value and the second parameter is greater than 1, or the first parameter is a negative value and the second parameter is less than 1.

4. The resource scheduling method according to claim 1, characterized in that: After the step of determining the schedulable resource amount of the target node according to the resource overselling coefficient and the initial configured resource amount, the resource scheduling method further includes: Obtaining the amount of deployed resources of the container set corresponding to the target node; If it is determined that the deployed resource amount is greater than or equal to the schedulable resource amount, the resource oversold coefficient is re-determined based on the deployed resource amount and the initial configured resource amount, and the process returns to the step of determining the schedulable resource amount of the target node based on the resource oversold coefficient and the initial configured resource amount.

5. The resource scheduling method according to claim 4, characterized in that: The step of re-determining the resource overselling coefficient according to the deployed resource amount and the initial configured resource amount includes: A ratio of the deployed resource amount to the initially configured resource amount is determined as the updated resource overselling coefficient.

6. The resource scheduling method according to any one of claims 1 to 5, characterized in that: Before the step of obtaining the historical resource utilization and the initial configuration resource amount of the target node of the container cluster, the resource scheduling method further includes: In response to intercepting the node information, determining a node corresponding to the node information as a target node; After the step of determining the schedulable resource amount of the target node according to the resource overselling coefficient and the initial configured resource amount, the resource scheduling method further includes: The schedulable resource amount in the node information is updated, and the updated node information is reported.

7. A resource scheduling device, characterized in that: The resource scheduling device comprises: The acquisition unit is configured to execute: acquiring historical resource utilization and initial configured resource amount of a target node of the container cluster; A coefficient determination unit is configured to execute: determining a resource overselling coefficient of the target node according to the historical resource utilization rate; wherein the resource overselling coefficient is a multiple of the resource amount after the initial configuration resource amount is oversold, and when the historical resource utilization rate is less than a preset resource utilization rate, the resource overselling coefficient is negatively correlated with the historical resource utilization rate; The scheduling determination unit is configured to execute: determining the schedulable resource amount of the target node according to the resource overselling coefficient and the initial configured resource amount; The coefficient determination unit is further configured to execute: if the historical resource utilization is less than the preset resource utilization, obtain the oversold constant of the target node, and index the historical resource utilization to obtain an indexed value, and use the difference between the oversold constant and the indexed value as the resource oversold coefficient; wherein the oversold constant is the sum of a predetermined maximum value of the resource oversold coefficient and a minimum value of the indexed value.

8. The resource scheduling device according to claim 7, characterized in that: The coefficient determination unit is further configured to perform: If the historical resource utilization is greater than or equal to the preset resource utilization, the resource overselling coefficient is determined to be 1.

9. The resource scheduling device according to claim 7, characterized in that: The coefficient determination unit is further configured to perform: Determine the product of the historical resource utilization and a first parameter; Determine the exponential value by taking the second parameter as the base and the product as the exponent; The first parameter is a positive value and the second parameter is greater than 1, or the first parameter is a negative value and the second parameter is less than 1.

10. The resource scheduling device according to claim 7, characterized in that: The acquisition unit is further configured to execute: acquiring the amount of deployed resources of the container set corresponding to the target node; The coefficient determination unit is further configured to execute: if it is determined that the deployed resource amount is greater than or equal to the schedulable resource amount, re-determine the resource overselling coefficient according to the deployed resource amount and the initial configured resource amount, and run the scheduling determination unit.

11. The resource scheduling device according to claim 10, characterized in that: The coefficient determination unit is further configured to perform: A ratio of the deployed resource amount to the initially configured resource amount is determined as the updated resource overselling coefficient.

12. The resource scheduling device according to any one of claims 7 to 11, characterized in that: The resource scheduling device also includes: The interception unit is configured to execute: in response to intercepting the node information, determining the node corresponding to the node information as the target node; The updating unit is configured to execute: updating the amount of schedulable resources in the node information, and reporting the updated node information.

13. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer executable instructions, When the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to execute the resource scheduling method according to any one of claims 1 to 6.

14. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by at least one processor of an electronic device, the electronic device is prompted to execute the resource scheduling method according to any one of claims 1 to 6.

15. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by at least one processor, the resource scheduling method according to any one of claims 1 to 6 is implemented.

Images