A data-driven online prediction method for realizing oversubscription of K8S resources

Abstract

The present invention relates to the field of cloud computing technology, and specifically to a method for data-driven online prediction and implementation of K8S resource oversubscription. The method effectively predicts the oversubscription information of nodes through a neural network algorithm, provides a decision for node oversubscription, and is beneficial to improving the resource utilization rate of cluster nodes. The beneficial effects are as follows: This application makes full use of historical data in a large-scale cluster, provides the ability to predict oversubscription for nodes during cluster operation, is conducive to improving the resource usage efficiency of the cluster, and can adaptively iterate with the business and production environment. For situations where prediction fails, goes out of the confidence interval, or scheduling fails, this solution provides an automatic node switching solution and provides a warning function for cluster expansion.

Description

Technical Field

[0001] The present invention relates to the technical field of cloud computing, and particularly to a method for data-driven online prediction and realizing oversubscription of K8S resources. Background Art

[0002] Each node in the cluster has limited CPU and memory resources. When scheduling Pods, only those nodes whose remaining resources can accommodate the current scheduling Pod request volume can be used as target nodes.

[0003] The scheduler of K8S determines which nodes can accept running the relevant Pod resources according to the resource requirements defined in the requests attribute of the container; for the resources of a node, for each running Pod object, the requested volume in its resource definition needs to be reserved until the node resources are consumed by all Pod objects.

[0004] The resource requirements of the container only ensure to meet the lower limit of its resource usage, and do not limit the upper limit of the available resources of the container. Therefore, there is no way to deal with the situation where the system resources are occupied for a long time due to bugs in the application itself. This requires defining the maximum available amount of resources through the limits attribute. When allocating resources, the control threshold of the compressible resource CPU can be freely adjusted, and the container process cannot obtain CPU resources exceeding its CPU quota.

[0005] However, if the container process exceeds its memory quota, it will lead to an out-of-memory situation (OOM), affecting the operation of the application. In most cases, the actual running resource consumption of the Pod is usually less than its requested resources. Therefore, in order to improve the utilization rate of cluster nodes, on the basis of ensuring the normal operation of Pod resources, oversubscription of cluster nodes should be realized as much as possible. The allocatable value of the node can be dynamically adjusted to make the allocatable resources of the computing node appear inflated, deceiving the K8S scheduler into thinking that the node has a large amount of allocatable resources and allowing more Pods to be scheduled to this node.

[0006] In the cluster, traditional scheduling methods generally adopt the default oversubscription strategy (i.e., fixed node oversubscription rate) or predict the node oversubscription rate based on empirical formulas (predict the selling ratio through an empirically constructed formula according to the remaining resources of the node and the resource requests of the Pod).

[0007] However, the above methods have disadvantages such as lack of flexibility, being relatively mechanical, and being unable to achieve effective online real-time prediction.

[0008] For any cluster, the scheduling information of Pod resources in the cluster is of great guiding significance for subsequent Pod scheduling and node oversubscription.

[0009] Among them, it is not difficult to see that there are the following rules:

[0010] 1. The higher the node load, the lower its selling ratio;

[0011] 2. The closer the resource request is to the remaining resources of the node, the lower the node selling ratio;

[0012] 3. The allocation of Pod scheduling resource requests and the resource consumption during actual operation on the node have a certain impact on the allocation of new resource requests;

[0013] 4. There is a certain numerical dependence relationship between resource requests, the remaining resources of the node, and node oversubscription, and the request should not exceed the oversubscription.

[0014] In recent years, data-driven methods such as machine learning that have emerged widely have the characteristics of not requiring precise modeling, being able to achieve a wide range of coverage and exploration of the solution space, continuously learning and evolving from data, and having strong algorithm generality. At the same time, there are tools such as a large number of open-source models and algorithm libraries to support.

[0015] Based on the above analysis, combining data-driven prediction methods with historical data during cluster operation (including relevant information such as the operation data of Pod resources and the remaining resources of nodes), and using Pod resource requests and node remaining resources as input features to predict the quota of its scheduling nodes is of great significance for realizing node oversubscription and improving cluster resource utilization rate. Summary of the Invention

[0016] The purpose of the present invention is to provide a data-driven online prediction and K8S resource oversubscription implementation method to solve the problems raised in the above background technology.

[0017] To achieve the above purpose, the present invention provides the following technical solutions:

[0018] A data-driven online prediction and K8S resource oversubscription implementation method, and the oversubscription prediction method includes the following steps:

[0019] S1: Predict the cluster operation situation, which is divided into a cold start stage and a prediction stage.

[0020] S2: The cold start stage, that is, the non-prediction stage, is used to accumulate data for later model prediction.

[0021] S3: The prediction stage is an online iterative process, which collects information on Pod resource scheduling and operation in the cluster in real time;

[0022] S4: Normalize and perform feature engineering on the collected data, and determine that the algorithm prediction target is the oversubscription rate of the corresponding node's CPU and memory;

[0023] S5: Train multiple neural network models in parallel by day, save the neural network model files, and distribute them to each node for subsequent prediction. For each Pod, sort its real historical running data of CPU and memory, and take the data at the 80% position in the sorting (Y0.8), compare it with 80% of the quota resources (Ylimit) of the node. If Y0.8 < 0.8 * Ylimit, put this data into the training set, otherwise discard it;

[0024] S6: For the Pod to be scheduled, call the K8S initial screening nodes, and use the default scoring function to sort and divide the priorities of the nodes, and select appropriate nodes for scheduling;

[0025] S7: Take the resources applied by the Pod and the resource situation of the node as the inputs of multiple neural networks, and the output of the neural network is the oversubscription rate of the node;

[0026] S8: Calculate the prediction variances of multiple neural networks and determine whether they are within the confidence interval;

[0027] If yes, update the oversubscription rate of the node by modifying the allocatable value of the node and perform scheduling;

[0028] If no, continue to determine whether the currently selected node is the highest-specification node;

[0029] If it is the highest-specification node and the remaining resources of the node are greater than the resources applied by the Pod, adjust the oversubscribed resources of the node to the Pod request volume and directly perform scheduling;

[0030] If no, issue an expansion warning;

[0031] If it is not the highest-specification node, switch the node to a higher-specification node and enter S7;

[0032] S9. Determine whether the scheduling is successful;

[0033] If successful, collect the data of the successfully scheduled Pod and put it into the database, and enter S3;

[0034] Otherwise, switch the scheduling node to a higher-specification node and enter S7.

[0035] Preferably, the cluster operation includes that Kubelet will recalculate the allocatable resources of the node every 10 seconds by default and send a patch request to the Kube-apiserver to report the status of the node.

[0036] Preferably, the cluster operation adjusts the oversubscription rate of nodes by dynamically modifying the value of the allocatable field of the Node object of the node through the MutatingAdmissionWebhook, where adjusting the oversubscription rate of the node includes oversubscription of CPU and memory.

[0037] Preferably, in the prediction stage, the historical data of the resources already running in the cluster is sampled through a neural network algorithm, features are extracted, and multiple neural network models are constructed.

[0038] Preferably, the neural network models are set to at least 4 groups. Multiple neural networks are used to predict the scheduled Pod resources collected in real time simultaneously. The prediction results are decision - made through confidence intervals, and the oversubscription rate of the nodes is adjusted based on the MutatingAdmissionWebhook to achieve oversubscription.

[0039] Preferably, the information during the scheduling and operation of Pod resources in the cluster includes: the label information of the Pod, the requested resources during Pod scheduling, the current oversubscription situation of the node, the current load situation of the node, and the true load information of the previous Pod resource on the corresponding node.

[0040] Preferably, before parallel training by day, the network parameters need to be randomly initialized, and the input data is input. The input data of the neural network includes: the label information of the Pod, the requested resources of the Pod, the available information of the node, the total resource information of the node, and the requested resources and the occupied resources during the actual operation of the previous Pod scheduled on the corresponding node.

[0041] Preferably, if there are multiple nodes in step six, nodes that meet its requirements are randomly selected for scheduling attempts. At the same time, the input features of the resources are input into multiple neural networks for predicting the oversubscription rate of the nodes.

[0042] Preferably, the cold - start process is the initial stage of cluster startup. There are few Pod scheduling resources in the cluster, and a non - predictive resource scheduling process is required. After the cluster has run for a period of time, it enters the prediction stage.

[0043] Compared with the prior art, the beneficial effects of the present invention are as follows: This application makes full use of the historical data in the large - scale cluster, provides the prediction ability of oversubscription for the nodes during cluster operation, is conducive to improving the resource utilization efficiency of the cluster, can adaptively iterate with the business and production environment. For the situations of prediction failure, out of the confidence interval or scheduling failure, this solution provides an automatic node switching solution and provides a warning function for the expansion of the cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a system flow chart of the data - driven node oversubscription prediction and implementation process of the present invention;

[0045] Figure 2 This is the feedforward neural network model diagram for node overbooking prediction of the present invention. Specific embodiments

[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0047] Please refer to Figures 1 to 2 , the present invention provides a technical solution:

[0048] Embodiment 1:

[0049] A data-driven online prediction and method for realizing K8S resource overbooking. The overbooking prediction method includes the following steps:

[0050] S1: Predict the cluster running status, which is divided into the cold start stage and the prediction stage.

[0051] S2: The cold start stage, that is, the non-prediction stage, is used to accumulate data for later model prediction. The cold start process is the initial stage of cluster startup. There are few Pod scheduling resources in the cluster, and a non-predictive resource scheduling process is required. After the cluster runs for a period of time, it enters the prediction stage;

[0052] S3: The prediction stage is an online iterative process, which collects information on Pod resource scheduling and runtime in the cluster in real time. The information includes: label information of the Pod, requested resources during Pod scheduling, overbooking status of the node at that time, load status of the node at that time, and real load information of the previous Pod resource on the corresponding node;

[0053] S4: Normalize and perform feature engineering on the collected data, and determine that the algorithm prediction target is the overbooking rate of the CPU and memory of the corresponding node;

[0054] S5: Train multiple neural network models in parallel by day. Before training in parallel by day, the network parameters need to be randomly initialized and the input data is input. The input data of the neural network includes: label information of the Pod, requested resources of the Pod, available information of the node, total resource information of the node, and requested resources and real occupied resources during the previous Pod scheduling on the corresponding node;

[0055] Save the neural network model file and distribute it to each node for subsequent prediction. For each Pod, sort its real historical running data of CPU and memory, and take the data at the 80% position in the sorting (Y0.8), compare it with 80% of the quota resources (Ylimit) of the node. If Y0.8 < 0.8 * Ylimit, put this data into the training set, otherwise discard it;

[0056] S6: For the Pod to be scheduled, call the K8S initial screening nodes, and use the default scoring function to sort and divide the priorities of the nodes, select appropriate nodes for scheduling. If there are multiple nodes, randomly select nodes that meet its requirements for scheduling attempts. At the same time, input the input features of this resource into multiple neural networks to predict the oversubscription rate of the nodes;

[0057] S7: Take the resources applied by the Pod and the resource situation of the node as the input of multiple neural networks, and the output of the neural network is the oversubscription rate of the node;

[0058] S8: Calculate the prediction variances of multiple neural networks and determine whether they are within the confidence interval;

[0059] If yes, update the oversubscription rate of the node by modifying the allocatable value of the node and perform scheduling;

[0060] If no, continue to determine whether the currently selected node is the highest-specification node;

[0061] If it is the highest-specification node and the remaining resources of the node are greater than the resources applied by the Pod, adjust the oversubscribed resources of the node to the Pod request amount and directly perform scheduling;

[0062] If no, issue an expansion warning;

[0063] If it is not the highest-specification node, switch the node to a higher-specification node and enter S7;

[0064] S9: Determine whether the scheduling is successful;

[0065] If successful, collect the data of the successfully scheduled Pod and put it into the database, and enter S3;

[0066] Otherwise, switch the scheduling node to a higher-specification node and enter S7.

[0067] The cluster operation includes that Kubelet recalculates the allocable resources of the node every 10 seconds by default, and sends a patch request to Kube-apiserver to report the status of the node. The cluster operation adjusts the oversubscription rate of the node by dynamically modifying the value of the allocatable field of the node Node object through MutatingAdmissionWebhook. The adjustment of the oversubscription rate of the node includes the oversubscription of CPU and memory.

[0068] In the prediction stage, through the neural network algorithm, the historical data of the resources already running in the cluster is sampled, features are extracted, and multiple neural network models are constructed.

[0069] The neural network models are set to at least 4 groups. Multiple groups of neural networks are used to predict the scheduled Pod resources collected in real time simultaneously. The prediction results are decision-making through the confidence interval, and the oversubscription is achieved by adjusting the oversubscription rate of the node based on MutatingAdmissionWebhook.

[0070] Embodiment 2:

[0071] On the basis of Embodiment 1, four groups of neural network models are set.

[0072] The present invention is used to solve the problem of oversubscription prediction of node resources in cloud computing. On the basis of ensuring the successful scheduling of resources, the oversubscription information of nodes is effectively predicted through the neural network algorithm, providing a decision for node oversubscription, which is beneficial to improving the resource utilization rate of cluster nodes.

[0073] During the cluster operation, Kubelet recalculates the allocable resources of the node every 10 seconds by default, and sends a patch request to Kube-apiserver to report the status of the node.

[0074] Based on this, the oversubscription rate of the node can be adjusted by MutatingAdmissionWebhook by dynamically modifying the value of the allocatable field of the node Node object, where the oversubscription rate of the node includes the oversubscription of CPU and memory.

[0075] The historical data of the resources already running in the cluster is sampled through the neural network algorithm, features are extracted, and multiple neural network models are constructed.

[0076] Due to the problem of overfitting in the neural network, 4 groups of neural networks are trained simultaneously to predict the currently scheduled Pod resources simultaneously. The prediction results are decision-making through the confidence interval, and the oversubscription is achieved by adjusting the oversubscription rate of the node based on MutatingAdmissionWebhook.

[0077] The specific steps are as follows:

[0078] The cold start process, that is, in the initial stage of cluster startup, there are fewer Pod scheduling resources in the cluster, and a non-predictive resource scheduling process is required, which is also the data collection process for later prediction. After the cluster runs for a period of time, it enters the prediction process.

[0079] The prediction process, the prediction and implementation process of the node resource overcommitment rate based on the algorithm is as follows:

[0080] Step 1: Collect information on Pod resource scheduling and runtime in the cluster in real time, including: Pod label information, requested resources (CPU request and memory request) when Pod is scheduled, the overcommitment situation of the node at that time (CPU and memory overcommitment rates), the load situation of the node at that time (CPU and memory loads), and the real load information of the previous Pod resource on the corresponding node (CPU and memory occupancy).

[0081] Step 2: Preprocess the information collected in Step 1, including normalization data processing of features, feature engineering on the input (one-hot encoding of label information), etc., and determine the algorithm prediction target as the overcommitment rates of CPU and memory on the corresponding node.

[0082] Step 2: Build 4 neural networks, randomly initialize network parameters, input data, and train them in parallel on a daily basis, that is, the training frequency is once a day. Low frequency can avoid bringing additional load to the cluster. The trained model files are saved and distributed to each node for later prediction of node resource overcommitment;

[0083] The input of the neural network is Pod label information, Pod requested resources (CPU and memory), available information of the node (CPU and memory), total resource information of the node (CPU and memory), and requested resources (CPU and memory) and real runtime occupancy resources (CPU and memory) of the previous Pod scheduled on the corresponding node.

[0084] Fully consider the resource applications of Pods that have been running or scheduled in the cluster during the current time period, the remaining resources of the corresponding nodes, and the resource consumption situation during the real runtime of the Pods, and fully mine and utilize these data.

[0085] The network is a two-layer feedforward neural network, with 40 and 10 neurons. The output dimension of the neural network is 2, corresponding to the CPU and memory overcommitment rates of the node. Among them, in order to improve the prediction ability of the model, the following screening process is adopted for the collected data:

[0086] For all Pods on the node, collect their data within the past ten days in units of Pods;

[0087] For each Pod, sort the historical running resource consumption (CPU and memory) for all time periods (n data), and calculate the 80% position Q with reference to the quartile calculation method, i.e., Q = (n + 1) * 0.8;

[0088] Judge the relationship between the resource consumption data Y0.8 corresponding to the data at the Qth position and its oversubscription limit ylimit;

[0089] If Y0.8 > 0.8 * Ylimit, ignore the Pod data; otherwise, collect it as training data;

[0090] This method can screen out data that ensures the oversubscription of nodes meets the Pod running requirements by analyzing the relationship between real historical running data and node quotas.

[0091] Step 4: When new resources are to be scheduled, according to the resource application situation and the load situation of the corresponding nodes, predict the oversubscribed resources of the corresponding scheduling nodes through a data-driven training model, and extract the requested resource information;

[0092] Obtain available nodes through the scheduling screening of K8S, and sort and divide the priorities of the nodes based on the scores of the built-in scoring function of K8S;

[0093] If there are multiple nodes, randomly select a node that meets its requirements for scheduling attempts, and at the same time input the input features of the resource into multiple neural networks to predict the oversubscription rate of the nodes.

[0094] Step 5: Simultaneously output the oversubscribed data (y1, y2, y3, y4) of the nodes through 4 neural networks, and calculate the variance of the prediction, where rCPU and rMem respectively represent the oversubscription rates of the node CPU and memory, is the error descriptor, and are the lower deviation limit and the upper deviation limit (generally, the interval length is 0.1 - 0.15, and 0.1 and 0.2 can be taken respectively), and judge whether it satisfies:

[0095] y i =[r CPU ,r Mem

[0096]

[0097]

[0098] Step 6: Propose to judge whether the neural network prediction is reliable based on the confidence interval. If it is lower than the lower limit, it means that the model gives a consistent decision for the current four models, suggesting that there may be deviations in the current data;

[0099] If it is higher than the upper limit, it means that the decisions given by the current model are inconsistent, that is, there are deviations in the model directly;​

[0100] When the data deviation is small and the model decisions are relatively consistent, that is, when this requirement is met, update the allocatable field value of the node through MutatingAdmissionWebhook according to the predicted overbooking rate, and perform scheduling;

[0101] If the confidence interval requirement is met, perform scheduling, and enter the judgment of whether the scheduling is successful. If the scheduling is successful, collect data and put it into the historical data for model training;

[0102] If the confidence interval requirement is not met, then judge whether the currently scheduled node is the highest-specification node;

[0103] If this node is not the highest-specification node, switch the scheduled node to a higher-specification node according to the node priority, and return to step 4;

[0104] If the currently scheduled node is the highest-specification node, then judge whether the remaining resources of the current node meet the resource request requirements;

[0105] If the resource request requirements are not met, issue a warning for capacity expansion;

[0106] If it is met, update the allocate of the node to the requested value of the resource and attempt to schedule.

[0107] Judge whether the scheduling is successful. If the scheduling is successful, collect data and put it into the historical data for model training.

[0108] If the scheduling fails, judge whether the current node is the current highest-specification node;

[0109] If it is, issue a warning for capacity expansion.

[0110] If it is not, then switch to a higher-specification node and return to step 4.

[0111] This method provides a method based on multi-model confidence interval prediction, which solves problems such as weak generalization ability caused by the deviation of training data in prediction and weak generalization ability caused by overfitting of neural network models. It can fully explore the law of resource occupation of nodes by resources in the cluster, provide a prediction function for node resource overbooking in the cluster, provide an automated scheduling node adjustment process, effectively improve the utilization efficiency of node resources, and provide guidance for the capacity expansion of nodes.

[0112] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A data-driven online prediction and method for realizing oversubscription of K8S resources, characterized in that: The method for overbooking prediction includes the following steps: S1: Predict the cluster operation status, which is divided into the cold start phase and the prediction phase; S2: The cold start phase, that is, the non-prediction phase, is used to accumulate data for later model prediction; S3: The prediction phase is an online iterative process that collects information on Pod resource scheduling and runtime in the cluster in real time; S4: Normalize the collected data, perform feature engineering processing, and determine that the algorithm prediction target is the overbooking rate of the corresponding node's CPU and memory; S5: Parallelly train multiple neural network models by day, save the neural network model files, and distribute them to each node for subsequent prediction. For each Pod, sort its real historical operation data of CPU and memory, and take the data at the 80% position of the sorting, denoted as Y0.8, and compare it with 80% of the node's quota resources, where the node's quota resources are denoted as Ylimit; If Y0.8 < 0.8 * Ylimit, then put this data into the training set, otherwise discard it; S6: For the Pod to be scheduled, call the K8S initial screening nodes, and use the default scoring function to sort and divide the priorities of the nodes, and select appropriate nodes for scheduling; S7: Take the resources applied by the Pod and the resource situation of the node as the inputs of multiple neural networks, and the output of the neural network is the node overbooking rate; S8: Calculate the prediction variances of multiple neural networks and determine whether they are within the confidence interval; If the judgment is yes, then update the node overbooking rate by modifying the allocatable value of the node and perform scheduling; If the judgment is no, then continue to judge whether the currently selected node is the highest-specification node; If it is the highest-specification node and the remaining resources of the node are greater than the resources applied by the Pod, adjust the overbooked resources of the node to the Pod request volume and directly perform scheduling; If the judgment is no, then issue an expansion warning; If it is not the highest-specification node, switch the node to a higher-specification node and enter S7; S9. Judge whether the scheduling is successful; If successful, then collect the data of the successfully scheduled Pod and put it into the database, and enter S3; Otherwise, switch the scheduling node to a higher-specification node and enter S7.

2. A method for data-driven online prediction and implementation of K8S resource oversubscription according to claim 1, characterized in that: The cluster operation includes that Kubelet will recalculate the allocatable resources of the node every 10 seconds by default, and send a patch request to Kube-apiserver to report the status of the node.

3. A data-driven online prediction and method for implementing oversubscription of K8S resources according to claim 2, characterized in that: The cluster operation adjusts the overbooking rate of the node by dynamically modifying the allocatable field value of the node Node object through MutatingAdmissionWebhook, where adjusting the overbooking rate of the node includes the overbooking of CPU and memory.

4. A data-driven online prediction and K8S resource oversubscription implementation method according to claim 1, characterized in that: In the prediction phase, through the neural network algorithm, sample the historical data of the resources already running in the cluster; Extract features and construct multiple neural network models.

5. A data-driven online prediction and K8S resource oversubscription implementation method according to claim 4, characterized in that: The neural network models are set to at least 4 groups. Multiple groups of neural networks are used to predict the resources of the scheduled Pod collected in real time simultaneously. Make decisions on the prediction results through the confidence interval, and realize overbooking by adjusting the overbooking rate of the node based on MutatingAdmissionWebhook.

6. A data-driven online prediction and method for realizing oversubscription of K8S resources according to claim 1, characterized in that : The information on Pod resource scheduling and runtime in the cluster includes the label information of the Pod, the requested resources during Pod scheduling, the oversubscription situation of the node at that time, the load situation of the node at that time, and the true load information of the previous Pod resource on the corresponding node.

7. A method for data-driven online prediction and realizing oversubscription of K8S resources according to claim 1, characterized in that: Before parallel training by day, it is necessary to randomly initialize the network parameters and input data. The input data of the neural network includes: the label information of the Pod, the requested resources of the Pod, the available information of the node, the total resource information of the node, and the requested resources and the occupied resources during the actual runtime of the previous Pod scheduling on the corresponding node.

8. A data-driven online prediction and method for implementing oversubscription of K8S resources according to claim 1, characterized in that: If there are multiple nodes in S6, randomly select nodes that meet its requirements for scheduling attempts, and at the same time input the input features of this resource into multiple neural networks for predicting the oversubscription rate of the nodes.

9. A data-driven online prediction and K8S resource overbooking implementation method according to claim 1, characterized in that: The cold start process is the initial stage of cluster startup. There are fewer Pod scheduling resources in the cluster, and a non-predictive resource scheduling process is required. After the cluster has been running for a period of time, it enters the prediction stage.

Images