A distributed scheduling method and device based on prediction

Abstract

The present invention discloses a prediction-based distributed scheduling method and device. The method includes: obtaining the historical resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster; inputting the historical resource utilization of the currently scheduled Pod into a deep learning model for training and reasoning to obtain the resource prediction utilization of the currently scheduled Pod; screening the available nodes in the cluster according to the resource prediction utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster to determine the target scheduling node of the currently scheduled Pod. The embodiment of the present invention predicts the resource utilization of the Pod by using a deep learning model, and scores the nodes in the cluster based on the predicted data, so as to optimize the situation in which the Pod is always in an abnormal state due to the cluster overload state, so as to automatically adjust the overload state of the cluster, thereby optimizing the performance upper limit of the cluster.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of cluster resource scheduling, and in particular to a prediction-based distributed scheduling method and device. Background Art

[0002] Kubernetes is a very popular container orchestration tool. It has attracted the attention of the industry with its advanced design concept and is widely used in actual production environments. An important task of Kubernetes is to select a suitable node to run a Pod (the smallest unit created and deployed in Kubernetes, which is a running instance). The load of the entire cluster is determined by the resource utilization of each node in the cluster, and the utilization of each node is closely related to the Pod running on the node. Therefore, the cluster scheduling strategy greatly affects the cluster's load status and resource utilization.

[0003] Generally speaking, when using the Kubernetes default scheduling algorithm to schedule computing tasks with large resource demand deviations, the cluster will experience server overload.

[0004] In the prior art, cluster status problems are generally solved by extending the Kubernetes scheduler, but this method either does not consider the time factor or ignores low-priority tasks, and both still need to be improved. Summary of the invention

[0005] The present invention provides a prediction-based distributed scheduling method and device to optimize the situation in which a Pod is always in an abnormal state due to the cluster's overload state, automatically adjust the cluster's overload state, and thus optimize the performance upper limit of the cluster.

[0006] In a first aspect, an embodiment of the present invention provides a prediction-based distributed scheduling method, which is applied in a Kubernetes cluster, including:

[0007] Get the historical resource utilization of the currently scheduled Pod and the resource utilization of available nodes in the cluster;

[0008] Inputting the historical resource utilization of the currently scheduled Pod into a deep learning model for training and reasoning to obtain the predicted resource utilization of the currently scheduled Pod;

[0009] According to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster, the available nodes in the cluster are screened to determine the target scheduling node of the currently scheduled Pod.

[0010] In a second aspect, an embodiment of the present invention further provides a prediction-based distributed scheduling device, configured in a Kubernetes cluster, comprising:

[0011] The acquisition module is used to obtain the historical resource utilization of the currently scheduled Pod and the resource utilization of available nodes in the cluster;

[0012] A prediction module, used to input the historical resource utilization of the currently scheduled Pod into a deep learning model for training and reasoning to obtain the predicted resource utilization of the currently scheduled Pod;

[0013] The determination module is used to screen the available nodes in the cluster according to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster to determine the target scheduling node of the currently scheduled Pod.

[0014] The present invention predicts the resource utilization of Pods by using a deep learning model, and scores the nodes in the cluster based on the predicted data to determine the optimal target scheduling node for the currently scheduled Pod, so as to optimize the situation in which the Pod is always in an abnormal state due to the cluster overload state, automatically adjust the cluster overload state, and thus optimize the performance upper limit of the cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A flowchart of a prediction-based distributed scheduling method provided in Embodiment 1 of the present invention;

[0016] Figure 2 A processing diagram of a prediction-based distributed scheduling method provided in Embodiment 1 of the present invention;

[0017] Figure 3 A flowchart of node scoring provided in Embodiment 1 of the present invention;

[0018] Figure 4 A working sequence diagram of the local working mode of the model provided in the second embodiment of the present invention;

[0019] Figure 5 A schematic diagram of CPU and memory utilization of cluster nodes in the prior art provided in the third embodiment of the present invention;

[0020] Figure 6 A schematic diagram of CPU and memory utilization of cluster nodes after using the prediction-based distributed scheduling method provided in the third embodiment of the present invention;

[0021] Figure 7 A schematic diagram of the structure of a prediction-based distributed scheduling device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0023] Embodiment 1

[0024] Figure 1 A flowchart of a prediction-based distributed scheduling method provided in Example 1 of the present invention is provided. This embodiment is applicable to the scheduling of Pods by a Kubernetes scheduler. The method can be executed by a prediction-based distributed scheduling device.

[0025] The prediction-based distributed scheduling method provided in this embodiment first obtains the resource usage of the Pod, that is, it is necessary to obtain the resource usage data of the Pod level in real time. After obtaining the data, the data is used to train the deep learning model. The trained model is deployed to the cluster to predict the resource occupancy of the Pod in the form of a service. When scheduling, the resource usage of the Pod will also be involved in the node scoring. Finally, a node with the highest score is selected to bind and run the Pod. For the specific processing process, see Figure 2 .

[0026] Furthermore, a prediction-based distributed scheduling method provided in an embodiment of the present invention specifically includes the following steps:

[0027] S110. Obtain the historical resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster.

[0028] In this embodiment, the historical resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster are obtained by deploying a cluster resource monitoring strategy of Prometheus+Grafana in the cluster.

[0029] The resources specifically include three indicators: CPU utilization, memory utilization, and IOwait rate. Specifically, this embodiment can obtain them from the data interface exposed by the prometheus server in real time through the following three formulas:

[0030] a)CPU utilization:

[0031] (1-sum(increase(node_cpu_seconds_total{mode="idle"}[10s]))by(instance) / sum(increase(no de_cpu_seconds_total[10s]))by(instance))*100;

[0032] b) Memory utilization:

[0033] (1-((node_memory_Buffers_bytes+node_memory_Cached_bytes+node_memory_MemFree_bytes) / node_memory_MemTotal_bytes))*100;

[0034] c)IOwait rate:

[0035] (sum(increase(node_cpu_seconds_total{mode="iowait"}[10s]))by(instance) / sum(increase(node_cpu_seconds_total[10s]))by(instance))*100.

[0036] Since this embodiment can use NodePort to expose services, the above data acquisition process can be completed on any node in the cluster, so that each node in the cluster will expose a port, and data can be obtained from the specified IP+Port. Another method is LoadBalancer, which exposes data in the form of a domain name. Whether inside or outside the cluster, the domain name can be used to access the data.

[0037] S120. Input the historical resource utilization of the currently scheduled Pod into a deep learning model for training and reasoning to obtain the predicted resource utilization of the currently scheduled Pod.

[0038] After obtaining the resource usage data of the Pod, the collected data is used for model training. The data for model training and reasoning can be obtained through the method provided by S110.

[0039] Since the data is single-dimensional after collection, it is necessary to preprocess the data. Specifically, the CPU utilization and memory utilization are taken as single samples, and the data length can be predicted according to the actual situation. The processed data is placed in a specified file to facilitate subsequent automated training.

[0040] For data prediction, the collected real-time data is fed into the model and the inference program is executed to obtain the prediction results for each node. The prediction results are then exposed through the specified endpoint for scoring the nodes during subsequent Pod scheduling.

[0041] The deep learning network model in this embodiment can be an RNN model, an LSTM model or an improved Multi-Attentional Characteristic Memory (M-ACM) model. Among them, the working unit of M-ACM consists of three working units, each of which uses a bidirectional LSTM structure with an attention mechanism added, and finally adds a fully connected layer for data output.

[0042] S130: Screen the available nodes in the cluster according to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster to determine the target scheduling node of the currently scheduled Pod.

[0043] In this embodiment, after the acquired data is used for training and prediction of the deep learning model, the prediction results are used in the subsequent scheduling strategy. The prediction results based on the Pod resource usage can be considered to solve the problem of unbalanced data allocation when allocating multi-dimensional resources. There are some problems in the resource allocation process of the cluster, that is, when a new service is deployed, some resources in the cluster are left over, but the new service cannot be deployed.

[0044] By monitoring the resource usage of the cluster, it is found that the cluster resources meet the resource requests of some Pods, but there are still a large number of Pods that are always in a waiting state, that is, they are always preparing before applying for nodes. There are many reasons for this state, such as: image pull failure, insufficient resources, restrictions on Pod scheduling rules, problems with taint settings, etc. However, the main reasons are the first two, so this embodiment considers ensuring that the CPU and memory load levels are as close as possible when the Pod is scheduled for the first time. When the Pod needs to be expelled and rescheduled to meet certain requirements, the previous resource usage of the Pod is sent to the model to obtain the predicted resource usage of the Pod, and the node is scored based on the predicted data.

[0045] Specifically, according to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster, the available nodes in the cluster are screened to determine the target scheduling node of the currently scheduled Pod, including:

[0046] Pre-screening the available nodes based on a built-in algorithm of Kubernetes to obtain candidate available nodes;

[0047] According to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the candidate available nodes, the candidate available nodes are screened to determine the target scheduling node of the currently scheduled Pod.

[0048] In this embodiment, in the preselection stage of the scheduler, the available nodes are pre-screened based on the built-in algorithm of Kubernetes to relax the conditions for node selection, that is, if the most basic scheduling conditions are met, they can pass the preselection and become candidate available nodes. Since the above-mentioned built-in algorithm of Kubernetes is an existing technology, it will not be described in detail.

[0049] Furthermore, after pre-screening the available nodes, the candidate available nodes that have passed the pre-screening are further scored and screened. Specifically, the candidate available nodes are screened according to the resource predicted utilization of the currently scheduled Pod and the resource utilization of the candidate available nodes to determine the target scheduling node of the currently scheduled Pod, including two parts:

[0050] a) Statistics and scoring of server remaining resource rate

[0051] That is, according to the resource utilization rate of each candidate available node, the first score of each candidate available node is determined, and the calculation formula of the first score is:

[0052] Score1=10-(U nc +U nm )*5;

[0053] Among them, Score1 represents the first score, U nc Indicates the CPU utilization of the candidate available node, U nm Represents the memory utilization of candidate available nodes.

[0054] The first scoring method above considers selecting nodes with low resource utilization as much as possible to ensure the coarse-grained load balancing of each node. The nodes are scored based on their usage. The lower the resource utilization of the node, the higher the score.

[0055] b) Statistical scoring of the difference between CPU and memory utilization

[0056] If the currently scheduled Pod is running for the first time, the second score of each candidate available node is determined based on the resource utilization of each candidate available node and the resource application ratio of the currently scheduled Pod. Specifically, the CPU application ratio and memory application ratio of the Pod are obtained, and added to the original resource utilization of the node to obtain the expected CPU utilization and memory utilization respectively. The smaller the difference between the two, the higher the score. The calculation formula for the second score is as follows:

[0057] Score2-1=10-(U nc +R pc -(U nm +R pm ))*10;

[0058] Score2-1 indicates the second score when the currently scheduled Pod is running for the first time. nc and R pc Represents the CPU utilization of candidate available nodes and the CPU application ratio of the currently scheduled Pod, U nm and R pm They represent the memory utilization of the candidate available nodes and the memory application ratio of the currently scheduled Pod respectively.

[0059] If the currently scheduled Pod is not running for the first time, the second score of each candidate available node is determined based on the resource utilization of each candidate available node and the predicted resource utilization of the currently scheduled Pod. That is, if it has been running on a certain node for a period of time, but due to some error, or it is forcibly deleted by some eviction policy, or it is manually deleted by the user, in this case, the resource usage results can be predicted based on the previous data using a deep learning model to obtain the expected CPU and memory occupancy in the future. In this case, the smaller the difference between the two, the higher the node score. At this time, the calculation formula for the corresponding second score is as follows:

[0060] Score2-2=10-(U nc +U pcp -(U nm +U pmp ))*10;

[0061] Among them, Score2-2 is the second score when the current scheduling is not the first operation, U nc and R pcp Respectively represent the CPU utilization of the candidate available nodes and the CPU predicted utilization of the currently scheduled Pod; U nm and R pmp They represent the memory utilization of candidate available nodes and the predicted memory utilization of the currently scheduled Pod, respectively.

[0062] Further, after obtaining the first score and the second score of each candidate node, the candidate available nodes are sorted according to the first score and the second score to determine the target scheduling node of the currently scheduled Pod, including: taking the sum of the first score and the second score as the target score of each candidate available node; and taking the candidate available node with the highest target score as the target scheduling node of the currently scheduled Pod.

[0063] Furthermore, based on the above embodiments, this embodiment also considers binding Pods that provide the same service to different physical nodes, that is, using anti-affinity to perform distributed deployment of Pods. By introducing a Pod anti-affinity algorithm, it is possible to avoid Pods that provide the same service from running on the same node, thereby improving the high availability of the service. In order to determine the anti-affinity between Pods, the Pods need to be labeled accordingly. If they have the same labels, these Pods are said to have anti-affinity. Then the more Pods that have the same labels as the currently scheduled Pod, the lower the score. The following is used to score the affinity of the node, where AAPodi represents the i-th Pod with an anti-affinity label.

[0064] Scores A =10-∑(AApod i ) / 10

[0065] In a Kubernetes cluster, if you want to exclude Pod nodes from each other, you can also use the Matchlabel strategy. However, this is a hard strategy. If the conditions are not met, the node will be directly filtered out. It is hoped that the scope of Pod operation can be relaxed, and it is also hoped that the Pod of the same application or providing the same service must communicate across nodes. On the other hand, using hard constraints to constrain Pod node selection may cause some Pods to fail to run if there are many Pods or replicas, or the number of nodes is insufficient. Therefore, a soft condition is used to constrain Pod anti-affinity. Only when the conditions are not met will the Pods providing the same service be scheduled to the same node. In this scheduling algorithm, the actual CPU and memory usage of the node and the change in resource usage after the Pod is scheduled are mainly considered. Therefore, when scoring, nodes with low load and small CPU and memory utilization differences after the resources requested by the Pod are allocated are given priority as the best scheduling nodes.

[0066] Figure 3 The workflow of S130 is shown. If the currently scheduled Pod is running for the first time, the algorithm's Score1 and Score2-1 will be used directly for scoring. If the Pod has been run before but was deleted for some reason, the algorithm's Score2-2 can be used for scoring based on the predicted resource utilization of the queried Pod.

[0067] The embodiment of the present invention uses deep learning technology to predict the usage of pod resources, and then schedules the Pod to a more suitable node in advance, rather than waiting until load imbalance actually occurs to make adjustments. This optimizes the situation in which the Pod is always in an abnormal state due to the cluster load imbalance, automatically adjusts the cluster load imbalance, and thus optimizes the performance ceiling of the cluster.

[0068] Embodiment 2

[0069] The prediction-based distributed scheduling method provided in this embodiment can extend the Kubernetes Scheduler in the form of an Extender, and embed the custom algorithm in the Extender. It is possible to obtain data from a specified endpoint in the Extender client, and this endpoint can be a URL, and the data is exposed in a Restful style.

[0070] 1. Algorithm workflow: The prediction-based distributed scheduling method proposed in this embodiment is added to the scheduling process of the Kubernetes platform in the form of extensions or plug-ins, which is convenient for subsequent expansion or replacement of different algorithms. After consulting the relevant documents of Kubernetes, consider using the Extender method to embed the algorithm into the Kubernetes scheduling process. The proposed prediction-based scheduling scoring algorithm takes effect in the priority stage of the Scheduler. In this stage, a list of objects of the current scheduled Pod object and available nodes is required, and a scored node list is output.

[0071] First, each node is traversed. When traversing each node, a basic score is given to the node based on the built-in algorithm of Kubernetes. The CPU and memory utilization of the node are obtained from the collector. The formula Score1 = 10-(U nc +U nm )*5 is used to calculate the score of the first part. In this part, the higher the load of the node, the lower the score, because we hope to select a server with a lower load to shorten the load gap between different servers.

[0072] For the second part of the scoring, we need to first judge the Pods in the current scheduling round. If the Pod is created for the first time, we cannot obtain the resource usage prediction result of the Pod. However, we should also try to maintain a balanced use of various resources. Based on the formula Score2-1=10-(U nc +R pc -(U nm +R pm))*10, pre-allocate resources first, and calculate the CPU and memory load differences based on the pre-allocation results. It is hoped that after the Pod is scheduled to a node, the CPU and memory load differences of the node will be as small as possible.

[0073] If the Pod is not running for the first time, you can perform prefix matching on the Pod name in ETCD, and then obtain the resource prediction results for the Pod. The stored data is the average CPU and memory usage in the last 30 minutes. The collection interval is 10 seconds. The average CPU and memory utilization of the Pod is used as the final prediction result, and the formula Score2-2=10-(U nc +U pcp -(U nm +U pmp ))*10 calculates the final score of the second part. In this part, if the difference between CPU and memory resource usage is higher, it is considered that scheduling the current Pod to this node will cause a higher degree of resource overload. In this case, the node score will be lowered to ensure multi-dimensional resource load balancing.

[0074] After traversing all nodes, based on the Scheduler Extender's Pod score, the final score of each available node is obtained, and the nodes are sorted according to the score. The score result is sent to Sheduler. Scheduler writes the ID of the best node into the NodeName field of the Pod, and writes the binding information to ETCD through the API server. After Kubelet monitors the change of Pod information, it will call Docker to create a container and write the container creation information to ETCD through the API Server. The Pod of the current scheduling round is actually running on a server node in the Kubernetes cluster, and then the next Pod in the Pod scheduling queue will execute the same process to select the best scheduling node.

[0075] 2. Model Embedding

[0076] Since the algorithm only needs to obtain the prediction results of the model, two approaches can be considered.

[0077] (1) Locally run the model

[0078] In this way, model training, inference, and prediction results can be exposed in the GPU servers in the cluster. In this way, the model works as follows:

[0079] 1) Data acquisition: Obtain and clean data from the data endpoints exposed by Prometheus.

[0080] 2) Model training: Preprocess the cleaned data, and then send the preprocessed data to the model for training. Consider putting the data preprocessing part in the model training code, so that you only need to get the cleaned data from Prometheus. In the preprocessing of the model, considering the shortcoming of the RNN family model that it has poor parallelism, standardize the data during data preprocessing to speed up the convergence of the model, reduce training time, and improve the efficiency of accommodating feature storage.

[0081] 3) Model reasoning: The trained model is saved in the local folder of the GPU server in the form of a weight file. For data reasoning, data is obtained from the Prometheus endpoint and fed into the model for reasoning.

[0082] 4) Exposure of prediction results: For the prediction results of the model, the latest prediction results are exposed through a client program at a fixed endpoint. The Extender program only needs to go to the specified endpoint to pull the prediction results.

[0083] Figure 4 The working sequence of the local working mode of the model is shown. In this mode, the obtained prediction results can be exposed to the specified endpoint through a client program. When the Pod is scheduled, the Extender program queries the response prediction result record from the specified endpoint for the scoring operation of the scheduling algorithm.

[0084] The advantage of this method is that the model parameters and training status are persistent, and there is no need to apply for PV volumes or other Pod storage persistence operations. The disadvantage is that the model itself and the data exposure endpoint are outside the cluster, and there is a certain delay in communication.

[0085] (2) Run the model in a container

[0086] You can first train the model, then package the model weight file, training code, and inference code into a Docker container, and then perform model inference and retraining in the form of a Pod. The model works as follows:

[0087] 1) Data acquisition: In this way, Prometheus does not need to expose the collected data outside the cluster. The client program running in the Pod can obtain data through the IP address of the server node in the cluster plus the port specified by Prometheus.

[0088] 2) Model training: Complete the model training work locally. This part of the process is similar to the working method of the local model.

[0089] 3) Model packaging: Package the trained model into a Docker image. If you use GPU for inference, you need to mount the specified Nvidia library file to a directory accessible to the specified container when deploying the model's Docker image, and deploy Nvidia-Device-Plugin in advance. If you use CPU for inference, you do not need to configure the GPU.

[0090] 4) Model reasoning: Running the trained model in Pod mode can also expose data to the outside in the form of Service. However, since it runs in Pod, data can be exposed only within the cluster in the form of cluster virtual IP plus port.

[0091] 5) Exposure of prediction results: In Extender, you can obtain data in the form of IP+Port. If you need to access it outside the cluster, you need to use NodePort or LoadBalance to expose the endpoint containing the prediction data.

[0092] This model works similarly to the local model, but the model training environment needs to be packaged into a Docker image and the prediction results need to be persisted to the disk through PV or PVC.

[0093] Both methods can complete data acquisition, model inference and result exposure. If you need to manually optimize the code regularly or need to retain data safely and conveniently, you can consider completing data collection, model training and inference, and inference result exposure directly locally.

[0094] 3. Algorithm Embedding

[0095] Based on the prediction results of the deep learning model, the cluster scheduling process is intervened to select relatively good nodes to run Pods, ensure cluster load balancing, reduce resource overload and increase the upper limit of cluster resource availability.

[0096] Furthermore, after adding the scheduling algorithm, the overall working process of the scheduling work is as follows:

[0097] (1) Data collection: First, deploy the Prometheus server in the cluster. Its built-in TSDB database can be used to store the collected cluster data. The latest data within two hours is stored in the memory, and other old data will be persisted to the local disk, but it is consistent with the life cycle of the Prometheus Server Pod. After the Prometheus Server is restarted, all data is lost. Therefore, consider writing the prediction results of Pod resource usage into the ETCD key-value pair storage system.

[0098] (2) Data cleaning: After acquiring the data, the data is sampled with a step size of 3, which increases the feature transformation frequency to a certain extent and predicts the occurrence of abnormal situations earlier. For samples with zero or very close to zero data, it is considered that the workload of the Pod during this period is almost zero, so samples with less than 0.1% are filtered out. Then the samples are written to the local file in a one-dimensional two-element array.

[0099] (3) Model training: After obtaining the resource usage data, the model can be read from the specified path. There are two ways to implement this method. One is to run the acquisition module and code training module in the Pod, and put the collected data in a virtual folder in the container. When training the model, the collected data can be read like a local file. Another method is to store the data locally and mount the local folder to the specified location in the container when deploying the model. After obtaining the data, use the data generator module to generate 180:60 data samples and send them to the model for training. If you want to train or infer on the GPU, you need to mount the Nvidia library used in the container. This method can only be used for training and inference on the specified GPU server after mounting, which will bring certain pressure to the GPU server in the cluster. After training, save the model to the specified directory and configure the path in the inference module.

[0100] (4) Resource usage reasoning: Read the most recent CPU and memory usage from the file written by the data collection module, load the trained model, and perform reasoning. This part only requires writing the timing prediction function of the prediction module. In the experiment, it takes less than 40 seconds to reason about 10,000 samples, so a single sample of 180 length takes less than 1 second. When the resource usage reasoning of all Pods is completed, the average value of the reasoning results is written to ETCD and stored in the form of key-value pairs, where the key uses the Pod name after removing the generated random number suffix as the identifier. For the value, I use the average value of CPU and memory utilization and store it in ETCD as a one-dimensional binomial array. If the homogeneous Pod is run multiple times, multiple prediction results will be generated. In order to avoid the occurrence of load imbalance as much as possible, the record with the largest difference between the average CPU and memory utilization is used to overwrite the current record.

[0101] (5) Scheduler Scheduling: The Scheduler will continuously scan the Pods with empty node names in ETCD through the API Server. These Pods will be organized into a queue and sent to the Scheduler together with the list of available nodes. The elements of the Pod and available node queues are corresponding object pointers. The Scheduler enters the pre-selection stage. This stage mainly selects which resources are sufficient to support the operation of the Pod, or some selector controllers strictly stipulate that it can only run on nodes of a specified type. The available nodes must meet these hard conditions. According to the built-in algorithm, the unavailable nodes are filtered out, and a list of available nodes and unavailable nodes and their reasons for unavailability are generated. After the Extender configuration is set, this information will be sent to the entry address specified by the Extender, and the Extender will perform secondary filtering. In this part, no additional processing is performed, and the available nodes only need to meet the indicator conditions specified by Kubernetes. Therefore, in the pre-selection stage, the original pre-selection results generated by the Scheduler based on the built-in pre-selection algorithm are returned. The Scheduler enters the optimization stage. After executing the built-in algorithm, the node scoring or sorting results and the Pod object pointer are sent to the entry address of the Extender. In this part, if there is a running record of the Pod before, the key value record of the ID prefix of the current Pod needs to be found from ETCD first. Then it is sent to the Priority module in Extender, and the nodes are scored based on the actual CPU and memory utilization of each node and the model prediction results, and the scoring results are updated and sent.

[0102] (6) Node Binding and Pod Running: After the Scheduler receives the scoring result from the Extender, the Node name field of the Pod is written with the node ID with the highest score. Then, the binding result of the Pod and the node is written to ETCD through the API Server. Then, the next round of Pod scheduling is carried out.

[0103] This embodiment provides a technical solution to the problem of Kubernetes cluster resource overload. It uses a multi-attention feature memory model to predict resource utilization, and designs a Kubernetes distributed scheduling strategy through a model that predicts resource utilization.

[0104] Embodiment 3

[0105] This embodiment provides an experimental verification method and verification results of a prediction-based distributed scheduling method.

[0106] 1. Experimental environment

[0107] The experimental environment in this embodiment is carried out on a Kubernetes distributed cluster with 26 nodes.

[0108] 2. Introduction to testing tools

[0109] This embodiment uses PowerfulSeal as a test case debugging tool, and the functions of the tool include:

[0110] (1) This tool can obtain detailed information about each running container object, so that you can manually modify or delete some required configurations during testing to simulate failures.

[0111] (2) Stop, start, and delete server nodes in the cluster

[0112] (3) Kill a container on the specified node.

[0113] (4) Find the specified Pod, deploys, and namespace through the API Server.

[0114] (5) Provides an interactive mode to facilitate instant viewing of cluster changes after operations.

[0115] In the experiment, PowerfulSeal is used to randomly delete Pods in the cluster to simulate the random scheduling scenario of multiple Pods. The process of cluster performance testing is introduced below.

[0116] 3. Cluster optimization pressure measurement performance and analysis

[0117] When testing the cluster performance, Kube-Stresscheck is used as the basic test case. By adjusting the upper limit of container resources when the test case Pod is running, six test applications with different degrees of overload are simulated and run in the cluster in the form of multiple copies. After the new Pod is added, the Pod is always in a waiting state to simulate the situation where the cluster resource usage reaches the upper limit. Then use Powerseal to delete the specified number of Pods. Due to the existence of the replica controller, the cluster will recreate the Pod and schedule it, but based on the current resource usage, the Pod may not be scheduled to the node bound for the first time. The reason why PowerfulSeal is needed to delete the Pod is that when creating the Pod, different Pod copies started by the same deployment or daemonset configuration file may put the Pod in a continuous position in the scheduling queue. If resources allow, these Pods will be immediately bound to the specified node and start running. It is necessary to manually delete some copies of Pods of different qualities to simulate the random scheduling of Pods with different CPU and memory application ratios.

[0118] Use the following table to test the performance limit of the cluster using six sample applications with different resource application ratios:

[0119]

[0120] During the operation of daily computing tasks, due to the different intermediate computing operations, the usage of different resources by Pods will continue to change. This change may cause a task that uses more CPU resources to occupy a larger proportion of memory than the CPU at a certain moment during execution, which will affect the judgment of the computing task type. In order to facilitate the verification of the performance of the algorithm, the test cases used will always occupy a number of resources close to the upper limit of resource allocation to avoid the change of task nature caused by different resource occupancy rates in different computing stages, which will affect the experimental results. In this experiment, three types of applications are used as test cases, with two applications with equal CPU and memory usage, two applications with more CPU usage, and two applications with more memory usage. Different granularity levels are used in the experiment for the same type of applications to ensure the accuracy of the experimental results. Among them, 1000m represents that the Pod applies for the use of one core CPU, 1Gi represents that the Pod applies for the use of 1GB of memory space, and 1Mi represents the application for 1MB of memory space.

[0121] When setting the Pod running limit of the nodes in the cluster to 200 and deploying the Pod, set the number of copies of the sample application to 90 in the configuration file. When all applications start running, randomly delete about half of the sample application copies until new Pods can be rescheduled to disrupt the scheduling order as much as possible. In the original cluster, use Prometheus to obtain the CPU utilization and memory utilization of each node server, and get the CPU utilization and memory utilization of the node as follows Figure 5 ,Through calculation, the average remaining rate of CPU is 11.4%, and the remaining idle rate of memory is 10.8%.

[0122] After adding the prediction-based distributed scheduling algorithm proposed in the embodiment of the present invention, the cluster state is set to the full load state using the example Pod again, and the CPU and memory utilization of all nodes in the cluster are obtained as follows: Figure 6 , the average remaining rate of CPU is 5.1%, and the average remaining rate of memory is 4.9%. It can be seen that compared with the original scheduling strategy, the data after adding the scheduling algorithm in this article has a certain improvement on the upper limit of CPU and memory utilization, and the degree of CPU and memory overload has also been reduced, which allows the remaining resources to accommodate tasks with more demanding resource requirements.

[0123] In this embodiment, the average value of the difference between the CPU and memory utilization of each node is used to measure the degree of node load imbalance, and the variance of memory and CPU utilization is used to measure the degree of node load balancing. For the cluster before optimization, the average degree of CPU and memory load imbalance is 3.89%, the variance of node CPU utilization is 10.30, and the variance of memory utilization is 10.97. For the cluster after adding the algorithm proposed in the embodiment of the present invention, the average degree of load imbalance is 2.27%, the variance of node CPU utilization is 6.41, and the variance of memory utilization is 3.94. It can be seen that the optimized cluster resource usage is more balanced.

[0124] 4. Summary of experimental optimization effects

[0125] In the embodiment of the present invention, the traditional model is mainly improved and optimized for cluster data, and a custom scheduling scoring algorithm is implemented based on the model. The situation in which the Pod is always in an abnormal state due to the cluster overload state is optimized to automatically adjust the cluster overload state, thereby optimizing the performance upper limit of the cluster.

[0126] The technical solution provided by the embodiment of the present invention considers the actual resource usage of the server and the predicted results of the resource usage of the Pod when determining the target scheduling node, and takes the difference between CPU and memory utilization as one of the scoring criteria. Scheduling is divided into two stages: pre-selection and optimization. The scheduling strategy proposed in this article does not impose secondary restrictions on the range of available nodes. As long as the basic Pod operating conditions are met, the node will be added to the list of available nodes. However, in the optimization stage, those Pods that have run before can use the previously collected information for prediction, and guide the Scheduler to make Pod scheduling decisions based on the prediction results. The scheduling algorithm proposed in the embodiment of the present invention mainly considers keeping each node in the cluster as stable as possible, so as to reduce the server load state caused by the continuous scheduling of multiple tasks with different application ratios. After experimental verification, using the prediction-based distributed scheduling method provided by the embodiment of the present invention, the average CPU idle rate of the Kubernetes cluster was reduced by 6.3%, the average memory idle rate was reduced by 5.9%, the CPU utilization load variance was reduced by 3.90%, and the memory utilization load variance was reduced by 7.02%.

[0127] Embodiment 4

[0128] See also Figure 7 The present invention provides a prediction-based distributed scheduling device, which includes:

[0129] The acquisition module 210 is used to obtain the historical resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster;

[0130] A prediction module 220 is used to input the historical resource utilization of the currently scheduled Pod into a deep learning model for training and reasoning to obtain the predicted resource utilization of the currently scheduled Pod;

[0131] The determination module 230 is used to screen the available nodes in the cluster according to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster to determine the target scheduling node of the currently scheduled Pod.

[0132] The determination module 210 includes a pre-screening unit and a optimization unit, wherein the pre-screening unit is used to pre-screen the available nodes based on the built-in algorithm of Kubernetes to obtain candidate available nodes;

[0133] The optimization unit is used to screen the candidate available nodes according to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the candidate available nodes to determine the target scheduling node of the currently scheduled Pod.

[0134] Furthermore, the optimization unit is specifically used to: determine a first score of each candidate available node according to the resource utilization rate of each candidate available node;

[0135] If the currently scheduled Pod is running for the first time, the second score of each candidate available node is determined according to the resource utilization rate of each candidate available node and the resource application ratio of the currently scheduled Pod;

[0136] If the currently scheduled Pod is not running for the first time, a second score of each candidate available node is determined according to the resource utilization of each candidate available node and the predicted resource utilization of the currently scheduled Pod;

[0137] The candidate available nodes are sorted according to the first score and the second score to determine the target scheduling node for the currently scheduled Pod.

[0138] The calculation formula of the first score is:

[0139] Score1=10-(U nc +U nm )*5;

[0140] Among them, Score1 represents the first score, U nc Indicates the CPU utilization of the candidate available node, U nm Represents the memory utilization of candidate available nodes.

[0141] If the currently scheduled Pod is running for the first time, the calculation formula for the second score is as follows:

[0142] Score2-1=10-(U nc +R pc -(U nm +R pm ))*10;

[0143] Score2-1 indicates the second score when the currently scheduled Pod is running for the first time. nc and R pc Represents the CPU utilization of candidate available nodes and the CPU application ratio of the currently scheduled Pod, U nm and R pm They represent the memory utilization of the candidate available nodes and the memory application ratio of the currently scheduled Pod respectively.

[0144] If the current schedule is not the first run, the calculation formula for the second score is as follows:

[0145] Score2-2=10-(U nc +U pcp -(U nm +U pmp ))*10;

[0146] Among them, Score2-2 is the second score when the current scheduling is not the first operation, U nc and R pcp Respectively represent the CPU utilization of the candidate available nodes and the CPU predicted utilization of the currently scheduled Pod; U nm and R pmp They represent the memory utilization of candidate available nodes and the predicted memory utilization of the currently scheduled Pod, respectively.

[0147] Specifically, sorting the candidate available nodes according to the first score and the second score to determine the target scheduling node of the currently scheduled Pod includes:

[0148] The sum of the first score and the second score is used as the target score of each candidate available node;

[0149] The candidate available node with the highest target score is used as the target scheduling node of the currently scheduled Pod.

[0150] Furthermore, the acquisition module specifically includes:

[0151] By deploying Prometheus and Grafana in the cluster, the cluster resource monitoring strategy can obtain the historical resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster.

[0152] The prediction-based distributed scheduling device provided in the embodiment of the present invention can execute the prediction-based distributed scheduling method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method, which will not be described in detail.

[0153] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A prediction-based distributed scheduling method, characterized in that: Applied in Kubernetes cluster, including: Get the historical resource utilization of the currently scheduled Pod and the resource utilization of available nodes in the cluster; Inputting the historical resource utilization of the currently scheduled Pod into a deep learning model for training and reasoning to obtain the predicted resource utilization of the currently scheduled Pod; According to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster, the available nodes in the cluster are screened to determine the target scheduling node of the currently scheduled Pod; According to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster, the available nodes in the cluster are screened to determine the target scheduling node of the currently scheduled Pod, including: Pre-screening the available nodes based on a built-in algorithm of Kubernetes to obtain candidate available nodes; According to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the candidate available nodes, the candidate available nodes are screened to determine the target scheduling node of the currently scheduled Pod; According to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the candidate available nodes, the candidate available nodes are screened to determine the target scheduling node of the currently scheduled Pod, including: Determine a first score of each candidate available node according to the resource utilization rate of each candidate available node; If the currently scheduled Pod is running for the first time, the second score of each candidate available node is determined according to the resource utilization rate of each candidate available node and the resource application ratio of the currently scheduled Pod; If the currently scheduled Pod is not running for the first time, a second score of each candidate available node is determined according to the resource utilization of each candidate available node and the predicted resource utilization of the currently scheduled Pod; The candidate available nodes are sorted according to the first score and the second score to determine the target scheduling node for the currently scheduled Pod.

2. The method according to claim 1, characterized in that The calculation formula of the first score is: Score1=10-(U nc +U nm )*5; Among them, Score1 represents the first score, U nc Indicates the CPU utilization of the candidate available node, U nm Represents the memory utilization of candidate available nodes.

3. The method according to claim 1, characterized in that If the currently scheduled Pod is running for the first time, the calculation formula for the second score is as follows: Score2-1=10-(U nc +R pc -(U nm +R pm ))*10; Score2-1 indicates the second score when the currently scheduled Pod is running for the first time. nc and R pc They represent the CPU utilization of the candidate available nodes and the CPU application ratio of the currently scheduled Pod, U nm and R pm They represent the memory utilization of the candidate available nodes and the memory application ratio of the currently scheduled Pod respectively.

4. The method according to claim 1, characterized in that: If the current schedule is not the first run, the calculation formula for the second score is as follows: Score2-2=10-(U nc +U pcp -(U nm +U pmp ))*10; Among them, Score2-2 is the second score when the current scheduling is not the first operation, U nc and R pcp Respectively represent the CPU utilization of the candidate available nodes and the CPU predicted utilization of the currently scheduled Pod; U nm and R pmp They represent the memory utilization of candidate available nodes and the predicted memory utilization of the currently scheduled Pod, respectively.

5. The method according to claim 1, characterized in that Sorting the candidate available nodes according to the first score and the second score to determine the target scheduling node for the currently scheduled Pod includes: The sum of the first score and the second score is used as the target score of each candidate available node; The candidate available node with the highest target score is used as the target scheduling node of the currently scheduled Pod.

6. The method according to claim 1, characterized in that Get the historical resource utilization of the currently scheduled Pod and the resource utilization of available nodes in the cluster, including: By deploying Prometheus and Grafana in the cluster, the cluster resource monitoring strategy can obtain the historical resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster.

7. A prediction-based distributed scheduling device, characterized in that: Configured in the Kubernetes cluster, including: The acquisition module is used to obtain the historical resource utilization of the currently scheduled Pod and the resource utilization of available nodes in the cluster; A prediction module, used to input the historical resource utilization of the currently scheduled Pod into a deep learning model for training and reasoning to obtain the predicted resource utilization of the currently scheduled Pod; A determination module, used to screen the available nodes in the cluster according to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the available nodes in the cluster to determine the target scheduling node of the currently scheduled Pod; Determine the module specifically for: Pre-screening the available nodes based on a built-in algorithm of Kubernetes to obtain candidate available nodes; According to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the candidate available nodes, the candidate available nodes are screened to determine the target scheduling node of the currently scheduled Pod; According to the predicted resource utilization of the currently scheduled Pod and the resource utilization of the candidate available nodes, the candidate available nodes are screened to determine the target scheduling node of the currently scheduled Pod, including: Determine a first score of each candidate available node according to the resource utilization rate of each candidate available node; If the currently scheduled Pod is running for the first time, the second score of each candidate available node is determined according to the resource utilization rate of each candidate available node and the resource application ratio of the currently scheduled Pod; If the currently scheduled Pod is not running for the first time, a second score of each candidate available node is determined according to the resource utilization of each candidate available node and the predicted resource utilization of the currently scheduled Pod; The candidate available nodes are sorted according to the first score and the second score to determine the target scheduling node for the currently scheduled Pod.

Images