Multi-dimensional auto-scaling of container-based clusters

Abstract

The invention relates to multi-dimensional automatic scaling based on clustering of containers. The present disclosure provides a method for determining a target configuration for a container-based cluster. The method generally includes determining, by a virtualization management platform configured to manage components of the cluster, a current state of the cluster; determining, by the virtualization management platform, at least one of a performance metric or a resource utilization metric for the cluster based on the current state of the cluster; processing the current state and at least one of the performance metric or the resource utilization metric with a model configured to generate candidate configurations for the cluster recommendations and thereby generate the candidate configurations; calculating a reward score for each of the candidate configurations; selecting the target configuration as a candidate configuration from the candidate configuration based on the reward score of the target configuration; and adjusting a configuration setting of the cluster based on the target configuration to change the current state of the cluster.

Description

Technical Field

[0001] The present disclosure relates to multi-dimensional autoscaling of container-based clusters. Background Art

[0002] Modern applications are applications designed to take advantage of the benefits of modern computing platforms and infrastructure. For example, modern applications can be deployed in a multi-cloud or hybrid cloud manner. Multi-cloud applications can be deployed across multiple clouds, which can be multiple public clouds provided by different cloud providers or the same cloud provider, or a mix of public and private clouds. The term "private cloud" refers to one or more local data centers that can have pooled resources allocated in a cloud-like manner. Hybrid cloud specifically refers to a combination of public and private clouds. Therefore, applications deployed across a hybrid cloud environment consume both cloud services executed in a public cloud and local services executed in a private data center (e.g., a private cloud). Within a public cloud or private data center, modern applications can be deployed to one or more virtual machines (VMs), containers, application services, and / or the like.

[0003] A container is a package that relies on virtual isolation to deploy and run applications that depend on a shared operating system (OS) core. A containerized application (also called a "containerized workload") may include a collection of one or more related applications encapsulated into one or more containers. In some orchestration systems, a group of one or more related containers (called a pod) that share storage and network resources is deployed as a unit of computing software. The container orchestration system automates the lifecycle of containers, including operations such as provisioning, deployment, monitoring, scaling (up and / or down), networking, and load balancing.

[0004] The software is an example open source container orchestration platform that automates the deployment and operation of such containerized applications. At a high level, the Kubernetes platform consists of a central database containing Kubernetes objects or persistent entities managed in the platform. Kubernetes objects are represented in configuration files such as JavaScript Object Notation (JSON) or YAML files, and describe the intended state of a Kubernetes cluster of interconnected nodes for running containerized applications. A node can be a physical machine or a VM configured to run on a physical machine running a monitor. The intended state of the cluster includes the intended infrastructure (e.g., container groups, containers, etc.) and containerized applications to be deployed in the cluster. In other words, Kubernetes objects are "records of intent," and once an object is created, the Kubernetes system will constantly work to ensure that the object is implemented in the deployment.

[0005] There are two object categories in Kubernetes: native Kubernetes objects and custom resource definition (CRD) objects (also referred to herein as "custom resources"). Native Kubernetes objects include container groups, services, storage volumes, namespaces, deployments, replication controllers, replica sets (ReplicaSets), and / or the like that are supported and can be created / manipulated by the Kubernetes application programming interface (API). The Kubernetes API is a resource-based (e.g., RESTful or representational state transfer architectural style) programming interface provided via HTTP. On the other hand, CRD objects are objects that extend the Kubernetes API or allow users to introduce their own APIs into a Kubernetes cluster.

[0006] Kubernetes is designed to accommodate any number and / or type of configurations, as long as certain limits are not exceeded (e.g., not exceeding a first threshold number of container groups per node, not exceeding a second threshold total number of container groups, not exceeding a third threshold total number of nodes, etc.). However, deriving an "optimal" configuration for deploying containerized applications is a technically challenging problem. In particular, an "optimal" configuration is one that balances cost, resource consumption, and application performance / availability while keeping platform operations simple. Understanding the number of container groups implemented in a container-based cluster, the number of nodes, node size and capacity, the number of replica sets (e.g., the number of container group replicas), container group resource requirements, container group resource limits, etc., so that these variables are optimized may not be obvious or easy to determine. In fact, determining the "optimal" configuration often requires a trial-and-error process that results in (1) many terminated / restarted container groups in under-allocated situations and (2) resource waste in over-allocated situations. In addition, as application requests change over time, such configurations may need to be constantly updated.

[0007] Some conventional methods for determining the "optimal" configuration for a container-based cluster are more empirical in nature. For example, one conventional method relies on the knowledge of a professional to identify the deployment of a cluster configuration. The expert may need to know the workload patterns, application implementation schemes, and understand infrastructure requirements and limitations when deriving the configuration. Because this is an inherently subjective method, it is not repeatable or scalable. Therefore, conventional manual methods for determining cluster configurations that attempt to optimize cost, resource consumption, application performance, and operational complexity may be ineffective.

[0008] Furthermore, in order to take into account the number of deployed applications and / or the changes in resource requests of different applications over time, conventional approaches have developed several tools, including horizontal container group autoscalers (HPA), vertical container group autoscalers (VPA), and cluster autoscalers (CA). These autoscalers are designed to help guarantee availability in Kubernetes by providing automatic scalability of application resources to adapt to changing loads.

[0009] HPA is a tool designed to automatically update workload resources, such as deployments and replica sets (e.g., designed to manage the deployment and scaling of a set of container groups), so that they scale to match the needs of applications in a container-based cluster. Horizontal scaling refers to the process of deploying additional container groups in a cluster in response to an increase in load and / or removing container groups in a container-based cluster in response to a decrease in load. In some cases, HPA is designed to automatically increase and / or decrease the number of container group replicas in a cluster based on actual utilization metrics, such as central processing unit (CPU) and / or memory utilization. In certain embodiments, HPA is implemented as a control loop to scale container group replicas based on a ratio between a desired metric value and a current metric value. The selection of the desired utilization metric value is such that a tradeoff between application availability and operating cost is enforced.

[0010] The VPA is a tool designed to automatically adjust resource limits and / or resource requests (e.g., with respect to CPU and / or memory) to help ensure that a container group always operates efficiently. The VPA determines the adjustments by analyzing the historical memory and / or CPU usage and the current memory and / or CPU usage of the containers running in the container group. In some embodiments, the VPA provides recommended values ​​for resource requests and / or limits that a user can use to manually update the configuration. In some embodiments, the VPA automatically updates the configuration based on these recommended values.

[0011] CA is a stand-alone program that expands or shrinks the number of nodes in a container-based cluster to help ensure that there are enough nodes to run all requested container groups and remove excess nodes from the cluster. For example, CA is designed to add nodes to the cluster when there is a waiting container group that cannot be scheduled on any of the existing nodes due to insufficient resources between the existing nodes. In addition, CA is designed to reduce the number of nodes in the cluster when at least one node is not needed all the time and when the container group running on the node can be transferred to a different node for execution. In this way, CA is designed to act in a reactive manner by adding and / or removing nodes based on the addition and / or removal of new container groups. In some cases, this causes the new container group added to the cluster to be in a waiting state until additional nodes are provisioned, thus affecting application performance. In some CA implementations, CA is deceived by reserving space to add additional nodes prematurely to deploy the suspended container group to improve this problem. However, this introduces additional operational complexity and cost into the system.

[0012] In other words, the autoscalers described above autoscalers automate the process of scaling applications to adapt to workload changes. However, the autoscalers themselves need to be configured and tuned over time, thus introducing significant setup and maintenance overhead. Furthermore, these autoscaler tools may not work well together (if not at all), and as such, may increase the complexity of setup, tuning, and maintenance of these autoscalers when deployed. For example, currently, HPA and VPA are not designed to be used on the same resource at the same time because they each attempt to optimize the resource in different ways.

[0013] It should be noted that the information contained in the background technology section of this document is only intended to provide a reference for the discussion of certain embodiments in the detailed description. All information contained in this background technology should not be regarded as an admission of prior art. Summary of the invention

[0014] One or more embodiments provide a method for determining a target configuration for a container-based cluster. The method generally includes determining, by a virtualization management platform configured to manage components of the container-based cluster, a current state of the container-based cluster. The method generally includes determining, by the virtualization management platform, at least one of a performance metric or a resource utilization metric of the container-based cluster based on the current state of the container-based cluster. The method generally includes processing the current state and at least one of the performance metric or the resource utilization metric using a model configured to generate a plurality of candidate configurations recommended for the container-based cluster and thereby generating the plurality of candidate configurations. The method generally includes calculating a reward score for each of the plurality of candidate configurations. The method generally includes selecting the target configuration as a candidate configuration from the plurality of candidate configurations based on the reward score of the target configuration. The method generally includes adjusting configuration settings of the container-based cluster based on the target configuration to change the current state of the container-based cluster.

[0015] Other embodiments include a non-transitory computer-readable storage medium comprising instructions that cause a computer system to perform the above method, and a computer system configured to perform the above method. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1A The diagram illustrates a computing system in which the embodiments described herein may be implemented.

[0017] Figure 1B The present invention is a method for implementing a method for Figure 1A A container-based cluster of computing systems that runs instances of containerized applications.

[0018] Figure 2 An example system configured to perform multi-dimensional autoscaling of container-based clusters is illustrated according to an example embodiment of the present disclosure.

[0019] Figure 3 An example method for determining a target configuration for a container-based cluster is illustrated according to an example embodiment of the present disclosure.

[0020] Figure 4 An example workflow for creating multiple training data instances to train a model to produce candidate configurations for a container-based cluster is illustrated according to an example embodiment of the present disclosure.

[0021] To facilitate understanding, identical reference numerals have been used, where possible, to designate identical elements that are common to the figures. It is contemplated that elements disclosed in one embodiment may be beneficially utilized on other embodiments without specific recitation. DETAILED DESCRIPTION

[0022] Techniques for performing multi-dimensional autoscaling in container-based clusters are described herein. Multi-dimensional autoscaling involves dynamically scaling up and / or down container groups (e.g., the number of container groups and / or resources allocated to container groups) and / or nodes (e.g., the number of nodes) based on workload changes in the container-based cluster.

[0023] In particular, the embodiments described herein utilize machine learning models to generate and evaluate different candidate configurations for a container-based cluster for the purpose of determining a target configuration for the cluster that provides improved workload performance and availability relative to other candidate configurations while also limiting cost, resource consumption, and overall complexity. In some embodiments, the identified target configuration is recommended to a user to prompt the user to manually update the configuration of the cluster. In some other embodiments, the identified target configuration is used to automatically adjust the configuration settings of the cluster. Adjusting the configuration settings of the cluster using the target configuration includes at least expanding and / or reducing the number of container groups, the amount of resources allocated to the container groups, and / or the number of nodes in the cluster.

[0024] The candidate configurations described herein contain values ​​for various configuration settings, including at least values ​​specifying the number of container groups, resource requests and / or limits assigned to each of the container groups, and the number of nodes to be deployed in the cluster. Each of the candidate configurations is generated by a machine learning model based on the current state, resource utilization metrics, and / or performance metrics collected for the cluster. The machine learning model is then configured to predict future application performance and resource utilization for each of the candidate configurations. A target configuration is selected among the candidate configurations based on the model predictions and, in some cases, an optimization goal identified by a user. The optimization goal may specify which factors are to be prioritized relative to other factors when selecting a target configuration. For example, an optimization goal may specify selecting a candidate configuration that prioritizes application performance and availability over cost optimization.

[0025] In addition, certain embodiments described herein provide a method for training a machine learning model to generate and evaluate different candidate configurations so that a target configuration for a cluster can be determined. The method includes obtaining a plurality of training data instances, and using the training data instances to train a model. The training data instances are obtained by collecting performance metrics and resource utilization metrics for different configurations and simulated loads applied to the cluster. Thus, each training data instance used to train the machine learning model includes: (1) training inputs, which include configuration information and information about simulated loads applied to the cluster; and (2) training outputs, which include performance metrics and resource utilization metrics collected for the corresponding configuration and load.

[0026] The techniques described herein for performing multi-dimensional autoscaling in container-based clusters using machine learning models provide significant technical advantages over conventional solutions, such as improved configuration optimization with respect to multiple aspects of a cluster. In addition, using a machine learning model in determining a target configuration for a cluster helps improve the efficiency of determining a configuration (e.g., an "optimal" configuration) for the cluster and application performance when the target configuration is deployed for the cluster. For example, a target configuration applied to a cluster may define resource requests and / or limits for applications running in the cluster such that the applications are neither under-provisioned nor over-provisioned, which typically results in poor application performance due to a lack of available resources or a negative impact on cluster capacity and overall cost, respectively.

[0027] Figure 1A is a block diagram illustrating a computing system 100 in which embodiments described herein may be implemented. The computing system 100 includes one or more hosts 102, a management network 180, a data network 170, and a virtualization management platform 140.

[0028] The hosts 102 may be geographically co-located servers on the same rack or on different racks in any arbitrary location in a data center. The hosts 102 may be located in a single host cluster or logically divided into multiple host clusters. Each host 102 may be configured to provide an abstraction of the processor, memory, storage, and networking resources of the hardware platform 108 of each host 102 into multiple VMs 1041-1044 running concurrently on the same host 102. N The virtualization layer (collectively referred to as VM 104 and individually referred to as VM 104) is also referred to as monitor 106.

[0029] The hosts 102 may be constructed on a server-class hardware platform 108, such as an x86 architecture platform. The hardware platform 108 of each host 102 includes components of a computing device, such as one or more processors (central processing units (CPUs)) 116, memory (random access memory (RAM)) 118, one or more network interfaces (e.g., physical network interface (PNIC) 120), storage device 122, and other components (not shown). The CPU 116 is configured to execute instructions that may be stored in the memory 118 and optionally in the storage device 122. The network interface enables the host 102 to communicate with other devices via a physical network, such as a management network 180 and a data network 170.

[0030] In some embodiments, monitor 106 runs in conjunction with an operating system (OS) (not shown) in host 102. In some embodiments, monitor 106 may be installed as system-level software directly on hardware platform 108 of host 102 (often referred to as a "bare metal" installation) and conceptually inserted between the physical hardware and the guest OS executing in VM 104. It should be noted that as used herein, the term "operating system" may refer to a monitor. One example of a monitor 106 that may be configured and used in the embodiments described herein is VMware 104, which is commercially available through VMware, Inc. of Palo Alto, CA. Part of the solution provided by VMware ESXi TM Monitor.

[0031] Each of the VMs 104 implements a virtual hardware platform that supports the installation of a guest OS 134 capable of executing one or more application programs 132. The guest OS 134 may be a standard commodity operating system. Examples of guest OSs include Microsoft Windows, Linux, and / or the like. The application programs 132 may be any software program, such as a word processor.

[0032] In certain embodiments, the computing system 100 includes a container orchestrator. The container orchestrator implements a container orchestration control plane (also referred to herein as a "control plane 142") (e.g., a Kubernetes control plane) to deploy and manage applications 132 and / or their services on a host 102 using containers 130. In particular, each VM 104 includes a container engine 136 installed therein and running as a guest application under the control of a guest OS 134. The container engine 136 is a process that implements the deployment and management of virtual instances (referred to herein as "containers") in conjunction with OS-level virtualization and a container orchestrator on a guest OS 134 within a VM 104. The container 130 provides isolation for user space processes executed therein. The container 130 encapsulates an application (and its associated application 132) into a single executable software package that bundles the application code with all the relevant configuration files, libraries, and dependencies required for its operation.

[0033] The control plane 142 runs on a cluster of hosts 102 and can deploy containerized applications 132 as containers 130 on the cluster of hosts 102. The control plane 142 manages compute, storage, and memory resources to enable the containers 130 to run in the cluster of hosts. In some embodiments, the monitor 106 is integrated with the control plane 142 to provide a "supervisor cluster" (i.e., a management cluster) that uses VMs 104 to implement both control plane nodes and compute objects managed by the Kubernetes control plane.

[0034] In certain embodiments, the control plane 142 deploys and manages applications as container groups of containers 130 that run on the host 102, either within a VM 104 or directly on the OS of the host 102. A container group is a group of one or more containers 130 and a specification of how to run the containers 130. A container group may be the smallest deployable computing unit that can be created and managed by the control plane 142.

[0035] Figure 1B The diagram in the figure shows a container-based cluster of instances used to run containerized applications. Figure 1B The example container-based cluster shown in FIG. 1 is a Kubernetes cluster 150, but in other examples, the container-based cluster may be another type of container-based cluster based on container technology, such as a Docker Swarm cluster. Figure 1B As illustrated in FIG, a Kubernetes cluster 150 is formed by a cluster of interconnected nodes, the interconnected nodes including: (1) one or more worker nodes 172 running one or more container groups 152 having containers 130; and (2) one or more control plane nodes 174 having control plane components running thereon that control the cluster (e.g., where a node is a physical machine such as a host 102 or a VM 104 configured to run on a host 102).

[0036] Each worker node 172 includes a kubelet 175. Kubelet 175 is an agent that helps ensure that one or more container groups 152 run on each worker node 172 according to the defined state of the container group 152 (e.g., defined in a configuration file). Each container group 152 may include one or more containers 130. Worker nodes 172 may be used to execute various applications 132 and software processes using containers 130. In addition, each worker node 172 may include a kube agent ( Figure 1B kube-proxy is a network proxy for maintaining network rules. These network rules allow network communication with container groups 152 from network sessions inside and / or outside the Kubernetes cluster 150.

[0037] The control plane 142 (e.g., running on one or more control plane nodes 174) includes components such as an application programming interface (API) server 162, a controller 164, a cluster storage device (etcd) 166, and a scheduler 168. The components of the control plane 142 make global decisions about the Kubernetes cluster 150 (e.g., scheduling) and detect and respond to cluster events.

[0038] The API server 162 operates as a gateway for the Kubernetes cluster 150. Thus, command line interfaces, web user interfaces, users and / or services communicate with the Kubernetes cluster 150 through the API server 162. One instance of the Kubernetes API server 162 is the kube-apiserver. The kube-apiserver is designed to scale horizontally, i.e., this component scales by deploying more instances. Several instances of the kube-apiserver can be run, and traffic between those instances can be balanced.

[0039] The controller 164 is responsible for running and managing the controller processes in the Kubernetes cluster 150. As described above, the control plane 142 may have (e.g., four) control loops called controller processes that monitor the state of the Kubernetes cluster 150 and attempt to modify the current state of the Kubernetes cluster 150 to match the intended state of the Kubernetes cluster 150.

[0040] Scheduler 168 is configured to assign new container group 152 to worker node 172 .

[0041] The cluster storage device (etcd) 166 is a data storage device used as a backing storage device for Kubernetes cluster 150 data, such as a consistent and highly available key value storage device. In some embodiments, the cluster storage device (etcd) 166 stores a configuration file 182, such as a JavaScript Object Notation (JSON) or YAML file, which is composed of one or more manifests declaring the established system infrastructure and workloads to be deployed in the Kubernetes cluster 150. Kubernetes objects or persistent entities can be created, updated, and deleted based on the configuration file 182 to represent the state of the Kubernetes cluster 150.

[0042] Kubernetes objects are "records of intent" and once an object is created, the Kubernetes system will continually work to ensure that the object is implemented in the deployment. One type of Kubernetes object is a custom resource definition (CRD) object (also referred to herein as a "custom resource (CR) 184") that enables the API server 162 to extend or allow users to introduce their own APIs into the Kubernetes cluster 150. In particular, Kubernetes provides a standard extension mechanism called custom resource definitions that enables the set of resources and objects that can be managed in a Kubernetes cluster to be extended.

[0043] As described above, Kubernetes is designed to accommodate any number and / or types of configurations (e.g., identified in configuration files 182), as long as certain limits are not exceeded. To determine a target configuration for a cluster that provides improved workload performance and availability while also limiting cost, resource consumption, and / or overall complexity, embodiments described herein utilize machine learning models. One or more machine learning models: (1) are trained to generate multiple candidate configurations for a cluster, (2) predict future application performance and resource utilization for each of the candidate configurations, and (3) select a target configuration among the candidate configurations based on the model predictions and, in some cases, an optimization goal identified by a user. Figure 2 and 3 The diagram in this article illustrates the use of a machine learning model to determine the target configuration for a cluster.

[0044] Specifically, Figure 2 An example system 200 configured to perform multi-dimensional automatic scaling of a container-based cluster according to an example embodiment of the present disclosure is illustrated. As illustrated, the system 200 includes a Kubernetes cluster 150 (e.g., Figure 1B Detailed description), virtualization management platform 140 and configuration prediction model 212. In some embodiments, Kubernetes cluster 150 includes automatic configuration update process controller 218. System 200 is configured to generate and evaluate candidate configurations 214 to determine target configuration 216 of Kubernetes cluster 150. In this example, the determined target configuration 216 is provided to Kubernetes cluster 150 to define a new intended state of Kubernetes cluster 150. The new intended state defined for Kubernetes cluster 150 can help improve the performance and availability of containerized applications 132 running in cluster 150.

[0045] Figure 3An example method 300 for determining a target configuration for a container-based cluster according to an example embodiment of the present disclosure is illustrated. The method 300 may be performed by the system 200 to determine the target configuration 216 of the Kubernetes cluster 150. As such, the following jointly describes Figure 2 and 3 .

[0046] like Figure 3 As illustrated in FIG. 3 , method 300 begins at operation 302, where a user interacts with a control plane of a container-based cluster to provide a configuration of the container-based cluster. The configuration defines a given state of the cluster. For example, in Figure 2 In the configuration file 190, a user may interact with the control plane node 174 to provide a configuration file 190. The configuration file 190 provided by the user consists of one or more manifests declaring a given system infrastructure and workloads to be deployed in the Kubernetes cluster 150. The configuration file 190 includes initial configuration settings for the cluster 150 that specify the number of container groups implemented in the Kubernetes cluster 150, the number of nodes, the node size and capacity, the number of replica sets (e.g., the number of container group replicas), container group resource requirements, container group resource limits, etc. As described in detail below, the initial configuration defined by the user in the configuration file 190 may be adjusted based on a target configuration 216 determined for the cluster 150 by the configuration prediction model 212.

[0047] Method 300 continues at operation 304, where the control plane modifies the current state of the container-based cluster to match the intended state of the cluster. Figure 2 , the controller 164 in the control plane node 174 works to modify the current state of the Kubernetes cluster 150 to match the intended state of the Kubernetes cluster 150 specified in the configuration file 190. In some embodiments, modifying the current state of the Kubernetes cluster 150 to match the intended state includes deploying the application 132 in the container 130 on the worker node 172.

[0048] The method 300 continues at operation 306, where a virtualization management platform (e.g., a component configured to manage a container-based cluster) determines performance metrics of containerized applications deployed and running in the container-based cluster. In addition, at operations 308 and 310, the virtualization management platform additionally collects resource utilization metrics of the container-based cluster and information about the current state of the container-based cluster, respectively.

[0049] For example, in Figure 2, at operations 306 , 308 , and 310 , the virtualization management platform 140 collects performance metrics 206 , resource utilization metrics 204 , and information about the current state 202 of the Kubernetes cluster 150 .

[0050] The performance metrics 206 may include performance information collected for each application 132 running in the Kubernetes cluster 150. The performance information may provide insight into the CPU usage of each application 132 (e.g., compared to a CPU limit assigned to the corresponding application 132 or a container group 152 running the corresponding application 132) and / or the memory usage of each application 132 (e.g., compared to a memory limit assigned to the corresponding application 132 or a container group 152 running the corresponding application 132). The performance information may also include details about application availability (e.g., service uptime), wait time, average response time (e.g., read and / or write), input / output (I / O) rate, user satisfaction, request error rate, and / or the like for each application 132 running in the Kubernetes cluster 150.

[0051] Resource utilization metrics 204 may include information about total CPU utilization and / or total memory utilization. Total CPU utilization is the average utilization percentage of CPU allocated to a particular component (e.g., container group 152, worker node 172, etc.) in a Kubernetes cluster 150 over an interval. Total memory utilization is the average utilization percentage of memory allocated to a particular component in a Kubernetes cluster 150 over an interval. Resource utilization metrics 204 may provide insight into whether resources for different components in a Kubernetes cluster 150 are adequate, under-provisioned, and / or over-provisioned.

[0052] As described above, the current state 202 of the Kubernetes cluster 150 may match the intended state declared for the cluster in the configuration file 190. The current state 202 determined by the virtualization management platform 140 may include information about the number of container groups deployed, the number of nodes deployed, resources allocated to different components, and / or other information about the infrastructure and / or applications 132 deployed in the Kubernetes cluster 150.

[0053] The method 300 continues at operation 312, where the performance metrics, resource utilization metrics, and current state of the container-based cluster are processed via a model configured to generate a plurality of candidate configurations recommended for the container-based cluster (e.g., the configuration prediction model 212). The model for processing these inputs can generate a plurality of candidate configurations for the cluster.

[0054] In some embodiments, the model is a digital twin model. The digital twin model builds a virtual model of the cluster based on the real-time current state, resource utilization metrics, and performance metrics collected for the modeled container-based cluster. Once the virtual model is informed of this data, the virtual model is used to run simulations, learn performance issues, and / or generate possible improvements to the configuration settings of the cluster (e.g., generate candidate configurations). In other words, the virtual representation created for the cluster is used to simulate the behavior of the cluster and predict application and cluster performance.

[0055] In certain embodiments, the digital twin model designed to generate candidate configurations for container-based cluster recommendations is a digital twin model based on a neural network. A neural network typically includes a plurality of connected units or nodes called artificial neurons. Each node typically has one or more inputs with associated weights, a net input function, and an activation function. The nodes are typically included in a plurality of connected layers, where the nodes of one layer are connected to the nodes of another layer, where various parameters govern the relationship between the nodes and the layers and the operation of the neural network. The layers may include an input layer, one or more hidden layers, and an output layer.

[0056] Model training components ( Figure 2 (not illustrated in the figure) is typically configured to train a model to generate recommended candidate configurations for different clusters. In some embodiments, the model training component uses a load generator ( Figure 2 The model is trained using training data from multiple training data instances analyzed (not illustrated in the figure), as described below with respect to Figure 4 Describe in detail.

[0057] Thus, at operation 312, Figure 2 , the virtualization management platform 140 provides the current state 202, resource utilization metrics 204, and performance metrics 206 as inputs into the configuration prediction model 212. The configuration prediction model 212 can be deployed and run on any node (e.g., VM), container group 152, and / or container 130 in the Kubernetes cluster 150. The configuration prediction model 212 uses this input information to generate a plurality of candidate configurations 214 for the Kubernetes cluster 150.

[0058] In this example implementation, the configuration prediction model 212 is a digital twin model based on a neural network. More specifically, the configuration prediction model 212 is a 3-layer neural network digital twin model with hidden layer sizes of 64, 32, and 64.

[0059] The configuration prediction model 212 is a model that is trained offline using various training techniques (e.g., before being deployed to predict candidate configurations 214 for the Kubernetes cluster 150). In some embodiments, the configuration prediction model 212 is trained using mean squared error (MSE). The mean squared error is calculated as the average of all squared differences between the true value and the predicted value generated by the configuration prediction model 212. A smaller MSE indicates that the predictions made by the model are more accurate.

[0060] In addition, the configuration prediction model 212 is validated using one or more validation techniques. In certain embodiments, the configuration prediction model 212 is validated using cross-validation techniques and more specifically 10-fold cross-validation during model tuning and testing. Cross-validation, such as k-fold cross-validation (e.g., where k is a positive integer), describes the process of dividing a data set into k folds or subsets, using (k-1) folds as training data to train a model, and using the remaining folds to validate the model. Cross-validation techniques are used to test the ability of a model to predict new data, identify problems (e.g., overfitting and / or selection bias), and / or give insight into how the model will generalize to independent data sets.

[0061] The candidate configurations 214 generated by the configuration prediction model may indicate different intended states of the Kubernetes cluster 150. The candidate configurations 214 may include one or more similar configuration settings and / or one or more different configuration settings. For example, all candidate configurations 214 generated for the Kubernetes cluster may specify that the number of container groups to be deployed is equal to 100, while the number of nodes indicated to be deployed for running these 100 container groups varies across different candidate configurations 214.

[0062] The method 300 then continues at operation 314, where a reward score is calculated for each of the plurality of candidate configurations recommended for the container-based cluster. In certain embodiments, the model is configured to calculate a reward score for each candidate configuration at operation 314. The reward score is a numerical value that identifies how good or bad the predicted candidate configuration is for the cluster.

[0063] In some embodiments, the bonus score calculated for each candidate configuration is calculated based on a parameter function having application metrics, memory and / or CPU usage and requests, and node utilization. For example, the bonus score can be calculated using the following equation:

[0064] R=α·R 应用程序 +β·R 使用率 +γ·R 资源

[0065] in,

[0066] R 应用程序 =fa(S请求错误率 ,S 响应时间 ,μ 响应时间 )

[0067] And S 请求错误率 is the request error rate, S 响应时间 is the request response time, and μ 响应时间 is the average request response time.

[0068] also,

[0069]

[0070] Where S 存储器使用率 is the memory usage, S 存储器限制 is the memory limit, and S 存储器请求 is the memory request. Although in this example, the parameter R 使用率 is calculated based only on memory usage, limits, and requests; however, in addition to or instead of memory usage, limits, and requests, other examples may calculate parameter R based on CPU usage, limits, and requests 使用率 .

[0071] also,

[0072] R 资源 =fr(S 利用率 ,t 利用率 )

[0073] Where S 利用率 is the current node utilization and t 利用率 is the target node utilization.

[0074] In the above equation, parameters α, β, and γ are weights set to define the priority of application availability and responsiveness over resource consumption. For example, in some cases, application availability and responsiveness are prioritized over resource consumption; therefore, weight parameter α=0.8 and weight parameter β=γ=1.

[0075] In some embodiments, when calculating the reward score for each candidate configuration, the user specifies which variables will be given priority relative to other variables. In particular, the user can interact with the model to specify an optimization goal. When calculating the reward score for each candidate configuration, the model can use this optimization goal. In some cases, the optimization goal may request that a higher reward score be given to a candidate configuration 214 with better application performance and availability, and the weight parameters α, β, and γ may be adjusted accordingly to perform this goal. In some other cases, the optimization goal may request that a higher reward score be given to a candidate configuration 214 with better resource utilization or a higher reward score be given to a candidate configuration 214 with less complexity, and the weight parameters α, β, and γ may be adjusted accordingly to perform each of these goals.

[0076] exist Figure 2 In the example illustrated in , at operation 314, the configuration prediction model 212 calculates a reward score for each of the candidate configurations 214 generated and recommended for the Kubernetes cluster 150. The configuration prediction model 212 is configured to calculate a reward score for each of the candidate configurations 214 based on the goal 208 (e.g., optimization goal) provided by the user 210.

[0077] The method 300 then continues at operation 316, where the target configuration is determined to be the candidate configuration from the plurality of candidate configurations having the highest score among the plurality of reward scores calculated for the plurality of candidate configurations. Figure 2 , the candidate configuration 214 having the highest calculated reward score is determined as the target configuration 216 for the Kubernetes cluster 150.

[0078] The method 300 then continues at operation 318, where a control plane of the container-based cluster is interacted with to provide a target configuration for the container-based cluster. In some embodiments, the target configuration is provided to a user, and the user interacts with the control plane to provide the target configuration, or in other words, to update the configuration settings for the cluster defined in the configuration file to match the determined target configuration. In some other embodiments, the system is configured to automatically update the configuration settings of the cluster based on the determined target configuration. The target configuration defines a new intended state for the container-based cluster; therefore, when the configuration settings are adjusted to match the intended cluster state defined in the target configuration, the controller of the control plane operates to modify the current state of the container-based cluster to match the new intended state of the cluster.

[0079] For example, in Figure 2, at operation 318, the configuration settings for the Kubernetes cluster 150 defined in the configuration file 190 are adjusted to match the target configuration 216. In certain embodiments, the automatic configuration update process controller 218 is configured to make such adjustments (e.g., updates) to the configuration file 190 in the Kubernetes cluster 150. The automatic configuration process controller 218 may be a custom controller 164 deployed in the Kubernetes cluster 150 that is responsible for automating configuration setting updates. For example, the automatic configuration update process controller 218 may monitor a new target configuration 216 (e.g., determined from the output of the configuration prediction model 212). When a new target configuration 216 is detected, the automatic configuration update process 218 may determine whether the new target configuration 216 is different from an existing configuration defined in the configuration file 190 for the Kubernetes cluster 150. If the target configuration 216 is different from the existing configuration, the automatic configuration update process 218 may automatically update the configuration settings in the configuration file 190 to match the target configuration 216. Although Figure 2 The automatic configuration update process controller 218 is illustrated to perform configuration setting updates, but in certain other embodiments, any other process deployed for the Kubernetes cluster 150 may be configured to perform the updates.

[0080] The controller 164 then operates to update the infrastructure and / or workloads in the Kubernetes cluster 150 based on the adjustment. For example, assuming that the Kubernetes cluster 150 was previously deployed with 50 container groups 152 and the configuration setting is updated to indicate that the cluster is instead intended to have 75 container groups 152, the controller (2) 164 operates to enable the deployment of 25 additional container groups 152 in the Kubernetes cluster 150.

[0081] The method 300 can be continuously and / or periodically performed to adjust the configuration settings of the container-based cluster to take into account the number of deployed applications and / or changes in application resource requirements. In this way, the configuration settings of the cluster can be continuously updated to help provide optimal application performance and availability while also limiting the cost, resource consumption, and overall complexity of the cluster.

[0082] As described above, a model for generating candidate configurations and determining a target configuration for a container-based cluster is trained before deployment (e.g., Figure 2The configuration prediction model 212 in FIG. 212 is used to train the model based on training data generated and collected during load simulation and / or for real-world loads applied to the cluster (e.g., loads that naturally occur as a function of the cluster and are used by real-world users and / or other systems). In particular, during load simulation, various loads can be generated and simulated for different cluster configuration settings. The performance of the cluster can be observed for each of the different configuration settings and simulated loads. The performance of the cluster can also be observed for different configuration settings and real-world loads applied to the cluster. This information can then be used to create training data instances to train the model to generate candidate configurations.

[0083] For example, Figure 4 An example workflow 400 for creating multiple training data instances generated and collected during a load simulation in order to train a model to generate candidate configurations for a container-based cluster is illustrated according to an example embodiment of the present disclosure. Each of the training data instances generated according to the workflow 400 includes a training input and a training output (generally referred to as a label). The training input includes configuration information and information about the load simulated for the cluster. The training output includes performance metrics and / or resource utilization metrics observed for the cluster when the cluster is configured based on the corresponding configuration information included in the training input and when the corresponding load included in the training input is generated for the cluster.

[0084] Workflow 400 begins at operation 402, where a user interacts with a control plane of a container-based cluster to provide a configuration of the container-based cluster. The configuration defines a desired state of the cluster. For example, a user may interact with a control plane node to provide configuration settings that are desired for the cluster in one or more configuration files. At operation 404, a controller in the cluster modifies the current state of the container-based cluster to match the desired state of the cluster (e.g., initially specified by the user). Modifying the current state to match the desired state involves deploying applications and the infrastructure (e.g., nodes, container groups, containers) needed to run those applications in the cluster.

[0085] Workflow 400 continues at operation 406, where a load generator configured to generate a simulated load of a containerized application running in a container-based cluster is deployed. Since the load generator was not previously deployed, operation 406 occurs at this time. However, as described below, when returning to operation 406, the load generator may not need to be redeployed. The load generator can be deployed and configured to run on any node (e.g., any VM), container group, and / or container in the container-based cluster.

[0086] Workflow 400 continues at operation 408, where a load generator generates a simulated load of the containerized application. Different loads and / or the same load can be applied to different applications running in the cluster.

[0087] At operation 410, a virtualization management platform configured to manage components of a container-based cluster determines performance metrics of containerized applications deployed and running in the container-based cluster. In addition, at operation 412, the virtualization management platform additionally collects resource utilization metrics of the container-based cluster.

[0088] Workflow 400 then continues at operation 414 where a training data instance is created, the training data instance comprising: training inputs including configuration information and information about simulated loads; and training outputs including performance metrics and resource utilization metrics.

[0089] Workflow 400 continues at operation 416, where it is determined whether all desired loads for these particular cluster configuration settings (e.g., provided at operation 402) have been simulated. If not all desired loads have been simulated, then workflow 400 returns to operation 408 to generate a new simulated load of the containerized application and performs operations 410 to 414 to create another training data instance. Alternatively, if all desired loads have been simulated, then workflow 400 continues to operation 418 to determine whether all desired configurations of the container-based cluster have been deployed and tested (e.g., against different simulated loads). If all desired configurations have been deployed and tested, then workflow 400 returns to operation 402 to adjust the configuration settings of the cluster and performs operations 404 to 416 to create additional training data instances for the new configuration settings. Alternatively, if all desired configurations have been deployed and tested, then workflow 400 is complete. The training data instances generated according to workflow 400 are then used to train the model to generate candidate configurations for various container-based clusters.

[0090] It should be understood that for any process described herein, there may be additional or fewer steps performed in similar or alternative orders, or in parallel, within the scope of the various embodiments, consistent with the teachings herein unless otherwise stated.

[0091] The various embodiments described herein may employ various computer-implemented operations involving data stored in a computer system. For example, these quantities may be in the form of electrical or magnetic signals, wherein the signals or representations of the signals can be stored, transmitted, combined, compared, or otherwise manipulated. In addition, such manipulations are generally referred to in terms such as generation, identification, determination, or comparison. Any operation described herein that forms part of one or more embodiments of the present invention may be a useful machine operation. In addition, one or more embodiments of the present invention also relate to a device or apparatus for performing these operations. The apparatus may be constructed specifically for a specific desired purpose, or it may be a general-purpose computer that is selectively activated or configured by a computer program stored in the computer. In particular, various general-purpose machines may be used together with computer programs written according to the teachings herein, or it may be more convenient to construct more specialized equipment to perform the desired operations.

[0092] The various embodiments described herein may be practiced with other computer system configurations including hand-held devices, microprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframe computers, and the like.

[0093] One or more embodiments of the present invention may be implemented as one or more computer programs or one or more computer program modules embodied in one or more computer readable media. The term computer readable media refers to any data storage device that can store data that can be later input into a computer system - the computer readable media can be based on any existing or subsequently developed technology for embodying a computer program in a manner that enables the computer program to be read by a computer. Examples of computer readable media include hard drives, network attached storage devices (NAS), read-only memory, random access memory (e.g., flash memory devices), CDs (Compact Discs) - CD-ROMs, CD-Rs or CD-RWs, DVDs (Digital Versatile Discs), magnetic tapes, and other optical and non-optical data storage devices. Computer readable media can also be distributed on network-coupled computer systems so that computer readable code is stored and executed in a distributed manner.

[0094] Although one or more embodiments of the present invention have been described in some detail for the sake of clarity of understanding, it should be understood that certain changes and modifications may be made within the scope of the claims. Therefore, it is intended that the described embodiments be regarded as illustrative rather than restrictive, and it is not intended that the scope of the claims be limited to the details given herein, but may be modified within the scope and equivalents of the claims. In the claims, elements and / or steps do not imply any particular order of operation unless explicitly stated in the claims.

[0095] The virtualization system according to various embodiments may be implemented as a managed embodiment, an unmanaged embodiment, or an embodiment that tends to blur the distinction between the two, all of which are contemplated. In addition, various virtualization operations may be implemented in whole or in part in hardware. For example, a hardware implementation may employ a lookup table to modify storage device access requests to protect non-disk data.

[0096] Some embodiments as described above relate to a hardware abstraction layer on top of a host computer. The hardware abstraction layer allows multiple contexts to share hardware resources. In one embodiment, these contexts are isolated from each other, and at least one user application is running in each context. The hardware abstraction layer thus provides the benefits of resource isolation and allocation among contexts. In the aforementioned embodiments, a virtual machine is used as an instance of a context, and a monitor is used as an instance of a hardware abstraction layer. As described above, each virtual machine includes a guest operating system in which at least one application is running. It should be noted that these embodiments can also be applied to other instances of contexts, such as containers that do not include a guest operating system, referred to herein as "OS-free containers" (see, for example, www.docker.com). OS-free containers implement operating system-level virtualization, in which an abstraction layer is set on top of an operating system kernel on a host computer. The abstraction layer supports multiple OS-free containers, each of which includes an application and its dependencies. Each OS-free container runs as an isolated process in the user space on the host operating system and shares the kernel with other containers. OS-free containers rely on the functionality of the kernel to utilize resource isolation (CPU, memory, block I / O, network, etc.) and separate namespaces and completely isolate the operating environment view of the application. By using OS-less containers, resources can be isolated, services can be limited, and processes can be provisioned with a private view of the operating system and its own process ID space, file system structure, and network interface. Multiple containers can share the same core, but each container can be constrained to use only a defined amount of resources, such as CPU, memory, and I / O. As used herein, the term "virtualized computing instance" is intended to cover both VMs and OS-less containers.

[0097] Regardless of the degree of virtualization, many variations, modifications, additions and improvements are possible. Virtualization software may therefore include components of a host, console or guest operating system that performs virtualization functions. Multiple instances may be provided as a single instance for the components, operations or structures described herein. The boundaries between various components, operations and data storage devices are arbitrary to some extent, and specific operations are illustrated in the context of a specific illustrative configuration. Other allocations of functionality are envisioned and may fall within the scope of the present invention. In general, the structure and functionality presented as separate components in an exemplary configuration may be implemented as a combined structure or component. Similarly, the structure and functionality presented as a single component may be implemented as a separate component. These and other variations, modifications, additions and improvements may fall within the scope of the appended claims.

Claims

1. A method for determining a target configuration for a container-based cluster, the method comprising: determining, by a virtualization management platform configured to manage components of the container-based cluster, a current state of the container-based cluster; determining, by the virtualization management platform, at least one of a performance metric or a resource utilization metric of the container-based cluster based on the current state of the container-based cluster; processing the current state and at least one of the performance metric or the resource utilization metric with a model configured to generate a plurality of candidate configurations recommended for the container-based cluster and thereby generating the plurality of candidate configurations; calculating a reward score for each of the plurality of candidate configurations; selecting the target configuration as a candidate configuration from the plurality of candidate configurations based on the reward score of the target configuration; and Configuration settings of the container-based cluster are adjusted based on the target configuration to change the current state of the container-based cluster.

2. The method of claim 1 , wherein selecting the target configuration as the candidate configuration from the plurality of candidate configurations based on the reward score of the target configuration comprises: The target configuration is selected as a candidate configuration from the plurality of candidate configurations having a highest reward score among the reward scores calculated for the plurality of candidate configurations.

3. The method of claim 1 , wherein adjusting the configuration settings of the container-based cluster comprises automatically adjusting at least one of: the number of container groups deployed in the container-based cluster, resource limits of one or more of the container groups deployed in the container-based cluster, A resource request of one or more of the groups of containers deployed in the container-based cluster, or a number of nodes deployed in the container-based cluster.

4. The method of claim 1 , wherein adjusting the configuration settings of the container-based cluster comprises: providing the target configuration to a user; and Updated configuration settings for the container-based cluster are received.

5. The method according to claim 1, wherein: The virtualization management platform determines the performance metric of the container-based cluster; and The performance metric includes at least one of: the central processing unit CPU usage of each application deployed in the container-based cluster, Memory usage of each application, Application availability, Waiting time, Average response time, Input / output I / O rate, or Request error rate.

6. The method according to claim 1, wherein: The virtualization management platform determines the resource utilization metric of the container-based cluster; and The resource utilization metric includes at least one of: an average utilization percentage of CPU allocated to one or more of the components of the container-based cluster, or An average utilization percentage of memory allocated to one or more of the components of the container-based cluster.

7. The method according to claim 1, further comprising: The model is trained on a plurality of training data instances to generate the plurality of candidate configurations, wherein each of the plurality of training data instances comprises: training inputs comprising information about configuration settings applied to the container-based cluster and information about one or more loads generated for the container-based cluster; and A training output includes at least one of a performance metric or a resource utilization metric collected for the container-based cluster when the corresponding configuration settings are applied and the corresponding one or more loads are generated.

8. The method of claim 7, further comprising generating each of the plurality of training data instances, wherein generating a training data instance in the plurality of training data instances comprises: applying the configuration settings associated with the training data instance to the container-based cluster; generating, by a load generator, the one or more loads associated with the training data instance; and At least one of the performance metric or the resource utilization metric associated with the training data instance is determined by the virtualization management platform.

9. The method according to claim 1, further comprising: An optimization objective is received, wherein calculating the reward score for each of the plurality of candidate configurations is based on the optimization objective.

10. A system comprising: one or more processors; and at least one memory, the one or more processors and the at least one memory being configured to: determining, by a virtualization management platform configured to manage components of a container-based cluster, a current state of the container-based cluster; determining, by the virtualization management platform, at least one of a performance metric or a resource utilization metric of the container-based cluster based on the current state of the container-based cluster; processing the current state and at least one of the performance metric or the resource utilization metric with a model configured to generate a plurality of candidate configurations recommended for the container-based cluster and thereby generating the plurality of candidate configurations; calculating a reward score for each of the plurality of candidate configurations; selecting the target configuration as a candidate configuration from the plurality of candidate configurations based on the reward score of the target configuration; and Configuration settings of the container-based cluster are adjusted based on the target configuration to change the current state of the container-based cluster.

11. The system of claim 10, wherein selecting the target configuration as the candidate configuration from the plurality of candidate configurations based on the reward score of the target configuration comprises: The target configuration is selected as a candidate configuration from the plurality of candidate configurations having a highest reward score among the reward scores calculated for the plurality of candidate configurations.

12. The system of claim 10, wherein adjusting the configuration settings of the container-based cluster comprises automatically adjusting at least one of: the number of container groups deployed in the container-based cluster, resource limits of one or more of the container groups deployed in the container-based cluster, A resource request of one or more of the groups of containers deployed in the container-based cluster, or a number of nodes deployed in the container-based cluster.

13. The system of claim 10, wherein adjusting the configuration settings of the container-based cluster comprises: providing the target configuration to a user; and Updated configuration settings for the container-based cluster are received.

14. The system of claim 10, wherein: The virtualization management platform determines the performance metric of the container-based cluster; and The performance metric includes at least one of: the central processing unit CPU usage of each application deployed in the container-based cluster, Memory usage of each application, Application availability, Waiting time, Average response time, Input / output I / O rate, or Request error rate.

15. The system of claim 10, wherein: The virtualization management platform determines the resource utilization metric of the container-based cluster; and The resource utilization metric includes at least one of: an average utilization percentage of CPU allocated to one or more of the components of the container-based cluster, or An average utilization percentage of memory allocated to one or more of the components of the container-based cluster.

16. The system of claim 10, wherein the one or more processors and the at least one memory are further configured to: The model is trained on a plurality of training data instances to generate the plurality of candidate configurations, wherein each of the plurality of training data instances comprises: training inputs comprising information about configuration settings applied to the container-based cluster and information about one or more loads generated for the container-based cluster; as well as A training output includes at least one of a performance metric or a resource utilization metric collected for the container-based cluster when the corresponding configuration settings are applied and the corresponding one or more loads are generated.

17. The system of claim 16, wherein the one or more processors and the at least one memory are further configured to generate each of the plurality of training data instances, wherein generating a training data instance in the plurality of training data instances comprises: applying the configuration settings associated with the training data instance to the container-based cluster; generating, by a load generator, the one or more loads associated with the training data instance; and At least one of the performance metric or the resource utilization metric associated with the training data instance is determined by the virtualization management platform.

18. The system of claim 10, wherein the one or more processors and the at least one memory are further configured to: An optimization objective is received, wherein calculating the reward score for each of the plurality of candidate configurations is based on the optimization objective.

19. A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of a computing system, cause the computing system to perform operations for determining a target configuration for a container-based cluster, the operations comprising: determining, by a virtualization management platform configured to manage components of the container-based cluster, a current state of the container-based cluster; determining, by the virtualization management platform, at least one of a performance metric or a resource utilization metric of the container-based cluster based on the current state of the container-based cluster; processing the current state and at least one of the performance metric or the resource utilization metric with a model configured to generate a plurality of candidate configurations recommended for the container-based cluster and thereby generating the plurality of candidate configurations; calculating a reward score for each of the plurality of candidate configurations; selecting the target configuration as a candidate configuration from the plurality of candidate configurations based on the reward score of the target configuration; and Configuration settings of the container-based cluster are adjusted based on the target configuration to change the current state of the container-based cluster.

20. The non-transitory computer-readable medium of claim 19, wherein selecting the target configuration as the candidate configuration from the plurality of candidate configurations based on the reward score of the target configuration comprises: The target configuration is selected as a candidate configuration from the plurality of candidate configurations having a highest reward score among the reward scores calculated for the plurality of candidate configurations.

Images