K8s cluster automatic scaling method and system based on pooled virtual machines

Abstract

This invention discloses a method and system for automatic scaling of a Kubernetes cluster based on pooled virtual machines, including the following steps: developing a VM Pool management service tool, which implements abstract management of the VM Pool and supports multi-cloud environments; executing tasks based on scheduled logic, obtaining the default configuration of a specified VM Pool, and comparing the current number and specifications of standby VMs with the expected number of standby VMs (standbySize) and specifications (vmSpec) in the YAML configuration. This invention can launch more suitable virtual machines according to business needs at different times, effectively improving resource utilization and reducing waste. Compared with the native elastic scaling service, the success rate of Node startup can be increased from 95% to 99.9%, and the average Node startup time can be shortened from about 2 minutes to about 20 seconds.

Description

Technical Field

[0001] The present invention relates to the technical field of cluster management, and in particular to a K8s cluster automatic scaling method and system based on pooled virtual machines. Background Art

[0002] Cluster AutoScaling (CA) means that when cluster resources are insufficient and new Pods cannot be scheduled to any Node, a new Node is automatically created and added to the cluster; when all Pods on a Node can be migrated to other Nodes, the Pods on the Node are expelled and the Node is removed from the cluster and released. It is usually implemented in combination with the ability of public clouds to elastically create and release virtual machines.

[0003] Cluster Autoscaler is an official component for implementing automatic scaling of K8s clusters. It has built-in support for public clouds such as AWS, Azure, Alibaba Cloud, and Tencent Cloud. Each cloud is called a Cloud Provider and implements the required interfaces in its own independent package (directory).

[0004] In the existing technology, directly using the official Cluster Autoscaler may encounter the following problems:

[0005] 1. Insufficient inventory of the specified virtual machine specifications: Cloud vendors have inventory limits for virtual machines of specified specifications in specified areas. When capacity expansion is required, there is no guarantee that virtual machines of specified specifications can be created, especially at hot times, such as 0:00. Many cloud tenants will create a large number of virtual machines at this time to perform scheduled tasks, which can easily lead to the sale of some popular virtual machine specifications.

[0006] 2. The virtual machine startup speed is slow and unstable: The startup speed of the virtual machine is unstable and generally slow when it is first started. We have observed that the slowest startup time can be more than ten minutes. Especially when continuous large-scale expansion is required, this instability and delay will seriously affect the normal operation of the business.

[0007] 3. The Autoscaler support of many clouds is based on elastic scaling groups, such as Azure's VMSS and Alibaba Cloud's ASG. They generally support redundant startup to improve the startup speed and success rate, but the redundant instances need to be billed normally, which will cause a waste of resources.

[0008] 4. When connecting to different cloud vendors, you need to use different Cloud Providers. Each cloud vendor provides different elastic scaling services and technical methods, and requires different configuration operations. There is no way to unify management, and the learning and maintenance costs are high.

[0009] Based on this, the present invention proposes a K8s cluster automatic scaling method and system based on pooled virtual machines, which can achieve:

[0010] (1) Lock the virtual machine inventory in advance to avoid the inability to expand capacity due to insufficient cloud inventory when needed, and dynamically adjust the specifications and quantity of pooled virtual machines according to different time periods to meet usage needs in different time periods.

[0011] (2) Pre-start the pooled virtual machines, configure them, and then shut them down to improve subsequent startup speed and ensure stability.

[0012] (3) Use the cloud’s no-downtime-charge feature to save costs.

[0013] (4) The cloud expansion and contraction operations are uniformly abstracted into operations on virtual machine pools, simplifying the configuration and operation and maintenance of multi-clouds. Summary of the invention

[0014] The purpose of the present invention is to provide a K8s cluster automatic scaling method and system based on pooled virtual machines to solve the problems raised in the above background technology.

[0015] To achieve the above object, the present invention provides the following technical solution: a K8s cluster automatic scaling method based on pooled virtual machines, comprising the steps of:

[0016] S1: Develop a VM Pool management service tool to implement abstract management of VM Pool and support multi-cloud.

[0017] S2: Execute the task based on the timing logic, obtain the default configuration of the specified VM Pool and the configuration matching the current time, compare the current number and specification of standby VMs with the expected number of standby VMs standbySize and specification vmSpec in the obtained configuration, and proceed to step S3 if they are different, and proceed to step S4 if they are the same;

[0018] S3: Create a standby VM or delete a standby VM;

[0019] S4: The scheduled task ends.

[0020] Preferably, the VM Pool management service tool in step S1 provides: obtaining the VM Pool list, obtaining all VMs under the specified VMPool, expanding the specified VM Pool, shrinking the specified instance in the specified VM Pool and other necessary APIs for ClusterAutoscaler to call; connecting to the SDKs of different cloud vendors, and supporting the VM Pool API by calling the cloud's create VM, start VM, stop VM, and delete VM interfaces; supporting the configuration of VM Pool in YAML format, multiple configurations can be made, and each VM Pool supports the configuration of different VM specifications and quantities in different time periods.

[0021] Preferably, before comparing the expected number and specifications of standby VMs in step S2, it is necessary to first obtain the default configuration of the specified VM Pool, and then compare the current time with the timeWindows configuration. If the current time is within a certain timeWindow, the configuration of the timeWindow is merged with the default configuration, and the priority of the timeWindow configuration is higher than the default configuration.

[0022] Preferably, the creation process of the standby VM logic in step S3 includes:

[0023] S301: Call the API of the corresponding cloud vendor to create a VM and poll the VM status. After the VM is started, perform necessary pre-configuration on it, such as SSH into the VM to pre-pull the necessary container image, so that the relevant container can be started directly when it is actually used, saving the time of pulling the container image;

[0024] S302: After all pre-configuration operations are completed, use the "no charge for downtime" function to shut it down (taking Alibaba Cloud as an example, add parameters when calling the shutdown interface), set a startup script so that it can automatically and quickly join the cluster the next time it is started, and then mark it as ready, that is, standby;

[0025] S303: If an error occurs during pre-creation and pre-configuration, the VM is marked as notReady and deleted.

[0026] Preferably, in step S1, the VM Pool management service tool also implements a custom Cloud Provider in ClusterAutoscaler, and implements cluster expansion and contraction by calling the VMPool API, thereby achieving decoupling from the cloud.

[0027] Preferably, the specific process of executing the capacity expansion request submitted by the VM Pool Provider includes: calling the VM API of the cloud vendor to start a specified number of standby VMs; if the number of current standby VMs is insufficient, executing the logic of creating standby VMs, but after pre-configuration, instead of shutting down the machine, directly executing the startup script to join the cluster immediately; the specific process of executing the capacity reduction request submitted by the VM Pool Provider includes: calling the VM API of the cloud vendor to delete the specified VM.

[0028] A K8s cluster automatic scaling system based on pooled virtual machines, wherein the cluster automatic scaling system is implemented based on the above method.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. Compared with native elastic scaling services, the present invention can improve the Node startup success rate from 95% to 99.9%, shorten the Node startup speed from an average of about 2 minutes to about 20 seconds, shorten the Pod startup speed from an average of about 5 minutes to about 1 minute, and the standby VMs in the VM Pool do not increase the cost much.

[0031] 2. The present invention can start more appropriate virtual machines according to business needs in different time periods, which can effectively improve resource utilization and reduce waste.

[0032] 3. The present invention achieves the unification of multi-cloud configuration by using an abstract VM Pool structure. It is no longer necessary to learn the implementation and configuration of a specific cloud provider and will not be affected by its changes in the future, thus reducing operation and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a flow chart of the method of the present invention;

[0034] Figure 2 The flowchart of implementing cluster expansion and reduction by calling VMPool API in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] See also Figure 1The present invention provides a technical solution: a K8s cluster automatic scaling method based on pooled virtual machines, comprising the steps of:

[0037] S1: Develop a VM Pool management service tool to implement abstract management of VM Pool and support multi-cloud.

[0038] S2: Execute the task based on the timing logic, obtain the default configuration of the specified VM Pool and the configuration matching the current time, compare the current number and specification of standby VMs with the expected number of standby VMs standbySize and specification vmSpec in the obtained configuration, and proceed to step S3 if they are different, and proceed to step S4 if they are the same;

[0039] S3: Create a standby VM or delete a standby VM;

[0040] S4: The scheduled task ends.

[0041] In this embodiment, the YAML sample configuration of VMPool is as follows:

[0042] vmPools:

[0043] - name: main

[0044] # The minimum pool size is 0, which means there can be no running VMs at all

[0045] minSize: 0

[0046] # The maximum pool size is 100, which means that a maximum of 100 VMs can be running at the same time

[0047] maxSize: 100

[0048] # The number of VMs on standby, which is the key feature of this patent,

[0049] # The number of VMs that are pre-created, pre-configured and in standby state can be specified through the configuration file.

[0050] # When cluster resources are insufficient, the standby VM can quickly and stably join the cluster and enter the running state

[0051] standbySize: 5

[0052] # Required VM specifications

[0053] vmSpec:

[0054] cloud: aliyun

[0055] # This is the specification name of Alibaba Cloud ECS

[0056] name: ecs.g7a.4xlarge

[0057] # Declare that the number of CPU cores for this specification is 16

[0058] cpuCores: 16

[0059] # Declare that the memory of this specification is 64G

[0060] memoryMB: 65536

[0061] # The above is the default configuration. You can also configure it according to the time period, as follows

[0062] timeWindows:

[0063] # There are a lot of scheduled tasks from 0:00 to 8:00, which will require a lot of CPU and memory resources.

[0064] # So here is a separate configuration for 0 o'clock ~ 8 o'clock

[0065] - from: '0:00'

[0066] to: '8:00'

[0067] # The number of VMs on standby during this period needs to be 8 times the usual number

[0068] standbySize: 40

[0069] vmSpec:

[0070] # During this period, there are many big data tasks, and the CPU-memory ratio of the VM needs to be 1:8.

[0071] # Instead of the usual 1:4, you need to specify other specifications.

[0072] # In addition, in order to sell out a single specification, you need to specify multiple specifications

[0073] names:

[0074] - ecs.r8a.4xlarge

[0075] - ecs.r6a.4xlarge

[0076] # Declare that the memory of this specification is 128G

[0077] memoryMB: 131072

[0078] In this embodiment, the VM Pool management service tool in step S1 provides: obtaining the VM Pool list, obtaining all VMs under the specified VM Pool, expanding the specified VM Pool, shrinking the specified instance in the specified VM Pool and other necessary APIs for Cluster Autoscaler to call; connecting to the SDKs of different cloud vendors, and supporting the VM Pool API by calling the cloud's create VM, start VM, stop VM, and delete VM interfaces; supporting the configuration of VM Pool in YAML format, and multiple VM Pools can be configured. Each VM Pool supports the configuration of different VM specifications and quantities in different time periods.

[0079] In this embodiment, before comparing the expected number and specifications of standby VMs in step S2, it is necessary to first obtain the default configuration of the specified VMPool, and then compare the current time with the timeWindows configuration. If the current time is within a certain timeWindow, the configuration of the timeWindow is merged with the default configuration, and the priority of the timeWindow configuration is higher than the default configuration.

[0080] In this embodiment, the creation process of the standby VM logic in step S3 includes:

[0081] S301: Call the API of the corresponding cloud vendor to create a VM and poll the VM status. After the VM is started, perform necessary pre-configuration on it, such as SSH into the VM to pre-pull the necessary container image, so that the relevant container can be started directly when it is actually used, saving the time of pulling the container image;

[0082] S302: After all pre-configuration operations are completed, use the "no charge for downtime" function to shut it down (taking Alibaba Cloud as an example, add parameters when calling the shutdown interface), set a startup script so that it can automatically and quickly join the cluster the next time it is started, and then mark it as ready, that is, standby;

[0083] S303: If an error occurs during pre-creation and pre-configuration, the VM is marked as notReady and deleted.

[0084] In this embodiment, the VM Pool management service tool in step S1 also implements a custom Cloud Provider in ClusterAutoscaler, and implements cluster expansion and reduction by calling the VMPool API, thereby achieving decoupling from the cloud.

[0085] See also Figure 2In this embodiment, the specific process of executing the capacity expansion request submitted by the VM Pool Provider includes: calling the VM API of the cloud vendor to start a specified number of standby VMs; if the number of current standby VMs is insufficient, the logic of creating a standby VM is executed, but after pre-configuration, the machine is not shut down but the startup script is directly executed to join the cluster immediately; the specific process of executing the capacity reduction request submitted by the VM Pool Provider includes: calling the VM API of the cloud vendor to delete the specified VM.

[0086] A K8s cluster automatic scaling system based on pooled virtual machines, wherein the cluster automatic scaling system is implemented based on the above method.

[0087] The specific implementation method of VM Pool Provider is to add a directory (Go package) to the K8s Cluster Autoscaler code and then implement and interface in it. The following methods in the interface are the most critical:

[0088] / / NodeGroup contains configuration info and functions to control aset

[0089] / / of nodes that have the same capacity and set of labels.

[0090] type NodeGroup interface {

[0091] / / TargetSize returns the current target size of the node group. It is possible that the

[0092] / / number of nodes in Kubernetes is different at the moment but should be equal

[0093] / / to Size() once everything stabilizes (new nodes finish startup andregistration or

[0094] / / removed nodes are deleted completely). Implementation required.

[0095] TargetSize() (int, error)

[0096] / / IncreaseSize increases the size of the node group. To delete anode you need

[0097] / / to explicitly name it and use DeleteNode. This function shouldwait until

[0098] / / node group size is updated. Implementation required.

[0099] IncreaseSize(delta int) error

[0100] / / DeleteNodes deletes nodes from this node group. Error is returnedeither on

[0101] / / failure or if the given node doesn't belong to this node group.This function

[0102] / / should wait until node group size is updated. Implementationrequired.

[0103] DeleteNodes([]*apiv1.Node) error

[0104] / / Nodes returns a list of all nodes that belong to this node group.

[0105] / / It is required that Instance objects returned by this method haveId field set.

[0106] / / Other fields are optional.

[0107] / / This list should include also instances that might have not become a kubernetes node yet.

[0108] Nodes() ([]Instance, error)

[0109] }

[0110] When implementing the above interface methods, the corresponding VM Pool API will be called as appropriate.

[0111] It is worth noting that: the present invention also provides definitions of terms:

[0112] A cluster is composed of nodes. Each Node corresponds to a virtual machine or physical machine and has fixed available resources such as CPU and memory.

[0113] Pod is a combination of one or more containers and is the smallest unit for K8s scheduling. Each container has its own resource requirements (Request) such as CPU and memory.

[0114] Scheduling is the process of K8s assigning Pods to Nodes based on factors such as the Pod's resource requirements (the sum of all containers) and the remaining available resources of the Node (the total minus the resources occupied by the assigned Pods).

[0115] Cluster AutoScaling (CA) means that when cluster resources are insufficient and new Pods cannot be scheduled to any Node, a new Node is automatically created and added to the cluster; when all Pods on a Node can be migrated to other Nodes, the Pods on the Node are expelled and the Node is removed from the cluster and released. It is usually implemented in combination with the ability of public clouds to elastically create and release virtual machines.

[0116] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0117] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0118] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0119] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only one. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0121] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0122] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0123] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0124] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0125] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A K8s cluster automatic scaling method based on pooled virtual machines, characterized in that: Includes steps: S1: Develop a VM Pool management service tool to implement abstract management of VM Pool and support multi-cloud. S2: Execute tasks based on timing logic and obtain the default configuration of the specified VM Pool and the separate configuration for each specified time period; compare the current time with each specified time period, and if the current time is within a specified time period, merge the separate configuration of the specified time period with the default configuration, and make the separate configuration of the specified time period have a higher priority than the default configuration; Compare the current number and specification of standby VMs with the expected number of standby VMs standbySize and specification vmSpec in the merged configuration, if they are different, proceed to step S3, if they are the same, proceed to step S4; S3: Create a standby VM or delete a standby VM; S4: The scheduled task ends.

2. The K8s cluster automatic scaling method based on pooled virtual machines according to claim 1 is characterized in that: The VM Pool management service tool in step S1 provides: obtaining the VM Pool list, obtaining all VMs under the specified VM Pool, expanding the specified VM Pool, and reducing the specified instance in the specified VM Pool. The necessary APIs are called by Cluster Autoscaler; access to SDKs of different cloud vendors, and support for VM Pool API by calling the cloud's create VM, start VM, stop VM, and delete VM interfaces; support for configuring VM Pool in YAML format, multiple configurations are possible, and each VM Pool supports configuring different VM specifications and quantities in different time periods.

3. The K8s cluster automatic scaling method based on pooled virtual machines according to claim 1 is characterized in that: The creation process of the standby VM logic in step S3 includes: S301: Call the API of the corresponding cloud vendor to create a VM and poll the VM status. After the VM is started, perform necessary pre-configuration on it and SSH into the VM to pre-pull the necessary container image. S302: After all pre-configuration operations are completed, use the free shutdown function to shut down the server, set a startup script so that it can automatically and quickly join the cluster the next time it is powered on, and then mark it as ready, that is, standby; S303: If an error occurs during pre-creation and pre-configuration, the VM is marked as notReady and deleted.

4. The K8s cluster automatic scaling method based on pooled virtual machines according to claim 1 is characterized in that: In step S1, the VM Pool management service tool also implements a custom CloudProvider in ClusterAutoscaler, and implements cluster expansion and reduction by calling the VMPool API, thereby achieving decoupling from the cloud.

5. The K8s cluster automatic scaling method based on pooled virtual machines according to claim 4 is characterized in that: The specific process of executing the capacity expansion request submitted by the VM Pool Provider includes: calling the cloud vendor's VM API to start a specified number of standby VMs; if the current number of standby VMs is insufficient, the logic of creating standby VMs is executed, but after pre-configuration, the power-on script is directly executed instead of shutting down to join the cluster immediately; the specific process of executing the capacity reduction request submitted by the VM Pool Provider includes: calling the cloud vendor's VM API to delete the specified VM.

6. K8s cluster automatic scaling system based on pooled virtual machines, characterized by: The cluster automatic scaling system is implemented based on the method described in any one of claims 1 to 5.

Images