Container group deployment method and apparatus

Abstract

A container group deployment method and device are disclosed. A first workload maintains a container group cache pool, and at least one container group is cached in the container group cache pool. In response to a creation of a second workload or a scaling request of the second workload, a container group is selected from the container group cache pool, the obtained container group is bound to the second workload, and the container group is moved into the second workload. In a further embodiment, when the container group is in the container group cache pool, a first part of a container runtime is started in advance in the container group, which is irrelevant to a function or an application. While in response to the container group being moved to the second workload, a second part of the container runtime is started in the container group, which is relevant to the function or the application corresponding to the second workload. Thus, the container group cache pool is maintained by the first workload which is independent of the tenant function in advance, so that when the tenant needs to scale, the cached container group resource can be directly extracted from the cache pool, realizing efficient deployment and rapid scaling of the container group.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of cloud computing, and in particular, to a deployment scheme of a container group. BACKGROUND

[0002] With the Internet technology entering the era of cloud computing, a "serverless" architecture has become an important technical component of cloud-native applications.

[0003] An important technical indicator of the serverless architecture is elasticity, which mainly manifests in two aspects: elastic capability and elastic performance. The serverless architecture is expected to provide customers with a good experience in terms of both elastic capability and elastic performance, so as to help customers maximize their value.

[0004] The problem that elasticity is concerned about is mainly the contradiction between capacity planning and actual cluster load. When the resources of the existing cluster cannot bear the traffic pressure, the scale of the cluster or the allocation of resources is adjusted to ensure the stability of the system. Similarly, when the cluster load is low, the resource configuration of the cluster is reduced as much as possible, so as to reduce the cost overhead caused by the waste of idle resources.

[0005] In order to improve the elastic capability and the elastic performance, efficient creation of a container group (hereinafter, it can also be referred to as "Pod") is very important.

[0006] In the original Pod production process of Kubernetes (K8s), the Pod creation process needs to go through multiple stages: Pod creation request submission (for example, submission to the etcd system), scheduling, Pod production, network initialization, container data volume mounting, container image downloading, and container startup. In the entire Pod production process, it is greatly affected by the infrastructure scheduling strategy and the resource state, resulting in a long time consumed for Pod creation.

[0007] On the other hand, due to the design principle of Kubernetes facing the terminal state, when the information in the template of the work load (Workload) and the template of the container group (Pod) changes, the Pod is rebuilt.

[0008] For this reason, it is very difficult to cache the Pod in Kubernetes. This is because Pod caching must be implemented in the case where a group of functions or applications have the same container runtime without changing the container configuration (Spec) such as environment variables and container images.

[0009] The existing Pod pooling solutions in the industry mainly target a single application or a single function, creating a batch of Pods as a cache pool. These cached Pods only serve one function to avoid Pod reconstruction.

[0010] Figure 1 The diagram schematically illustrates the relationship between Pods, function instances, and workloads in an existing Pod management solution.

[0011] like Figure 1 As shown, the Pod pool maintained by the Pod pool workload caches multiple Pods (Pods are schematically represented by heptagonal graphics in the attached figure). Multiple function instances Fn-A, Fn-B, and Fn-C select Pods from the Pod pool to use. However, these function instances are all in the same workload. For the Pod management of each function or application, it is not possible to use the existing workload method of Kubernetes. Pods related to the specified function or application can only be retrieved through custom labels (Labels). When the workload that creates the pool changes, the Pods assigned to multiple functions or applications will be affected by the corresponding workload changes and will be rebuilt. If the workload that manages these Pods is deleted, all Pods will be deleted.

[0012] In this way, it is almost impossible to effectively manage Pods in scenarios with multiple functions and applications.

[0013] Therefore, there is still a need to design and implement a general pooled cache architecture based on the K8s architecture to help improve the elastic performance of applications and functions in Serverless scenarios, achieve extremely fast instance startup, and help users implement a more flexible computing model. Summary of the Invention

[0014] A technical problem to be solved by the present disclosure is to provide a container group pooled cache deployment solution that can achieve rapid startup of containers.

[0015] According to a first aspect of the present disclosure, a container group deployment method is provided, including: maintaining a container group cache pool by a first workload, caching at least one container group in the container group cache pool; and in response to a request to create a second workload or to expand the capacity of the second workload, selecting a container group from the container group cache pool, binding the obtained container group to the second workload, and moving the container group to the second workload.

[0016] Optionally, the first workload maintains multiple container group cache pools, wherein the container groups cached in each container group cache pool have the same resource configuration specifications, and the container groups cached in different container group cache pools have different resource configuration specifications. The step of selecting a container group from the container group cache pool may include: selecting a container group from a container group cache pool that matches the second workload based on the container group resource configuration specification requirements of the second workload.

[0017] Optionally, when the container groups in one container group cache pool are consumed at a faster rate, container groups are deployed from other container group cache pools to supplement the one container group cache pool.

[0018] Optionally, the first workload is a general workload constructed for functions or applications using the same image.

[0019] Optionally, the second workload is a workload corresponding to the function or application that is created in response to creation of the function or application, and the function or application uses the same image.

[0020] Optionally, the method may further include: when the container group is in the container group cache pool, starting in advance a first part of the container runtime that is unrelated to the function or application in the container group; and / or in response to the container group being moved to the second workload, starting in the container group a second part of the container runtime that is related to the function or application corresponding to the second workload; and / or after the container group in the container group cache pool starts in advance the first part of the container runtime that is unrelated to the function or application, placing the containers in the container group in a standby state to wait for the container group to be bound to the second workload so as to start the second part of the container runtime that is related to the function or application corresponding to the second workload.

[0021] Optionally, the first part includes at least one of the following: container startup and container runtime framework initialization.

[0022] Optionally, the second part includes at least one of the following: dynamic injection of environment variables, information or code loading of functions or applications.

[0023] Optionally, the method may further include: after the container group in the container group cache pool starts the first part of the container runtime in advance, controlling the activity state and the ready state of the containers in the container group: so that the containers meet the activity state requirement but do not meet the ready state requirement; or so that the containers meet both the activity state requirement and the ready state requirement.

[0024] Optionally, the method may further include: in response to the container group being bound to the second workload, performing at least one of the following modifications on the container group: updating a label of the container group so that a container group selector corresponding to the second workload can retrieve the container group; and modifying the owner information of the container group to the second workload.

[0025] Optionally, the method may further include: after binding the obtained container group to the second workload, maintaining the status of the container group, where the statuses include: successful binding, successful startup, failed, and deleted. The successful binding status indicates that the container group has been selected from the container group cache pool and bound to the second workload. The successful startup status indicates that the container runtime of the successfully bound container group has successfully started and entered the ready state. The failed status indicates that the container runtime of the successfully bound container group has failed to start or timed out, or that the container runtime of the successfully started container group has encountered an error. For container groups that enter the failed state due to a container runtime startup failure or timeout, the successful binding status is returned to retry starting the container runtime. For container groups that enter the failed state due to a container runtime error, the successful startup status is restored after the container runtime has resolved the fault and returned to the ready state. The deleted status indicates that the container group is about to be deleted. If the container runtime of the successfully bound container group fails to start multiple times, or if the container runtime of the failed container group fails to resolve the fault, the container group enters the deleted state.

[0026] According to a second aspect of the present disclosure, a container group deployment device is provided, comprising: a caching device for maintaining a container group cache pool using a first workload, wherein at least one container group is cached in the container group cache pool; and a migration device for selecting a container group from the container group cache pool in response to a request to create a second workload or to expand the capacity of the second workload, binding the obtained container group to the second workload, and moving the container group to the second workload.

[0027] Optionally, the device may further include: a first starting device for starting, in advance, a first portion of the container runtime that is unrelated to the function or application in the container group when the container group is in the container group cache pool; and / or a second starting device for starting, in response to the container group being moved to the second workload, a second portion of the container runtime that is related to the function or application corresponding to the second workload in the container group; and / or a standby device for placing the containers in the container group in a standby state after the container group in the container group cache pool starts in advance the first portion of the container runtime that is unrelated to the function or application, waiting for the container group to be bound to the second workload so as to start the second portion of the container runtime that is related to the function or application corresponding to the second workload. The first workload is a general workload constructed for functions or applications using the same image, and the second workload is a workload corresponding to the function or application created in response to the creation of the function or application, and the function or application uses the same image.

[0028] According to a third aspect of the present disclosure, a computing device is provided, comprising: a processor; and a memory on which executable code is stored, and when the executable code is executed by the processor, the processor executes the method described in the first aspect above.

[0029] According to a fourth aspect of the present disclosure, a computer program product is provided, comprising an executable code. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method as described in the first aspect above.

[0030] According to a fifth aspect of the present disclosure, a non-transitory machine-readable storage medium is provided, on which executable code is stored. When the executable code is executed by a processor of an electronic device, the processor executes the method described in the first aspect above.

[0031] This provides a universal pooling capability, such as one suitable for multi-tenant scenarios. This capability can be used in multiple scenarios, accelerating Pod creation in Kubernetes-based scenarios, improving Pod startup performance, and enabling mechanism elasticity. A container group cache pool is maintained in advance by a primary workload independent of tenant functions, allowing tenants to directly retrieve cached container group resources from the cache pool when capacity expansion is required, enabling efficient deployment and rapid expansion of container groups.

[0032] Furthermore, in an embodiment, by starting the common part of the Pod's container runtime in advance, the cached Pod resources are directly used when services need to be provided or capacity expanded, and the personalized part of the container runtime is further started, thereby achieving the purpose of efficient deployment and rapid capacity expansion.

[0033] Furthermore, in the embodiment, the objective differences in elasticity requirements of different functions or applications are utilized to achieve maximum utilization of the cache resource pool in a multi-tenant mode for multiple functions or applications. At the same time, a mutually isolated Pod management solution is also provided for multiple functions or applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0035] Figure 1 The diagram schematically illustrates the relationship between Pods, function instances, and workloads in an existing Pod management solution.

[0036] Figure 2 is a schematic flowchart of a container group deployment method according to the present disclosure.

[0037] Figure 3 is a schematic block diagram of a container group deployment device according to the present disclosure.

[0038] Figure 4 A schematic diagram showing a Pod cache pool with different resource specifications.

[0039] Figure 5 A schematic flowchart of a container group deployment method according to a further embodiment of the present disclosure is shown.

[0040] Figure 6 A schematic block diagram of a container group deployment device according to a further embodiment of the present disclosure is shown.

[0041] Figure 7 The following schematically shows a system architecture diagram that can be used to implement the Pod deployment solution according to the present disclosure.

[0042] Figure 8 The diagram schematically illustrates a phased startup scheme for a container runtime.

[0043] Figure 9 The diagram schematically illustrates the Pod lifecycle management solution adopted by the container group deployment solution of the present disclosure.

[0044] Figure 10 A schematic diagram of the structure of a computing device that can be used to implement the above container group deployment method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0045] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0046] First, several terms involved in this application are briefly introduced.

[0047] Kubernetes (abbreviated as "K8s") is a container cluster management system that provides container-based application orchestration, application deployment, maintenance, expansion mechanism and other functions. Kubernetes can be used to easily manage containerized applications running across machines.

[0048] A pod is the smallest and simplest unit of Kubernetes creation or deployment. A pod represents a single process running on a cluster. A pod encapsulates an application container (or multiple containers), storage resources, a dedicated network IP address, and policy options to manage and control how the container runs. In Kubernetes, a pod represents a group of running containers on a cluster.

[0049] A workload is an application running on Kubernetes. Whether a workload is a single component or consists of multiple components working together, it can run in a set of Pods in Kubernetes. In other words, a workload manages a set of Pod resources with a unified function.

[0050] Containers, the fundamental building blocks of Pod resources, provide the essential operating environment for application components. A Pod orchestrates a group of containers, defining their basic dependencies and lifecycles. The operating status of each container collectively constitutes the Pod's operating state.

[0051] Container Runtime refers to a runtime for serverless computing, a program that can create and run containers based on code obtained online or files in a specified location in the container.

[0052] Pod pooling refers to resource orchestration based on the Kubernetes immutable infrastructure architecture, with Pod as the smallest resource granularity. In Serverless scenarios, by preparing Pod resources in advance, existing Pod resources can be directly reused when capacity expansion is needed, improving deployment and expansion efficiency.

[0053] Below, a container group (Pod) pooled cache deployment solution according to the present disclosure is described with reference to the accompanying drawings.

[0054] Figure 2 It is a schematic flow chart of the Pod deployment method according to the present disclosure.

[0055] Figure 3 It is a schematic block diagram of a Pod deployment device according to the present disclosure.

[0056] like Figure 3 As shown, the Pod deployment device 100 according to the present disclosure may include a cache device 110 and a migration device 120 .

[0057] like Figure 2 As shown, in step S110, the first workload may maintain a Pod cache pool, for example, through the cache device 110. The Pod cache pool may cache at least one Pod.

[0058] Then, in step S120, in response to a request to create a second workload or expand the capacity of the second workload, for example, the migration device 120 can select a Pod from the Pod cache pool, bind the obtained Pod to the second workload, and move it to the second workload.

[0059] The first workload, also known as a "pooled cache workload" or "pool workload," produces Pod resources and caches them in the Pod cache pool.

[0060] The first workload is a general workload constructed for functions or applications using the same image (the same type of container runtime). The first workload does not reflect information related to a specific function or application.

[0061] When multiple functions or applications using the same image initiate capacity expansion requests, a specified number of Pods are selected from the first workload. These Pods are then bound and moved to the function or application's own workload (the second workload described below, also referred to as the "function workload" or "application workload").

[0062] In this way, Pods that have been moved to the function or application's own second workload will no longer be affected by the Pod cache pool maintained by the first workload. Even if the Pod cache pool is rebuilt, it will not affect the reconstruction of Pods in the function or application's second workload.

[0063] Here, regarding the type of the first workload, you can directly use the official K8s workload, such as Deployment resources, or you can use third-party open source or customized workload resources.

[0064] The second workload, also called a "function workload" or "application workload," is a workload created in response to the creation of a function or application. The created function or application uses the same image as the first workload.

[0065] When creating a function or releasing an application, a specific number of Pod instances need to be created for the specified application or function. To do this, you can create a workload that uses pooled resources, i.e., a second workload, to manage the Pods actually allocated from the cache pool to specific functions or applications.

[0066] The second workload is related to a specific application and function. The second workload may include some information related to the function or application, including the function / application name, the runtime information it depends on, etc.

[0067] After creating the second workload or receiving a request to expand the capacity of the second workload, you can bind and move Pods from the Pod cache pool that matches the type of function or application based on the second workload's requirement for the number of Pods and the type of function or application.

[0068] After completing the allocation and scheduling of the cache Pod, you can send a request to the specified container to complete the operation of the function or application according to the configuration (Spec) requirements.

[0069] After the Pod move is completed, the second workload of each function or application manages the Pod resources allocated to it respectively.

[0070] The second workload can use a custom workload type to provide consistency of Replicas when Pods are moved.

[0071] Here, in response to the Pod being bound to the second workload, some modifications may be made to the Pod.

[0072] First, you can update the labels of the Pod selected in the pool so that the Pod selector corresponding to the second workload can retrieve the Pod.

[0073] Secondly, you can also modify the owner resource (OwnerReference) information of the Pod selected from the pool and change the owner (Owner) information of the Pod to the second workload.

[0074] In addition, in order to improve the resource utilization and allocation speed of the Pod cache pool, a Pod hierarchical management solution can also be adopted.

[0075] Figure 4 A schematic diagram showing a Pod cache pool with different resource specifications.

[0076] like Figure 4 As shown, the first workload can maintain multiple Pod cache pools. The Pods cached in each Pod cache pool can have the same resource configuration specifications, while the Pods cached in different Pod cache pools can have different resource configuration specifications. A single first workload can maintain multiple Pod cache pools, or multiple first workloads can each maintain multiple Pod cache pools.

[0077] In this way, when a Pod is to be selected from the Pod cache pool, a Pod that matches the second workload can be selected from the Pod cache pool based on the second workload's requirements for Pod resource configuration specifications.

[0078] Figure 4 The figure shows three types of Pod cache pools with different resource specifications: 2C4G (2 cores with 4G memory), 1C2G (1 core with 2G memory), and 0.5C1G (0.5 core with 1G memory).

[0079] These Pod cache pools with different resource specifications can be combined into a Pod cache pool set.

[0080] Then, the number of replicas assigned to each workload is dynamically adjusted based on the resource usage of the entire collection.

[0081] When the Pods in a Pod cache pool are consumed quickly, Pods can be deployed from other Pod cache pools to supplement the Pod cache pool.

[0082] If the 2C4G Pod is consumed at a high rate, you can directly use in-place VPA (vertical automatic scaling of Pod) to schedule Pods from the Pod cache pool with smaller resource specifications (such as 1C2G) to supplement the large-scale Pod resources and achieve dynamic allocation of Pods.

[0083] Optionally, the pooled cache solution disclosed herein can support a single cache pool or multiple cache pools in cascade. Each cache pool can correspond to a specific resource configuration specification, just like a Bin in a Malloc Bin Class.

[0084] In other words, two modes can be provided: one is a single cache pool mode based on the premise of in-place resource upgrade feature; the other is a cache pool cascade mode with multiple resource configuration specifications.

[0085] Therefore, using in-place VPA can maintain a single cache pool with uniform resource configuration specifications, and can also provide cache containers for other workloads with different resource configuration specifications, thereby improving the utilization of the single cache pool.

[0086] Moreover, while saving resources, the Pod cache hit rate can be improved.

[0087] The above describes the pod caching and migration solution. Based on this, we can further optimize the runtime deployment of the container within the pod.

[0088] To adapt to Pod pooling and accelerate the startup of pooled container runtimes, the container runtime can be enabled to support multiple startup phases, and to achieve phased startup whenever possible, so that parts unrelated to functions or applications and parts related to functions or applications can be started in phases. When the Pod is in the Pod cache pool, the first part of the container runtime that is unrelated to the function or application can be supported to start in advance. When the Pod in the pool is assigned (or "bound") to a specific function or application and thus moved to the second workload, the startup of the container runtime that is closely related to the specific function or application is completed.

[0089] Specifically, some of the logic in a function or application is universal and implemented in a general programming language. It is independent of the specific programming language and can be started in advance in the Pod cache pool. The other part of the logic is related to a specific function or application and is often implemented in the programming language corresponding to the function or application. Therefore, it can only be started after being bound to the second workload corresponding to the function or application.

[0090] To support the universality of pooling and avoid maintaining multiple independent Pod cache pools for the container runtime of each programming language, the container runtime can be divided into two parts within the container image: the general part (first part) that is not related to the specific programming language and the part (second part) that is related to the specific programming language. For Pods in the Pod cache pool, the first part of the runtime that is not related to the specific programming language can be started in advance. After the Pod is bound to a specific function or application, the second part of the runtime that is related to the specific programming language is started during the bootloader running phase of the second part. In this way, the Pod cache pool can serve the second workload of different programming languages ​​without the need to maintain a cache pool for each programming language.

[0091] Figure 5 A schematic flowchart of a Pod deployment method according to a further embodiment of the present disclosure is shown.

[0092] Figure 5 Steps S110 and S120 in the process may be the same as those described above.

[0093] Figure 6 A schematic block diagram of a Pod deployment device according to a further embodiment of the present disclosure is shown.

[0094] like Figure 6 As shown, in addition to the cache device 110 and the migration device 120 described above, the Pod deployment device 100 may further include a first starting device 115 , a second starting device 125 , and a standby device 130 .

[0095] like Figure 5 As shown, in step S115, when the Pod is in the Pod cache pool, the first part of the container runtime that is not related to the function or application can be started in advance in the Pod, for example, by the first starting device 115.

[0096] The first part can be, for example, language-independent. Part of the function or application's logic is language-independent and can be implemented using a general-purpose programming language (such as RUST). This logic is independent of the function or application and can be started in advance.

[0097] The first part may, for example, include at least one of the following: container startup and container runtime framework initialization.

[0098] In step S125 , in response to the Pod being bound to the second workload, a second portion related to the function or application corresponding to the second workload during container runtime may be started in the Pod, for example, by the second starting device 125 .

[0099] The second part, for example, can be a startup part that contains many programming languages ​​used by functions or applications. This part of the logic is directly related to the function or application and cannot be started in advance. It needs to be started accordingly after being bound to the second workload.

[0100] The second part may include, for example, at least one of the following: dynamic injection of environment variables, information or code loading of functions or applications.

[0101] In other words, when caching pods in the pod cache pool, some common parts across functions, applications, and voices can be started in advance. Then, during or after the pod is moved to the second workload, the personalized parts closely related to the function, application, voice, etc. can be started.

[0102] To subsequently select cache Pods that meet the requirements, you can define multiple container runtime types, such as runtimes for specific programming languages ​​and runtimes for specific functions or application types.

[0103] In this way, while the Pod is being moved, some of the Pod container runtime logic can be loaded. The Pod movement and the running of the container runtime in the Pod can be completed concurrently.

[0104] For example, in the scenario of function computing, the life cycle of the runtime container used for function computing can be divided into multiple stages: container startup stage, function runtime framework (Runtime Framework) startup (initialization) stage, code loading stage, and code execution stage.

[0105] These phases (parts) can be divided into two major phases, namely the first part of early startup (stateless startup) in the Pod cache pool and the second part of startup when bound to a specific function or application.

[0106] First, in the first major phase, some logic (or processing / operations) is pre-started statelessly in the Pod cache pool. This mainly includes parts unrelated to specific functions or applications, such as container startup and function runtime framework initialization.

[0107] After completing this phase of startup, the container can be placed in a standby state, waiting for the second phase (second part) to start. For example, the standby device 130 can be used to pre-start the first part of the container runtime that is not related to the function or application in the Pod in the Pod cache pool, and then put the container in the Pod into a standby state, waiting for the Pod to be bound to the second workload, so as to start the second part of the container runtime that is related to the function or application corresponding to the second workload.

[0108] In this state, the container can be in the Running state. Based on this, two levels of support are provided for the Liveness and Readiness states based on K8s:

[0109] (1) One scenario is that the container that completes the first phase must pass the active state requirement, but may not pass the ready state requirement;

[0110] (2) Another scenario is that both the active state requirement and the ready state requirement can be passed.

[0111] In other words, after the Pods in the Pod Cache Pool pre-start the first part of the container runtime, the liveness and readiness of the containers in the Pods can be controlled. A container can meet the liveness requirement but not the readiness requirement, or it can meet both the liveness and readiness requirements.

[0112] Regarding this point, different settings can be made for various specific scenarios.

[0113] Then, in the second major phase, another piece of logic (or processing / operation) is started when the pooled Pod is bound to a specific function or application.

[0114] When the pooled Pod is bound to a specific running instance (function or application), the second phase of startup is initiated for the container.

[0115] For example, dynamic injection of environment variables, loading of application information / code, etc. can be performed to cooperate with the runtime to complete the startup of the second stage.

[0116] So far, the Pod pooling caching solution according to the present disclosure has been described in detail.

[0117] Figure 7 The following schematically shows a system architecture diagram that can be used to implement the Pod deployment solution according to the present disclosure.

[0118] like Figure 7 As shown, multiple Pods are cached in multiple Pod cache pools maintained by the first workload.

[0119] The cache pool manager manages the Pod cache pool. Since the Pods in the Pod cache pool can start some logic in advance and put them in a standby state, the Pod cache pool can also be called the "standby Pod cache pool."

[0120] After the user creates functions Fn-1, Fn-2, ..., Fn-n, a corresponding second workload will be created accordingly.

[0121] The elasticity decider can decide how many Pods should be allocated to each function, thereby achieving elasticity. Figure 7 As shown, three Pods are allocated to Fn-1, one Pod is allocated to Fn-2, and two Pods are allocated to Fn-n.

[0122] Based on the elasticity decider's decisions, the cache pool operator schedules pods cached in the pod cache pool to meet the elasticity decider's requirements. Using a dynamic pod selection channel, the selected pods are migrated from the pod cache pool to the corresponding secondary workloads of each function. The cache pool operator can also reconcile the secondary workloads accordingly.

[0123] The runtime manager can listen to the current status of each Pod in the standby Pod cache pool, and extract the code corresponding to the corresponding function from the code cache system at the appropriate time in the above two stages to start the runtime of the container in the Pod in stages.

[0124] The Pod monitoring controller can obtain Pod-related indicators corresponding to each function from the elasticity decider, and collect corresponding indicators from the second workload corresponding to the corresponding function to monitor and control the Pod.

[0125] The cache pool manager may include, for example, a carrier controller designed for the final state.

[0126] The workload controller can achieve two functions: Pod scheduling and expansion and Pod operation and maintenance movement.

[0127] First, describe Pod scheduling and scaling.

[0128] Upon receiving a request to scale out a function or application, the workload controller retrieves the actual number and status of Pods for the workload (secondary workload) corresponding to the requested function or application. For example, the workload controller can determine whether scaling out is necessary by comparing the resource specifications (Spec) of the workload with the actual number of Pods required for the secondary workload corresponding to the function or application. If scaling out is necessary, the workload controller can select a specified number of Pods from the standby Pod cache pool based on a specific scheduling strategy to fulfill the scaling request.

[0129] Of course, you can also first determine whether there are sufficient Pod instances in the Pod cache pool to meet the capacity expansion requirements. If the number of Pods in the Pod cache pool is less than the required number of Pods for capacity expansion, the workload transfer controller can prioritize obtaining the currently available number of Pods from the Pod cache pool. The remaining Pods that cannot be obtained from the Pod cache pool can be obtained through cold starts to meet the capacity expansion requirements.

[0130] After the workload controller selects an available Pod from the Pod cache pool, it can bind it to the second workload corresponding to a specific function or application.

[0131] After the Pods are successfully bound to the second workload corresponding to a specific function or application, these Pods, although still in the cache pool, already belong to the second workload corresponding to the specific function or application.

[0132] The runtime manager can include a Carrier Agent. After successful binding, the Carrier Agent takes over the specific Pods selected from the cache pool, thus completing the binding of the Pods to the specific container runtimes corresponding to the function or application. Once the Carrier Agent completes the runtime binding and confirms that the runtime has started and can normally accept requests, the entire expansion request is complete.

[0133] Generally speaking, this process accomplishes two things: binding between Pods and workloads; and binding between Pods and the corresponding runtimes of functions or applications.

[0134] Next, we describe Pod operation and maintenance migration.

[0135] After the Pod in the Pod cache pool is bound to the second workload corresponding to a specific function or application, the Pod and the second workload corresponding to the specific function or application maintain an owner relationship, that is, the Pod belongs to the second workload.

[0136] At this point, although the capacity expansion has been completed from the perspective of service, from the perspective of operation and maintenance, these Pods bound to the second workload corresponding to the function or application have not yet belonged to the application group where the second workload corresponding to the function or application is located.

[0137] To address the operational and maintenance issues of Pods in the Pod cache pool belonging to secondary workloads corresponding to different functions or applications and to isolate Pods belonging to secondary workloads corresponding to different functions or applications, Pod migration logic can be introduced. Specifically, after the ownership relationship between the Pod and the secondary workload corresponding to the function or application is determined, the bound Pod is completely removed from the current standby Pod cache pool from an operational and maintenance perspective.

[0138] In other words, by binding the Pod to the secondary workload, an ownership relationship is established between the two. In some cases, the runtime in the Pod may even be ready to run. However, to meet operational and tenant isolation requirements, the Pod needs to be moved to the secondary workload corresponding to the specified function or application.

[0139] In this way, after the Pod in the Pod cache pool is bound to the second workload corresponding to a specific function or application, the Pod will be moved to the second workload corresponding to the function or application to which it belongs and registered to the function or application's own application group, thereby achieving isolation at the operation and maintenance level.

[0140] It should be understood that Pod movement does not require binding to the runtime. Pod movement can begin after the Pod is bound to the secondary workload corresponding to a specific function or application. This allows the Pod movement process to be fully parallelized with the Pod binding process to the container runtime, further enabling Pod availability at both the service level (binding) and the operations level (movement).

[0141] Furthermore, this solution does not rely on the backend standby Pod cache pool for workload handling. It can be designed using a plug-in approach, allowing the standby Pod cache pool to be hot-swappable at any time.

[0142] The standby Pod cache pool is a resource that can be queried during capacity expansion based on the secondary workload information corresponding to the current function or application. If available backend Pod cache pool resources are available, capacity expansion is performed from that available backend cache pool. If no backend cache pool resources are available, a fallback solution can be used, such as a regular cold start solution, to scale Pods.

[0143] Figure 8 The diagram schematically illustrates a phased startup scheme for a container runtime.

[0144] The runtime manager listens to pods in the pod cache pool and pulls runtime code from the code cache system to prepare the container runtime for the pod in stages.

[0145] After functions Fn-1 and Fn-n each select a Pod from the Pod cache, they perform an in-place update on the selected Pod, first adding the code extracted from the code cache system to each Pod and then starting the runtime. Once the runtime is ready, the Pods assigned to the workloads corresponding to Fn-1 and Fn-n can run the corresponding functions normally, achieving their desired functionality.

[0146] This disclosure provides a pooled cache implementation solution suitable for multi-tenant isolation, in which cached Pod resources are isolated from the Pods that actually run functions or applications, so that the two Pods belong to different workloads.

[0147] In this way, Pods assigned to functions or applications from the pool will be out of the management of the K8s backend pool workload.

[0148] On the other hand, the present disclosure provides a general pooled backend workload (first workload), which utilizes the backend workload resource quota VPA (vertical automatic scaling of Pod) capability to achieve multi-level resource pooling allocation without Pod reconstruction.

[0149] This not only saves pooled resources, but also improves the pool Pod's ability to respond to requests of different resource specifications, thereby improving the pool's resource utilization and cache hit rate.

[0150] In the original Kubernetes Pod production process, the Pod creation process needs to go through multiple stages: scheduling, Pod production, container image download, container volume mounting, container startup, etc. During the entire Pod production process, it is greatly affected by the infrastructure scheduling strategy and resource state, resulting in a long time for Pod creation.

[0151] Through the Pod pooling scheme of the present disclosure, the Pod container runtime can be modified to create and start the container in the Pod in advance. When a Pod needs to be created, the Pod in the pool can be directly used to quickly implement the service and save the time required for Pod creation.

[0152] The present disclosure uses a custom pooling workload implementation. Based on the implementation of the Pod cache, after the cached Pod is attributed to different functions or applications, the Pod can be moved from the original cache pool workload to the function or application's own workload for management.

[0153] In addition, the present disclosure also optimizes the pool cache level to provide a multi-level resource specification cache pool. Pods of different resource specifications can be moved to the matching pool through VPA to maximize the utilization of the backend pool resources.

[0154] By using the Pod pooling cache scheme of the present disclosure, the pooled Pod will be moved to the function or application's own workload for management after being bound to the specific function or application, thereby being separated from the original pool workload responsible for creating the Pod.

[0155] Thus, the cached Pod resources in the pool and the Pod that actually runs the function or application are isolated, and the cached Pod in the pool and the Pod that actually participates in the function or application logic are attributed to different workloads.

[0156] This scheme can be applied to a multi-tenant scenario. That is, multiple tenants can share the Pod cache pool maintained by the first workload, and each tenant can create a second workload. Once a tenant needs resources, the Pod allocated from the Pod cache pool can be attributed to the tenant's own workload, and they do not affect each other.

[0157] Thus, the isolation between Pods of different tenants can be maintained, and the operation and maintenance flexibility of the backend shared pool can be greatly improved.

[0158] On the other hand, this multi-tenant architecture can maximize the utilization of the elastic difference between tenants to achieve the most efficient use of elastic resources.

[0159] Moreover, such isolation is very important at the real product level.

[0160] In addition, the present disclosure further proposes a Pod lifecycle management solution.

[0161] During the Pod expansion process, the expected expansion time is in milliseconds.

[0162] However, existing Kubernetes lifecycle management solutions are based on readiness and liveness. Readiness is a synchronous detection mechanism that must be maintained throughout the entire pod lifecycle. Therefore, when configuring parameters, the detection interval cannot be set very quickly, typically with a granularity of 5 to 10 seconds. As a result, existing Kubernetes lifecycle management solutions are not suitable for state changes with millisecond granularity, and therefore cannot support millisecond-level capacity expansion requirements.

[0163] The Pod lifecycle management solution proposed in this paper does not rely on the readiness of the Pod, but uses its own maintained state, which can achieve millisecond-level Pod expansion.

[0164] The Pod lifecycle management solution proposed in this disclosure manages the Pod status from the perspective of the second workload, including selecting a Pod from the Pod cache pool, starting the container runtime, and moving to the Pod call lifecycle of the second workload.

[0165] Figure 9 The diagram schematically illustrates the Pod lifecycle management solution adopted by the container group deployment method according to the present disclosure.

[0166] like Figure 9 As shown, the Pod lifecycle management solution disclosed in this disclosure can maintain four states: binding success, startup success, failure, and deletion.

[0167] First, the binding success status is described.

[0168] With the container group deployment solution disclosed herein, the second workload corresponding to a function or application does not need to create a Pod from scratch. Instead, it selects and calls a pre-created and cached Pod from the Pod cache maintained by the first workload. As previously mentioned, the first part (first phase) of the container runtime, independent of the function or application, can be started while the Pod is still in the Pod cache.

[0169] Once a Pod is selected from the Pod cache pool for a function or application and bound to the second workload corresponding to the function or application, the Pod enters the successful binding state. Binding establishes a binding relationship between a Pod and a function or application, setting the Pod to correspond to or belong to a specific function or application, or assigning or scheduling the Pod to a specific function or application.

[0170] The successful binding status indicates that the desired Pod has been selected from the Pod cache and bound to the second workload. After detecting this status, the workload agent starts the second part (second phase) of the container runtime related to the function or application in the corresponding Pod based on the binding information.

[0171] In other words, the successful binding status indicates that the Pod, while still in the Pod pool, has been bound to the function or application requesting scaling, but has not yet been moved to the secondary workload. In other words, the Pod has been successfully scheduled in the pool, and a Pod has been prepared for this specific function or application. In other words, a Pod has been selected from the Pod pool, assigned to the function or application, and bound.

[0172] From the perspective of a function or application, the successful binding status indicates that a Pod has been "created" for that function or application. This Pod has actually been created and cached in the Pod cache pool. A Pod has been selected for scheduling. From the perspective of the function or application, it appears as if a Pod has been "created" for it.

[0173] Before moving a Pod, you need to know where to move it. Therefore, a Pod can only be moved if it is in a successfully bound state.

[0174] Furthermore, a successful binding status indicates that a Pod resource has been found in the Pod cache pool and can be allocated to the second workload corresponding to the function or application requesting capacity expansion. Logically, a Pod is created for the second workload corresponding to the function or application instantly.

[0175] It's understandable that with the Pod cache deployment solution disclosed herein, if the first workload has already maintained a Pod cache pool, "creating" a Pod for the second workload simply means binding it. The previously complex and time-consuming Pod creation process can now be completed instantly. Subsequent operations involve starting the container runtime and moving the Pod.

[0176] On the other hand, pods can be moved as soon as they are bound to a function or application. Binding and moving can be parallel, asynchronous processes. Here, the successful binding status can be understood as an event, and the move begins after the event occurs. In other words, an event-driven mechanism can be used, with the "successful binding" event driving the move operation.

[0177] Next, the startup success status is described.

[0178] Based on the successful binding, when the container runtime starts successfully and enters the ready state, the Pod enters the successful startup state.

[0179] After the Pod enters the startup success state, functions or applications can call the Pod normally to implement corresponding functions.

[0180] As mentioned earlier, while the Pod is still in the cache pool, the first part of the container runtime, which is independent of the function or application, has already been started. Therefore, after successful binding, only the second part of the container runtime, which is related to the function or application, needs to be started. This further reduces the time it takes to reach the successful startup state after successful binding.

[0181] Next, the failure state is described.

[0182] If the container runtime of a Pod in the successful binding state fails to start, or the container runtime startup times out, the Pod enters the failed state. For Pods that enter the failed state due to container runtime startup failure / timeout, you can return to the successful binding state and try to restart the container runtime.

[0183] On the other hand, if an error occurs during the runtime of a container in a Pod that is in the Startup Success state, the Pod will also enter the Failed state. After the failure is eliminated and the container is restored to the Ready state, it can be restored to the Startup Success state.

[0184] Next, the deletion state is described.

[0185] If a Pod that is successfully bound to the second workload (binding success state) fails to start the container runtime multiple times, for example, if the set startup failure / start timeout limit is exceeded, or if the Pod enters the failed state due to an error in the container runtime and the fault cannot be eliminated, the Pod can be marked as deleted so that the Pod can be deleted and rebuilt.

[0186] When the Pod is deleted, a new Pod instance can be cold-started and the cold-started new Pod instance can enter the binding success state.

[0187] The "delete" state here indicates that the Pod is about to be deleted. The specific deletion operation can be performed by, for example, the transport workload agent after detecting the deletion state. In this case, the Pod can be deleted first in the transport workload, and then deleted from the secondary workload and / or Pod cache pool.

[0188] Here, you can not only delete the Pod from the second workload corresponding to the function or application, but also delete the Pod from the Pod cache pool. In other words, you can completely destroy the Pod.

[0189] After a Pod is destroyed, there are two optional processing modes for capacity expansion requests for a function or application: one mode, such as "Fallback_To_Cold_Boot", in which the second workload of the function or application performs a cold start and adds a Pod to itself; the other mode, such as "Always_From_CachePool", in which the second workload of the function or application continues to search for available cached Pods in the Pod cache pool.

[0190] Here, these state changes and Pod movement are two parallel processes. As mentioned earlier, Pod movement requires that the Pod be marked as successfully bound (i.e., the function or application to which the Pod belongs has been determined). Furthermore, for some special functions or applications, it can be configured to require the Pod to be in the successfully started state (the second part of the container runtime associated with the function or application has successfully started) before it can be moved.

[0191] This way, the lifecycle of the Pod can be maintained from the time it is in the cache state to the time it is moved into the second workload corresponding to the function or application.

[0192] For example, after obtaining a Pod from the Pod cache pool, you can determine whether the Pod is available without waiting for the Pod to be fully moved to the second workload. Pod availability means that the Pod is successfully bound and the second component associated with the function or application is successfully started during the container runtime. In other words, the present disclosure can separate the availability determination and Pod movement into two independent logic processes.

[0193] Typically, you can first determine whether the Pod is available and then complete the Pod move.

[0194] Figure 10 A schematic diagram of the structure of a computing device that can be used to implement the above container group deployment method according to an embodiment of the present invention is shown.

[0195] See also Figure 10 , the computing device 1000 includes a memory 1010 and a processor 1020 .

[0196] The processor 1020 can be a single core processor or a multiple core processor. In some embodiments, the processor 1020 can include a general purpose processor and one or more special purpose processors such as graphics processors (GPUs), digital signal processors (DSPs), and the like. In some embodiments, the processor 1020 can be implemented using a custom circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0197] The memory 1010 can include various types of storage units such as a system memory, a read-only memory (ROM), and a permanent storage device. The ROM can store static data or instructions that the processor 1020 or other modules of the computer need. The permanent storage device can be a read and write memory device. The permanent storage device can be a non-volatile storage device that does not lose stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (e.g., a magnetic or optical disk, a flash memory) as a permanent storage device. In other embodiments, the permanent storage device can be a removable storage device (e.g., a floppy disk, an optical disk). The system memory can be a read and write memory device or a volatile read and write memory device such as a dynamic random access memory. The system memory can store some or all of the instructions and data that the processor needs at runtime. In addition, the memory 1010 can include a combination of any computer readable storage media including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks and / or optical disks. In some embodiments, the memory 1010 can include a read and / or write removable storage device such as a compact disk (CD), a read-only digital versatile disk (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disk, an ultra density disk, a flash memory card (e.g., SD card, min SD card, Micro-SD card, and the like), a magnetic floppy disk, and the like. The computer readable storage media does not include carrier waves and transitory electronic signals propagating through a wire or cable or a wireless transmission.

[0198] The memory 1010 stores executable code that, when processed by the processor 1020, can cause the processor 1020 to perform the container group deployment method described above.

[0199] The container group deployment scheme according to the present application has been described in detail above with reference to the accompanying drawings.

[0200] Furthermore, the method according to the present application can also be implemented as a computer program or computer program product comprising computer program code instructions for executing the above steps defined in the above method of the present application.

[0201] Alternatively, the present application can also be implemented as a non-transitory machine readable storage medium (or computer readable storage medium, or machine readable storage medium) having stored thereon executable code (or computer program, or computer instruction code) which, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform the steps of the above method according to the present application.

[0202] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or combinations of both.

[0203] The flow diagrams and block diagrams in the drawings are representative of the architecture, functionality, and operation of possible implementations of systems and methods according to the present application. In this regard, each block in the flow diagrams and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0204] Embodiments of the present application have been described above, and the description is intended to be illustrative of the embodiments of the present application and not exhaustive, and is not limited to the embodiments disclosed. Numerous modifications and variations are possible in light of the above teachings without departing from the scope and spirit of the described embodiments. The choice of words in this document is intended to best explain the principles of the embodiments, practical application, or improvement over the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A container group deployment method, comprising: The first workload maintains a container group cache pool, in which at least one container group is cached; as well as In response to a request to create a second workload or to expand the capacity of the second workload, select a container group from the container group cache pool, bind the obtained container group to the second workload, and move the container group to the second workload; After binding the obtained container group to the second workload, the method further includes: maintaining the status of the container group, wherein the status includes: binding success, startup success, failure, and deletion, wherein: The successful binding status indicates that the container group has been selected from the container group cache pool and bound to the second workload; The successful startup status indicates that the container runtime of the successfully bound container group has successfully started and entered the ready state; The failed state indicates that the container runtime of a successfully bound container group failed to start or timed out, or that the container runtime of a successfully started container group encountered an error. A container group that entered the failed state due to a container runtime failure or timeout will return to the successful binding state to retry starting the container runtime. A container group that entered the failed state due to a container runtime error will return to the successful start state after the container runtime resolves the problem and returns to the ready state. The Deletion state indicates that the container group is about to be deleted. A container group enters the Deletion state if the container runtime of a successfully bound container group fails to start successfully multiple times, or if the container runtime of a failed container group fails to eliminate the fault.

2. The method according to claim 1, wherein The first workload maintains multiple container group cache pools. The container groups cached in each container group cache pool have the same resource configuration specifications, and the container groups cached in different container group cache pools have different resource configuration specifications. The step of selecting a container group from the container group cache pool includes: selecting a container group from the container group cache pool that matches the second workload according to a requirement of the second workload on a container group resource configuration specification.

3. The method according to claim 2, wherein: When the container groups in one container group cache pool are consumed at a faster rate, container groups are allocated from other container group cache pools to supplement the one container group cache pool.

4. The method according to claim 1, wherein The first workload is a generic workload constructed for functions or applications using the same image; and / or The second workload is a workload corresponding to a function or application that is created in response to creation of the function or application, and the function or application uses the same image.

5. The method according to claim 4, further comprising: When the container group is in the container group cache pool, start in advance a first portion of the container runtime that is not related to the function or application in the container group; and / or In response to the container group being bound to the second workload, starting a second portion of the container runtime associated with the function or application corresponding to the second workload in the container group; and / or After the container group in the container group cache pool starts the first part of the container runtime that is not related to the function or application in advance, the containers in the container group are placed in a standby state to wait for the container group to be bound to the second workload so as to start the second part of the container runtime that is related to the function or application corresponding to the second workload.

6. The method according to claim 5, wherein: The first part includes at least one of the following: container startup, container runtime framework initialization part; and / or The second part includes at least one of the following: dynamic injection of environment variables, information or code loading of functions or applications.

7. The method according to claim 5, further comprising: After the container group in the container group cache pool starts the first part of the container runtime in advance, the active state and the ready state of the containers in the container group are controlled: Causes the container to meet the liveness requirement but not the readiness requirement; or This ensures that the container meets both active and ready state requirements.

8. The method according to claim 1, further comprising: In response to the container group being bound to the second workload, performing at least one of the following modifications on the container group: Updating the label of the container group so that the container group selector corresponding to the second workload can retrieve the container group; and The owner information of the container group is modified to the second workload.

9. A container group deployment device, comprising: A cache device, configured to maintain a container group cache pool using a first workload, wherein the container group cache pool caches at least one container group; as well as a migration device, configured to select a container group from the container group cache pool in response to a request to create a second workload or to expand the capacity of the second workload, bind the obtained container group to the second workload, and move the container group to the second workload; After binding the obtained container group to the second workload, the device is further configured to maintain the status of the container group, wherein the status includes: successful binding, successful startup, failure, and deletion. The successful binding status indicates that the container group has been selected from the container group cache pool and bound to the second workload; The successful startup status indicates that the container runtime of the successfully bound container group has successfully started and entered the ready state; The failed state indicates that the container runtime of a successfully bound container group failed to start or timed out, or that the container runtime of a successfully started container group encountered an error. A container group that entered the failed state due to a container runtime failure or timeout will return to the successful binding state to retry starting the container runtime. A container group that entered the failed state due to a container runtime error will return to the successful start state after the container runtime resolves the problem and returns to the ready state. The Deletion state indicates that the container group is about to be deleted. A container group enters the Deletion state if the container runtime of a successfully bound container group fails to start successfully multiple times, or if the container runtime of a failed container group fails to eliminate the fault.

10. The apparatus according to claim 9, further comprising: A first starting device is configured to start, in advance, a first portion of the container runtime that is not related to a function or an application in the container group when the container group is in a container group cache pool; and / or a second starting device for starting, in response to the container group being moved to the second workload, a second portion of the container runtime associated with the function or application corresponding to the second workload in the container group; and / or A standby device is configured to, after a container group in a container group cache pool starts a first portion of a container runtime that is unrelated to a function or application in advance, place the containers in the container group in a standby state, waiting for the container group to be bound to a second workload so as to start a second portion of the container runtime that is related to a function or application corresponding to the second workload, Among them, the first workload is a general workload constructed for functions or applications using the same image, and the second workload is a workload corresponding to the function or application created in response to the creation of the function or application, and the function or application uses the same image.

11. A computing device comprising: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to perform the method according to any one of claims 1 to 8.

12. A computer program product comprising executable codes, which, when executed by a processor of an electronic device, causes the processor to perform the method according to any one of claims 1 to 8.

13. A non-transitory machine-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method according to any one of claims 1 to 8.

Images