Dynamic scaling for workload execution

By using sidecar containers and ephemeral containers for dynamic resource management in a cloud computing environment, the problem of excessively long Pod and container startup times is solved, enabling efficient workload scaling and task execution.

CN115964120BActive Publication Date: 2026-11-17INTERNATIONAL BUSINESS MACHINE CORPORATION
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202211131690.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2021-10-11
Filing Date
2022-09-16
Publication Date
2026-11-17
Estimated Expiration
2042-09-16

AI Technical Summary

Technical Problem

Existing technologies have the problem of excessive time required to start Pods and containers when dynamically scaling workloads in cloud computing environments, which leads to task delays, especially under high instantaneous throughput or pipelined tasks.

Method used

By generating sidecar containers and ephemeral containers, container resources are dynamically created and monitored. Pod managers and horizontal Pod autoscalers are used to scale Pods up or down, avoiding the need for a complete Pod and container initialization process for each task.

Benefits of technology

It reduces workload execution time and improves task processing efficiency, especially under high load conditions, by dynamically adjusting container resources to meet demand.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115964120B_ABST
    Figure CN115964120B_ABST
Patent Text Reader

Abstract

This disclosure relates to dynamic scaling for workload execution. Aspects of the invention include receiving a workload comprising one or more tasks by a controller; generating a first Pod including a first sidecar container; generating one or more ephemeral containers for the first Pod based on the workload and one or more resource allocation metrics of the Pod; executing the one or more tasks in the one or more ephemeral containers; monitoring the one or more resource allocation metrics of the Pod; and generating at least one new ephemeral container in the first Pod based on the one or more resource allocation metrics of the Pod and the workload.
Need to check novelty before this filing date? Find Prior Art

Description

Background Technology

[0001] This invention relates generally to data management, and more specifically to dynamically scaling workload execution in cloud environments.

[0002] Cloud computing refers to the practice of delivering information computing services (i.e., cloud services) using a network of remote servers hosted on a public network (e.g., the Internet) instead of on local servers. The network architecture (e.g., a virtualized information processing environment including hardware and software) through which these cloud services are provided to service consumers (i.e., cloud service consumers) is referred to as a "cloud," and can be a public cloud (e.g., cloud services publicly offered to cloud service consumers), a private cloud (e.g., a private network or data center that provides cloud services only to a designated group of cloud service consumers within an enterprise), a community cloud (e.g., a collection of cloud services publicly offered to a limited set of cloud service consumers (e.g., to an organization with a specific state / region or set of states / regions), a dedicated / hosted private cloud, or other emerging cloud service delivery models. The fundamental intent of cloud computing is to provide cloud service consumers with easy, scalable access to computing resources and information technology (IT) services.

[0003] Cloud services can be broadly categorized into four types: Infrastructure as a Service (IaaS), Platform as a Service (PaaS), Software as a Service (SaaS), and Managed Services. Infrastructure as a Service refers to the virtualized computing infrastructure that provides cloud services (e.g., virtual server space, network connectivity, bandwidth, IP addresses, load balancers, etc.). Platform as a Service in the cloud refers to a set of software and product development tools hosted on the cloud, enabling developers (i.e., a type of cloud service consumer) to build applications and services using the cloud. Software as a Service refers to applications hosted and available on demand by cloud service consumers via the cloud. Managed Services refer to services such as backup management, remote system management, application management, and security services, which are enabled by the managed service provider for any cloud service.

[0004] Generally, cloud services have three distinct characteristics that differentiate them from traditional managed services. The first is that they are sold to service consumers on demand (e.g., by the minute or the hour). The second is that they are dynamic (e.g., service consumers can have as much or as little service as they want at any given time). The third characteristic, which applies specifically to public clouds rather than private or hybrid clouds, is that the service is entirely managed by the cloud service provider (e.g., service consumers only need appropriately equipped client devices and network connectivity). This third feature is particularly relevant to public clouds. However, private clouds can be managed by an internal IT department or through an ITO (IT Outsourcing) contract. In these examples, the I&O (Infrastructure and Operations) administrator acts as the cloud provider, and therefore, this third feature will have similar relevance.

[0005] Business and IT users are rapidly adopting the cloud as a way to make their organizations more efficient and cost-effective. With this opportunity comes new pain points and significant risks for businesses. For example, business users are quickly investing in their own cloud capabilities (e.g., IaaS, PaaS, and SaaS) to meet business needs, while application developers want to move quickly without involving IT to provide tools and environments. These actions are posing a significant threat to IT management, which is concerned about factors such as management costs, billing, capacity, and resources resulting from unrestricted / unplanned cloud expansion. Summary of the Invention

[0006] Embodiments of the present invention relate to dynamically scaling workload execution in a cloud environment. A non-limiting example computer implementation of a method includes receiving a workload comprising one or more tasks by a controller; generating a first Pod including a first sidecar container; generating one or more ephemeral containers for the first Pod based on the workload and one or more resource allocation metrics of the Pod; executing the one or more tasks in the one or more ephemeral containers; monitoring the one or more resource allocation metrics of the Pod; and generating at least one new ephemeral container in the first Pod based on the one or more resource allocation metrics of the Pod and the workload.

[0007] Other embodiments of the present invention implement the features of the above-described method in computer systems and computer program products.

[0008] Additional technical features and benefits are achieved through the technology of this invention. Embodiments and aspects of the invention are described in detail herein and are considered part of the claimed subject matter. For a better understanding, refer to the specific embodiments and accompanying drawings. Attached Figure Description

[0009] The details of the exclusive rights described herein are specifically pointed out and clearly claimed in the claims at the conclusion of this specification. The foregoing and other features and advantages of embodiments of the invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: Figure 1 A cloud computing environment according to one or more embodiments of the present invention is shown; Figure 2 An abstract model layer according to one or more embodiments of the present invention is shown; Figure 3 A block diagram of a computer system for implementing one or more embodiments of the present invention is shown; Figure 4 The diagram shows compute nodes using full virtualization and OS-level virtualization; Figure 5 The diagram shows compute nodes using full virtualization and OS-level virtualization; Figure 6 A block diagram of a Kubernetes Pod according to one or more embodiments of the present invention is shown; Figure 7 A block diagram of a Pod manager resource for managing Pods according to one or more embodiments of the present invention is shown; Figure 8 A block diagram of a Pod manager and a horizontal Pod autoscaler according to one or more embodiments of the present invention is shown; Figure 9 A block diagram of a controller module communicating with a Pod manager and a workload queue according to one or more embodiments of the present invention is shown; Figure 10 A system for dynamically scaling workload execution in a cloud environment is illustrated according to one or more embodiments of the present invention; and Figure 11 A flowchart is shown for a method for dynamically scaling workload execution in a cloud environment according to one or more embodiments of the present invention.

[0010] The figures shown herein are illustrative. Many variations of the figures or operations described herein are possible without departing from the spirit of the invention. For example, actions may be performed in a different order, or actions may be added, deleted, or modified. Furthermore, the term "coupled" and its variations describe a communication path between two elements and do not imply a direct connection between the elements without any intermediate elements / connections between them. All such variations are considered part of the specification. Detailed Implementation

[0011] One or more embodiments of the present invention provide systems and methods for dynamically scaling workload execution in a cloud environment. A cloud environment can be a platform for managing containerized workloads and services, such as Kubernetes. Kubernetes represents the compute, networking, and storage infrastructure for orchestrating user workloads. Aspects of the present invention provide systems and methods for scaling execution environments (Pods) in a Kubernetes platform. A Pod is the basic structural unit of Kubernetes—the smallest and simplest unit created and deployed in the Kubernetes object model. A Pod represents a running process in a distributed computing environment (e.g., a computer cluster). A Pod encapsulates an application container (or, in some cases, multiple containers) and includes storage resources, a unique network IP, and options for managing how the container should run. A Pod represents a deployment unit: a single instance of an application in Kubernetes, which may consist of a single container or a small number of tightly coupled containers sharing resources. A further aspect of the present invention allows multiple containers to be created within a Pod based on resource allocation metrics monitored for the Pod. These multiple containers allow tasks to be executed in parallel without continuously generating new Pods and / or containers for each task. Furthermore, for larger workloads, additional Pods with these multiple containers can be generated, and these larger workloads are managed by a Pod manager and controller to execute multiple tasks in parallel. Pods and / or containers can be created and / or terminated when resource availability changes. This allows for maximizing task execution by scaling Pods and containers to execute tasks in parallel.

[0012] An emerging information technology (IT) delivery model is cloud computing, through which shared resources, software, and information are provided on demand to computers and other devices via the internet. Cloud computing can significantly reduce IT costs and complexity while improving workload optimization and service delivery. In this approach, application instances can be hosted and made available from internet-based resources accessible via a regular web browser over HTTP. An example application could be one that provides a set of common messaging functionalities such as email, calendar, contact management, and instant messaging. Users then access the service directly over the internet. Using this service, businesses place their email, calendar, and / or collaboration infrastructure in the cloud, and end users use appropriate clients to access their emails or perform calendar operations.

[0013] Cloud computing resources are typically housed in large server clusters that run one or more web applications. These clusters typically employ virtualization architectures, where applications run within virtual servers, or so-called "virtual machines" (VMs), which are mapped to physical servers within a data center facility. VMs usually run on top of a hypervisor, which controls the allocation of physical resources to the VMs. Modern hypervisors frequently utilize hardware-assisted virtualization, which provides efficient and full virtualization by leveraging virtualization-specific hardware capabilities (primarily derived from the host CPU).

[0014] Operating system (OS) level virtualization is another virtualization approach. OS-level virtualization allows for the partitioning of a computer's resources through support for multiple isolated user-space instances (often referred to as containers) via the operating system kernel. Therefore, this virtualization approach is often called container-based virtualization. To the end user, a container may not be indistinguishable from a separate machine. Container-based virtualization is widely used in cloud environments. For example, in many of today's data centers, workloads run inside containers. Containers offer greater flexibility and orchestration to adapt to changing workload demands. Many technologies are used in container-based virtualization. These technologies include, for example, namespaces and cgroups, discussed below.

[0015] In a typical container cloud computing environment, the host machine runs an operating system, such as the Linux kernel. As mentioned above, the term "container" refers to an OS-level virtualization mechanism used to run isolated computing workloads (containers) on a controlling host using a single operating system kernel. This approach effectively divides resources managed by a single operating system into isolated groups to better balance conflicting demands for resource usage between these groups. Compared to other types of virtualization, it requires neither instruction-level emulation nor just-in-time compilation. Furthermore, containers can run instructions native to the core CPU without any special interpretation mechanisms. By providing a way to create and import containers, the operating system gives applications the illusion of running on separate machines while simultaneously sharing many underlying resources.

[0016] It should be understood that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings described herein is not limited to a cloud computing environment. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0017] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with service providers. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0018] The characteristics are as follows: On-demand self-service: Cloud consumers can automatically and unilaterally configure computing power, such as server time and network storage, as needed, without human interaction with the service provider.

[0019] Extensive network access: Capabilities are available through networks and accessed via standard mechanisms that facilitate the use of heterogeneous thin client platforms or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0020] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated as needed. There is a sense of location independence because consumers typically do not have control or knowledge of the exact location of the resources provided, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0021] Rapid elasticity: Capacity can be provided quickly and flexibly (in some cases, automatically) to shrink rapidly and expand rapidly. For consumers, the available supply capacity often appears unlimited and can be purchased in any quantity at any time.

[0022] Measurement services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the service type (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both service providers and consumers.

[0023] Infrastructure as a Service (IaaS): This provides consumers with the capability to offer processing, storage, networking, and other basic computing resources, where consumers can deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but rather have control over the operating system, storage, deployed applications, and potentially limited control over selected networking components (e.g., host firewalls).

[0024] The deployment model is as follows: Private cloud: A cloud infrastructure that operates solely for an organization. It can be managed by the organization or a third party and can exist on-site or off-site.

[0025] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It can be managed by an organization or a third party and can exist on-site or off-site.

[0026] Public cloud: Cloud infrastructure that is made available to the general public or large industry groups and is owned by an organization that sells cloud services.

[0027] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardized or proprietary technologies that enable data and applications to be ported together (e.g., cloud bursts for load balancing between clouds).

[0028] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure comprising a network of interconnected nodes.

[0029] See now Figure 1 An illustrative cloud computing environment 50 is shown. As illustrated, the cloud computing environment 50 includes one or more cloud computing nodes 10 to which local computing devices used by cloud consumers can communicate, such as, for example, personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N. The nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 50 to provide infrastructure, platforms, and / or software as services that cloud consumers do not need to maintain on their local computing devices. It should be understood that... Figure 1 The types of computing devices 54A-N shown are intended to be illustrative only, and computing node 10 and cloud computing environment 50 can communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0030] See now Figure 2 This demonstrates the 50 (cloud computing environment) Figure 1 This provides a set of functional abstractions. It should be understood beforehand. Figure 2 The components, layers, and functions shown are intended to be illustrative only, and embodiments of the invention are not limited thereto. As described, the following layers and corresponding functions are provided: The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: a mainframe 61; a RISC (Reduced Instruction Set Computer) based server 62; a server 63; a blade server 64; a storage device 65; and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0031] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75.

[0032] In one example, management layer 80 may provide the following functionalities: Resource Provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and Pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and bills or invoices for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User Portal 83 provides access to the cloud computing environment for consumers and system administrators. Service Level Management 84 provides cloud resource allocation and management to ensure that required service levels are met. Service Level Agreement (SLA) Planning and Fulfillment 85 provides pre-scheduling and procurement of cloud resources, anticipating future requirements for those resources according to the SLA.

[0033] Workload layer 90 provides examples of functionalities that can be leveraged in a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analytics and processing 94; transaction processing 95; and dynamic scaling for workload execution 96.

[0034] refer to Figure 3 An embodiment of a processing system 300 for implementing the teaching content of this document is shown. The processing system 300 can be implemented as a computing node 10. Furthermore, some or all of the functions of the elements of the processing system 300 can be implemented as a node 10 of a cloud 50 (in...). Figure 1 and 2(As shown in the diagram). Cloud computing node 10 is merely one example of a suitable cloud computing node and is not intended to impose any limitation on the use or scope of functionality of the embodiments of the invention described herein. In this embodiment, processing system 300 has one or more central processing units (processors) 21a, 21b, 21c, etc. (collectively or commonly referred to as processor 21). In one or more embodiments, each processor 21 may include a Reduced Instruction Set Computer (RISC) microprocessor. Processor 21 is coupled to system memory 34 and various other components via system bus 33. Read-only memory (ROM) 22 is coupled to system bus 33 and may include a Basic Input / Output System (BIOS) that controls certain basic functions of processing system 300.

[0035] Figure 3 Further shown are input / output (I / O) adapter 27 and network adapter 26 coupled to system bus 33. I / O adapter 27 may be a Small Computer System Interface (SCSI) adapter that communicates with hard disk 23 and / or tape storage device 25 or any other similar component. I / O adapter 27, hard disk 23, and tape storage device 25 are collectively referred to herein as mass storage device 24. Operating system 40 for execution on processing system 300 may be stored in mass storage device 24. Network adapter 26 interconnects system bus 33 with external network 36, enabling processing system 300 to communicate with other such systems. Screen (e.g., display monitor) 35 is connected to system bus 33 via display adapter 32, which may include a graphics adapter and video controller for improving performance in graphics-intensive applications. In one embodiment, adapters 27, 26, and 32 may be connected to one or more I / O buses that are connected to system bus 33 via an intermediate bus bridge (not shown). A suitable I / O bus for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically includes common protocols such as Peripheral Component Interconnect (PCI). Additional input / output devices are shown connected to the system bus 33 via user interface adapter 28 and display adapter 32. Keyboard 29, mouse 30, and speaker 31 are all interconnected to the system bus 33 via user interface adapter 28, which may include, for example, a super I / O chip integrating multiple device adapters into a single integrated circuit.

[0036] In an exemplary embodiment, the processing system 300 includes a graphics processing unit 41. The graphics processing unit 41 is a dedicated electronic circuit designed to manipulate and modify memory to accelerate the creation of images in a frame buffer intended for output to a display. Generally, the graphics processing unit 41 is highly efficient in manipulating computer graphics and image processing and has a highly parallel architecture, making it more efficient than a general-purpose CPU for algorithms in which large blocks of data are processed in parallel.

[0037] Therefore, as Figure 3 The processing system 300, configured as described above, includes processing capabilities in the form of a processor 21, storage capabilities including system memory 34 and mass storage device 24, input devices such as a keyboard 29 and a mouse 30, and output capabilities including a speaker 31 and a display 35. In one embodiment, a portion of the system memory 34 and mass storage device 24 jointly store coordinated data. Figure 3 The operating system shows the functions of each component.

[0038] Figure 4 and 5 Compute nodes 400 and 500, using full virtualization and OS-level virtualization respectively, are shown. Some embodiments of the invention can be used with any of these types of compute nodes, as well as in hybrid environments with combinations of these compute nodes spanning single or multiple compute nodes.

[0039] like Figure 4 As shown, each of the compute nodes 400 includes hardware 406, which may include a processor (or CPU) 407, memory 408, a network interface card (NIC) 409, and a disk drive 410. The disk drive 410 may include a solid-state drive or a hard disk drive, or some combination of both. In terms of hardware, the compute node 400 runs a host operating system 405. The compute node 400 also includes a hypervisor 404 to share and manage the hardware 406, thereby allowing multiple distinct environments isolated from each other to be executed on the same physical machine 400. The hypervisor 404 may use hardware-assisted virtualization to provide efficient and full virtualization by utilizing virtualization-specific hardware capabilities primarily derived from the host CPU 407. Each compute node 400 includes one or more virtual machines 401, each virtual machine including a guest operating system 403 and one or more applications 402 running on the guest operating system 403.

[0040] Similarly, such as Figure 5As shown, each of the compute nodes 500 includes hardware 506, which may include a processor (or CPU) 507, memory 508, a network interface card (NIC) 509, and a disk drive 510. The disk drive 510 may include a solid-state drive or a hard disk drive, or some combination of both. In terms of hardware, the compute node 500 runs a host operating system 505. Each compute node 500 includes one or more containers 501, where each container includes one or more applications 502.

[0041] According to some embodiments, compute node 500 may include one or more Pods 503, each Pod including one or more containers 501, and each container 501 including one or more applications 502. For example, in Kubernetes, containers run within Pods.

[0042] Kubernetes is a portable, scalable, open-source platform for managing containerized workloads and services. It promotes both declarative configuration and automation. Kubernetes orchestrates compute, networking, and storage infrastructure on behalf of user workloads. Kubernetes is an instance of an orchestration framework. Containerized workloads across multiple compute nodes can be managed by a Container Orchestration Manager (COM). An example of a Container Orchestration Manager (COM) is the Kubernetes Master. Several binary components (e.g., the Master, Node, and Addons) are used to deliver a functioning Kubernetes cluster. The Master provides the control plane of the Kubernetes cluster (also known as the "Kubernetes control plane"). The Master can include, but is not limited to, the Kube API server, etcd, the Kube scheduler, the Kube controller manager, and the cloud controller manager. The Master makes global decisions about the Kubernetes cluster. For example, the Master handles scheduling. Furthermore, the Master is utilized in detecting and responding to cluster events. For example, when the "replica" field of a replication controller is not satisfied, the Master is responsible for starting a new Pod. The Master can run on any machine in the cluster. However, setup scripts typically start all Masters on the same machine and do not run user containers on that machine.

[0043] Node components run on each compute node in a Kubernetes cluster. Node components are responsible for maintaining running Pods and providing the Kubernetes runtime environment. Node components can include, but are not limited to, Kubelets, Kube brokers, and container runtimes. A Kubelet is a broker that ensures containers are running within Pods. Kubelets ensure that containers specified in a set of PodSpecs provided through various mechanisms are running and healthy. A Kube broker is a network broker. Kube brokers implement Kubernetes service abstractions by maintaining network rules and performing connection forwarding on compute nodes.

[0044] A container runtime is the software responsible for running containers. More specifically, a container runtime is a node component that handles the lifecycle of containers. The container runtime implements fundamental concepts such as creating, starting, stopping, and removing container workloads. Kubernetes supports multiple runtimes, including but not limited to Docker, containerd, CRI-O, and rkt1et.

[0045] More generally, Kubernetes supports any implementation of the Container Runtime Interface (CRI) provided by Kubernetes. CRI makes it easy to plug in various container runtimes. Prior to the introduction of CRI in Kubernetes 1.5, only the default Docker image library and its default OCI-compliant runtime, runC, were used. The Open Container Initiative (OCI) created a runtime specification that details the API for OCI-compliant container runtimes. runC, runV, and Intel's Transparent Containers (also known as "cc-runtime") are examples of OCI-compliant container runtimes. runC has built-in support for CRIU—the userspace checkpoint / recovery described below for inspecting and recovering containers. runV is OCI's hypervisor-based Docker runtime; runV is also known as "Hyper runV".

[0046] The CRI runtime operates at a higher level of abstraction and should not be confused with OCI-compatible runtimes. The CRI runtime is also known as a "CRIshim." CRIshims include cri-containerd, CRI-O, dockersshim, and frakti. Some CRIshims (e.g., CRI-containerd, CRI-O, and dockersshim) invoke OCI-compatible runtimes, while others (e.g., frakti) are all-in-one solutions.

[0047] At least some CRIshims support multiple runtimes running on a single compute node. For example, CRI-O supports the concept of trusted and untrusted sandboxes. In Kubernetes, a mix of one or more VM-based Pods and one or more cgroup / namespace-based Pods can run on a single compute node based on Pod annotations and the default CRI-O configuration. Containers running inside VM-based Pods can be isolated and managed via namespaces and cgroups, similar to what runC does.

[0048] Addons are Pods and services responsible for implementing cluster features. Addons include, but are not limited to, cluster DNS (i.e., DNS servers that provide DNS records for Kubernetes services), dashboards (i.e., a web-based UI for Kubernetes clusters that allows users to manage and troubleshoot applications running in the cluster as well as the cluster itself), container resource monitoring (i.e., responsible for logging general time-series metrics about containers in a central database, and providing a UI for browsing the data recorded in that database), and cluster-level logging (i.e., responsible for saving container logs to a central log store with a search / browse interface).

[0049] Turning now to an overview of the technology more specifically related to various aspects of this invention, a Kubernetes platform for managing containerized workloads and services requires a startup process to launch Pods comprising one or more containers. This Pod startup process involves three steps. The first step is an initialization / preparation step. The second step is the creation of a sandboxed container via the Container Runtime Interface (CRI). The final (third) step is the creation of the actual container. The first and second steps can take a significant amount of time (e.g., 2-3 seconds). For each task, the Kubernetes Pod must go through this process end-to-end, which can take a considerable amount of time for running short workloads. For example, there are two exemplary scenarios where this timeframe can cause problems. The first scenario is when there is high transient throughput in a cloud environment where one or more Pods need to be launched in a Kubernetes cluster for each user logic. This results in a large number of similar Pods being launched. These similar Pods can be used for similar, short workloads, such as calling an API. Starting a Pod in 2-3 seconds or longer represents a significant portion of the overall workload. The second scenario is when a pipeline exists and a step can include multiple tasks, and similar Kubernetes needs to launch Pods for each task. If the number of tasks in the pipeline is large, not all Pods can be launched due to hardware limitations.

[0050] One or more embodiments of the present invention address the aforementioned drawbacks of the prior art by providing systems and processes for dynamically scaling workload execution in a cloud environment. Aspects include dynamically launching multiple containers to execute workloads within containers of a Kubernetes cluster. Resource metrics and statistics can be collected to allocate resources for executing workloads in these Pods and containers. Pods can be scaled up and / or scaled down to other nodes in the Kubernetes cluster based on workload queues. Furthermore, containers can be associated with workloads and monitored as workloads are executed.

[0051] Figure 6 A block diagram of a Kubernetes Pod according to one or more embodiments of the present invention is shown. A Kubernetes Pod represents a running process in a distributed computing environment (e.g., a computer cluster). A Pod encapsulates an application container (or in some cases, multiple containers) and includes storage resources, a unique network IP, and options for managing how the container should run. In a typical configuration, the operation of a Kubernetes Pod 602 must follow the three-step creation process described above for the main container 604. Here, the main container 604 is used to perform the workload tasks. Moreover, as mentioned above, container creation may take 2-3 seconds, resulting in a delay in the execution of tasks (especially tasks that do not require long execution times). In one or more embodiments of the present invention, a Kubernetes Pod 606 is proposed to be created to perform workloads. In this case, the main container operates as a sidecar container 608. The sidecar container 608 prevents the Pod 606 from exiting and transfers results by executing, for example, a lightweight loop. Furthermore, one or more ephemeral containers 610 are spawned within the Pod 606. The workload and associated tasks can be executed on these ephemeral containers 610, while the sidecar container 608 prevents the Pod from exiting. Using the sidecar container 608 avoids the need to initialize and sandbox a new container for each task, which reduces workload execution time.

[0052] In one or more embodiments of the present invention, a Kubernetes Pod 606 can be managed by a Pod management module that tracks resource metrics for ephemeral containers. Currently, setting resources for ephemeral containers is not allowed. However, in one or more embodiments, the Pod manager can dynamically create ephemeral containers through resource limits to avoid exceeding Pod resource limits and for resources to be grabbed by one or more specific containers. Figure 7A block diagram illustrating Pod manager resources for managing Pods according to one or more embodiments of the present invention is described. In one or more embodiments, Pod manager 702 may obtain and / or receive resource utilization metrics from resource metric API 730. Pod manager 702 may calculate how many containers can be launched, which may be based on analysis of available resources regarding workloads and / or user-predefined metrics. Pod manager 702 then creates and terminates containers with allocated resources in Cgroup 710. These steps may be repeated as needed for workload execution. Pod manager 702 interacts with Container Runtime Interface (CRI) 704 and Container Runtime Engine 705 to dynamically create and terminate containers 720. As previously described, sidecar container 718 executes, for example, a lightweight loop to prevent Pod 706 from executing. Container 720 may be built using namespaces 708 within the Pod. The Linux kernel has a feature called "namespaces". Linux kernel namespaces (which are the main building blocks of Linux containers) isolate applications within different "user spaces" such as network, process, user, and file system. Namespaces isolate and virtualize system resources used for collections of processes. Some examples of resources that can be virtualized include process IDs, hostnames, user IDs, etc. Namespaces typically refer to namespace types and specific instances of these types. The Linux operating system initializes with a single instance of each namespace type. After initialization, additional namespaces can be created or joined. Furthermore, the Linux kernel provides a feature called "control groups," also known as "cgroups," which allows for the limitation and prioritization of resources (CPU, memory, block I / O, network, etc.). cgroups functionality limits different host resources, such as CPU count and usage, disk performance, memory, and other process limits.

[0053] In one or more embodiments of the present invention, ephemeral containers can be created and terminated within a Pod based on resource allocation and / or workload. Furthermore, in one or more embodiments of the present invention, a Pod can be scaled up and / or shrunk. As the number of ephemeral containers within a Pod increases, Pod resources will approach the limit of no longer creating ephemeral containers within the Pod. In this case, a horizontal Pod autoscaler (HPA) can be used to scale up and / or shrink the Pod. Figure 8A block diagram of a Pod manager and a horizontal Pod autoscaler according to one or more embodiments of the present invention is shown. Pod manager 802 may again receive workload 850 and, using resource metric information collected from resource metric API 830, create Pod 810 with sidecar container 814 and one or more ephemeral containers 812. Based on resource and / or workload queues, Pod manager 802 may scale up or down Pods by triggering HPA 806 through updating custom metric API 804 and configuration. Thus, more Pods 810 may be created to perform workload 850. Further, HPA 806 may scale down when the workload queue is small (below a threshold), terminating one or more Pods 810 to save resources. In some embodiments, workload 850 may be received from a workload queue manager. Replication controller 808 may scale Pods by creating multiple Pods 810. This is referred to as "replication". Replicated Pods may be created and managed as a group through an abstraction called a controller.

[0054] In one or more embodiments of the present invention, multiple containers can be scaled up and / or shrunk. When multiple containers are created, a controller module is needed to monitor the workload and communicate with the Pod manager and workload queue manager to maintain the workload. Figure 9 A block diagram of a controller module communicating with a Pod manager and a workload queue according to one or more embodiments of the present invention is shown. In one or more embodiments, controller 904 may communicate with Pod manager 902, which creates Pod 910 having a sidecar container 914 and one or more ephemeral containers 912. Controller 904 also communicates with queue manager 906, which manages workload queue 908. In one or more embodiments, workload queue 908 may be a Message Queuing Telemetry Transport (MQTT) queue. MQTT queues are exemplary and are not intended to limit the type of queue used herein. In one or more embodiments of the invention, controller 904 obtains and / or receives task definitions from an API server. Controller 904 may place tasks into queue 908 via queue manager 906. Controller 904 then triggers Pod manager 902 to create ephemeral containers 912 and sidecar containers 914. Sidecar container 914 monitors ephemeral containers 912 and sends results to message queue 908. Controller 904 obtains results from message queue 908 and updates tasks accordingly.

[0055] Figure 10 A system for dynamically scaling workload execution in a cloud environment is illustrated according to one or more embodiments of the present invention. System 1000 includes... Figures 6 to 9The system 1000 comprises many components as described herein. Thus, the system 1000 includes a Pod manager 1002, a controller 1004, a queue manager 1006, a resource metrics API 1008, an HPA 1010, a replication controller 1012, a container runtime interface 1014, and a workload / message queue 116. One or more tasks 1018 can be reached through the API server 1024 and sent to the controller 1004. The controller 1004 places tasks into the queue 1016 via the queue manager 1006. The controller 1004 can trigger the Pod manager 1002 to create a ephemeral container 1034 and a sidecar container 1036 in a Pod 1032 for monitoring and sending results to the queue 1016. The resource metrics API 1008 monitors resource metrics for container 1032. The HPA 1010 and the replication controller 1012 can add and / or terminate containers based on tasks, resources, and / or workloads. The Pod manager 1002 manages each Pod 1032 and can add and / or terminate ephemeral containers based on resource allocation metrics 1034.

[0056] In one or more embodiments, system 1000 also includes Kubernetes components such as Etcd 1022 and the node component Kubelet 1026. Etcd is a persistent, lightweight, distributed key-value data store that reliably stores configuration data for the Kubernetes cluster, representing the overall state of the cluster at any given point in time. Kubelet is a broker that ensures containers are running in Pods. Kubelet ensures that containers specified in a set of PodSpecs provided through various mechanisms are running and healthy.

[0057] Figure 11 A flowchart illustrating a method for dynamically scaling workloads in a cloud environment, according to one or more embodiments of the present invention, is shown. For example, it may be performed by one or more servers (such as...) Figure 1The public cloud 120 and / or private cloud 130 shown in the diagram execute at least a portion of method 1100. Method 1100 includes receiving a workload comprising one or more tasks by a controller, as shown in box 1102. In box 1104, method 1100 includes generating a first Pod including a first sidecar container. For example, the first Pod is a Kubernetes Pod. At box 1106, method 1100 includes generating one or more ephemeral containers for the first Pod based on the workload and one or more resource allocation metrics for that Pod. Method 1100 may include executing the one or more tasks within the one or more ephemeral containers at box 1108. The sidecar containers execute, for example, lightweight loops to prevent the Pod from exiting. At box 1110, method 1100 includes monitoring one or more resource allocation metrics for the Pod. The resource allocation metrics may be taken from a resource monitoring API. And at box 1112, method 1100 includes generating at least one new ephemeral container in the first Pod based on one or more resource allocation metrics for the Pod and the workload.

[0058] It may also include other processes. It should be understood that... Figure 11 The processes described herein are illustrative, and other processes may be added or existing processes may be removed, modified or rearranged without departing from the scope and spirit of this disclosure.

[0059] This document describes various embodiments of the invention with reference to the accompanying drawings. Alternative embodiments of the invention may be devised without departing from the scope of the invention. In the following description and drawings, various connections and positional relationships (e.g., above, below, adjacent, etc.) are illustrated between elements. Unless otherwise specified, these connections and / or positional relationships may be direct or indirect, and the aspects of the invention herein are limiting. Therefore, the connection of entities may refer to direct or indirect connections, and the positional relationship between entities may be direct or indirect positional relationships. Furthermore, the various tasks and process steps described herein may be incorporated into a more comprehensive procedure or process with additional steps or functions not described in detail herein.

[0060] One or more of the methods described herein can be implemented using any or a combination of the following techniques, each of which is well known in the art: discrete logic circuit(s) having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having appropriately combined logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0061] For the sake of brevity, this document may or may not describe in detail conventional techniques associated with the manufacture and use of aspects of the invention. Specifically, various aspects of computing systems and specific computer programs used to implement the different technical features described herein are well known. Consequently, for the sake of brevity, many conventional implementation details are only briefly mentioned or omitted entirely herein, without providing well-known system and / or process details.

[0062] In some embodiments, different functions or actions may occur at a given location and / or in combination with the operation of one or more devices or systems. In some embodiments, a portion of a given function or action may be performed at a first device or location, and the remainder of the function or action may be performed at one or more additional devices or locations.

[0063] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to also include the plural forms. It should also be understood that when the terms “comprises” and / or “comprising” are used in this specification, they specify the presence of the stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or combinations thereof.

[0064] All means or steps in the following claims, plus corresponding structures, materials, actions, and equivalents of the functional elements, are intended to include any structure, material, or action for performing the function in conjunction with other claimed elements as specifically claimed. This disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of this disclosure. These embodiments were chosen and described in order to best explain the principles and practical application of this disclosure, and to enable others skilled in the art to understand this disclosure with respect to different embodiments having different modifications suitable for the particular intended use.

[0065] The figures shown herein are illustrative. Many variations may be made to the figures or steps (or operations) described herein without departing from the spirit of this disclosure. For example, actions may be performed in a different order, or actions may be added, deleted, or modified. Furthermore, the term "coupled" describes a signal path between two elements and does not imply a direct connection between elements without intermediate elements / connections. All such variations are considered part of this disclosure.

[0066] The following definitions and abbreviations will be used to interpret the claims and specification. As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having,” “contains,” or “containing,” or any other variations thereof, are intended to cover non-exclusive inclusion. For example, a composition, mixture, process, method, article, or apparatus that comprises a list of elements is not necessarily limited to those elements, but may include other elements not expressly listed or inherent to such composition, mixture, process, method, article, or apparatus.

[0067] Furthermore, the term "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation or design described herein as "exemplary" is not necessarily to be construed as superior to or better than other implementations or designs. The terms "at least one" and "one or more" should be understood to include any integer greater than or equal to one, i.e., one, two, three, four, etc. The term "multiple" should be understood to include any integer greater than or equal to two, i.e., two, three, four, five, etc. The term "connection" can include both indirect "connection" and direct "connection."

[0068] The terms “about,” “substantially,” “approximately,” and their variations are intended to include the degree of error associated with a measurement based on a specific quantity of equipment available at the time of application submission. For example, “about” could include a range of ±8%, 5%, or 2% of a given value.

[0069] This invention can be a system, method, and / or computer program product at any possible level of technical detail integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to execute aspects of the invention.

[0070] Computer-readable storage media can be tangible devices capable of retaining and storing instructions used by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital universal disk (DVD), memory sticks, floppy disks, mechanical encoding devices such as punch cards or protrusions in slots having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0071] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device or to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the corresponding computing / processing device.

[0072] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​(such as Smalltalk, C++, etc.) and procedural programming languages ​​(such as the "C" programming language or similar programming languages). The computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as a standalone software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)) or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuitry in order to perform aspects of this invention.

[0073] This document describes various aspects of the invention with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0074] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner, such that the computer-readable storage medium storing the instructions includes an article of manufacture containing instructions that implement aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0075] These computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, thereby causing these instructions to perform on the computer, other programmable apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0076] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than indicated in the figures. For example, depending on the functions involved, two consecutively shown blocks may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0077] Various embodiments of the invention have been described for illustrative purposes, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, their practical application, or technical improvements over those found in the market, or to enable those skilled in the art to understand the embodiments described herein.

Claims

1. A computer-implemented method for dynamically scaling the execution of a workload, comprising: The controller receives a workload comprising one or more tasks, wherein the one or more tasks are stored in a workload queue; A first Pod is generated, the first Pod including a main container configured as a first sidecar container, wherein the first sidecar container executes a loop to prevent the first Pod from exiting; Within the first Pod, the Pod manager generates one or more ephemeral containers for the first Pod; Executing the one or more tasks in one or more ephemeral containers, wherein executing the one or more tasks includes: Execute the one or more tasks in one or more transient containers; Based on the execution of the ephemeral container, monitor one or more resource allocation metrics used for the first Pod; and Based on the workload queue and the monitoring, the first Pod is scaled using a Pod Manager and a Horizontal Pod Autoscaler (HPA) to perform one or more tasks in the workload queue, wherein scaling the first Pod includes the following steps: The Pod manager dynamically generates one or more additional ephemeral containers in the first Pod based on the one or more resource allocation metrics used for the first Pod and the workload, wherein all ephemeral containers are set with resource limits to avoid exceeding the Pod resource limits; Based on the workload queue and resource allocation metrics, the first Pod is scaled using the HPA, wherein scaling the first Pod when the workload queue exceeds a predefined threshold includes creating an additional Pod with one or more ephemeral containers in the Kubernetes cluster; and The one or more tasks are executed in multiple ephemeral containers.

2. The computer-implemented method according to claim 1, further comprising: At least one of the one or more ephemeral containers in the first Pod is terminated based on the one or more resource allocation metrics.

3. The computer-implemented method according to claim 1, wherein, The workload queue includes a message queue and a telemetry transport queue.

4. The computer-implemented method according to claim 1, wherein, The first Pod includes a Kubernetes Pod.

5. A computer system for dynamically scaling the execution of workloads, comprising: Memory, containing computer-readable instructions; as well as One or more processors are configured to execute the computer-readable instructions, which control the one or more processors to perform the computer-implemented method of any one of claims 1 to 4.

6. A computer program product for dynamically scaling the execution of a workload, comprising program instructions executable by one or more processors to cause the one or more processors to perform a computer-implemented method according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Resource scheduling method and device based on kubernetes, equipment and storage medium

    CN112199194A

  • Task processing method and device, electronic equipment and readable storage medium

    CN112799854A