Dynamic Pod Resource Limit Adjustment Based on Data Analysis

Abstract

Dynamic adjustment of pod resource limits is provided at runtime for a container orchestration platform pod. The process includes the step of deploying a container orchestration platform pod having one or more pod resources in the computing environment, the pod resources having one or more associated pod resource limits. Furthermore, the process includes the step of monitoring the runtime resource usage of the container orchestration platform pod and the step of predicting future resource usage of the container orchestration platform pod using a trained machine learning model. The prediction step uses at least partially the monitored runtime resource usage. Furthermore, the process includes the step of dynamically adjusting the pod resource limits of the one or more pod resource limits of the container orchestration platform pod in the computing environment. The dynamic adjustment step is based on the monitored runtime resource usage and the predicted future resource usage.

Description

Technical Field

[0001] One or more aspects generally relate to facilitating processing within a computing environment, particularly to dynamically adjusting pod resource limits of container orchestration platform pods within a computing environment at runtime.

[0002] Container orchestration platforms and / or tools provide a framework for managing container and microservices architectures in scaling. Container orchestration automates the development, management, scaling, and networking of containers and can be used by enterprises that need to deploy and manage hundreds or even thousands of containers.

[0003] Containers provide microservice-based applications with application deployment units and self-contained execution environments. Containers enable multiple parts of an application to be individually executed as microservices on the same hardware with greater control over individual application components and lifecycles.

[0004] Container orchestration platforms typically support clusters, control planes, agents, and one or more pods. A cluster is a control plane and one or more compute machines, or nodes. The control plane is a collection of processes that control container orchestration platform nodes and is the location where task assignment occurs. An agent is a service that runs on a node, reads container manifests, and ensures that defined containers are started and running. A pod is a group of one or more containers deployed on a single node. Containers within a pod share an IP address, IPC, hostname, and other resources.

Summary of the Invention

[0005] Through the provision of a computer implementation method that facilitates processing within a computing environment, certain shortcomings of the prior art are overcome and additional advantages are provided herein. The computer implementation method includes: deploying a container orchestration platform pod having one or more pod resources in the computing environment. The pod resources are associated with one or more pod resource limits. Furthermore, the computer implementation method includes the steps of monitoring the runtime resource usage of the container orchestration platform pod and predicting future resource usage of the container orchestration platform pod using a trained machine learning model, wherein the predicting step uses at least partially the monitored runtime resource usage. In addition, the computer implementation method includes the step of dynamically adjusting the pod resource limits of the one or more pod resource limits of the container orchestration platform pod in the computing environment. The dynamically adjusting step is based on the monitored runtime resource usage and the predicted future resource usage. Advantageously, processing is facilitated within a container architecture-based computing environment by providing enhanced resource utilization. By dynamically adjusting pod resource limits, cost-effective use of computing resources is provided, along with efficient and dynamic resource utilization, thereby supporting on-demand resource allocation without unnecessary waste. Furthermore, by using computing resources more efficiently in container-based computing environments, processes result in cost savings, a lower carbon footprint, and reduced energy consumption.

[0006] In one implementation, the computer implementation method further includes the step of obtaining a trained machine learning model by training the machine learning model on the historical resource usage data of the container orchestration platform pod. Furthermore, in one embodiment, the computer implementation method further includes the step of using the predicted future resource usage of the container orchestration platform pod to continue training the trained machine learning model. In one example, the machine learning model includes a linear regression model. Advantageously, by using the trained machine learning model to predict the future resource usage of the container orchestration platform pod, it becomes easier to dynamically adjust the pod resource limits of the container orchestration platform pod to provide, for example, cost-effective use of computing resources, effective and dynamic utilization of resources, and support on-demand resource allocation without wasting resources.

[0007] In one implementation, the deployment step includes a step of initializing and deploying the container orchestration platform pod. The initialization and deployment step includes, for example, a step of obtaining a dynamic resource definition used for the dynamic allocation of pod resources for the container orchestration platform pod being deployed. The dynamic resource definition includes a resource usage expression. Furthermore, the initialization and deployment step includes a step of creating a deployment object used for deploying the container orchestration platform pod. In one embodiment, the creation step includes a step of parsing the resource usage expression of the dynamic resource definition, and a step of generating one or more initial pod resource limits for use in initializing the container orchestration platform pod based on the parsing of the resource usage expression of the dynamic resource definition. Furthermore, the initialization and deployment steps include initializing and deploying the container orchestration platform pod having the one or more pod resources in the computing environment, wherein the initialization steps apply the generated one or more initial pod resource limits to the deployment of the container orchestration platform pod having the one or more pod resources. Advantageously, resource utilization is further enhanced by generating one or more initial pod resource limits for use in initializing the container orchestration platform pod based on a resource usage formula of dynamic resource definitions, for example, by setting initial pod resource limits via a resource usage formula to efficiently initialize the container orchestration platform pod with appropriate pod resource limits.

[0008] In one implementation, the monitored runtime resource usage includes the runtime central processing unit (CPU) usage and runtime memory usage of the container orchestration platform pod. Advantageously, the step of dynamically adjusting pod resource limits based on the runtime central processing unit (CPU) usage and runtime memory usage of the container orchestration platform pod allows for more efficient use of computing resources in a container-based computing environment, thereby saving processing costs, lowering the carbon footprint, and reducing energy consumption.

[0009] In one embodiment, the dynamically adjusting step includes dynamically increasing the pod resource limit of the container orchestration platform pod in the computing environment during runtime. In another implementation, the dynamically adjusting step includes dynamically decreasing the pod resource limit of the container orchestration platform pod in the computing environment during runtime. Advantageously, the step of dynamically increasing and / or decreasing the pod resource limit of the container orchestration platform pod in the computing environment during runtime provides cost-effective use of computing resources, as well as effective and dynamic resource utilization, and supports on-demand resource allocation without allocating unnecessary resources.

[0010] In one implementation, the dynamically adjusting step is further based on the autoscaling mode for the container orchestration platform pod. In one example, the autoscaling mode is a post-intervention mode, and the dynamically adjusting step is further based on the number of instances of the container orchestration platform pod that remain at a specified pod instance limit over a set period of time. In another example, the autoscaling mode is a first-in mode, and the dynamically adjusting step proceeds before the autoscaling of the number of instances of the container orchestration platform pod. Advantageously, the dynamic adjustment of pod resource limits based on the autoscaling mode provides the ability to dynamically adjust pod resource limits, after or before horizontal pod autoscaling. This further enhances the effective and dynamic use of resources to better balance load handling and avoid inefficient system behaviors such as slow computing system execution.

[0011] In one implementation, the computer implementation method further includes a step of dynamically adjusting the pod resource limits of one or more other container orchestration platform pods in the computing environment based on the dynamic adjustment of the pod resource limits of the container orchestration platform pod. Advantageously, by dynamically adjusting one or more pod resources or one or more other container orchestration platform pods in the computing environment based on the dynamic adjustment of pod resource limits, resource scalability within the computing environment is facilitated by scaling up and / or down resources across multiple container orchestration platform pods based on needs within the container-based computing environment. This facilitates processing within the container architecture-based computing environment and enhances resource utilization. Furthermore, the dynamic utilization of resources across multiple container orchestration platform pods leads to better balancing of load processing and enhances the efficient operation of the container-based computing environment.

[0012] Computer systems and computer program products relating to one or more embodiments are also described and claimed herein. Furthermore, services relating to one or more embodiments are also described and claimed herein.

[0013] Additional features and advantages are realized through the technologies described herein. Other embodiments and aspects are described in detail herein and are considered to be part of the claimed embodiments. [Brief explanation of the drawing]

[0014] One or more embodiments are specifically pointed out and claimed individually as examples in the claims at the end of this specification. The purposes, features, and advantages of the foregoing, as well as of one or more embodiments, will become apparent from the following detailed description in conjunction with the accompanying drawings.

[0015] [Figure 1] An example of a computing environment including and / or using one or more aspects of the present invention is shown.

[0016] [Figure 2] An embodiment of a computer program product having a pod resource limit adjustment module according to one or more aspects of the present invention is shown.

[0017] [Figure 3] An embodiment of a pod resource limit adjustment process according to one or more aspects of the present invention is shown.

[0018] [Figure 4] Another example of a container-based computing environment including and / or using one or more aspects of the present invention is shown.

[0019] [Figure 5] An example of a service mesh of a container-based computing environment including adjustment of pod resource limits according to one or more aspects of the present invention is shown.

[0020] [Figure 6] An example of a workflow of a formula control component of the control system of FIG. 4 according to one or more aspects of the present invention is shown.

[0021] [Figure 7A] An embodiment of a workflow of a prediction component of the control system of FIG. 4 according to one or more aspects of the present invention is shown.

[0022] [Figure 7B] Another example of a computing environment including and / or using one or more aspects of the present invention is shown.

[0023] [Figure 7C]An example of a prediction workflow of the prediction component of FIG. 4 according to one or more aspects of the present invention is shown.

[0024] [Figure 8] An example of a workflow of the message service component of the control system of FIG. 4 according to one or more aspects of the present invention is shown.

[0025] [Figure 9] An example of a workflow of the vertical pod auto-scaler (VPA) engine component of the control system of FIG. 4 according to one or more aspects of the present invention is shown.

[0026] [Figure 10] A further exemplary workflow of the vertical pod auto-scaler engine component of the control system of FIG. 4 according to one or more aspects of the present invention.

MODE FOR CARRYING OUT THE INVENTION

[0027] The accompanying drawings, which are incorporated in and form a part of this specification, further illustrate the present invention and serve to explain aspects of the present invention together with this detailed description of the present invention. In this regard, it should be noted that descriptions of well-known systems, devices, processing methods, etc. are omitted so as not to unnecessarily obscure the present invention in unnecessary detail. However, it should be understood that the detailed description and this specific example illustrate, but do not limit, aspects of the present invention and are provided merely as examples. For those skilled in the art of the present disclosure, various substitutions, modifications, additions, and / or other arrangements will be apparent within the spirit or scope of the underlying inventive concept. It should be further noted that numerous aspects or features of the present invention are disclosed herein, and unless conflicting, each disclosed aspect or feature can be combined with any other disclosed aspect or feature as desired for a particular application of the disclosed concept.

[0028] It should also be noted that the exemplary embodiments described below are merely examples and not limiting, using specific code, designs, architectures, protocols, layouts, schematics, or tools. Furthermore, the exemplary embodiments are described in specific cases using specific software, hardware, tools, or data processing environments, merely as examples to clarify the description. The exemplary embodiments may be used in relation to other comparable or similar structures, systems, applications, or architectures. One or more aspects of the exemplary embodiments may be implemented in hardware, software, or a combination thereof.

[0029] As will be understood by those skilled in the art, the program code referred to herein may include software and / or hardware. For example, the program code in certain embodiments of the present invention may utilize a software-based implementation of the described function, while other embodiments may include hardware for fixed functions. Certain embodiments may combine both types of program code. An example of program code, also referred to as one or more programs, is shown in Figure 1, including an operating system 122 and a pod resource limit adjustment module 200 stored in persistent storage 113.

[0030] One or more aspects of the present invention are incorporated into a computing environment and run and / or used by the computing environment. For example, the computing environment can be a variety of architectures and various types, including but not limited to personal computing, client-server, distributed, virtual, emulate, partitioned, unpartitioned, cloud-based, quantum, grid, time-sharing, cluster, peer-to-peer, mobile, having one or more nodes, having one or more processors, and / or any other type of environment and / or configuration capable of running a process (or more processes) that performs pod resource limit adjustment processing, such as disclosed herein. The aspects of the present invention are not limited to any particular architecture or environment.

[0031] Before further describing detailed embodiments of the present invention, an example of a computing environment including and / or using one or more aspects of the present invention is discussed below with reference to Figure 1.

[0032] Various aspects of this disclosure are described by explanatory text, flowcharts, block diagrams of computer systems, and / or block diagrams of mechanical logic included in embodiments of computer program products (CPPs). With respect to any flowchart, depending on the technology involved, operations may be performed in a different order than those shown in a given flowchart. For example, also depending on the technology involved, two operations shown in consecutive blocks of a flowchart may be performed in reverse order, as a single integrated stage, simultaneously, or with at least partial time overlap.

[0033] Embodiments of a computer program product ("CPP Embodiment" or "CPP") are terms used in this disclosure to describe any set of one or more storage media ("mediums") that are collectively included in a set of one or more storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device capable of holding and storing instructions for use by a computer processor. Computer-readable storage media may be, but are not limited to, electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, mechanical storage media, or any preferred combination of those described above. Some known types of storage devices, including these media, include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices (such as pits / lands formed on the main surface of a punch card or disk), or any suitable combination of those described above. When the term "computer-readable storage medium" is used in this disclosure, it shall not be construed as storage in the form of transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, optical pulses passing through optical fiber cables, electrical signals transmitted through wires, and / or other transmission media.As will be understood by those skilled in the art, data is typically moved at several intermittent points during the normal operation of a storage device, such as during access, defragmentation, or garbage collection. However, since data is not transient while it is stored, this does not mean that the storage device is transient.

[0034] The computing environment 100 includes an example of an environment for executing at least some of the computer code involved in performing the method of the present invention, such as a pod resource limit adjustment module block 200. In addition to block 200, the computing environment 100 includes, for example, a computer 101, a wide area network (WAN) 102, an end-user device (EUD) 103, a remote server 104, a public cloud 105, and a private cloud 106. In this embodiment, the computer 101 includes a processor set 110 (including processing circuits 120 and a cache 121), a communication fabric 111, volatile memory 112, persistent storage 113 (including the operating system 122 and block 200 as identified above), a peripheral device set 114 (including a user interface (UI), a device set 123, storage 124, and an Internet of Things (IoT) sensor set 125), and a network module 115. The remote server 104 includes a remote database 130. The public cloud 105 includes a gateway 140, a cloud orchestration module 141, a host physical machine set 142, a virtual machine set 143, and a container set 144.

[0035] Computer 101 may take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device that is currently known or may be developed in the future, capable of running programs, accessing networks, or querying databases such as remote database 130. As is well understood in the field of computer technology, and depending on the technology, the execution of a computer implementation may be distributed among multiple computers and / or multiple locations. On the other hand, in this description of the computing environment 100, in order to make the explanation as concise as possible, the detailed discussion will focus on a single computer, specifically computer 101. Although computer 101 is not shown in the cloud in Figure 1, it may be located in the cloud. On the other hand, computer 101 does not need to be located in the cloud, except to any extent that may be definitively shown.

[0036] The processor set 110 includes one or more computer processors of any type currently known or to be developed in the future. The processing circuitry 120 may be distributed across multiple packages, for example, multiple interconnected integrated circuit chips. The processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. The cache 121 is memory located within the processor chip package and is typically used for data or code that should be available for high-speed access by threads or cores running on the processor set 110. The cache memory is typically organized into multiple levels depending on its relative proximity to the processing circuitry. Alternatively, some or all of the cache for the processor set may be located "off-chip". In some computing environments, the processor set 110 may operate using qubits and be designed to perform quantum computing.

[0037] Computer-readable program instructions are typically loaded onto computer 101, causing the processor set 110 of computer 101 to execute a series of operational steps, thereby enabling the computer implementation method. As a result, the instructions thus executed instantiate the method specified in the flowcharts and / or descriptions of the computer implementation method contained herein (collectively referred to as the “Method of the Invention”). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 110 to control and direct the execution of the Method of the Invention. In computing environment 100, at least some of the instructions for executing the Method of the Invention may be stored in blocks 200 in persistent storage 113.

[0038] The communication fabric 111 is a signal conduction path that enables various components of the computer 101 to communicate with one another. Typically, this fabric is made up of switches and conductive paths, such as buses, bridges, and physical input / output ports. Other types of signal communication paths, such as optical fiber communication paths and / or wireless communication paths, may be used.

[0039] Volatile memory 112 is any type of volatile memory currently known or to be developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Volatile memory is typically characterized by random access, but this is not mandatory unless explicitly stated. In computer 101, volatile memory 112 is located in a single package and resides inside computer 101, but alternatively or additionally, volatile memory may be distributed across multiple packages and / or located externally to computer 101.

[0040] The persistent storage 113 is any form of non-volatile storage for a computer, currently known or to be developed in the future. The non-volatility of this storage means that the stored data is maintained regardless of whether power is supplied to the computer 101 and / or directly to the persistent storage 113. The persistent storage 113 may be read-only memory (ROM), but typically at least a portion of the persistent storage allows for writing, deleting, and rewriting of data. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. The operating system 122 may take several forms, such as various known proprietary operating systems or open-source portable operating system interface type operating systems that utilize a kernel. The code contained in block 126 typically includes at least a portion of computer code involved in performing the method of the present invention.

[0041] The peripheral device set 114 includes a set of peripheral devices for the computer 101. Data communication connections between the computer 101's peripheral devices and other components may be implemented in various ways, such as Bluetooth® connections, near-field communication (NFC) connections, connections formed by cables (such as Universal Serial Bus (USB) type cables), insert-type connections (e.g., Secure Digital (SD) cards), connections formed through local area communication networks, and even connections formed through wide area networks such as the Internet. In various embodiments, the UI device set 123 may include components such as a display screen, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. Storage 124 is external storage such as an external hard drive, or insertable storage such as an SD card. Storage 124 may be persistent and / or volatile. In some embodiments, storage 124 may take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 requires a large amount of storage (for example, when computer 101 locally stores and manages a large database), this storage may be provided by peripheral storage devices designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 125 consists of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer and another may be a motion detector.

[0042] The network module 115 is a collection of computer software, hardware, and firmware that enables computer 101 to communicate with other computers via the WAN 102. The network module 115 may include hardware such as a modem or Wi-Fi signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for transmitting data over the internet. In some embodiments, the network control and network forwarding functions of the network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing Software-Defined Networking (SDN)), the control and forwarding functions of the network module 115 are performed on physically separate devices, such that the control function manages several different network hardware devices. Computer-readable program instructions for performing the method of the present invention can typically be downloaded to computer 101 from an external computer or external storage device via a network adapter card or network interface included in the network module 115.

[0043] WAN102 is any wide area network (e.g., the Internet) that can communicate computer data over non-local distances using any currently known or future-developed technology for communicating computer data. In some embodiments, the WAN may be replaced and / or supplemented by a local area network (LAN), such as a Wi-Fi network, designed to communicate data between devices located in a local area. The WAN and / or LAN typically include computer hardware such as copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0044] An end-user device (EUD) 103 is any computer system used and controlled by an end-user (e.g., an enterprise customer operating computer 101) and can take any of the forms discussed above in relation to computer 101. EUD 103 typically receives useful and valuable data from the operation of computer 101. For example, in a hypothetical case where computer 101 is designed to provide recommendations to an end-user, these recommendations would typically be transmitted from computer 101's network module 115 to EUD 103 via WAN 102. Thus, EUD 103 can display or otherwise present recommendations to the end-user. In some embodiments, EUD 103 may be a client device such as a thin client, heavy client, mainframe computer, or desktop computer.

[0045] The remote server 104 is any computer system that provides at least some data and / or functions to computer 101. The remote server 104 may be controlled and used by the same entity that operates computer 101. The remote server 104 represents a machine that collects and stores useful and valuable data for use by other computers, such as computer 101. For example, in a hypothetical case where computer 101 is designed and programmed to provide recommendations based on historical data, this historical data may be provided to computer 101 from the remote database 130 of the remote server 104.

[0046] The public cloud 105 is any computer system available for use by multiple entities, providing on-demand availability of computer system resources and / or other computer functions, particularly data storage (cloud storage) and computing power, without direct active management by the user. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct and active management of the computing resources of the public cloud 105 is performed by the computer hardware and / or software of the cloud orchestration module 141. The computing resources provided by the public cloud 105 are typically implemented by virtual computing environments running on various computers that make up the host physical machine set 142, which is a universe of physical computers located within and / or available to the public cloud 105. The virtual computing environment (VCE) typically takes the form of virtual machines from the virtual machine set 143 and / or containers from the container set 144. These VCEs can be stored as images and transferred either as images or after VCE instantiation, among and between hosts on various physical machines. The cloud orchestration module 141 manages the transfer and storage of images, deploys new VCE instantiations, and manages the active instantiation of VCE deployments. The gateway 140 is a collection of computer software, hardware, and firmware that enables the public cloud 105 to communicate through the WAN 102.

[0047] Here, some further explanation of virtualized computing environments (VCEs) is provided. A VCE can be stored as an "image." A new active instance of a VCE can be instantiated from an image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature in which the kernel allows for the existence of multiple isolated user-space instances called containers. These isolated user-space instances typically behave like actual computers in terms of the programs running within them. Computer programs running on a normal operating system can utilize all of that computer's resources, including connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running inside a container can only use the contents of the container and the devices allocated to the container; this feature is known as containerization.

[0048] The private cloud 106 is similar to the public cloud 105, except that its computing resources are available only for use by a single enterprise. While the private cloud 106 is shown as being in communication with the WAN 102, in other embodiments, the private cloud may be completely isolated from the internet and accessible only via a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. Each of the multiple clouds remains a separate discrete entity, but the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the multiple configuration clouds. In this embodiment, both the public cloud 105 and the private cloud 106 are part of a larger hybrid cloud.

[0049] The computing environment described above is merely one example of a computing environment that incorporates, runs, and / or uses one or more embodiments of the present invention. Other examples are possible. Furthermore, in one or more embodiments, one or more of the components / modules in Figure 1 do not need to be included in the computing environment and / or are not used for one or more embodiments of the present invention. Furthermore, in one or more embodiments, additional and / or other components / modules may be used. Other variations are possible.

[0050] As shown, in one example, computing environment 100 supports containers. Containers may reside in clouds such as public clouds (e.g., public cloud 105), private clouds (e.g., private cloud 106), hybrid clouds, and / or on-premises (e.g., computer 101). In one example, containers are managed by one or more container orchestration platforms. One example of such a platform is Kubernetes®, which is an open-source, extensible, and portable container management platform. Kubernetes is a registered trademark of the Linux Foundation in San Francisco, California. Other platforms may also be used. In Kubernetes®, for example, a container has its own central processing unit share, file system, processing space, memory, etc. Furthermore, due to their relaxed isolation characteristics, containers may share an operating system (OS) among applications; containers are isolated from the underlying infrastructure; containers are portable across operating system distributions and clouds; and each container is repeatable. Containers are intended to be stateless and immutable; the code of a running container is not modified, instead a new container image containing any code changes is built.

[0051] As should be noted, container orchestration platforms typically partially support one or more pods, where a pod is a group of one or more containers deployed on a single node. Containers within a pod typically share an IP address, IPC, hostname, and other resources. In one example, a computing environment employs a platform such as Kubernetes® and / or another platform to manage containers. Kubernetes® is a platform for running and managing containers from multiple container runtimes. A computing environment may include one or more nodes, an operating system shared by one or more nodes, and underlying hardware used by one or more nodes, such as processing units. Nodes can be virtual or physical machines, and they can be on-premises (e.g., within computer 101 and / or other computing devices) and / or in cloud environments (e.g., public cloud 105, private cloud 106, hybrid cloud environment, and / or other cloud environment). In one example, a node may include a container runtime; one or more pods; a proxy; and an agent. An example of a proxy is kube-proxy, which is a network proxy that runs on each node in a cluster and implements some of the Kubernetes® service concepts. kube-proxy maintains network rules on its nodes, which enable network communication to pods from network sessions inside or outside the cluster. An example of an agent is kubelet, which runs on each node. It can register nodes using one or more of the following indicators: hostname, flags, or other indicators, using an application programming interface (API) server that validates and configures data for objects (e.g., pods).In other examples where the platform is not Kubernetes®, proxies and agents may be specific to that platform. Many examples are possible.

[0052] In one example, a container runtime interface is provided, which is a plug-in interface that allows an agent (e.g., kubelet) to use various container runtimes without requiring the cluster components to be recompiled. Furthermore, in another example, a pod contains one or more containers, and a container contains a container image having one or more applications, for example, one or more libraries and / or one or more binaries and / or text resources. The container image is deployed on the node.

[0053] Regarding resource control for pods and containers, developers can specify the amount of resources (e.g., memory, CPU, storage, etc.) required to support a pod at runtime. For example, there are typically two different static settings for pod resource control. These include a request or lower limit, which defines the amount of resources required as a minimum requirement (i.e., a lower limit), and a limit, which defines an upper limit (i.e., an upper limit) on resource usage. For example, an exemplary pod specification might include: apiVersion: v1 kind: Pod metadata: name: frontend spec: containers: - name: app image: images.my-company.example / app:v4 resources: lower limits: Memory: "64Mi" CPU: "250M" upper limits: memory: "128Mi" CPU: "500M" Note that in the example pod specification above, Mi (Mebibytes) and m (milli) are Kubernetes® resource units. The container orchestration platform scheduler schedules pod and / or container deployments based on the definition of pod resource limits and the resources available on existing nodes. Specific resource limits for different resource types, such as memory and CPU, typically differ in practice.

[0054] Based on the current specifications, the deployer optionally provides specific lower and upper limits for different resource types. By default, these lower and upper limits are static values ​​based on estimates of the amount of resources required for pod deployment and runtime processing. This approach presents several challenges. For example, the deployer must understand application and existing infrastructure details to make acceptable estimates, such as the amount of resources available on typical cluster nodes and the amount of resources remaining on existing nodes. Furthermore, after deployment, the workload or requests for processing can be unpredictable, making it difficult to define static values ​​for pod resource lower and upper limits. While scaling the number of pods is possible, there is no solution to address the challenges associated with pod resource lower and upper limits.

[0055] To address these challenges, one or more embodiments of pod resource limit adjustment modules and processes are disclosed herein and first described with reference to Figures 2-3. Figure 2 shows one embodiment of a pod resource limit adjustment module 200 including code or instructions for performing pod resource limit adjustment processing according to one or more embodiments of the present invention, and Figure 3 shows one embodiment of a pod resource limit adjustment process according to one or more embodiments of the present invention.

[0056] Referring to Figures 1 and 2, the pod resource limit adjustment module 200 includes, in one example, various submodules used to perform processing according to one or more aspects of the present invention. The submodules are, for example, computer-readable program code (e.g., instructions) and computer-readable media (e.g., persistent storage (e.g., persistent storage 113, e.g., disk) and / or cache (e.g., cache 121)). The computer-readable media may be part of a computer program product and may be executed by and / or using one or more computers such as computer 101; processors such as processors of processor set 110; and / or processing circuits such as processing circuits of processor set 110.

[0057] In the embodiment shown in Figure 2, exemplary submodules of the Pod Resource Limit Adjustment Module 200 include, for example, a Dynamic Resource Definition Acquisition submodule 202 that acquires a dynamic resource definition used for the dynamic allocation of pod resources for a container orchestration platform pod to be deployed, where the dynamic resource definition includes a resource usage expression; a Deployment Object Creation submodule 204 that creates a deployment object used for deploying the container orchestration platform pod; a Container Orchestration Platform Pod Initialization and Deployment submodule 206 for deploying a container orchestration platform pod having one or more pod resources in a computing environment, where initialization applies the generated one or more pod resource limits; and a Monitoring of Runtime Resource Usage of the Container Orchestration Platform Pod. The system includes a runtime resource usage monitoring submodule 208; a pod resource usage prediction submodule 210 that predicts future resource usage of a container orchestration platform pod using a trained machine learning model, where the prediction uses at least partially monitored runtime resource usage; a pod resource limit dynamic adjustment submodule 212 used for dynamic adjustment of one or more pod resource limits for a container orchestration platform pod in a computing environment; and a pod resource limit dynamic adjustment propagation submodule 214 used for dynamic adjustment of one or more other pod resource limits for one or more other container orchestration platform pods in a computing environment, based on the dynamic adjustment of the pod resource limits of the container orchestration platform pod. Advantageously, processing is facilitated within a container architecture-based computing environment by providing enhanced resource utilization.By dynamically adjusting pod resource limits, cost-effective use of computing resources is provided, along with efficient and dynamic resource utilization, thereby supporting on-demand resource allocation without unnecessary waste. Furthermore, by using computing resources more efficiently in container-based computing environments, the disclosed processes result in cost savings, a lower carbon footprint, and reduced energy consumption. While various submodules are described, it should be noted that the processing of pod resource limit adjustment modules as disclosed herein may use or include additional, fewer, and / or different submodules. Certain submodules may contain additional or less code, including code from other submodules. Furthermore, additional and / or other modules may be used. Many variations are possible.

[0058] In one or more embodiments, submodules according to one or more aspects of the present invention are used to perform pod resource limit adjustment processing. Figure 3 shows one example of a pod resource limit adjustment process as disclosed herein. In one or more embodiments, the process is performed by a computer (e.g., computer 101 (Figure 1)), and / or a processor or processing circuit (e.g., processor set 110 in Figure 1). In one embodiment, the code or instructions that implement the process are part of a module, such as the pod resource limit adjustment module 200. In other embodiments, the code may be contained in one or more other modules, and / or in one or more submodules of such one or more other modules. A variety of options are available.

[0059] As one example, a pod resource limit adjustment process 300 running on a computer (e.g., computer 101 in Figure 1), a processor (e.g., a processor in processor set 110 in Figure 1), and / or processing circuitry (e.g., processing circuitry in processor set 110) obtains a dynamic resource definition 302, which is used for the dynamic allocation of pod resources for a container orchestration platform pod to be deployed, where the dynamic resource definition includes a resource usage expression. The pod resource limit adjustment process 300 further includes a step 304, which creates a deployment object used for the deployment of the container orchestration platform pod. In one embodiment, the step of creating a deployment object may include, for example, a step of parsing the resource usage expression of the dynamic resource definition, and a step of generating one or more pod resource limits for use in the initialization of the container orchestration platform pod based on the parsing of the resource usage expression of the dynamic resource definition.

[0060] In one embodiment, the pod resource limit adjustment process 300 further includes a step 306 of initializing and deploying a container orchestration platform pod having one or more pod resources in a computing environment. The initialization step may include a step of applying the generated one or more pod resource limits.

[0061] Furthermore, the pod resource limit adjustment process 300 includes, in one embodiment, a step 308 of monitoring the runtime resource usage of the container orchestration platform pod, and a step 310 of predicting the future resource usage of the container orchestration platform pod using a trained machine learning model. In one example, the prediction step may be, for example, a step of predicting the future resource usage requirements for a defined time interval, for example, the next x seconds or y minutes.

[0062] In addition, in one embodiment, the pod resource limit adjustment process 300 includes a step 312 of dynamically adjusting the pod resource limits of one or more pod resource limits for a container orchestration platform pod in the computing environment. In one embodiment, the dynamic adjustment step is based on monitored runtime resource usage and predicted future resource usage. Furthermore, in one embodiment, the pod resource limit adjustment process 300 may include a step 314 of selectively propagating one or more dynamic adjustments of pod resource limits to one or more other container orchestration platform pods. For example, the process may include a step of dynamically adjusting one or more other pod resource limits of one or more other container orchestration platform pods in the computing environment based on the dynamic adjustment of the pod resource limits of a container orchestration platform pod.

[0063] In an example of another computing environment, Figure 4 shows a container-based architectural computing environment 400, which (in one embodiment) may reside within or be similar to the computing environment 100 described above in relation to Figure 1. The computing environment 400 includes a control system 410 having program code configured to implement one or more embodiments of a container orchestration platform as disclosed herein, and one or more embodiments of a pod resource limit adjustment facility. In addition, the computing environment 400 includes one or more nodes 405, and underlying hardware used by the control system 410 and one or more nodes 405, such as processing units. The nodes may be virtual or physical machines, and they may be on-premises (e.g., within computer 101 (Figure 1)), and / or in other computer devices, and / or in cloud environments (e.g., public cloud 105, private cloud 106, hybrid cloud environment, and / or other cloud environment). In one example, the computing environment 400 may employ a platform such as Kubernetes® and / or other platforms to manage pods and containers, further comprising components and / or features according to one or more aspects of the present invention.

[0064] For example, the control system 410 includes a call chain management component 412, which in one embodiment includes a service message configuration reader (SMCR) that automatically creates channels and subscriptions based on the SMCR. Furthermore, the call chain management component 412 is configured in one embodiment to customize which service change events different pods subscribe to. In addition, the control system 410 includes a service mesh configuration data store 414 and a messaging server (MQ) 416 that facilitates subscribers taking action when they receive events and publishers taking action when changes occur, i.e., sending change events to channels. The application programming interface (API) server 418 of the control system 410 receives the configuration of an application 401, which may include dynamic resource definitions used for the dynamic allocation of pod resources for deployed container orchestration platform pods, according to one or more aspects of the present invention. In one or more embodiments, the dynamic resource definition includes, for example, a resource usage expression that facilitates the dynamic adjustment of pod resource limits for one or more container orchestration platform pods. Note that the configuration may be stored in database 420. In one example, database 420 may be an etcd store, which is an open-storage distributed key-value store used to hold and manage information for a distributed system, and in particular can manage configuration data, state data, and metadata about Kubernetes®.

[0065] The metrics server 430 obtains runtime resource usage data for pods 406 and / or containers within pods via container monitoring agents 407 associated with one or more nodes 405 having one or more pods 406. The runtime resource usage data is transferred by the metrics server 430 to a database 432 of historical data. In one or more embodiments, the runtime resource usage data may include runtime data for central processing unit (CPU) usage, memory usage, etc., for pods and / or containers running on the nodes. Historical data may be referenced by a prediction component 434 configured to collect historical resource usage data and current runtime usage data as input for use in predicting future resource usage of one or more container orchestration platform pods running on the nodes using a trained machine learning model. In one implementation, the prediction component 434 predicts incoming request traffic according to the collected resource usage data. The expression control component 436 or expression manager accesses configuration information received from the data store 420 and usage history data from the history data database 432. In one or more embodiments, the expression control component 436 is used to register expressions and expression handlers, to determine through the expressions when resources (i.e., resource limits) are set, and if so, parses the syntax of the received configuration expression. In one implementation, expressions may be parsed at runtime, expression handlers in the expression data store 420 may be found, and calls are made to the handlers to obtain configuration values. The process is repeated until all expressions have been parsed and the respective values ​​based on the expressions have been determined.

[0066] The vertical pod auto-scale (VPA) engine component 440 communicates with the prediction component 434 and the expression control component 436 to facilitate the dynamic adjustment of pod resource limits for one or more pod resource limits of a container orchestration platform pod in a computing environment, for example, where the dynamic adjustment is based on monitored runtime resource usage and predicted future resource usage. In one implementation, the VPA engine component 440 communicates with the message service component 442 to facilitate the sending and receiving of events for propagating, for example, pod resource limit changes to one or more other container orchestration platform pods, as discussed herein. In addition, the VPA engine component 440 deploys any adjustments to pod resource limits via the deployment component 450 and, in one embodiment, the replica set component 452. In one embodiment, the deployment component 450 further communicates with the horizontal pod auto-scaler (HPA) component 435. In one or more implementations, the vertical pod autoscaler engine component 440 works together with the horizontal pod autoscaler component 435 to, for example, in post-intervention mode, proceed with dynamic adjustment of the pod resource limit based on the number of instances of the container orchestration platform pod that remain at a specified pod instance limit over a set period of time. Alternatively, if the autoscaling mode is first-in mode, the dynamic adjustment proceeds before the autoscaling of the number of instances of the container orchestration platform pod. For example, in this mode, the dynamic pod resource limit setting is applied first, and the horizontal pod autoscaler is not applied until it is detected that the pod resource limit has not changed significantly over a period of time.In this regard, it should be noted that the horizontal pod autoscaler initiates and deploys instances of additional container orchestration platform pods in the computing environment, and the vertical pod autoscaling engine component controls the dynamic adjustment of one or more pod resource limits for one or more container orchestration platform pods in the computing environment, according to one or more embodiments disclosed herein.

[0067] As an example, Figure 5 illustrates one embodiment of how pod resource limit adjustment processing as disclosed herein may occur. Assume that multiple services are deployed, including services running on a pod 510 of a container-based architectural computing environment 400', such as the computing environments 100 and 400 described above in relation to Figures 1 and 4. The container-based architectural computing environment 400' includes a control system 410, as described above in relation to Figure 4, and multiple pods 510, 520 running on one or more nodes of the computing environment. In this example, pod 510 provides a first service, such as travel-related services, and the first service triggers one or more second services, such as a second service running on multiple other container orchestration platform pods 520. In the example, the travel service 510 receives a request 501 and, as part of processing the request, may access one or more other services running on one or more other pods related to the travel service being provided. In one example, if the first service requires high runtime resource usage, the VPA engine component 440 in Figure 4 may automatically scale up one or more pod resource limits for, for example, pod 510 to facilitate the handling of high-volume requests, and one or more other container orchestration platform pods 520 that at least partially support the container orchestration platform pod 510 may also have their pod resource limits scaled up one or more times based on the call chain management component 412 (Figure 4).

[0068] In one or more implementations, processing within a container-based computing environment is facilitated in three distinct phases according to one or more embodiments disclosed herein. Phase 1 is the preparation phase, in which processing deploys all necessary settings and configurations, initializes message queues, and sets up monitoring to collect resource usage data for storage as historical data. Phase 2 is the resource determination and / or prediction phase, in which the VPA engine dynamically adjusts resource usage limits based on expression control components, inputs from prediction components, events propagated from message services, and / or real-time metrics at runtime. Phase 3 includes propagation of VPA engine results, in which the VPA engine propagates resource limit adjustment events to subsequent services, e.g., second services running in multiple other container orchestration platform pods (e.g., pod 520 in the example in Figure 5). In this manner, resource usage limits for secondary services are also dynamically adjusted, facilitating on-demand resource allocation across pod levels.

[0069] Referring to Figure 4, in one embodiment, preparation phase 1 includes the call chain management component 412 periodically synchronizing call chain microservices based on the service mesh configuration 414. The call chain management component 412 establishes a message queue (MQ) based on the call chain information. In one embodiment, an end user or end user system provides a dynamic resource definition, such as example.yaml 401 in Figure 4, to facilitate resource deployment, and the definition is based on one or more expressions stored in the definition data store 420 (or expression registry).

[0070] The expression control component 436 parses applicable expressions and, in one implementation, may maintain a cache for each expansion. In one embodiment, the expression control component 436 includes a syntax parser, a semantic analysis component, an expression handler, an expression registry, and an expression optimization component. In one or more implementations, the syntax parser may include basic syntax checking to ensure that the input is in a valid format and that a syntax tree can be constructed from the input. The semantic analysis component may be a runtime component that performs semantic analysis to identify and extract expressions such as max(avg(30day),avg(2weeks)) and looks up the corresponding handler in the expression registry. The expression handlers include built-in and user-defined handlers that facilitate the determination of the results of expressions entered, for example, by a system user. The expression register may manage the registration of expressions and allow users to customize expressions within handlers. In one embodiment, the expression registry may be part of the definition data store 420. The expression optimization component may be based on compiler principles for performing expression optimization. For example, max(avg(30days),avg(30days)) may be optimized as avg(30days). When creating a deployment object, the expression control component parses the dynamic resource definition for the pod and generates one or more initial pod resource limits for pod initialization and deployment. One embodiment of the expression control component workflow 600 is shown in Figure 6.

[0071] Referring to Figure 6, this specification assumes that a dynamic resource definition includes, for example, an expression for resource usage such as avg(last(30days)) or 100Mi. This is an example of an expression that may be used for initialization. An expression is a mathematical expression consisting of multiple functions and operations. In the embodiment of Figure 6, the workflow begins with a semantic analysis of the resource definition received by the control system 601 to determine 602 whether any expression was used. If "no", the process ends 603. Assuming the resource definition includes an expression, the process 604 parses the expression, which may include calling an expression handler component 606 to facilitate the parsing of the expression. The expression may be obtained, for example, from an expression registry or data store 420, and the expression handler component may refer to historical data 432 to obtain the data necessary to determine one or more limit values ​​610 using the expression. Once the limit values ​​are determined, the pod deployment object 612 may be updated with the determined initial pod resource limit. In one or more embodiments, the expression control component may perform steps multiple times to refer to the expression data store, parse the expression, and invoke the expression handler in order to fully parse a particular expression. For example, in the example resource definition above, the expression control component first performs these steps for the term "last" and then for the term "avg". Depending on the expression, a series of actions may then be applied as needed to obtain the expression value for the expression. Note that in the case of initialization, if the historical data is insufficient, a default value may be specified by the expression control component. For example, in the resource definition described, 100Mi may be used as the initial value for the pod resource limit if the resource usage data is insufficient to dynamically determine the limit value.

[0072] Once deployed, the metrics server 430 (Figure 4) facilitates real-time monitoring of resource usage by the deployed pods or objects and maintains the historical data store 432 for runtime metrics such as central processing unit (CPU) usage and / or memory usage by the container orchestration platform pods.

[0073] In Phase 2 processing, actual resource usage is determined, and future resource usage is predicted. For example, during runtime, the expression control component 436 interacts with the VPA engine component to provide historical values ​​for deployed resource usage in order to determine dynamic resource usage limits. For example, in one example, the dynamic resource definition of application configuration 401 may be as specified in Figure 4. In the exemplary configuration, the resource usage expression avg(last(30days)) × 2 may be used to facilitate dynamic adjustment of the pod resource limits of the container orchestration platform pod. Note that in this example configuration and expression, pod resource usage is determined at runtime from historical data, as in only one example, the expression includes the last function (last(30days)), which means that data from the past 30 days is used.

[0074] The prediction component 434 periodically monitors historical resource usage data and uses the monitored runtime resource usage, at least partially, to predict future resource usage for the container orchestration platform pod. Based on the predicted future resource usage, prediction alerts may be generated (for example, if the predicted usage exceeds a threshold) and provided to the VPA engine component.

[0075] Figures 7A–7C illustrate one embodiment of the workflow and environment associated with the prediction component 434 and the VPA engine component 440. In Figure 7A, a container-based computing environment 700 is shown, similar to the container-based computing environment 400 in Figure 4, where the prediction model implementation 710 includes, in one embodiment, both a model training phase and a model testing phase. As shown, the prediction model implementation component 710 receives resource metrics 701 such as CPU and / or memory usage, arbitrary custom metrics 702 such as queries per second / response time (QPS / RT), and historical data usage metrics 703. In the model training phase, historical resource usage data 712 is input to one or more machine learning models 714, which facilitate the training of a predictive machine learning model 716 to predict future resource usage of container orchestration platform pods, as discussed herein. In the model testing phase, new runtime resource usage data 720 is applied to a trained predictive machine learning model 716 to generate one or more predictions 722, which may form the basis for one or more actions, such as dynamic adjustment of one or more pod resource limits of a container orchestration platform pod. As shown in Figure 7, to facilitate the dynamic deployment 450 of the adjustment of one or more pod resource limits, the predicted future resource usage may be applied, for example, to the VPA engine component 440 of the control system 410 in Figure 4, and the adjustment of one or more pod resource limits is propagated to one or more pods 406 of the container-based computing environment.

[0076] Further description reveals that Figure 7B shows one embodiment of a computing environment that may incorporate or implement one or more embodiments of the embodiments of the present invention. In one or more implementations, the control system 410 may be implemented as part of a computing environment such as the computing environment 100 described above in relation to Figure 1. The control system 410 includes and / or utilizes one or more computing resources that implement, for example, one or more embodiments of a module or equipment as disclosed herein and execute program code 740 including a cognitive engine or agent 742 that trains and / or utilizes one or more machine learning models 716 as described herein. Based on the specific application of a machine learning model to facilitate the achievement of the disclosed dynamic pod resource limit adjustment workflow, data, e.g., historical resource usage data, runtime resource usage data, or other data associated with the generation of a predictive model for dynamic pod resource adjustment in one or more aspects disclosed herein, is obtained from data sources 712, 720 and used by the cognitive agent 742 to train a model 716 to predict future resource usage of the container orchestration platform pods (for example). In an implementation, the control system 410 may include or utilize one or more networks for exchanging various aspects of computing resources, as well as one or more data sources 712, 720 that provide data, and one or more components, systems, etc. that receive the output, actions, etc. 722 of the machine learning model 716, etc., to facilitate the execution of one or more system operations. For example, a network can be a telecommunications network, a local area network (LAN), a wide area network (WAN) such as the Internet, or a combination thereof, and may include wired, wireless, and fiber optic connections.The network may include, as discussed herein, one or more wired and / or wireless networks capable of receiving and transmitting data, including training data for machine learning models, and output solutions, recommendations, and actions of the machine learning models.

[0077] In one or more implementations, the computing resource houses and / or executes program code 740 configured to perform a method according to one or more aspects of the present invention. For example, the computing resource may be a computing system implementation resource. Furthermore, for illustrative purposes only, the computing resource in Figure 7B is shown as a single computing system. This is a non-limiting example of implementation. In one or more other implementations, as discussed herein, one or more aspects of the processing may be implemented at least partially in multiple separate computing resources or systems, such as one or more computing systems in a cloud-hosted environment, for example. In this regard, it should be noted that, although described herein as part of the control system 410 of the container-based computing environment 400 in Figure 4, one or more aspects of training and / or testing machine learning models may be performed by a computing system separately from the control system 410. Many variations are possible.

[0078] In one embodiment, program code 740 runs a cognitive engine or agent 742 that trains one or more models 716. The models may be trained using training data that may include various types of data depending on the model and data source. In one or more embodiments, program code 740 running on one or more computing resources applies one or more algorithms of the cognitive agent 742 to generate and train models, which the program code then uses to determine, for example, values ​​for future resource usage of a container orchestration platform pod, and the predictions use, for example, the monitored runtime resource usage of the pod. The values ​​may be used to dynamically adjust the pod resource limits of one or more pod resource limits of the container orchestration platform pod in the computing environment, as described herein. In an initialization or learning stage, program code 740 trains one or more machine learning models 716 using acquired training data, which in one or more embodiments includes, for example, usage history data or other data used by an artificial intelligence system workflow to generate predictions for future resource usage of a container orchestration platform pod, as described herein.

[0079] In one or more embodiments, the data used to train the model may include various types of data, e.g., heterogeneous data generated by one or more data sources, and / or data stored in one or more databases accessible by computing resources. In embodiments of the present invention, program code may perform data analysis to generate data structures that include algorithms used by the program code to predict and / or perform actions. As is known, machine learning-based modeling solves problems that cannot be solved by numerical means alone. In one example, program code extracts features / attributes from training data, which may be stored in memory or in one or more databases. The extracted features may be used to develop a predictive function h(x), also called a hypothesis, which the program code uses as a model. In identifying machine learning models, various techniques for selecting features (elements, patterns, attributes, etc.), including but not limited to diffusion mapping, principal component analysis, recursive feature reduction (a brute-force method for selecting features), and / or random forests, may be used to select attributes relevant to a particular model. The program code may utilize one or more algorithms to train the model (e.g., the algorithms used by the program code), including providing weights for conclusions, so that the program code can train any predictors or performance functions included in the model. The conclusions may be evaluated by quality metrics. By selecting a diverse set of training data, the program code trains the model to identify and weight various attributes (e.g., features, patterns) that correlate with the model's enhanced performance.

[0080] In one or more embodiments, program code running on one or more processors utilizes existing cognitive analysis tools or agents (currently known or to be developed later) to tune a model based on data acquired from one or more data sources. In one or more embodiments, the program code interacts with an application programming interface to perform cognitive analysis on the acquired data. Specifically, in one or more embodiments, a particular application programming interface includes a cognitive agent (e.g., a learning agent) which includes one or more programs, including but not limited to natural language classifiers, search and ranking services that can reveal the most relevant information from a set of documents, conceptual / visual insights, trade-off analysis, document transformation, and / or relationship extraction. In embodiments, one or more programs analyze data acquired by program code across various sources that utilize one or more of the natural language classifier, search and ranking application programming interfaces, and trade-off analysis application programming interfaces.

[0081] In one or more embodiments of the present invention, program code may utilize one or more neural networks to analyze training data and / or collected data to generate an operational machine learning model. Neural networks are a programming paradigm that enables computers to learn from observed data. This learning is referred to as deep learning, which is a set of techniques for learning in neural networks. Neural networks, including modular neural networks, enable pattern (e.g., state) recognition with speed, accuracy, and efficiency in situations where datasets are interactive and extensible, including across distributed networks, including but not limited to cloud computing systems. Modern neural networks are nonlinear statistical data modeling tools. They are typically used to model complex relationships between inputs and outputs or to identify patterns (e.g., states) in data (i.e., neural networks are nonlinear statistical data modeling or decision tools). Generally, program code that utilizes neural networks can model complex relationships between inputs and outputs, and patterns identified in data. In particular when parsing multiple complex datasets, neural networks and deep learning offer solutions to many problems in multi-source processing, due to their speed and efficiency, which in embodiments of the present invention can be used to implement machine learning models as described herein, as program code can be used.

[0082] For further example, Figure 7C illustrates another embodiment of the prediction workflow of the prediction component 434 of Figure 4, according to one or more aspects of the present invention. The workflow begins by collecting historical resource usage data for a container orchestration platform pod and preprocessing and cleaning the historical data by removing any missing or outlier values. For example, based on the application specification 401 of Figure 4, for central processing unit (CPU) usage, the process may refer to historical resource usage data for the past 30 days.

[0083] In the workflow shown in Figure 7C, the cleaned historical resource usage data is divided into a training dataset and a test dataset, with the training dataset being used to train a machine learning model (e.g., a linear regression model). For example, processing may remove any columns containing missing values ​​or any values ​​greater than three standard deviations from the mean. In one embodiment, to facilitate the training of a linear regression model, processing may divide the historical usage data so that 70% is used for the training dataset and 30% for the test dataset.

[0084] In the 7C workflow, the performance of a linear regression model is evaluated on a test dataset of historical resource usage data.754 For example, R measures how well the model fits the data. 2 The value can be determined.

[0085] The process collects runtime or real-time data about the resource usage of the pods and preprocesses the real-time data by removing any missing or outlier values. For example, in one embodiment, runtime data about the CPU usage of a container orchestration platform pod is obtained. For example, in one embodiment, the data may be collected and stored every minute. Furthermore, in one embodiment, any column containing missing values ​​may be removed, and / or any value greater than (e.g.) 3 standard deviations from the mean may be removed.

[0086] A trained linear regression model (i.e., a trained machine learning model) can then be used to predict possible future values ​​of pod resource usage based on preprocessed real-time data.758 For example, a trained machine learning model can be used to predict memory and CPU usage values ​​for the next minute based on the current value of CPU usage. The predicted values ​​of pod resource usage can be monitored in real time, and adjustments can be made to the trained machine learning model (e.g., a linear regression model) and / or to the pod configuration, i.e., to one or more pod resource limits of the container orchestration platform pod.760

[0087] Continuing with the resource determination phase, the VPA engine component 440 of the control system 410 in Figure 4 receives results from both the formula control component 436 and the prediction component 434 and initiates any adjustments based on those results. For example, based on the results, the VPA engine component may initiate updates to the definitions of deployment objects, such as pod configuration limits. For example, the VPA engine component may dynamically change the pod memory limit, for example, from 100 Mi to 150 Mi, as needed. The VPA engine component may also, in one or more embodiments, work with the horizontal pod autoscaling component 435 to dynamically control pod-level resource usage, as described herein.

[0088] In one or more implementations, when the VPA engine component receives any event from the prediction component 434 or the message service component 442, it retrieves real-time metrics data from the metrics server 430. In response to this event, the VPA engine component checks the auto-scaling mode and, if the current state requires intervention, makes a final decision to scale up or down the pod resource limits and may notify the deployment component 450 to adjust the resource limits as appropriate.

[0089] As described, in one or more embodiments, the third phase includes propagation of the results of the vertical pod autoscaler. Suppose the VPA engine component 440 decides to scale up the pod resource limit for one service, namely service A, which runs on pod 510 in Figure 5. After the resources have been scaled by the VPA engine component, the change in the resource limit may be sent to a message queue through the message service (publisher) 442 to notify other services of the resource change in service A, such as a second service on pod 520 in the example of Figure 5.

[0090] The corresponding VPA engine component receives a message from the message service (subscriber) indicating that the previous service, i.e., service A, has changed its resource limits, and triggers call chain management to find the next service, for example, service B. In one embodiment, the VPA engine takes action to propagate the change for service B using phase 2 processing by calculating a new resource limit for service B. If intervention by the VPA engine component is required, the component also notifies the deployment component to adjust one or more pod resource limits for service B.

[0091] Figure 8 shows an exemplary workflow for publishing and subscribing to limit event changes. The message service component publishes limit changes across bus 810 to message service subscribers 800 who subscribe to the relevant resource limit changes. Subscribers 820 to limit changes can customize which service change events should be subscribed to, for example, which services are relevant, and take action when a limit event change is received. The published limit change equipment 800 can publish when a limit change occurs and send the change event to a channel or bus 810. For example, a memory limit increased by 20% may be indicated in the change event payload 802, which may be propagated to subscribers by default 804.

[0092] As described, in one or more implementations, dynamic adjustment of the pod resource limits of a container orchestration platform pod may propagate for the dynamic adjustment of one or more pod resource limits of one or more other container orchestration platform pods in the computing environment. Dynamic changes to pod resource limits may operate in relation to the horizontal pod autoscaling component and may further depend on the autoscaling mode of the container orchestration platform pod. For example, a post-intervention mode may be configured as a mode in which the control system intervenes when the HPA divides by the existing pod instance value over a set period, i.e., the number of instances of the container orchestration platform pod remains at a specified pod instance limit over a set period. For example, the HPA component may be defined such that the expected memory usage is 80%, the minimum pod instance is 3, and the maximum pod instance is 9. The control system may intervene when the number of instances remains at 3 or 9 for an extended period. For example, if the number of instances is 9 for an extended period, it means that the memory allocation may be too small. Based on this, the control system may be configured to dynamically increase the memory limit by 20% (or another customized value). After running for a period of time, the process then checks whether the number of pod instances has decreased. If not, the memory may continue to increase by 20% until the number of pod instances decreases. Figure 9 shows one example of this process.

[0093] As shown in Figure 9, the process begins with step 900 to detect the autoscaling mode, in particular step 902 to determine if the mode is post-intervention mode. If "no", the process ends with the workflow in Figure 9 903. Otherwise, in post-intervention mode, the process monitors the HPA's minimum and maximum pod instance settings and determines whether the instances are at their limits over a set period, such as over a specified (long) time interval 904. If "no", the process ends with the process 903. Otherwise, the process determines the result of adjusting the pod resource limits from the control system's expression control manager 906 and checks the predicted future resource usage values ​​for the container orchestration platform pods obtained from the prediction component using a trained machine learning model 908. In addition, runtime data usage metrics from the metrics server 430 are checked 910 and a determination is made whether to increase or decrease the pod resource limits by a set percentage, such as 20% (in one example) 912. After increasing or decreasing the limit by the set percentage, the process returns to monitoring the HPA settings to determine whether the number of pod instances remains above the limit for the specified length of time.

[0094] As a concrete example, let's assume the HPA definition is as follows: minReplicas: 3 (Min Pod) maxReplicas: 9 (Max Pod) efficiently: - Type: Municipal resource: name: memory target: type: Utilization averageUtilization: 80 (80% with expected memory usage rate).

[0095] Furthermore, a dynamic resource definition for a pod could be as follows in one example: resources: hpaMode: mode: post-in (use HPA first) increment: 20% (per once) stabilizationWindowMinute: 120 (stable window) upper limits: memory: avg(last(30days))* 2 or 100Mi (Twice the average of the last 30 days or 100Mi) max: 300Mi (Max300Mi) lower limits: memory: avg(last(30days)) * 0.5 or 10Mi (Half the average of the last 30 days or 10Mi) Max: 50Mi (Max 50Mi).

[0096] In the example above, if the expected memory usage is 80% and the HPA is defined with a minimum of 3 pod instances and a maximum of 9 pod instances, the control system may intervene if the number of instances remains at 3 or 9 over a set period, for example, over a set long period (e.g., 1 hour or longer). For example, if the number of pod instances remains at 9 over a long period, it means that the memory allocation may be too small, and as described, the memory may be increased by a custom value, such as 20%. After running for a period, the process monitors whether the number of pod instances has decreased, and if not, continues to increase the memory limit allocation to the pods until the number of pod instances decreases.

[0097] Figure 10 shows another example of the process described above. This example assumes, for example, that a normal workload uses 5 pod instances, and each instance uses 230 Mi of memory. The default value for pod initialization and deployment may be 100 Mi. In this situation, the process begins by determining whether there is historical resource usage data for the pod over the past 30 days 1000 (i.e., an example where the resource usage expression refers to resource usage over the past 30 days). Assuming otherwise, the default value of 100 Mi for Container Orchestration Platform pod initialization and deployment is used. Furthermore, the process determines that the autoscaling mode is in post-intervention mode 1006. At the start of runtime 1010, the process determines the status of the pod instances over a certain time interval, for example, the past 120 minutes 1012, and whether the number of pod instances is at a limit, for example, 3 pods or 9 pods in the example above. If not, the process ends 1013. In the example above, after deployment, it is determined that the number of pod instances has increased to 9, the maximum allowed by the HPA definition in this example, after running for a period of time (e.g., 120 minutes). Once the deployed replica count reaches the maximum replica setting, in one embodiment, the VPA can then check the pod resource usage. For example, if the process is at the pod instance limit, e.g., 9 pod instances, which is the maximum replica setting in this example, the VPA process determines whether the maximum memory usage is exceeded by, for example, a 20% increase in the allocated pod memory limit 1014. If "yes", the process terminates 1013. Otherwise, the pod memory limit is increased by, for example, 20%, and then the process waits until the next evaluation window and repeats the process 1016, in one embodiment, the process is interrupted by a runtime interruption 1018.

[0098] In another mode, the autoscaling mode may be first-in mode, in which case dynamic pod resource limit settings are applied first, and horizontal pod autoscaling is not applied until it is detected that indicators such as pod CPU usage or pod memory usage are at their limits and have not changed significantly over a specified period of time.

[0099] Those skilled in the art will note from the description provided herein that, in one embodiment, a computer implementation method is provided that facilitates processing within a computing environment. The computer implementation method includes: deploying a container orchestration platform pod having one or more pod resources in the computing environment. The pod resources are associated with one or more pod resource limits. Furthermore, the computer implementation method includes monitoring the runtime resource usage of the container orchestration platform pod and predicting future resource usage of the container orchestration platform pod using a trained machine learning model, the predicting step using at least partially the monitored runtime resource usage. In addition, the computer implementation method includes dynamically adjusting the pod resource limits of the one or more pod resource limits of the container orchestration platform pod in the computing environment. The dynamically adjusting step is based on the monitored runtime resource usage and the predicted future resource usage. Advantageously, processing is facilitated within a container architecture-based computing environment by providing enhanced resource utilization. By dynamically adjusting pod resource limits, cost-effective use of computing resources is provided, along with efficient and dynamic resource utilization, thereby supporting on-demand resource allocation without unnecessary waste. Furthermore, by using computing resources more efficiently in container-based computing environments, processes result in cost savings, a lower carbon footprint, and reduced energy consumption.

[0100] In one implementation, the computer implementation method further includes the step of obtaining a trained machine learning model by training the machine learning model on the historical resource usage data of the container orchestration platform pod. Furthermore, in one embodiment, the computer implementation method further includes the step of using the predicted future resource usage of the container orchestration platform pod to continue training the trained machine learning model. In one example, the machine learning model includes a linear regression model. Advantageously, by using the trained machine learning model to predict the future resource usage of the container orchestration platform pod, it becomes easier to dynamically adjust the pod resource limits of the container orchestration platform pod to provide, for example, cost-effective use of computing resources, effective and dynamic utilization of resources, and support on-demand resource allocation without wasting resources.

[0101] In one implementation, the deployment step includes a step of initializing and deploying the container orchestration platform pod. The initialization and deployment step includes, for example, a step of obtaining a dynamic resource definition used for the dynamic allocation of pod resources for the container orchestration platform pod being deployed. The dynamic resource definition includes a resource usage expression. Furthermore, the initialization and deployment step includes a step of creating a deployment object used for deploying the container orchestration platform pod. In one embodiment, the creation step includes a step of parsing the resource usage expression of the dynamic resource definition, and a step of generating one or more initial pod resource limits for use in initializing the container orchestration platform pod based on the parsing of the resource usage expression of the dynamic resource definition. Furthermore, the initialization and deployment steps include initializing and deploying the container orchestration platform pod having the one or more pod resources in the computing environment, wherein the initialization steps apply the generated one or more initial pod resource limits to the deployment of the container orchestration platform pod having the one or more pod resources. Advantageously, resource utilization is further enhanced by generating one or more initial pod resource limits for use in initializing the container orchestration platform pod based on a resource usage formula of dynamic resource definitions, for example, by setting initial pod resource limits via a resource usage formula to efficiently initialize the container orchestration platform pod with appropriate pod resource limits.

[0102] In one implementation, the monitored runtime resource usage includes the runtime central processing unit (CPU) usage and runtime memory usage of the container orchestration platform pod. Advantageously, the step of dynamically adjusting pod resource limits based on the runtime central processing unit (CPU) usage and runtime memory usage of the container orchestration platform pod allows for more efficient use of computing resources in a container-based computing environment, thereby saving processing costs, lowering the carbon footprint, and reducing energy consumption.

[0103] In one embodiment, the dynamically adjusting step includes dynamically increasing the pod resource limit of the container orchestration platform pod in the computing environment during runtime. In another implementation, the dynamically adjusting step includes dynamically decreasing the pod resource limit of the container orchestration platform pod in the computing environment during runtime. Advantageously, the step of dynamically increasing and / or decreasing the pod resource limit of the container orchestration platform pod in the computing environment during runtime provides cost-effective use of computing resources, as well as effective and dynamic resource utilization, and supports on-demand resource allocation without allocating unnecessary resources.

[0104] In one implementation, the dynamically adjusting step is further based on the autoscaling mode for the container orchestration platform pod. In one example, the autoscaling mode is a post-intervention mode, and the dynamically adjusting step is further based on the number of instances of the container orchestration platform pod that remain at a specified pod instance limit over a set period of time. In another example, the autoscaling mode is a first-in mode, and the dynamically adjusting step proceeds before the autoscaling of the number of instances of the container orchestration platform pod. Advantageously, the dynamic adjustment of pod resource limits based on the autoscaling mode provides the ability to dynamically adjust pod resource limits, after or before horizontal pod autoscaling. This further enhances the effective and dynamic use of resources to better balance load handling and avoid inefficient system behaviors such as slow computing system execution.

[0105] In one implementation, the computer implementation method further includes a step of dynamically adjusting the pod resource limits of one or more other container orchestration platform pods in the computing environment based on the dynamic adjustment of the pod resource limits of the container orchestration platform pod. Advantageously, by dynamically adjusting one or more pod resources or one or more other container orchestration platform pods in the computing environment based on the dynamic adjustment of pod resource limits, resource scalability within the computing environment is facilitated by scaling up and / or down resources across multiple container orchestration platform pods based on needs within the container-based computing environment. This facilitates processing within the container architecture-based computing environment and enhances resource utilization. Furthermore, the dynamic utilization of resources across multiple container orchestration platform pods leads to better balancing of load processing and enhances the efficient operation of the container-based computing environment.

[0106] Advantageously, computer implementations, computer systems, and computer program products are disclosed herein that facilitate the control of resource limits for container orchestration platform pods using dynamic data analysis and machine learning model-based predictions. The processing may, in one or more embodiments, include dynamic resource allocation through the use of trained formulas, dynamic prediction of incoming workloads or traffic, pre-initiating scaling actions by collecting usage history and runtime data as inputs for predictions by trained machine learning models, dynamically configuring resources in the microservice call chain, and introducing the integration of different modes, including post-intervention mode and first-in mode, with a horizontal pod autoscaler. In one or more implementations, the computer implementations, computer systems, and computer program products disclosed herein efficiently avoid over-allocation and resource waste within a container-based computing environment. Furthermore, resource allocation is more needs-based, reducing the accumulation of requests and / or service interruptions caused by insufficient resources. Additionally, dynamic pod resource limit adjustment based on data analysis, as described herein, achieves lower technical barriers and reduced reliance on experts.

[0107] The terms used herein are intended to describe only specific embodiments and are not intended to limit the invention. Where used herein, unless the context otherwise explicitly indicates, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well. It will be further understood that the terms “comprise” (and any form of comprise, e.g., “comprises” and “comprising”), “have” (and any form of have, e.g., “has” and “having”), “include” (and any form of include, e.g., “includes” and “including”), and “contain” (and any form of contain, e.g., “contains” and “containing”) are open-ended linking verbs. As a result, a method or device that “comprises,” “has,” “includes,” or “contains” one or more stages or elements will hold, but will not be limited to, those one or more stages or elements. Similarly, any step or element of a method or device that “companies,” “has,” “includes,” or “encompasses” one or more features retains, but is not limited to retaining only, one or more of those features. Furthermore, a device or structure configured in a particular way is configured in at least that way, but may be configured in ways not listed.

[0108] Any means or steps plus functional elements in the following claims are intended to include, if applicable, corresponding structures, materials, actions, and, if equivalents thereof, structures, materials, or actions for performing a function in combination with other claimed elements, as specifically claimed. The descriptions of one or more embodiments are presented for illustrative and explanatory purposes, but are not intended to be exhaustive or to limit oneself to the disclosed forms. Many modifications and variations will be apparent to those skilled in the art. Embodiments have been selected and described in order to best illustrate various aspects and practical applications, and to enable those skilled in the art to understand various embodiments with various modifications to suit a particular intended use.

Claims

1. A computer implementation method for facilitating processing within a computing environment, wherein the computer implementation method is: In the computing environment, the step of deploying a container orchestration platform pod having one or more pod resources is that one or more pod resource limits are associated with each of the one or more pod resources; The step of monitoring the runtime resource usage of the container orchestration platform pod; A step in which a trained machine learning model predicts the future resource usage of the container orchestration platform pod, the prediction step using at least partially the monitored runtime resource usage; and A step of dynamically adjusting the pod resource limits of one or more pod resource limits of the container orchestration platform pod in the computing environment, wherein the dynamic adjustment step is based on the monitored runtime resource usage and the predicted future resource usage. A computer implementation method comprising the above.

2. The computer implementation method according to claim 1, further comprising the step of obtaining the trained machine learning model by training the machine learning model on the historical resource usage data of the container orchestration platform pod.

3. A computer implementation method according to any of the preceding claims, further comprising a step of using the predicted future resource usage of the container orchestration platform pod to continue training the trained machine learning model.

4. The computer implementation method according to any of the above claims, wherein the machine learning model includes a linear regression model.

5. The aforementioned deployment step includes a step of initializing and deploying the container orchestration platform pod, and the aforementioned initialization and deployment step is: Step 1: Obtain a dynamic resource definition used for the dynamic allocation of pod resources for the container orchestration platform pods to be deployed, wherein the dynamic resource definition includes a resource usage expression; The step of creating a deployment object used to deploy the container orchestration platform pod is as follows: The step of parsing the expression for the resource usage of the dynamic resource definition; and The steps include: generating one or more initial pod resource limits for use in initializing the container orchestration platform pod, based on the parsing of the resource usage expression of the dynamic resource definition; and A step of initializing and deploying the container orchestration platform pod having the one or more pod resources in the computing environment, wherein the initialization step applies the generated one or more initial pod resource limits to the deployment of the container orchestration platform pod having the one or more pod resources. A computer implementation method according to any of the preceding claims, including the method described above.

6. The computer implementation method according to any of the above claims, wherein the monitored runtime resource usage includes runtime central processing unit usage of the container orchestration platform pod and runtime memory usage of the container orchestration platform pod.

7. The computer implementation method according to any of the preceding claims, wherein the dynamically adjusting step includes a step of dynamically increasing the pod resource limit of the container orchestration platform pod in the computing environment during runtime.

8. The computer implementation method according to any of the preceding claims, wherein the dynamically adjusting step includes a step of dynamically reducing the pod resource limit of the container orchestration platform pod in the computing environment during runtime.

9. The computer implementation method according to any of the above claims, wherein the dynamically adjusting step is further based on an autoscaling mode for the container orchestration platform pod.

10. The computer implementation method according to claim 9, wherein the auto-scaling mode is a post-intervention mode, and the dynamically adjusting step is further based on the number of instances of the container orchestration platform pod that remain within a specified pod instance limit over a set period of time.

11. The computer implementation method according to claim 9, wherein the auto-scaling mode is a first-in mode, and the dynamically adjusting step is performed before the auto-scaling of the number of instances of the container orchestration platform pod.

12. A computer implementation method according to any of the above claims, further comprising the step of dynamically adjusting the one or more pod resource limits of one or more other container orchestration platform pods in the computing environment based on the dynamic adjustment of the pod resource limits of the container orchestration platform pod.

13. A computer system for facilitating processing within a computing environment, wherein the computer system is: memory; and The computer system comprises at least one processor communicating with the memory, wherein the computer system is configured to perform a method, and the method is: In the computing environment, the step of deploying a container orchestration platform pod having one or more pod resources is that one or more pod resource limits are associated with each of the one or more pod resources; The step of monitoring the runtime resource usage of the container orchestration platform pod; A step in which a trained machine learning model predicts the future resource usage of the container orchestration platform pod, the prediction step using at least partially the monitored runtime resource usage; and A step of dynamically adjusting the pod resource limits of one or more pod resource limits of the container orchestration platform pod in the computing environment, wherein the dynamic adjustment step is based on the monitored runtime resource usage and the predicted future resource usage. Including computer systems.

14. The computer system according to claim 13, further comprising the step of obtaining the trained machine learning model by training the machine learning model on the historical resource usage data of the container orchestration platform pod.

15. The aforementioned deployment step includes a step of initializing and deploying the container orchestration platform pod, and the aforementioned initialization and deployment step is: Step 1: Obtain a dynamic resource definition used for the dynamic allocation of pod resources for the container orchestration platform pods to be deployed, wherein the dynamic resource definition includes a resource usage expression; The step of creating a deployment object used to deploy the container orchestration platform pod is as follows: The step of parsing the expression for the resource usage of the dynamic resource definition; and The steps include: generating one or more initial pod resource limits for use in initializing the container orchestration platform pod, based on the parsing of the resource usage expression of the dynamic resource definition; and A step of initializing and deploying the container orchestration platform pod having the one or more pod resources in the computing environment, wherein the initialization step applies the generated one or more initial pod resource limits to the deployment of the container orchestration platform pod having the one or more pod resources. A computer system according to any one of claims 13 to 14, including the computer system according to any one of claims 13 to 14.

16. The computer system according to any one of claims 13 to 15, wherein the dynamically adjusting step is further based on an auto-scaling mode for the container orchestration platform pod.

17. The computer system according to claim 16, wherein the auto-scaling mode is a post-intervention mode, and the dynamically adjusting stage is further based on the number of instances of the container orchestration platform pod that remain at a specified pod instance limit over a set period of time.

18. The computer system according to claim 16, wherein the autoscaling mode is a first-in mode, and the dynamically adjusting step is performed before the autoscaling of the number of instances of the container orchestration platform pod.

19. The computer system according to any one of claims 13 to 18, further comprising the step of dynamically adjusting the one or more pod resource limits of one or more other container orchestration platform pods in the computing environment based on the dynamic adjustment of the pod resource limits of the container orchestration platform pod.

20. A computer program product for facilitating processing within a computing environment, wherein the computer program product is: The system comprises one or more computer-readable storage media and program instructions embodied therein, wherein the program instructions are: In the computing environment, the step of deploying a container orchestration platform pod having one or more pod resources is that one or more pod resource limits are associated with each of the one or more pod resources; The step of monitoring the runtime resource usage of the container orchestration platform pod; A step in which a trained machine learning model predicts the future resource usage of the container orchestration platform pod, the prediction step using at least partially the monitored runtime resource usage; and A step of dynamically adjusting the pod resource limits of one or more pod resource limits of the container orchestration platform pod in the computing environment, wherein the dynamic adjustment step is based on the monitored runtime resource usage and the predicted future resource usage. A computer program product that is readable by a processing circuit, causing the processing circuit to execute a method including the following.

Images