Serverless Runtime Container Allocation

Abstract

Methods, systems, and computer program products are provided for implementing automated serverless runtime container allocation. The method includes defining a number and associated characteristics of runtime containers required for each of a plurality of worker nodes for execution of a specified workload. The specified workload is dispatched to the plurality of worker nodes, and a specified portion of the specified workload is assigned to each worker node. An application executing a universal runtime container is generated, the universal runtime container including the latent application runtime and associated supported software versions in a tiered modifiable format, and unused tiers are removed from the universal runtime container. The specified workload is executed via the universal runtime container, and a set of available universal runtime containers is replenished on the associated work nodes.

Description

[Background technology]

[0001] The present invention generally relates to methods for automatically generating serverless runtime containers, and more particularly, to methods and related systems for improving runtime container software technology related to generating applications that run in universal runtime containers containing a potential application runtime and associated supported software versions within a layered modifiable format, and executing specified workloads via the universal runtime container. A typical serverless environment must verify that the provided hardware / software capacity is sufficient to handle elastic workloads that are scheduled without notice to system customers. Similarly, providers may need to ensure that operational costs fall within a predetermined threshold range. In an overprovisioned system, operational costs exceed an upper threshold. The present method and related system are configured to address several modifications to improve serverless systems to balance operational costs and system performance. Summary of the Invention

[0002] A first aspect of the present invention provides a serverless runtime container generation method including: defining, by a processor of a centralized maintenance device, a number and associated characteristics of runtime containers required for each worker node of a plurality of worker nodes for execution of a specified workload; dispatching, by the processor via a plurality of cooperative controllers, the specified workload to the plurality of worker nodes; allocating, by the processor via the plurality of cooperative controllers, specified portions of the specified workload to respective worker nodes; generating, by the processor based on a result of the allocating, an application executing a universal runtime container including a plurality of potential application runtimes and associated supported software versions in a layered modifiable format including a plurality of layers; removing, by a processor executing a refill agent component, unused layers of the plurality of layers of the universal runtime container; executing, by the processor in response to generating the universal runtime container, the specified workload via the universal runtime container; and replenishing, by the processor via the plurality of cooperative controllers in response to the executing, a set of available universal runtime containers on associated worker nodes of the plurality of work nodes.

[0003] Some embodiments of the present invention further provide a process for determining that each worker node includes a designated number of hardware and software resources for enabling universal runtime containers, a process for negotiating workload capacity among multiple cooperative controllers, and a process for determining whether each container in the set of available universal runtime containers includes an initialization container or an invalidation container configured for initialization. These embodiments advantageously provide an efficient means for generating and allocating specific pre-built runtime containers, including universal runtime containers, a centralized maintenance mechanism including a refiller component, and a dispatch mechanism to enable improved capacity utilization.

[0004] A second aspect of the present invention is a computer program product including a computer readable hardware storage device having computer readable program code stored thereon, the computer readable program code including an algorithm that, when executed by a processor of a centralized maintenance device, implements a serverless runtime container generation method, the method comprising: defining, by the processor, a number and associated characteristics of runtime containers required for each worker node of a plurality of worker nodes for execution of a specified workload; dispatching, by the processor via a plurality of cooperative controllers, the specified workload to the plurality of worker nodes; allocating, by the processor via the plurality of cooperative controllers, designated portions of the specified workload to each worker node; and determining, by the processor, a performance indicator based on a result of the allocating. generating, by a processor executing a replenishment agent component, an application that executes a universal runtime container that includes a plurality of potential application runtimes and associated supported software versions within a layered modifiable format that includes a plurality of layers based on the universal runtime container; removing, by a processor executing a replenishment agent component, an unused layer from the plurality of layers of the universal runtime container; executing, by the processor in response to generating the universal runtime container, a specified workload through the universal runtime container; and replenishing, by the processor via the plurality of cooperative controllers in response to executing, a set of available universal runtime containers on associated worker nodes from the plurality of work nodes.

[0005] Some embodiments of the present invention further provide a process for determining that each worker node includes a designated number of hardware and software resources for enabling universal runtime containers, a process for negotiating workload capacity among multiple cooperative controllers, and a process for determining whether each container in the set of available universal runtime containers includes an initialization container or an invalidation container configured for initialization. These embodiments advantageously provide an efficient means for generating and allocating specific pre-built runtime containers, including universal runtime containers, a centralized maintenance mechanism including a replenishment component, and a dispatch mechanism to enable improved capacity utilization.

[0006] A third aspect of the present invention provides a centralized maintenance device including a processor coupled to a computer readable memory unit, the memory unit including instructions that, when executed by the processor, implement a serverless runtime container generation method, the method including: defining, by the processor, a number and associated characteristics of runtime containers required for each worker node of a plurality of worker nodes for execution of a specified workload; dispatching, by the processor via a plurality of cooperative controllers, the specified workload to the plurality of worker nodes; allocating, by the processor via the plurality of cooperative controllers, designated portions of the specified workload to the respective worker nodes; and generating, by the processor based on results of the allocating, a layered modifiable format including a plurality of layers. generating, by a processor executing a replenishment agent component, an application that runs a universal runtime container, the universal runtime container including a plurality of potential application runtimes and associated supported software versions within the universal runtime container; removing, by a processor executing a replenishment agent component, an unused layer of the plurality of layers of the universal runtime container; executing, by the processor in response to generating the universal runtime container, a specified workload through the universal runtime container; and replenishing, by the processor via the plurality of cooperative controllers in response to executing, a set of available universal runtime containers on associated worker nodes of the plurality of work nodes.

[0007] Some embodiments of the present invention further provide a process for determining that each worker node includes a designated number of hardware and software resources for enabling universal runtime containers, a process for negotiating workload capacity among multiple cooperative controllers, and a process for determining whether each container in the set of available universal runtime containers includes an initialization container or an invalidation container configured for initialization. These embodiments advantageously provide an efficient means for generating and allocating specific pre-built runtime containers, including universal runtime containers, a centralized maintenance mechanism including a replenishment component, and a dispatch mechanism to enable improved capacity utilization.

[0008] The present invention advantageously provides a simple method and associated system that can automatically generate applications that run in serverless runtime containers. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 illustrates a system for improving runtime container software techniques related to generating applications that run in a universal runtime container, including a latent application runtime and associated supported software versions in a layered modifiable format, and running a specified workload via the universal runtime container, according to an embodiment of the present invention. [Figure 2]FIG. 2 illustrates an algorithm detailing a process flow enabled by the system of FIG. 1 for generating an application that executes a universal runtime container, including a latent application runtime and associated supported software versions in a layered modifiable format, and for improving runtime container software technology associated with executing a specified workload via the universal runtime container, in accordance with an embodiment of the present invention. [Figure 3] 2 is an internal structural diagram of the machine learning software / hardware structure and / or circuit / software of FIG. 1 according to an embodiment of the present invention. [Figure 4A] FIG. 1 illustrates a process for enabling improved capacity handling of a dispatcher component, according to an embodiment of the present invention. [Figure 4B] FIG. 1 illustrates a process for enabling improved capacity handling of a dispatcher component, according to an embodiment of the present invention. [Figure 4C] FIG. 1 illustrates a process for enabling improved capacity handling of a dispatcher component, according to an embodiment of the present invention. [Figure 4D] FIG. 1 illustrates a process for enabling improved capacity handling of a dispatcher component, according to an embodiment of the present invention. [Figure 5] 2 illustrates a computer system used by the system of FIG. 1 for generating applications that run in a universal runtime container, including a latent application runtime and associated supported software versions in a layered modifiable format, and for improving runtime container software techniques related to running a specified workload via the universal runtime container, according to an embodiment of the present invention. [Figure 6] FIG. 1 illustrates a cloud computing environment according to an embodiment of the present invention. [Figure 7] FIG. 1 illustrates a set of functional abstraction layers provided by a cloud computing environment in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] FIG. 1 illustrates a system 100 for improving runtime container software technology related to generating a (software) application that executes a universal runtime container, including a latent application runtime and associated supported software versions in a layered modifiable format, and executing a specified workload via the universal runtime container, according to an embodiment of the present invention. In a typical serverless environment, a system provider may need to verify that the hardware / software capacity provided (e.g., memory, CPU, etc.) is sufficient to handle elastic workloads that are scheduled without notice to system customers. Similarly, the provider may need to ensure that operational costs fall within a predetermined threshold range. An over-provisioned system will cause operational costs to exceed an upper threshold. Similarly, a system configured to operate with minimal hardware may be completely unable to handle the required workload, forcing cloud providers to balance operational costs with system performance. The system 100 is configured to address several modifications to improve serverless systems to balance operational costs and system performance. Improvements to serverless systems include: 1. Creating and allocating specific pre-built runtime containers, including universal runtime containers; 2. Creating a centralized maintenance organization that includes a replenishment component; 3. Creating a dispatch mechanism to enable improved capacity utilization; may include:

[0011] The system 100 of FIG. 1 includes an edge server 107, controllers 106a and 106b, a data store 112, a messaging component 109, a replenishment component 102, and worker nodes 104a...104n, interconnected through a network 117. The replenishment component 102 includes hardware and software components configured to maintain a workload provisioning plan that defines the number of runtime containers required for each workload. The hardware and software components of the replenishment component 102 include a workload monitor component 102a, a workload protector component 102b, and a capacity manager component 102c. The replenishment component 102 is configured to create and apply a provisioning plan that is rolled out to all worker nodes 104a...104n. The provisioning plan consists of the type and number of containers that each designated worker node should maintain within its pre-warm pool. Consuming the message engine topics allows the replenishment component 102 to determine which requests are currently waiting to be executed by a worker node. A second data source is pulled from the data store 112, which contains information about previously executed workloads. The capacity manager component 102c uses all of the data sources to create a provisioning plan for each individual worker node.

[0012] The worker nodes 104a...104n are configured to execute the submitted workload.

[0013] Each worker node 104a...104n (e.g., as shown in close-up 122 of worker node 104a) includes a container bridge 119 (i.e., for spawning and managing runtime containers 108), an actual running container 114, an invoker component 121, a replenishment agent 123 (i.e., an agent that listens for replenishment component 102 and executes replenishment requests issued by central replenishment), and a universal runtime container 127. The replenishment component 102, controllers 106a and 106b, and worker nodes 104a...104n each include dedicated circuitry (which may include dedicated software), sensors, and machine learning software code / hardware structures (i.e., including machine learning software code). The sensors may include any type of internal or external sensor, including ultrasonic 3D sensor modules, temperature sensors, ultrasonic sensors, optical sensors, video search devices, audio search devices, humidity sensors, voltage sensors, pressure sensors, etc., among others. The replenishment component 102, the controllers 106a and 106b, and the worker nodes 104a...104n may each include an embedded device. An embedded device is defined herein as a dedicated device or computer that includes a combination of computer hardware and software (either fixed-capability or programmable) specifically designed to perform a dedicated function. A programmable embedded computer or device may include a dedicated programming interface. In one embodiment, the replenishment component 102, the controllers 106a and 106b, and the worker nodes 104a...104n may each include a dedicated hardware device that includes dedicated (non-general-purpose) hardware and circuitry (i.e., dedicated, discrete, non-general-purpose analog, digital, and logic-based circuitry) for performing (independently or in combination) the processes described with respect to Figures 1-10.The dedicated, discrete, non-general-purpose analog, digital, and logic-based circuitry may include unique, purpose-designed components (e.g., dedicated integrated circuits, such as application-specific integrated circuits (ASICs), designed solely for implementing automated processes to improve runtime container software technology associated with generating a universal runtime container containing a potential application runtime and associated supported software versions in a layered, modifiable format and executing a specified workload via the universal runtime container). Network 117 may include any type of network, including, among others, a 5G telecom network, a local area network (LAN), a wide area network (WAN), the Internet, a wireless network, etc. Alternatively, network 117 may include an application programming interface (API).

[0014] System 100 improves container instantiation by using a universal runtime container instead of using specific pre-built runtime containers associated with standard programming languages. System 100 is configured to build a universal runtime container that includes all necessary runtimes and supported software versions. Before placing a workload into the container and running it, system 100 removes all runtime layers not required for the workload. All runtimes / versions are represented as layers within the container and are easily modified and / or removed.

[0015] System 100 reduces the number of different containers that require pre-provisioning. Previous container systems required that a different container be created for each supported runtime. Because a universal runtime container can be valid for any workload, system 100 enables processes that require only one universal container, thereby improving system flexibility. Thus, a calling component does not need to create individual containers based on current requirements, but may use an existing universal container for any workload, thereby simplifying container management so that only one container requires maintenance (e.g., software updates, validation, etc.).

[0016] The system 100 enables an improved container supply management process through a replenishment component 102 (i.e., a central replenishment component) mechanism. The replenishment component 102 is configured to maintain a provisioning plan that defines the number of runtime containers required within each caller component 121. The system 100 then communicates the provisioning plan to each worker node 104a...104n via an agent model. Similarly, the replenishment agent 123 applies the provisioning plan by creating or deleting containers as needed. For example, when a workload is assigned, the replenishment agent 123 applies the provisioning plan to reduce the size of the runtime container by removing unnecessary runtimes, thereby resulting in additional space being available on the associated node. The provisioning plan may indicate that the replenishment agent 123 may allocate a new universal container when the total available space exceeds the size of the universal container.

[0017] System 100 enables a caller component (e.g., caller component 121) on each worker node (e.g., worker node 104a) to determine whether it must create a new runtime container or can use an existing runtime container after consuming the upcoming workload item. Replenishment component 102 can enable a transfer of responsibility from caller component 121 to replenishing component 102. Additionally, the creation and preparation process associated with the container can begin before the actual workload arrives at the caller component.

[0018] The replenishment component 102 may be further improved by adding machine learning capabilities to analyze historical data in relation to creating provisioning plans, which may improve predictions of expected workloads for associated actions. 1. Faster worker node startup 2. Optimized cold start performance for faster workload execution 3. Prepare for reoccurring workload patterns 4. Improved resource management with optimized hardware / software cleanup procedures This may result in better performance for

[0019] System 100 further provides improved capacity handling for the dispatcher component. For example, two instances of the controller component can be enabled to dispatch incoming workloads to worker nodes based on the required and available hardware / software / memory capacity. Thus, both controllers are enabled to manage half the capacity for each individual worker node, thereby allowing the controllers to negotiate the ratio of the capacities associated with their management. If a controller is unable to allocate new workloads due to insufficient capacity, an additional controller is configured to provide the missing capacity for the associated node. If the additional controller is able to provide the requested capacity, the request is approved, and the associated capacity for the worker node is reduced by the requested amount. The (original) controller then increases the overall capacity for the worker node by the requested amount and allocates the workload. Additionally, all requests and responses include information related to the capacity managed by each controller. When communication synchronization problems exist, periodic exchange of relevant state information is used to repair capacity management.

[0020] The following example illustrates the process involved in executing a workload on a worker node from a placement request to the workload being executed.

[0021] The process is enabled to run within a cloud structure and provide a pre-packaged runtime component. The process includes enabling a calling component to receive a workload placement request, locating containers with associated software requirements, assigning workloads to the associated containers, and initializing and running the workloads. The process is further detailed as follows:

[0022] A request for workload placement is received by a caller component. The request includes a specification related to the required runtime or set of runtimes. The specification includes a declarative description of the minimum set of software components required to execute the workload. The specification may additionally include a description of the required software and its respective minimum version numbers. The software may include a runtime (e.g., JRE or Python® runtime) and a set of required libraries. The specification is associated with the workload requirements. The caller component then generates a request for an inactive container that matches the runtime specification from the container inventory on the node. If a container that meets the specification exists, it is marked as active in the inventory. Similarly, the caller component places the workload in the container for execution. If the container inventory does not contain a container related to the workload specification, the container registry is analyzed to determine whether it contains a template related to the workload specification that can be instantiated to provide the required container. The container registry presents available container templates and associated installed software. If the container registry contains a template that matches the workload requirements, the caller component creates and starts an instance of the template and deploys and runs the workload in the container. A prerequisite for template instantiation is that the associated node must have sufficient resources to fit into the instantiated container. Sufficient management of container resources is explained below.

[0023] If the container registry does not contain a template that matches the requirements of the workload to be deployed, a template management process is triggered to create an appropriate container template based on the workload requirements. The created container template is assigned to the registry. The template management process can include manual or automated processes such as: 1. The template requirements are validated for correctness. For example, the specification associated with a deployment request may contain a non-existent or incompatible version number, preventing the system from locating the template. 2. Process dependencies are associated with required software and versions, e.g., the terms of use of required software are validated to ensure that the template being built meets the cloud provider's policies and the legal terms of the contract between the cloud provider and the workload requestor. 3. Requirements are validated against the resource limitations (hardware or software) of the cloud environment to ensure that the instantiation template can execute the workload as expected. 4. The necessary software packages are downloaded and installed to create the template. 5. The template is deployed in the container registry.

[0024] The system 100 is configured to set up a cloud environment for workload processing such that a cloud provider can initiate a template registry containing a set of standard templates generated based on programming language popularity or as a result of customer surveys. Alternatively, the template registry may be initially configured as an empty structure, with all template creation associated with the template management process, as described above. Initiating the process with a set of standard templates can speed response times for early deployment.

[0025] The preceding discussion of container resource management results in the optimization of resource consumption, such that the period occurring between receiving a placement request and the workload running on the container is referred to as the “time-until-up” period. Because the container initialization process can have a significant impact on the time-until-up period, fast response times may require containers to be pre-provisioned on nodes. Key success factors for enabling minimal time-until-up periods across workloads may include maintaining an appropriate set of templates in the container registry and pre-provisioning a mix of templates that are likely to match upcoming workload requirements. Incorrect decisions may additionally have a negative impact on the time-until-up period because additional steps may be required to remediate, such as deleting pre-provisioned containers. For example, if containers are pre-provisioned on a node based on runtimes A, B, and C, and a customer initiates a request for a specified number of workloads requiring runtimes D and E, containers may need to be swapped on the node, and new containers may need to be instantiated, which may incur greater costs than placing workloads on already pre-provisioned containers.

[0026] A Universal Container is defined herein as a container template (structure) that includes all pre-installed software packages required to meet the specifications of all incoming workloads. Universal Containers include partitioning capabilities provided by the container (software / hardware) engine to allow for easy removal of unnecessary software components to free up resources for future workload placement. Software components may be determined to be incompatible, such that they may not be able to coexist within a single container. Therefore, multiple Universal Containers may be required.

[0027] Universal container processing may include analyzing workload requirements relative to software present in the universal container. If an inconsistency is detected, the calling component may add the required software to the universal container. Similarly, if a universal container matches the workload requirements, the inventory is checked against inactive universal containers, unnecessary software components are removed from the universal container, and the workload is placed into the container and executed. The container is marked as active in the inventory.

[0028] Universal Containers offers the following benefits:

[0029] Simplifying decisions related to locating matching containers. For example, workload requirements may only need to be matched once against the software present in a Universal Container. The matching process only needs to be run on nodes containing at least one inactive container, since all pre-provisioned inactive containers contain an instance of the Universal Container image. If a Universal Container does not meet the workload requirements, it can be extended with the necessary software during the template management process. The added software will then be present for use by future workloads.

[0030] Universal containers may require more storage space than containers configured to address specific workload requirements. Therefore, because universal containers are tuned to the minimum software required for a workload, overhead exists only while the container is inactive or while a workload is placed and during the container resizing process, which includes removing unnecessary software. A replenishment component can be configured to manage available space by intelligently allocating new universal containers.

[0031] FIG. 2 illustrates an algorithm detailing a process flow enabled by the system 100 of FIG. 1 for generating an application that executes a universal runtime container, including a latent application runtime and associated supported software versions in a layered modifiable format, and for improving runtime container software technology associated with executing a specified workload via the universal runtime container, according to an embodiment of the present invention. The steps in the algorithm of FIG. 2 may be performed in any order by a computer processor executing computer code. Additionally, the steps in the algorithm of FIG. 2 may be performed in combination by the edge server 107, controllers 106a and 106b, messaging component 109, replenishment component 102, and worker nodes 104a...104n of FIG. 1. In step 200, the number and associated characteristics of runtime containers required for each of a plurality of worker nodes are defined (by a centralized maintenance device) for execution of a specified workload. In step 202, the specified workload is dispatched to the plurality of worker nodes via a cooperative controller. Similarly, workload capacities between the cooperative controllers may be negotiated, and the dispatch process may be performed based on the results of the negotiation. The dispatch process may be performed based on the hardware and software capacities of the cooperative controllers.

[0032] In step 204, a designated portion of the designated workload is assigned to each worker node via the collaborative controller. In step 208, an application running a universal runtime container is generated based on the results of step 204. The universal runtime container includes a latent application runtime and associated supported software versions in a layered modifiable format that includes multiple layers. Additionally, each worker node may be determined to include a designated number of hardware and software resources for enabling the universal runtime container.

[0033] In step 210, unused layers of the plurality of layers are removed by execution of a replenishment agent component. The removal process may further include removing application runtimes of latent application runtimes that are not required for execution of each specified portion of the specified workload. The replenishment agent component may include a statistical process or machine learning capability configured to analyze historical data associated with previous instances of executing the specified workload and generating future instances of the universal runtime container.

[0034] In step 212, the specified workload is executed by the universal runtime container. In step 214, the set of universal runtime containers available on the associated worker node is replenished. Additionally, it may be determined whether each container (in the set of available universal runtime containers) includes an initialization container or an invalidation container configured for initialization.

[0035] 3 shows a detailed diagram of a worker node 300 configured to execute a workload, according to an embodiment of the present invention. The worker node 300 includes a caller component 302, a replenishment agent component 304, and a container bridge 308 that is responsible for handling all runtime container-related operations (for running containers 310 and pre-warm containers 312), such as create, suspend, and remove operations. The caller component 302 is configured to manage the entire lifecycle of incoming workloads, as well as the running containers 310 and pre-warm containers 312. The worker node 300 is associated with three types of workload execution: cold execution, pre-warm execution, and warm execution.

[0036] In terms of performance and latency, warm execution is associated with faster operation than pre-warm and cold execution. Warm execution allows the caller component 302 to reuse running user containers to execute incoming workload requests. Similarly, cold execution requires new containers for initialization. To minimize the amount of cold execution, the caller component 302 includes a pool of pre-warm containers that are initialized but not yet customized for a specific customer or workload. The distribution and quantity of pre-warm containers is stored as an evaluation configuration during the caller component's startup. The configuration settings are static and can only be changed with deployment.

[0037] The replenishment agent component 304 is responsible for managing the pool of warm and pre-warm containers. The replenishment agent component 304 enables the capability to pre-provision a specified type and number of containers based on the current needs in the system. Thus, the opportunity to run a workload as a cold execution is dramatically reduced.

[0038] The replenishment agent component 304 enables a process to reduce processing wait times or avoid cold runs. This process begins when the caller component 302 consumes a workload and determines whether the workload requires starting a new container (cold run) or whether a (pre-)warm container is available to run the workload. When a worker node has already started the required container, the determination may be made before the caller component 302 receives the workload, thereby avoiding a cold run. Similarly, if the replenishment agent component 304 cannot start a container immediately, the wait time until the workload is ready to run may be reduced.

[0039] The replenishment agent component 304 enables the system 100 of FIG. 1 to prepare for respawning workloads by evaluating historical data. The replenishment agent component 304 can detect the need for specific container types, and based on this information, a provisioning plan is generated to start the required containers before the actual workload is executed by the associated worker node. Thus, the system 100 (of FIG. 1) can respond in a timely manner without dealing with cold executions. The replenishment agent component 304 can additionally detect sequences of previously executed detected workloads. In this case, the replenishment agent component 304 can additionally provision the required containers before the actual workload arrives at the worker node. The use of historical data enables the replenishment agent component 304 to enable the system to predict the type and amount of incoming workloads. The replenishment agent component 304 provides pluggable prediction capabilities that can interface with both simple statistical modules and advanced model-based prediction systems, thereby improving system performance by reducing the ratio of cold executions to warm executions.

[0040] 4A-4D illustrate a process for enabling improved capacity handling of a dispatcher component in accordance with an embodiment of the present invention. For example, when a controller is unable to place a workload on a specified worker node due to a lack of capacity associated with a request, additional capacity may be allocated from an additional controller instead of attempting to locate another worker node for workload placement. The amount of required capacity and the total amount of currently managed capacity may be transmitted with the request. The additional controller may verify whether it can provide the requested amount of capacity on a particular worker node and, based on the available capacity, accept or reject the request. If the additional worker node accepts the request, it reduces its available capacity by the requested amount. The associated response indicates the amount of current capacity associated with the controller management.

[0041] 4A-4D show an example implementation involving two controllers initially managing 8 GB of memory on a worker node containing 16 GB of capacity. Both controllers occupy the worker node with 6 GB of existing workload each, leaving each controller with 2 GB of remaining capacity.

[0042] FIG. 4A illustrates a first example process that results in a successful request to shift workload capacity. The process begins when a workload 404a requiring 3 GB of capacity is requested at controller 400a. Controller 400a is unable to place the workload on a worker node because it only has 2 GB of remaining capacity. In response, controller 400a initiates a request for 1 GB of additional capacity from controller 402a. Controller 402a accepts the request because it can provide 1 GB of memory to the worker node. Through the process described above, the total amount of memory it manages is reduced to 7 GB, and an acceptance response is sent. In response to receiving the response, controller 400a increases the amount of capacity it manages to 9 GB and places workload 404a on the worker node.

[0043] FIG. 4B illustrates a second example process involving an unsuccessful request to shift capacity. The process begins when a workload 404b requiring 5 GB of capacity is requested at controller 400b. Controller 400b is unable to place workload 404b on a worker node because it only has 2 GB of remaining capacity. In response, controller 400b initiates a request for 3 GB of additional capacity from controller 402a. Controller 402b rejects the request because it cannot provide 3 GB of memory for the worker node. Similarly, controller 402b maintains the total amount of memory it manages at 8 GB and sends a rejection response. In response to receiving the response, controller 400b must either place workload 404b on a different worker node or postpone the requested workload 404b until capacity is again available.

[0044] FIG. 4C illustrates a third example process involving a self-healing result in a successful capacity shift request. Due to a previous miscommunication, controller 402c assumes it currently manages only 6 GB of memory on the worker node, and a workload 404c requiring 3 GB of capacity is requested at controller 400c. Controller 400c is unable to place workload 404c on the worker node because it only has 2 GB of remaining capacity. In response, controller 400c requests 1 GB of additional capacity from controller 402a. Similarly, controller 400c sends a request along with additional information indicating that controller 400c currently manages a total of 8 GB of memory. In response, controller 402c adds up both total amounts of capacity it manages and determines that there is an additional 2 GB difference to add to the amount of capacity it manages. Based on the new total capacity, controller 402c can grant 1 GB of memory on the worker node and therefore accepts the request. Controller 402c adjusts the total amount of memory it manages to 7 GB and sends an acceptance response. In response to receiving the response, controller 400c increases the amount of capacity it manages to 9 GB and places workload 404c on the worker node.

[0045] FIG. 4D illustrates a fourth example process involving an unsuccessful request to shift capacity due to self-healing results. Due to a previous miscommunication, controller 400d determines that it currently manages only 9 GB of memory on a worker node, and a workload 404d requiring 5 GB of capacity is requested at controller 400d. Controller 400d is unable to place workload 404d on the worker node because it only has 3 GB of remaining capacity. In response, controller 400d requests 2 GB of additional capacity from controller 402d. Similarly, controller 400d sends a request along with additional information indicating that controller 400d currently manages a total of 9 GB of memory. In response, controller 402d adds up both amounts of managed capacity, determines that 1 GB of memory is missing, and therefore removes 1 GB from the amount of managed capacity. Controller 402d rejects the request because it cannot provide 2 GB of memory on the worker node. Similarly, controller 402d adjusts the total amount of memory it manages to 7 GB and sends a rejection response. In response to receiving the response, controller 400d must either place workload 404d on a different worker node or postpone the requested workload 404d until capacity is again available.

[0046] FIG. 5 illustrates a computer system 90 (e.g., edge server 107, controllers 106a and 106b, messaging component 109, replenishment component 102, and worker nodes 104a...104n of FIG. 1) used by or configured with the system of FIG. 1 to improve runtime container software techniques related to generating applications that run universal runtime containers containing latent application runtimes and associated supported software versions in a layered modifiable format and execute specified workloads via the universal runtime containers, according to an embodiment of the present invention.

[0047] Aspects of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all of which may be generally referred to herein as a "circuit," "module," or "system."

[0048] The present invention may be a system, a method, or a computer program product, or a combination thereof. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the present invention.

[0049] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, punch cards, or mechanically encoded devices such as ridge structures in grooves that allow instructions to be recorded thereon, and any suitable combination of the foregoing. As used herein, computer-readable storage media should not be construed as ephemeral signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., light pulses passing through fiber optic cable), or electrical signals transmitted through electrical wires.

[0050] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium in each computing / processing device.

[0051] Computer-readable program instructions for carrying out the operations of the present invention may be either source code or object code written in any combination of one or more programming languages, including assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or object-oriented programming languages ​​such as Smalltalk®, C++, and conventional procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (e.g., through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions by individualizing the electronic circuitry using state information of the computer-readable program instructions to perform aspects of the present invention.

[0052] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0053] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, a mobile device, a smart watch, or other programmable data processing device to manufacture a machine, such that the instructions, when executed by the processor of the computer or other programmable data processing device, create means for performing the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams. These computer-readable program instructions may also be stored on a computer-readable storage medium that may direct a computer, programmable data processing device, or other device, or combination thereof, to function in a particular manner, such that the computer-readable storage medium having instructions stored therein comprises an article of manufacture containing instructions that implement aspects of the functions / acts specified in one or more blocks of the flowcharts and / or block diagrams.

[0054] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing device, or other device to cause a series of operational steps to be executed on the computer, other programmable device, or other device to generate a computer-implemented process, such that the instructions executing on the computer, other programmable device, or other device perform the functions / operations specified in one or more blocks of the flowcharts and / or block diagrams.

[0055] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may actually be implemented as a single step, executed concurrently, substantially concurrently, partially, or fully overlapping in time, or the blocks may sometimes be executed in the reverse order depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or operations or executes a combination of dedicated hardware and computer instructions.

[0056] The computer system 90 shown in FIG. 5 includes a processor 91, an input device 92 coupled to the processor 91, an output device 93 coupled to the processor 91, and memory devices 94 and 95, respectively, coupled to the processor 91. The input device 92 may be, among other devices, a keyboard, a mouse, a camera, a touchscreen, etc. The output device 93 may be, among other devices, a printer, a plotter, a computer screen, a magnetic tape, a removable hard disk, a floppy disk, etc. The memory devices 94 and 95 may be, among other devices, a hard disk, a floppy disk, a magnetic tape, optical storage such as a compact disk (CD) or a digital video disk (DVD), a dynamic random access memory (DRAM), a read-only memory (ROM), etc. The memory device 95 includes computer code 97. Computer code 97 includes an algorithm (e.g., the algorithm of FIG. 2) for improving runtime container software technology related to generating a universal runtime container that includes a latent application runtime and associated supported software versions in a layered, modifiable format, and executing a specified workload via the universal runtime container. Processor 91 executes computer code 97. Memory device 94 includes input data 96. Input data 96 includes input required by computer code 97. Output device 93 displays output from computer code 97. Either or both of memory devices 94 and 95 (or one or more additional memory devices, such as a read-only memory (ROM) device or firmware 85) may include an algorithm (e.g., the algorithm of FIG. 2) and may be used as a computer-usable medium (or computer-readable medium or program storage device) having computer-readable program code embodied therein and / or other data stored thereon, the computer-readable program code including computer code 97.Generally, the computer program product (or alternatively, article of manufacture) of computer system 90 may include a computer-usable medium (or program storage device).

[0057] In some embodiments, rather than being stored and accessed from a hard drive, optical disk, or other writable, rewritable, or removable hardware memory device 95, the stored computer program code 84 (e.g., including algorithms) may be stored on, or accessed by the processor 91 directly from, a static, non-removable, read-only storage medium such as a ROM device or firmware 85. Similarly, in some embodiments, the stored computer program code 97 may be stored as, or accessed by the processor 91 directly from, such a ROM device or firmware 85, rather than from a more dynamic or removable hardware data storage device 95 such as a hard drive or optical disk.

[0058] Additionally, any of the components of the present invention may be created, integrated, hosted, maintained, deployed, managed, serviced, etc., by a service provider offering to improve runtime container software technology related to generating a universal runtime container that includes a latent application runtime and associated supported software versions in a layered, modifiable format and executing a specified workload via the universal runtime container. Accordingly, the present invention discloses a process for deploying, creating, integrating, hosting, maintaining, or integrating a computing infrastructure, or combinations thereof, comprising integrating computer-readable code into a computer system 90. The code in combination with the computer system 90 is capable of executing a method that enables a process for improving runtime container software technology related to generating a universal runtime container that includes a latent application runtime and associated supported software versions in a layered, modifiable format and executing a specified workload via the universal runtime container. In another embodiment, the present invention provides a business method that performs the process steps of the present invention on a subscription, advertising, or fee basis, or a combination thereof. That is, a service provider, such as a solutions integrator, may offer to generate a universal runtime container that includes a potential application runtime and associated supported software versions in a layered, modifiable format, as well as enable a process for improving the runtime container software technology associated with running a specified workload via the universal runtime container. In this case, the service provider may create, maintain, support, etc., the computer infrastructure that performs the process steps of the present invention for one or more customers.In exchange, the service provider may receive payments from the customer under a subscription and / or fee agreement, and / or the service provider may receive payments from the sale of advertising content to one or more third parties.

[0059] 5 illustrates computer system 90 as a configuration of hardware and software, any configuration of hardware and software known to those skilled in the art may be utilized for the purposes described above in conjunction with computer system 90 of FIG. 5. For example, memory devices 94 and 95 may be part of a single memory device rather than separate memory devices.

[0060] Cloud Computing Environment Although this disclosure includes detailed descriptions of cloud computing, it should be understood that implementations of the teachings herein are not limited to cloud computing environments. Rather, embodiments of the invention can be practiced in conjunction with any other type of computing environment now known or later developed.

[0061] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal administrative effort or service provider interaction. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0062] The characteristics are as follows:

[0063] On-Demand Self-Service: Cloud consumers can unilaterally provision computing capabilities such as server time and network storage automatically as needed, without the need for human interaction with the service provider.

[0064] Wide network access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin-client or thick-client platforms (e.g., cell phones, laptops, and PDAs).

[0065] Resource Sharing: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically allocated and reallocated according to demand. Location independence is implied in that consumers generally have no control or knowledge over the exact location of the resources provided, but may be able to specify location at a higher level of abstraction (e.g., country, state, or data center).

[0066] Rapid scalability: Capabilities can be quickly and elastically provisioned, sometimes automatically, to instantly scale out, and quickly released to instantly scale in. To the consumer, the capabilities available for provisioning often appear unlimited and can be purchased at any time, in any quantity.

[0067] Services are meterable: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at a level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the services being used.

[0068] The service model is as follows:

[0069] Software as a Service (SaaS): The capability offered to the consumer is the use of the provider's applications running on a cloud infrastructure. The applications are accessible from a variety of client devices through thin-client interfaces such as web browsers (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0070] Platform as a Service (PaaS): The capability offered to the consumer is to deploy consumer-created or consumer-acquired applications, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. The consumer does not manage or control the underlying cloud infrastructure, including the network, servers, operating systems, or storage, but does have control over the deployed applications and, to the extent possible, the application hosting environment configuration.

[0071] Infrastructure as a Service (IaaS): The capability offered to consumers is to provision processing, storage, network, and other basic computing resources onto which they can deploy and run any software, which may include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure, but does exercise control over the operating systems, storage, deployed applications, and possibly limited control over select networking components (e.g., host firewalls).

[0072] The deployment model is as follows:

[0073] Private Cloud: The cloud infrastructure is operated exclusively for an organization. The cloud infrastructure may be managed by the organization or a third party and may be on-premise or off-premise.

[0074] Community Cloud: Cloud infrastructure is shared by multiple organizations to support a specific community with shared concerns (e.g., mission, security requirements, policy, and compliance considerations). The cloud infrastructure may be managed by the organizations or a third party and may reside on-premise or off-premise.

[0075] Public Cloud: Cloud infrastructure is made available to the general public or large industry organizations and is owned by an organization that sells cloud services.

[0076] Hybrid Cloud: A cloud infrastructure is a composite of two or more clouds (private, community, or public) that remain unique entities but are joined by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0077] Cloud computing environments are service-oriented, with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure that includes a network of interconnected nodes.

[0078] Referring now to FIG. 6, an exemplary cloud computing environment 50 is shown. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which local computing devices used by cloud consumers, such as a personal digital assistant (PDA) or mobile phone 54A, a desktop computer 54B, a laptop computer 54C, or an automotive computer system 54N, or combinations thereof, may communicate. The nodes 10 may communicate with each other. They may be physically or virtually grouped in one or more networks (not shown), such as a private cloud, a community cloud, a public cloud, or a hybrid cloud, or combinations thereof, as described above. This enables the cloud computing environment 50 to offer infrastructure, platform, or software, or combinations thereof, as a service without the cloud consumer having to maintain resources on their local computing device. The types of computing devices 54A, 54B, 54C, and 54N shown in FIG. 6 are intended to be exemplary only, and it is understood that computing node 10 and cloud computing environment 50 may communicate with any type of computerized device via any type of network or network-addressable connection or combination thereof (e.g., using a web browser).

[0079] Referring now to Figure 7, a set of functional abstraction layers provided by cloud computing environment 50 (see Figure 6) is shown. It should be understood in advance that the components, layers, and functions shown in Figure 7 are intended to be merely exemplary, and embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functions are provided:

[0080] Hardware and software layer 60 includes hardware and software components. Examples of hardware components include mainframe 61, reduced instruction set computer (RISC) architecture-based server 62, server 63, blade server 64, storage device 65, and network and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0081] The virtualization layer 70 provides an abstraction layer at which the following examples of virtual entities may be provided: virtual servers 71, virtual storage 72, virtual networks including virtual private networks 73, virtual applications and operating systems 74, and virtual clients 75.

[0082] In one embodiment, management layer 80 may provide the following functions: Resource provisioning 81 provides dynamic procurement of computing and other resources utilized to execute tasks within the cloud computing environment. Metering and pricing 82 provides cost tracking as resources are utilized within the cloud computing environment and billing or invoicing for the consumption of these resources. In one embodiment, these resources may include application software licenses. Security provides protection for data and other resources as well as identity verification for cloud consumers and tasks. User portal 83 provides consumers and system administrators with access to the cloud computing environment. Service level management 87 provides cloud computing resource allocation and management to ensure required service levels are met. Service level agreement (SLA) planning and fulfillment 88 provides pre-allocation and procurement of cloud computing resources in anticipation of future requirements according to SLAs.

[0083] The Workloads layer 101 provides examples of functionality for which a cloud computing environment may be utilized. Examples of workloads and functionality that may be provided from this layer include mapping and navigation 99, software development and lifecycle management 103, virtual classroom education delivery 133, data analytics processing 134, transaction processing 106, and improving runtime container software technology 110 related to generating a universal runtime container that includes a potential application runtime and associated supported software versions in a layered modifiable format and running specified workloads via the universal runtime container.

[0084] While embodiments of the present invention have been described herein for purposes of illustration, many modifications and changes will become apparent to those skilled in the art. It is, therefore, intended in the appended claims to cover all such modifications and changes as fall within the true spirit and scope of the invention.

Claims

1. 1. A serverless runtime container allocation method, comprising: defining, by a processor of the centralized maintenance device, the number and associated characteristics of runtime containers required for each worker node of a plurality of worker nodes for execution of a specified workload; dispatching the designated workload by the processor via a plurality of cooperative controllers to the plurality of worker nodes; assigning, by the processors via the plurality of cooperative controllers, designated portions of the designated workload to each of the worker nodes; generating, by the processor based on a result of the allocating, an application executing universal runtime container including a plurality of potential application runtimes and associated supported software versions within a layered modifiable format including a plurality of layers; removing, by the processor executing a replenishment agent component, unused layers of the plurality of layers of the universal runtime container; executing, by the processor in response to generating the universal runtime container, the specified workload via the universal runtime container; replenishing, by the processor via the plurality of cooperative controllers in response to said executing, a set of universal runtime containers available on associated worker nodes of said plurality of work nodes; A serverless runtime container allocation method, comprising:

2. 2. The method of claim 1, wherein the removing further comprises removing application runtimes of the latent application runtimes that are not required for execution of the specified portions of each of the specified workloads.

3. 3. The method of claim 1 or 2, wherein the supplemental agent component includes a statistical process or machine learning capability configured to analyze historical data associated with previous instances that execute the specified workload and generate future instances of the universal runtime container.

4. 4. The method of claim 1, further comprising determining, by the processor, that each of the worker nodes includes a designated number of hardware and software resources for enabling the universal runtime container.

5. 5. The method of claim 1, further comprising: negotiating, by the processor, workload capacity among the plurality of cooperative controllers, wherein the dispatching of the specified workload to the plurality of worker nodes is performed based on a result of the negotiating.

6. 6. The method of claim 1, further comprising determining, by the processor, whether each container in the set of available universal runtime containers includes an initialization container or an invalidation container configured for initialization.

7. The method of claim 1 , wherein the dispatching is performed based on hardware and software capacities of the plurality of cooperative controllers.

8. 8. The method of claim 1, further comprising providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer readable code on a server hardware device, the code being executed by the processor to perform the defining, dispatching, allocating, generating, removing, executing, and replenishing.

9. A computer program product for causing a computer to carry out the method according to any one of claims 1 to 8.

10. A computer-readable recording medium having the computer program according to claim 9 recorded thereon.

11. 1. A centralized maintenance device including a processor coupled to a computer-readable memory unit, the memory unit containing instructions that, when executed by the processor, implement a serverless runtime container allocation method, the method comprising: defining, by the processor, the number and associated characteristics of runtime containers required for each worker node of a plurality of worker nodes for execution of a specified workload; dispatching the designated workload by the processor via a plurality of cooperative controllers to the plurality of worker nodes; assigning, by the processors via the plurality of cooperative controllers, designated portions of the designated workload to each of the worker nodes; generating, by the processor based on a result of the allocating, an application executing universal runtime container including a plurality of potential application runtimes and associated supported software versions within a layered modifiable format including a plurality of layers; removing, by the processor executing a replenishment agent component, unused layers of the plurality of layers of the universal runtime container; executing, by the processor in response to generating the universal runtime container, the specified workload via the universal runtime container; replenishing, by the processor via the plurality of cooperative controllers in response to said executing, a set of universal runtime containers available on associated worker nodes of said plurality of work nodes; A centralized maintenance device, including:

12. 12. The centralized maintenance device of claim 11, wherein the removing further comprises removing application runtimes of the latent application runtimes that are not required for execution of the specified portions of each of the specified workloads.

13. 13. The centralized maintenance device of claim 11 or 12, wherein the replenishment agent component includes a statistical process or machine learning capability configured to analyze historical data associated with previous instances that execute the specified workload and generate future instances of the universal runtime container.

14. The method comprises:

14. The centralized maintenance device of claim 11, further comprising determining, by the processor, that each of the worker nodes includes a designated number of hardware and software resources for enabling the universal runtime container.

15. The method comprises:

15. The centralized maintenance device of claim 11, further comprising: negotiating, by the processor, workload capacity among the plurality of cooperative controllers, wherein the dispatching of the specified workload to the plurality of worker nodes is performed based on a result of the negotiating.

Images