Serverless runtime container allocation

Abstract

A method, system, and computer program product for implementing automated serverless runtime container allocation is provided. The method includes defining a plurality of runtime containers and a plurality of worker nodes each of which performs a plurality of runtime containers and associated characteristics required for a specified workload. The specified workload is dispatched to the plurality of worker nodes and a specified portion of the specified workload is assigned to each worker node. An application executing a generic runtime container is generated, the generic runtime container including potential application runtimes and associated supported software versions within a layered modifiable format, and unused layers are removed from the generic runtime container. The specified workload is executed via the generic runtime container and a set of available generic runtime containers are repopulated on the associated worker nodes.

Description

Background Technology

[0001] This invention generally relates to a method for automatically generating serverless runtime containers, and more specifically to a method and associated system for improving runtime container software technology associated with generating applications that execute generic runtime containers and executing specified workloads via these generic runtime containers, which include potential application runtimes and associated supporting software versions within a layered, modifiable format. Typical serverless environments require verification that the provided hardware / software capacity is sufficient to handle highly resilient workloads being scheduled without notifying clients of the system. Similarly, providers may need to ensure that operating costs remain within predetermined thresholds. Over-provisioning of the system drives operating costs beyond the upper threshold. The method and associated system of this invention are configured to address several modifications for improving serverless systems in balancing operating costs and system performance. Summary of the Invention

[0002] A first aspect of the present invention provides a serverless runtime container generation method, comprising: defining, by a processor of a centralized maintenance facility, a plurality of runtime containers and associated features required for each of a plurality of worker nodes to execute a specified workload; dispatching, by the processor, the specified workload to the plurality of worker nodes via a plurality of coordinated controllers; assigning, by the processor, a specified portion of the specified workload to each worker node via the plurality of coordinated controllers; generating, by the processor, an application executing a generic runtime container based on the result of the assignment, the generic runtime container including a plurality of potential application runtimes within a layered modifiable format and associated supported software versions, the layered modifiable format including a plurality of layers; removing unused layers from the plurality of layers of the generic runtime container by a processor performing a repopulation of proxy components; executing the specified workload via the generic runtime container in response to generating the generic runtime container; and, in response to the execution, repopulating a set of available generic runtime containers on associated worker nodes of the plurality of worker nodes by the processor via the plurality of coordinated controllers.

[0003] Some embodiments of the invention also provide for determining each worker node including processes for: enabling a specified number of hardware and software resources for a generic runtime container; negotiating workload capacity among multiple coordinating controllers; and determining whether each generic runtime container in the set of available generic runtime containers includes an initialized container or a disabled container configured for initialization. These embodiments advantageously provide efficient means for generating and allocating specific, pre-built runtime containers including generic runtime containers; generating centralized maintenance mechanisms including refiller components; and generating allocation mechanisms for achieving improved capacity utilization.

[0004] A second aspect of the invention provides a computer program product including a computer-readable hardware storage device storing computer-readable program code including an algorithm for implementing a serverless runtime container generation method when executed by a processor of a centralized maintenance device. The method includes: defining, by the processor, the number of runtime containers required for each of a plurality of worker nodes to execute a specified workload and associated characteristics; assigning the specified workload to the plurality of worker nodes via a plurality of coordinated controllers; assigning a specified portion of the specified workload to each worker node via the plurality of coordinated controllers; generating an application that executes a generic runtime container based on the assignment result, the generic runtime container including a plurality of potential application runtimes within a hierarchical modifiable format and associated supported software versions, the hierarchical modifiable format including a plurality of layers; removing unused layers from the plurality of layers of the generic runtime container by the processor performing a repopulation of agent components; executing the specified workload via the generic runtime container in response to generating the generic runtime container; and in response to the execution, repopulating a set of available generic runtime containers on associated worker nodes of the plurality of worker nodes via the plurality of coordinated controllers.

[0005] Some embodiments of the invention also provide processes for: determining that each worker node includes a specified number of hardware and software resources for enabling a generic runtime container; negotiating workload capacity among multiple coordinated controllers; and determining whether each generic runtime container in the set of available generic runtime containers includes an initialized container or a disabled container configured for initialization. These embodiments advantageously provide efficient means for generating and allocating specific, pre-built runtime containers including generic runtime containers; generating a centralized maintenance mechanism including a refiller component; and generating an allocation mechanism for achieving improved capacity utilization.

[0006] A third aspect of the invention provides a centralized maintenance device including a processor coupled to a computer-readable storage unit, the storage unit including instructions that, when executed by the processor, implement a serverless runtime container generation method, the method comprising: defining by the processor the number of runtime containers required for each of a plurality of worker nodes to execute a specified workload and associated characteristics; assigning the specified workload to the plurality of worker nodes by the processor via a plurality of coordinated controllers; assigning a specified portion of the specified workload to each worker node by the processor via the plurality of coordinated controllers; generating by the processor, based on the assignment result, an application executing a generic runtime container, the generic runtime container including a plurality of potential application runtimes within a hierarchical modifiable format and associated supported software versions, the hierarchical modifiable format including a plurality of layers; removing unused layers from the plurality of layers of the generic runtime container by the processor performing a repopulation of agent components; executing the specified workload by the processor via the generic runtime container in response to generating the generic runtime container; and in response to the execution, repopulating a set of available generic runtime containers on associated worker nodes of the plurality of worker nodes by the processor via the plurality of coordinated controllers.

[0007] Some embodiments of the invention also provide processes for: determining that each worker node includes a specified number of hardware and software resources for enabling a generic runtime container; negotiating workload capacity among multiple coordinated controllers; and determining whether each generic runtime container in the set of available generic runtime containers includes an initialized container or a disabled container configured for initialization. These embodiments advantageously provide efficient means for generating and allocating specific, pre-built runtime containers including generic runtime containers; generating centralized maintenance mechanisms including repopulation components; and generating allocation mechanisms for achieving improved capacity utilization.

[0008] The present invention advantageously provides a simple method and associated system for automatically generating applications that execute serverless runtime containers. Attached Figure Description

[0009] Figure 1 A system for improving runtime container software technology according to an embodiment of the present invention is illustrated. This runtime container software technology is associated with generating applications that execute generic runtime containers and executing specified workloads via the generic runtime containers, which include potential application runtimes within a layered, modifiable format and associated supported software versions.

[0010] Figure 2 An algorithm according to an embodiment of the present invention is shown, which details the process by which... Figure 1The system implements a process flow for improving runtime container software technology, which is associated with generating applications that execute generic runtime containers and executing specified workloads via these generic runtime containers, which include potential application runtimes within a layered, modifiable format and associated supported software versions.

[0011] Figure 3 An embodiment of the present invention is shown. Figure 1 Internal structure diagram of machine learning software / hardware architecture and / or circuit system / software.

[0012] Figures 4A-4D An improved capacity handling process for implementing a dispatcher component according to an embodiment of the present invention is illustrated.

[0013] Figure 5 The invention is illustrated by an embodiment of the invention. Figure 1 The system uses a computer system for improving and generating applications that execute generic runtime containers and runtime container software technologies associated with executing specified workloads via generic runtime containers, wherein the generic runtime container includes a potential application runtime within a layered, modifiable format and associated supported software versions.

[0014] Figure 6 A cloud computing environment according to an embodiment of the present invention is shown.

[0015] Figure 7 A set of functional abstraction layers provided by a cloud computing environment is shown according to an embodiment of the present invention. Detailed Implementation

[0016] Figure 1 A system 100 for improving runtime container software technology according to an embodiment of the present invention is illustrated. This runtime container software technology is associated with generating (software) applications that execute generic runtime containers and executing specified workloads via these generic runtime containers, which include potential application runtimes within a hierarchical, modifiable format and associated supported software versions. In a typical serverless environment, system providers may need to verify that the provided hardware / software capacity (e.g., memory, CPU, etc.) is sufficient to handle highly elastic workloads being scheduled without notifying the system's customers. Similarly, providers may need to ensure that operating costs fall within predetermined thresholds. Over-provisioning of systems drives operating costs beyond the upper limit threshold. Likewise, systems configured to operate with minimal hardware may be unable to handle the required workloads at all, forcing cloud providers to balance operating costs and system performance. System 100 is configured to address several modifications for improving serverless systems in balancing operating costs and system performance. Serverless system improvements may include:

[0017] 1. Generate and allocate specific, pre-built runtime containers, including generic runtime containers.

[0018] 2. Generate a centralized maintenance mechanism that includes refiller components.

[0019] 3. Generate a scheduling mechanism to achieve improved capacity utilization.

[0020] Figure 1 System 100 includes edge servers 107 interconnected via network 117, controllers 106a and 106b, a data repository 112, a messaging component 109, a refiller component 102, and worker nodes 104a…104n. The refiller component 102 includes hardware / software components configured to maintain a workload provisioning plan that defines the number of runtime containers required for each workload. The hardware / software components of the refiller component 102 include a workload monitor component 102a, a workload protector component 102b, and a capacity manager component 102c. The refiller component 102 is configured to create provisioning plans and roll them out to all worker nodes 104a…104n. The provisioning plan consists of the type and number of containers that each specified worker node should maintain in its preheating pool. The consumption of the messaging engine's topics enables the refiller component 102 to determine which requests are currently waiting to be executed via the worker nodes. A second data source is pulled from data store 112. Data repository 112 contains information about previously executed workloads. The Capacity Manager component 102c uses all data sources to create supply plans for each individual worker node.

[0021] Worker nodes 104a…104n are configured to execute the submitted workloads.

[0022] Worker nodes 104a…104n (e.g., as illustrated in expanded view 122 of worker node 104a) each include a container bridge 119 (i.e., for producing and managing runtime containers 108), an actual running container 114, a caller component 121, a refiller agent 123 (i.e., an agent that listens to refiller component 102 and executes refill requests issued by a central refiller), and a general runtime container 127. Refiller component 102, controllers 106a and 106b, and worker nodes 104a…104n each include a dedicated circuit system (which may include dedicated software), sensors, and machine learning software code / hardware architecture (i.e., including machine learning software code). Sensors may include any type of internal or external sensor, particularly including ultrasonic 3D sensor modules, temperature sensors, ultrasonic sensors, optical sensors, video retrieval devices, audio retrieval devices, humidity sensors, voltage sensors, pressure sensors, etc. Refiller component 102, controllers 106a and 106b, and worker nodes 104a…104n each may include embedded devices. Embedded devices are defined herein as dedicated devices or computers, including a combination of computer hardware and software (fixed or programmable) specifically designed to perform dedicated functions. Programmable embedded computers or devices may include dedicated programming interfaces. In one embodiment, refiller component 102, controllers 106a and 106b, and worker nodes 104a, ... 104, 104n each may include dedicated hardware devices that include functions for (independently or in combination) performing related functions. Figure 1-1 0 describes the dedicated (non-general-purpose) hardware and circuitry (i.e., dedicated discrete non-general-purpose analog, digital, and logic-based circuitry) of the process. The dedicated discrete non-general-purpose analog, digital, and logic-based circuitry may include proprietary, specially designed components (e.g., application-specific integrated circuits, such as those designed solely to implement automated processes for improving runtime container software technologies associated with generating general-purpose runtime containers and executing specified workloads via these general-purpose runtime containers, which include potential application runtimes within a layered, modifiable format and associated supported software versions). Network 117 may include any type of network, particularly including 5G telecommunications networks, local area networks (LANs), wide area networks (WANs), the Internet, wireless networks, etc. Alternatively, network 117 may include application programming interfaces (APIs).

[0023] System 100 provides improved container instantiation by using a generic runtime container instead of using specific pre-built runtime containers associated with standard programming languages. System 100 is configured to build a generic runtime container that includes all necessary runtimes and supported software versions. Before injecting a workload into the container and executing it, System 100 removes all runtime layers that are unnecessary for the workload. Each runtime / version is represented as a layer within the container and is easily modified and / or removed.

[0024] System 100 reduces the need for multiple different containers to be pre-provisioned. Previous container systems required creating different containers for each supported runtime. System 100 improves system flexibility by requiring only a single generic container, as the generic runtime container can be enabled for any workload. Therefore, the caller component does not need to create individual containers based on current needs, but can use an existing generic container for any workload, simplifying container management so that only a single container needs maintenance (e.g., software updates, verification, etc.).

[0025] System 100 implements an improved container provisioning management process via a re-provisioning component 102 (i.e., a central re-provisioning component). The re-provisioning component 102 is configured to maintain a provisioning plan that defines the number of runtime containers required within each caller component 121. System 100 then transmits the provisioning plan to each worker node 104a…104n via an agent model. Similarly, re-provisioning agents 123 apply the provisioning plan by creating or deleting containers as needed. For example, when allocating workloads, re-provisioning agents 123 may apply a provisioning plan related to the size of runtime containers reduced by removing unnecessary runtimes, resulting in additional available space on the relevant nodes. The provisioning plan may indicate that when the total available space exceeds the size of a generic container, re-provisioning agents 123 may assign a new generic container.

[0026] System 100 enables the caller component (e.g., caller component 121) on each worker node (e.g., worker node 104a) to determine whether a new runtime container must be created or whether an existing runtime container can be used after consuming an upcoming workload item. Refiller component 102 can facilitate the transfer of responsibility from caller component 121 to refiller component 102. Furthermore, the creation and preparation processes associated with the container can be initiated before the actual workload reaches the caller component.

[0027] The refiller component 102 can be further improved by adding machine learning capabilities to analyze historical data on supply plan creation, thereby improving the prediction of the expected workload for associated actions. This improvement can lead to better performance in the following areas:

[0028] 1. Faster worker node startup.

[0029] 2. Faster workload execution due to optimized cold start performance.

[0030] 3. Preparation for re-emerging workload patterns.

[0031] 4. Improved resource management due to optimized hardware / software cleanup processes.

[0032] System 100 also provides improved capacity handling for the scheduler component. For example, two instances of the controller component can be enabled to distribute incoming workloads to worker nodes based on required and available hardware / software / memory capacity. Thus, both controllers are enabled to manage half the capacity of each individual worker node, allowing the controllers to negotiate the capacity ratio associated with their management. If a controller is unable to allocate new workloads due to insufficient capacity, the additional controller is configured to provide the missing capacity for the associated node. If the additional controller is able to provide the requested capacity, the request is approved, and the associated capacity of the worker node is reduced by the requested amount. Subsequently, the (original) controller increases its total capacity allocated to the worker node by the requested amount and assigns the workload. Additionally, all requests and responses include information associated with the capacity managed by each controller. If communication synchronization issues exist, continuous exchange of associated status information is used to resolve capacity management problems.

[0033] The following example illustrates the process associated with the execution of a workload on a worker node from the placement of a request until the workload is executed.

[0034] This process is enabled to execute within a cloud architecture and provides pre-packaged runtime components. The process includes: enabling the caller component to receive workload placement requests, locating containers with associated software requirements, assigning workloads to the associated containers, and initializing and running the workloads. This method is further detailed below:

[0035] The invoking component receives a request for workload placement. This request includes a specification associated with the required runtime or set of runtimes. The specification includes a declarative description of the minimum set of software components necessary to run the workload. The specification may additionally include a description of the required software and its corresponding minimum version number. The software may include a runtime (e.g., a JRE or Python runtime) and a set of required libraries. This specification is associated with the workload's requirements. The invoking component then generates requests for inactive containers matching the runtime specification from the container manifest on the node. If a container that meets the specification exists, it is marked as active in the manifest. The invoking component then places the workload within the container for execution. If the container manifest does not contain a container associated with the workload specification, the container registry is analyzed to determine if it contains a template associated with the workload specification, which can be instantiated to provide the required container. The container registry presents available container templates and associated installed software. If the container registry contains a template that matches the workload's requirements, the invoking component creates and starts an instance of the template, places the workload within the container, and executes the workload within the container. As a prerequisite for template instantiation, the associated nodes must have sufficient resources to accommodate the instantiated container. The adequate management of container resources is described below:

[0036] If the container registry does not contain a template that meets the requirements of the workload to be placed, a template management process is triggered to create an appropriate container template based on the workload's needs. The created container template is then assigned to the registry. The template management process can include manual or automatic procedures as follows:

[0037] 1. Verify the correctness of the template requirements. For example, the system may be unable to locate the template because the specification associated with the placement request contains a non-existent or incompatible version number.

[0038] 2. Associate process dependencies with the required software and versions. For example, verify the terms and conditions of the required software to ensure that the template to be built matches the cloud provider's policies and to enter into legal terms between the cloud provider and the workload requester.

[0039] 3. Verify the requirements regarding cloud environment resource limitations (hardware or software) to ensure that the instantiated template can run workloads as expected.

[0040] 4. Download and install the required software packages for template creation.

[0041] 5. Place the template in the container registry.

[0042] System 100 is configured to establish a cloud environment for workload processing, enabling the cloud provider to initiate a template registry comprising a set of standard templates generated based on the popularity of programming languages ​​or as a result of customer surveys. Alternatively, the template registry can be initially configured with an empty structure, and all template creation is associated with the template management process described above. Initiating this process with respect to a set of standard templates can accelerate response times for early deployments.

[0043] The above description associated with container resource management leads to optimizations in resource consumption, such that the time period between receiving a placement request and the workload running on the container is referred to as the "time-until-time" period. Fast response times may require pre-provisioning containers on nodes, as the container initialization process can significantly impact the time-until-time period. Key success factors for achieving a minimal time-until-time period across workloads may include maintaining an appropriate set of templates within the container registry and pre-provisioning a mix of templates that may match the requirements of incoming workloads. Incorrect decisions can further negatively impact the time-until-time period, as additional remedial steps may be required, such as removing pre-provisioned containers. For example, if nodes are pre-provisioned with containers based on runtimes A, B, and C, and a client initiates a request for a specified number of workloads requiring runtimes D and E, the containers may need to be swapped on the nodes, and new containers may need to be instantiated, which could be more costly than placing the workload on containers that are already pre-provisioned.

[0044] A generic container is defined here as a container template (structure) that includes all pre-installed software packages required to meet the specifications of all incoming workloads. A generic container includes partitioning capabilities provided by the container (software / hardware) engine to make it easy to remove unnecessary software components to free up resources for future workload placement. Software components may be determined to be incompatible, meaning they may not be able to coexist within a single container. Therefore, multiple generic containers may be required.

[0045] Generic container processing can include analyzing workload requirements for the software present within the generic container. If a mismatch is detected, the caller component can add the required software to the generic container. Similarly, if the generic container matches the workload requirements, an inactive generic container checklist is used to remove unnecessary software components from the generic container, and the workload is placed inside the container and executed. The container is then marked as active in the checklist.

[0046] General-purpose containers offer the following advantages:

[0047] Simplification of decisions associated with locating and matching containers. For example, workload requirements may only need to be matched once regarding the software existing within a generic container. The matching process only needs to be performed on nodes that include at least one inactive container, since all pre-provisioned inactive containers include instances of the generic container image. If the generic container does not meet the workload requirements, it can be extended with the required software during the template management process. The added software will be available for future workloads.

[0048] General-purpose containers may require more storage space than containers configured to address specific workload requirements. Therefore, since general-purpose containers are scaled to the minimum software needed for a workload, overhead exists only when the container is inactive, during the time the workload is placed, and during container resizing processes that include removing unnecessary software. The repopulator component can be configured to manage available space by intelligently assigning new general-purpose containers.

[0049] Figure 2 An algorithm according to an embodiment of the present invention is shown, which details the process by which... Figure 1 The system 100 implements a process flow for improving runtime container software technology, which is associated with generating applications that execute generic runtime containers and executing specified workloads via the generic runtime containers, which include potential application runtimes within a layered, modifiable format and associated supporting software versions. Figure 2 Each step in the algorithm can be enabled and executed in any order by (multiple) computer processors that execute computer code. Furthermore, Figure 2 Each step in the algorithm can be derived from... Figure 1 Edge servers 107, controllers 106a and 106b, message sending and receiving component 109, refill component 102, and worker nodes 104a…104n are enabled and executed in combination. In step 200, multiple runtime containers and associated characteristics required by each of the multiple worker nodes for executing a specified workload are defined (by a centralized maintenance device). In step 202, the specified workload is distributed to the multiple worker nodes via a coordinating controller. Similarly, workload capacity can be negotiated between the coordinating controllers, and the distribution process can be performed based on the negotiation results. The distribution process can also be performed based on the hardware and software capabilities of the coordinating controllers.

[0050] In step 204, a designated portion of the specified workload is assigned to each worker node via a coordinating controller. In step 208, an application executing a generic runtime container is generated based on the results of step 204. The generic runtime container includes a potential application runtime within a modifiable format and associated supported software versions; this hierarchical modifiable format comprises multiple layers. Additionally, each worker node may be determined to include a specified number of hardware and software resources for enabling the generic runtime container.

[0051] In step 210, unused layers from multiple layers are removed by performing a repopulation proxy component. The removal process may also include removing application runtimes from potential application runtimes that are not needed for each specified portion of the specified workload. The repopulation proxy component may include statistical processes or machine learning capabilities configured to analyze historical data associated with previous instances of the specified workload and to generate future instances of a generic runtime container.

[0052] In step 212, the specified workload is executed via a generic runtime container. In step 214, a set of available generic runtime containers on the associated worker node is repopulated. Additionally, it can be determined whether each generic runtime container (of the set of available generic runtime containers) includes either an initialized container or a disabled container configured for initialization.

[0053] Figure 3 A detailed view of a worker node 300 configured to execute workloads according to an embodiment of the present invention is shown. The worker node 300 includes a caller component 302, a refiller agent component 304, and a container bridge 308 responsible for handling all runtime container-related operations (for running container 310 and warming up container 312), such as creation, pause, and removal operations. The caller component 302 is configured to manage incoming workloads and the entire lifecycle of running container 310 and warming up container 312. The worker node 300 is associated with three types of workload execution: cold execution, warming up execution, and hot execution.

[0054] Regarding performance and latency, hot execution is associated with operations that are faster than pre-warming and cold execution. Hot execution allows the caller component 302 to reuse running user containers to execute incoming workload requests. Similarly, cold execution requires new containers for initialization. To minimize the amount of cold execution, the caller component 302 includes a pool of pre-warmed containers that have been initialized but not yet customized for a specific customer or workload. The distribution and quantity of pre-warmed containers are stored as a configuration for evaluation during caller component startup. The configuration settings are static and can only be changed with deployment.

[0055] The refill agent component 304 manages the pool of hot and preheated containers. The refill agent component 304 enables the pre-supply of specific types and numbers of containers based on the current needs within the system. Therefore, the chances of workloads being executed as cold executions are significantly reduced.

[0056] The refiller proxy component 304 enables processes to reduce processing wait times or avoid cold execution. This process is initiated when the caller component 302 consumes the workload and determines whether the workload requires starting a new container (cold execution) or whether a (pre)warmed container is available to execute the workload. This determination can be performed before the caller component 302 receives the workload, thereby avoiding cold execution because the worker node has already started the required container. Similarly, if the refiller proxy component 304 cannot start the container immediately, the wait time for the workload to be ready for execution can be reduced.

[0057] Refiller proxy component 304 enables Figure 1 System 100 is able to prepare for re-emerging workloads by evaluating historical data. The refill agent component 304 is able to detect the need for a specific container type, and based on this information, a provisioning plan is generated to launch the required containers before the associated worker nodes execute the actual workload. Therefore, ( Figure 1 System 100 can respond in a timely manner without handling cold executions. The refill agent component 304 can additionally detect sequences of previously executed detected workloads. In this case, the refill agent component 304 can additionally supply the required containers before the actual workload arrives at the worker node. The use of historical data allows the refill agent component 304 to enable the system to predict the type and amount of incoming workloads. The refill agent component 304 provides pluggable predictive capabilities that can be connected to simple statistical modules as well as advanced model-based predictive systems, thereby improving system performance by reducing the ratio of cold to hot executions.

[0058] Figures 4A-4D An improved capacity handling process for implementing a dispatcher component according to an embodiment of the invention is illustrated. For example, when the controller cannot place a workload on a designated worker node due to a lack of capacity associated with the request, additional capacity from an additional controller can be assigned instead of attempting to locate another worker node for workload placement. The amount of capacity required and the total amount of capacity currently managed can be sent along with the request. The additional controller can verify whether it can free up the requested amount of capacity on a particular worker node and accept or reject the request based on the available capacity. If the additional worker node accepts the request, it reduces its available capacity by the requested amount. The associated response indicates the current amount of capacity associated with the controller's management.

[0059] Figures 4A to 4D An implementation example is shown, associated with two controllers managing 8GB of memory on worker nodes that initially have a capacity of 16GB. Both controllers occupy worker nodes, each with an existing workload of 6GB, thus leaving 2GB of spare capacity for each controller.

[0060] Figure 4A A first example associated with a process that triggers a successful request to transfer workload capacity is shown. This process is initiated when a workload 404a requests 3GB of capacity at controller 400a. Controller 400a cannot place the workload on the worker node because it only has 2GB of remaining capacity. In response, controller 400a initiates a request for an additional 1GB of capacity from controller 402a. Controller 402a is able to free up 1GB of memory for the worker node and therefore accepts the request. The above process reduces the total managed memory to 7GB and sends an acceptance response. In response to receiving the response, controller 400a increases the amount of capacity it manages to 9GB and places workload 404a on the worker node.

[0061] Figure 4B A second example associated with a process that causes an unsuccessful request for capacity conversion is shown. This process is initiated when a workload 404b requiring 5GB of capacity is requested at controller 400b. Controller 400b cannot place workload 404b on a worker node because it only has 2GB of remaining capacity. In response, controller 400b initiates a request for an additional 3GB of capacity from controller 402b. Controller 402b cannot free up 3GB of memory for the worker node and therefore rejects the request. Similarly, controller 402b retains its total managed storage at 8GB and sends a rejection response. Upon receiving this response, controller 400b is required to place workload 404b on a different worker node or defer the requested workload 404b until the capacity becomes available again.

[0062] Figure 4CA third example is shown associated with the process leading to a successful request to transfer capacity using the self-healing result. Due to previous erroneous communication, controller 402c assumes it currently only manages 6GB of memory on worker nodes, and a workload 404c requiring 3GB of capacity is requested at controller 400c. Controller 400c cannot place workload 404c on the worker node because it only has 2GB of remaining capacity. In response, controller 400c requests an additional 1GB of capacity from controller 402c. Again, controller 400c sends this request along with additional information specifying its current total managed memory of 8GB. In response, controller 402c adds the totals of the two managed capacities and determines there is an additional 2GB difference, which it adds to the amount of capacity it manages. Based on the new total capacity, controller 402c is able to free up 1GB of memory on the worker node and therefore accepts the request. Controller 402c adjusts its total managed memory to 7GB and sends an acceptance response. In response to receiving the response, controller 400c increases the amount of capacity it manages to 9GB and places workload 404c on worker nodes.

[0063] Figure 4D A fourth example is shown associated with a process that causes an unsuccessful request to transfer capacity as a result of self-healing. Due to previous erroneous communication, controller 400d determines that it currently only manages 9GB of memory on worker nodes, and a workload 404d requiring 5GB of capacity at controller 400d is requested. Controller 400d cannot place workload 404d on the worker node because it only has 3GB of remaining capacity. In response, controller 400d requests an additional 2GB of capacity from controller 402d. Again, controller 400d sends the request along with additional information specifying the total amount of memory it currently manages (9GB). In response, controller 402d adds the total amounts of the two managed capacities and determines that it is short 1GB of memory, and therefore removes 1GB from the amount of capacity it manages. Controller 402d cannot free up 2GB of memory on the worker node and therefore rejects the request. Again, controller 402d adjusts its total managed memory to 7GB and transmits a rejection response. Upon receiving this response, controller 400d needs to place workload 404d on a different worker node or defer the requested workload 404d until capacity is available again.

[0064] Figure 5 The invention is illustrated by an embodiment of the invention. Figure 1 The system uses or includes computer systems 90 (e.g., Figure 1Edge servers 107, controllers 106a and 106b, message transceiver component 109, refiller component 102, and worker nodes 104a…104n) are used to improve runtime container software technology associated with generating applications that execute generic runtime containers, which include potential application runtimes and associated supported software versions within a layered, modifiable format, and execute specified workloads via the generic runtime container.

[0065] Various aspects of the present invention may take the form of a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which may be collectively referred to herein as a “circuit,” a “module,” or a “system.”

[0066] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to perform aspects of the invention.

[0067] A computer-readable storage medium can be a tangible device capable of retaining and storing instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital universal disc (DVD), memory sticks, floppy disks, mechanical encoding devices (such as punched cards or raised structures in recesses on which instructions are recorded), and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0068] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to a computer-readable storage medium within the corresponding computing / processing device.

[0069] Computer-readable program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data of an integrated circuit system, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk and C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry devices, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs) may execute computer-readable program instructions by utilizing state information from the computer-readable program instructions to personalize the electronic circuitry devices in order to perform aspects of this invention.

[0070] This document describes aspects of the invention with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0071] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / actions specified in one or more boxes of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable storage medium having the instructions stored therein includes an article of writing comprising instructions for implementing aspects of the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0072] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions, which execute on the computer, other programmable apparatus, or other device, perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0073] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of instructions comprising one or more executable instructions for implementing a specified logical function(s). In some alternative implementations, the functions indicated in the blocks may occur in a different order than indicated in the drawings. For example, two blocks shown consecutively may actually be implemented as a single step, executed simultaneously, substantially simultaneously, with partial or complete time overlap, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It will also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified function or action or executes a combination of dedicated hardware and computer instructions.

[0074] Figure 5The computer system 90 shown includes a processor 91, an input device 92 coupled to the processor 91, an output device 93 coupled to the processor 91, and memory devices 94 and 95, each coupled to the processor 91. The input device 92 may in particular be a keyboard, mouse, camera, touchscreen, etc. The output device 93 may in particular be a printer, plotter, computer screen, magnetic tape, removable hard disk, floppy disk, etc. The memory devices 94 and 95 may in particular be hard disks, floppy disks, magnetic tapes, optical storage such as CDs or DVDs, dynamic random access memory (DRAM), read-only memory (ROM), etc. The memory device 95 includes computer code 97, which includes algorithms for improving runtime container software technology (e.g., ...). Figure 2 The runtime container software technology is associated with generating a generic runtime container that includes a potential application runtime and an associated supporting software version within a hierarchical, modifiable format, and executing a specified workload via the generic runtime container. Processor 91 executes computer code 97, memory device 94 includes input data 96, which includes the inputs required by computer code 97, output device 93 displays the output from computer code 97, and one or both of memory devices 94 and 95 (or one or more additional memory devices, such as read-only memory (ROM) devices or firmware 85) may include algorithms (e.g., Figure 2 The algorithm), and can be used as a computer-usable medium (or computer-readable medium or program storage device) containing computer-readable program code and / or storing other data therein, wherein the computer-readable program code includes computer code 97. Typically, the computer program product (or, alternatively, article of manufacture) of computer system 90 may include computer-usable medium (or program storage device).

[0075] In some embodiments, instead of being stored and accessed from a hard disk drive, optical disk, or other writable, rewritable, or removable hardware storage device 95, the stored computer program code 84 (e.g., including algorithms) may be stored on a static, non-removable, read-only storage medium (such as a ROM device or firmware 85), or may be directly accessed by the processor 91 from such a static, non-removable, read-only medium. Similarly, in some embodiments, the stored computer program code 97 may be stored as a ROM device or firmware 85, or may be directly accessed by the processor 91 from such a ROM device or firmware 85, rather than from a more dynamic or removable hardware data storage device 95 such as a hard disk drive or optical disk.

[0076] Furthermore, any component of this invention can be created, integrated, hosted, maintained, deployed, managed, and serviced by a service provider that provides improved runtime container software technology for generating generic runtime containers and executing specified workloads via these generic runtime containers, the runtime containers including potential application runtimes within a layered, modifiable format and associated supported software versions. Therefore, this invention discloses a process for deploying, creating, integrating, hosting, maintaining, and / or integrating computing infrastructure, comprising integrating computer-readable code into a computer system 90, wherein the code combined with the computer system 90 is capable of executing a method for implementing a process for improving runtime container software technology associated with generating generic runtime containers and executing specified workloads via these generic runtime containers, the generic runtime containers including potential application runtimes within a layered, modifiable format and associated supported software versions. In another embodiment, this invention provides a business method for performing the process steps of this invention on a subscription, advertising, and / or charging basis. That is, service providers such as solution integrators can provide processes for implementing improved runtime container software technology, which is associated with generating generic runtime containers and executing specified workloads via these generic runtime containers, including potential application runtimes within a layered, modifiable format and associated supported software versions. In this case, the service provider can create, maintain, support, etc., the computer infrastructure for performing the processing steps of the present invention for one or more customers. In return, the service provider may receive payments from (multiple) customers according to subscription and / or fee agreements, and / or may receive payments from the sale of advertising content to one or more third parties.

[0077] Although Figure 5 The computer system 90 is shown as a hardware and software configuration, but any hardware and software configuration known to those skilled in the art can be used in the above combination. Figure 5 For the purposes described in the computer system 90, for example, memory devices 94 and 95 may be part of a single memory device rather than separate memory devices.

[0078] Cloud computing environment

[0079] It should be understood that although this disclosure includes a detailed description of cloud computing, the implementation of the teachings set forth herein is not limited to a cloud computing environment. Rather, embodiments of the invention can be implemented in conjunction with any other type of computing environment now known or developed hereafter.

[0080] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, storage, applications, virtual machines, and services) that can be rapidly provisioned and released with minimal management effort or interaction with a service provider. This cloud model may include at least five features, at least three service models, and at least four deployment models.

[0081] The features are as follows:

[0082] On-demand self-service: Cloud consumers can automatically and unilaterally provision computing power, such as server time and network storage, on demand, without human interaction with the service provider.

[0083] Extensive network access: Capabilities are available through the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0084] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically assigned and reassigned as needed. Location independence is significant because consumers typically do not have control or knowledge of the exact location of the resources provided, but can specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0085] Rapid flexibility: In some situations, supply capacity can be rapidly and flexibly supplied automatically to shrink quickly and expand rapidly. For consumers, the available supply capacity often appears unlimited and can be purchased in any quantity at any time.

[0086] Measuring services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at an abstraction layer appropriate to service types (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both service providers and consumers.

[0087] The service model is as follows:

[0088] Software as a Service (SaaS): This provides consumers with the ability to use the provider's applications running on cloud infrastructure. These applications are accessible from various client devices through thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including the network, servers, operating system, storage devices, or even the individual application capabilities, with possible exceptions such as limited user-specific application configuration settings.

[0089] Platform as a Service (PaaS): This provides consumers with the ability to deploy applications created or acquired by the consumer onto cloud infrastructure using programming languages ​​and tools supported by the provider. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage devices, but they have control over the deployed applications and, if any, the configuration of the application hosting environment.

[0090] Infrastructure as a Service (IaaS): This provides consumers with the capability to deploy and run any software, including operating systems and applications, providing them with processing, storage, networking, and other basic computing resources. Consumers do not manage or control the underlying cloud infrastructure, but have control over the operating system, storage devices, deployed applications, and, possibly, limited control over the selection of networking components (e.g., host firewalls).

[0091] The deployment model is as follows:

[0092] Private cloud: Cloud infrastructure operated solely by an organization. It can be managed by the organization or a third party and can exist on-site or off-site.

[0093] Community cloud: A cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., tasks, security requirements, policies, and compliance considerations). It may be managed by an organization or a third party and may reside on-site or off-site.

[0094] Public cloud: Cloud infrastructure that is made available to the general public or large groups of industries and is owned by an organization that sells cloud services.

[0095] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community, or public) that maintain distinct entities but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursts for load balancing between clouds).

[0096] Cloud computing environments are service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability. At the heart of cloud computing is the infrastructure of a network of interconnected nodes.

[0097] Now for reference Figure 6The diagram illustrates an illustrative cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 to which local computing devices used by cloud consumers can communicate. These local computing devices include, for example, personal digital assistants (PDAs) or cellular phones 54A, desktop computers 54B, laptop computers 54C, and / or automotive computer systems 54N. The nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as private clouds, community clouds, public clouds, or hybrid clouds, or combinations thereof, as described above. This allows the cloud computing environment 50 to provide infrastructure, platform, and / or software as a service, without requiring cloud consumers to maintain resources on their local computing devices. It should be understood that... Figure 6 The types of computing devices 54A, 54B, 54C, and 54N shown are for illustrative purposes only, and computing node 10 and cloud computing environment 50 can communicate with any type of computerized device via any type of network and / or network-addressable connection (e.g., using a web browser).

[0098] Now for reference Figure 7 This demonstrates the use of cloud computing environments 50 (see [link]). Figure 6 This provides a set of functional abstractions. It should be understood beforehand that... Figure 7 The components, layers, and functions shown are for illustrative purposes only, and embodiments of the invention are not limited thereto. As depicted, the following layers and corresponding functions are provided:

[0099] The hardware and software layer 60 includes hardware and software components. Examples of hardware components include: a mainframe 61; a RISC (Reduced Instruction Set Computer) based server 62; a server 63; a blade server 64; a storage device 65; and a network and network components 66. In some embodiments, software components include network application server software 67 and database software 68.

[0100] The virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage device 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75.

[0101] In one example, management layer 80 can provide the following functionalities: Resource Provisioning 81 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Metering and Pricing 82 provides cost tracking when utilizing resources in the cloud computing environment, as well as billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User Portal 83 provides access to the cloud computing environment for consumers and system administrators. Service Level Management 87 provides cloud resource allocation and management to ensure that required service levels are met. Service Level Agreement (SLA) Planning and Fulfillment 888 provides pre-scheduling and procurement of cloud resources, where future needs are anticipated according to the SLA.

[0102] Workload layer 101 provides examples of functionalities that can be leveraged in a cloud computing environment. Examples of workloads and functionalities that can be provided from this layer include: mapping and navigation 99; software development and lifecycle management 103; virtual classroom education delivery 133; data analytics and processing 134; transaction processing 106; and improved runtime container software technology associated with generating a generic runtime container and executing specified workloads via a generic runtime container 110, which includes potential application runtimes and associated supported software versions within a layered, modifiable format.

[0103] Although embodiments of the invention have been described herein for illustrative purposes, many modifications and alterations will become apparent to those skilled in the art. Therefore, the appended claims are intended to cover all such modifications and alterations that fall within the true spirit and scope of the invention.

Claims

1. A method for allocating containers in a serverless runtime environment, comprising: The processor of the centralized maintenance device defines multiple runtime containers and associated features required for each worker node in a series of worker nodes to execute a specified workload; The processor distributes the specified workload to the multiple worker nodes via multiple coordinating controllers; The processor assigns a specified portion of the specified workload to each worker node via the plurality of coordinated controllers; The processor generates an application that executes a generic runtime container based on the result of the assignment. The generic runtime container includes multiple potential application runtimes within a layered, modifiable format and associated supported software versions. The layered, modifiable format includes multiple layers. The processor that performs the refilling of the proxy component removes unused layers from the plurality of layers of the general runtime container; The processor, in response to the generation of the general runtime container, executes the specified workload via the general runtime container; as well as In response to the execution, the processor repopulates a set of available general runtime containers on the associated worker nodes of the plurality of worker nodes via the plurality of coordinated controllers.

2. The method of claim 1, wherein the removal further comprises: removing application runtimes from the potential application runtime that are not required to execute each specified portion of the specified workload.

3. The method of claim 1 or 2, wherein the repopulation agent component includes a statistical process or machine learning capability configured to analyze historical data associated with previous instances of the specified workload and to generate future instances of the general runtime container.

4. The method according to claim 1 or 2, further comprising: The processor determines that each worker node includes a specified number of hardware and software resources for enabling the general runtime container.

5. The method according to claim 1 or 2, further comprising: The processor negotiates workload capacity among the plurality of coordinated controllers, wherein the assignment of the specified workload to the plurality of worker nodes is performed based on the result of the negotiation.

6. The method according to claim 1 or 2, further comprising: The processor determines whether each container in the set of available generic runtime containers includes either an initialized container or a disabled container configured for initialization.

7. The method of claim 1 or 2, wherein the dispatch is performed based on the hardware and software capabilities of the plurality of coordinating controllers.

8. The method according to claim 1 or 2, further comprising: Provide at least one support service for creating, integrating, hosting, maintaining, and deploying at least one of computer-readable code in the centralized maintenance equipment, the code being executed by the computer processor to perform: the definition, the dispatch, the assignment, the generation, the removal, the execution, and the repopulation.

9. A computer program product comprising a computer-readable hardware storage device storing computer-readable program code, the computer-readable program code including an algorithm that, when executed by a processor of a centralized maintenance device, implements a serverless runtime container allocation method, the method comprising: The processor defines multiple runtime containers and associated features required for each of the multiple worker nodes to execute a specified workload; The processor distributes the specified workload to the multiple worker nodes via multiple coordinating controllers; The processor assigns a specified portion of the specified workload to each worker node via the plurality of coordinated controllers; The processor generates an application that executes a generic runtime container based on the result of the assignment. The generic runtime container includes multiple potential application runtimes within a layered, modifiable format and associated supported software versions. The layered, modifiable format includes multiple layers. The processor that performs the refilling of the proxy component removes unused layers from the plurality of layers of the general runtime container; The processor, in response to the generation of the general runtime container, executes the specified workload via the general runtime container; as well as In response to the execution, the processor repopulates a set of available general runtime containers on the associated worker nodes of the plurality of worker nodes via the plurality of coordinated controllers.

10. The computer program product of claim 9, wherein the removal further includes removing application runtime from the potential application runtime that is not required to execute each specified portion of the specified workload.

11. The computer program product of claim 9 or 10, wherein the repopulation agent component includes a statistical process or machine learning capability configured to analyze historical data associated with previous instances of the specified workload and to generate future instances of the general runtime container.

12. The computer program product according to claim 9 or 10, wherein the method further comprises: The processor determines that each worker node includes a specified number of hardware and software resources for enabling the general runtime container.

13. The computer program product according to claim 9 or 10, wherein the method further comprises: The processor negotiates workload capacity among the plurality of coordinated controllers, wherein the assignment of the specified workload to the plurality of worker nodes is performed based on the result of the negotiation.

14. The computer program product according to claim 9 or 10, wherein the method further comprises: The processor determines whether each container in the set of available generic runtime containers includes either an initialized container or a disabled container configured for initialization.

15. The computer program product of claim 9 or 10, wherein the dispatch is performed based on the hardware and software capabilities of the plurality of coordinated controllers.

16. A centralized maintenance device including a processor coupled to a computer-readable storage unit, the storage unit including instructions, when executed by the processor, implementing a serverless runtime container allocation method, the method comprising: The processor defines multiple runtime containers and associated features required for each of the multiple worker nodes to execute a specified workload; The processor distributes the specified workload to the multiple worker nodes via multiple coordinating controllers; The processor assigns a specified portion of the specified workload to each worker node via the plurality of coordinated controllers; The processor generates an application that executes a generic runtime container based on the result of the assignment. The generic runtime container includes multiple potential application runtimes within a layered, modifiable format and associated supported software versions. The layered, modifiable format includes multiple layers. The processor that performs the refilling of the proxy component removes unused layers from the plurality of layers of the general runtime container; The processor, in response to the generation of the general runtime container, executes the specified workload via the general runtime container; as well as In response to the execution, the processor repopulates a set of available general runtime containers on the associated worker nodes of the plurality of worker nodes via the plurality of coordinated controllers.

17. The centralized maintenance device of claim 16, wherein the removal further includes removing application runtimes from the potential application runtimes that are not required to perform each specified portion of the specified workload.

18. The centralized maintenance facility of claim 16 or 17, wherein the refill agent component includes a statistical process or machine learning capability configured to analyze historical data associated with previous instances of the specified workload and to generate future instances of the general runtime container.

19. The centralized maintenance equipment according to claim 16 or 17, wherein the method further comprises: The processor determines that each worker node includes a specified number of hardware and software resources for enabling the general runtime container.

20. The centralized maintenance equipment according to claim 16 or 17, wherein the method further comprises: The processor negotiates workload capacity among the plurality of coordinated controllers, wherein the assignment of the specified workload to the plurality of worker nodes is performed based on the result of the negotiation.

Images