Container reuse methods, systems, and programs for pipeline workloads

Abstract

A container reuse is provided. A pipeline workload manager is communicated using an agent daemon of the particular container that the particular container has finished executing a step of a pipeline workload. Pipeline workload information corresponding to the pipeline workload is checked using the pipeline workload manager to determine whether the particular container is reused to execute a particular step in a different pipeline workload. Based on determining according to the pipeline workload information that the particular container is reused to execute the particular step in a different pipeline workload, a particular container is provided to be reused to execute a particular step in a different pipeline workload without performing a container environment preparation sub-step of the particular step.

Description

Technical Field

[0001] This disclosure generally relates to a container orchestration environment, and more particularly to optimizing resources for a pipeline workload composed of multiple steps in a container orchestration environment by reusing containers that have finished executing steps of a pipeline workload on a host node to execute specific steps within different pipeline workloads.

Background Art

[0002] For example, container orchestration environments such as Kubernetes® (a registered trademark of the Linux Foundation, San Francisco, California) provide a platform for automating the deployment, scaling, and operation of containers across a cluster of host nodes. A host node is either a physical or virtual machine on which containers (i.e., application workloads) are deployed. A pod is a group of one or more containers that has shared storage and network resources, as well as specifications on how to execute the containers. The contents of a pod are always co-located, co-scheduled, and operate in a shared context. A host node hosts pods that are components of an application workload.

[0003] A scheduler selects on which host node unscheduled pods will operate based on the resource availability of each host node. A pod is the basic unit managed by the scheduler. The scheduler tracks the resource utilization rate on each host node to ensure that workloads are not scheduled to exceed the available resources.

[0004] One current solution optimizes hardware resource utilization by automatically assessing and allocating virtualized resources (e.g., central processing and graphics processing resources) by building a watcher to monitor workflow queues and placing workflows associated with workloads on the workflow queues to maximize resource utilization. Another current solution presents workload performance on the storage system, predicts the performance load on the storage system resulting from implementing potential changes, and displays the predicted characteristics of one or more workloads running on the storage system. Yet another current solution calculates and continuously refines the pod size for all pods of an application workload based on actual application usage patterns to provide the expected performance for the estimated workload. However, none of these current solutions optimize resources for application workloads consisting of multiple steps used for artificial intelligence processing on a cluster of host nodes. [Overview of the Initiative]

[0005] According to one exemplary embodiment, a computer implementation method for reusing containers is provided. The computer uses an agent daemon for a particular container to communicate to the computer's pipeline workload manager that the particular container has finished executing a step in a pipeline workload. The computer uses the pipeline workload manager to check pipeline workload information corresponding to the pipeline workload to determine whether the particular container can be reused to execute a particular step in a different pipeline workload. Based on the determination, according to the pipeline workload information, that the particular container can be reused to execute that particular step in a different pipeline workload, the computer provides a particular container that can be reused to execute that particular step in a different pipeline workload without having to perform the prepare container environment sub-step for that particular step. According to another exemplary embodiment, a computer system and computer program product for reusing containers are provided. As a result, the exemplary embodiment provides technical benefits and practical applications in the field of container orchestration by reducing costs by reusing containers between pipeline workloads and eliminating the need to perform a container environment preparation substep for each reused container, thereby reducing overall container execution time and improving performance in the container orchestration environment.

[0006] The exemplary embodiment also optionally uses a pipeline workload manager to select a different step from among several steps in a pipeline workload to form a selection step to run on a previously used container based on a set of rules, and uses the agent daemon of the previously used container to run the selection step in the pipeline workload on the previously used container without performing the container environment preparation substep of the selection step in order to reduce the runtime of the selection step and improve computer performance. As a result, the exemplary embodiment may save, for example, more than 30% of container execution time by not performing the container environment preparation substep for reused containers in the pipeline workload. In other words, the exemplary embodiment may be able to terminate the pipeline workload using, for example, less than 70% of normal container execution time, thereby reducing costs and improving the performance of the container orchestration environment. [Brief explanation of the drawing]

[0007] [Figure 1] This is a graphical representation of a network of a data processing system in which an exemplary embodiment is implemented. [Figure 2] This is a diagram of a data processing system in which an exemplary embodiment is implemented. [Figure 3] This figure shows an example of a pipeline workload management system according to an exemplary embodiment. [Figure 4] This figure shows an example of a pipeline workload management process according to an exemplary embodiment. [Figure 5] This figure shows an example of container sharing in a host cluster-level process according to an exemplary embodiment. [Figure 6] This figure shows an example of a step selection table according to an exemplary embodiment. [Figure 7A]This flowchart illustrates a process for executing steps of a pipeline workload on a previously used container to reduce step execution time, according to an exemplary embodiment. [Figure 7B] This flowchart illustrates a process for executing steps of a pipeline workload on a previously used container to reduce step execution time, according to an exemplary embodiment. [Figure 8] This is a flowchart illustrating the process for container sharing according to an exemplary embodiment. [Modes for carrying out the invention]

[0008] The present invention may be a system, method, or computer program product, or a combination thereof, at any possible level of technical detail of integration. The computer program product may include a computer-readable storage medium (or multiple mediums) having computer-readable program instructions for causing a processor to perform an aspect of the present invention.

[0009] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium may, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of those described above. A non-exclusive list of more specific embodiments of computer-readable storage media includes portable computer diskettes, hard disks, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random-access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disks (DVDs), memory sticks, floppy(R) disks, mechanically encoded devices such as punch cards or grooved raised structures on which instructions are recorded, and any suitable combination of those described above. The computer-readable storage media used herein should not be interpreted as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0010] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers the computer-readable program instructions for storage on a computer-readable storage medium within each computing / processing device.

[0011] The computer-readable program instructions for performing the operation of the present invention may be either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk(R) and C++, and procedural programming languages ​​such as the C programming language or a similar programming language. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, for example, an electronic circuit including a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) may execute computer-readable program instructions by individualizing the electronic circuit using state information of computer-readable program instructions in order to carry out aspects of the present invention.

[0012] Aspects of the present invention are described herein with reference to flowcharts or block diagrams, or both, of methods, apparatus (systems), and computer program products according to embodiments of the invention. Each block in the flowchart or block diagram, or both, and any combination of blocks in the flowchart or block diagram, or both, should be understood to be implemented by computer-readable program instructions.

[0013] These computer-readable program instructions may be provided to a computer or a processor of another programmable data processing device for manufacturing machines, so that instructions executed by the processor of a computer or other programmable data processing device generate means for performing functions / operations specified in one or more blocks of a flowchart or block diagram or both. These computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct a computer, a programmable data processing device, or other device, or a combination thereof, to function in a particular manner.

[0014] Computer-readable program instructions may also be loaded onto a computer, other programmable device, or other device to perform a series of operational steps on the computer, other programmable device, or other device to create a computer-executed process, so that the instructions executed on the computer, other programmable device, or other device perform a function / operation specified in one or more blocks of a flowchart or block diagram, or both.

[0015] The flowcharts and block diagrams in the drawings illustrate the architecture, function, and operation of possible embodiments of the systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or part of an instruction containing one or more executable instructions for performing a specified logical function. In some alternative embodiments, the functions described within a block may occur in an order other than that shown in the drawings. For example, two consecutively shown blocks may actually be executed simultaneously, substantially simultaneously, partially or entirely in overlapping time, to be realized as a single step, or the blocks may be executed in reverse order depending on the functionality involved. It should also be noted that each block in a block diagram or flowchart, or both, and any combination of blocks in a block diagram or flowchart, or both, is performed by a dedicated hardware-based system that performs a specified function or operation, or a combination of dedicated hardware and computer instructions.

[0016] Referring here to the drawings, and in particular to Figures 1-3, diagrams of a data processing environment in which exemplary embodiments are carried out are provided. Figures 1-3 are meant to be merely examples and should be understood as not intended to claim or imply any limitations regarding environments in which different embodiments are carried out. Many modifications may be made to the illustrated environment.

[0017] Figure 1 shows a graphical representation of a network of data processing systems in which an exemplary embodiment is implemented. The network data processing system 100 is a network of computers, data processing systems, and other devices in which the exemplary embodiment is implemented. In this embodiment, the network data processing system 100 represents a container orchestration environment such as Kubernetes(R). However, it should be understood that Kubernetes(R) is intended merely as an exemplary architecture and not as an limitation to the exemplary embodiment. In other words, the exemplary embodiment may utilize any kind of container orchestration platform, architecture, infrastructure, or environment that provides automated deployment, scaling, and operation of containers across host nodes.

[0018] The network data processing system 100 includes a network 102, which is a medium used to provide communication links between computers, data processing systems, and other devices connected to each other within the network data processing system 100. The network 102 may include connections such as wired communication links, wireless communication links, and fiber optic cables.

[0019] In the illustrated embodiment, servers 104 and 106 are connected to network 102 along with storage 108. Servers 104 and 106 may be, for example, server computers having high-speed connectivity to network 102. Alternatively, servers 104 and 106 may each represent multiple servers in one or more data centers. Alternatively, servers 104 and 106 may each represent multiple computing nodes in one or more cloud environments.

[0020] In addition, servers 104 and 106 may represent a cluster of physical and virtual host nodes in a container orchestration environment that executes pipeline application workloads for client devices. The pipeline application workload may include any type of workload, such as artificial intelligence processing, natural language processing, image processing, computer vision, scientific computing, forecasting, prediction, recommendation, data processing, transaction processing, etc. Further, the pipeline application workload is composed of multiple steps. Further, servers 104 and 106 may optimize resource utilization for pipeline workloads executed on server 104 or server 106 in a container orchestration environment. For example, server 104 may optimize resource utilization for a pipeline workload by reusing containers that have finished executing steps of the pipeline workload on server 104 to execute specific steps (e.g., the same step or different steps) within different pipeline workloads on server 104 or server 106. The term container is generally used in the Kubernetes(R) paradigm, but the terms used herein are not limited to that environment. Rather, they refer to any type of container in which a pipeline application workload is deployed and that holds the executing applications, libraries, and their dependencies.

[0021] Clients 110, 112, and 114 are also connected to network 102. Clients 110, 112, and 114 are clients of servers 104 and 106. In this embodiment, clients 110, 112, and 114 are shown as desktop or personal computers with a wired communication link to network 102. However, it should be noted that clients 110, 112, and 114 are merely embodiments and could represent other types of data processing systems with a wired or wireless link to network 102, such as network computers, laptop computers, handheld computers, smartphones, smartwatches, smart TVs, smart cars, smart home appliances, gaming devices, and virtual reality devices. Users of clients 110, 112, and 114 can use clients 110, 112, and 114 to submit requests to run pipeline workloads on servers 104 and 106.

[0022] Storage 108 is a network storage device capable of storing any type of data in a structured format or an unstructured format. Additionally, storage 108 may represent a plurality of network storage devices. Further, storage 108 may be, for example, a workload information center that stores identifiers and network addresses for a plurality of servers (such as host nodes), identifiers for a plurality of pipeline workload managers located on the plurality of servers, identifiers and network addresses for a plurality of client devices, pipeline workload information corresponding to a plurality of different pipeline workloads, identifiers for steps including the plurality of pipeline workloads, identifiers for a plurality of containers that execute the steps, and the like. Further, storage 108 may store other types of data such as authentication or credit data that may include, for example, usernames, passwords, etc., associated with client device users and container orchestration environment administrators.

[0023] In addition, it should be noted that the network data processing system 100 may include any number of additional servers, clients, storage devices, and other devices not shown. The program code located in the network data processing system 100 may be stored on a computer-readable storage medium or a set of computer-readable storage media and may be downloaded to a computer or other data processing device for use. For example, the program code may be stored on a computer-readable storage medium on server 104 and may be downloaded to client 110 via network 102 for use on client 110.

[0024] In the illustrated embodiment, the network data processing system 100 may be implemented as several different types of communication networks, such as the Internet, an intranet, a wide area network, a local area network, a telecommunications network, or any combination thereof. Figure 1 is intended as merely one embodiment and not as an architectural limitation to different exemplary embodiments.

[0025] As used herein, when used in reference to an item, “a number of” means one or more items. For example, “a number of different types of communication networks” means one or more different types of communication networks. Similarly, when used in reference to an item, “a set of” means one or more items.

[0026] Furthermore, when the term "at least one of" is used with a list of items, it means that one or more different combinations of the listed items may be used, and only one of each item in the list may be required. In other words, "at least one of" means that any combination and any number of items from the list may be used, but not all of the items in the list may be required. An item may be a specific object, thing, or category.

[0027] For example, rather than being limiting, “at least one of item A, item B, or item C” could include item A, item A and item B, or item B. This embodiment could also include item A, item B, and item C, or item B and item C. Naturally, any combination of these items may exist. In some exemplary examples, “at least one of ~” could be, for example, two item A, one item B, and ten item C, four item B and seven item C, or any other suitable combination.

[0028] Referring now to Figure 2, a diagram of a data processing system is shown according to an exemplary embodiment. The data processing system 200 is an embodiment of a computer, such as the server 104 in Figure 1, which may house computer-readable program code or instructions that implement the container sharing process of the exemplary embodiment. In this embodiment, the data processing system 200 includes a communication fabric 202, which provides communication between the processor unit 204, memory 206, persistent storage 208, communication unit 210, input / output (I / O) unit 212, and display 214.

[0029] The processor unit 204 is responsible for executing instructions for software applications and programs loaded into memory 206. Depending on the particular embodiment, the processor unit 204 may be a set of one or more hardware processor devices or a multi-core processor.

[0030] Memory 206 and persistent storage 208 are embodiments of the storage device 216. As used herein, a computer-readable storage device or computer-readable storage medium is any part of hardware capable of temporarily or persistently storing information such as, for example, data, computer-readable program code in functional form, or other suitable information, or a combination thereof. Furthermore, computer-readable storage devices or computer-readable storage mediums exclude propagating media such as transient signals. Furthermore, computer-readable storage devices or computer-readable storage mediums may represent a set of computer-readable storage devices or a set of computer-readable storage media. In these embodiments, memory 206 may be, for example, random-access memory (RAM), or any other suitable volatile storage device, or a non-volatile storage device such as flash memory. Persistent storage 208 may take various forms depending on the particular embodiment. For example, persistent storage 208 may include one or more devices. For example, persistent storage 208 may be a disk drive, a solid-state drive, a rewritable optical disk, a rewritable magnetic tape, or any combination of the above. The media used by persistent storage 208 may be removable. For example, a removable hard drive may be used for persistent storage 208.

[0031] In this embodiment, persistent storage 208 stores the pipeline workload manager 218. However, while the pipeline workload manager 218 is shown residing in persistent storage 208, it should be noted that in alternative exemplary embodiments, the pipeline workload manager 218 may be a separate component of the data processing system 200. For example, the pipeline workload manager 218 may be a hardware component coupled to the communication fabric 202, or a combination of hardware and software components.

[0032] The pipeline workload manager 218 controls the process of optimizing resources for pipeline workloads consisting of multiple steps in a container orchestration environment by reusing containers that have finished executing steps of a pipeline workload on the data processing system 200 to execute the same or different steps in different pipeline workloads on different host nodes in the data processing system 200 or the container orchestration environment, based on pipeline workload information registered in an external workload information center. The external workload information center may be, for example, storage 108 in Figure 1. As a result, the data processing system 200 operates as a dedicated computer system, enabling the pipeline workload manager 218 within the data processing system 200 to share containers to execute the same or different steps in a pipeline workload, thereby reducing costs and improving performance in the container orchestration environment. In particular, the pipeline workload manager 218 transforms the data processing system 200 into a dedicated computer system compared to currently available general-purpose computer systems that do not have a pipeline workload manager 218.

[0033] In this embodiment, the communication unit 210 provides communication with other computers, data processing systems, and devices via a network such as network 102 in Figure 1. The communication unit 210 may provide communication through the use of both physical communication links and wireless communication links. A physical communication link may be established for the data processing system 200 using, for example, wires, cables, universal serial buses, or any other physical technology. A wireless communication link may be established using, for example, shortwave, high frequency, ultra-high frequency, microwave, Wireless Fidelity (Wi-Fi), Bluetooth® technology, GSM (global system for mobile communications), code division multiple access (CDMA), second generation (2G), third generation (3G), fourth generation (4G), 4G LTE (4G Long Term Evolution), LTE Advanced, fifth generation (5G), or any other wireless communication technology or standard for establishing a wireless communication link for the data processing system 200.

[0034] The input / output unit 212 enables data input and output using other devices connected to the data processing system 200. For example, the input / output unit 212 may provide a connection for user input through a keypad, keyboard, mouse, microphone, or any other suitable input device, or a combination thereof. The display 214 provides a mechanism for displaying information to the user and may include, for example, touchscreen capabilities to allow the user to make on-screen selections through a user interface or input data.

[0035] Instructions for an operating system, application, or program, or a combination thereof, may reside in a storage device 216, which communicates with the processor unit 204 through a communication fabric 202. In this exemplary embodiment, the instructions are in function form on persistent storage 208. These instructions may be loaded into memory 206 for execution by the processor unit 204. Processes in different embodiments may be executed by the processor unit 204 using computer implement instructions, which may reside in memory such as memory 206. These program instructions are called program code, computer-readable program code, or computer-readable program code, which are read and executed by the processor in the processor unit 204. In different embodiments, program instructions may be embodied on different physical computer-readable storage devices such as memory 206 or persistent storage 208.

[0036] The program code 220 may reside in a functional form on a selectively removable computer-readable medium 222 and be loaded onto or transported to a data processing system 200 for execution by a processor unit 204. The program code 220 and the computer-readable medium 222 form a computer program product 224. In one embodiment, the computer-readable medium 222 may be a computer-readable storage medium 226 or a computer-readable signal medium 228.

[0037] In these exemplary embodiments, the computer-readable storage medium 226 is not a medium for propagating or transmitting the program code 220, but a physical or tangible storage device used to store the program code 220. The computer-readable storage medium 226 may include, for example, an optical disk or a magnetic disk, which is inserted into or placed in a drive or other device that is part of the persistent storage 208 for transfer to a storage device such as a hard drive that is part of the persistent storage 208. The computer-readable storage medium 226 may also take the form of persistent storage such as a hard drive, thumb drive, or flash memory connected to the data processing system 200.

[0038] Alternatively, the program code 220 may be transmitted to the data processing system 200 using a computer-readable signal medium 228. The computer-readable signal medium 228 may be, for example, a propagated data signal containing the program code 220. For example, the computer-readable signal medium 228 may be an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals may be transmitted over a communication link such as a wireless communication link, an optical fiber cable, a coaxial cable, a wire, or any other suitable type of communication link.

[0039] Furthermore, as used herein, “computer-readable media 222” may be singular or plural. For example, program code 220 may reside in computer-readable media 222 in the form of a single storage device or system. In another embodiment, program code 220 may reside in computer-readable media 222 distributed across multiple data processing systems. In other words, some instructions within program code 220 may reside in one data processing system, while other instructions within program code 220 may reside in one or more other data processing systems. For example, part of program code 220 may reside in computer-readable media 222 in a server computer, while another part of program code 220 may reside in computer-readable media 222 located in a set of client computers.

[0040] The different components illustrated for the data processing system 200 do not imply an architecture limitation to how different embodiments are carried out. In some exemplary embodiments, one or more components may be incorporated into another component or form part of another component. For example, memory 206 or part thereof may be incorporated into processor unit 204 in some exemplary embodiments. Different exemplary embodiments may be carried out in a data processing system including components in addition to, or instead of, those shown for the data processing system 200. Other components shown in Figure 2 may vary from those in the illustrated exemplary embodiments. Different embodiments may be carried out using any hardware device or system capable of executing program code 220.

[0041] In another embodiment, the bus system may be used to implement the communication fabric 202 and may consist of one or more buses, such as a system bus or input / output buses. Naturally, the bus system may be implemented using any suitable type of architecture that provides data transfer between different components or devices attached to the bus system.

[0042] Pipeline workload management is widely used for data processing, such as artificial intelligence processing. The entire application workload is divided into multiple steps, which are executed sequentially to obtain results. Each step represents a task performed for the application workload, and the execution of this sequence of tasks encompasses the entire application workload.

[0043] Each step in the application workload executes task commands within a container on a host node in the container orchestration environment cluster. In addition, each step includes two substeps: the "Environment Preparation" substep and the "Command Execution" substep. An agent daemon residing within each container running on the host node controls the logic.

[0044] The agent daemon uses an environment preparation substep to prepare the environment for containers that will run task commands. The environment preparation substep includes actions such as downloading container images, hardware checks, and quality-of-service checks, and typically takes 3–5 seconds to complete. After the environment preparation substep for the containers is complete, the agent daemon uses a command execution substep to execute task commands for the application workload within the containers. The command execution substep includes actions such as starting the containers, executing task commands, and outputting results, and typically takes 1–20 seconds to complete, with an average duration of 10 seconds. Therefore, the duration of the container environment preparation substep can account for more than 30% of the total execution time of the entire step. As a result, the execution cost of the container environment preparation substep for millions of steps in a pipeline workload is substantial in terms of overall execution time and system performance. Consequently, it is necessary to reduce the time required to execute steps within the pipeline workload.

[0045] An exemplary embodiment optimizes resources for pipeline application workloads on a cluster of host nodes in a container orchestration environment by reusing containers among pipeline workloads on host nodes within the cluster. Reusing containers among pipeline workloads reduces costs by eliminating the need to perform container environment preparation substeps for each reused container. This reduces overall container execution time, thereby improving host node performance in the cluster.

[0046] An exemplary embodiment provides a workload information center in a container orchestration environment. The workload information center stores container information and pipeline workload information corresponding to different pipeline workloads running in the container orchestration environment. The workload information center may be located on, for example, storage unit nodes, database server nodes, controller nodes, host nodes, compute nodes, etc., in the container orchestration environment. The exemplary embodiment also provides multiple pipeline workload managers in the container orchestration environment. Each pipeline workload manager may be located on, for example, host nodes, compute nodes, etc., which execute steps of application workloads within containers. Furthermore, each pipeline workload manager registers pipeline workload information, which includes, for each pipeline workload and container registered in the workload information center, for example, a pipeline workload identifier, task commands, task command parameters, identifiers corresponding to a set of pipeline workload managers that can use or reuse a particular container within the pipeline workload, a container identifier, a step identifier corresponding to the container identifier, etc. In summary, in this specification, “pipeline workload information” means information registered in the workload information center that can be used to determine whether a container can be used or reused, and may include at least (a) a pipeline workload identifier, (b) a step identifier, (c) a container identifier, (d) a task command, (e) parameters of a task command, (f) an identifier corresponding to a set of pipeline workload managers that can use or reuse the container, and (g) a step identifier corresponding to the container identifier.

[0047] In an exemplary embodiment, pipeline workload information may be registered in the workload information center for low-priority pipeline workloads that do not need to be executed immediately, based on custom rules such as, for example, that high-priority workloads should be executed before low-priority workloads. In an exemplary embodiment, pipeline workload manager B registers its pipeline workload information (for example, that step 1 corresponding to pipeline workload B needs to be executed on container X) in the workload information center. When a step being executed on any container X in another pipeline workload is completed, pipeline workload manager B may reuse container X to execute step 1, saving the cost of executing the container environment preparation substep for container X.

[0048] The exemplary embodiment also generates an agent daemon and injects it into each container running on the host node of the container orchestration environment. The exemplary embodiment makes the agent daemon the default command for each container. Furthermore, the agent daemon in a particular container communicates with the pipeline workload manager corresponding to that particular container. In the exemplary embodiment, when the agent daemon is started on a container as the default command for that container, the agent daemon communicates with the corresponding pipeline workload manager to obtain pipeline workload information (e.g., task commands and their parameters) from the workload information center for the pipeline workload that a user (e.g., customer, client, tenant, etc.) wants to run on the container orchestration environment. Thus, the exemplary embodiment utilizing the agent daemon can support custom task commands for containers.

[0049] The pipeline workload manager corresponding to a container checks pipeline workload information obtained from the workload information center to determine whether the container can be reused for a specific step within a particular pipeline workload on the same or a different host node in the cluster in response to the container completing a step. Based on the pipeline workload information obtained from the workload information center, the pipeline workload manager provides the container to be reused by that specific step within a particular pipeline workload. Furthermore, based on a predefined set of custom rules, the pipeline workload manager may select the first step to be executed on the reused container from among several steps registered in the workload information center that correspond to that pipeline workload.

[0050] As a result, the exemplary embodiment may save, for example, more than 30% of container execution time by not performing the container environment preparation substep for reused containers in the pipeline workload. In other words, the exemplary embodiment may terminate the pipeline workload using, for example, less than 70% of normal container execution time, thereby reducing costs and improving the performance of the container orchestration environment.

[0051] Accordingly, the exemplary embodiments provide one or more technical solutions that overcome technical problems, thereby reducing container execution time and improving performance in a container orchestration environment. As a result, these one or more technical solutions provide technical benefits and practical applications in the field of container orchestration environments.

[0052] Referring here to Figure 3, a diagram illustrating an example of a pipeline workload management system is shown according to an exemplary embodiment. The pipeline workload management system 300 may be implemented in a network of data processing systems, such as the network data processing system 100 in Figure 1. The pipeline workload management system 300 is a system of hardware and software components for optimizing resources for pipeline workloads consisting of multiple steps by reusing containers that have finished executing steps in a pipeline workload to execute specific steps (e.g., identical or different steps) within different pipeline workloads.

[0053] In this embodiment, the pipeline workload management system 300 includes node 302, host node A304, and host node B306. However, it should be noted that the pipeline workload management system 300 is intended merely as an example and not as an limitation to the exemplary embodiment. In other words, the pipeline workload management system 300 may include any number of nodes and components not shown.

[0054] Node 302 may be a storage node, such as storage 108 in Figure 1. Alternatively, node 302 may be a server node, a database node, a controller node, etc. Node 302 includes a workload information center 308. The workload information center 308 includes registered information corresponding to a set of pipeline workloads on one or more host clusters.

[0055] In this embodiment, host nodes A304 and B306 constitute a cluster of host nodes within a container orchestration environment. However, it should be noted that a container orchestration environment may contain any number of host nodes and clusters. Host node A304 includes a pipeline workload manager A310, such as pipeline workload manager 218 in Figure 2. Host node A304 also runs pipeline workload A312. Pipeline workload A312 includes steps 1 314, 2 316 to 318. Step 1 314 consists of an environment preparation substep 320 and a command execution substep 322. Similarly, step 2 316 consists of an environment preparation substep 332 and a command execution substep 334. It should be noted that each step within pipeline workload A312 includes an environment preparation substep and a command execution substep.

[0056] The pipeline workload manager A310 uses agent daemon 326 to execute step 1 314 on container X324. Agent daemon 326 is the default command for container X324. Task command 328 represents a task, job, action, etc., that is executed to produce results for step 1 314 in response to executing the command execution substep 322. Task command 328 includes parameter 330. Parameter 330 represents a set of constraints corresponding to task command 328. Similarly, the pipeline workload manager A310 uses agent daemon 338 to execute step 2 316 on container Y336. Agent daemon 338 is the default command for container Y336. Task command 340 represents a task, job, action, etc., that is executed to produce results for step 2 316 in response to executing the command execution substep 334. Task command 340 includes parameter 342. Parameter 342 represents a set of constraints corresponding to task command 340. In 343, pipeline workload manager A310 registers pipeline workload information (e.g., steps, containers, task commands, parameters, etc.) corresponding to pipeline workload A312 in workload information center 308.

[0057] Host node B306 includes pipeline workload manager B344. Host node B306 also runs pipeline workload B346, which may be the same pipeline workload as pipeline workload A312, or a different pipeline workload. Pipeline workload B346 includes steps 1 314 to "N" 348. Note that step 1 314 of pipeline workload B346 is identical to step 1 314 in pipeline workload A312. Consequently, step 1 314 of pipeline workload B346 also consists of an environment preparation substep 320 and a command execution substep 322. Therefore, once step 1 314 of pipeline workload A312 has finished executing on container X324 on host node A304, pipeline workload manager A310 may share container X324 with host node B306 so that it can be reused to execute step 1 314 of pipeline workload B346 without having to perform environment preparation substep 320 on host node B306.

[0058] Referring now to Figure 4, a diagram illustrating an example of the pipeline workload management process is shown according to an exemplary embodiment. The pipeline workload management process 400 may be implemented in a pipeline workload management system, such as the pipeline workload management system 300 shown in Figure 3.

[0059] In this embodiment, the pipeline workload management process 400 includes, for example, a workload information center 402, a pipeline workload manager A404, and a pipeline workload manager B406, such as the workload information center 308, pipeline workload manager A310, and pipeline workload manager B344 in Figure 3. In stage 1 408, step 1 410 of pipeline workload A412 is completed on container X414. Step 1 410, pipeline workload A412, and container X414 may be, for example, step 1 314, pipeline workload A312, and container X324 in Figure 3.

[0060] In Stage 2, 416, pipeline workload manager A404 checks the pipeline information registered in workload information center 402. In Stage 3, 418, pipeline workload manager A404 updates container X414 with the registered pipeline information and shares container X414 with pipeline workload manager B406. In Stage 4, 420, pipeline workload manager B406 executes the command execution substep 422 of step 1 410 in pipeline workload B426 to execute task commands in container X414. Container X414 was shared with pipeline workload manager B406 by pipeline workload manager A404 without executing the environment preparation substep 428 of step 1 410. In stage 5, 430, pipeline workload manager B406 returns container X414 to pipeline workload manager A404 after step 1, 410 of pipeline workload B426 has finished executing.

[0061] Therefore, once a container has finished executing a step in a pipeline workload, the pipeline workload manager corresponding to the container checks the pipeline workload information obtained from the workload information center to determine whether the container can be reused for a specific step (e.g., the same or a different step) in a different pipeline workload. For example, once the task command for a step has finished executing in the container, the pipeline workload manager checks the pipeline workload information registered in the workload information center for the next action. For example, if the container cannot be reused based on the registered pipeline workload information, the container successfully completes its step and terminates. If the container can be reused for that specific step in another pipeline workload and does not need to be restarted, the current pipeline workload manager shares the container and its pipeline workload information (e.g., task command and its parameters) with the target pipeline workload manager. The target pipeline workload manager then sends the task command and its parameters to the container in order to execute the task command on that container. If a container can be reused by another pipeline workload and needs to be restarted, the current pipeline workload manager restarts the container with a restart command, such as the command restart{container_id}, and shares the restarted container and its pipeline workload information with the target pipeline workload manager.

[0062] Referring now to Figure 5, a diagram illustrating an example of container sharing in a host cluster-level process is shown according to an exemplary embodiment. Container sharing in a host cluster-level process 500 can be implemented, for example, in a network of data processing systems such as the network data processing system 100 in Figure 1.

[0063] The container sharing in the host cluster-level process 500 includes a host cluster 502. The host cluster 502 represents a cluster of host nodes, such as servers 104 and 106 in Figure 1, that run the pipeline workload. In this embodiment, the host cluster 502 includes host node A504, host node B506, host node C508, and host node D510. However, it should be noted that the host cluster 502 is intended merely as an example and not as an limitation to the exemplary embodiment. In other words, the host cluster 502 may include more or fewer host nodes than those illustrated.

[0064] It should be noted that containers are reused by multiple steps within the same or different pipeline workloads on the same or different host nodes. In other words, a container can be reused many times before it terminates. At the host cluster level, the same step can run on the same reused container in a cluster of host nodes, saving the cost of running the container environment preparation substep for that same step each time the step is executed.

[0065] For example, container X512 executes step 1 514 on host node A504 after executing both the environment preparation substep 516 and the command execution substep 518. After container X512 has finished executing step 1 514, the pipeline workload manager 520 on host node A504 shares container X512 with host node B506 to reuse it to execute step 1 514 without executing the environment preparation substep 516 of step 1 514. Note that host node B506 may, as host node A504, be executing the same or a different pipeline workload, including step 1 514. Similarly, the pipeline workload manager 520 on host node A504 shares container X512 with host node C508, which is reused to execute step 1 514 in the pipeline workload on host node C508 without executing the environment preparation substep 516 of step 1 514, and shares container X512 with host node D510, which is reused to execute step 1 514 in the pipeline workload on host node D510 without executing the environment preparation substep 516 of step 1 514. As a result, the performance of the host cluster 502 is improved by saving processing costs by not executing the environment preparation substep 516 on multiple host nodes.

[0066] Referring now to Figure 6, a diagram illustrating an example of a step selection table is shown according to an exemplary embodiment. The step selection table 600 may be implemented in a pipeline workload manager, such as the pipeline workload manager 310 in Figure 3.

[0067] In this embodiment, the step selection table 600 includes step 602, step source 604, priority 606, submission time 608, and submitter name 610. Step 602 identifies the step of the pipeline workload. Step source 604 identifies the specific pipeline workload corresponding to each step in step 602. Priority 606 identifies the priority level corresponding to each step in step 602. Submission time 608 identifies the specific time when each step in step 602 was submitted. Submitter name 610 identifies the specific person who submitted each step in step 602.

[0068] The pipeline workload manager uses the step selection table 600 to select specific steps within a pipeline workload to run on a host node's container, based on a predefined set of custom rules generated by a user, such as the person who requested the performance of the pipeline workload. For example, when multiple steps can run on the same container, the pipeline workload manager may select the step to run first based on a user-defined set of custom rules (e.g., by priority, submission time, submitter name, etc.). The pipeline workload manager uses an agent daemon inserted into the container to run the selected step first on that particular container. The agent daemon inserted into the container may be, for example, agent daemon 326 inserted into container X324 in Figure 3.

[0069] In this embodiment, step X from pipeline workload A has a priority of 1 and was submitted by Tom at 9:00 AM. Step Y from pipeline workload B has a priority of 1 and was submitted by Tom at 9:30 AM. Step Z from pipeline workload B has a priority of 2 and was submitted by Jack at 11:00 AM. If the user's predefined rule indicates priority first, the pipeline workload manager selects step Z to run first on the container because step Z has a priority of 2. In this embodiment, a priority of 2 is higher than a priority of 1. Similarly, if the user's predefined rule indicates submission time first, the pipeline workload manager selects step X to run first on the container because it has an earlier submission time of 9:00 AM, in contrast to step Y with a submission time of 9:30 AM and step Z with a submission time of 11:00 AM.

[0070] Referring here to Figures 7A and 7B, flowcharts illustrating the process for executing steps of a pipeline workload on a previously used container to reduce step execution time are shown according to an exemplary embodiment. The process shown in Figures 7A and 7B may be implemented on a computer, for example, server 104 in Figure 1 or data processing system 200 in Figure 2.

[0071] The process begins when a computer registers pipeline workload information with the Workload Information Center in the container orchestration environment, using the computer's pipeline workload manager, for a pipeline workload consisting of multiple steps to be executed on the computer, and the computer is one of a cluster of computers in the container orchestration environment (step 702). Note that each step within the multiple steps consists of two substeps, including a container environment preparation substep and an execute task command substep. In addition, the computer generates an agent daemon, which is the default command for containers that run the multiple steps containing the pipeline workload on the computer in order to communicate with the pipeline workload manager (step 704). The computer inserts the agent daemon into each container that runs each step of the pipeline workload on the computer (step 706).

[0072] The computer uses the agent daemon of a specific container to communicate to the pipeline workload manager that the specific container has finished executing a step in the pipeline workload (step 708). The computer uses the pipeline workload manager to check the pipeline workload information corresponding to the pipeline workload registered in the workload information center to determine whether the specific container can be reused to execute a specific step in a different pipeline workload (step 710). Note that the specific step may be the same step that has been completed or a different step. Also, the different pipeline workload may be on the computer or on different computers in the cluster. Based on the determination made by the pipeline workload manager, according to the pipeline workload information registered in the workload information center, that the specific container can be reused to execute that specific step in a different pipeline workload, the computer provides the specific container to be reused to execute that specific step in the different pipeline workload without having to perform the container environment preparation substep for that specific step (step 712).

[0073] Furthermore, the computer uses the pipeline workload manager to select another step from among several steps in the pipeline workload to form a selection step to run on a previously used container, based on a predefined set of custom rules (step 714). Note that the previously used container may be on the computer or received from another computer in the cluster. The computer uses the agent daemon of the previously used container to run the selection step in the pipeline workload on the previously used container without executing the container environment preparation substep of the selection step, in order to reduce the runtime of the selection step and improve the computer's performance (step 716).

[0074] The computer then determines whether all of the steps in the pipeline workload have been executed (step 718). If the computer determines that not all of the steps in the pipeline workload have been executed and the output of step 718 is no, the process returns to step 708. In step 708, the computer uses the agent daemon of a particular container to communicate to the pipeline manager that the particular container has finished executing the steps in the pipeline workload. If the computer determines that all of the steps in the pipeline workload have been executed and the output of step 718 is yes, the process then terminates.

[0075] Referring now to Figure 8, a flowchart illustrating the process for container sharing is shown according to an exemplary embodiment. The process shown in Figure 8 may be implemented on a computer, for example, the server 104 in Figure 1 or the data processing system 200 in Figure 2.

[0076] The process begins when the computer receives an indicator from an agent daemon inserted into a container that a container located on the computer has finished executing a step in the pipeline workload used for artificial intelligence processing (step 802). In response to receiving the indicator, the computer uses its pipeline workload manager to retrieve pipeline workload information corresponding to the container and pipeline workload from an external workload information center, such as the workload information center 308 in Figure 3 (step 804).

[0077] The computer uses a pipeline workload manager to update the container with pipeline workload information corresponding to the container and pipeline workload, retrieved from an external workload information center (step 806). The computer uses the pipeline workload manager to send the container, updated with the pipeline workload information, to a different pipeline workload manager based on the pipeline workload information (step 808). Note that the different pipeline workload manager may be located on another computer in the computer cluster that includes the computer. The computer then uses the pipeline workload manager to receive the container from the different pipeline workload manager after the container has finished executing a specific step in the different pipeline workload (step 810). The specific step may be the same or a different step, and the different pipeline workload may be on the computer or another computer in the cluster.

[0078] Accordingly, exemplary embodiments of the present invention provide a computer implementation method, computer system, and computer program product for optimizing resources for a pipeline workload consisting of multiple steps used for artificial intelligence processing on a cluster of host nodes in a container orchestration environment by reusing containers that have finished executing steps of a pipeline workload on a host node to execute the same steps in different pipeline workloads on different host nodes. The description of various embodiments of the present invention is presented for illustrative purposes only and is not intended to be exhaustive or limit to the disclosed embodiments. Many changes and modifications will be apparent to those skilled in the art without departing from the scope of the described embodiments. The terminology used herein has been selected to best describe the principles of the embodiments, their practical applications, or technical improvements to the technology available on the market, or to make the embodiments disclosed herein understandable to others skilled in the art.

Claims

1. A method for reusing containers through computer information processing, The computer uses an agent daemon for a specific container to communicate to the computer's pipeline workload manager that the specific container has finished executing the steps of the pipeline workload, The computer uses the pipeline workload manager to check the pipeline workload information corresponding to the pipeline workload (including at least (a) a pipeline workload identifier, (b) a step identifier, (c) a container identifier, (d) a task command, (e) parameters of the task command, (f) an identifier corresponding to a set of pipeline workload managers that can use or reuse the container, and (g) a step identifier corresponding to the container identifier), and determines, based on the pipeline workload information, whether the particular container can be reused to perform a particular step in a different pipeline workload, Based on the computer using the pipeline workload manager to determine, according to the pipeline workload information, that the particular container can be reused to execute the particular step in the different pipeline workload, the particular container is provided to be reused to execute the particular step in the different pipeline workload without having to execute the container environment preparation substep for the particular step. Methods that include...

2. The computer, using the pipeline workload manager, selects another step from among several steps in the pipeline workload to form a selection step to run on a previously used container based on a set of rules (including selection rules based on priority, submission time, and submitter name). The computer, using the agent daemon of the previously used container, executes the selection step in the pipeline workload on the previously used container without executing the container environment preparation substep of the selection step, in order to reduce the runtime of the selection step and improve the performance of the computer. The method according to claim 1, further comprising:

3. The method according to claim 1, further comprising the computer registering pipeline workload information corresponding to a pipeline workload consisting of a plurality of steps to be executed on the computer with a workload information center in a container orchestration environment, wherein the computer is one of a cluster of computers in the container orchestration environment, and each step in the plurality of steps consists of two substeps, including the container environment preparation substep and the task command execution substep.

4. The computer generates the agent daemon for communicating with the pipeline workload manager, wherein the agent daemon is a default command for a container that executes multiple steps including the pipeline workload, The computer inserts the agent daemon into each container that executes each step of the pipeline workload, The method according to claim 1, further comprising:

5. The computer receives an indication from the agent daemon inserted into the particular container that the particular container has finished executing the step in the pipeline workload, The computer uses the pipeline workload manager to retrieve the pipeline workload information corresponding to the specific container and the pipeline workload from the workload information center, The method according to claim 1, further comprising:

6. The computer updates the specific container with the pipeline workload information corresponding to the specific container and the pipeline workload retrieved from the workload information center, using the pipeline workload manager. The computer uses the pipeline workload manager to send the specific container, updated with the pipeline workload information, to a different pipeline workload manager. The method according to claim 1, further comprising:

7. The method according to claim 1, further comprising the computer using the pipeline workload manager to receive the particular container from a different pipeline workload manager after the particular container has finished performing the particular step in the different pipeline workload.

8. A computer system for reusing containers, Bus system and, A storage device connected to the bus system, wherein the storage device stores program instructions, A processor connected to the bus system, wherein the processor Using the agent daemon of a specific container, the system communicates to the pipeline workload manager of the computer system that the specific container has finished executing the steps of the pipeline workload. Using the pipeline workload manager, check the pipeline workload information corresponding to the pipeline workload (including at least (a) a pipeline workload identifier, (b) a step identifier, (c) a container identifier, (d) a task command, (e) parameters of the task command, (f) an identifier corresponding to a set of pipeline workload managers that can use or reuse the container, and (g) a step identifier corresponding to the container identifier), and determine whether the particular container can be reused to perform a particular step in a different pipeline workload based on the pipeline workload information. Using the pipeline workload manager, and based on the determination that the particular container can be reused to execute the particular step in the different pipeline workload according to the pipeline workload information, the particular container is provided to be reused to execute the particular step in the different pipeline workload without having to execute the container environment preparation substep for the particular step. The processor that executes the program instructions, A computer system equipped with [a certain feature].

9. The aforementioned processor, Using the pipeline workload manager, select another step from among several steps in the pipeline workload to form a selection step to run on a previously used container based on a set of rules (including selection rules based on priority, submission time, and submitter name). To reduce the runtime of the selection step and improve the performance of the computer system, the agent daemon of the previously used container is used to execute the selection step in the pipeline workload on the previously used container without executing the container environment preparation substep of the selection step. The computer system according to claim 8, further executing the program instructions.

10. The aforementioned processor, Using the pipeline workload manager, the pipeline workload information corresponding to the pipeline workload, which consists of multiple steps executed on the computer system, is registered in the workload information center within the container orchestration environment, and the computer system is one of a cluster of computers within the container orchestration environment, and each step in the multiple steps consists of two substeps, including the container environment preparation substep and the task command execution substep. The computer system according to claim 8, further executing the program instructions.

11. The aforementioned processor, The agent daemon is generated for communicating with the pipeline workload manager, and the agent daemon is the default command for a container that executes multiple steps including the pipeline workload. The agent daemon is inserted into each container that executes each step of the pipeline workload. The computer system according to claim 8, further executing the program instructions.

12. The aforementioned processor, An indicator that the particular container has finished executing the step in the pipeline workload is received from the agent daemon inserted into the particular container. Using the pipeline workload manager, retrieve the pipeline workload information corresponding to the specific container and pipeline workload from the workload information center. The computer system according to claim 8, further executing the program instructions.

13. The aforementioned processor, Using the pipeline workload manager, update the specific container with the pipeline workload information corresponding to the specific container and pipeline workload retrieved from the workload information center. Using the pipeline workload manager, send the specific container updated with the pipeline workload information to a different pipeline workload manager. The computer system according to claim 8, further executing the program instructions.

14. A computer program that causes a computer to perform the method according to any one of claims 1 to 7.

15. A storage medium in which the computer program described in claim 14 is stored in a computer-readable storage medium.

Images