Image optimization method and system for pipelined workloads

Abstract

A method and system for optimizing images for pipeline workloads is provided that first determines shared layers that exist between images selected for update management, where the images include executable code that is executed to create containers, then detects changes to the shared layers in the shared layers for images in the images, and, in response to detecting the changes to the shared layers for the images, updates the shared layers in the shared layers for a set of images that have the shared layers.

Description

[Technical Field]

[0001] The present disclosure relates generally to improved computer systems, and more particularly to optimizing pipeline workloads. Even more particularly, the present disclosure relates to methods, apparatus, systems, and computer programs for managing image optimization for pipeline workloads.

[0002] For example, container orchestration environments such as Kubernetes® (a registered trademark of the Linux Foundation, San Francisco, California) provide a platform for automating the deployment, scaling, and operation of containers across a cluster of host nodes. Host nodes are either physical or virtual machines on which containers are deployed. A pod is a group of one or more containers with shared storage and network resources and specifications for how the containers should run. Containers may run workloads. An image represents the executable code that is executed to create a container.

[0003] Pipeline workloads are often executed in a container orchestration environment. Pipeline workloads can include any type of workload. For example, workloads can be artificial intelligence processing, training machine learning models, natural language processing, image processing, computer vision, scientific computing, predictions, forecasts, recommendations, data processing, transaction processing, and the like.

[0004] A pipeline workload divides a workload into many stages, which can be executed sequentially by the pipeline workload to obtain a result. In a container orchestration environment, containers are used to implement these stages, with each container representing one stage among the multiple stages executed by the pipeline workload. For example, pipeline workloads are used in artificial intelligence processing, where a workload is divided into stages that are executed sequentially to perform processing.

[0005] In a pipeline workload, there is an image for each stage. When there are hundreds or thousands of stages, maintaining this number of images can be difficult. For example, if an operating system-level patch is used for some images, these images are currently patched one by one. This process is complex and error-prone. For example, there are 1,000 images, and 800 images use a first Linux version and 200 images use a second Linux version. Currently, if the first Linux version needs a patch, a user must repeatedly build the patched image and upload the image to the Docker server 800 times. This process is tedious, time-consuming, and error-prone.

[0006] One solution involves using a script to handle patch requests. A user writes a build using a script that contains the build and upload logic. The script is executed for each image. However, using a script can still result in errors because some images with the first Linux version may already have been modified, while some images with the first Linux version may not have been modified by the update. Even with a script, collecting image information regarding which images need to be modified still requires manual collection by the user. For example, a user checks all images to identify which images need to be modified and which images have already been updated. Maintaining this list can be tedious, time-consuming, and error-prone.

[0007] As another example, a user can include a script in an image. When the image is loaded to run a container, the script is executed before running the image to create the container. Before the container is run, the script finds patches and applies the patches to files containing code for the container. This process of retrieving patches and patching files in the image before creating the container uses resources such as time and bandwidth. This approach has an increasing performance impact as the number of images with patched scripts increases. When there are thousands of images for a pipeline workload, the performance impact can be significant.

[0008] It would therefore be desirable to have a method and apparatus that takes into account at least some of the challenges discussed above, as well as other potential challenges, such as overcoming the technical problems associated with image preservation. Summary of the Invention

[0009] According to one exemplary embodiment, a computer-implemented method manages image updates. A computer system determines shared layers that exist between images selected for update management. The images include executable code that is executed to create a container. The computer system detects changes to shared layers in shared layers for images in the images. In response to detecting changes to the shared layers for the images, the computer system updates shared layers in shared layers in a set of images having shared layers. According to other exemplary embodiments, a computer system and computer program product for managing image updates are provided. As a result, the exemplary embodiments may provide the technical effects of improving performance by reducing errors in updated images and improving performance of image updates by eliminating the need to individually update code in each image.

[0010] Exemplary embodiments may, in response to detecting the change to the shared layer for the image, permissively update the shared layer in the shared layers in the set of images that have the shared layer by determining a new unique identifier for the shared layer for the changed image; identify the set of images that have the changed shared layer; and update the shared layer for the set of images with the new unique identifier for the changed shared layer. As a result, exemplary embodiments may provide the technical effect of improving performance by reducing errors in updated images and improving performance of image updates by eliminating the need to individually update code in each image. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram of a computing environment in accordance with an illustrative embodiment.

[0012] [Figure 2]1 is a block diagram of an image environment in accordance with an illustrative embodiment;

[0013] [Figure 3] 3 is an illustration of images managed by an image manager in accordance with an example embodiment;

[0014] [Figure 4] FIG. 1 is a block diagram illustrating determining shared layers that exist between images, according to an example embodiment.

[0015] [Figure 5] FIG. 4 is an illustration of a table identifying shared layers according to an example embodiment.

[0016] [Figure 6] FIG. 1 is a block diagram illustrating detecting changes to a shared layer, according to an example embodiment.

[0017] [Figure 7] FIG. 10 is a block diagram illustrating updating a shared layer according to an example embodiment.

[0018] [Figure 8] FIG. 2 is an illustration of an image manager for images used in a pipeline workload, according to an example embodiment.

[0019] [Figure 9] 10 is a flowchart of a process for identifying shared layers in an image selected for automatic updating, according to an example embodiment.

[0020] [Figure 10] 10 is a flowchart of a process for detecting changes to shared layers between images in accordance with an example embodiment.

[0021] [Figure 11]10 is a flowchart of a process for monitoring a service port to detect changes to shared layers between images, in accordance with an example embodiment.

[0022] [Figure 12] 10 is a flowchart of a process for updating a shared layer between images in accordance with an example embodiment.

[0023] [Figure 13] 10 is a flowchart of a process for updating a shared layer for an image in accordance with an example embodiment.

[0024] [Figure 14] 10 is a flowchart of a process for deleting a selected image in accordance with an illustrative embodiment.

[0025] [Figure 15] 10 is a flowchart of a process for determining whether shared layers exist between images in accordance with an example embodiment.

[0026] [Figure 16] 10 is a flowchart of a process for determining whether shared layers exist between images in accordance with an example embodiment.

[0027] [Figure 17] 10 is a flowchart of a process for detecting changes to a shared layer in accordance with an example embodiment.

[0028] [Figure 18] 10 is a flowchart of a process for detecting a command to update an image in accordance with an illustrative embodiment.

[0029] [Figure 19] 10 is a flowchart of a process for updating a shared layer in accordance with an example embodiment.

[0030] [Figure 20] 1 is a block diagram of a data processing system in accordance with an illustrative embodiment; DETAILED DESCRIPTION OF THE INVENTION

[0031] Various aspects of the present disclosure are described through text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in computer program product (CPP) embodiments. For any flowchart, depending on the technology involved, operations may be performed in an order different from that shown in a given flowchart. For example, depending again on the technology involved, two operations shown in successive flowchart blocks may be performed in the reverse order, as a single integrated step, simultaneously, or in an at least partially overlapping manner.

[0032] A computer program product embodiment ("CPP embodiment" or "CPP") is a term used in this disclosure to describe any set of one or more storage media (also referred to as "media"), collectively contained in one or more storage devices, that collectively contain machine-readable code corresponding to instructions and / or data for performing the computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. The computer-readable storage medium may be, but is not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanically encoded device (such as pits / lands formed on a major surface of a punch card or disk), or any suitable combination of the foregoing. Computer-readable storage media, as the term is used in this disclosure, is not to be construed as storage in the form of a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses passing through fiber optic cables, electrical signals transmitted through wires, and / or other transmission media. As will be appreciated by those skilled in the art, data is typically moved at some infrequent time during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but this does not make the storage device temporary because the data is not temporary while it is stored.

[0033] Referring now to the figures, and in particular to FIG. 1 , a block diagram of a computing environment according to an exemplary embodiment is shown. Computing environment 100 includes an example environment for execution of at least a portion of computer code involved in performing the methods of the present invention, such as improved image manager code 190. In this illustrative example, improved image manager code 190 operates to manage images. For example, images or pipeline workloads may be managed by improved image manager code 190. In the illustrative example, improved image manager code 190 may detect that a shared layer in an image has been updated and automatically update corresponding shared layers in other images that include the shared layer. In addition to improved image manager code 190, computing environment 100 includes, for example, computer 101, wide area network (WAN) 102, end user device (EUD) 103, remote server 104, public cloud 105, and private cloud 106. In this embodiment, computer 101 includes a set of processors 110 (including processing circuitry 120 and cache 121), a communications fabric 111, volatile memory 112, persistent storage 113 (including operating system 122 and improved image manager code 190, as identified above), a set of peripheral devices 114 (including a set of user interface (UI) devices 123, storage 124, and a set of Internet of Things (IoT) sensors 125), and a network module 115. Remote server 104 includes a remote database 130. Public cloud 105 includes a gateway 140, a cloud orchestration module 141, a set of host physical machines 142, a set of virtual machines 143, and a set of containers 144.

[0034] Computer 101 may take the form of a desktop computer, a laptop computer, a tablet computer, a smartphone, a smartwatch or other wearable computer, a mainframe computer, a quantum computer, or any other form of computer or mobile device now known or later developed that is capable of executing programs, accessing a network, or querying a database, such as remote database 130. As is well understood in the field of computer technology, and depending on the technology, execution of a computer-implemented method may be distributed among multiple computers and / or multiple locations. However, in this presentation of computing environment 100, to keep the presentation as concise as possible, the detailed discussion focuses on a single computer, specifically computer 101. Although computer 101 is not shown in FIG. 1 within a cloud, it may be located within a cloud. However, computer 101 is not required to reside within a cloud except to any extent expressly indicated.

[0035] Processor set 110 includes one or more computer processors of any type now known or to be developed in the future. Processing circuitry 120 may be distributed across multiple packages, e.g., multiple tailored integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within the processor chip package and is typically used for data or code that should be available for fast access by threads or cores executing on processor set 110. Cache memory is typically organized into multiple levels depending on relative proximity to the processing circuitry. Alternatively, some or all caches for a processor set may be located “off-chip.” In some computing environments, processor set 110 may be designed to operate with qubits and perform quantum computing.

[0036] Computer-readable program instructions are typically loaded onto computer 101 and cause processor set 110 of computer 101 to execute a series of operational steps, thereby enabling a computer-implemented method, such that the instructions so executed instantiate the methods specified in the computer-implemented method flowcharts and / or descriptions contained herein (collectively referred to as the "methods of the present invention"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by processor set 110 to control and direct the execution of the methods of the present invention. In computing environment 100, at least a portion of the instructions for executing the methods of the present invention may be stored in enhanced image manager code 190 in persistent storage 113.

[0037] Communications fabric 111 is the signal-conducting pathway that allows various components of computer 101 to communicate with one another. Typically, this fabric is made up of switches and conductive pathways, such as switches and conductive pathways that make up buses, bridges, physical input / output ports, and the like. Other types of signal communication pathways may be used, such as fiber optic communication pathways and / or wireless communication pathways.

[0038] Volatile memory 112 may be any type of volatile memory now known or later developed. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory 112 is characterized by random access, although this is not required unless expressly indicated. In computer 101, volatile memory 112 is located in a single package and is internal to computer 101; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or located external to computer 101.

[0039] Persistent storage 113 is any form of non-volatile storage for a computer, now known or later developed. The non-volatility of this storage means that stored data is maintained regardless of whether power is supplied to computer 101 and / or to persistent storage 113 directly. Persistent storage 113 can be read-only memory (ROM), but typically at least a portion of persistent storage allows data to be written, data to be erased, and data to be rewritten. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. Operating system 122 can take several forms, including various known proprietary operating systems or open-source Portable Operating System Interface-type operating systems that employ a kernel. The code included in improved image manager code 190 typically includes at least a portion of the computer code involved in performing the methods of the present invention.

[0040] The peripheral device set 114 includes a set of peripheral devices of the computer 101. Data communication connections between the peripheral devices and other components of the computer 101 can be implemented in various ways, such as Bluetooth connections, Near-Field Communication (NFC) connections, connections made by cable (such as a Universal Serial Bus (USB)-type cable), pluggable connections (e.g., a Secure Digital (SD) card), connections made through a local area communication network, and even connections made through a wide area network such as the Internet. In various embodiments, the UI device set 123 can include components such as a display screen, speakers, microphones, wearable devices (such as goggles and smartwatches), keyboards, mice, printers, touchpads, game controllers, and haptic devices. The storage 124 can be external storage, such as an external hard drive, or insertable storage, such as an SD card. The storage 124 can be persistent and / or volatile. In some embodiments, the storage 124 can take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where computer 101 is required to have a large amount of storage (e.g., where computer 101 stores and manages large databases locally), this storage may be provided by a peripheral storage device designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple, geographically distributed computers. IoT sensor set 125 consists of sensors used in Internet of Things applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

[0041] Network module 115 is a collection of computer software, hardware, and firmware that enables computer 101 to communicate with other computers over WAN 102. Network module 115 may include hardware such as a modem or Wi-Fi® signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data over the Internet. In some embodiments, the network control and network forwarding functions of network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing Software-Defined Networking (SDN)), the control and forwarding functions of network module 115 are performed on physically separate devices, such that the control function manages multiple different network hardware devices. Computer-readable program instructions for implementing the methods of the present invention may be downloaded to computer 101 from an external computer or external storage device, typically through a network adapter card or network interface included in network module 115.

[0042] WAN 102 is any wide area network (e.g., the Internet) capable of communicating computer data over non-local distances using any technology for communicating computer data now known or later developed. In some embodiments, WAN 102 may be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area, such as a Wi-Fi network. WANs and / or LANs typically include copper transmission cables, optical fiber transmissions, wireless transmissions, and computer hardware such as routers, firewalls, switches, gateway computers, and edge servers.

[0043] End-user device (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of the enterprise operating computer 101) and may take any of the forms discussed above in connection with computer 101. EUD 103 typically receives useful and useful data from the operation of computer 101. For example, in a hypothetical case where computer 101 is designed to provide recommendations to the end user, the recommendations would typically be communicated from computer 101's network module 115 over WAN 102 to EUD 103. In this manner, EUD 103 can display or otherwise present the recommendations to the end user. In some embodiments, EUD 103 may be a client device, such as a thin client, a heavy client, a mainframe computer, a desktop computer, and the like.

[0044] Remote server 104 is any computer system that provides at least some data and / or functionality to computer 101. Remote server 104 may be controlled and used by the same entity that operates computer 101. Remote server 104 represents a machine that collects and stores useful and useful data for use by other computers, such as computer 101. For example, in the hypothetical case where computer 101 is designed and programmed to provide recommendations based on past data, this past data may be provided to computer 101 from remote database 130 of remote server 104.

[0045] A public cloud 105 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computer capabilities, particularly data storage (cloud storage) and computing power, without direct active management by users. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. Direct active management of public cloud 105 computing resources is performed by computer hardware and / or software in cloud orchestration module 141. The computing resources provided by public cloud 105 are typically implemented by virtual computing environments running on various computers comprising host physical machine set 142, which is the universe of physical computers within and / or available in public cloud 105. Virtual computing environments (VCEs) typically take the form of virtual machines from virtual machine set 143 and / or containers from container set 144. It is understood that these VCEs are stored as images and can be transferred among and between various physical machine hosts, either as images or after instantiation of the VCEs. Cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of VCE, and manages active instantiations of VCE deployments. Gateway 140 is a collection of computer software, hardware, and firmware that enables public cloud 105 to communicate over WAN 102.

[0046] Some further discussion of virtualized computing environments (VCEs) is now provided. A VCE can be stored as an "image." A new, active instance of a VCE can be instantiated from an image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to a feature of an operating system in which the kernel allows the existence of multiple isolated user space instances, called containers. These isolated user space instances typically behave as actual computers from the perspective of programs running within them. A computer program running on a typical operating system can utilize all of the computer's resources, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running inside a container can only use the contents of the container and of the devices assigned to the container; this feature is known as containerization.

[0047] A private cloud 106 is similar to a public cloud 105, except that its computing resources are available only for use by a single enterprise. While the private cloud 106 is shown in communication with the WAN 102, in other embodiments, the private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a composite of multiple clouds of different types (e.g., private, community, or public cloud types), often each implemented by a different vendor. While each of the multiple clouds remains a separate, discrete entity, the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the constituent clouds. In this embodiment, both the public cloud 105 and the private cloud 106 are part of a larger hybrid cloud.

[0048]

[0009] Exemplary embodiments provide a method, apparatus, system, and computer program product for managing image updates. A computer system determines shared layers that exist between images selected for update management. The images include executable code that is executed to create a container. The computer system detects changes to shared layers in shared layers for images in the images. In response to detecting the changes to the shared layers for the images, the computer system updates shared layers in shared layers in a set of images that have shared layers. One or more of the illustrative examples described herein may provide the technical effect of improving performance by reducing errors in image updates and improving image update performance by eliminating the need to individually update code in each image.

[0049] As used herein, a "set of," when used in reference to an item, means one or more items. For example, a set of images is one or more images.

[0050] Referring now to Figure 2, a block diagram of an image environment is shown in accordance with an illustrative embodiment. In this illustrative example, image environment 200 includes components implemented in hardware, such as the hardware shown in computing environment 100 in Figure 1.

[0051] In this illustrative example, an image management system 202 in an image environment 200 manages an image 204. As shown, the image 204 includes executable code that is executed to create a container 206 from the image 204. In this example, the code in the image 204 is contained in binary files, libraries, and other types of files that are executed.

[0052] In this illustrative example, image 204 may form pipeline workload 205. In one illustrative example, pipeline workload 205 formed using image 204 may be artificial intelligence pipeline 207. In this example, image 204 is for container 206, which executes stages in artificial intelligence pipeline 207. If artificial intelligence pipeline 207 includes a machine learning model, stages such as data retrieval, dataset construction, data cleanup, data transformation, model training, model evaluation, and other stages may be executed in artificial intelligence pipeline 207. Additionally, if artificial intelligence pipeline 207 involves training a machine learning model, the training may include stages such as dataset splitting, algorithm selection, training data analysis, and hyperparameter tuning.

[0053] Container 206 is a container running on container orchestration platform 208, which may be, for example, a Kubernetes® architecture, environment, or the like. However, it should be understood that the description of the illustrative example using Kubernetes® is intended only as an example architecture and not as a limitation on the example embodiment. In this example, container 206 is for a workload, which is an application running on container orchestration platform 208.

[0054] As shown, image management system 202 comprises a computer system 210 and an image manager 212. Image manager 212 is located within computer system 210.

[0055] Image manager 212 may be implemented in software, hardware, firmware, or a combination thereof. If software is used, the operations performed by image manager 212 may be implemented in program instructions configured to run on hardware, such as a processor unit. If firmware is used, the operations performed by image manager 212 may be implemented in program instructions and data and stored in persistent memory for execution on a processor unit. If hardware is employed, the hardware may include circuitry that operates to perform the operations in image manager 212.

[0056] In illustrative examples, the hardware may take the form of at least one selected from a circuit system, an integrated circuit, an application specific integrated circuit (ASIC), a programmable logic device, or any other suitable type of hardware configured to perform multiple operations. In a programmable logic device, the device may be configured to perform multiple operations. The device may be reconfigured at a later point in time or may be permanently configured to perform multiple operations. Programmable logic devices include, for example, programmable logic arrays, programmable array logic, field programmable logic arrays, field programmable gate arrays, and other suitable hardware devices. Additionally, these processes may be implemented in organic components integrated with inorganic components or may be composed entirely of organic components to the exclusion of humans. For example, these processes may be implemented as circuits in organic semiconductors.

[0057] Computer system 210 is a physical hardware system and includes one or more data processing systems. When more than one data processing system is present in computer system 210, the data processing systems communicate with each other using a communication medium. The communication medium may be a network. The data processing systems may be selected from at least one of a computer, a server computer, a tablet computer, or any other suitable data processing system.

[0058] As used herein, the phrase "at least one of," when used in conjunction with a list of items, means that one or more different combinations of the listed items may be used, and that only one of each item in the list may be required. In other words, "at least one of" means that any combination of items and multiple items from the list may be used, but not all items in the list are required. An item may be a specific object, thing, or category.

[0059] For example, "at least one of item A, item B, or item C" may include, but is not limited to, item A, item A and item B, or item B. This example may also include item A, item B, and item C, or item B and item C. Of course, any combination of these items may be present. In some illustrative examples, "at least one of" may be, for example, but is not limited to, two of item A; one of item B and ten of item C; four of item B and seven of item C; or other suitable combinations.

[0060] As shown, computer system 210 includes multiple processor units 214 capable of executing program instructions 216 that implement the processes in the illustrative example. As used herein, a processor unit in multiple processor units 214 is a hardware device, made up of hardware circuitry, such as hardware circuitry on an integrated circuit, that responds to and processes program instructions and instructions that operate a computer. When multiple processor units 214 execute program instructions 216 for processing, multiple processor units 214 are one or more processor units that may reside on the same computer or on different computers. In other words, the processing may be distributed among processor units on the same or different computers within the computer system.

[0061] Furthermore, the multiple processor units 214 may be the same or different types of processor units, for example, the multiple processor units 214 may be selected from at least one of a single-core processor, a dual-core processor, a multi-processor core, a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or some other type of processor unit.

[0062] Image manager 212 may manage changes to images 204. In this illustrative example, images 204 are selected for management regarding shared layers 218 that exist between images 204. Shared layers 218 in images 204 mean that these layers exist between all or a subset of images 204. Image manager 212 determines the shared layers 218 that exist between images 204 selected for update management.

[0063] In this example, image manager 212 detects changes 219 to shared layer 220 in shared layer 218 in image 222 in image 204. Changes 219 may be the result of, for example, a patch in the form of an operating system, a library update, or other change to shared layer 218. Shared layer 220 with changes 219 becomes updated shared layer 224.

[0064] In this example, in response to detecting a change to the shared layer 220 in an image 222, the image manager 212 updates the shared layer 220 in the shared layers 218 in the set of images 204 that have the shared layer 220.

[0065] An update for a set of images 204 having a shared layer 220 may be performed by including the shared layer 220 with the changes 219 in the set of images 204 for the shared layer 220 without the changes 219. In other words, there is no need to patch the executable code for the corresponding layer in the set of images 204. Instead, the shared layer 220 with the changes 219 may be used in place of the shared layer 220 without the changes 219 that was present in the set of images 204 prior to the update performed by the image manager 212.

[0066] For example, image 204 may include 600 images 204, with 350 of the images 204 having a shared layer 220. As a result, if a change 219 is made to the shared layer 220 in one of the 350 images, the other 349 images are updated to use the shared layer 220 with the change 219. This change to the other 349 images in image 204 is made without patching the actual code in the files for the shared layer 220. Instead, these layers may be updated to reference or point to the updated shared layer 224, which is the shared layer 220 with the change 219. Thus, image manager 212 automatically updates the shared layer 218 for image 204 in response to detecting a change to the shared layer 218.

[0067] Additionally, a selected image 226 within images 204 may be deleted from images 204. This deletion of the selected image 226 may be performed in response to a user input to suspend automatic updates for the selected image 226. In response to the deletion of the selected image 226 from images 204, image manager 212 no longer considers the selected image 226 for updates.

[0068] For example, if shared layer 220 is an operating system, an updated version of the operating system may not work with other layers in image 204. Images for which updating the shared layer causes problems may be removed from image 204 so that automatic updates by image manager 212 are not performed for selected images 226. These problems may be, for example, execution errors, performance hits, or other issues.

[0069] In one exemplary embodiment, one or more technical solutions exist that overcome the technical problems associated with managing image updates when a large number of images exist. As a result, the one or more technical solutions may provide a technical effect of managing images through identifying shared layers that are common between images, detecting changes to the shared layers, and updating the shared layers in the images that correspond to the updated shared layers. As one example, there may be a technical effect of reducing errors and improving performance in updated images by eliminating the need to individually update code in each image.

[0070] Referring to Figure 3, an illustrative diagram of images managed by an image manager is shown, according to an exemplary embodiment. In the illustrative example, the same reference number may be used in more than one figure. This reuse of a reference number in different figures represents the same element in the different figures.

[0071] An example of the information that may be present in an image 222 is shown in this figure. As shown, image 222 includes file 301, image information 302, and layer context information 303.

[0072] In this example, file 301 contains executable code for layer 304 that is executed to create a container for image 222. This executable code may be in the form of machine-readable code, such as binary code and libraries, that is executed by a processor unit in a computer.

[0073] Image information 302 is metadata that includes information about image 222. In this example, image information 302 may include image name, owner, created file, size, created time, and other information.

[0074] In this illustrative example, layer context information 303 is metadata that includes information about layers 304 in file 301. For example, layer context information 303 may be a list of unique identifiers (UIDs) 306 for layers 304 for image 222. Thus, unique identifiers 306 are pointers to layers 304 for image 222 and may be used to obtain file 301 for layer 304 if file 301 for layer 304 is not stored with image 222. Using unique identifiers 306 for layers 304 may reduce storage space for the image.

[0075] As shown in the illustrative example, file 301 for layer 304 for image 222 may be stored in a separate location from image information 302 and layer context information 303 and referenced at that location by layer context information 303. In this illustrative example, when used to implement image 204 managed by image manager 212, image 222 may include image information 302 and layer context information 303 without file 301. In other words, image manager 212 stores image information 302 and layer context information 303, but does not store file 301 for layer 304. As a result, file 301 for layer 304, including shared layer 218 in FIG. 2, may be retrieved when file 301 is needed to execute image 204 in FIG. 2.

[0076] When image 222 is sent to a requestor for use, the requestor receives image 222 with image information 302 and layer context information 303. The requestor may use unique identifier 306 and layer context information 303 to obtain file 301 for layer 304. In this illustrative example, unique identifier 306 references where file 301 can be found. For example, unique identifier 306 points to a container repository that contains the file for layer 304. With file 301 retrieved, image 222 is in a format that can be executed to create a container.

[0077] 4, a block diagram illustrating determining shared layers that exist between images is shown, in accordance with an example embodiment. Information about the images 204 can be used to determine the existence of shared layers 218 between the images 204.

[0078] Image manager 212 may use context information 400 about images 204 to determine shared layers 218 that exist between images 204 selected for update management. In this illustrative example, determining shared layers 218 that exist between images 204 selected for update management may be performed by image manager 212 using layer context information about images 204.

[0079] For example, image manager 212 may compare unique identifiers (UIDs) 402 for layers 404 for images 204 with each other. Layers 404 between images 204 that have matching unique identifiers form shared layers 218 between those images. For example, a selected image within images 204 may have a selected layer with a unique identifier UID1 that identifies the selected layer as a Linux operating system. Each layer in another image within image 204 that has UID1 as its unique identifier is a shared layer with the selected layer.

[0080] Thus, layers 404 for these images with unique identifier UID1 are a set of shared layers 218 within layers 404. In other words, layers 404 for images 204 with UID1 as a unique identifier are a set of shared layers 218.

[0081] Multiple sets of shared layers may exist between images 204. For example, another unique identifier UID2 in a layer in a selected image may be for a C library. A comparison of the unique identifier UID2 with the unique identifiers (UIDs) 402 for other images in image 204 may be used to identify the use of the same layer for some or all of image 204 to form another set of shared layers 218. As a result, layer 404 with unique identifier UID2 is a set of shared layers 218.

[0082] In this illustrative example, image manager 212 gathers information 405 about the relationships between images 204 and shared layers 218 from determining the existence of shared layers 218 between images 204. Information 405 may be stored in data structure 406. Data structure 406 may include at least one of a database, a table, a linked list, a flat file, or any other suitable type of data structure that may be used to store information 405 about shared layers 218 and images 204 that include shared layers 218.

[0083] 5, an illustration of a table identifying shared layers is shown, according to an example embodiment. In this illustrative example, shared layers table 500 is an example of a table in a database used to implement data structure 406 in FIG.

[0084] As shown in shared layer table 500, layer column 501 identifies the layer name, UID column 502 identifies a unique identifier for the layer, and layer creation date 504 identifies the date the shared layer was created. This creation date in shared layer table 500 can be compared to the creation date for the layer in the image being examined for changes. The comparison can be made to determine whether a change occurred between when the shared layer was added to shared layer table 500 and when the image with the shared layer is examined. Additionally, in shared layer table 500, image name 505 identifies the image in which the layer with the unique identifier resides.

[0085] As shown in this simplified example, in line 510, the shared layer is Linux® with a unique identifier of UIDA. This shared layer was created on February 16, 2022 at 5:18 AM. In this example, UIDA uniquely identifies the shared layer Linux within the repository. UIDA can take various forms, such as a hash number, an alphanumeric value, a universal resource locator, or some other value that uniquely identifies Linux from other layers. In this example, multiple images, Image X, Image Y, and Image Z, are images with Linux as a layer with the unique identifier UIDA. The unique identifier can be used as an index to find images with the shared layer.

[0086] As another example, row 512 in shared layer table 500 is named centos:7 with unique identifier UIDB. The creation date of this shared layer is April 7, 2022, 10:18. The images that have this shared layer are Image A, Image C, and Image F. In other words, all three of these images have the same shared layer.

[0087] In yet another example, row 514 is for a shared layer named Data Collection with a unique identifier UIDC. The creation date of this shared layer is June 11, 2022, 17:21. In this example, the images that have this shared layer are Image A and Image X.

[0088] In addition to identifying images that have shared layers, the shared layer table 500 can also be used to identify shared layers within an image. For example, an image name can be used to look up shared layers for an image with the image name in the shared layer table 500. As shown, image A has two shared layers with unique identifiers UIDB and UIDC. In addition, image X has two shared layers identified with unique identifiers UIDA and UIDC. Image Y and image Z have a single shared layer with unique identifier UIDA. Image C and image F have a single shared layer with unique identifier UIDB.

[0089] The illustration of shared layers table 500 is provided as a simplified illustration of how information is determined from identifying shared layers between images and how that information is stored and used by image manager 212 in Figure 2. In other illustrative examples, shared layers table 500 may have hundreds or thousands of rows.

[0090] In this illustrative example, the image name in the image name 505 column uniquely identifies the image from other images. In other illustrative examples, an image identifier may be used in addition to or instead of the image name. In other illustrative examples, a separate table may be used in which each image is represented by a row with the layer name and unique identifier for layers that are shared layers in the image.

[0091] As another example, the shared layer table 500 may also include additional information. For example, a creation date for an image may be stored with the image name. The creation date may be used to determine whether changes have occurred in the image since the image was examined to identify shared layers for the shared layer table 500.

[0092] 6, a block diagram illustrating detecting changes to a shared layer is shown, according to an example embodiment. In this illustrative example, image manager 212 may detect changes 219 to shared layer 220 in shared layer 218 in a number of different ways. For example, image manager 212 may detect local file changes 606 to shared layer 218 for image 204 by checking images 204 stored on image server 602, monitoring a service port 604 for uploading images 204 to image server 602, or monitoring local file changes 606. In this example, image server 602 may be a Docker server.

[0093] For example, image manager 212 checks images 204 stored on image server 602 for changed images. For example, image manager 212 may detect changes to image 222 using image information 302 for image 222, which indicates that image 222 has been changed. In this illustrative example, any updates to image 222 or the application of any patches to image 222 are reflected in image information 302. For example, a change to a layer in image 222 results in the creation time being updated to the latest build. This information obtained for image 222 may be compared to the image information stored in data structure 406 for image 222.

[0094] In response to identifying the changes to image 222 using image information 302, image manager 212 may determine that shared layer 220 in image 222 has changed using layer context information 303 for image 222 that indicates that shared layer 220 for image 222 has changed in response to detecting the changes to image 222. For example, the UID in layer context information 303 for image 222 may be compared to layer context information stored for image 222. Layer context information 303 may be used for the comparison and may be stored in data structure 406 in the form of one or more tables, such as shared layer table 500 in FIG.

[0095] In another illustrative example, changes to image 204 may be monitored through service port 604. For example, if image 222 is uploaded to service port 604, detecting the upload of image 222 may cause image manager 212 to determine whether changes have occurred to image 222 and whether the changes include shared layers. In this example, image 204 is checked when image 204 is uploaded to image server 602 using service port 604.

[0096] In another example, local file change 606 may be used to determine whether an update of a shared layer for image 222 has occurred. For example, image manager 212 may detect command 608 used as part of local file change 606 to update image 222. For example, image manager 212 may detect command 608 to update image 222. Image manager 212 may determine that shared layer 220 for image 222 has changed based on command 608 for layer for image 222. In this illustrated example, the command may indicate that shared layer 220 in layer 218 has changed in response to detecting command 608 to update image 222.

[0097] For example, if the image is used in a container orchestration platform such as Kubernetes®, the commands 608 may be located in a Dockerfile 609. These commands are executed to build the image 222. A user may create a Dockerfile 609 that includes commands 608 that include unique identifiers for layers for the image 222. The user may then run a Docker build command using the Dockerfile 609 that includes commands 608.

[0098] In this illustrated example, image manager 212 may detect execution of command 608 in Dockerfile 609 and identify the layer referenced by the command. The command may use a unique identifier, which may then be used to determine whether a change has occurred in shared layer 218 for image 222.

[0099] 7, a block diagram illustrating updating a shared layer is shown, according to an example embodiment. In this illustrative example, image manager 212 may update shared layer 220 in shared layers 218 in a set of images 204 that have shared layer 220. This update may be performed by image manager 212 in response to detecting a change to shared layer 220.

[0100] In this illustrative example, image manager 212 determines that shared layer 220 for image 222 has been modified and has a new unique identifier 700 instead of the previous unique identifier 702. In this example, shared layer 220 is for modified image 222.

[0101] Determining the previous unique identifier 702 may be done using the creation date for the shared layer 220 and the information in the data structure 406, which may be the shared layer table 500 in FIG. 5. For example, the creation date for the shared layer 218 for the image 222 may be compared to the creation dates for those shared layers stored in the data structure 406. Unmodified shared layers in the shared layers 218 have the same creation date. When compared to the entry for the shared layer 220 in the data structure 406, a newer creation date for the shared layer 220 indicates that the shared layer 220 has been modified.

[0102] In this example, the unique identifier for the shared layer 220 in the data structure 406 is different from the new unique identifier 700 for the shared layer 220. This unique identifier in the data structure 406 for the shared layer 220 is the previous unique identifier 702.

[0103] Additionally, the data structure 406 can be used to identify the shared layers 220 for other images within the image 204. Thus, the image manager 212 identifies the set of images 204 that have the modified shared layers 220. The shared layers 218 for the images 204 can be determined using the data structure 406, which has the previous unique identifier 702 for the shared layer 220 changed to the new unique identifier 700.

[0104] The image manager 212 updates the shared layer 220 for the set of images with the new unique identifier 700 for the changed shared layer. The shared layer 220 for the image 204 identified using the data structure 406 can be updated to change the previous unique identifier 702 to the new unique identifier 700. The update can be performed in several different ways.

[0105] As a result, the set of shared layers 218 in the set of images 204 is updated. This update can be performed without having to apply patches to change code in files in the set of images 204.

[0106] 2 may be configured to perform at least one of the steps, operations, or actions described in the different illustrative examples using software, hardware, firmware, or a combination thereof. As a result, computer system 210 operates as a special-purpose computer system that enables image manager 212 within computer system 210 to automatically manage images. In particular, image manager 212 transforms computer system 210 into a special-purpose computer system compared to currently available general-purpose computer systems that do not have image manager 212.

[0107] In an illustrative example, the use of image manager 212 in computer system 210 integrates the process into the actual application by managing image updates for containers, thereby improving the performance of computer system 210. In other words, image manager 212 in computer system 210 targets the actual application of the process integrated into image manager 212 in the computer system to identify shared layers in an image, detect changes to the shared layers in the image, and update the shared layers as the image changes.

[0108] The illustration of image environment 200 and different components in Figures 2-7 are not meant to imply physical or architectural limitations to the manner in which example embodiments may be implemented. Other components in addition to or instead of those shown may be used. Some components may be unnecessary. Also, blocks are presented to illustrate some functional components. One or more of these blocks may be combined, divided, or combined and divided into different blocks when implemented in an example embodiment.

[0109] For example, image 204 may be used to create container 206 on one or more container orchestration platforms in addition to, or instead of, container orchestration platform 208. As another example, other tables in addition to shared layers table 500 may be used for data structure 406. For example, an image table may be used to identify an image and the shared layers for that image. In other words, an image name may be indexed to identify the shared layers that the image uses.

[0110] Referring now to FIG. 8, an illustration of an image manager for images used in a pipeline workload is shown, according to an exemplary embodiment. In this illustrative example, image manager 800 is an example implementation for image manager 212 in FIG. 2. In this example, image manager 800 manages image X 802, image Y 804, and image Z 806. Each of these images may be executed to create a container for an artificial intelligence pipeline in this example. Only three images are shown to illustrate different aspects of how image manager 800 may manage images. When implemented, image manager 800 may manage many more images, such as hundreds or thousands of images.

[0111] In this example, image X 802, image Y 804, and image Z 806 have been selected for management by image manager 800. In this example, image X 802 includes a data collection layer 810 and a Linux layer 812. Image Y 804 includes a data cleaning layer 814 and a Linux layer 816, and image Z 806 includes a module training layer 818 and a Linux layer 820.

[0112] In this illustrated example, image manager 800 examines the image and identifies shared layers 822, which include Linux layer 812, Linux layer 816, and Linux layer 820. In this example, the shared layers have the same unique identifier. Image manager 800 builds a shared list 824 for these images and shared layers 822.

[0113] The identity of the shared layer 822 may be determined using a unique identifier for the shared layer 822 found in the layer context information for the image. In this illustrative example, the layer context information may be obtained by inspecting the image or from detecting a command that creates the image.

[0114] In this illustrative example, image manager 800 may monitor the images to detect updates made to one or more of the images. For example, image manager 800 may detect changes to shared layer 822 by checking images stored on image server 830, monitoring a service port for uploading images, or monitoring local file changes.

[0115] For example, a user may perform a local file change that applies a patch to image X 802. This patching changes Linux layer 812, which results in a change to the unique identifier for Linux layer 812. In response to detecting this change, image manager 800 may update the unique identifier for Linux layer 816 in image Y 804 and for Linux layer 820 for image Z 806. With this update, all shared layers 822 now have the same unique identifier. This update is also made by image manager 800 to share list 824 for future comparison.

[0116] In this illustrative example, when images are updated, the updated images may be uploaded to another location, such as image server 830 in cloud 832. Image server 830 may be a Docker server. Thus, image manager 800 may manage complex image updates for a large number of images for a pipeline workload. These images for a pipeline workload may number in the hundreds of thousands of images. Image manager 800 may identify shared layers between images, monitor the images for changes to the shared layers, and, in response to detecting changes to the shared layers, update corresponding shared layers for other images.

[0117] If automatic updates of shared layers are not desired, the image may be removed from the shared list 824. For example, if updating a shared layer in an image in the shared list 824 would cause other layers to run incorrectly or with errors, the image may be removed from the shared list 824. In this way, errors caused by current techniques for updates can be avoided, and the quality of the pipeline workload can be improved.

[0118] Referring now to Figure 9, a flowchart of a process for identifying shared layers in an image selected for automatic updating is shown, according to an example embodiment. The process in Figure 9 may be implemented in hardware, software, or both. If implemented in software, the process may take the form of program instructions executed by one or more processor units located in one or more hardware devices in one or more computer systems. For example, the process may be implemented in image manager 212 in computer system 210 in Figure 2 and image manager 800 in Figure 8.

[0119] The process begins by identifying images for automatic update (step 900). The process then identifies shared layers in all images selected for automatic update (step 902). In step 902, identifying the shared layers may be performed by identifying unique identifiers for the shared layers. These unique identifiers may be used to retrieve files for the shared layers from a container repository.

[0120] The process compares layers between the images to identify layers that are the same between the images (stage 904). This comparison may be performed by comparing unique identifiers between layers within the images. Layers that have the same unique identifier between the images are shared layers.

[0121] The process builds a list of images and shared layers (step 906). The process then terminates. In step 906, this list may be in the form of a data structure such as data structure 406 in Figure 4, and may be in the form of shared layer table 500 in Figure 5.

[0122] This process can be repeated if new images are added. In addition, images are removed from the list so that no automatic updates occur for those removed images.

[0123] 10, a flowchart of a process for detecting changes to shared layers between images is shown, according to an example embodiment. The process in FIG. 10 may be implemented in hardware, software, or both. If implemented in software, the process may take the form of program instructions executed by one or more processor units located in one or more hardware devices in one or more computer systems. For example, the process may be implemented in image manager 212 in computer system 210 in FIG. 2 and image manager 800 in FIG. 8.

[0124] The process begins by detecting an image change in an image that is being managed for automatic updates (step 1000). The process identifies shared layers in the image (step 1002). The process compares the shared layers with corresponding shared layers in other images to determine whether the shared layers are the same between the image with the change and the other images (step 1004).

[0125] Based on the comparison, the process determines whether any of the changes to the layers are to the set of shared layers in the image with the changes (step 1006). The process then ends.

[0126] 11, a flowchart of a process for monitoring a service port to detect changes to shared layers between images is shown, according to an example embodiment. The process in FIG. 11 may be implemented in hardware, software, or both. If implemented in software, the process may take the form of program instructions executed by one or more processor units located in one or more hardware devices in one or more computer systems. For example, the process may be implemented in image manager 212 in computer system 210 in FIG. 2 and image manager 800 in FIG. 8.

[0127] The process begins by detecting an image upload to the image server through a service port (step 1100). The process determines whether the image is an image subscribed for management (step 1102). If the updated image is not an image subscribed for management, the process ends.

[0128] If not, the process obtains information about layers for the uploaded image (step 1104). In step 1104, the layer information is layer context information obtained from one of multiple folders in which the image is stored. For example, these folders may reside in a container repository. A data structure containing shared layer information, such as shared layer table 500 in FIG. 5, may be used to determine which images have shared layers to be compared with each other.

[0129] The process compares each shared layer in the uploaded image with the corresponding shared layer in the other images to determine whether there are any changes to the shared layer in the uploaded image (stage 1106). This determination may be made by comparing the unique identifier in the shared layer in the uploaded image with the corresponding shared layer in the other images. The process then terminates.

[0130] 12, a flowchart of a process for updating shared layers between images is shown, according to an example embodiment. The process in FIG. 12 may be implemented in hardware, software, or both. If implemented in software, the process may take the form of program instructions executed by one or more processor units located in one or more hardware devices in one or more computer systems. For example, the process may be implemented in image manager 212 in computer system 210 in FIG. 2 and image manager 800 in FIG. 8.

[0131] The process begins by identifying an image that has a shared layer for update (step 1200). The process deletes the unique identifier for the shared layer in the image to be updated (step 1202). The process replaces the deleted unique identifier with a new unique identifier for the detected updated shared layer (step 1204). The process then ends.

[0132] In this illustrative example, when a shared layer is updated, the updated shared layer may be stored in a repository, such as a container repository. A new unique identifier is created for the updated shared layer. This new unique identifier is used to replace the previous one, so that the image now points to the updated shared layer instead of the old shared layer without the update.

[0133] Only a single file is needed for a shared layer that is used in multiple images. The unique identifier is used to fetch the shared layer when the image is ready to be run to create a container. As a result, a file containing the code for the updated shared layer can be fetched for use in the image, rather than fetching it and then patching the file before running the image.

[0134] Referring now to Figure 13, a flowchart of a process for updating a shared layer for an image is shown, according to an example embodiment. The process in Figure 13 may be implemented in hardware, software, or both. If implemented in software, the process may take the form of program instructions executed by one or more processor units located in one or more hardware devices in one or more computer systems. For example, the process may be implemented in image manager 212 in computer system 210 in Figure 2 and image manager 800 in Figure 8.

[0135] The process begins by determining shared layers that exist between images selected for update management, where the images include executable code that is executed to create a container (step 1300). The process detects changes to shared layers in shared layers for images in the images (step 1302). In response to detecting changes to shared layers for the images, the process updates shared layers in shared layers for the set of images that have shared layers (step 1304). The process then terminates.

[0136] Referring now to Figure 14, a flowchart of a process for deleting a selected image is shown, according to an exemplary embodiment. The process shown in Figure 14 is an example of additional steps used in conjunction with the steps in Figure 13. The process deletes the selected image from the images identified for management in response to a user input to stop automatic updates for the selected image (step 1400). The process then terminates.

[0137] Referring to Figure 15, a flowchart of a process for determining whether shared layers exist between images is shown, according to an example embodiment. The process shown in Figure 15 is an example of one implementation for step 1300 in Figure 13. The process uses layer context information for the images to determine whether shared layers exist between the images selected for update management (step 1500). The process then ends.

[0138] 16, a flowchart of a process for determining whether shared layers exist between images is shown, in accordance with an example embodiment. The process shown in FIG. 16 is an example of one implementation for stage 1300 in FIG. 13.

[0139] The process determines the shared layers that exist between the images selected for update management using the commands used to build the images, step 1600. The process then ends.

[0140] 17, a flowchart of a process for detecting changes to a shared layer is shown, according to an example embodiment. The process shown in FIG. 17 is an example of one implementation for stage 1302 in FIG. 13.

[0141] The process begins by detecting a change to an image using image information for the image that indicates the image has been changed (step 1700). In response to detecting the change to the image, the process determines that a shared layer for the image has changed using layer context information for the image that indicates the shared layer has been changed (step 1702). The process then ends.

[0142] 18, a flowchart of a process for detecting a command to update an image is shown, in accordance with an example embodiment. The process shown in FIG. 18 is an example of one implementation for stage 1302 in FIG. 13.

[0143] The process begins by detecting a command to update the image (step 1800). In response to detecting the command to update the image, the process determines that a shared layer for the image has changed based on a command for a layer for the image indicating that a shared layer within the layer has changed (step 1802). The process then ends.

[0144] 19, a flowchart of a process for updating a shared layer is shown, according to an example embodiment. The process shown in FIG. 19 is an example of one implementation for stage 1304 in FIG. 13.

[0145] The process begins by determining a new unique identifier for a shared layer for a modified image (step 1900). The process identifies a set of images that have a modified shared layer (step 1902). The process updates the shared layer for the set of images with the new unique identifier for the modified shared layer (step 1904). The process then terminates.

[0146] The flowcharts and block diagrams in the different illustrated embodiments illustrate the architecture, functionality, and operation of some possible implementations of the apparatus and methods in the example embodiments. In this regard, each block in the flowcharts or block diagrams may represent at least one of a module, segment, function, or portion of an operation or stage. For example, one or more of these blocks may be implemented as program instructions, hardware, or a combination of program instructions and hardware. If implemented in hardware, the hardware may take the form of, for example, an integrated circuit that is manufactured or configured to perform one or more operations in the flowcharts or block diagrams. If implemented as a combination of program instructions and hardware, the implementation may take the form of firmware. Each block in the flowcharts or block diagrams may be implemented using dedicated hardware and dedicated hardware systems that perform different operations or combinations of program instructions executed by the dedicated hardware.

[0147] In some alternative implementations of the exemplary embodiments, one or more functions noted in the blocks may occur in an order other than that noted in the figures. For example, in some cases, two blocks shown in succession may be executed substantially concurrently, or the blocks may even be executed in the reverse order, depending on the functionality involved. Also, other blocks may be added in addition to the blocks shown in a flowchart or block diagram.

[0148] Referring now to Figure 20, a block diagram of a data processing system is shown in accordance with an illustrative embodiment. Data processing system 2000 may be used to implement the computers and computing devices in computing environment 100 in Figure 1. Data processing system 2000 may also be used to implement computer system 210 in Figure 2. In this illustrative example, data processing system 2000 includes a communications framework 2002 that provides communications between a processor unit 2004, a memory 2006, persistent storage 2008, a communications unit 2010, an input / output (I / O) unit 2012, and a display 2014. In this example, communications framework 2002 takes the form of a bus system.

[0149] The processor unit 2004 functions to execute instructions for software loaded into the memory 2006. The processor unit 2004 includes one or more processors. For example, the processor unit 2004 may be selected from at least one of a multi-core processor, a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a network processor, or any other suitable type of processor. Furthermore, the processor unit 2004 may be implemented using one or more heterogeneous processor systems in which a primary processor resides on a single chip along with secondary processors. As another illustrative example, the processor unit 2004 may be a symmetric multiprocessor system including multiple processors of the same type on a single chip.

[0150] Memory 2006 and persistent storage 2008 are examples of storage devices 2016. A storage device is any number of pieces of hardware capable of temporarily, permanently, or both temporarily and permanently storing information such as, for example, but not limited to, data, program instructions in a functional form, or other suitable information. In these illustrative examples, storage devices 2016 may also be referred to as computer-readable storage devices. Memory 2006 in these examples may be, for example, random access memory or any other suitable volatile or non-volatile storage device. Persistent storage 2008 may take various forms, depending on the particular implementation.

[0151] For example, persistent storage 2008 may include one or more components or devices. For example, persistent storage 2008 may be a hard drive, a solid-state drive (SSD), a flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above. The media used by persistent storage 2008 may also be removable. For example, a removable hard drive may be used for persistent storage 2008.

[0152] The communications unit 2010 in these illustrative examples provides for communication with other data processing systems or devices. In these illustrative examples, the communications unit 2010 is a network interface card.

[0153] Input / output unit 2012 allows for the input and output of data with other devices connected to data processing system 2000. For example, input / output unit 2012 may provide a connection for user input through at least one of a keyboard, a mouse, or some other suitable input device. Further, input / output unit 2012 may send output to a printer. Display 2014 provides a mechanism for displaying information to a user.

[0154] Instructions for at least one of the operating system, applications, or programs may be located in storage devices 2016, which are in communication with processor unit 2004 through communications framework 2002. The processes of the different embodiments may be executed by processor unit 2004 using computer-implemented instructions that may be located in a memory, such as memory 2006.

[0155] These instructions are referred to as program instructions, computer-usable program instructions, or computer-readable program instructions, which are read and executed by a processor in processor unit 2004. The program instructions in different embodiments may be embodied on different physical or computer-readable storage media, such as memory 2006 or persistent storage 2008.

[0156] The program instructions 2018 are located in a functional form on a computer-readable medium 2020 that is selectively removable and that may be loaded onto or transferred to the data processing system 2000 for execution by the processor unit 2004. In these illustrative examples, the program instructions 2018 and the computer-readable medium 2020 form a computer program product 2022. In the illustrative example, the computer-readable medium 2020 is a computer-readable storage medium 2024.

[0157] The computer readable storage medium 2024 is not a medium that propagates or transmits the program instructions 2018, but rather a physical or tangible storage device used to store the program instructions 2018. As used herein, the computer readable storage medium 2024 is not to be construed as a transitory signal per se, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0158] Alternatively, the program instructions 2018 may be transferred to the data processing system 2000 using a computer-readable signal medium. The computer-readable signal medium may be a signal, such as a propagated data signal containing the program instructions 2018. For example, the computer-readable signal medium may be at least one of an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals may be transmitted over a connection, such as a wireless connection, fiber optic cable, coaxial cable, a wire, or any other suitable type of connection.

[0159] Additionally, as used herein, "computer-readable medium 2020" may be singular or plural. For example, the program instructions 2018 may be located in computer-readable medium 2020 in the form of a single storage device or system. In another example, the program instructions 2018 may be located in computer-readable medium 2020 distributed across multiple data processing systems. In other words, some instructions in the program instructions 2018 may be located in one data processing system, while other instructions in the program instructions 2018 may be located in a single data processing system. For example, some portion of the program instructions 2018 may be located in computer-readable medium 2020 in a server computer, while another portion of the program instructions 2018 may be located in computer-readable medium 2020 located in a set of client computers.

[0160] The different components illustrated for data processing system 2000 are not meant to provide architectural limitations to the manner in which different embodiments may be implemented. In some illustrative examples, one or more of the components may be incorporated into or otherwise form a part of another component. For example, memory 2006, or portions thereof, may be incorporated into processor unit 2004 in some illustrative examples. The different illustrative embodiments may be implemented in a data processing system including components in addition to or instead of those illustrated for data processing system 2000. Other components illustrated in FIG. 20 may be varied from the illustrated illustrative example. The different embodiments may be implemented using any hardware device or system capable of executing program instructions 2018.

[0161] Thus, exemplary embodiments of the present invention provide computer-implemented methods, apparatus, systems, and computer program products for managing image updates. According to one exemplary embodiment, a computer-implemented method manages image updates. A computer system determines shared layers that exist between images selected for update management. The images include executable code that is executed to create a container. The computer system detects changes to shared layers in shared layers for images in the images. In response to detecting the changes to the shared layers for the images, the computer system updates shared layers in shared layers in the set of images having shared layers. According to other exemplary embodiments, computer systems and computer program products for managing image updates are provided. As a result, the exemplary embodiments may provide the technical effect of improving performance by reducing errors in image updates and improving image update performance by eliminating the need to individually update code in each image.

[0162] In an illustrative example, errors are reduced through automated updates of shared layers rather than patching images individually. Additionally, performance is improved through replacing pointers, such as unique identifiers, to shared layers in multiple images rather than applying patches or copying patched files into images. As a result of using a new unique identifier for an updated shared layer in images that have a shared layer, the images share the updated layer without having to patch the file individually for each image. In an illustrative example, an image manager does not store files for shared layers. Instead, updates to shared layers are made by updating a unique identifier that points to the location of the shared layer. As a result, an updated shared layer can be pulled from a repository using the updated shared layer's new unique identifier, as opposed to patching individual files in an image after the files have been retrieved for use in creating a container.

[0163] Additionally, errors are reduced and performance is improved in pipeline workloads that may have hundreds or thousands of stages, where each stage is performed by a container formed from running files in an image.

[0164] The descriptions of different illustrative embodiments are presented for purposes of illustration and description and are not intended to be exhaustive or limited to the embodiments in the form disclosed. The different illustrative examples describe components that perform actions or operations. In the illustrative embodiments, the components may be configured to perform the described actions or operations. For example, the components may have a configuration or design for the structure that provides the components with the ability to perform the actions or operations described in the illustrative examples as being performed by the components. Furthermore, to the extent the terms "includes," "including," "has," "contains," and variations thereof are used herein, such terms are intended to be inclusive in a manner similar to the term "comprises" as an open-ended transition, without excluding any additional or other elements.

[0165] The description of various embodiments of the present invention has been presented for illustrative purposes and is not intended to be comprehensive or limited to the disclosed embodiments. Not all embodiments include all of the features described in the illustrative examples. Furthermore, different exemplary embodiments may provide different features compared to other exemplary embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been selected to best explain the principles of the embodiments, practical applications, or technical improvements over technologies found in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. 1. A computer-implemented method for managing image updates, comprising: determining, by the computer system, shared layers that exist between the images selected for update management, wherein the images include executable code that is executed to create a container; the computer system detecting a change in the shared layer for an image in the shared layer; updating the shared layer in the shared layer for the set of images having the shared layer in response to the computer system detecting the change to the shared layer for the image. A computer-implemented method comprising:

2. the computer system deleting the selected image from the images identified for management in response to a user input to stop automatic updates for the selected image. The computer-implemented method of claim 1 further comprising:

3. The step of the computer system determining the shared layers that exist between the images selected for the update management includes: the computer system using layer context information for the images to determine the shared layers that exist between the images selected for the update management. The computer-implemented method of claim 1 , comprising:

4. The step of the computer system determining the shared layers that exist between the images selected for the update management includes: the computer system determining the shared layers that exist between the images selected for update management using commands used to build the images. The computer-implemented method of claim 1 , comprising:

5. The step of the computer system detecting the change of the shared layer in the shared layer for the image in the image includes: the computer system detecting changes to the image using image information about the image that indicates the image has been changed; determining, in response to the computer system detecting the change to the image, that the shared layer for the image has changed using layer context information for the image that indicates that the shared layer has changed. The computer-implemented method of claim 1 , comprising:

6. The step of the computer system detecting the change of the shared layer in the shared layer for the image in the image includes: detecting a command to update the image by the computer system; determining, in response to the computer system detecting the command to update the image, that the shared layer for the image has changed based on the command for a layer for the image indicating that the shared layer within the layer has changed. The computer-implemented method of claim 1 , comprising:

7. In response to the computer system detecting the change to the shared layer for the image, updating the shared layer in the shared layers in the set of images having the shared layer includes: the computer system determining a new unique identifier for the shared layer for the modified image; the computer system identifying the set of images having the shared layer modified; the computer system updating the shared layer for the set of images with the new unique identifier for the modified shared layer. The computer-implemented method of claim 1 , comprising:

8. The computer-implemented method of claim 1 , wherein the images form an artificial intelligence pipeline, and the images are for the containers that perform stages in the artificial intelligence pipeline.

9. Multiple Processing Units 1. A computer system comprising: determining shared layers that exist between images selected for update management, where the images include executable code that is executed to create a container; detecting a change in a shared layer in the shared layer for an image in the image; In response to detecting the change to the shared layer for the image, updating the shared layer in the shared layers for the set of images having the shared layer. A computer system that executes program instructions for

10. The plurality of processor units: In response to a user input to stop automatic updates for the selected image, removing the selected image from the images identified for management.

10. The computer system of claim 9, wherein the computer system executes program instructions for:

11. In the step of determining the shared layers that exist between the images selected for the update management, the plurality of processor units: Using layer context information for the images, determine the shared layers that exist between the images selected for the update management.

10. The computer system of claim 9, wherein the computer system executes program instructions for:

12. In the step of determining the shared layers that exist between the images selected for the update management, the plurality of processor units: Using the commands used to build the images, determine the shared layers that exist between the images selected for update management.

10. The computer system of claim 9, wherein the computer system executes program instructions for:

13. In the step of detecting the change in the shared layer for the image in the image, the plurality of processor units: detecting changes to the image using image information about the image that indicates the image has been changed; In response to detecting the change to the image, determining that the shared layer for the image has changed using layer context information for the image that indicates that the shared layer has changed.

10. The computer system of claim 9, wherein the computer system executes program instructions for:

14. In the step of detecting the change in the shared layer for the image in the image, the plurality of processor units: Detecting a command to update the image; In response to detecting the command to update the image, determining that the shared layer for the image has changed based on the command for a layer for the image indicating that the shared layer within the layer has changed.

10. The computer system of claim 9, wherein the computer system executes program instructions for:

15. In response to detecting the change to the shared layer for the image, updating the shared layer in the shared layers in the set of images having the shared layer, the plurality of processor units: determining a new unique identifier for the shared layer for the modified image; identifying the set of images having the shared layer modified; updating the shared layer for the set of images with the new unique identifier for the modified shared layer; 10. The computer system of claim 9, wherein the computer system executes program instructions for:

16. 10. The computer system of claim 9, wherein the images form an artificial intelligence pipeline, the images being for the containers that execute stages in the artificial intelligence pipeline.

17. 1. A computer program product for managing image updates, comprising: a computer-readable storage medium having program instructions embodied thereon, the program instructions configuring a computer system to: the computer system determining shared layers that exist between the images selected for update management, where the images include executable code that is executed to create a container; the computer system detecting a change in the shared layer for an image in the shared layer; and updating the shared layer in the shared layer for the set of images having the shared layer in response to the computer system detecting the change to the shared layer for the image. a computer program product executable by said computer system to cause said computer system to perform the method of

18. the computer system deleting the selected image from the images identified for management in response to a user input to stop automatic updates for the selected image.

20. The computer program product of claim 17, further comprising:

19. The computer system determines the shared layers that exist between the images selected for update management by: the computer system using layer context information about the images to determine the shared layers that exist between the images selected for update management.

20. The computer program product of claim 17, comprising:

20. The computer system determines the shared layers that exist between the images selected for updating by: The computer system determines the shared layers that exist between the images selected for update management using commands used to build the images.

20. The computer program product of claim 17, comprising:

Images