IMAGE OPTIMIZATION FOR PIPELINE WORKLOADS

Abstract

A computer-executed method, apparatus, system, and computer program product manages updates of images. A computer system determines shared layers that exist between the images selected for update management. The images include executable code that executes to create containers. The computer system detects a change in a shared layer in the shared layers for an image in the images. The computer system updates the shared layer in the shared layers in a set of the images that have the shared layer in response to detecting the change to the shared layer for the image. According to further illustrative embodiments, a computer system and a computer program product for managing updates of images are provided.

Description

BACKGROUND1. Area:

[0001] The disclosure generally relates to an improved computer system, and more particularly to optimizing pipeline workloads. More specifically, the present disclosure relates to a computer-executed method, apparatus, system, and computer program product for managing image optimization for pipeline workloads. 2. Description of related technology:

[0002] A container orchestration environment such as Kubernetes ®(a registered trademark of the Linux Foundation of San Francisco, California), provides a platform for automating the deployment, scaling, and operation of containers across clusters of host nodes. A host node is a machine, either physical or virtual, on which containers are deployed. A pod is a group of one or more containers with shared storage and network resources and a specification for how the containers should run. The containers can run workloads. Images represent the executable code that runs to build the containers.

[0003] Pipeline workloads are often run in container orchestration environments. A pipeline workload can be any type of workload. For example, the workload can be AI processing, training a machine learning model, natural language processing, image processing, computer vision, scientific computing, forecasting, prediction, recommendation, data processing, transaction processing, and the like.

[0004] A pipeline workload divides the workload into many steps. These steps can be processed in a sequence to produce a result for the pipeline workload. In a container orchestration environment, containers are used to execute these steps. Each container serves a step in the steps performed by the pipeline workload. For example, pipeline workloads are used with container processing, where the workload is divided into steps that are executed in a sequence to complete the process.

[0005] In a pipeline workload, there is an image for each step. When there are hundreds or thousands of steps, managing this number of images can be difficult. For example, if an operating system-level patch is applied to several images, these images are currently patched one after the other. This process is complex and error-prone. For example, there are 1,000 images, and 800 of the images use a first Linux version and 200 of the images use a second Linux version. If the first Linux version requires a patch, the user currently repeats the process of building the image with the patch and uploading the image to a Docker server 800 times. This process is tedious, time-consuming, and error-prone.

[0006] One solution involves handling patch requests using scripts. The user creates a build with the script, which contains the build and upload logic. The script is executed for each image. However, using scripts can still lead to errors because some images may have already been modified with the initial Linux release, while some images may not have been modified with the initial Linux release with an update. Even with scripts, a collection of image information still needs to be manually collected by a user regarding which images require changes. For example, the user checks all images to indicate which images require changes and which have already been updated. Maintaining this list is tedious, time-consuming, and can be error-prone.

[0007] As another example, the user can include a script in the image. Once the image is loaded to run the container, the script is executed before the image is executed to build the container. The script finds the fix and applies the fix to the files containing the code for the container before the container is executed. This process of obtaining a fix and fixing the files in the image before the container is built consumes resources such as time and bandwidth. This approach has a performance issue that becomes more severe as the number of images that have scripts applying fixes increases. When there are thousands of images for a pipelined workload, the performance issue can be severe.

[0008] It would therefore be desirable to have a method and apparatus that address at least some of the problems discussed above, as well as other potential problems. For example, it would be desirable to have a method and apparatus that overcomes a technical problem with image management. SUMMARY

[0009] According to a single illustrative embodiment, a method executed by a computer manages updates to images. A computer system determines shared layers that exist between the images selected for update management. The images include executable code that executes to create containers. The computer system detects a change in a shared layer in the shared layers for an image in the images. The computer system updates the shared layer in the shared layers in a set of the images that have the shared layer in response to detecting the change to the shared layer for the image. According to further illustrative embodiments, a computer system and a computer program product for managing updates to images are provided.Thus, the illustrative embodiments may provide a technical effect of increasing performance by reducing errors in updated images and increasing performance in updating images by eliminating a need to update code in each image individually.

[0010] The illustrative embodiments may permissibly update the shared layer in the shared layers in the set of images that have the shared layer in response to detecting the change to the shared layer for the image by determining a new unique identifier for the shared layer for the image that has changed; indicate the set of images that have the shared layer that has changed; and update the shared layer for the set of images using the new unique identifier for the shared layer that has changed.Thus, the illustrative embodiments may provide a technical effect of increasing performance by reducing errors in updated images and increasing performance in updating images by eliminating a need to update code in each image individually. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram of a computing environment according to an illustrative embodiment; Fig. 2 is a block diagram of an image environment according to an illustrative embodiment; Fig. 3 is a diagram of an image that may be managed by an image manager according to an illustrative embodiment; Fig. 4 is a block diagram illustrating determining shared layers present between images according to an illustrative embodiment; Fig. 5 is a representation of a table indicating shared layers according to an illustrative embodiment; Fig. 6 is a block diagram illustrating detection of a change in a shared layer according to an illustrative embodiment; Fig. 7 is a block diagram illustrating updating shared layers according to an illustrative embodiment; Fig. 8 is a diagram of an image manager for images used in a pipeline workload according to an illustrative embodiment; Fig. 9 is a flowchart of a process for specifying shared layers in images selected for automatic updates, according to an illustrative embodiment; Fig. 10 is a flowchart of a process for detecting changes to shared layers between images according to an illustrative embodiment; Fig. 11 is a flowchart of a process for monitoring a service port to detect changes to shared layers between images, according to an illustrative embodiment; Fig. 12 is a flowchart of a process for updating shared layers between images according to an illustrative embodiment; Fig. 13 is a flowchart of a process for updating a shared layer for images according to an illustrative embodiment; Fig. 14 is a flowchart of a process for removing a selected image according to an illustrative embodiment; Fig. 15 is a flowchart of a process for determining the presence of shared layers between images according to an illustrative embodiment; Fig. 16 is a flowchart of a process for determining the presence of shared layers between images according to an illustrative embodiment; Fig. 17 is a flowchart of a process for detecting a change to a shared layer according to an illustrative embodiment; Fig. 18 is a flowchart of a process for detecting commands to update an image according to an illustrative embodiment; Fig. 19 is a flowchart of a process for updating a shared layer according to an illustrative embodiment; and Fig. 20 is a block diagram of a data processing system according to an illustrative embodiment. DETAILED DESCRIPTION

[0011] Various aspects of the present disclosure are described by descriptive text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in computer program product (CPP) embodiments. With respect to all flowcharts, depending on the associated technology, the operations may be performed in a different order than that shown in a particular flowchart. For example, again depending on the associated technology, two operations shown in consecutive blocks of flowcharts may be performed in reverse order, as a single integrated step, concurrently, or in an at least partially temporally overlapping manner.

[0012] A computer program product embodiment ("CPP embodiment" or "CPP") is a term used in the present disclosure to describe any set of one or more storage media (also referred to as "media") collectively included in a set of one or more storage units, which collectively contain machine-readable code corresponding to instructions and / or data for performing computer operations specified in a particular CPP claim. A "storage unit" is any tangible unit capable of retaining and storing instructions for use by a computer processor.Without limitation, the computer-readable storage medium may be an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some common types of storage devices incorporating these media include: floppy disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), DVD (digital versatile disc), memory stick, floppy disk, a mechanically encoded device (such as punched cards or pits / ridges formed in a large surface of a disk), or any suitable combination of the foregoing.A computer-readable storage medium, as used in this disclosure, should not be construed as storing transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, light pulses traveling through an optical fiber, electrical signals transmitted through a wire, and / or other transmission media. As one of ordinary skill in the art will understand, data is typically moved from time to time during normal operations of a storage device, such as during access, defragmentation, or garbage collection, but this does not make the storage device volatile because the data is non-volatile while stored.

[0013] With reference to the figures, in particular with reference to Fig. 1, a block diagram of a computing environment is shown according to an illustrative embodiment. Computing environment 100 includes an example of an environment for executing at least a portion of the computer code involved in performing the inventive methods, such as enhanced image manager code 190. In this illustrative example, enhanced image manager code 190 is for managing images. For example, images or the workload of pipelines may be managed by enhanced image manager code 190. In the illustrative example, enhanced image manager code 190 may detect that a shared layer in one image is updated and automatically update shared layers corresponding to that shared layer in other images that include the shared layers.In addition to the enhanced image manager code 190, the computing environment 100 includes, for example, a computer 101, a wide area network (WAN) 102, an end user device (EUD) 103, a remote server 104, a public cloud 105, and a private cloud 106. In this embodiment, the computer 101 includes a processor set 110 (including processing circuitry 120 and a cache 121), a transmission fabric 111, a volatile main memory 112, a persistent memory 113 (including an operating system 122 and enhanced image manager code 190, as noted above), a peripheral device set 114 (including a user interface (UI) device set 123, a memory 124, and an Internet of Things (IoT) sensor set 125) and a network module 115. The remote server 104 includes a remote database 130.The public cloud 105 includes a gateway 140, a cloud orchestration module 141, a host physical machine set 142, a virtual machine set 143, and a container set 144.

[0014] The COMPUTER 101 may take the form of a desktop computer, a laptop computer, a tablet computer, a smartphone, a smartwatch, or other computer attached to a user's body, a mainframe computer, a quantum computer, or any other form of computer or mobile device now known or to be developed in the future that can execute a program, access a network, or query a database such as the remote database 130. As is generally known in computer technology, and depending on the technology, the performance of a method performed by a computer may be distributed among multiple computers and / or between multiple locations. On the other hand, in this illustration of the computing environment 100, the detailed explanation focuses on a single computer, specifically the computer 101, in order to keep the illustration as simple as possible.The computer 101 may be located in a cloud, although it may be in . Fig. 1 is not shown in a cloud. On the other hand, it is not required that the computer 101 be located in a cloud, except to the extent that may be expressly stated.

[0015] Processor set 110 includes one or more computer processors of any type now known or later developed. Processing circuitry 120 may be distributed across multiple packages, for example, multiple coordinated integrated circuit chips. Processing circuitry 120 may execute multiple processor threads and / or multiple processor cores. Cache 121 is main memory located within the processor chip package(s) and is typically used for data or code that should be available for rapid access by the threads or cores executing on processor set 110. Caches are typically organized into multiple levels depending on their relative proximity to the processing circuitry. Alternatively, some or all of the cache for the processor set may be located "off-chip."In some computing environments, the processor set 110 may be designed to operate on qubits and perform quantum computing.

[0016] Computer-readable program instructions are typically loaded onto computer 101 to cause a series of process steps to be performed by processor set 110 of computer 101, thereby effecting a computer-executed method, such that the instructions thus executed instantiate the methods specified in flowcharts and / or narrative descriptions of computer-executed methods included in this document (collectively, "the inventive methods"). These computer-readable program instructions are stored in various types of computer-readable storage media, such as cache 121 and the other storage media discussed below.The program instructions and associated data are accessed by a processor set 110 to control and direct the execution of the inventive methods. In the data processing environment 100, at least some of the instructions for performing the inventive methods may be stored in the enhanced image manager code 190 in persistent memory 113.

[0017] The transmission structure 111 is the signal transmission path that enables the various components of the computer 101 to exchange data with each other. Typically, this structure consists of switches and electrically conductive paths, such as the switches and electrically conductive paths that form buses, bridges, physical input / output ports, and the like. Other types of signal transmission paths may be used, such as fiber optic transmission paths and / or wireless transmission paths.

[0018] Volatile main memory 112 is any type of volatile main memory now known or developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile main memory 112 is characterized by random access, but this is only required if explicitly stated. In computer 101, volatile main memory 112 is located in a single package and within computer 101; however, alternatively or additionally, volatile memory may be distributed across multiple packages and / or external to computer 101.

[0019] Persistent memory 113 is any form of non-volatile computer memory known now or developed in the future. The non-volatility of this memory means that the stored data is retained regardless of whether power is applied to computer 101 and / or power is applied directly to persistent memory 113. Persistent memory 113 may be read-only memory (ROM), but typically at least a portion of persistent memory allows data to be written, erased, and rewritten. Some known forms of persistent memory include magnetic disks and semiconductor memory devices. Operating system 122 may take several forms, such as various well-known proprietary operating systems or open-source Portable Operating System Interface-type operating systems that utilize a kernel.The code included in the improved image manager code 190 typically includes at least a portion of the computer code involved in carrying out the methods of the invention.

[0020] The peripheral unit set 114 includes the set of peripheral units of the computer 101. Data transmission connections between the peripheral units and the other components of the computer 101 can be implemented in various ways, such as Bluetooth connections, Near-Field Communication (NFC) connections, connections made using cables (such as Universal Serial Bus (USB) cables), insert connections (e.g., Secure Digital (SD) cards), connections made through local area networks, and even connections made through wide area networks such as the Internet. In various embodiments, the UI unit set 123 can include components such as a display, a speaker, a microphone, wearables (such as glasses and smartwatches), a keyboard, a mouse, a printer, a touchpad, game controllers, and haptic units.The memory 124 is external storage, such as an external hard drive, or pluggable storage, such as an SD card. The memory 124 can be persistent and / or volatile. In some embodiments, the memory 124 can take the form of a quantum computing memory device to store data in the form of qubits. In embodiments where the computer 101 needs to have a large storage capacity (for example, where the computer 101 locally stores and manages a large database), this memory can be provided by peripheral storage devices designed to store very large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The IoT sensor set 125 consists of sensors that can be used in Internet of Things applications.For example, one sensor can be a thermometer and another sensor can be a motion detector.

[0021] The network module 115 is the collection of computer software, hardware, and firmware that enables the computer 101 to communicate with other computers through the wide area network 102. The network module 115 may include hardware such as modems or Wi-Fi signal transceivers, software to packetize and / or depacketize data for transmission over the broadcast network, and / or web browser software to transmit data over the Internet. In some embodiments, the network control functions and network forwarding functions of the network module 115 are performed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control functions and forwarding functions of the network module 115 are performed on physically separate devices, such that the control functions manage multiple different network hardware devices.Computer-readable program instructions for performing the methods of the invention can typically be downloaded to the computer 101 from an external computer or external storage device through a network adapter card or network interface included in the network module 115.

[0022] The WAN 102 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances using any computer data transmission technology now known or developed in the future. In some embodiments, the WAN 102 may be replaced and / or supplemented by local area networks (LANs) configured to transmit data between devices located on a local area network, such as a Wi-Fi network. The WAN and / or LANs typically include computer hardware such as copper transmission cables, fiber optic transmission lines, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers.

[0023] The end user device (EUD) 103 is any computer system used and controlled by an end user (for example, a customer of a company operating the computer 101) and may take any of the forms discussed above in connection with the computer 101. The EUD 103 typically receives helpful and useful data from the operations of the computer 101. For example, in a hypothetical case where the computer 101 is configured to provide a recommendation to an end user, that recommendation would typically be transmitted from the network module 115 of the computer 101 through the WAN 102 to the EUD 103. In this way, the EUD 103 can display or otherwise present the recommendation to an end user.In some embodiments, the EUD 103 may be a client device, such as a thin client, a heavy client, a mainframe computer, a desktop computer, and so on.

[0024] The REMOTE SERVER 104 is any computer system that provides at least some data and / or functionality to the computer 101. The remote server 104 may be controlled and used by the same entity that operates the computer 101. The remote server 104 represents the machine(s) that collects and stores helpful and useful data for use by other computers, such as the computer 101. For example, in a hypothetical case where the computer 101 is designed and programmed to provide a recommendation based on historical data, this historical data may be provided to the computer 101 from the remote database 130 of the remote server 104.

[0025] The PUBLIC CLOUD 105 is any computer system available for use by multiple entities that provides on-demand availability of computer system resources and / or other computer functions, particularly data storage (cloud storage) and computing power, without direct active management by the user. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. The direct and active management of the computing resources of the public cloud 105 is performed by the computer hardware and / or software of the cloud orchestration module 141.The computing resources provided by the public cloud 105 are typically executed by virtual computing environments running on various computers that constitute the computers of the physical machine set 142 of the host, which represents the totality of the physical computers in the public cloud 105 and / or is available to the public cloud 105. The virtual computing environments (VCEs) typically take the form of virtual machines from the virtual machine set 143 and / or containers from the container set 144. It should be noted that these VCEs can be stored as images and transferred among and between the various hosts of the physical machines, either as images or after instantiation of the VCE.The cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of VCEs, and manages active instantiations of VCE deployments. The gateway 140 is the collection of computer software, hardware, and firmware that enables the public cloud 105 to exchange data through the WAN 102.

[0026] Virtualized computing environments (VCEs) will now be explained in more detail. VCEs can be stored as "images." A new active instance of the VCE can be instantiated from the image. Two common types of VCEs are virtual machines and containers. A container is a VCE that makes use of operating system-level virtualization. This refers to an operating system feature where the kernel allows the existence of multiple isolated user-space instances, called containers. From the perspective of programs running in them, these isolated user-space instances typically behave like real computers. A computer program running on a regular operating system can utilize all of that computer's resources, such as attached devices, files and folders, shared network space, CPU power, and quantifiable hardware features.However, programs running in a container can only use the contents of the container and the units assigned to the container, a feature known as containerization.

[0027] The PRIVATE CLOUD 106 is similar to the public cloud 105, except that the computing resources are available for use only by a single enterprise. While the private cloud 106 is depicted as being in communication with the WAN 102, in other embodiments, a private cloud may be completely disconnected from the Internet and accessible only through a local / private network. A hybrid cloud is a collection of multiple clouds of different types (e.g., private, community, or public cloud), often implemented by different providers accordingly. Each of the multiple clouds remains a separate and discrete entity; however, the architecture of the larger hybrid cloud is held together by standardized or proprietary technology that enables orchestration, management, and / or portability of data / applications between the multiple sub-clouds.In this embodiment, the public cloud 105 and the private cloud 106 are both part of a larger hybrid cloud.

[0028] The illustrative embodiments provide a method, apparatus, system, and computer program product for managing updates to images. A computer system determines shared layers that exist between the images selected for update management. The images include executable code that executes to create containers. The computer system detects a change in a shared layer in the shared layers for an image in the images. The computer system updates the shared layer in the shared layers in a set of images that have the shared layer in response to detecting the change to the shared layer for the image.One or more of the illustrative examples described herein may provide a technical effect of increasing performance by reducing errors when updating images and increasing performance when updating images by eliminating a need to update the code in each image individually.

[0029] As used herein, a "set of" when used with reference to items means one or more items. For example, a set of images means one or more images.

[0030] With reference to Fig. Figure 2 shows a block diagram of an image environment according to an illustrative embodiment. In this illustrative example, an image environment 200 includes components that may be implemented in hardware, such as the hardware implemented in the computing environment 100 in Fig. 1 is shown.

[0031] In this illustrative example, an image management system 202 manages images 204 in the image environment 200. As shown, the images 204 include executable code that is executed to create containers 206 from the images 204. In this example, the code in the images 204 is contained in binaries, libraries, and other types of files that can be executed.

[0032] In this illustrative example, images 204 may form a pipeline workload 205. In an illustrative example, the pipeline workload 205 formed using images 204 may be an AI pipeline 207. In this example, images 204 are for containers 206 that perform steps in AI pipeline 207. If AI pipeline 207 includes a machine learning model, steps such as retrieving data, creating a dataset, cleaning data, transforming data, training a model, evaluating a model, and other steps may be performed in AI pipeline 207. In addition, if the AI ​​pipeline 207 includes training the machine learning model, training may include steps such as splitting a dataset, selecting an algorithm, evaluating training data, and optimizing hyperparameters.

[0033] The containers 206 are containers running on a container orchestration platform 208, which may be, for example, a Kubernetes° architecture, environment, or the like. However, it should be understood that a description of illustrative examples using Kubernetes ® is intended merely as an example architecture and not as a limitation of illustrative embodiments. In this example, containers 206 are intended for workloads that are applications running on container orchestration platform 208.

[0034] As illustrated, the image management system 202 includes a computer system 210 and an image manager 212. The image manager 212 is located in the computer system 210.

[0035] The image manager 212 may be embodied in software, hardware, firmware, or a combination thereof. If software is used, the operations performed by the image manager 212 may be embodied in program instructions configured to execute on hardware, such as a processing unit. If firmware is used, the operations performed by the image manager 212 may be embodied in program instructions and data and stored in persistent memory for execution on a processing unit. If hardware is employed, the hardware may include circuitry operative to perform the operations in the image manager 212.

[0036] In the illustrative examples, the hardware may take a form selected from a circuit system, an integrated circuit, an application-specific integrated circuit (ASIC), a programmable logic unit, or other suitable type of hardware configured to perform a number of operations. For a programmable logic unit, the unit may be configured to perform the number of operations. The unit may be reconfigured at a later time or permanently configured to perform the number of operations. Programmable logic units include, for example, a programmable logic array, a programmable array logic, a field-programmable logic array, a field-programmable gate array, and other suitable hardware units.Furthermore, the processes can be carried out in organic components integrated with inorganic components, and they can consist entirely of organic components, excluding a human. For example, the processes can be carried out as circuits in organic semiconductors.

[0037] Computer system 210 is a physical hardware system and includes one or more data processing systems. If more than one data processing system is present in computer system 210, these data processing systems exchange data with each other using a transmission medium. The transmission medium may be a network. The data processing systems may be selected from a computer and / or a server computer and / or a tablet computer or another suitable data processing system.

[0038] As used herein, the phrase "and / or" when used with a list of items means that various combinations of one or more of the listed items may be used, and that only one of each item in the list may be required. In other words, "and / or" means that any combination of items and any number of items from the list may be used, but not all items in the list are required. The item can be a specific object, thing, or category.

[0039] For example, but not limited to, "Item A and / or Item B and / or Item C" may include Item A, Item A and Item B, or Item B. This example may also include Item A, Item B, and Item C, or Item B and Item C. Of course, any combination of these items may be present. In some illustrative examples, "and / or" may be, for example, but not limited to, two of Item A; one of Item B; and ten of Item C; four of Item B and seven of Item C; or other suitable combinations.

[0040] As shown, the computer system 210 includes a number of processor units 214 that can execute program instructions 216 that perform processes in the illustrative examples. As used herein, a processor unit in the number of processor units 214 is a hardware unit and consists of hardware circuitry, such as that on an integrated circuit, that responds to and processes instructions and program instructions that control a computer. When the number of processor units 214 execute program instructions 216 for a process, the number of processor units 214 is one or more processor units, which may be located on the same computer or on different computers. In other words, the process may be divided between processor units on the same or different computers in a computer system.

[0041] Furthermore, the number of processor units 214 may be of the same type or may be different types of processor units. For example, the number of processor units 214 may be selected from a single-core processor, a dual-core processor, a multi-core processor, a general-purpose central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), or another type of processor unit.

[0042] The image manager 212 can manage changes to images 204. In this illustrative example, the images 204 were selected for management with respect to shared layers 218 that exist between the images 204. Shared layers 218 in the images 204 means that these layers exist between all or a subset of the images 204. The image manager 212 identifies shared layers 218 that exist between the images 204 selected for update management.

[0043] In this example, the image manager 212 detects a change 219 in the shared layer 220 in the shared layers 218 in an image 222 in the images 204. The change 219 may result, for example, from a patch in the form of an operating system, a library update, or another change to the shared layer 218. The shared layer 220 with the change 219 becomes an updated shared layer 224.

[0044] In this example, the image manager 212 updates the shared layer 220 in the shared layers 218 in a set of images 204 that have the shared layer 220 in response to detecting the change to the shared layer 220 in the image 222.

[0045] Updating the set of images 204 that have the shared layer 220 may occur by including the shared layer 220 with the change 219 in the set of images 204 of the shared layer 220 without the change 219. In other words, executable code for the corresponding layer in the set of images 204 does not need to be corrected. Rather, the shared layer 220 with the change 219 may be used instead of the shared layer 220 without the change 219 that existed in the set of images 204 prior to the update performed by the image manager 212.

[0046] For example, images 204 include 600 images 204, and 350 of the images 204 may have shared layer 220. Consequently, if change 219 is made to shared layer 220 in one of the 350 images, the other 349 images may be updated to use shared layer 220 with change 219. This change to the other 349 images in images 204 is made without applying any patches to the actual code in files for shared layer 220. Rather, these layers may be updated to reference or point to the updated shared layer 224, which is shared layer 220 with change 219.Thus, the image manager 212 automatically updates the shared layers 218 for the images 204 in response to detecting changes in the shared layers 218.

[0047] Additionally, a selected image 226 in the images 204 may be removed from the images 204. This removal of the selected image 226 may be performed in response to a user input to pause automatic updates for the selected image 226. In response to the removal of the selected image 226 from the images 204, the image manager 212 no longer considers the selected image 226 for updates.

[0048] For example, if the shared layer 220 is an operating system, an updated version of the operating system may not work with other layers in the images 204. Because an image in which updates to a shared layer cause problems may be removed from the images 204, preventing automatic updates by the image manager 212 for the selected image 226 from being performed. These problems may include, for example, execution errors, performance degradation, or other issues.

[0049] In a single illustrative example, there are one or more technical solutions that overcome a technical problem with managing updates to images when there are a large number of images. Consequently, one or more technical solutions may provide a technical effect that manages images by specifying shared layers that are common between images, detecting changes in a shared layer, and updating the shared layers in the images corresponding to the updated common layer. In a single illustrative example, there may be a technical effect that reduces errors in updated images and increases performance by eliminating the need to update code in each image individually.

[0050] With a view to Fig. Figure 3 shows a representation of an image that may be managed by an image manager, according to an illustrative embodiment. In the illustrative examples, the same reference number may be used in more than one figure. This reuse of a reference number in different figures represents the same element in the different figures.

[0051] Examples of information that may be present in image 222 are shown in this figure. As shown, image 222 includes files 301, image information 302, and layer context information 303.

[0052] In this example, files 301 include executable code for layers 304 that can be executed to create the container for image 222. This executable code may be in the form of machine-readable code that can be executed by a processing unit in a computer, such as binary code and libraries.

[0053] The image information 302 is metadata that contains information about the image 222. In this example, the image information 302 may include the image name, owner, information about the file creation, size, creation time, and other information.

[0054] In this illustrative example, the layer context information 303 is metadata that contains information about the layers 304 in the files 301. For example, the layer context information 303 may be a list of unique identifiers (UIDs) 306 for the layers 304 for the image 222. Thus, the unique identifiers 306 are pointers to the layers 304 for the image 222 and can be used to obtain files 301 for the layers 304 if the files 301 for the layers 304 are not stored with the image 222. By using unique identifiers 306 for the layers 304, storage space for images can be reduced.

[0055] As shown in the illustrative examples, the files 301 for the layers 304 for the image 222 may be stored in a different storage location than that of the image information 302 and the layer context information 303 and referenced at that storage location by the layer context information 303. In this illustrative example, the image 222, when used to execute the images 204 managed by the image manager 212, may include image information 302 and layer context information 303 without the files 301. In other words, the image manager 212 stores the image information 302 and the layer context information 303, but not the files 301 for the layers 304. Consequently, the files 301 for the layers 304, including the shared layers 218, may be in Fig. 2 when the files 301 are needed to create the images 204 in Fig. 2 to execute.

[0056] When the image 222 is sent to a requesting user for use, the requesting user receives the image 222 with the image information 302 and the layer context information 303. The requesting user can obtain the files 301 for the layers 304 using the unique identifiers 306 and the layer context information 303. In this illustrative example, the unique identifiers 306 point to storage locations where the files 301 can be found. For example, the unique identifiers 306 point to a container repository containing files for the layers 304. With the retrieval of the files 301, the image 222 has a form that can be executed to create a container.

[0057] Next, with reference to Fig. Figure 4 shows a block diagram illustrating detecting shared layers present between images, according to an illustrative embodiment. Information for images 204 may be used to detect the presence of shared layers 218 between images 204.

[0058] The image manager 212 may determine shared layers 218 that exist between the images 204 selected for update management using context information 400 for the images 204. In this illustrative example, the determination of shared layers 218 that exist between the images 204 selected for update management may be performed by the image manager 212 using layer context information for the images 204.

[0059] For example, the image manager 212 may compare the unique identifiers (UIDs) 402 for the layers 404 for the images 204. The layers 404 between the images 204 that have matching unique identifiers form the shared layers 218 between those images. For example, a selected image in the images 204 may have a selected layer with a unique identifier UID1 that specifies the selected layer as a Linux operating system. Any layer in other images in the images 204 that has UID1 as a unique identifier is a shared layer for the selected layer.

[0060] Thus, the layers 404 for those images that have the unique identifier UID1 are a set of shared layers 218 within the layers 404. In other words, the layers 404 for the images 204 that have UID1 as a unique identifier are a set of shared layers 218.

[0061] Multiple sets of shared layers may exist between images 204. For example, another unique identifier UID2 in a layer in the selected image may be a unique identifier for a C library. A comparison of the unique identifier UD2 with the unique identifiers (UIDs) 402 for other images in images 204 may be used to indicate the use of the same layer for some or all of images 204 to form another set of shared layers 218. Thus, the layers 404 that have the unique identifier UID2 are one set of shared layers 218.

[0062] In this illustrative example, the image manager 212 captures information 405 about relationships between images 204 and shared layers 218 by determining the presence of shared layers 218 between images 204. The information 405 may be stored in a data structure 406. The data structure 406 may comprise a database, a table, a linked list, a flat file, or any other suitable type of data structure that can be used to store information 405 about shared layers 218 and images 204 that contain shared layers 218.

[0063] Next, with a view to Fig. 5 shows a representation of a table indicating shared layers, according to an illustrative embodiment. In this illustrative example, a shared layer table 500 is an example of the table in a database that can be used to implement the data structure 406 in Fig. 4 to execute.

[0064] As shown in the shared layers table 500, a layer column 501 indicates the name of the layer, a UID column 502 indicates the unique identifier for the layer, and a layer creation date 504 indicates the date the shared layer was created. This creation date in the shared layers table 500 can be compared to a creation date for a layer in an image being examined for changes. The comparison can be made to determine if a change has occurred between the time the shared layer was added to the shared layers table 500 and the time the image containing the shared layer is examined. Furthermore, an image name 505 in the shared layers table 500 indicates the images in which the layer with the unique identifier is present.

[0065] As shown in this simplified example, the shared layer at line 510 is Linux with a unique identifier of UIDA. This shared layer was created on February 16, 2022, at 5:18 AM. In this example, UIDA uniquely identifies the Linux shared layer in a repository. UIDA can take several forms, such as a hash number, an alphanumeric value, a Universal Resource Locator, or another value that uniquely distinguishes Linux from other layers. In this example, the images Image X, Image Y, and Image Z are images that have Linux as a layer with the unique identifier UIDA. The unique identifier can be used as an index to find images that have the shared layer.

[0066] As another example, row 512 in the shared layer table 500 has the name centos:7 with a unique identifier of UIDB. The creation date of this shared layer is 04 / 07 / 2022, 10:18 AM. Images that have this shared layer are Image A, Image C, and Image F. In other words, these three images all have the same shared layer.

[0067] In yet another example, line 514 represents a shared layer named Data Collection with a unique identifier of UIDC. The creation date of this shared layer is 11 / 06 / 2022, 5:21 PM. In this example, the images that have this shared layer are Image A and Image X.

[0068] In addition to specifying images that have shared layers, the shared layers table 500 can also be used to specify shared layers within an image. For example, an image name can be used to search for shared layers specific to the image with that image name in the shared layers table 500. As shown, image A has two shared layers with the unique identifiers UIDB and UIDC. Additionally, image X has two shared layers specified using the unique identifiers UIDA and UIDC. Image Y and image Z have a single shared layer with the unique identifier UIDA. Image C and image F have a single shared layer with the unique identifier UIDB.

[0069] A representation of the shared layers table 500 is provided as a simplified representation showing how information is determined from the specification of shared layers between images and how the information is stored and used by the image manager 212 in Fig. 2 can be used. In further illustrative examples, the shared layer table 500 may have hundreds or thousands of rows.

[0070] In this illustrative example, the image name in the image name column 505 uniquely distinguishes the image from other images. In further illustrative examples, an image identifier may be used in addition to the image name or instead of the image name. In further illustrative examples, a separate table may be used in which each image is represented by a row containing layer names and unique identifiers for layers that are shared layers in the image.

[0071] As another example, the shared layers table 500 may also contain additional information. For example, image creation dates may be stored for the image names. The creation dates may be used to determine whether a change has occurred in an image since the image was examined to specify shared layers for the shared layers table 500.

[0072] Next, with reference to Fig. 6 is a block diagram illustrating detecting a change in a shared layer, according to an illustrative embodiment. In this illustrative example, the image manager 212 may detect a change 219 in the shared layer 220 in the shared layers 218 in several different ways. For example, the image manager 212 may detect a local file change 606 to the shared layers 218 for the images 204 by checking the images 204 stored on the image server 602, monitoring a service port 604 for an upload of images 204 to the image server 602, or monitoring the local file change 606. In this example, the image server 602 may be a Docker server.

[0073] For example, the image manager 212 checks images 204 stored on the image server 602 for a changed image. For example, the image manager 212 may detect a change to the image 222 using image information 302 for the image 222, which indicates that the image 222 has changed. In this illustrative example, any updates or fixes to the image 222 are reflected in the image information 302. For example, a change to a layer in the image 222 causes the build time to be updated to the most recent build. This information obtained for the image 222 may be compared to image information stored for the image 222 in the data structure 406.

[0074] In response to indicating a change to image 222 using image information 302, image manager 212 may determine that shared layer 220 in image 222 has changed using layer context information 303 for image 222, which, in response to detecting the change to image 222, indicates that shared layer 220 for image 222 has changed. For example, UIDs in layer context information 303 for image 222 may be compared to stored layer context information for image 222. Layer context information 303 may be used for comparison and stored in data structure 406 in the form of one or more tables, such as shared layer table 500 in Fig. 5, are saved.

[0075] In another illustrative example, changes to images 204 may be monitored through service port 604. For example, when image 222 is uploaded to service port 604, detecting the upload of image 222 may cause image manager 212 to determine whether changes have occurred to image 222 and whether the changes include shared layers. In this example, images 204 are checked when images 204 are uploaded to image server 602 using service port 604.

[0076] In another example, the local file change 606 may be used to determine if a shared layer update for the image 222 has occurred. For example, the image manager 212 may detect commands 608 used as part of a local file change 606 to update the image 222. For example, the image manager 212 may detect commands 608 to update the image 222. The image manager 212 may determine that the shared layer 220 for the image 222 has changed based on commands 608 for layers for the image 222. In this illustrated example, in response to detecting the commands 608 to update the image 222, the commands may indicate that the shared layer 220 in layers 218 has changed.

[0077] For example, the commands 608 can be located in a Dockerfile 609 when images are deployed in a container orchestration platform such as Kubernetes ® These commands are executed to build image 222. A user can create Dockerfile 609, which contains commands 608 that include unique identifiers for the layers for image 222. The user can then run a Docker build command using Dockerfile 609, which contains commands 608.

[0078] In this illustrated example, the image manager 212 detects the execution of commands 608 in the Dockerfile 609 and can specify the layers referenced by the commands. The commands can use unique identifiers, which can then determine whether changes have occurred in the shared layers 218 for the image 222.

[0079] With a view to Fig. Figure 7 shows a block diagram illustrating updating shared layers, according to an illustrative embodiment. In this illustrative example, image manager 212 may update shared layer 220 in shared layers 218 in a set of images 204 that have shared layer 220. This update may be performed by image manager 212 in response to detecting the change to shared layer 220.

[0080] In this illustrative example, the image manager 212 determines that the shared layer 220 for the image 222 has changed and has a new unique identifier 700 instead of the previous unique identifier 702. In this example, the shared layer 220 is for the image 222 that has changed.

[0081] The determination of the previous unique identifier 702 may be made using a creation date for the shared layer 220 and information in the data structure 406, which is the shared layer table 500 in Fig. 5. For example, the creation dates for the shared layers 218 for the image 222 can be compared with the creation dates for those shared layers stored in the data structure 406. Unchanged shared layers in the shared layers 218 have the same creation date. A more recent creation date for the shared layer 220, when compared with the entry for the shared layer 220 in the data structure 406, indicates that the shared layer 220 has changed.

[0082] In this example, the unique identifier for shared layer 220 in data structure 406 is different from the new unique identifier 700 for shared layer 220. This unique identifier in data structure 406 for shared layer 220 is the previous unique identifier 702.

[0083] Furthermore, data structure 406 can be used to specify the shared layer 220 for other images in images 204. Thus, image manager 212 indicates the set of images 204 that have the shared layer 220 that has changed. The shared layers 218 for images 204 can be determined using data structure 406, which has the previous unique identifier 702 for shared layer 220 that has changed to the new unique identifier 700.

[0084] The image manager 212 updates the shared layer 220 for the set of images using the new unique identifier 700 for the shared layer that has changed. Updates can be made to change the previous unique identifier 702 to the new unique identifier 700 for the shared layer 220 for the images 204 specified using the data structure 406. The update can be performed in several different ways.

[0085] Consequently, the set of shared layers 218 in the set of images 204 is updated. This updating can be performed without having to apply patches to change code in files in the set of images 204.

[0086] The computer system 210 in Fig. 2 may be configured to perform at least one of the steps, operations, or actions described in the various illustrative examples using software, hardware, firmware, or a combination thereof. Consequently, computer system 210 operates as a special-purpose computer system, with image manager 212 within computer system 210 enabling the automatic management of images. In particular, image manager 212 transforms computer system 210 into a special-purpose computer system compared to currently available general-purpose computer systems that do not include image manager 212.

[0087] In the illustrative example, the use of image manager 212 in computer system 210 integrates processes into a practical application by managing updates to images for containers, which increases the performance of computer system 210. In other words, image manager 212 in computer system 210 is directed toward a practical application of processes integrated with image manager 212 in the computer system that specifies shared layers in images, detects changes to shared layers in the images, and updates shared layers with changes in the images.

[0088] The representation of the image environment 200 and the various components in the Fig. Figures 2 through 7 are not intended to imply any physical or architectural limitations on the manner in which an illustrative embodiment may be implemented. Other components besides or in place of those illustrated may be used. Some components may not be necessary. Also, the blocks are shown for illustrative purposes only. Some functional components are shown. One or more of these blocks may be combined, separated into different blocks, or combined and separated into different blocks when implemented in an illustrative embodiment.

[0089] For example, images 204 can be used to create containers 206 on one or more container orchestration platforms alongside or instead of container orchestration platform 208. As another example, in addition to shared layers table 500, another table can be used for data structure 406. For example, an image table can be used to specify images and shared layers for images. In other words, the image name can be indexed to specify shared layers that the image uses.

[0090] With reference to Fig. 8 shows a diagram of an image manager for images used in a pipeline workload according to an illustrative embodiment. In this illustrative example, image manager 800 is an example of an implementation of image manager 212 in Fig. 2. In this example, the Image Manager 800 manages an image X 802, an image Y 804, and an image Z 806. Each of these images can be executed to create containers for an AI pipeline in this example. Only three images are shown to illustrate the different aspects of how the Image Manager 800 can manage images. When executed, the Image Manager 800 can manage numerous additional images, such as hundreds or thousands of images.

[0091] In this example, image X 802, image Y 804, and image Z 806 have been selected for management by image manager 800. In this example, image X 802 has layers of data collection 810 and layers of Linux 812. Image Y 804 has layers of data cleansing 814 and layers of Linux 816, and image Z 806 has layers of module training 818 and layers of Linux 820.

[0092] In this illustrated example, the image manager 800 examines the images and identifies shared layers 822 that include Linux layers 812, Linux layers 816, and Linux layers 820. The shared layers in this example have the same unique identifiers. The image manager 800 creates a shared list 824 for these images and the shared layers 822.

[0093] The identification of the shared layers 822 can be determined using unique identifiers for the shared layers 822 located in layer context information for the images. In this illustrative example, layer context information can be obtained by examining the images or by identifying commands that create the images.

[0094] In this illustrative example, the image manager 800 may monitor the images to detect an update performed on one or more of the images. For example, the image manager 800 may detect a change to the shared layers 822 by checking images stored in the image server 830, monitoring the service port for an image upload, or monitoring a local file change.

[0095] For example, a user may make a local file change that corrects image X 802. This patch changes layers of Linux 812, resulting in a change in the unique identifiers for layers of Linux 812. In response to detecting this change, image manager 800 may update the unique identifier for layers of Linux 816 in image Y 804 and layers of Linux 820 for image Z 806. With this update, all shared layers 822 now have the same unique identifier. This update is also made to the shared list 824 by image manager 800 for future comparisons.

[0096] In this illustrative example, when images are updated, these updated images may be uploaded to other storage locations, such as the image server 830 in the cloud 832. The image server 830 may be a Docker server. Thus, the image manager 800 may manage complex image updates for a large number of images for a pipeline workload. These images for a pipeline workload may number in the hundreds of thousands. The image manager 800 may specify shared layers between images, monitor images for changes to shared layers, and update corresponding shared layers for other images in response to detecting a change to shared layers.

[0097] An image may be removed from the shared list 824 if automatic updates to the shared layers are not desired. For example, if an update to the shared layer in an image in the shared list 824 causes other layers to perform incorrectly or incorrectly, that image may be removed from the shared list 824. In this way, errors caused by current update techniques can be avoided and the quality of a pipeline workload can be increased.

[0098] Next, with a view to Fig. 9 is a flowchart of a process for specifying shared layers in images selected for automatic updates, according to an illustrative embodiment. The process in Fig. 9 may be executed in hardware, software, or both. When executed in software, the process may take the form of program instructions executed by one or more processor units located in one or more hardware units in one or more computer systems. For example, the process may be executed in image manager 212 in computer system 210 in Fig. 2 and in Image Manager 800 in Fig. 8 can be executed.

[0099] The process begins by specifying images for automatic update (step 900). The process specifies the shared layers in all images selected for automatic updates (step 902). In step 902, the specification of the shared layers can be performed by specifying the unique identifiers for the shared layers. These unique identifiers can be used to retrieve files for the shared layers from a container repository.

[0100] The process compares the layers between the images to indicate layers that are the same between images (step 904). This comparison can be performed by comparing unique identifiers between layers in the images. Layers that have the same unique identifiers between images are shared layers.

[0101] The process creates a list of images and shared layers (step 906). The process then ends. In step 906, this list can be stored in the form of a data structure, such as data structure 406 in Fig. 4 and in the form of the table 500 of the shared layers in Fig. 5 are present.

[0102] This process can be repeated if new images need to be added. Images can also be removed from the list so that automatic updates for these removed images are not performed.

[0103] With reference to Fig. 10 is a flowchart of a process for detecting changes to shared layers between images according to an illustrative embodiment. The process in Fig. 10 may be executed in hardware, software, or both. When executed in software, the process may take the form of program instructions executed by one or more processor units located in one or more hardware units in one or more computer systems. For example, the process may be executed in image manager 212 in computer system 210 in Fig. 2 and in Image Manager 800 in Fig. 8 can be executed.

[0104] The process begins by detecting a change in an image among the images managed for automatic updates (step 1000). The process identifies shared layers in the image (step 1002). The process compares the shared layers with corresponding shared layers in other images to determine if the shared layers are the same between the image with the change and the other images (step 1004).

[0105] The process determines whether any of the changes to the layers affect a set of shared layers in the image based on the comparison (step 1006). The process then exits.

[0106] With reference to Fig. Figure 11 is a flowchart of a process for monitoring a service port to detect changes to shared layers between images, according to an illustrative embodiment. The process in Fig. 11 may be executed in hardware, software, or both. When executed in software, the process may take the form of program instructions executed by one or more processor units located in one or more hardware units in one or more computer systems. For example, the process may be executed in image manager 212 in computer system 210 in Fig. 2 and in Image Manager 800 in Fig. 8 can be executed.

[0107] The process begins by detecting an image upload through a service port to an image server (step 1100). The process determines whether the image is a subscribed-for-management image (step 1102). If the updated image is not a subscribed-for-management image, the process ends.

[0108] Otherwise, the process obtains information about the layers for the updated image (step 1104). In step 1104, the layer information is layer context information, which can be obtained from a folder of folders in which images are stored. For example, these folders can be located in a container repository. A data structure containing information about shared layers, such as the shared layers table 500 in Fig. 5, can be used to determine which images have shared layers that should be compared.

[0109] The process compares each shared layer in the uploaded image with the corresponding shared layers in other images to determine if a change in a shared layer exists in the uploaded image (step 1106). This determination can be made by comparing the unique identifier in a shared layer in the uploaded image with corresponding shared layers in other images. The process then terminates.

[0110] With reference to Fig. Figure 12 is a flowchart of a process for updating shared layers between images according to an illustrative embodiment. The process in Fig. 12 may be executed in hardware, software, or both. When executed in software, the process may take the form of program instructions executed by one or more processor units located in one or more hardware units in one or more computer systems. For example, the process may be executed in image manager 212 in computer system 210 in Fig. 2 and in Image Manager 800 in Fig. 8 can be executed.

[0111] The process begins by specifying images with shared layers to update (step 1200). The process removes the unique identifier for the shared layer in the images to be updated (step 1202). The process replaces the removed unique identifier with a new unique identifier for the updated shared layer that was detected (step 1204). The process then ends.

[0112] In this illustrative example, when a shared layer is updated, the updated shared layer can be stored in a repository, such as a container repository. A new unique identifier is created for the updated shared layer. This new unique identifier can be used to replace the previous one, so that the image now points to the updated shared layer instead of the old, unupdated shared layer.

[0113] Only a single file is required for a shared layer used across multiple images. The unique identifier is used to retrieve this shared layer when the image is ready to run to create a container. Consequently, the file containing code for the updated shared layer can be retrieved for use in the image, rather than correcting the file after retrieval before the image is run.

[0114] With a view to Fig. 13 is a flowchart of a process for updating a shared layer for images according to an illustrative embodiment. The process in Fig. 13 may be executed in hardware, software, or both. When executed in software, the process may take the form of program instructions executed by one or more processor units located in one or more hardware units in one or more computer systems. For example, the process may be executed in image manager 212 in computer system 210 in Fig. 2 and in Image Manager 800 in Fig. 8 can be executed.

[0115] The process begins by identifying shared layers that exist between images selected for update management, where the images have executable code that is executed to create containers (step 1300). This process detects a change in a shared layer in the shared layers for an image in the images (step 1302). The process updates the shared layer in the shared layers for a set of the images that have the shared layer in response to detecting the change to the shared layer for the image (step 1304). The process then terminates.

[0116] With a view to Fig. 14 is a flowchart of a process for removing a selected image according to an illustrative embodiment. Fig. The process illustrated in Figure 14 is an example of an additional step that can be used with the steps in Fig. 13. The process removes a selected image from the images specified for management in response to a user input to stop automatic updates for the selected image (step 1400). The process then terminates.

[0117] With reference to Fig. 15 is a flowchart of a process for determining that shared layers exist between images, according to an illustrative embodiment. Fig. The process illustrated in Figure 15 is an example of a single execution for step 1300 in Fig. 13. The process determines the shared layers that exist between the images selected for update management using layer context information for the images (step 1500). The process then terminates.

[0118] Next, with a view to Fig. 16 is a flowchart of a process for determining that shared layers exist between images, according to an illustrative embodiment. Fig. The process illustrated in Figure 16 is an example of a single execution for step 1300 in Fig. 13.

[0119] The process determines the shared layers that exist between the images selected for update management using commands used to create the images (step 1600). The process then terminates.

[0120] With a view to Fig. 17 is a flowchart of a process for detecting a change to a shared layer according to an illustrative embodiment. Fig. The process illustrated in Figure 17 is an example of a single execution for step 1302 in Fig. 13.

[0121] The process begins by detecting a change to the image using image information for the image that indicates that the image has changed (step 1700). The process determines that the shared layer for the image has changed using layer context information for the image that indicates that the shared layer has changed in response to detecting the change to the image (step 1702). The process then ends.

[0122] With reference to Fig. Figure 18 is a flowchart of a process for detecting image update commands, according to an illustrative embodiment. Fig. The process illustrated in Figure 18 is an example of a single execution for step 1302 in Fig. 13.

[0123] The process begins by detecting image update commands (step 1800). The process determines that the shared layer for the image has changed based on the layer commands for the image indicating that the shared layer in the layers has changed in response to detecting the image update commands (step 1802). The process then ends.

[0124] With a view to Fig. 19 is a flowchart of a process for updating a shared layer according to an illustrative embodiment. Fig. The process illustrated in Figure 19 is an example of a single execution for step 1304 in Fig. 13.

[0125] The process begins by determining a new unique shared layer identifier for the image that has changed (step 1900). The process identifies the set of images that have the shared layer that has changed (step 1902). The process updates the shared layer for the set of images using the new unique shared layer identifier that has changed (step 1904). The process then ends.

[0126] The flowcharts and block diagrams in the various illustrated embodiments illustrate the architecture, functionality, and operation of some possible implementations of devices and methods in an illustrative embodiment. In this regard, each block in the flowcharts or block diagrams may represent a module and / or a segment and / or a function or part of an operation or step. For example, one or more of the blocks may be embodied as program instructions, hardware, or a combination of the program instructions and hardware. When embodied in hardware, the hardware may, for example, take the form of integrated circuits fabricated or configured to perform one or more operations in the flowcharts or block diagrams.When executed as a combination of program instructions and hardware, the execution may take the form of firmware. Each block in the flowcharts or block diagrams may be implemented using special-purpose hardware systems that perform the various operations or combinations of special-purpose hardware and program instructions executed by the special-purpose hardware.

[0127] In some alternative versions of an illustrative embodiment, the function(s) indicated in the blocks may occur in a different order than that indicated in the figures. For example, in some cases, two blocks shown in succession may be performed substantially concurrently, or the blocks may sometimes be performed in the reverse order depending on the corresponding functionality. Also, additional blocks may be added in addition to those illustrated in a flowchart or block diagram.

[0128] With a view to Fig. Figure 20 shows a block diagram of a data processing system according to an illustrative embodiment. The data processing system 2000 may be used to integrate computers and data processing devices in a data processing environment 100 into Fig. 1. The data processing system 2000 may also be used to execute the computer system 210 in Fig. 2. In this illustrative example, data processing system 2000 includes a transfer framework 2002 that provides transfers between a processor unit 2004, a main memory 2006, a persistent memory 2008, a transfer unit 2010, an input / output (I / O) unit 2012, and a display 2014. In this example, transfer framework 2002 takes the form of a bus system.

[0129] The processor unit 2004 is used to execute instructions for software that can be loaded into the main memory 2006. The processor unit 2004 includes one or more processors. For example, the processor unit 2004 may be selected from a multi-core processor and / or a central processing unit (CPU) and / or a graphics processing unit (GPU) and / or a physics accelerator (PPU), and / or a digital signal processor (DSP), and / or a network processor, or any other suitable type of processor. Furthermore, the processor unit 2004 may be implemented using one or more heterogeneous processor systems in which a main processor with secondary processors is present on a single chip.As another illustrative example, the processor unit 2004 may be a symmetric multiprocessor system that includes multiple processors of the same type on a single chip.

[0130] Main memory 2006 and persistent memory 2008 are examples of memory devices 2016. A memory device is a hardware component that can store information such as, but not limited to, data and / or program instructions in a functional form, or other suitable information, either temporarily, permanently, or both temporarily and permanently. Memory devices 2016 may also be referred to as computer-readable memory devices in these illustrative examples. Main memory 2006 in these examples may be, for example, random-access memory or any other suitable volatile or non-volatile memory device. Persistent memory 2008 may take various forms depending on the particular implementation.

[0131] For example, persistent storage 2008 may include one or more components or units. For example, persistent storage 2008 may be a hard disk drive, a solid-state drive (SSD), flash memory, a rewritable optical disk, a rewritable magnetic tape, or a combination of the foregoing. The storage media used by persistent storage 2008 may also be removable. For example, a removable hard disk drive may be used for persistent storage 2008.

[0132] The transmission unit 2010 in these illustrative examples provides for transmissions with other data processing systems or devices. In these illustrative examples, the transmission unit 2010 is a network interface card.

[0133] Input / output unit 2012 enables input and output of data with other devices that may be connected to data processing system 2000. For example, input / output unit 2012 may provide a connection for user input through a keyboard and / or a mouse or other suitable input device. Furthermore, input / output unit 2012 may send the output to a printer. Display screen 2014 provides a mechanism for displaying information to a user.

[0134] Instructions for the operating system and / or applications or programs may be located in the memory units 2016, which are in communication with the processor unit 2004 through the communication framework 2002. The processes of the various embodiments may be performed by the processor unit 2004 using computer-executed instructions, which may be located in a main memory, such as the main memory 2006.

[0135] These instructions are referred to as program instructions, computer-usable program instructions, or computer-readable program instructions that can be read and executed by a processor in the processing unit 2004. The program instructions in the various embodiments may be embodied on various physical or computer-readable storage media, such as the main memory 2006 or the persistent memory 2008.

[0136] The program instructions 2018 are located in operative form on a selectively removable computer-readable medium 2020, and can be loaded onto or transferred to the data processing system 2000 for execution by the processor unit 2004. The program instructions 2018 and the computer-readable medium 2020 constitute a computer program product 2022 in these illustrative examples. In the illustrative example, the computer-readable medium 2020 is a computer-readable storage medium 2024.

[0137] The computer-readable storage medium 2024 is a physical or tangible storage device used to store program instructions 2018, and not a medium that carries or transmits program instructions 2018. The computer-readable storage medium 2024, as used herein, should not be construed as ephemeral signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., pulses of light traveling through fiber optic cables), or electrical signals transmitted through a wire.

[0138] Alternatively, the program instructions 2018 may be transmitted to the data processing system 2000 using a computer-readable signal carrier. The computer-readable signal carrier is a signal and may, for example, be a relayed data signal containing the program instructions 2018. For example, the computer-readable signal carrier may be an electromagnetic signal and / or an optical signal, or any other suitable type of signal. These signals may be transmitted over connections such as wireless connections, fiber optic cables, coaxial cables, a wire, or any other suitable type of connection.

[0139] Furthermore, as used herein, "computer-readable medium" 2020 may be a singular or plural term. For example, the program instructions 2018 may be located in the computer-readable medium 2020 in the form of a single storage device or storage system. In another example, the program instructions 2018 may be located in the computer-readable medium 2020 distributed across multiple data processing systems. In other words, some instructions in the program instructions 2018 may be located in one data processing system, while other instructions in the program instructions 2018 may be located in a different data processing system.For example, a portion of the program instructions 2018 may be located in the computer-readable medium 2020 in a server computer, while another portion of the program instructions 2018 may be located in the computer-readable medium 2020 located in a set of client computers.

[0140] The various components illustrated for data processing system 2000 are not intended to impose any architectural limitations on the manner in which various embodiments may be implemented. In some illustrative examples, one or more of the components may be integrated with, or otherwise form part of, another component. For example, in some illustrative examples, main memory 2006 or portions thereof may be integrated with processor unit 2004. The various illustrative embodiments may be implemented in a data processing system that includes components in addition to, or instead of, the components illustrated for data processing system 2000. Further in Fig.Components shown in Figure 20 may differ from the illustrative examples shown. The various embodiments may be implemented using a hardware device or system capable of executing the program instructions 2018.

[0141] Thus, illustrative embodiments of the present invention provide a computer-executed method, apparatus, system, and computer program product for managing updates to images. According to a single illustrative embodiment, a computer-executed method manages updates to images. A computer system determines shared layers that exist between images selected for update management. The images include executable code that executes to create containers. The computer system detects a change in a shared layer among the shared layers for an image in the images.The computer system updates the shared layer in the shared layers in a set of images that have the shared layer in response to detecting the change to the shared layer for the image. According to further illustrative embodiments, a computer system and a computer program product for managing image updates are provided. Consequently, the illustrative embodiments may provide a technical effect of increasing performance in reducing image update errors and increasing image update performance by eliminating the need to update the code in each image individually.

[0142] In the illustrative examples, errors are reduced by automated updating of shared layers, rather than by fixing images individually. Furthermore, performance is improved by replacing pointers, such as unique identifiers, to a shared layer across multiple images, rather than fixing images or copying fixed files between images. Using a new unique identifier to an updated shared layer in images that have the shared layer causes those images to share the updated layers without having to fix files for each image individually. In the illustrative example, the Image Manager does not store the files for the shared layers.Rather, updates to shared layers are made by updating unique identifiers that point to the shared layer locations. Consequently, unlike correcting individual files in the images after the files have been retrieved for use in container creation, an updated shared layer can be pulled from a repository using the updated shared layer's new unique identifier.

[0143] It also reduces errors and increases performance in pipeline workloads that may have hundreds or thousands of steps, with each step performed by a container formed by executing files in an image.

[0144] The description of the various illustrative embodiments has been presented for the purpose of illustration and explanation and is not intended to be exhaustive or limited to the embodiments in the form disclosed. The various illustrative examples describe components that perform actions or operations. In an illustrative embodiment, a component may be configured to perform the described action or operation. For example, the component may have a configuration or design for a structure that provides the component with a capability to perform the action or operation described in the illustrative examples as if performed by the component.Furthermore, to the extent the terms “comprises,” “comprising,” “has,” “includes,” and variations thereof are used herein, these terms are intended to be inclusive in a manner similar to the term “comprises” as an open transition word, without excluding additional or different elements.

[0145] The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the disclosed embodiments. Not all embodiments include all of the features described in the illustrative examples. Furthermore, different illustrative embodiments may provide different features than other illustrative embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiment. The terminology used herein has been chosen to best explain the principles of the embodiment, practical application, or technical improvement over existing technology, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

[1] A computer-implemented method for managing image updates, the computer-implemented method comprising: Determining, by a computer system, shared layers existing between the images selected for update management, the images comprising executable code that is executed to create containers; Detecting, by the computer system, a change in a shared layer in the shared layers for an image in the images; and Updating, by the computer system, the shared layer in the shared layers for a set of the images that have the shared layer in response to detecting the change to the shared layer for the image. [2] A computer-implemented method according to claim 1, further comprising: Removing, by the computer system, a selected image from the images specified for management, in response to user input, stopping automatic updates for the selected image. [3] The computer-implemented method of claim 1, wherein determining, by the computer system, the shared layers present between the images selected for update management comprises: Determining, by the computer system, the shared layers that exist between the images selected for update management using layer context information for the images. [4] The computer-implemented method of claim 1, wherein determining, by the computer system, the shared layers present between the images selected for update management comprises: Determining, by the computer system, the shared layers that exist between the images selected for update management, using commands used to create the images. [5] The computer-implemented method of claim 1, wherein detecting, by the computer system, the change in the shared layer in the shared layers for the image in the images comprises: Detecting, by the computer system, a change to the image using image information for the image indicating that the image has changed; and Determining, by the computer system, that the shared layer for the image has changed using layer context information for the image that indicates that the shared layer has changed in response to detecting the change to the image. [6] The computer-implemented method of claim 1, wherein detecting, by the computer system, the change in the shared layer in the shared layers for the image in the images comprises: Detecting, by the computer system, commands to update the image; and based on the commands for layers for the image that, in response to detecting the commands to update the image, indicate that the shared layer in the layers has changed, determining, by the computer system, that the shared layer for the image has changed. [7] The computer-implemented method of claim 1, wherein updating, by the computer system, the shared layer in the shared layers in the set of images having the shared layer in response to detecting the change to the shared layer for the image comprises: Determining, by the computer system, a new unique identifier for the shared layer for the image that has changed; Indicating, by the computer system, the set of images that have the shared layer that has changed; and Updating, by the computer system, the shared layer for the set of images using the new unique identifier for the shared layer that has changed. [8] The computer-executed method of claim 1, wherein the images form an AI pipeline and the images are for the containers that perform steps in the AI ​​pipeline. [9] Computer system comprising: a number of processor units, wherein the number of processor units executes program instructions to: identify shared layers that exist between images selected for update management, where the images include executable code that is executed to create containers; detect a change in a shared layer in the shared layers for an image in the images; and update the shared layer in the shared layers for a set of the images that have the shared layer in response to detecting the change to the shared layer for the image. [10] The computer system of claim 9, wherein the number of processor units execute program instructions to: remove a selected image from the images specified for management in response to a user input to pause automatic updates for the selected image. [11] The computer system of claim 9, wherein, upon determining the shared layers existing between the images selected for update management, the number of processor units executes program instructions to: Determine the shared layers that exist between the images selected for update management using layer context information for the images. [12] The computer system of claim 9, wherein, upon determining the shared layers existing between the images selected for update management, the number of processor units executes program instructions to: Determine the shared layers that exist between the images selected for update management using commands used to create the images. [13] The computer system of claim 9, wherein upon detecting the change in the shared layer in the shared layers for the image in the images, the number of processor units executes program instructions to: detect a change to the image using image information for the image that indicates that the image has changed; and determine that the shared layer for the image has changed, using layer context information for the image that indicates that the shared layer has changed in response to detecting the change to the image. [14] The computer system of claim 9, wherein upon detecting the change in the shared layer in the shared layers for the image in the images, the number of processor units executes program instructions to: Detect commands to update the image; and Determine that the shared layer for the image has changed based on the layer commands for the image that indicate that the shared layer in the layers has changed in response to the image update commands being detected. [15] The computer system of claim 9, wherein, in updating the shared layer in the shared layers in the set of images having the shared layer, in response to detecting the change to the shared layer for the image, the number of processor units executes program instructions to: determine a new unique identifier for the shared layer for the image that has changed; specify the set of images that have the shared layer that has changed; and update the shared layer for the set of images using the new unique identifier for the shared layer that has changed. [16] The computer system of claim 9, wherein the images form an AI pipeline and the images are intended for the containers that perform steps in the AI ​​pipeline. [17] A computer program product for managing updates of images, the computer program product comprising a computer-readable storage medium having program instructions embodied thereon, the program instructions being executable by a computer system to cause the computer system to perform a method to: to identify layers shared by the computer system that exist between the images selected for update management, the images comprising executable code that is executed to create containers; by the computer system to detect a change in a shared layer in the shared layers for an image in the images; and by the computer system updating the shared layer in the shared layers for a set of the images that have the shared layer in response to detecting the change to the shared layer for the image. [18] The computer program product of claim 17, further comprising: Removing, by the computer system, a selected image from the images specified for management, in response to user input, stopping automatic updates for the selected image. [19] The computer program product of claim 17, wherein determining, by the computer system, the shared layers present between the images selected for update management comprises: Determining, by the computer system, the shared layers that exist between the images selected for update management using layer context information for the images. [20] The computer program product of claim 17, wherein determining, by the computer system, the shared layers present between the images selected for updating comprises: Determining, by the computer system, the shared layers that exist between the images selected for update management, using commands used to create the images.

Images