Container instantiation with union file system layer loading

Abstract

Computing system enhancements make container instantiation faster, reduce layer content storage requirements, and make more container image formats available. The container instantiation location sends a container image pull request to a container registry, receives an image manifest, sends a layer load request to the registry instead of a layer content download request, receives layer loads, optionally repeats for additional layers, creates a union file system spanning the layers, and launches the container process based on the union file system without first downloading all layer content. Inefficiencies and technical limitations of some other approaches are avoided, such as loopback loading for snapshot expansion, creation or transfer of extra snapshots or extra container image clones, cluttered layer content with virtual machine settings, container system vendor lock-in, lack of container instantiation at a local system due to insufficient local storage, and lack of system call optimization due to storage driver plug-in usage.

Description

BACKGROUND

[0001] In the context of computing technology, "virtualization" can be used to create versions of things that have greater universality or greater flexibility than the original versions of the things. For example, computer hardware platforms, digital storage devices, and computer network resources each have non-virtualized versions that are tied to specific physical hardware items, and virtualized versions that are implemented with software that can execute on a variety of different types of physical hardware. Virtualization can have greater universality (e.g., the ability to run on different hardware) and greater flexibility (e.g., relatively easy to make functional changes) than their non-virtualized counterparts, because a greater amount of the virtualized functionality is implemented as software.

[0002] Virtualization can be performed on a wide variety of physical items, and can be performed at a variety of levels. For example, in the context of cloud computing, virtualization helps provide different levels of functionality on top of underlying hardware. These levels include "Infrastructure as a Service" (IaaS), "Platform as a Service" (PaaS), and "Software as a Service" (SaaS). Cloud computing is often performed with a finer granularity using "virtual machines" as virtualization of physical computer systems; virtual machines run on a hypervisor, which is a virtualization of underlying computing, storage, and networking hardware. SUMMARY

[0003] Some embodiments use or perform operations that instantiate a container at an instantiation location in a cloud or other computing system. From the perspective of the instantiation location, these operations can include sending a container image pull request to a container registry, where the pull request identifies a container image; receiving an image manifest that lists a plurality of layers that collectively make up the container image, where each layer has a respective layer identification and respective layer content; sending a layer load request to the container registry, where the layer load request identifies a layer of the container image; receiving a layer load that provides access to content of a registry-resident form of the layer identified in the layer load request without providing access to all content of all layers that collectively make up the container image; creating a union file system that spans the layers that collectively make up the container image and that is based on at least one received layer load; and launching a container process at the instantiation location based on the union file system. These operations avoid transferring at least part of the layer content of the identified layer from the container registry to the instantiation location over a network before launching the container process.

[0004] From the perspective of the container registry, the operations performed in support of instantiating the container at the instantiation location can include: the container registry receiving a container image pull request, the pull request identifying a container image; the container registry sending an image manifest, the image manifest listing a plurality of layers that collectively make up the container image, each layer having a respective layer identification and a respective layer content; the container registry receiving a layer load request, the layer load request identifying a layer of the container image; and the container registry sending a layer load, the layer load providing access to the content of the layer identified in the layer load request in a registry-resident form without providing access to all of the content of all of the layers that collectively make up the container image. The container registry avoids sending at least a portion of the layer content of the identified layer prior to the container process being started at the instantiation location.

[0005] Some embodiments of the teachings presented herein include or communicate with a container instantiation function that includes digital hardware configured to perform certain operations. The configured hardware includes a memory and a processor, both located at an instantiation location such as a particular virtual machine or server. The operations can include: (a) sending a container image pull request to a container registry, the pull request identifying a container image, (b) receiving an image manifest listing a plurality of layers that collectively make up the container image, each layer having a respective layer identification and a respective layer content, (c) sending a layer load request to the container registry, the layer load request identifying a layer of the container image, (d) receiving a layer load, the layer load providing access to the layer content in a registry-resident form identified in the layer load request without providing access to all of the content of all of the layers that collectively make up the container image, (e) creating a union file system that spans the layers that collectively make up the container image, and (f) starting a container process at the instantiation location based on the union file system without first obtaining all of the content of all of the layers that collectively make up the container image at the instantiation location. The memory is configured at least by the image manifest and the union file system; the processor is configured by instructions that perform the operations.

[0006] Other technical activities related to the teachings herein will be apparent to those of ordinary skill in the art in light of the teachings hereof. The examples given are exemplary only. The summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. Rather, the purpose of the summary is to present some of the technical concepts discussed in a simplified form as a prelude to the more detailed description that is to follow. The claimed subject matter is defined by the claims and, accordingly, the claims should be consulted to determine the true scope of the claimed subject matter. In the event of a conflict between the summary and the claims, the claims shall control. BRIEF DESCRIPTION OF DRAWINGS

[0007] A more particular description will be rendered by reference to the appended drawings. These drawings are included only to provide a full and enabling disclosure of the application and are not intended to limit the scope of the application. The drawings are included solely for purposes of exemplification.

[0008] Figure 1 is a block diagram illustrating a computer system in general, and also illustrating a configured storage medium in general;

[0009] Figure 2 is a block diagram illustrating a container instantiation computing environment and a container registry computing environment;

[0010] Figure 3 is a block diagram illustrating some examples of a container instantiation environment;

[0011] Figure 4 is a block diagram illustrating aspects of a system configured to use a union file system and a container layer loading function for fast container instantiation;

[0012] Figure 5 is a block diagram illustrating some container instantiation communications;

[0013] Figure 6 is a block diagram illustrating some relationships that can be implemented between layer loading and one or more static layers of a container;

[0014] Figure 7 is a data flow diagram illustrating a container instantiation method in which all contents of a layer are downloaded without using layer loading;

[0015] Figure 8 is a data flow diagram illustrating a container instantiation method in which most or all contents of a layer are downloaded only after a container including the layer has been started;

[0016] Figure 9 is a flow diagram illustrating steps in some container instantiation methods; and

[0017] Figure 10 is a flow diagram further illustrating steps in some container instantiation methods. DETAILED DESCRIPTION

[0018] SUMMARY

[0019] Many innovations extend beyond their origins, but understanding the origins of an innovation can help people more fully appreciate the innovation. In the present case, some of the teachings described herein were inspired by the technical challenge of reducing startup time for cold-start containers (i.e., containers that are not cached locally). Other technical challenges addressed by the innovations taught here will also be apparent to those skilled in the art from the discussion provided below.

[0020] Containers are virtualized. In this regard, they are similar to virtual machines. But whereas a virtual machine virtualizes computer hardware, so that each virtual machine typically has its own copy of an operating system, a container system virtualizes an operating system. Different containers can provide respective users address spaces with respective visibility into a shared underlying operating system. Thus, multiple workloads in the form of containers can run on a single operating system instance. A computer program running on an operating system on a computer without container virtualization has access to (at least by default) most, if not all, of the computer's computing resources, such as connected devices, files and directories, network shares, and processors. However, a program running within a container can only see the contents of the container and the devices that have been assigned to the container.

[0021] Depending on the context, a container can be referred to by other names, such as "OS-level virtualization instance," "zone," "virtual private server," "partition," "virtual environment," "virtual kernel," or "jail." However, those skilled in the art will recognize that some of these names also have other meanings that do not necessarily relate to containers, e.g., "partition" can refer to a disk partition, and "zone" can refer to a data center availability zone.

[0022] A container has layers, which have respective contents. A container runs as a container process. Thus, "container" as used herein can refer to a process that accesses the layers, the layers that make up a container, the layer contents, metadata such as a list of layers, or combinations thereof. The intended meaning will be clear to those skilled in the art, by the presence of qualifying terms such as "container process" or by context. Any remaining ambiguity should be resolved in a manner that gives the description or claims their broadest possible effect and operative meaning.

[0023] In some systems, the execution lifecycle of a container can include pulling (i.e., downloading) contents from a remote repository, unpackage the contents, creating a union file system on top of the contents, and finally creating a process based on the newly defined file system. Unpackaging the contents can include extracting selected contents from surrounding data, e.g., copying the contents of layer L from a larger image file that includes not only the contents of layer L but also the contents of other layers. Unpackaging the contents can also include unpacking, which is used herein as a synonym for "decompressing." Unpackaging the contents can also include decrypting the contents. In some systems, the majority of the time taken after a container instantiation is requested and before the execution of the container process begins is taken during the pulling and unpackaging of the contents.

[0024] Some embodiments of the teachings provided herein allow for fast instantiation of containers using hybrid file systems. A container is "instantiated" when the container process is launched and has (directly or via layer loading) access to all the layer content belonging to the container. As used herein, "fast" instantiation of a container as a container process is an instantiation sequence that (a) uses one or more layer loads to give the container process access to the layer content prior to the container process being launched, and (b) takes less than one-tenth the time that would be used by an alternative instantiation sequence that downloads (and also can unpackage) the layer content prior to the container process being launched, instead of using one or more layer loads. Fast container instantiation uses layer loading as taught herein, instead of downloading and (when needed) unpackage the layer content prior to the container process being launched. Some embodiments greatly reduce or eliminate pulling and unpackage of content, allowing them to achieve near-instant container instantiation.

[0025] Some embodiments described herein can be reviewed by some in a broader context. For example, concepts such as content, downloading, identifying, launching, listing, and computing processes can generally be considered relevant to particular embodiments. However, it does not follow from the availability of the broader context that exclusive rights are sought in abstract concepts herein; they are not. Rather, the present disclosure focuses on providing proper particular embodiments whose technical effects wholly or partly solve particular technical problems, such as slow instantiation of containers and unnecessary downloading of layer content. Storage media, systems, and processes involving other configurations of content, downloading, identifying, launching, listing, or computing processes are generally outside the scope. Thus, ambiguity, mere abstraction, lack of technical character, and attendant proof problems are also avoided with a proper understanding of the present disclosure.

[0026] More generally, those skilled in the art will recognize that not every part of the present disclosure or any specific detail therein need meet legal standards such as enablement, written description, or best mode. Moreover, embodiments are not limited to particular file systems, software development environments, programming languages, container format examples, development tools, identifiers, files, data structures, symbols, control flow, pseudocode, security mechanisms, or other implementation choices described herein. Any apparent conflict with any other patent disclosure, even by the owner of the present innovation, does not operate in interpreting claims presented in the present patent disclosure.

[0027] Technical Features

[0028] The technical features of the embodiments described herein will be apparent to those of ordinary skill in the art, and will be apparent in a number of respects to those of ordinary skill and attention in the art. Some embodiments address technical activities such as instantiating containers and measuring container instantiation performance, activities that are deeply rooted in computing technology. Some of the technical mechanisms discussed include, for example, layer loading, union file systems, change sets, manifests, caches, repositories, and registries. Some of the technical effects discussed include, for example, faster container instantiation, avoiding image cloning, avoiding virtual machine settings in layer content, layer sharing between containers, instantiating containers despite lack of local storage sufficient to hold layer content for all containers, and accessing layer content without using snapshots. Thus, purely mental processes are precluded. Some embodiments improve the functioning of computing systems by making containers more quickly available in practice, by reducing layer content storage requirements, or by expanding the range of container image formats available for use without incurring performance hits. Other advantages of the technical features based on the teachings provided will be apparent to those of skill in the art, based on the description provided.

[0029] Notes on hyperlinks

[0030] This disclosure can contain various URIs, hyperlinks, IP addresses, and / or other items that can be considered browser-executable code. These items are included in this disclosure as a courtesy only, and not to reference the contents of websites or files identified as necessary support for the description of the embodiments. The Applicant does not intend for these URIs, hyperlinks, IP addresses, or other such code to be active links. None of these items are intended to be imported as references to material located outside of the disclosure document. Therefore, inclusion of these items herein should not be opposed. In the event that these items have not been disabled, it is assumed that the Patent Office will disable them (make them inactive as links) when preparing the text of this document for loading into their official web database. See, e.g., U.S. Patent and Trademark Office Manual of Patent Examining Procedures § 608.01(VII).

[0031] Acronyms, Abbreviations, Names, and Symbols

[0032] Some acronyms, abbreviations, names, and symbols are defined below. Others are defined elsewhere herein, or need not be defined herein for those of skill in the art to understand.

[0033] ALU: arithmetic and logic unit

[0034] API: application program interface

[0035] BIOS: basic input / output system

[0036] CD: compact disc

[0037] CPU: Central Processing Unit

[0038] DVD: Digital Versatile Disk or Digital Video Disk

[0039] FPGA: Field Programmable Gate Array

[0040] FPU: Floating Point Unit

[0041] GPU: Graphics Processing Unit

[0042] GUI: Graphical User Interface

[0043] IaaS or IAAS: Infrastructure as a Service

[0044] ID: Identification or Identity

[0045] IDE: Integrated Development Environment, sometimes also referred to as "Interactive Development Environment"

[0046] IoT: Internet of Things

[0047] IP: Internet Protocol

[0048] LAN: Local Area Network

[0049] OS: Operating System

[0050] PaaS or PAAS: Platform as a Service

[0051] RAM: Random Access Memory

[0052] ROM: Read-Only Memory

[0053] SSD: Solid State Drive

[0054] UEFI: Unified Extensible Firmware Interface

[0055] URI: Uniform Resource Identifier

[0056] URL: Uniform Resource Locator

[0057] VM: Virtual Machine

[0058] VHD: Virtual Hard Disk

[0059] VHDX: Virtual Hard Disk Extension

[0060] WAN: Wide Area Network

[0061] Some Additional Terminology

[0062] This document refers to exemplary embodiments as shown in the drawings and uses specific language to describe the exemplary embodiments herein. However, variations and further modifications of the features shown herein and additional applications of the abstract principles underlying the specific embodiments shown herein are to be considered within the scope of the claims, which are to be afforded the broadest reasonable interpretation available, to those skilled in the relevant art, having the benefit of the present disclosure.

[0063] The meaning of terms is set forth in the disclosure, and the claims should be carefully read to understand these explications. Specific examples are given, but those skilled in the relevant art(s) will understand that other examples can also fall within the meaning of the terms used and within the scope of one or more claims. The terms do not necessarily have the same meaning as they do in general usage, particularly in non-technical usage, or in usage in a specific industry, or in a specific dictionary or dictionary set. Reference signs can be used with various phrases to help illustrate the breadth of the terms. Omission of reference signs from a given text does not necessarily mean that the text is not discussing the content of the figure. The present inventor asserts and exercises the right to specific and selective lexicography. The cited terms are explicitly defined, but the terms can also be implicitly defined without the use of quotes. The terms can be explicitly or implicitly defined in the DETAILED DESCRIPTION herein and / or elsewhere in the application file.

[0064] As used herein, a "computer system" (i.e., a "computing system") can include, for example, one or more servers, motherboards, processing nodes, laptop computers, tablet computers, personal computers (portable or non-portable), personal digital assistants, smart phones, smart watches, smart bands, cellular or mobile phones, other mobile devices having at least a processor and memory, video game systems, augmented reality systems, holographic projection systems, televisions, wearable computing systems, and / or other device(s) that provide one or more processors controlled at least in part by instructions. The instructions can be in the form of firmware or other software in memory and / or dedicated circuitry.

[0065] A "multithreaded" computer system is a computer system that supports multiple threads of execution. The term "thread" should be understood to include code that is capable of or subject to scheduling and possibly synchronization. Outside of this disclosure, a thread can also be referred to by an alias such as "task," "process," or "coroutine." However, a distinction is made herein between threads and processes because a thread defines an execution path within a process. In addition, threads of a process share a given address space, while different processes have different respective address spaces. Threads of a process can run in parallel, sequentially, or in a combination of parallel execution and sequential execution (e.g., time-slicing).

[0066] A "processor" is a thread processing unit, such as a core in a simultaneous multi-threading implementation. A processor includes hardware. A given chip can host one or more processors. Processors can be general purpose, or they can be customized for a particular use, such as vector processing, graphics processing, signal processing, floating point arithmetic processing, encryption, I / O processing, machine learning, and the like.

[0067] A "kernel" includes an operating system, a hypervisor, a virtual machine, BIOS or UEFI code, and similar hardware interface software.

[0068] "Code" means processor instructions, data (which includes constants, variables, and data structures), or both instructions and data. "Code" and "software" are used interchangeably herein. Executable code, interpreted code, and firmware are some examples of code.

[0069] "Program" is used broadly herein to include applications, kernels, drivers, interrupt handlers, firmware, state machines, libraries, and other code written and / or generated by programmers, also referred to as developers.

[0070] A "service" refers to a consumable program that provides access to resources or resource access for multiple programs in a cloud computing environment or other networked or computing system environment.

[0071] "Cloud" refers to a pooled resource for computing, storage, and networking that is elastically available to measured demand. A cloud can be private, public, community, or hybrid, and cloud services can be provided in the form of infrastructure as a service (IaaS), platform as a service (PaaS), software as a service (SaaS), or another service. Unless otherwise noted, any discussion of reading from or writing to a file includes reading / writing a local file or reading / writing over a network, which can be a cloud network or other network, or both (local and networked read / write).

[0072] “IOT” or “Internet of Things” means any networked collection of addressable, embedded computing nodes. These nodes are examples of computer systems as defined herein, but they also have at least two of the following characteristics: (a) no local human-readable display; (b) no local keyboard; (c) the raw source of input is a sensor that tracks non-linguistic data sources; (d) no local rotating disk storage device, with RAM chips or ROM chips providing only local memory; (e) no CD or DVD drive; (f) embedded in a home appliance or home fixture; (g) embedded in an implanted or wearable medical device; (h) embedded in a vehicle; (i) embedded in a process automation control system; or (j) designed with a focus on one of: environmental monitoring, civic infrastructure monitoring, industrial equipment monitoring, energy usage monitoring, human or animal health monitoring, physical security, or physical transportation system monitoring. IoT storage devices can be targets of unauthorized access attempted via the cloud, via another network, or via direct local access.

[0073] In the context of container instantiation, “local” herein means the local of the instantiation location 204. The instantiation location 204 is also not necessarily the local of the developer 104 that requested instantiation or whose actions are directed to the instantiation, and is also not necessarily the local of the integrated development environment or virtual machine or other tool 122 that manages the instantiated container.

[0074] As used herein, “comprises” or “comprising” allows additional elements (i.e., comprising means including).

[0075] “Optimization” means improvement, not necessarily perfection. For example, a program or algorithm that has been optimized can be further improved.

[0076] “Process” is sometimes used herein as a term in the field of computing science, and includes in that technical sense a resource user, i.e., for example, a co-operating program, thread, task, interrupt handler, application process, kernel process, program, and object method. “Process” is also used herein as a term in patent law, for example, in describing a process claim as opposed to a system claim or an article of manufacture (configured storage medium) claim. Similarly, “method” is sometimes used herein as a technical term in the field of computing science (a “routine”), and also as a term in the field of patent law. “Process” and “method” in the sense of patent law are used interchangeably herein. Those skilled in the art will understand which sense is intended in a particular instance, and will also understand that a given claimed process or method (in the sense of patent law) can sometimes be implemented using one or more processes or methods (in the sense of computing science).

[0077] "Automatically" means using automation (e.g., general purpose computing hardware configured by software for the particular operations and technical effects discussed herein) as opposed to without automation. Specifically, steps performed "automatically" are not performed by hand on paper or in a person's mind, but they can be initiated by a person or interactively guided by a person. The automated steps utilize machines to perform to obtain one or more technical effects that could not be achieved without the technical interaction provided thereby.

[0078] Those skilled in the art understand that technical effects are the presumed purpose of technical embodiments. For example, the fact that computations are involved in embodiments and that some computations can be performed without technical components (e.g., by paper and pencil, or even as mental steps) does not eliminate the existence of technical effects or change the specific and technical nature of the embodiments. Operations such as sending or receiving mirror pull requests or manifests or layer load requests or layer loading, creating union file systems, starting container processes, and many other operations discussed are understood herein to be intrinsically digital. Human minds cannot interface directly with CPUs or other processors or with RAM or other digital storage to read and write the necessary data to perform the container instantiation steps taught herein. Those skilled in the art will fully appreciate this point in light of the present disclosure, but it can sometimes be necessary to inform or remind others of these facts.

[0079] "Computationally" likewise means that a computing device (at least a processor plus memory) is being used, and excludes obtaining a result by human thought alone or by human action alone. For example, doing arithmetic with paper and pencil is not doing arithmetic computation as understood herein. Computing results faster, wider, deeper, more accurately, more consistently, more comprehensively, and / or otherwise providing technical effects beyond the range of human performance. A "computational step" is a step performed computationally. "Automatically" or "computationally" do not necessarily mean "immediately." "Computationally" and "automatically" are used interchangeably herein.

[0080] "Proactively" means without a direct request from a user. In fact, a user can not even be aware that a proactive step of an embodiment is possible until the result of that step has been presented to the user. Unless otherwise noted, any computational and / or automatic steps described herein can also be done proactively.

[0081] In this document, the use of optional plural "plurality," "multiple," or "a plurality of" means that there is one or more of the indicated features. For example, "a (plurality of) processor" means "one or more processors" or equivalently "at least one processor."

[0082] For purposes of United States law and practice, the use of the word "step" in this document, in the claims or elsewhere is not intended to invoke 35 U.S.C. § 112, paragraph 6 / 35 U.S.C. § 112(f) claim interpretation. Any presumption to the contrary is hereby expressly disavowed.

[0083] For purposes of United States law and practice, claims are not intended to invoke the apparatus-plus-function interpretation unless the claim uses the phrase "apparatus for". If there is any intent to invoke the apparatus-plus-function language, it will be explicitly recited in the claim language. When the apparatus-plus-function interpretation is applied, the apparatus recited in the specification as associated with the given name or the given verb should be understood to be linked to the claim language, and is linked together in this document by any of the following: appearing within the same block in a block diagram of the drawings, being represented by the same or similar name, being represented by the same reference number, functional relationship depicted in any drawing, functional relationship noted in the disclosure text. For example, if a claim limitation references a "zac widget" and the claim limitation becomes subject to the apparatus-plus-function interpretation, then all structures identified in any block, paragraph or example in the specification that refer to a "zac widget" or that are bound by any reference number assigned to a zac widget, or that are disclosed to have a functional relationship with a zac widget structure or operation, will be considered to be part of the structure identified in the claim for the zac widget and will help define the set of equivalents for the zac widget structure.

[0084] Those skilled in the art will recognize that the present innovation discusses various data values and data structures, and recognizes that these items reside in memory (RAM, disk, etc.), thereby configuring the memory. Those skilled in the art will also recognize that the present innovation discusses various algorithmic steps that are implemented in executable code in a given implementation, and that such code also resides in memory, and it effectively configures any general purpose processor executing it, thereby transforming it from a general purpose processor into a special purpose processor dedicated to the function of the code.

[0085] Thus, the skilled person will not erroneously consider the memory recited in (a) claim as non-overlapping item with the data structure or data value or code recited in (b) claim. The data structures and data values and code are understood to reside in the memory even if the recitation of the residence for each data structure or data value or code fragment is not explicitly recited in the claim. Thus, an explicit recitation of such residence is not required. However, they are not prohibited either and one or both of the alternative recitations can exist for emphasis without thereby excluding all other data values and data structures and code from the residence. Likewise, the code functionality recited in the claim is understood to configure the processor irrespective of whether the configuring quality is explicitly recited in the claim.

[0086] Throughout this document, unless explicitly stated to the contrary, any reference to a step in a process assumes that the step can be performed directly by the interested party and / or indirectly by the party through intervening agencies and / or intervening entities and still be within the scope of the step. That is, the interested party is not required to perform the step directly unless direct performance is explicitly stated as the requirement. For example, steps that involve the interested party action with respect to a destination or other subject (such as, accessing, determining, based on, caching, cloning, configuring, constituting, creating, extending, identifying, initiating, listing, locating, measuring, loading, omitting, packaging, providing, receiving, retrieving, sending, sharing, spanning, storing, transmitting (sending or receiving or both), unifying, uninstalling, unpackaging, using (as well as accessing, determined, etc., and avoiding any step step) can involve intervening action by other parties, such as forwarding, copying, uploading, downloading, encoding, decoding, compressing, decompressing, encrypting, decrypting, authenticating, invoking, etc. (including any action recited in this document) still be understood as being performed directly by the interested party with respect to the destination or other subject.

[0087] Whenever reference is made to data or instructions, it is understood that these items of data or instructions are stored in computer-readable storage media, which is computer- readable storage memory or computer-readable storage media, which is non-transitory computer- readable storage memory or computer-readable storage media. It is further understood that the data or instructions can be retrieved from the computer-readable storage media and loaded into the computer-readable storage media for execution. The computer-readable storage media can be a transitory or non-transitory computer-readable storage medium. The computer-readable storage media can be a computer-readable storage memory or a computer-readable storage media. The computer-readable storage media can be a non-transitory computer-readable storage memory or a non-transitory computer-readable storage media. The computer-readable storage media can be a tangible computer-readable storage media. The computer-readable storage media can be a volatile computer-readable storage media or a non-volatile computer-readable storage media. The computer-readable storage media can be a computer-readable storage memory or a computer-readable storage media. The computer-readable storage media can be a non-transitory computer-readable storage memory or a non-transitory computer-readable storage media. The computer-readable storage media can be a tangible computer-readable storage media. The computer-readable storage media can be a volatile computer-readable storage media or a non-volatile computer-readable storage media. Whenever a computer-readable storage medium or media is / were referenced in this document, it is understood that the computer-readable storage medium or media is / are computer-readable storage memory or computer-readable storage media, which is / are non-transitory computer-readable storage memory or computer-readable storage media. It is further understood that the data or instructions can be retrieved from the computer-readable storage medium or media and loaded into the computer-readable storage medium or media for execution. The computer-readable storage medium or media can be a transitory or non-transitory computer-readable storage medium or media. The computer-readable storage medium or media can be a computer-readable storage memory or a computer-readable storage media. The computer-readable storage medium or media can be a non-transitory computer-readable storage memory or a non-transitory computer-readable storage media. The computer-readable storage medium or media can be a tangible computer-readable storage medium or media. The computer-readable storage medium or media can be a volatile computer-readable storage medium or a non-volatile computer-readable storage medium. Whenever a computer-readable storage medium or media is / were referenced in this document, it is understood that the computer-readable storage medium or media is / are computer-readable storage memory or computer-readable storage media, which is / are non-transitory computer-readable storage memory or computer-readable storage media. It is further understood that the data or instructions can be retrieved from the computer-readable storage medium or media and loaded into the computer-readable storage medium or media for execution. The computer-readable storage medium or media can be a transitory or non-transitory computer-readable storage medium or media. The computer-readable storage medium or media can be a computer-readable storage memory or a computer-readable storage media. The computer-readable storage medium or media can be a non-transitory computer-readable storage memory or a non-transitory computer-readable storage media. The computer-readable storage medium or media can be a tangible computer-readable storage medium or media. The computer-readable storage medium or media can be a volatile computer-readable storage medium or a non-volatile computer-readable storage medium.

[0088] Also, unless explicitly described otherwise, in (a) one aspect, computer- readable storage media and computer-readable memories are to be understood as discrete

[0089] An "embodiment" herein is an example. The term "embodiment" is not interchangeable with the term "invention." Embodiments can freely share or borrow aspects to create other embodiments (as long as the result is operable), even if the resulting combination of aspects is not explicitly described herein. It is unnecessary for every and each allowed combination to be explicitly and individually described to the skilled person, and would be contrary to the strategy of recognizing that a patent specification is written for a reader skilled in the art. Formal combinatorial calculations and informal common intuition about the number of possible combinations resulting from even a small number of combinable features would also indicate that a large number of combinations of aspects exist for the aspects described herein. Therefore, requiring every and each combination to be explicitly recited would be contrary to the strategy of keeping the patent specification concise and requiring the reader to have knowledge of the relevant art.

[0090] List of reference signs

[0091] The following list is provided for convenience and to support the drawings and as part of the specification text describing the invention by reference to multiple items. Items not listed here can still be part of a given embodiment. For better readability of the text, some but not all recitations of an item are referenced near the recitation of the given drawing reference. The same drawing reference is used to refer to different examples or different instances of a given item. The list of reference signs is:

[0092] 100 operating environment, also referred to as computing environment;

[0093] 102 computer system, also referred to as system of computing or computing system;

[0094] 104 user;

[0095] 106 peripheral device;

[0096] 108 network, typically including e.g. LAN, WAN, software-defined network, cloud, and other wired or wireless networks;

[0097] 110 processor;

[0098] 112 computer-readable storage media, e.g., RAM, hard disk;

[0099] 114 computer-readable storage media that is removable configured;

[0100] 116 instructions executable by a processor; can be on removable storage media or in other memory (volatile or non-volatile or both);

[0101] 118 data

[0102] 120 kernel(s), e.g., operating system(s), BIOS, UEFI, device drivers;

[0103] 122 tools, e.g., anti-virus software, firewall, packet sniffer software, intrusion detection system, intrusion prevention system, debugger, profiler, compiler, interpreter, software development tools and tool suites, hardware development tools and tool suites, diagnostic program;

[0104] 124 applications, e.g., word processor, web browser, spreadsheet, game, email tool;

[0105] 126 display screen, also referred to as "display";

[0106] 128 computing hardware not associated with reference numerals 106, 108, 110, 112, 114;

[0107] 202 instantiation environment

[0108] 204 instantiation location, e.g., particular virtual machine, server, network node, URI or URL, or cluster, or hardware box, or virtual location;

[0109] 206 registry, i.e., list or other data structure that identifies computing resources stored in one or more repositories and locations of those resources; "registry" also refers to a server or other computing system that contains the list of computing resources;

[0110] 208 registry environment; includes the registry and all repositories storing the computing resources identified in the registry;

[0111] 210 container, e.g., computing construct that provides user-space virtualization and does not itself include an operating system;

[0112] 212 container image; also refers to file(s) holding the container image;

[0113] 214 container layer; also refers to file(s) holding the container layer;

[0114] 216 a number or other electronic communication to instantiate a container;

[0115] 302 cloud; can also be referred to as a "cloud computing environment";

[0116] 304 virtual machine, e.g., a computing construct that provides hardware virtualization and includes an operating system;

[0117] 306 production environment;

[0118] 402 container process;

[0119] 404 union file system;

[0120] 406 container layer content;

[0121] 408 container image manifest;

[0122] 410 layer load, i.e., a URI or other data that identifies a location of layer content and can be used at a load point to provide a container process with access to the layer content;

[0123] 412 change set, e.g., a set of instructions to add, delete, or otherwise modify a file system of a container layer;

[0124] 414 file system of a container layer;

[0125] 416 container image configuration, e.g., an ordered set of root file system changes and corresponding execution parameters for use in a container runtime;

[0126] 418 metadata repository;

[0127] 502 a request to pull a copy of a container image from a registry or repository to another location;

[0128] 504 a request for a layer load 410;

[0129] 506 an identification of a layer; also referred to as a "layer identifier"; e.g., can be a hash of layer content; also refers to the act of identifying a layer;

[0130] 600 a relationship that maps one or more layer loads to one or more static layers;

[0131] 602 one-to-one, also referred to as 1 -to-1 or 1 : 1, means that one layer load corresponds to one static layer, and in particular, one layer load corresponds to one and only one static layer's content;

[0132] 604 One-to-many, also referred to as 1-to-many or 1:N, means that one layer load corresponds to multiple static layers, and specifically, one layer load corresponds to more than one static layer of content, but not to all static layers of a particular container of interest;

[0133] 700 Illustrates a data flow diagram of container instantiation with content download, prior to container process startup and without use of layer loading;

[0134] 702 Download of container image or container layer; also refers to the act of downloading; the art recognizes that unsealing typically occurs after downloading;

[0135] 704 Container image cache;

[0136] 706 Container image identifier; also refers to the act of identifying an image;

[0137] 708 Container image repository;

[0138] 710 Container image repository identifier;

[0139] 712 Container image tag, such as a version indicator or other metadata;

[0140] 714 Size of container image, such as gigabytes or other unit of storage;

[0141] 716 Container layer cache;

[0142] 718 Sending container image pull request 502; unless otherwise specified, all sending and receiving acts are assumed to include transmission of digital communications over a computer network;

[0143] 720 Receiving container image manifest 408;

[0144] 722 Sending request for delta layer, such as a layer within a surrounding container image that also stores other layers;

[0145] 724 Delta layer; can typically have the same content as layer 214;

[0146] 726 Receiving layer URL;

[0147] 728 Layer URL;

[0148] 730 Request for layer content stored at the URL in the request;

[0149] 732 Compressed version of layer content 406;

[0150] 800 Illustrates a data flow diagram of container instantiation with content download, after container process startup and with use of layer loading;

[0151] 802 Request layer load 410; it also refers to the action of sending layer load request 504;

[0152] 804 Receiver layer load 410;

[0153] 806 Extension (i.e., never compressed or not compressed now, never encapsulated or encapsulated before and now decapsulated) layer 214; used interchangeably with "decapsulated" in the context of caching;

[0154] 900 flowchart; 900 also refers to Figure 9 The container instantiation method shown in the flowchart or consistent with it;

[0155] 902 lists the layers that make up the container image, such as layers in an identity manifest or other data structure;

[0156] 904 constitutes a container image, that is, at least the static (pre-run) layer content of the container image is commonly defined;

[0157] 906 Access layer content, for example, providing access mechanisms; also refers to the actions of accessing layer content by reading or writing or both;

[0158] 908 Create a union file system;

[0159] 910 Across layers, for example, by providing access to layer content through a single unified file system;

[0160] 912 Based on, for example, derived from or dependent on;

[0161] 914 Transmission, i.e., sending or receiving via digital or other electronic functions; similar to other actions discussed herein, the action may be indicated to be avoided or prevented before “avoidance”, the avoidance itself being an action;

[0162] 916 Container startup process;

[0163] 918 has a location, situated at a specific location (physical or virtual);

[0164] 920 Instantiate the container;

[0165] 1000 Flowchart; 1000 also refers to the process by which... Figure 10 Flowchart (which is incorporated) Figure 9 The steps are illustrated in the diagram or a container instantiation method consistent with them;

[0166] 1002. Measure the elapsed time;

[0167] 1004 elapsed time; can be expressed in processor cycles, milliseconds, or other units;

[0168] 1006 performance gain;

[0169] 1008 performance gain, e.g., faster completion of a defined task, lower use of memory or network bandwidth or another computing resource, or improved security (data confidentiality, data integrity, or data availability);

[0170] 1010 receive container image pull request;

[0171] 1012 send container manifest;

[0172] 1014 receive layer load request;

[0173] 1016 send layer load;

[0174] 1018 send layer content;

[0175] 1020 compress (i.e., package) layer content;

[0176] 1022 decompress (i.e., unpack) layer content;

[0177] 1024 store data in cache;

[0178] 1026 retrieve data from cache;

[0179] 1028 share layer content among multiple containers;

[0180] 1030 use loopback load; refers to avoiding use of loopback load before the “avoid”;

[0181] 1031 loopback load;

[0182] 1032 clone container image; refers to avoiding cloning of container image before the “avoid”;

[0183] 1034 use or store virtual machine settings in a container layer; refers to avoiding use or storage of virtual machine settings in a container layer before the “avoid”;

[0184] 1036 virtual machine settings;

[0185] 1038 store layer content in a file;

[0186] 1040 file;

[0187] 1042 access (read or write) layer content;

[0188] 1044 container image snapshot;

[0189] 1046 determining that a copy of the tier content is stored locally;

[0190] 1048 local storage, such as a disk, a RAM cache, or a high-speed network- connected storage device;

[0191] 1050 omitting tier L from the load request, i.e., not requesting tier loading for the tier L;

[0192] 1052 using a storage driver plug-in; "avoid" refers to avoiding use of a storage driver plug-in before "avoiding";

[0193] 1054 storage driver plug-in;

[0194] 1056 any step discussed in this disclosure that is not assigned some other reference numeral.

[0195] Operating Environment

[0196] Reference Figure 1 For one embodiment, operating environment 100 includes at least one computer system 102. Computer system 102 can or can not be a multiprocessor computer system. The operating environment can include one or more machines in a given computer system, which can be clustered, client-server networked, and / or peer-to-peer networked in the cloud. A single machine is a computer system, and a cooperating set of machines is also a computer system. Given computer system 102 can be configured for end users (e.g., with applications), for administrators, as a server, as a distributed processing node, and / or otherwise.

[0197] Human user 104 can interact with computer system 102 through use of a display, a keyboard, and other peripheral devices 106 via typed text, touch, voice, movement, computer vision, gestures, and / or other forms of I / O. Screen 126 can be a removable peripheral device 106 or can be an integral part of system 102. The user interface can support interaction between an embodiment and one or more human users. The user interface can include a command line interface, a graphical user interface (GUI), a natural user interface (NUI), a voice command interface, and / or other user interface (UI) presentations, which can be presented as distinct options or can be integrated.

[0198] System administrators, network administrators, cloud administrators, security analysts and other security personnel, operators, developers, testers, engineers, auditors, and end users are each a particular type of user 104. Automated agents, scripts, playback software, devices, and the like that act on behalf of one or more people can also be users 104, e.g., to facilitate testing of the system 102. Depending on their detachability from the processor 110, in some embodiments storage devices and / or networking devices can be considered peripheral devices, while in other embodiments storage devices and / or networking devices can be considered part of the system 102. For example, Figure 1 Other computer systems not shown in FIG. 1 can interact with the computer system 102 in a technical way, or with another system embodiment using one or more connections to a network 108 via a network interface device.

[0199] Each computer system 102 includes at least one processor 110. The computer system 102 (like other suitable systems) also includes one or more computer- readable storage media 112. The storage media 112 can be of different physical types. The storage media 112 can be volatile memory, non-volatile memory, media fixed in place, media transportable from place to place, magnetic media, optical media, solid-state media, and / or other types of physical, durable storage media (as opposed to a signal or energy alone). In particular, storage media 114 configured such as a portable (i.e., external) hard drive, a CD, a DVD, a memory stick, or other transportable non-volatile memory media can become functionally a technical part of the computer system when plugged in or otherwise installed, such that its contents are accessible by the processor 110 for interaction and use with the processor 110. The transportable configured storage media 114 is an example of the computer-readable storage media 112. Some other examples of the computer-readable storage media 112 include built-in RAM, ROM, hard disks, and other memory storage devices that are not easily removable by the user 104. To comply with current U.S. patent requirements, a computer-readable medium or computer-readable storage medium or computer-readable memory is not a signal per se or energy alone under any pending or issued U.S. claims.

[0200] The storage medium 114 is configured with binary instructions 116 executable by the processor 110; "executable" is used broadly here to include, for example, machine code, interpretable code, bytecode, and / or code that runs on a virtual machine. The storage medium 114 is also configured with data 118 that is created, modified, referenced, and / or otherwise used by execution of the instructions 116 for technical effects. The instructions 116 and data 118 configure the memory or other storage medium 114 on which they reside; they also configure a given computer system when that memory or other computer readable storage medium is a functional part of the computer system. In some embodiments, a portion of the data 118 represents real-world items, such as product characteristics, inventory, physical measurements, settings, images, readings, targets, volumes, and the like. Such data is also transformed by backup, restore, commit, abort reformatting, and / or other technical operations.

[0201] A given operating environment 100 can include an integrated development environment (IDE) 122, which itself is a coordinated set of computing technology development tools 122 such as compilers, interpreters, disassemblers, assemblers, decompilers, source code editors, profilers, debuggers, emulators, fuzzers, registry and repository access tools, version control tools, optimizers, collaboration tools, and the like provided to a developer. In particular, some suitable operating environments for some software development embodiments include or help create a visualization development environment (a trademark of Microsoft Corporation). Some suitable operating environments include environment (a trademark of Oracle America, Inc.), and some include environments that utilize languages such as C++ or C# ("C sharp"), although many of the teachings herein can apply to various programming languages, programming models, and programs.

[0202] While embodiments can be described as being implemented as software instructions executed by one or more processors in a computing device (e.g., a general purpose computer, server, or cluster), such a description is not meant to exhaust all possible embodiments. Those of skill in the art will understand that the same or similar functionality can also be implemented, in whole or in part, in hardware logic directly, to provide the same or similar technical effect. Alternatively or in addition to software implementations, the technical functions described herein can be performed, at least in part, by one or more hardware logic components. For example and without limitation, embodiments can include hardware logic components 110, 128 such as field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip components (SOCs), complex programmable logic devices (CPLDs), and similar components. For example, components of embodiments can be grouped into interacting functional modules based on their inputs, outputs, and / or technical effects.

[0203] In addition to processor 110 (e.g., CPU, ALU, FPU, and / or GPU), memory / storage medium 112, and display 126, the operating environment can also include other hardware 128 such as a battery, bus, power supply, wired and wireless network interface cards. The terms “screen” and “display” are used interchangeably herein. Display 126 can include one or more touchscreens, screens responsive to input from a pen or input pad, or screens for output only. In some embodiments, peripheral devices 106 such as human user I / O devices (screens, keyboards, mice, input pads, microphones, speakers, motion sensors, etc.) will be in operable communication with one or more processors 110 and memory.

[0204] In some embodiments, the system includes multiple computers connected through wired and / or wireless network 108. Networking interface devices 128 can provide access to network 108 using network components such as packet-switched network interface cards, wireless transceivers, or telephone network interfaces, which can be present in a given computer system. Virtualization of networking interface devices and other network components such as switches or routers or firewalls can also be present in, for example, software-defined networks or sandboxed or other secure cloud computing environments. A given embodiment can also communicate technical data and / or technical instructions through direct memory access, removable non-volatile storage media, or other information storage retrieval and / or transmission methods.

[0205] Those of skill in the art will appreciate that the foregoing aspects and other aspects presented herein under “Operating Environment” can form part of a given embodiment. The headings of this document are not intended to strictly categorize features as sets of embodiment and non-embodiment features.

[0206] One or more items are illustrated in phantom or listed in parentheses to emphasize that they are not necessarily part of the operation environment or all embodiments shown, but are merely present in the environment or interact with items in the environment or some embodiments as discussed herein. In any of the figures or any of the embodiments, items not in outline or parentheses form are not necessarily required. In particular, Figure 1 provided for convenience; Figure 1 The inclusion of an item in a document of composition should not be construed as a representation that the item is or will be known in the art prior to the filing date of the present innovation.

[0207] More about the system

[0208] With reference to Figures 1 to 8 Some embodiments use or provide a functionally enhanced system 400. The functional enhancement facilitates container instantiation efficiency by providing technical mechanisms that can (a) instantiate containers at instantiation locations that are remote from a container registry without first downloading all layer content to the instantiation location, (b) avoid downloading layer content that does not belong to a particular container not only before instantiation but even after a container process has started, and (c) avoid various technical limitations and requirements of existing approaches, including items or circumstances such as loopback mounting, snapshots, container image cloning, virtual machine setup, container system vendor lock-in, local storage insufficiency, and storage driver plug-ins. Some of these technical mechanisms include layer mounting, on-the-fly downloading of layer content, storage of layer content in each layer file rather than each image file, and data structure formats that conform to industry specifications known as the Open Container Initiative (OCI) (see, e.g., opencontainers dot org). The teachings presented herein can be selectively applied and mixed in various ways. Not every embodiment provides the same set or range of container instantiation efficiency, and other efficiencies and advantages not listed in this paragraph can also be provided by some embodiments. Moreover, not every embodiment provides or uses the same set of technical mechanisms in the same way, and other technical mechanisms not listed in this paragraph can also be used in some embodiments.

[0209] Figure 2 Figures illustrate a container instantiation computing environment 100, 202 and a container registry computing environment 100, 208. While shown with separate blocks for clarity, in a given implementation, the environments 202 and 208 can overlap or one environment can include the other.

[0210] The illustrated instantiation environment 202 includes an instantiation location 204 where one or more containers 210 are to be instantiated 920. The instantiation location 204 has a specialized instantiation technology, which can include specialized hardware 128 or can include general-purpose hardware 110, 112 that is configured with data 118 and instructions 116 to perform container instantiation operations as taught herein.

[0211] The illustrated registry environment 208 includes a container registry 206, and thus implicitly includes the repository 708 referenced by the registry 206. The repository 708 stores container images 212. Each image includes layers 214. Typically, the layers 214 stored in the registry environment 208 are static, and a dynamic (writable) layer 214 is added to the static layers 214 at the instantiation location 204 when a container is instantiated. The instantiation environment 202 and the registry environment 208 communicate through container instantiation communications 216.

[0212] Figure 3 Some examples of container instantiation environments 202 are shown. The shown examples include a network 108, a cloud 302, a virtual machine 304, and a production environment 306. However, those skilled in the art will recognize that containers 210 can also run in other environments, and some of these environments can overlap, e.g., a virtual machine 304 can reside in a cloud 302.

[0213] Figure 4 A system 102, 400 configured for fast container instantiation is illustrated; instantiation includes launching of a container process 402 that is configured. The system 400 includes a union file system 404 that spans layers 214 of an image 212 of an instantiated container 210. The layers 214 are identified in an image manifest 408. Content 406 of at least one layer 214 is accessible through a layer load 410; other layer content 406 can be stored locally at the instantiation location 204. The container can also include a set of changes 412 that describe the file system of the layers 214, an image configuration 416, or both, in accordance with OCI or in other contexts.

[0214] Figure 5 Some container instantiation communications 216 are illustrated. The shown examples include an image manifest 408, a layer load 410, a container image pull request 502, and a layer load request 504. In Figure 7 and Figure 8 Some other instantiation communications 216, 406, 410, 702, 718, 720, 722, 724, 726, 728, 730, 802, 804 are shown in

[0215] Figure 6Some relationships 600 between layer loads 410 and one or more static container layers 214 are illustrated. The illustrated example includes a one-to-one relationship 602 (one layer load to one static layer) and a one-to-many (but not all) relationship 604 (one layer load to multiple static layers of a container but not all static layers). Some conventional approaches put all static layers 214 of a container into a single image file 1040 in a one-to-one relationship not shown in Figure 6

[0216] Figure 7 A container instantiation method is illustrated in which all content 406 of a layer 214 is downloaded 702 without using layer loads 410. In some cases using the illustrated method, container instantiation can only occur after all static layers are downloaded, but in other cases, less than all static layers are downloaded prior to instantiation and at least one layer load 410 is used in place of pre-instantiation downloads. Thus, while Figure 7 Some aspects are consistent with some strictly conventional approaches to container instantiation, but Figure 7 Some aspects are also consistent with some non-conventional innovative approaches to container instantiation, and in other cases the suggestion is misleading.

[0217] In the illustrated method and architecture, the local system 102 used as the instantiation location 204 has an image cache 704 and a layer cache 716. In alternative embodiments, one or both of these caches is not present at the local system. Figure 7 The illustrated image cache 704 includes zero or more image entries, one of which is shown for illustration purposes. The illustrated image cache entry includes an image ID 706, a repository ID 710, tag fields 712 for metadata such as version number, author, license, revision history, developer notes, etc., and a size field 714. In alternative embodiments, one or more of these entry fields is not present, and additional fields can also be present in the image cache entry.

[0218] The illustrated layer cache 704 includes zero or more layer entries, one of which is shown for illustration purposes. The illustrated layer cache entry includes a layer ID 506. In alternative embodiments, additional fields can also be present in the layer cache entry, and locally resident layer content 406 can be present in the cache. Figure 7

[0219] Figure 7

[0220] Figure 7 ​​​​A registry 206 is also shown with one or more member repositories 708. For convenience, when an item is actually stored at a repository after the item is located by data stored at the registry, the item stored at the registry can be referenced. In this sense, the illustrated registry holds compressed container layers 732, 214. The registry also holds container manifests 408; these are typically stored locally at the registry itself, rather than in the repositories.

[0221] Figure 7 Data movement is also illustrated from the perspective of the local system 102, 204 in the form of container instantiation communications 216. Thus, Figure 8 Sending 718 container image pull requests 502 is shown explicitly and receiving 1010 pull requests is shown implicitly, sending 1012 container image manifests 408 is shown implicitly and receiving 720 manifests is shown explicitly, sending 722 container delta layer 724 requests is shown explicitly and receiving delta layer requests is shown implicitly, sending layer URLs 728 is shown implicitly and receiving 726 delta layer URLs is shown explicitly, sending 730 requests for layer content 406 is shown explicitly and receiving layer content requests is shown implicitly, and sending layer content 406 is shown implicitly and receiving 702 layer content 406 is shown explicitly. Those skilled in the art understand that sending and receiving are exchanged when the perspective is changed between the local system 102, 204 and the registry 206. That is, a communication 216 that is a send operation from the local system perspective is a receive operation from the registry perspective, and vice versa.

[0222] Figure 7 A container instantiation method is illustrated in which most or all of the content of a layer 214 is downloaded 702 only after a container that includes the layer has been started. In addition to some of the items and operations discussed above in connection with Figure 8 Figure 8 Expanded (uncompressed) layer content 406 is shown, referred to for convenience as expanded layer 806. Figure 9 Sending 802 layer load requests 504 explicitly and receiving 1014 the request implicitly is also shown, and sending 1016 layer loads 410 implicitly and receiving 804 layer loads explicitly is shown.

[0223] ​Some embodiments use or provide a container instantiation system 400, 102 that includes a memory 112 and a processor 110 in operable communication with the memory at an instantiation location 204. The processor is configured to perform the steps of: (a) sending 718 a container image pull request 502 to a container registry 206, the pull request identifying 706 a container image 212, (b) receiving 720 an image manifest 408, the image manifest 408 listing 902 a plurality of layers 214 that collectively make up 904 the container image, each layer having a respective layer identification 506 and respective layer content 406, (c) sending 802 a layer load request 504 to the container registry, the layer load request identifying 506 a layer of the container image, (d) receiving 804 a layer load 410, the layer load 410 providing access 906 to the layer content in the registry-resident form identified in the layer load request without providing access to all of the content of all of the layers that collectively make up the container image, (e) creating 908 a union file system 404 that spans 910 the layers that collectively make up the container image, and (f) based on 912 the union file system, starting 916 a container process 402 at the instantiation location without first obtaining 914 all of the content of all of the layers that collectively make up the container image at the instantiation location. In some embodiments, the memory 112 includes at least (e.g., contains, is configured by) the image manifest 408 and the union file system 404. Those skilled in the art will recognize that the union file system implements union loading for other file systems and can be implemented using software such as unionfs software, aufs software, overlayfs software, or other software that provides union loading functionality.

[0224] In these embodiments, the skilled person will recognize that the layer load is not a snapshot ID because the layer load provides access to the content of the layer identified in the layer load request in the registry-resident form without providing access to all of the content of all of the static layers that collectively make up the container image. A snapshot ID provides access to all of the content of all of the static layers of the container.

[0225] Those skilled in the art will also understand that the container process 402 is started without first downloading and extracting all of the static layers 214 because the container process is started "without first obtaining all of the content of all of the layers that collectively make up the container image at the instantiation location."

[0226] Some embodiments support dynamically flattening certain layers, but still allow the 1 : 1 relationship 602. In some embodiments, multiple layer loads 410 are received at the instantiation location 204, and each of the received layer loads corresponds to one static layer 214 of a container image 212 in a one-to-one manner 602. In some embodiments, the received layer loads 410 correspond to multiple static layers of a container image in a one-to-many manner 604, but the received layer loads do not correspond to all of the static layers that collectively make up 904 the stored container image.

[0227] Some embodiments support hybrid file system overlays. In some embodiments, the launched union file system 404 includes a layer load 410 represented here as a local load that provides access to unencapsulated locally-resident layer content 406 that resides in a local storage device 106, 112 local to the instantiation location 204, and the launched union file system 404 also includes a layer load 410 represented here as a registry load that provides access 906 to unencapsulated registry-resident layer content 806 that resides in a registry storage device 112 that is non-local to the instantiation location, but can also be local to, for example, a registry server or a repository server.

[0228] In some embodiments, the local system can launch 916 a container 210 whose image 212 is too large to be stored locally. Assume that prior to sending 718 a container image pull request 502 to the container registry 206, the amount of storage available to hold a container from instantiation of a container image 212 is no more than F at the instantiation location 204. Despite the apparent lack of sufficient space, the use of layer loads 410 allows the container to be launched. That is, even though the container image 212 occupies more than F amount of storage at the container registry 206 (or, implicitly, the registry's repository 708), the container is instantiated at the local location 204.

[0229] In some embodiments, the union file system 404 unifies different instances of a single file system 414, but in other embodiments, the union file system 404 unifies different file systems 414. That is, in some embodiments, each of the multiple layers 214 of a container image 212 has a respective file system 414, and the union file system 404 combines at least two layer file systems 414 that are different from one another, for example, one can be a ZFS® (Zettabyte File System) file system 414, and the other can be a NTFS® (New Technology File System) file system 414. Or, the file systems 414 can differ in the sense that they are based on different distributions.

[0230] Some embodiments are actively compatible with open container. In some embodiments, the storage 112 is configured by the change set 412 describing the file system 414 of the layer, and the storage 112 is also configured as the configuration 416 of the container image 212 describing the ordering of the plurality of layers of the container image. Unless otherwise noted, it should be understood that the change set 412 and the configuration 416 are each compatible with the most recently published version of the open container active specification of the respective item available at the priority date of the present disclosure under applicable patent laws.

[0231] Other system embodiments are also described herein, either directly or derivable as system versions of the described processes or configured media, informed by the extensive discussion of computing hardware herein.

[0232] Although specific architectural examples are shown in the figures, embodiments can depart from these examples. For example, items shown in different figures can be included together in embodiments, items shown in the figures can be omitted, functionality shown in different items can be combined into fewer items or a single item, items can be renamed, or items can be connected differently to one another.

[0233] Examples are provided herein to aid in illustration of aspects of the technology, but the examples given herein do not describe all possible embodiments. Embodiments are not limited to the specific component names, optimizations, communication sequences, environments, algorithmic choices, data, data types, configurations, implementations, arrangements, displays, features, methods, or scenarios provided herein. Given embodiments can include, for example, additional or different technical features, mechanisms, sequences, data structures, or functionality, and can depart from the examples provided herein in other ways.

[0234] Processes (also referred to as: methods)

[0235] Figure 10The method 900 is illustrated, which is an example of a method that can be performed or assisted by the enhanced system 400. The illustrated method includes (a) sending 718 a container image pull request 502 to the container registry 206, the pull request identifying 706 a container image 212, (b) receiving 720 an image manifest 408, the image manifest 408 listing 902 a plurality of layers 214 that collectively make up 904 the container image, (c) sending 802 a layer load request 504 to the container registry, the layer load request identifying 506 a layer of the container image, (d) receiving 804 a layer load 410, the layer load 410 providing access 906 to some but not all of the contents of a static layer that collectively make up the container image, (e) creating 908 a union file system 404 that spans 910 the layers that collectively make up the container image, and (f) starting 916 a container process 402, the container process 402 being located 918 at an instantiation location and based 912 on the union file system, while avoiding transferring 914 at least some layer contents prior to starting the process 402.

[0236] Figure 9 Further illustrated is a method (also referred to as a process) adapted for use during container instantiation 920, the method including Figure 9 refinements, supplements, or contextual actions of the illustrated steps. Unless otherwise indicated, the technical processes illustrated in the figures or otherwise disclosed are to be performed automatically (e.g., by container system software or cloud 302 infrastructure software). The processes can also be performed partially automatically and partially manually to the extent that actions involving a human administrator or other person are involved, e.g., in some embodiments a person can specify which union file system software is needed. No process disclosed herein is intended to be entirely manual.

[0237] In a given embodiment, zero or more of the illustrated steps of the process can be repeated, possibly with different parameters or data being operated on. Steps in embodiments can also be performed in a different order than the top-down order illustrated in Figure 10 and Figure 8 The steps can be performed in series, in partially overlapping fashion, or entirely in parallel. In particular, the flowchart 900 action items or flowchart 1000 action items are traversed to indicate the order of steps performed during a process can vary from one execution of the process to another execution of the process. The flowchart traversal order can also vary from one process embodiment to another process embodiment. Steps can also be omitted, combined, renamed, regrouped, performed on one or more machines, or otherwise diverge from the illustrated flow if the performed process is operable and complies with at least one claim.

[0238] Some embodiments use or provide a method for instantiating a container at an instantiation location in a computing system, the method comprising: sending 718 a container image pull request 502 to a container registry 206, the pull request identifying 706 a container image 212; receiving 720 an image manifest 408, the image manifest listing a plurality of layers 214 that collectively make up 904 the container image, each layer having a respective layer identification 506 and respective layer content 406; sending 802 a layer load request 504 to the container registry, the layer load request identifying 506 a layer of the container image; receiving 804 a layer load 410, the layer load 410 providing access 906 to the layer content in the form resident at the registry identified in the layer load request, without providing access to all content of all layers that collectively make up the container image; creating 908 a union file system 404, the union file system 404 spanning 910 the layers that collectively make up the container image and based on 912 the received at least one layer load; based on the union file system, starting 916 a container process 402 at the instantiation location; and prior to starting the container process, avoiding 914 transferring at least a portion of the layer content of the identified layer from the container registry to the instantiation location over a network 108.

[0239] Some embodiments provide 1006 a significant performance gain 1008. In some embodiments, a first time 1004, represented here as a new time, measures 1002 the elapsed clock time at the time of sending 718 the container image pull request, at the time of receiving 720 the image manifest, at the time of sending 802 one or more layer load requests, the layer load requests collectively identifying all layers of the container image that are not stored locally at the instantiation location, at the time of receiving 804 one or more layer loads in response to the one or more layer load requests, and at the time of completing the creation of the union file system based on the received at least one layer load. A second time 1004, represented here as an old time, measures 1002 the elapsed clock time at the time of sending 718 the container image pull request, at the time of receiving 720 the image manifest, at the time of downloading 702 from the container registry to the instantiation location the layer content of all layers of the container image that have not yet been stored locally at the instantiation location, at the time of decompressing 1022 any downloaded layer content that was downloaded in compressed form, and at the time of creating 908 the localized union file system that spans the layers that collectively make up the container image. In some of these embodiments, the old time is at least 15 times the new time. In some embodiments, the old time is at least 30 times the new time. In some cases, the old time is 15 to 30 times the new time.

[0240] Some embodiments efficiently share a layer cache memory across different containers. In some embodiments, the method includes storing 1024 layers of content 406 in cache 716 and sharing 1028 cached layers of content between two different container processes 402. That is, a single cached copy of the static layer content is accessed by each of two or more different container processes 402.

[0241] In some embodiments, a first execution of the container instantiation method 920 includes: receiving a layer load 804, the layer load providing access 906 to the contents of a registry-resident form of the layer identified in the first layer load request, but not providing access 908 to the contents of all layers that collectively constitute the first container image; creating 908 a first union file system spanning 910 the layers that collectively constitute the first container image 904; and starting a container A process 402 at a first instantiation location based on the first union file system. A second execution of the container instantiation method 920 includes: receiving the same layer load 804, the layer load providing access 906 to the same contents of the same layer identified in the second layer load request, but not providing access to the contents of all layers that collectively constitute the second container image 904; creating 908 a second union file system spanning the layers that collectively constitute the second container image; and starting a container B process 402 at a second instantiation location based on the second union file system. The first union file system is also based on layer 214, denoted herein as layer A, which is not part of container B, and the second union file system is also based on layer 214, denoted herein as layer B, which is not part of container A. Therefore, this embodiment relates to a single instance sharing 1028 layers of content between two containers, while also providing each of the two containers with layer content that is not part of the other container.

[0242] In some embodiments, the communication and operation as taught herein, as well as the use of data structures and software, result in one or more of the following: avoiding 1030 using loopback load 1031 to extend snapshot 1044; avoiding 1032 cloning container images to obtain copies of layer content to be downloaded; avoiding 1034 storing virtual machine settings 1036 in static layer content 406; storing each layer of the container 1038 in a file 1040 different from the other layers of the container; or avoiding 1052 using storage driver plugins 1054 to populate the container with layer content. In some embodiments, the method avoids 1052 sending layer load requests 802 using any storage driver plugins and also avoids 1052 receiving layer loads 804 using any storage driver plugins.

[0243] Some embodiments check to determine which layers 214 are already in the local cache 716 or other local repositories 1046 to avoid unnecessary content 406 downloads. Some embodiments include determining that layers L listed in the image manifest 1046 are stored locally at the instantiation location, and omitting the identifier of layer L 1050 from any layer load request sent to the container registry as part of instantiating the container.

[0244] In some embodiments, mounting of an extended or compressed file system format may be provided. In some embodiments, the method includes: receiving 804 a first-layer mount 410, the first-layer mount 410 providing access to first content of a decapsulated registry-resident form 806 belonging to a first layer of a container image; and receiving 804 a second-layer mount 410, the second-layer mount 410 providing access to second content of a encapsulated registry-resident form 732 belonging to a second layer of a container image.

[0245] Configured storage media

[0246] Some embodiments include a configured computer-readable storage medium 112. Storage medium 112 may include a disk (magnetic, optical, or other), RAM, EEPROM or other ROM and / or other configurable memory, particularly including computer-readable storage media (which are not only transmitted signals). Specifically, the configured storage medium may be a removable storage medium 114 such as a CD, DVD, or flash memory. General-purpose memory may be removable or non-removable, and may be volatile or non-volatile. General-purpose memory may be configured to use embodiments of items in the form of data 118 and instructions 116 read from the removable storage medium 114 and / or another source such as a network connection (e.g., layer loading 410, change set 412, image configuration 416, container instantiation communication 216, union file system 404, and implementations). Figure 9 The data stream shown or Figure 10 and Figure 9 The software of the method shown forms a configured storage medium. As disclosed herein, the configured storage medium 112 enables computer system 400 or 206, or both, to perform technical process steps for container instantiation 920. The accompanying drawings thus help to illustrate embodiments of the configured storage medium and processes (also referred to as methods), as well as system and process embodiments. Specifically, Figure 10 or Example Any process steps shown or taught herein may be used to help configure the storage medium to form a configured storage medium embodiment.

[0247] Some embodiments use or provide computer-readable storage media 112, 114 configured with data 118 and instructions 116 that, when executed by a processor 110, perform a registry method that supports container instantiation 920 at an instantiation location 204 in a computing system. The method includes: a container registry 206 receiving 1010 a container image pull request 502, the pull request identifying a container image 212; the container registry sending 1012 an image manifest 408, the image manifest listing a plurality of layers that collectively make up 904 the container image, each layer having a respective layer identification 506 and a respective layer content 406; the container registry receiving 1014 a layer load request 504, the layer load request identifying a layer of the container image; the container registry sending 1016 a layer load 410, the layer load 410 providing access to a registry-resident form of content of the layer identified in the layer load request without providing access to all content of all layers that collectively make up the container image; and the container registry avoiding 914 sending, to the instantiation location remote from the container registry, at least a portion of the layer content of the identified layer prior to launch of a container process based on the container image at the instantiation location.

[0248] In some embodiments, a plurality of layer loads are sent 1016 from the container registry, and each said layer load corresponds 602 to one layer of the container image in a one-to-one manner.

[0249] In some embodiments, the method further includes: storing 1024 the layer content in a cache, and sharing 1028 the cached layer content between two different container processes.

[0250] In some embodiments, the method is characterized by at least one of the following ways: the method avoids cloning 1032 the container image in response to the layer load request; each layer content is free of virtual machine settings 1036; each layer content is accessible 1042 without accessing any snapshot 1044; or, each layer of the container is stored 1038 in a different file 1040 from another layer of the container.

[0251] In some embodiments, the method includes: the container registry sending 1016 a first layer load, the first layer load providing access to a first content that belongs to an un- encapsulated registry-resident form 806 of a first layer of the container image; and the container registry sending 1016 a second layer load, the second layer load providing access to a second content that belongs to an encapsulated registry-resident form 732 of a second layer of the container image.

[0252] Additional embodiments and observations

[0253] Those skilled in the art will recognize that not every part of this disclosure or any specific detail thereof must meet legal standards such as enablement, written description, or best mode. Furthermore, embodiments are not limited to the specific networks, tools, identifiers, fields, data structures, functions, constants, or other implementation choices described herein. Any apparent conflict with any other patent disclosure, even by the owner of this invention, shall not be effective in interpreting the claims set forth in this patent disclosure. With this understanding relating to all parts of this disclosure, some additional examples and observations are provided.

[0254] Some remarks about the snapshots Some implementations reduce the startup time of containers that are not locally cached (e.g., cold-start containers). In one internal implementation, during container fetching, the local system 400 sends... The container registry 206 (a trademark of Microsoft Corporation) communicates with the local system requesting decapsulated content 806 (if available). Registry 206 responds 1016 with a list of mount points 410 (if it is encapsulated) associated with each layer in the image. The local system stores these mount points in a metadata repository 418 and associates them with a list 408 returned from the registry. The local system retrieves these mount points and combines them using a union file system such as aufs or overlayfs 908 to allow the image to run.

[0255] This internal implementation is built to align with Open Container Initiative (OCI), which supports all familiar container image formats. This implementation is not limited to only... It works with container systems (a trademark of Docker, Inc.) and does not require a corresponding storage driver plugin. Some of the methods taught in this article allow developers to further develop and use them at various runtimes, as they are not limited to... The context of container software. Some of the methods taught in this paper can be optimized using specific system calls for the relevant file system subsystems. Some of the methods taught in this paper support hybrid file system 414 overlay 404; for example, one layer 214 in container 210 can be represented as an ext4 file system, another layer can be represented as a public internet file system (CIFS), another layer can be represented as a file system in user space (FUSE), and another layer can be represented as a loopback mount 1031. However, note that using loopback mounts to extend container snapshots 1044 and image clones can be avoided as sources of layer content because layer 214 can be mounted separately using layer mount 410, and layer 214 can be stored 1038 in separate corresponding files 1040. In some embodiments, each layer 214 in the container can be set as a separate mount, or each image can be represented as a separate mount.

[0256] Some additional test results In many (if not all) traditional container contexts, snapshot 1044 represents a checkpoint from virtual machine 304. Therefore, snapshots of containers include additional information 1036 beyond the container itself (such as virtual machine settings and virtual disks). Storage snapshots can also save the active running state from virtual machines or other files created from other programs. Thus, snapshots can be broadly compared to… Compared to some disks (specifically VHD / VHDX format) in Hypervisor 120 (a Microsoft trademark) or another Hypervisor 120, a single snapshot can be used to compose an entire container in a conventional approach. This is reminiscent of a flat filesystem where the entire container is represented by a single file. Layer 214 is flattened so that mounting any layer will provide a complete view of the filesystem that can be used by the container. However, in some cases, frames (such as...) At least one version of the software does not have the concept of flattened layers. When such a framework pulls a mirror image, it obtains all layers and passes each layer to the driver using operations that apply differences. As a result, a computationally expensive clone is produced.

[0257] Conversely, in some embodiments taught herein, load 410 represents one or more layers 214. An embodiment may have the ability to dynamically flatten certain layers, but still allow a 1:1 relationship. In the conventional model, a single snapshot represents all layers 214 of the container in a flat format.

[0258] Furthermore, in some embodiments according to the teachings herein, mount 410 may provide access to extended or compressed file system formats. Specifically, some may support VHD / VHDX / VMStore or extended file systems. A conventional approach only allows VMStore representations from snapshots, which are extended using loopback mounts. By extension, this innovative embodiment does not require any layer content 1036 that extends beyond the container scope (more specifically, beyond the scope of one or more layers). Layer content 406 may be kept without virtual machine settings, virtual machine disks, and storage information.

[0259] Moreover, when traditional approaches represent the entire file system of a container, the approach does not support efficiently sharing 1028 layer cache memory across different containers. For example, consider container A and container B. If they share layer Z, then the traditional approach uses two different snapshots: snapshot F and snapshot G. Snapshots F and G will not be able to share memory because they are different files and thus will perform worse than at least some of the approaches taught herein. According to the teachings herein, containers A and B can directly share 1028 layer Z, where layer Z is represented by a single file (e.g., in VHD / VHDX / other disk form) or many files (in extended form), thus sharing local memory-in-cache from the kernel.

[0260] Moreover, during the load process, at least some of the approaches taught herein do not waste any memory or compute unnecessary copies of the layer content 406. The enhanced system 400 can simply perform one load for each layer; since the load represents one or more layers, there is no waste like in traditional approaches that make or use snapshot clones for multi-layer images. Some of these traditional approaches effectively discard all information from anywhere but the top layer, since the top layer has all the information in flat format. Thus, for any image with a sufficient number of layers, embodiments taught herein will perform better than traditional approaches. For example, internal tests show similar performance with images having fewer layers in the case of a 35-layer image. Even for the 35-layer image, internal tests reveal a thirty to thirty-five speedup factor from traditional downloads from the image 212.

[0261] Conclusions One particular internal test set used a 35-layer jupyter notebook container image 212 that consumes approximately 5.75 GB on disk and approximately 2 GB in compressed form. As a local system, a virtual machine 304 with 8 virtual CPUs, 30 GB of virtual disk, and 32 GB of memory was employed in these tests, with each of two disk options: standard SSD and premium SSD. Without the fast container instantiation enhancements taught herein, instantiation of this jupyter notebook container took 10 minutes 4 seconds with standard SSD and 6 minutes 44 seconds with premium SSD. In stark contrast, for the fast container instantiation enhancements taught herein, instantiation 920 of this jupyter notebook container took 0 minutes 15 seconds with standard SSD and 0 minutes 13 seconds with premium SSD. Thus, in this test, the performance gain for the standard SSD system was a speedup factor of over 40, and the performance gain for the premium SSD was a speedup factor of over 31.

[0262] Some additional combinations and variations

[0263] Any combination of these combinations of code, data structures, logic, components, communications, and / or their functional equivalents can also be combined with any of the systems described above and variations thereof. Processes can include any subset or combination or sequence of the steps described herein that are operable. Each variation can occur alone, or in combination with any one or more other variations. Each variation can occur with any process, and each process can be combined with any one or more other processes. Each process or combination of processes (including variations) can be combined with any of the configurations of storage media described above and variations thereof.

[0264] Figure 9

[0265] In short, the teachings provided herein can be applied to enhance a computing system 102 to make container instantiation 920 faster, reduce layer 214 content 406 storage requirements, and make more container image 212 formats available. A system 102, 400 at a container instantiation location 204 sends 718 a container image pull request 502 to a container registry 206, receives 720 an image manifest 408, sends 802 a layer load request 504 to the registry instead of a layer content download request 730 (which is sent later after instantiation 920 and when and if the content is actually needed), receives 804 a layer load 410, optionally repeats steps 802 and 804 to obtain layer loads 410 for additional layers, creates 908 a union file system 404 spanning 910 the layers, and launches 916 a container process 402 based on 912 the union file system, without first downloading 702 all of the static layer content. Inefficiencies and technical limitations of some other approaches are avoided, such as avoiding 1030 loopback mounts 1031 for snapshot 1044 extensions, avoiding creating or transferring extra snapshots 1044 or extra container image clones 1032, avoiding layer content 406 commingling 1034 with virtual machine settings 1036, avoiding container system vendor lock-in (e.g., system lock-in), avoiding lack of container instantiation 920 at a local system due to insufficient local storage, and allowing system call optimizations due to avoiding 1052 use of storage driver plugins 1054. System lock-in), avoiding lack of container instantiation 920 at a local system due to insufficient local storage, and allowing system call optimizations due to avoiding 1052 use of storage driver plugins 1054.

[0266] Embodiments are understood to also include or benefit from tested and appropriate security controls and privacy controls (such as the General Data Protection Regulation (GDPR)), for example, it should be understood that appropriate security measures should be taken to help prevent misuse of container instantiation and to help avoid tampering with any personal or private information that a running container can handle, for the purpose of injecting or propagating malware. Use of the tools and techniques taught herein are compatible with use of such controls.

[0267] Although specific embodiments are clearly illustrated and described herein as processes, configured storage media, or systems, it should be understood that discussion of one type of embodiment generally extends to other types of embodiments. For example, in conjunction with Figure 10 and ​ The description of the process also helps to describe the configured storage medium and the technical effects and operation of the systems and articles of art, as discussed in conjunction with other figures. There is no requirement to read from one embodiment into another. Specifically, the process is not necessarily limited to the data structures and arrangements presented when discussing systems or articles of art, such as the configured memory.

[0268] Those skilled in the art will understand that implementation details may involve specific code, such as specific APIs, specific fields, specific kinds of components, and specific sample programs, and therefore need not appear in every embodiment. Those skilled in the art will also understand that program identifiers and some other terms used in the discussion of details are implementation-specific and therefore do not need to be addressed in every embodiment. Nevertheless, while the presence of these details is not necessarily required herein, they can assist some readers by providing context and / or may exemplify some of the many possible implementations of the techniques discussed herein.

[0269] Unless such exclusion is expressly stated herein, embodiments having certain features X referenced herein and embodiments having certain features Y referenced elsewhere herein do not exclude embodiments of this disclosure having both features X and features Y. All possible negative claims are within the scope of this disclosure, and in this sense, any feature that is part of an embodiment may be explicitly removed from inclusion in another embodiment, even if no specific exclusion is given in any example herein. The term “embodiment” is used herein only as a more convenient form of “process, system, article of manufacture, configured computer-readable storage medium and / or other example of the application of the teachings herein in a manner consistent with applicable law.” Therefore, a given “embodiment” may include any combination of features disclosed herein, provided that the embodiment is consistent with at least one claim.

[0270] Not every item shown in the figures is required in every embodiment. Rather, embodiments can include items not explicitly shown in the figures. While some possibilities are illustrated by specific examples in the text and figures, embodiments can depart from these examples. For example, a specific technical effect or technical feature of an example can be omitted, renamed, differently grouped, repeated, instantiated differently in hardware and / or software, or a mix of effects or features appearing in two or more examples. In some embodiments, functionality shown at one location can also be provided at a different location; one of skill recognizes that functional modules can be defined in a variety of ways in a given implementation without necessarily omitting a desired technical effect from the overall set of interacting modules. Different steps can be shown together in a single block for reasons of space or convenience, but different steps can still be performed separately, e.g., one step can be performed without another step in a given method execution.

[0271] Reference numbers refer to the drawings throughout. Any apparent inconsistency in a phrase associated with a given reference number in the drawings or text should be understood to simply broaden the scope referred to by that reference number. Different instances of a given reference number can refer to different embodiments, even if the same reference number is used. Similarly, a given reference number can be used to refer to a verb, a noun, and / or to refer to a corresponding instance of each, e.g., processor 110 can process 110 instructions by executing the instructions.

[0272] As used herein, terms such as "a," "one," and "the" include one or more of the items or steps. Specifically, reference to items or steps in a claim means at least one such item or step, and reference to steps means at least one instance of the step is performed. Similarly, "is" and other singular verb forms are to be interpreted as encompassing "are" and other plural forms, as allowed by context, to avoid grammatical errors or misinterpretations.

[0273] Headings are for convenience only; information on a given topic can be found outside the section whose heading indicates that topic.

[0274] All claims and the abstract submitted are part of the specification.

[0275] To the extent any term used herein relates to or otherwise refers to an industry standard, and to the extent applicable law requires identification of a particular version of the standard, the present disclosure shall be understood to refer to the most recent version of the standard that was published in at least draft form (if final form is more recent) as of the earliest priority date of the present disclosure under the applicable patent law.

[0276] While example embodiments are shown in the drawings and described above, it is to be understood that the application is not limited to the disclosed embodiments and modifications are possible without departing from the principles and concepts as set forth in the claims. Although the subject matter has been described in language specific to structural features and / or process acts, it is to be understood that the subject matter defined in the claims is not necessarily limited to the specific features or acts described above. Each device or aspect or technological effect identified in a given definition or example is not necessarily present or utilized in every embodiment. Rather, the specific features and acts, and effects described are disclosed as examples of implementing the claims as contemplated.

[0277] All modifications are intended to be within the scope of the claims and their equivalents.

Claims

1. A container instantiation system, comprising: a memory at an instantiation location; a processor in operable communication with the memory at the instantiation location, the processor configured to perform steps comprising: (a) sending a container image pull request to a container registry, the pull request identifying a container image, (b) receiving an image manifest, the image manifest listing a plurality of layers that collectively comprise the container image, each layer having a respective layer identification and a respective layer content, (c) sending a layer load request to the container registry, the layer load request identifying a layer of the container image, (d) receiving a layer load, the layer load providing access to the content of the layer identified in the layer load request in a registry-resident form without providing access to all content of all layers that collectively comprise the container image, (e) creating a union file system spanning the layers that collectively comprise the container image, and (f) launching a container process at the instantiation location based on the union file system without first obtaining all content of all layers that collectively comprise the container image at the instantiation location; whereby the memory is configured by at least the image manifest and the union file system.

2. The system of claim 1, wherein a plurality of layer loads are received at the instantiation location, and each received layer load corresponds in a one-to-one manner to a static layer of the container image.

3. The system of claim 1, wherein the received layer load corresponds in a one-to-many manner to a plurality of static layers of the container image, but the received layer load does not correspond to all of the layers that collectively comprise the container image.

4. The system of claim 1, wherein the federated file system launched comprises: wherein the layer load is represented here as a local load, the local load providing access to an unencapsulated locally-resident layer content that resides in a local storage device local to the instantiation location, and wherein the launched union file system further comprises: a layer load represented here as a registry load, the registry load providing access to an unencapsulated registry-resident layer content that resides in a registry storage device non-local to the instantiation location.

5. The system of claim 1, wherein prior to sending the container image pull request to the container registry, an amount of storage available at the instantiation location for saving a container from the container image instantiation is no more than F, and wherein the container image occupies more than F amount of storage at the container registry.

6. The system of claim 1, wherein each of a plurality of layers of the container image has a respective file system, and wherein the union file system combines at least two layer file systems that are different from each other.

7. The system of claim 1, wherein the memory is further configured by: a configuration of the container image that describes a set of changes to file systems of the layers, and that describes an ordering of a plurality of layers of the container image.

8. A method of instantiating a container at an instantiation location in a computing system, the method comprising: sending a container image pull request to a container registry, the pull request identifying a container image; receiving an image manifest, the image manifest listing a plurality of layers that collectively make up the container image, each layer having a respective layer identification and a respective layer content; sending a layer load request to the container registry, the layer load request identifying a layer of the container image; receiving a layer load, the layer load providing access to the content of the layer identified in the layer load request in a registry-resident form without providing access to all content of all layers that collectively make up the container image; creating a union file system that spans layers that collectively make up the container image and that is based on at least one received layer load; based on the union file system, starting a container process at the instantiation location; and prior to starting the container process, avoiding transmission of at least part of the layer content of the identified layer from the container registry to the instantiation location over a network.

9. The method of claim 8, wherein a clock time at which a time measurement elapses is represented here as new time at the time of sending the container image pull request, at the time of receiving the image manifest, at the time of sending one or more layer load requests, the layer load requests collectively identifying all layers of the container image that are not stored locally to the instantiation location, at the time of receiving one or more layer loads in response to the one or more layer load requests, and at the time of completing creation of the union file system that is based on at least one received layer load; wherein a clock time at which a time measurement elapses is represented here as old time at the time of sending the container image pull request, at the time of receiving the image manifest, at the time of downloading the layer content of all layers of the container image from the container registry to the instantiation location, at the time of decompressing any downloaded layer content that was downloaded in compressed form, and at the time of creating a localized union file system that spans the layers that collectively make up the container image; and wherein old time is at least fifteen times new time.

10. The method of claim 8, wherein the method further comprises: storing the layer content in a cache; and sharing the cached layer content between two different container processes.

11. The method of claim 8, wherein: a first execution of the container instantiation method includes receiving the layer load that provides access to the content of the layer identified in a first layer load request in the registry-resident form without providing access to all content of all layers that collectively make up a first container image, creating a first union file system that spans the layers that collectively make up the first container image, and based on the first union file system, starting a container A process at a first instantiation location; The second execution of the container instantiation method includes receiving the same layer load that provides access to the same content in the same registry-resident form of the same layer identified in the second layer load request without providing access to all content of all of the layers that collectively comprise the second container image, creating a second union file system that spans the layers that collectively comprise the second container image, and based on the second union file system, starting a container B process at a second instantiation location; wherein the first union file system is further characterized as also being based on a layer load of a layer here denoted as Layer A that is not part of container B, and wherein the second union file system is further characterized as also being based on a layer load of a layer here denoted as Layer B that is not part of the container A; thereby sharing a single instance of layer content between two containers while also providing each of the two containers with layer content that is not part of the other container.

12. The method of claim 8, wherein the method is further characterized by at least one of the following ways: The method avoids using a loopback load to extend a snapshot; The method avoids cloning the container image; Each layer content is free of virtual machine settings; or Each layer of the container is stored in a different file than another layer of the container.

13. The method of claim 8, further comprising: determining that a layer L listed in the image manifest is stored locally to the instantiation location; and and omitting an identification of the layer L from any layer load request sent to the container registry as part of the container instantiation.

14. The method of claim 8, wherein the method avoids using any storage driver plug-in to send the layer load request and also avoids using any storage driver plug-in to receive the layer load.

15. The method of claim 8, wherein the method comprises: receiving a first layer load that provides access to a first content in an un- encapsulated registry-resident form of a first layer that belongs to the container image; and receiving a second layer load that provides access to a second content in an encapsulated registry-resident form of a second layer that belongs to the container image.

16. A computer-readable storage medium configured with data and instructions that, when executed by a processor, perform a registry method that supports instantiation of a container at an instantiation location in a computing system, the method comprising: a container registry receiving a container image pull request that identifies a container image; the container registry sending an image manifest that lists a plurality of layers that collectively comprise the container image, each layer having a respective layer identification and a respective layer content; the container registry receiving a layer load request that identifies a layer of the container image; and the container registry sending a layer load that provides access to the layer content in the registry-resident form of the layer identified in the layer load request. The container registry sends a layer load that provides access to the content of the layer identified in the layer load request in a registry-resident form without providing access to all content of all layers that collectively constitute the container image; and The container registry avoids sending at least part of the layer content of the identified layer to an instantiation location remote from the container registry prior to launch at the instantiation location of a container process based on the container image.

17. The storage medium of claim 16, wherein multiple layer loads are sent from the container registry, and each of the layer loads corresponds to one layer of the container image in a one-to-one manner.

18. The storage medium of claim 16, wherein the method further comprises: storing the layer content in a cache; and sharing the cached layer content between two different container processes.

19. The storage medium of claim 16, wherein the method is further characterized by at least one of the following: The method avoids cloning the container image in response to a layer load request; Each layer content is free of virtual machine settings; Each layer content is accessible without accessing any snapshot; or Each layer of the container is stored in a different file from another layer of the container.

20. The storage medium of claim 16, wherein the method comprises: The container registry sends a first layer load that provides access to a first content in an unencapsulated, registry-resident form that belongs to a first layer of the container image; and The container registry sends a second layer load that provides access to a second content in an encapsulated, registry-resident form that belongs to a second layer of the container image. ​ ​

Images