Container management system with layout manager system

Abstract

Methods, systems, and computer storage media for providing a universal flat file set in a synthetic image that can be mounted as a container (i.e., a synthetic container) to support isolation and interoperability of computing resources. A synthetic image file system engine provides container management for a container management system, the synthetic image file system engine performing synthetic operations to support resource isolation and operating system (OS) virtualization functionality. In particular, a layout manager operates with a synthetic engine interface to support generating a synthetic image with an optimized configuration (i.e., pre-alignment of executable files and pre-computed hashes thereof). In operation, a plurality of files are accessed for generating a synthetic image. The synthetic image is generated for the plurality of files while pre-processing executable files, where pre-processing one or more files includes pre-aligning executable files in the synthetic image or pre-computing hashes of executable files in the synthetic image.

Description

Background Technology

[0001] Users often rely on computing resources such as applications and services to perform various computing tasks. Distributed computing environments can support the building, deployment, and management of applications and services. Users and enterprises are moving away from traditional computing environments and running their applications and services in distributed computing environments. Distributed computing environments implement operating system (OS) level virtualization (e.g., container management systems) to support multiple isolated computing instances within the distributed computing environment.

[0002] For example, a container management system (or platform) with a container engine (e.g., DOCKER) can support the development and delivery of software in packages called containers. Containers are created from images that specify their exact contents. In particular, container management systems can be used to support (e.g., manage, create, mount, and access) containers. A container management system includes several components that are integrated and communicate to provide container functionality. Conventional container management systems primarily support container images with traditional container image layouts. For example, a traditional container image layout may include a directory tree structure on disk, which can lead to inefficient container management operations. As distributed computing environments increasingly support applications and services, it is important to provide operations for efficiently creating and managing container resources to improve computing operations, especially improving computing operations for resource isolation functionality in distributed computing environments. Summary of the Invention

[0003] The technologies described herein generally relate to systems, methods, and computer storage media used to provide a common flat set of files in synthetic images, which can be mounted as containers (i.e., synthetic containers) to support the isolation and interoperability of computing resources. Specifically, container management is provided to container management systems based on a synthetic image file system engine (i.e., the API of the synthetic image file system "CIMFS"). The synthetic image file system engine ("synthetic engine") provides a wide variety of synthetic image file system engine operations ("synthetic operations") that are performed to support resource isolation and operating system (OS) virtualization functionality. For example, the DOCKER platform (a container management system) can integrate with components of the file system (i.e., CIMFS) (e.g., the API and the machine code of the synthetic operations). The API can support different types of CIMFS functionality, such as generating (e.g., via the container engine interface) different types of CIMFS images (i.e., a base synthetic image, a synthetic image with shared functionality, a synthetic image with pre-computed hashes, or a synthetic image with pre-aligned executables). The API and container engine interfaces can further support the execution of compositional operations for container management with the container engine, including mounting different types of CIMFS images and communicating with drivers (i.e., remote interfaces or client interfaces) that support access to or execution from different types of CIMFS images.

[0004] At a higher level, compositing operations are integrated into the container management system to enable the compositing engine to implement the multi-purpose features of the compositing image (i.e., flat file image) (e.g., container engine interface, remote share manager, and layout manager system). The compositing engine can be a library that defines the logic (i.e., algorithms or instructions) for the compositing image file system "CIMFS". This library defines the interface for calling specific behaviors of CIMFS, including instructions on how to create and access a common set of flat files within the compositing image. This logic includes operations (or instructions) that can be performed using the structure of CIMFS (e.g., flat files and namespaces). The compositing engine is shared (i.e., not dedicated to kernel mode or user mode) but can be used by multiple configurations that are resource-isolated and virtualized, and these configurations may not have different types of connections to each other. The compositing engine is organized so that it can be reused by independent programs or subroutines unaware of the library details, while interface handles provide CIMFS functionality (e.g., mounting, sharing, and accessing different resources).

[0005] In operation, the compositing engine performs compositing operations on compositing images with different configurations and host environments. Specifically, the compositing engine provides a wide variety of compositing operations that are performed to support resource isolation and operating system virtualization functionality. The compositing engine comprises several different components, including a container engine interface, a remote share manager, a layout manager, and additional APIs (i.e., the remote interface and the layout manager client interface). These different components can be integrated into the container engine or the host (or host driver or interface) to provide the functionality described herein. Thus, several aspects of the technical solutions described in this disclosure are aimed at improving container management in a container management system based on a compositing engine. The compositing engine, container engine interface, remote share manager, layout manager, and additional APIs (i.e., the remote interface and the layout manager client interface) provide efficient creation and management of compositing images and compositing containers to improve resource isolation functionality in distributed computing environments.

[0006] The present invention is provided in a simplified form to introduce a series of concepts further described in the following detailed description. The present invention is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used to help determine the scope of the claimed subject matter. Attached Figure Description

[0007] The technology described herein will now be described in detail with reference to the accompanying drawings, wherein:

[0008] Figure 1A This is a block diagram of an example container management system environment for providing synthesis operations using a synthetic image file system engine, applicable to the technical aspects described herein;

[0009] Figure 1B This is a block diagram illustrating an example composite (flat file) image applicable to implementing the technical aspects described herein;

[0010] Figure 2A This is an example container management system environment based on the technical aspects described in this article for providing synthesis operations using a synthetic image file system engine;

[0011] Figure 2B This is an example container management system environment based on the technical aspects described in this article for providing synthesis operations using a synthetic image file system engine;

[0012] Figure 3A This document provides example methods for using a synthetic image file system engine to provide synthetic operations based on the technical aspects described herein.

[0013] Figure 3B This document provides example methods for using a synthetic image file system engine to provide synthetic operations based on the technical aspects described herein.

[0014] Figure 4A This document provides example methods for using a synthetic image file system engine to provide synthetic operations based on the technical aspects described herein.

[0015] Figure 4B This document provides example methods for using a synthetic image file system engine to provide synthetic operations based on the technical aspects described herein.

[0016] Figure 5A This document provides example methods for using a synthetic image file system engine to provide synthetic operations based on the technical aspects described herein.

[0017] Figure 5B This document provides example methods for using a synthetic image file system engine to provide synthetic operations based on the technical aspects described herein.

[0018] Figure 6 A block diagram of an example container management system environment suitable for implementing the technical aspects described herein is provided;

[0019] Figure 7 Block diagrams are provided for example distributed computing environments suitable for implementing the technical aspects described in this article; and

[0020] Figure 8 This is a block diagram of an example computing environment applicable to implementing the technical aspects described in this article. Detailed Implementation

[0021] Overview of technical issues, technical solutions, and technical improvements

[0022] Distributed computing environments implement resource isolation and operating system (OS)-level virtualization (e.g., container management systems) to support multiple isolated computing instances within the distributed computing environment. In this context, container management systems (or platforms) (e.g., DOCKER) can support the development and delivery of software in packages called containers. Containers are generally isolated from each other and bundle their own software, libraries, and configuration files; however, they can communicate with each other through well-defined channels. Containers run from a single operating system kernel, which is more lightweight than virtual machines. A single server or virtual machine can run several containers simultaneously. Containers are created from images that specify their exact contents. In particular, container management systems can be used to support (e.g., manage, create, mount, and access) containers that comprise several components that communicate to provide container functionality.

[0023] Conventional container management systems primarily support container images with a traditional container image layout, which introduces an inefficient directory tree structure into container management operations. Specifically, a classic container image consists of layers of containers, where these layers are composed of files arranged on the host file system. When implementing a classic container image (e.g., via a compressed .tar file), these files are extracted. Traditionally, layers are a hierarchical directory structure of files in a tree structure, which are slowly extracted from the .tar and stored on disk. Container management can include operations on container images, including the extraction, manipulation, and access of container images that put pressure on the host operating system supporting the container image. For example, when container extraction is slow, this can cause bottlenecks in container initialization and other operations within the container management system. Therefore, alternative approaches to container management systems that provide extraction, manipulation, and access to container management operations with zero latency improve compute operations for more efficient resource isolation and operating system virtualization functionality.

[0024] Embodiments of this invention provide a simple and effective method, system, and computer storage medium for providing a common flat file set within a composite image, which can be mounted as a composite container to support the isolation and interoperability of computing resources. Specifically, container management is provided to a container management system based on a composite image file system engine (i.e., the API of the composite image file system "CIMFS"). This engine provides a wide variety of composite image file system engine operations ("composite operations") that are performed to support resource isolation and operating system (OS) virtualization functionality. The composite engine comprises several different components, including a container engine interface, a layout manager, and a composite engine remote interface. These different components can be integrated into the container engine or the host to provide the functionality described herein.

[0025] For example, a container engine (e.g., DOCKER) can support APIs and machine code, which provide compositing operations for the host's compositing engine interface, different layouts for compositing images, and remote interfaces to support CIMFS. Specifically, CIMFS works in conjunction with compositing images, which are flat images representing the entire container image and are also composed of multiple flat files. Furthermore, compositing images can be implemented in different types of configurations and different types of compute "host" environments, as discussed in more detail below. In this respect, container management systems with CIMFS and compositing images can operate more efficiently (in several different configurations and compute environments) to meet different compute needs for resource isolation and operating system virtualization.

[0026] In operation, the Container Engine interface supports the creation and mounting of synthesized images or synthesized containers, as well as providing access to and from synthesized images or containers (via the Container Engine). Synthetic images and containers are based on a container image layout designed to support flat files (i.e., a generic flat fileset). The Synthetic Engine interface can further support layering and deduplication during synthesized image generation. A remote share manager operates with the Synthetic Engine interface to support the generation of synthesized images configured for split-layer storage shares, split-layer direct-access storage shares, or dynamic base images. Synthetic images generated based on remote share manager functionality operate with remote interfaces (e.g., host drivers) that support sharing functionality between components of the container system environment (i.e., containers, VMs, and hosts). Remote interfaces support access to synthesized images and implementation of synthesized file system functionality in different compute environments (i.e., host environments). Remote interfaces (e.g., the Synthetic Engine API on the host machine) allow the Container Engine to operate with different types of host environment configurations and functionalities. The remote interface provides operations that support split-layer memory sharing, split-layer direct access to memory, and other types of composite images and functionalities.

[0027] The layout manager system comprises a layout manager and a layout manager client interface. The layout manager operates in conjunction with the composition engine interface to support the generation of composed images with optimized configurations (i.e., pre-alignment and pre-computed hashes) for executables. The composed images generated based on the layout manager's functionality (e.g., optimized composed images) operate with the layout manager client interface, which supports the implementation of optimized configurations when the composed image is mounted as a composition container. Furthermore, the layout manager supports generating composed images while aligning the executable code within the image. The layout manager also supports pre-computed hashes of the binary so that these hashes can be used to verify the binary on the host machine. Images comprising aligned executable code and one or both of the pre-computed hashes can be accessed using the layout manager client on the host machine. The layout manager client interface further supports using the pre-computed hashes to execute the pre-aligned executable code and verify the binary. Thus, the composition engine, container engine interface, remote share manager, layout manager, and additional APIs and machine code (i.e., remote interface and layout manager client interface) provide efficient creation and management of composed images and composition containers to improve resource isolation functionality in distributed computing environments.

[0028] An overview of an example environment using a synthesis engine to provide container management.

[0029] Various aspects of the technical solution can be illustrated through examples and references. Figure 1A , Figure 1B , Figure 2A and Figure 2B To describe. Original reference Figure 1A , Figure 1A A container management system environment 100 (i.e., a technical solution environment "container management system") comprising a synthesis engine 10 and a host 20 is disclosed. The synthesis engine 10 has a container engine interface 12, a remote share manager 14, and a layout manager 16. The host 20 has a remote interface 14X and a layout manager client interface 16X. A container engine 30 runs an instance of the synthesis engine 10X. The container management system may have similar features to the referenced... Figure 6 This paper describes the features and functionality of the container management system environment 600.

[0030] At higher levels, a common flat set of files within a composite image can be mounted as composite containers to support the isolation and interoperability of compute resources. Specifically, container management is provided to the container management system based on composite operations performed by a composite engine that supports resource isolation and operating system (OS) virtualization functionality. The composite engine comprises the aforementioned components, which are integrated into the host running the container engine to provide the functionality described herein.

[0031] Synthesis Engine Interface

[0032] The Compositing Engine Interface 12 manages compositing operations, including creating and mounting compositing images or containers, and (using the container engine) providing access to and from compositing images or containers. Compositing images and containers are based on a container image layout designed to support flat files (i.e., a generic flat file set). The Compositing Engine Interface 12 is responsible for creating compositing images as flat file images. A compositing image (i.e., a flat file image) represents a container composed of a generic flat file set, including one or more object ID files, one or more region files, and one or more metadata (file system) files. Region files contain file metadata and encodings of data used for the image, called object repositories. Each file can be stored as a file object and the file's contents. Each directory can be stored as a file object plus a list of directory entries, and so on. Different region types are stored in different files; some region files store metadata, some store page-aligned file data (for larger files), and some store unaligned file data (for smaller files).

[0033] Object pointers can be stored as "region offsets." For example, a region offset can include a 48-bit byte offset and a 16-bit region index in the region file. Therefore, there can be a maximum of 65,535 regions, yet a new image can be created using at most one region file per type. The object ID file can contain a mapping from the SHA512 digest of each region object to its region offset. These are used for data deduplication during image building: if a new object is being added, and that object already exists in an existing region of the same type (determined by looking up its digest in the object ID file), the object does not need to be persisted again and can be referenced at its existing region offset. The object ID file is not used at runtime. The filesystem file contains region offsets for filesystem objects, including the region offset for the file table directory, the file ID for the root directory, and the region offset for the uppercase table used for string case comparison. Reference Figure 1B , Figure 1B An illustration is provided of a composite (flat file) image 110 having the corresponding features discussed above. The composite flat file image 110 includes an object repository 120 and an object ID 130. The object repository 120 also includes metadata 120A and data 120B. In some embodiments, the object repository may include metadata 140, small data 150, and large data 160, where metadata 140 has region types 142 and 144, as discussed above.

[0034] The Synthesis Engine Interface 12 (e.g., via Synthesis Engine Interface 10X) is also responsible for mounting the synthesized image as a synthesized container (or file system). Synthesis Engine Interface 12 (i.e., as a mount driver) performs the mount operation, which creates a file system instance (i.e., a synthesized container) for the synthesized image. The file system (i.e., the synthesized image mounted as a synthesized container) is created as its own volume, while the synthesized container still has access to the host file system volume. Specifically, Synthesis Engine Interface 12 can be an Application Programming Interface (API) that can be used to mount synthesized images as synthesized containers. The Synthesis Engine Interface supports providing synthesized images as synthesized containers, whereas previously, classic container image files were simply mounted to a location on the host file system.

[0035] Classic container images can be distinguished from synthetic containers by comparing how each type of container is written (i.e., random versus sequential writes). Classic container images use a tree data structure to allocate writes to disk at random locations. Tree-based writes are inherently slower compared to the sequential writes supported when mounting synthetic images. When mounting synthetic images, writes to disk files sequentially to improve the speed of writing to the specified regions.

[0036] Then, the synthesized image, when mounted as a synthesis container (filesystem), can have files associated with the filesystem exposed in its corresponding tree structure. The tree structure is stored as metadata, which is accessed on the synthesis container to display files in the tree structure, even if the files are "flat" on disk. The metadata file includes the directory tree structure and points to the location where the data is stored in the region file. Metadata can include file attributes, timestamps, and extended attributes. This contrasts with the region file, which includes the data; however, the metadata points to the location of the data in the region file. The synthesis engine interface 12 reads the synthesized image file and directory structure metadata written sequentially to present the files to the user in a tree structure format via the interface. Advantageously, when presented in a tree structure format, the files are not modified on disk (i.e., the order is maintained).

[0037] Layering

[0038] The Compositing Engine Interface 12 also supports layering or modifying layered compositing (or flat) images, where existing compositing images can be modified or extended by adding additional region files. New flat images can be built from older flat images by creating links from existing region files to new locations and extending existing regions. Layering supports the portability of compositing files because it is not necessary to create new compositing images from scratch. Existing compositing images can be used to add .NET layers. Layering supports the modularity of compositing images, where compositing images (e.g., metadata, small data, or large data) can be attached to or detached from other compositing images. Layering a compositing image involves adding more region files to an existing set of region files, which can be composited into a new layered compositing image. For a first compositing image that already has a first set of files, a second set of files can be merged to generate a second “layered” compositing image. The second compositing image has new sets of files that can be referenced, and these new sets of files can be operated as layered compositing containers when the second layered compositing image is mounted. The added sets of files can include new volumes, new files, and directories, which can be integrated with or operated independently of the original sets of files. In particular, the metadata file of the new composite file may include a tree structure to support access to the first composite image file and the second composite image file.

[0039] For example, a customer wants to create a new composite image for the .NET Framework. The customer already has a base that the .NET Framework can use. Instead of starting with a new composite image and building a base layer and then adding .NET Framework layers, the customer can use the .NET Framework as its own common (composite image) set of files, and then "composite" the new layered composite image by merging the base layer and the common flat set of files for the new .NET Framework. Layering improves portability because the base layer (common flat set of files) can be reused as needed. Layering also improves how storage space is used because the base layer can be shared between a composite container that uses only the base layer and another composite container that uses both the base layer and the .NET Framework layers. In this respect, composite images can be described as stackable because composite images can consist of completely different individual module parts. It is possible to layer different layers of this common (flat) set of files, either separately on the base layer or on the .NET Framework and the base layer (e.g., an APP layer).

[0040] As layers are added, each new layer's metadata layer stores at least some metadata about the files in the previous layer. While the base layer's metadata file may only know its general flat set of files, the .NET Framework layer's metadata can understand both the first and second layers. The .NET Framework metadata file can use pointers to track the location of files from each different layer. The second-layer's metadata can be used to build a directory structure to view the first and second layers. Each additional stacked layer has its own metadata file to manage further pointers and references to the previous layer.

[0041] Additionally, a single image can contain any number of file systems; this allows container and VM images to be combined into a shared set of regions. To modify an existing file system (e.g., when extracting a container image layer), the modified files and directories are added to a new region file. To reference these new files, existing directories are updated using a copy-on-write approach, by copying them, along with their modifications, to the new region file. The root directory is eventually updated, and a new file system object is constructed to reference it. Finally, a new file system file pointing to this new object can be written. Alternatively, a peer file system can be built using existing region files but starting with an empty root directory. Any data shared between peer file systems will be deduplicated via an object hash file, but does not need to be shared within the file system directory tree.

[0042] Deduplication

[0043] Advantageously, as mentioned, layering can help save storage space because common files between layers are deduplicated. For example, files already existing in the .NET Framework layer in the base layer are not added to the composite image. Pointers are stored in metadata, so files can be shared. Deduplication refers to the ability of the composition engine interface 12 to know what files already exist in the composite image (e.g., using a tracking mechanism) and to add pointers to the file data that needs to be shared. Deduplication can occur when the .NET Framework layer is merged with the base layer. This is particularly useful for container file systems that often have several common small files.

[0044] share

[0045] The compositing engine interface supports sharing compositing image files and can operate in conjunction with remote interfaces running on the host computer. Remote interfaces can refer to the compositing engine API, machine code, or drivers and driver components added to components or programs on the host computer to support understanding and using CIMFS, as discussed in more detail below. Go to Figure 2A and Figure 2B , Figure 2A and Figure 2B Each of these discloses a container management system (e.g., a container management system environment 100, in which the features of the present invention (e.g., container image “CIM” layer volumes, container image file system “CIMFS” layer, isolation points (SILO), filters, and server message blocks (SMB)) and functionality can be implemented. Specifically, Figure 2A Container A 240 is disclosed, which has CIMFS layer 0 volume 210, CIMFS layer 1 volume 220, host volume 230 (including CIM layer 0 230A and CIM layer 1 230B), filter 242 and file system temporary volume 244, and container B 250, filter 252 and file system temporary volume 254. Figure 2B Disclosed are guest VM 240B (based on hypervisor VM) having CIM layer 0 210B, CIM layer 1 220B, host disk 230B, guest VM 240B with filter 242B, CIMFS layer 0 volume 262, CIMFS layer 1 volume 264 and SMB client 270A, and guest VM 250B layer having filter 252B, CIMFS layer 0 volume 266, CIMFS layer 1 volume 260 and SMB client 270B.

[0046] At higher levels, composite images and CIMFS can be used in two different types of host scenarios. In the first host model, Figure 2AThe host supports process-based containers, where the container does not mount the synthesized container but can access the synthesized container files. Process-based containers do not have a guest-to-host transition framework; there is no hypervisor separating these operating system virtualization instances. Thus, the synthesized container is mounted on the host computer, while the container (sharing the same kernel) can access the synthesized container (e.g., using a remote interface on the host). In the second-host model, Figure 2B The host supports isolated VM containers (via hypervisor-based VMs), and composite images can be mounted (i.e., directly accessed) within the VM as composite containers to avoid host-to-host translation for visitors with certain types of data requests to the composite container's region files. In both host scenarios, CIMFS supports file sharing.

[0047] Split-layer memory shared components

[0048] As background, classic containers can share images (i.e., shared memory) to save storage space. Composite images can be built in a first configuration, with all layers in a single composite image. However, composite images can also be built in a second configuration, where each layer has its own composite image (i.e., a common flat file set). This second configuration is called split-layer composite image. Memory sharing is possible when composite images are created with split layers. For efficiency purposes in operating system virtualization, a large number of containers are often implemented on the same host (i.e., memory density). If containers share the same base layer, files are stored efficiently. For example, for a single file used across different containers, a single page is loaded only once. For instance, instead of having a base layer, the .NET Framework layer and the App layer can be combined in a single composite image (with common flat files), each composite image having its own layer with common flat files. Exposing individual layers allows these layers to share memory across composite containers because if the layers were not unpacked, the container engine might not know to allow composite containers to share the same flat files.

[0049] The container engine running the composition engine can manage which composition containers are using a specific layer. The container engine may not support sharing because containers cannot share the same base layer when the layer is not split into two separate composition containers. However, if the layer is split, each container can access and share the same composition container layer. Thus, the first container can open the first file, and the second container can access and use the same file. Split-layer memory sharing can operate specifically on the host machine, where containers (process-based containers) use a general-purpose kernel; however, in user mode, containers have their own filesystem and isolation.

[0050] The compositing engine interface, operating in conjunction with the container engine running on the host, can support split-layer storage sharing for preparing and using compositing images associated with a set of compositing images used for split-layer storage sharing. Each compositing image in the set has a corresponding generic flat file set. This implementation includes separate compositing images for each layer. Each compositing image can be mounted as a read-only file system device and can access the files and directories contained in the container image using a file system interface or driver (e.g., the Windows Container Isolation File System "WCIFS" filter driver). For example, if a single compositing image represents a base, .NET, and application, then each layer is mounted as its own volume. Thus, for example, a first compositing container could use only the base layer, while a second compositing container could use the base layer (shared with the first compositing container) and the .NET layer. In this way, process-based containers can share the same base compositing image and files. This implementation does not preclude having all three layers in a compositing image; however, it allows sharing between compositing containers.

[0051] For example, at runtime, the synthesized image is exposed as a read-only volume, and WCIFS uses this volume as the source image, similar to how WCIFS uses single-layer container images. WCIFS can operate in this way because the base layer of the existing container image corresponds to a regular file system image without special metadata. There are several ways to expose this volume to the container so that WCIFS can use the Server Message Block Protocol (e.g., vSMB). For VM-based containers using existing container image formats, the image file is exposed to the VM via the SMB protocol through a transport used for the synthesized device (e.g., VMBus transport). The contents of the synthesized image can also be exposed in this way by linking a flat image resolver to a dynamic link library "DLL" (e.g., vSMBDLL).

[0052] Split-layer direct access memory shared components

[0053] As background, the above reference can be used specifically when there is a need to improve the security and isolation of virtual machines. Figure 2B The second-host model discussed still uses container images (i.e., VM-based containers or guest operating systems "OS"). In this second-host model, each guest OS has its own OS kernel (i.e., an isolated kernel). This is sometimes referred to as a lightweight VM (a hybrid between VMs and containers). With the guest OS having its own kernel, the split layers of the synthetic image are still used for memory sharing. However, these guest OSs require direct access to the memory shares. The synthetic image can support direct access to the memory shares using a file-sharing interface (e.g., a vSMB client). The region files of the split layers can be shared on the synthetic image mounted on the guest OS.

[0054] Files stored on the split layer experience improved access performance through a composition container mounted within the guest OS. Typically, the guest OS accesses the split layer via a server message block "SMB" client, as the guest OS must translate between the guest OS and the host. The guest OS can share the split layer's memory using a host SMB client on the host, which receives requests for files in the split layer from the guest OS. The file is mapped into the host's memory, and this file memory is directly shared with the guest OS. In this respect, the host and guest OS can use the same memory in the split layer (e.g., pages). Even if the guest OS is operationally isolated, several different composition containers can have split layer memory mapped directly from the host to the guest OS.

[0055] Advantageously, region files of the split layer can be shared here. Once the synthesis container is mounted on the guest OS, any file that is part of the synthesis image can be directly accessed (in storage). In particular, metadata files that are constructively mapped to the guest OS have their own virtual space, meaning that information stored in the metadata files can be efficiently accessed without going through an SMB client. Accessing the metadata file is much faster when directly mapped, eliminating guest-to-host translation at the individual file levels. Note that the first access to the metadata file still requires a guest-to-host translation; however, after the file is loaded on the guest OS, the metadata file is accessed directly. Any region files (e.g., metadata) that are part of the synthesis image file are mapped once to improve read efficiency. Access to the region file data via an SMB client is still required to retrieve the individual data. Nevertheless, any files already in storage (e.g., hot files) can be accessed quickly while also avoiding guest-to-host translation.

[0056] The composition engine can support several different types of remote interfaces, as shown in the shared interface. An interface typically refers to the part of the composition engine that helps programs or applications understand and operate with CIMFS. For example, a traditional VM-based container might have a guest OS that cannot mount composition images. For instance, the traditional VM might not have a driver that supports mounting composition images. At a higher level, the SMB client on the host (e.g., vSMB) can be updated to support mounting composition containers. The SMB client exposes the composition container to the guest VM. Direct access is not possible in this configuration because the zone files are provided across hypervisor boundaries using the SMB client.

[0057] In operation, at runtime, the synthesized image is exposed as a read-only volume, and WCIFS uses this volume as the source image, similar to how WCIFS uses single-layer container images. WCIFS can operate in this way because the base layer of the existing container image corresponds to a regular file system image without special metadata. There are several ways to expose this volume to the container so that WCIFS can use the Server Message Block Protocol (e.g., vSMB). For VM-based containers using existing container image formats, the image file is exposed to the VM via the SMB protocol through a transport of the synthesized device (e.g., VMBus transport). The contents of the synthesized image can also be exposed in this way by linking a flat image resolver to a dynamic link library "DLL" (e.g., vSMBDLL). The advantage of this approach is that it works with traditional container images. This compatibility is important for ensuring that flat images can become a proprietary image format for new container hosts. The main disadvantage is that metadata operations still require explicit guest-to-host communication, so the observed performance improvement is limited. Additionally, this approach only works with VM-based containers and not with silo-based containers.

[0058] Furthermore, traditionally, splitting layers is not necessary for achieving memory sharing within VM containers using direct-access memory because memory sharing in this configuration occurs at the region file level of the image rather than at the individual file level within the image. Containers running directly on the host share memory by sharing individual files, thus requiring splitting layers to increase the opportunity for sharing between layers. If N-1 layers are split and mounted as individual CIM volumes, containers using layers (1...N-1) can only share memory with containers using layers (1...N) of the same image.

[0059] In contrast, a composite image with all split layers can be mounted within a VM container while still using a subset of those layers to share storage with other VM containers. A container using N-1 layers can share storage for all its layers with another VM container using all N layers because they can both directly map region files corresponding to the same N-1 layers. For example, a composite image can be built such that, in addition to mounting each split layer individually, it can also be mounted as a composite of N split layers, allowing for optional storage sharing by using the same composite image in both host-based and VM-container-based environments. In operation, because the image can be built as a composite of split layers, each layer has its own region file, which can be directly mapped across VM boundaries.

[0060] Dynamic base image

[0061] The synthesis engine interface supports the creation of dataless or dynamic base images and can operate in conjunction with a remote interface running on the host to provide access to dataless images. Synthetic images can be built as dataless images because a generic flat fileset can be included in the synthetic image without data files (e.g., small or large data). Instead, the synthetic image uses metadata files from the generic flat fileset to redirect to other files (e.g., files on the host). The metadata file can be a placeholder file. The metadata file is configured to indicate the location of files within the synthetic image's region files, so that placeholder files can be used to locate files in other locations.

[0062] Contextually, process-based containers may want to utilize as much of the host OS as possible (e.g., kernel files). Synthetic images can be used to narrow down the scope visible from the host OS. Placeholder files can be used by containers to share copies of files between the container and the host. This avoids duplication of the host OS. A host can have several different namespaces for the host OS to share some host OS files. Namespaces allow containers to securely access files from the host OS associated with that namespace without sharing the entire host OS. In this way, synthetic containers essentially provide a limited view of the host filesystem. Dataless image operations reduce disk space and storage usage so that image files are not copied to the local synthetic container volume. The nature of dataless images is that they are immutable, so the directory tree of the content can be pre-computed and stored in metadata area files. The pre-computed metadata information of the content circumvents any processing or storage of data files. Dataless images can be created on the host or pre-built in a build lab. Synthetic containers access metadata at runtime to request access to data stored in the metadata. To make a read request, the synthetic container uses re-parse points and the host filesystem to access the requested data.

[0063] Even though the guest OS can use its own base image (e.g., .NET and APP layers) separate from the host OS, there are situations where the guest OS might need files from the host OS. Files in the host OS can be assigned namespaces, and a composite container (a dataless image) can be mapped to these namespaces. The guest OS then mounts the composite container and uses it (as a file system) to access the namespace of the files in the host OS. Data requests using placeholders can use guest-to-host translation to access files in the namespace.

[0064] Advantageously, synthetic containers can be used to virtually change the attributes of files on the host OS. Metadata files can be used to change filenames or paths, so accessing files through the synthetic container's file system differs from accessing them on the host OS. For example, files on the host may be accessible without any restrictions, but on the container or guest OS, the security of specific files can be enhanced, and only trusted access can be granted. Other types of changes can be made to files using the metadata files within the synthetic container.

[0065] Executable mirror data alignment

[0066] The layout manager can support file build operations (e.g., executable image data alignment and code integrity pre-computation) to create composite images, and the composition interface engine can operate with remote interfaces running on hosts that support runtime functionality. Contextually, the storage (or layout) of a file with executable code on disk (disk alignment) can differ from its layout in memory (memory alignment). This is called data alignment, i.e., how each individual part of the data is laid out on disk. Data alignment on disk can differ from data alignment in memory. In particular, on-disk (data) alignment may be smaller than in-memory (data) alignment, where data alignment is associated with sections of executable code. For example, a new page is used for a given number of bytes (e.g., 512 bytes). If a section stops midway through a page, the remaining portion of the page is filled, and a new section begins on a new page. The memory manager (e.g., the dynamic linker) is responsible for realigning files in memory from disk alignment to memory alignment.

[0067] On the one hand, composite images contain many executable files, and storing each of these files with smaller page alignments can be beneficial. For example, the base OS includes several DLL files and the .NET Framework, while the app includes executable files, with each executable having additional padding as each section ends before the end of the page. On the other hand, smaller page alignments on disk were originally designed to save space (e.g., saving space on consumer machines). Currently, in commercial scenarios, the focus is more aligned with the efficiency of performing computational operations (e.g., computation speed). The inefficient operation of the memory manager to realign files from disk alignments to memory alignments does not alleviate the efficiency problem. In commercial environments, bypassing realignment takes precedence over the storage benefits achieved through smaller disk alignments per page.

[0068] The composite file is configured to have the same disk alignment as the memory alignment in the memory where the executable code will be executed. The disk alignment can be larger than a traditional disk alignment, with each page padded to meet the disk alignment requirements. However, significant efficiency is achieved because the file is not remapped when it is loaded into memory. Since the file in the composite image is already in memory alignment, instructions are passed to the memory manager to bypass the mapping from disk alignment to memory alignment. Advantageously, for OS virtualization, boot time metrics are improved. Thus, even with a decrease in storage efficiency, boot time is improved, where bottlenecks typically occur because the memory manager realigns from disk alignment to memory alignment.

[0069] In another scenario, the composite image comprises an executable image (PE binary). A PE binary can consist of several headers and sections that instruct the memory manager (e.g., the dynamic linker, which loads the shared libraries required by the executable at runtime and links them into RAM) how to map the PE binary into memory. When the PE is loaded into memory for execution, each binary section must be aligned to the binary's section alignment format. For example, the system page size is 4KB. However, when stored on disk, these sections are aligned only according to the binary's alignment format. For example, each section is 512 bytes. When the binary is loaded from disk, the memory manager reads the binary sections from their locations on disk into the appropriately aligned memory.

[0070] This means that if a typical binary file is mapped directly from its disk representation into storage, it cannot be executed in-place but needs to be copied to a new location. This is not a problem for existing container image formats; when a container needs an image mapping for a binary file, the host storage manager can create such a mapping and provide it to the container, thereby performing deduplication on this mapping across all containers on the system.

[0071] However, this presents a problem for composite images because the host storage manager cannot parse the PE binaries from the middle of the region files. In other words, the region files are inaccessible to the PE binaries. While individual filesystem drivers can build compatible image mappings at runtime, deduplication across all containers on the system is difficult. To address this, the PE binaries are parsed and stored in region files during flat image building, with their sections aligned to allow for in-place execution. Parsing PE binaries in this way doesn't allow for direct mapping to data files, but normal data reading can be performed by reverse unrolling at runtime. One difference between this and existing solutions is that the host no longer relocates the binaries before mapping them to the guest OS.

[0072] Code integrity

[0073] The layout manager can support file build operations (e.g., performable image data alignment and code integrity pre-computation) to create composite images, and the composition interface engine can operate with remote interfaces running on host machines that support runtime functionality. Code integrity of the composite image is provided by pre-computing hashes of the binaries within the composite image. These pre-computed hashes are stored within the composite image. The pre-computed hashes are accessible within the composition container to ensure code integrity. For example, the pre-computed hashes are provided to the code integrity (CI) driver as a pre-computed hash extension attribute of the binaries. In this way, hashing the composite image bypasses the pre-caching of validation results for extension attributes stored as binaries.

[0074] Code integrity is used to verify OS components (e.g., kernel components) that the OS wants to execute in a binary file. Verifying a binary file means verifying that the binary file is signed. Different binaries have different signing requirements (e.g., kernel system state versus normal system processes). Verifying a signature as part of code integrity can be challenging. Specifically, the entire executable file must be read and a hash of that executable file must be generated. This hash is then verified using a binary signature (e.g., a signature embedded in the binary file). In other cases, a directory file with a list of hashes can be used instead. The binary file is associated with extended properties used to store the verification results. Accessing these extended properties to store cached verification results (e.g., before executing the binary file) can avoid performing the expensive verification process.

[0075] Containers used on a host machine are typically not intended for long-term operation. Furthermore, containers rely on rapid startup times to quickly deliver the functionality they support (e.g., applications or services). The host machine's operation, designed to support containers, cannot adequately support regular PC file system caching for code integrity. To mitigate this issue, existing classic containers pre-validate each binary and cache the results on the host file system.

[0076] For synthesized images, caching for code integrity is not straightforward because the caching code operates in kernel mode and expects to receive handles to binary files; it cannot operate on region segments. Worse still, the caching code needs to access directory files to find the signatures of most container image binaries, and these directory files are not accessible via ordinary file system APIs during container image import. Moreover, even for classic container images, this caching scheme has problems. The extended attributes storing cached results are versioned, and the CI driver in older guest OSes may not be able to interpret newer cached results generated by the host CI driver. Additionally, the CI driver uses various system inputs, such as the USN log ID, to determine the validity of the cache, and these inputs are difficult to reliably synchronize between the container and the host.

[0077] To maintain acceptable CI performance for synthesized images, a new approach is needed. A fully pre-cached approach, independent of signature verification, is proposed, where only the hash of each binary image is pre-computed and stored within the synthesized image, made available to the container CI driver via a new extended attribute. During container startup, the CI driver can verify the binary's signature based on this hash, but it doesn't need to access every page of the binary or spend significant CPU time calculating the hash. Since the image hash is stable and easily computed across CI driver versions and doesn't need to be invalidated due to any changes in the container's OS state, it never needs to be computed at runtime. This hash calculation handles approximately 85% of the uncached CI verification process, so its pre-computation should be sufficient to allow the removal of existing pre-caching schemes.

[0078] Furthermore, once this hash-accelerated CI verification occurs, the CI caching scheme can continue to operate normally on the container staging volume, so subsequent use of a given binary does not require re-evaluation of the signature. Caching only on the staging volume is more reliable and correct because its invalidation semantics are simply a function of the container's operating system state.

[0079] Aspects of the technical solutions disclosed herein have been described with reference to container management components, such as a composition engine with a composition engine interface, a remote share manager, and a layout manager, wherein the composition engine operates in conjunction with the remote interface and the layout manager client interface to provide the functionality described herein. Specifically, the composition engine provides composition operations for container management using a container engine. These operations include mounting different types of CIMFS images and communicating with drivers (i.e., remote interfaces or client interfaces) that support the execution of different types of CIMFS images. Composition operations are performed to support resource isolation and operating system (OS) virtualization functionality. In general, practical application of several aspects of the described technical solutions results in improvements based on less CPU computation, smaller memory requirements, increased container management efficiency, and greater flexibility. Used for synthetic mirror images A component system engine provides an exemplary approach to container management.

[0080] refer to Figure 3A , Figure 3B , Figure 4A , Figure 4B and Figure 5A as well as Figure 5B A flowchart is provided illustrating methods for providing container management based on a synthetic image file system engine. These methods can be executed using the container management environment described herein. In embodiments, one or more computer storage media containing computer-executable instructions thereon can cause one or more processors to execute the methods in the container management environment when executed by one or more processors.

[0081] Go to Figure 3A The document provides a flowchart of method 300A, which is used to provide container management based on a composite image file system engine. Initially, at box 310, multiple files for generating a composite image are received. At box 320, a composite image of the multiple files is generated. The composite image includes a generic flat fileset. At box 330, the composite image is passed to trigger its mounting. Mounting the composite image is based on metadata files within the generic flat fileset.

[0082] Go to Figure 3BThe diagram provides a flowchart of method 300B, which provides container management based on a synthetic image file system engine. Initially, at box 340, the synthetic image is accessed, generated from multiple files. The synthetic image includes a generic flat file set. At box 350, the synthetic image is mounted, based on metadata files from the generic flat file set. Mounting the synthetic image further includes creating a file system instance corresponding to the container used for that synthetic image. At box 360, access to the files in the generic flat file set is provided. The generic flat file set is configured to be presented in a tree structure based on metadata from a tree structure of files within the generic flat file set.

[0083] Go to Figure 4A The document provides a flowchart of method 400A, which provides container management based on a composite image file system engine. Initially, at box 410, multiple files used to generate the composite image are accessed, and a remote shared configuration for generating the composite image is selected. The remote shared configuration is selected as one of the following: a split-layer storage shared configuration, a split-layer direct access storage shared configuration, or a dynamic base image configuration. At box 420, a composite image for multiple files is generated. The composite image includes a generic flat file set. The composite image is generated as a split-layer storage shared image, a split-layer direct access storage shared image, or a dynamic base image. At box 430, the composite image is passed to trigger its mounting. Mounting the composite image is based on the metadata file from the generic flat file set and the selection of the remote shared configuration.

[0084] Go to Figure 4B The diagram provides a flowchart of method 300B, which is used to provide container management based on a synthetic image file system engine. Initially, at box 440, the synthetic image is accessed. The synthetic image comprises a common flat file set. At box 450, the type of remote share configuration the synthetic image is configured for is determined. The type of remote share configuration corresponds to one of the following: split-layer storage share configuration, split-layer direct access storage share configuration, or dynamic base image configuration. At box 460, based on the type of remote share configuration of the synthetic image, the synthetic image is provided for accessing files in the common file set.

[0085] Go to Figure 5AThe diagram provides a flowchart of method 500A, which provides container management based on a synthetic image file system engine. Initially, at box 510, multiple files used to generate the synthetic image are accessed. At box 520, the synthetic image of the multiple files is generated. The synthetic image includes a common flat file set. At box 530, the generated synthetic image of the multiple files includes: pre-aligning one or more executable files from the multiple files based on memory alignment for the memory used to execute the executable. The one or more executable files are stored in the synthetic image.

[0086] Go to Figure 5B The diagram provides a flowchart of method 500B, which provides container management based on a synthetic image file system engine. Initially, at box 540, multiple files used to generate the synthetic image are accessed. At box 550, the synthetic image of the multiple files is generated. The synthetic image includes a common flat file set. At box 560, the generated synthetic image of the multiple files includes pre-computed hashes of one or more executable files from the multiple files. The hashes are stored in the synthetic image.

[0087] Example container management system environment

[0088] Referring to a container management system environment 600 (“Container Management System”), which includes a container management system for executing functional embodiments of the supporting technologies described herein. The container management system includes distributed components that are implemented in conjunction with other integrated components that implement aspects of the technologies. The container management system environment 600 refers to the hardware and software architecture supporting the functionality of the technologies.

[0089] At a higher level, container management systems (e.g., DOCKER) support the development and delivery of software in the packaging of objects called containers within distributed computing systems. Containers are created from images that specify exact contents. Specifically, container management systems can be used to support (e.g., manage, create, mount, and access) containers. A container management system includes several components that integrate and communicate to provide container functionality. For example, a container management system could be a suite of platform-as-a-service offerings that use OS-level virtualization to provide containers that are isolated from each other and bundled with their own software, libraries, and configuration files. Containers can communicate with each other through well-defined channels. Containers can run in a single operation.

[0090] Continue to refer to Figure 6 , Figure 6A high-level architecture file system environment 600 with components implemented according to this disclosure is shown. It should be understood that the arrangement described herein is illustrated by way of example only, and other arrangements besides those shown are contemplated. Among other components not shown, the container management system includes a container engine 610, an interface 612, containers 620 (including APP 622, .NET 624, and BASE 628), and images 630. The container management system further includes clients 640, composition 650, a registry 660, a SWARM cluster 670, and an operating system 680 (including compute services 682, control groups 684, namespaces 686, tier capabilities 688, and other OS functionalities 690).

[0091] Container management systems can support containers and virtual machines (VMs) with features including isolated environments, portability between host machines, and resource governance. Containers and VMs can be distinguished based on virtualization level (i.e., OS virtualization of containers compared to hardware virtualization of VMs), OS functionality (i.e., containers share the OS kernel mode with other containers and container hosts, while VMs can be full OSs with dedicated kernel modes available), and modular architectural models (e.g., containers share the underlying resources of the container host and build the images needed to run applications, while VMs are built with a full OS and depend on application thin-features).

[0092] Container management systems can support the kernel mode of an operating system, which has been implemented for drivers requiring unrestricted access to the underlying hardware. Container management systems can also allow users to access hardware or memory using OS APIs. Code running in kernel mode can directly access resources and share the same memory locations and virtual address space with the operating system and other kernel drivers. In user mode, code runs in a separate process (e.g., user space) with its own dedicated set of memory locations (private virtual address space). Because each application's virtual address space is private, one application cannot modify data belonging to another application. Each application runs in isolation; if one application crashes, the crash is limited to that application.

[0093] Container management systems provide a complete environment for executing and running applications, including an operating system with compute services, control groups, namespaces, tier capabilities, and other OS functionalities. Container management systems also include container engines to support container-related actions based on command and client interfaces. In operation, the container engine continuously listens for and processes API requests via its interface. Clients can interact with the container engine (e.g., through the client's command-line interface) to manage (e.g., synthesize) container instances. Clients send commands to the container engine to execute operations.

[0094] A container engine can include persistent background processes (e.g., daemons) that manage images, containers, networking, and storage volumes. An image can reference a read-only binary template that can be used to build containers. Images can be used to store and deliver applications. Images can be used to build containers or customized to add other elements to extend the image's current configuration. A container is a packaged environment for running an application. Containers can be defined by images and provide any additional configuration operations on starting a container, including but not limited to network connectivity and storage options.

[0095] The container engine provides support for communication between isolated container components and other components using different drivers. For example, an overlay driver can support swarm services (i.e., clustering and scheduling service tools) for multiple applications, or the host driver can remove network isolation between containers when it is not required. The container engine includes a registry that provides locations from which images can be stored and downloaded (i.e., the container engine registry contains repositories for one or more images). In this way, the container engine and container management system support general resource isolation and operating system virtualization.

[0096] Example Distributed Computing Environment

[0097] Now for reference Figure 7 , Figure 7 The illustration shows an example distributed computing environment 700 that can be implemented using this disclosure. Specifically, Figure 7 A high-level architecture of an example cloud computing platform 710 that can host a technology solution environment or a portion thereof (e.g., a data trustee environment) is illustrated. It should be understood that such and other arrangements described herein are merely illustrative examples. For instance, as mentioned above, many of the elements described herein can be implemented as discrete or distributed components, or in combination with other components, and in any suitable combination and location. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, and functional groupings) may be used in addition to or instead of those shown.

[0098] The data center can support a distributed computing environment 700, which includes a cloud computing platform 710, racks 720, and nodes 730 (e.g., computing devices, processing units, or blades) within the racks 720. The technical solution environment can be implemented using the cloud computing platform 710, which runs cloud services across different data centers and geographic regions. The cloud computing platform 710 can implement a structure controller 740 component for provisioning and managing the allocation, deployment, upgrades, and management of cloud services. Typically, the cloud computing platform 710 is used to store data or run service applications in a distributed manner. The cloud computing infrastructure 710 in the data center can be configured to host and support the operation of endpoints for specific service applications. The cloud computing infrastructure 710 can be a public cloud, a private cloud, or a dedicated cloud.

[0099] Node 730 may be equipped with a host 750 (e.g., an operating system or runtime environment) on which a defined software stack runs. Node 730 may also be configured to perform specialized functionalities (e.g., compute nodes or storage nodes) within the cloud computing platform 710. Node 730 is assigned to run one or more portions of a tenant's service application. A tenant may refer to a customer who uses the resources of the cloud computing platform 710. The service application components of the cloud computing platform 710 that support a particular tenant may be referred to as tenant infrastructure or rental housing. The terms service application, application, or service are used interchangeably herein and refer generally to any software or software portion that runs on top of or accesses storage and compute equipment locations within a data center.

[0100] When node 730 supports more than one individual service application, node 730 can be partitioned into virtual machines (e.g., virtual machine 752 and virtual machine 754). Physical machines can also run individual service applications simultaneously. Virtual machines or physical machines can be configured as personalized computing environments supported by resources 760 (e.g., hardware and software resources) and content engine 762 in the cloud computing platform 710. It is conceivable that resources can be configured for specific service applications. Furthermore, each service application can be partitioned into functional parts so that each functional part can run on a separate virtual machine. In the cloud computing platform 710, multiple servers can be used to run service applications and perform data storage operations in a cluster. In particular, servers can perform data operations independently but are exposed as a single device referred to as a cluster. Each server in the cluster can be implemented as a node.

[0101] Client device 780 can be linked to service applications in cloud computing platform 710. Client device 780 can be any type of computing device, which can correspond to the reference... Figure 8The described computing device 800, for example, client device 780, can be configured to issue commands to cloud computing platform 710. In embodiments, client device 780 can communicate with service applications via Virtual Internet Protocol (IP) and load balancers or other components that direct communication requests to specified endpoints within cloud computing platform 710. Components of cloud computing platform 710 can communicate with each other via a network (not shown), which may include, but is not limited to, one or more local area networks (LANs) and / or wide area networks (WANs).

[0102] Example operating environment

[0103] Having briefly described an overview of embodiments of the invention, the following describes an example operating environment in which embodiments of the invention may be implemented, in order to provide a general background for various aspects of the invention. In particular, reference is made first to... Figure 8 The illustration shows an example operating environment for implementing embodiments of the present invention and is generally designated as computing device 800. Computing device 800 is merely one example of a suitable computing environment and is not intended to imply any limitation on the scope of the purpose or functionality of the invention. Computing device 800 should also not be construed as having any dependency or requirement on any of the components illustrated or a combination thereof.

[0104] This invention can be described in the general context of computer code or machine-usable instructions, including computer-executable instructions, such as program modules, that are executed by a computer or other machine such as a personal data assistant or other handheld device. Generally, a program module, including routines, programs, objects, components, data structures, etc., refers to code that performs a specific task or implements a specific abstract data type. This invention can be practiced in a wide variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, and more specialized computing devices. This invention can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network.

[0105] refer to Figure 8 The computing device 800 includes a bus 810 that is directly or indirectly coupled to the following devices: a memory 812, one or more processors 814, one or more presentation components 816, an input / output port 818, an input / output component 820, and an illustrative power supply 822. The bus 810 can represent one or more buses (such as an address bus, a data bus, or a combination thereof). For clarity of concept, Figure 8 The various boxes are shown with lines, and other arrangements of the described components and / or component functionality are also envisioned. For example, a presentation component such as a display device can be considered as an I / O component. Furthermore, the processor has memory. We recognize this as essential to the art and reiterate... Figure 8The diagrams are merely illustrative of example computing devices that may be used in conjunction with one or more embodiments of the present invention. No distinction is made between categories such as “workstation,” “server,” “laptop,” and “handheld device,” as all devices are... Figure 8 It is envisioned within the scope of "computing devices".

[0106] Computing device 800 typically includes a wide variety of computer-readable media. Computer-readable media can be any available media accessible by computing device 800 and includes volatile and non-volatile media, removable and non-removable media. By way of example and not limitation, computer-readable media can include computer storage media and communication media.

[0107] Computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile disk (DVD) or other optical disc storage, magnetic tape cassettes, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store the required information and can be accessed by a computing device 800. Computer storage media itself does not include signals.

[0108] Communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and include any information delivery medium. The term "modulated data signal" refers to a signal having one or more characteristics that can be set or altered in a manner that encodes information in the signal. By way of example and not limitation, communication media include wired media such as wired networks or direct wired connections, and wireless media such as acoustic, RF, infrared, and other wireless media. Any combination of the above media should also be included within the scope of computer-readable media.

[0109] Memory 812 includes a computer storage medium in the form of volatile and / or non-volatile memory. The memory may be removable, non-removable, or a combination of both. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. Computing device 800 includes one or more processors that read data from various entities such as memory 812 or I / O components 820. Multiple presentation components 816 present data indications to a user or other device. Exemplary presentation components include display devices, speakers, printing components, vibration components, etc.

[0110] I / O port 818 allows computing device 800 to be logically coupled to other devices including I / O components 820, some of which may be built-in. Illustrative components include microphones, joysticks, game controllers, satellite antennas, scanners, printers, wireless devices, etc.

[0111] In conjunction with the technical solution environment described herein, the embodiments described herein support the technical solutions described herein. The components of the technical solution environment may be integrated components including hardware architecture and software framework that support constraint computation and / or constraint query functionality within the technical solution system. Hardware architecture refers to physical components and their interrelationships, while software framework refers to software that provides functionalities that can be implemented using hardware embodied on a device.

[0112] End-to-end software-based systems can run within system components to operate computer hardware to provide system functionality. At a low level, the hardware processor executes instructions selected for a given processor from a machine language (also known as machine code or native code) instruction set. The processor recognizes native instructions and executes the corresponding low-level functions, such as those related to logic, control, and memory operations. Low-level software written in machine code can provide more complex functionality to higher-level software. As used herein, computer-executable instructions include any software, including low-level software written in machine code, high-level software such as application software, and any combination thereof. In this respect, system components can manage resources and provide services for system functionality. Embodiments of the invention are contemplated by any other variations and combinations.

[0113] For example, a technical solution system may include an API library, which contains specifications for routines, data structures, object classes, and variables that can support interaction between the hardware architecture of a device and the software framework of the technical solution system. These APIs include configuration specifications for the technical solution system to enable different components within it to communicate with each other, as described herein.

[0114] Having identified the various components used herein, it should be understood that any number of components and arrangements can be employed to achieve the desired functionality within the scope of this disclosure. For example, for clarity of concept, components in the embodiments depicted in the figures are shown with lines. Other arrangements of these and other components may also be implemented. For example, while some components are depicted as single components, many elements described herein can be implemented as discrete or distributed components or in combination with other components, and can be implemented in any suitable combination and location. Some elements may be omitted entirely. Furthermore, the various functions described herein as being performed by one or more entities can be performed by hardware, firmware, and / or software, as described below. For example, various functions can be performed by a processor executing instructions stored in memory. Therefore, other arrangements and elements (e.g., machines, interfaces, functions, sequences, and functional groups) may be used in addition to or in lieu of those shown.

[0115] The embodiments described in the following paragraphs can be combined with one or more of the specifically described alternatives. In particular, the claimed embodiments may include alternative references to more than one other embodiment. The claimed embodiments may specify further limitations on the claimed subject matter.

[0116] The subject matter of embodiments of the present invention has been specifically described herein to satisfy legal requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have considered that the claimed subject matter may also be embodied in other ways in combination with other existing or future techniques to include different steps or combinations of steps similar to those described in this document. Furthermore, although the terms “step” and / or “box” may be used herein to imply different elements of the method employed, these terms should not be construed as implying any particular order among or between the various steps disclosed herein, unless the order of the various steps is explicitly described.

[0117] For the purposes of this disclosure, the word "comprising" has the same broad meaning as the word "including," and the word "access" includes "receiving," "quoting," or "retrieval." Furthermore, the word "communication" has the same broad meaning as the words "receiving" or "transmitting," meanings facilitated by a software- or hardware-based bus, receiver, or transmitter using the communication medium described herein. Additionally, unless otherwise stated, words such as "a" and "an" include both plural and singular. Thus, for example, the constraint of "feature" is satisfied when one or more features are present. Furthermore, the term "or" includes conjunctions, antonymous conjunctions, and both (therefore, a or b includes a or b, as well as a and b).

[0118] For the purposes of the detailed discussion above, embodiments of the present invention have been described with reference to a distributed computing environment; however, the distributed computing environment described herein is merely exemplary. Components may be configured to perform novel aspects of the embodiments, wherein the term "configured to" may mean "programmed to" use code to perform a particular task or implement a particular abstract data type. Furthermore, while embodiments of the present invention may refer generally to the technical environment and illustrations described herein, it should be understood that the described techniques can be extended to other implementation contexts.

[0119] Embodiments of the invention have been described with respect to specific examples, which are illustrative in all respects and not restrictive. Alternative embodiments will become apparent to those skilled in the art without departing from the scope of the invention.

[0120] As can be seen from the above, the present invention is well suited to achieve all the above-described purposes and objectives, as well as other obvious and inherent structural advantages.

[0121] It should be understood that certain features and sub-combinations are useful and can be used without reference to other features or sub-combinations. This is conceivable and within the scope of the claims.

Claims

1. A computerized system, comprising: One or more computer processors; as well as A computer memory storing computer-usable instructions, which, when used by the one or more computer processors, cause the one or more computer processors to perform operations, including: Access multiple files used to generate the composite image; and Generate the composite image for the plurality of files, wherein the composite image includes a common flat file set, and generating the composite image includes: Preprocessing is performed on one or more executable files from the plurality of files, wherein the preprocessing of the one or more executable files includes at least one of the following: Pre-align the one or more executable files in the composite image to have the same disk alignment as the memory alignment of the memory where the executable file is configured to be executed; or Adjust the disk alignment of one or more sections of the executable file to match the corresponding section alignment.

2. The system of claim 1, wherein the synthesized image includes an object repository, wherein the preprocessed one or more executable files are stored in the object repository as region files.

3. The system of claim 1, wherein the executable file is a portable executable PE binary file having sections stored in a section-aligned manner so that the PE binary file can be executed in-situ.

4. The system of claim 1, wherein pre-alignment of the one or more executable files avoids repositioning the one or more executable files before mapping them for execution.

5. The system of claim 1, wherein the one or more computer processors further perform pre-computation of hashes of the one or more executable files, causing evasion of pre-cached verification results.

6. The system according to claim 5, further comprising: The composite image is provided so that the layout manager client interface causes execution of the one or more executable files as pre-aligned executable code, or causes verification of the one or more executable files using the pre-computed hash.

7. The system of claim 5 or 6, wherein the pre-computed hash is stored as an extended attribute of the one or more executable files.

8. One or more computer storage media having computer-executable instructions embodied thereon, which, when executed by a computing system having a processor and a memory, cause the processor to: Access multiple files used to generate the composite image; as well as Generate the composite image for the plurality of files, wherein the composite image includes a common flat file set, and generating the composite image includes: Preprocessing is performed on one or more executable files from the plurality of files, wherein the preprocessing of the one or more executable files includes at least one of the following: The one or more executable files in the composite image are pre-aligned to have the same disk alignment as the memory alignment of the memory in which the executable file is configured to be executed; or Adjust the disk alignment of one or more sections of the executable file to match the corresponding section alignment.

9. The medium of claim 8, wherein the synthesized image includes an object repository, wherein the preprocessed one or more executable files are stored in the object repository as region files.

10. The medium of claim 8, wherein the executable file is a portable executable PE binary file having sections stored in a section-aligned manner such that the PE binary file can be executed in-situ.

11. The medium of claim 8, wherein pre-alignment of the one or more executable files avoids repositioning the one or more executable files prior to mapping them for execution.

12. The medium of claim 8, wherein the one or more computer processors further perform pre-computation of hashes of the one or more executable files, causing evasion of pre-cached verification results.

13. The medium according to claim 12, further comprising: The composite image is provided so that the layout manager client interface causes execution of the one or more executable files as pre-aligned executable code, or causes verification of the one or more executable files using the pre-computed hash.

14. The medium according to claim 12 or 13, wherein the pre-computed hash is stored as an extended attribute of the one or more executable files.

15. A computer-implemented method, the method comprising: Access multiple files used to generate the composite image; as well as Generate the composite image for the plurality of files, wherein the composite image includes a common flat file set, and generating the composite image includes: Preprocessing is performed on one or more executable files from the plurality of files, wherein the preprocessing of the one or more executable files includes at least one of the following: The one or more executable files in the composite image are pre-aligned to have the same disk alignment as the memory alignment of the memory in which the executable file is configured to be executed; or Adjust the disk alignment of one or more sections of the executable file to match the corresponding section alignment.

16. The method of claim 15, wherein the synthesized image includes an object repository, wherein the preprocessed one or more executable files are stored in the object repository as region files.

17. The method of claim 15, wherein the executable file is a portable executable PE binary file having sections stored in a section-aligned manner such that the PE binary file can be executed in-situ.

18. The method of claim 15, wherein pre-alignment of the one or more executables avoids repositioning the one or more executables prior to mapping the one or more executables for execution.

19. The method of claim 15, wherein the one or more computer processors further perform pre-computation of hashes of the one or more executables, causing evasion of pre-cached verification results.

20. The method of claim 19, further comprising: The composite image is provided so that the layout manager client interface causes execution of the one or more executable files as pre-aligned executable code, or causes verification of the one or more executable files using the pre-computed hash.

Images