CONTAINERIZED APPLICATION MANIFESTOS AND VIRTUAL PERSISTENT VOLUMES

Abstract

A non-transitory, machine-readable medium (504) that stores instructions for a policy engine (242) of a storage virtualization system (130) that are executable by a processing resource (102; 502), wherein the non-transitory, machine-readable medium (504) comprises: Instructions to read from a container orchestrator (122; 222) a manifest (282) of a containerized application (124; 224) to which a virtual persistent volume (280) is assigned; Instructions for identifying an application data store profile (284) that matches information in the manifest (282) from a plurality of application data store profiles, each containing preconfigured storage parameters for a particular application type; and Instructions for changing the virtual persistent volume (280) according to the identified application data storage profile (284), wherein, if the manifest (282) information indicates increased scaling of the containerized application (124; 224), the identify instructions cause the identification of an application datastore profile (284) that matches the increased scaling, and the modify instructions cause the increase of the virtual persistent volume (280) according to the application datastore profile (284) that matches the increased scaling, and if the manifest information (282) indicates a reduced scaling of the containerized application (124; 224), the instructions for identifying the cause for identifying an application datastore profile (284) that matches the reduced scaling, and the instructions for changing the cause for shrinking the virtual persistent volume (280) according to the application datastore profile (284) that matches the reduced scaling.

Description

BACKGROUND

[0001] Containers are a type of virtualization. A container can contain an application packaged together with dependencies and libraries. A containerized application can use or generate persistent data. US 2014 / 0 156 877 A1 describes techniques for analyzing storage resources. US 2017 / 0 315 795 A1 describes methods and systems for monitoring containerized applications. US 2017 / 0 087 470 A1 describes methods and systems for measuring a user's capabilities in order to anticipate future challenges for the user using predictive analytics. It is an object of the invention to propose a non-transitory, machine-readable storage medium, a method, and a system for modifying a virtual persistent volume according to an identified application data storage profile.This object is achieved by an inventive, non-transitory, machine-readable storage medium according to claim 1, by an inventive method according to claim 7 and by an inventive system according to claim 13. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Various examples are described below with reference to the following figures. Fig. Figure 1 shows an example system that creates a virtual persistent volume that aggregates a variety of underlying storage and presents it to a containerized application. Fig. Figure 2 shows an example system that modifies a virtual persistent volume according to an application data store profile that matches the information in a manifest of a containerized application. Fig. 3 shows an example method that includes modifying a virtual persistent volume according to an identified application data storage profile. Fig. 4 shows another example method that includes modifying a virtual persistent volume according to an identified application data storage profile. Fig. 5 shows an example system with a machine-readable medium containing instructions, including instructions for modifying a virtual persistent volume according to an identified application data storage profile. Fig. 6 shows another example system having a machine-readable medium containing instructions, including instructions for modifying a virtual persistent volume according to an identified application data storage profile. DETAILED DESCRIPTION

[0003] Container technology is a paradigm of computer virtualization in which an application, along with dependencies and libraries, is packaged in a container to provide an isolated environment for running the application. Such an application can be referred to as a containerized application. Many containers can run on top of a single operating system, but each container is inherently isolated from other containers. In this way, the container paradigm can be thought of as the virtualization of the operating system. Containers can be more lightweight than other forms of virtualization, such as virtual machines that virtualize hardware. For example, each virtual machine can have its own copy of an operating system kernel, whereas, in contrast, multiple containers can share an operating system kernel.

[0004] Containerized applications may require storage to retain persistent data. Container orchestrators (e.g., Kubernetes) can provide the ability to provision some storage to a container. However, there are many types of storage, including but not limited to storage that is local to the container, remote to the container, hardware (e.g., a locally attached drive), software-defined (e.g., a file system, virtualized or containerized storage, storage provided via an API, etc.), or storage that combines all of the above. Previous attempts to provide container storage may not provide the flexibility of storage configuration and data services that users and administrators of container environments desire. For example, a container orchestrator may be limited to providing one type of storage to an application.Other types of systems may attempt to concatenate multiple volumes into a single volume, but such concatenation may not have the flexibility to provide certain data services without disrupting user access to the data.

[0005] Additionally, persistent storage can be provided for a containerized application using predefined storage classes. However, these storage classes may not be tailored or optimized for the containerized application, especially if the storage requirements of the containerized application change over time.

[0006] To address the above technical challenges, the examples described here refer to a storage virtualization system and its policy engine that reads a manifest of a containerized application assigned a virtual persistent volume from a container orchestrator. An application datastore profile is identified that matches information in the manifest. The application datastore profile is one of many profiles preconfigured with storage parameters selected for different application types and configurations based on domain expertise and / or testing and analysis. The virtual persistent volume attached to the containerized application is then modified according to the identified application datastore profile.Because the virtual persistent volume abstracts the underlying storage from the containerized application, the type of persistent storage can be changed to a more current application-centric storage policy in a way that is transparent and non-intrusive to the containerized application.

[0007] Fig. 1 shows an example of a computer system 100 that supports and participates in a container environment 120. The computing system 100 includes a processing resource 102, which may include a microcontroller, a microprocessor, one or more central processing unit cores, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc. The computing system 100 includes a machine-readable medium 104, which may be non-transitory and may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, a hard disk drive, etc.

[0008] The processing resource 102 can execute instructions 105 (i.e., programming or software code) stored on the machine-readable medium 104 to perform functions of the computer system 100, such as providing a container orchestrator 122 and a containerized storage system 130, as described further below. In particular, each of the components of the containerized storage virtualization system 130 can be implemented as executable instructions 105, including the data service provider 134, the storage virtualizer 136, the volume manager 138, the adapters 139, the policy engine 142, and the container storage interface API 144. Containerized applications 124, 126 and container storage interface plug-ins 128 can also be implemented as instructions included in the executable instructions 105.Additionally or alternatively, the processing resource 102 may include electronic circuitry for performing the functionality described herein.

[0009] Computer system 100 may also include other hardware components, such as physical memory 106. Physical memory 106 may comprise any physical storage device, such as a hard disk drive, a solid-state drive, or the like, or a plurality of such storage devices (e.g., an array of hard disks), and may be locally attached (i.e., installed) within computer system 100. In some implementations, physical memory 106 may be accessed as a block storage device.

[0010] In some cases, computer system 100 may also include a local file system 108, which may be implemented as a layer above physical storage 106. For example, an operating system 107 may be running on computer system 100 (due to processing resource 102 executing certain instructions 105 related to the operating system), and operating system 107 may provide a file system 108 for storing data on physical storage 106.

[0011] Computer system 100 may communicate with other computer systems, such as computer system 110, e.g., via a wired and / or wireless network. The other computer systems may be similar to computer system 100 and may each include at least one processing resource and a machine-readable medium. Computer systems 100 and 110 may each execute software (i.e., processing resource 102 executes specific instructions 105) to deploy nodes of a container orchestrator 122. In other words, container orchestrator 122 may have a cluster architecture that includes the container orchestrator nodes of computer systems connected in a cluster. Container orchestrator 122 acts as a platform for deploying and managing containerized applications across the cluster of computer systems.The container orchestrator 122, the containerized applications it provides, and other container resources (such as container storage) are considered part of a container environment 120. In contrast, other elements may operate outside of the container environment 120, such as the local file system 108 and an operating system 107 of the computing system 100.

[0012] In Fig. 1, for illustrative purposes, containerized applications 124 and 126 are deployed locally on computing system 100 via container orchestrator 122. Applications 124, 126 can therefore be said to be remote from other nodes, such as computer system 110. Containerized applications 124, 126 may represent microservices, user applications, or the like and are sometimes simply referred to as containers. Container environment 120 may also host containerized applications (not shown) that are local to other computer systems in the cluster.

[0013] The container orchestrator 122 also employs a containerized storage virtualization system 130, which is described in more detail below. In the example of Fig. 1, the containerized storage virtualization system 130 is also executed locally on the computer system 100 and its processing resource 102. In some implementations, other containerized storage virtualization systems of other computer systems in the cluster are also included in the container environment 120, although in Fig. 1 are not shown. For example, the containerized storage virtualization system 130 may serve as a node in a storage virtualization platform having a clustered architecture in which multiple containerized storage virtualization systems (at least some of which may be running on different and separate physical computing systems) cooperate to store and manage data in a storage virtualization layer.

[0014] The container orchestrator 122 may also include a standardized container storage interface 123. The container storage interface 123 has a plug-in architecture and may provision storage in the form of a persistent volume from a storage source using a corresponding one of a variety of available container storage interface plug-ins 128. "Provisioning" storage may refer to the process of allocating a certain amount of storage capacity and making that allocation available to a consumer. Plug-ins 128 may be provided by various vendors and may expose an associated storage system to the container storage interface 123 (i.e., make it available for use and / or consumption). Non-limiting examples of plug-ins include a block protocol plug-in (e.g., based on the Internet Small Computer Systems Interface or iSCSI protocol), a file protocol plug-in (e.g.,based on the Network File System or NFS protocol, the Common Internet File System or CIFS protocol, the Server Message Block or SMB protocol), a public cloud persistent volume plug-in, and other plug-ins based on any other storage type (e.g., custom drivers). For convenience, individual ones of the plug-ins 128 may be referred to herein as a plug-in 128 (e.g., a block protocol plug-in 128 or a file protocol plug-in 128). A plug-in 128 may undergo an installation and setup process in the container environment 120 as required by that plug-in (e.g., populating credentials or other configuration details). In some cases, one or more of the plug-ins 128 may run as a containerized application. However, a persistent volume provisioned via the plug-in 128 may be limited to a single storage type corresponding to that plug-in.In contrast, the containerized storage virtualization system 130 disclosed herein may be advantageously used to create persistent volumes that mix multiple underlying storage types.

[0015] The containerized storage virtualization system 130 (also referred to as storage virtualization system 130) runs in one or more containers and may be implemented in executable instructions 105. As will be seen, the storage virtualization system 130 provides an additional storage virtualization layer between a requesting application and one or more storage allocations provided via the container storage interface 123. The storage virtualization system 130 includes a data path 132 and a control plane 140. The data path 132 includes the data service provider 134, the storage virtualizer 136, and a volume manager 138. The data path 132 may also include storage adapters 139 for accessing local storage (i.e., storage on the same computer system 100 on which the storage virtualization system 130 is hosted), such asa block adapter for mounting local physical storage 106, stacked file and block adapters for accessing the local file system 108 as a block device, or other storage adapters.

[0016] The control plane 140 includes a policy engine 142 and a container storage interface API (Application Programming Interface) 144. During an initialization phase of the storage virtualization system 130, the control plane 140 may receive a list of available container storage interface plug-ins 128 in the container environment 120 from an administrator or the container orchestrator 122. The control plane 140 may also confirm the storage adapters 139 available for the data path 132 for mounting local storage. The control plane 140 may also maintain a list of characteristics of the storage associated with each of these available plug-ins 128, such as: B. Performance characteristics (e.g. latency, IOPS or input / output operations per second, etc.), security features (e.g. encryption, isolation, etc.), data protection features (e.g. available RAID or redundant Array of Independent Disks levels), cost characteristics (e.g.dollars per GB) or other features.

[0017] The functionality of the data path functions 132 and the control plane functions 140 will now be described in the context of providing storage to containerized applications 124, 126. A containerized application, such as application 124, may request storage from the container orchestrator 122. For example, the application 124 may require a persistent volume to store data. In some implementations, the application 124 may pass one or more requirements with the request, such as a capacity requirement, a performance requirement (e.g., latency, IOPS, etc.), a data protection requirement (e.g., RAID level), a cost requirement (e.g., dollars per GB), a security requirement, a tiering requirement (e.g., specific amounts of hot and cold storage), or another requirement.In some implementations, the container orchestrator 122 may manage a storage abstraction called an “Orchestrator Persistent Volume” in response to the request.

[0018] The container orchestrator 122 may use the container storage interface 123 to send the request to the storage virtualization system 130 via the container storage interface API 144 of the control plane 140 (with the interface 144 acting as a server). In this way, the containerized storage virtualization system 130 may be understood as acting as a storage provider for the container orchestrator 122. In some implementations, the control plane 140 may assign an identifier to the request so that each request can be individually identified, particularly with respect to the storage provided for each request in the manner described herein.

[0019] The policy engine 142 of the container orchestrator 122 analyzes the request to determine which storage types meet the requirements of the request. For example, the control plane 140 may have one or more of the following types available for storage provisioning: physical storage 106, the local file system 108, a remote storage system 160, a virtualized storage 162, or a public cloud storage 164. Additionally, more than one of the illustrated storage types may be present, but is not shown for clarity. For example, multiple remote storages 160 may be available from which storage allocations can be provided.

[0020] The Fig. The remote storage system 160 illustrated in Figure 1 may represent either a file system (e.g., a NAS file server), a block storage system (e.g., a SAN storage device), or any other type of storage system that is remote from the computer system 100 and, thus, also from the containerized storage virtualization system 130. Being remote may mean, for example, that the computer system 100 communicates with the remote storage system 160 via a network connection or the like.

[0021] Virtualized storage 162 may represent any existing persistent volume in a virtual environment, such as a persistent volume in container environment 120, including container storage provisioned via container storage interface 123 independently of container storage virtualization system 130. In some examples, virtualized storage 162 may represent container storage provided by a container storage virtualization system other than system 130 (e.g., hosted on a different node than computer system 100). In some examples, virtualized storage 162 may represent storage provided by a virtual machine or a hypervisor-based software-defined storage platform.

[0022] The policy engine 142 may determine a mix of the aforementioned storage types. For example, the policy engine 142 may compare the request to the available storage types to determine the closest possible match. To illustrate, the request may require a certain amount of high-speed storage and a certain amount of low-cost archival-class storage. The policy engine may determine that physical storage 106 meets the high-speed storage requirements (e.g., due in part to locality and because it is a high-speed medium in this example) and a block storage device 160 meets the low-cost archival storage requirements. Additional example implementations of the policy engine 142 are described further below, e.g., with reference to Fig. 2.

[0023] The control plane 140 then uses the adapters 139 and / or container storage interface plug-ins 128 to provision each storage type in the specified mix and obtain a mount point for each provisioned storage. A mount point enables access to the provisioned storage by a consumer, such as the data path 132, as described further below.

[0024] As an example of a local deployment, control plane 140 may use a block adapter from adapters 139 to provision a mapping from physical storage 106 and acquire a local block mount point 170 (e.g., a local host device mount point) to access this mapping. As another example, control plane 140 may use stacked file and block adapters to provision a mapping from local file system 108 and acquire a local file system mount point 172 to access this mapping as a block device (i.e., "file as block device").

[0025] To provision storage via plug-ins 128, control plane 140 communicates with storage interface 123 via container storage interface API 144 to request that a plug-in 128 provide a mapping from its associated storage and provide a mount point back to control plane 140. Assuming remote storage system 160 represents a remote block device (e.g., a SAN storage array external to computer system 100), control plane 140 may request (via 144 and 123) that a block protocol plug-in 128 (e.g., based on the iSCSI protocol) provide a mapping from remote block storage system 160 and provide a remote volume mount point 174 (e.g., iSCSI target and LUN or Logical Unit Number) to access this mapping. As another example, the remote storage system 160 may be a remote file device (e.g.,A remote file storage system 160 may represent a network-attached storage device (NAS file server), and the control plane 140 may request (via 144 and 123) that a file protocol plug-in 128 (e.g., based on the NFS protocol) provide a mapping from the remote file storage system 160 and provide a remote volume mount point 174 (e.g., an IP address and export name under NFS) for accessing that mapping. In some implementations, the control plane 140 may use a block protocol plug-in 128 to provide from the physical storage 106 or a file protocol plug-in 128 to provide from the local file system 108, rather than using an adapter 139.

[0026] As another example of provisioning via a plug-in, the control plane 140 (via 144 and 123) may request that a plug-in 128 that matches the virtualized storage 162 provide a mapping from the virtualized storage 162 and provide a mount point to the virtualized storage 176 for accessing this mapping. As another example, the control plane 140 (via 144 and 123) may request that a public cloud plug-in 128 provide a mapping from the public cloud storage 164. In turn, the public cloud plug-in 128 may provide a public cloud persistent volume mount point 178 for accessing this mapping.

[0027] Although in Fig. 1 illustrates a local block mount point 170, a local file system mount point 172, a remote volume mount point 174, a virtualized storage mount point 176, and a public cloud persistent volume mount point 178, more or fewer mount points and mount points in any combination may be requested by the control plane 140 and acquired via the adapters 139 or plug-ins 128. In various cases, multiple local block mount points 170, multiple local file system mount points 172, multiple remote volume mount points 174, multiple virtualized storage mount points 176, and / or multiple persistent public cloud volume mount points 178 may be requested and acquired by the control plane 140.Furthermore, storage system 160 and remote mount point 174 may each represent one or more of the same or different types of remote storage and a mount point thereof, including block, file, or other types of remotely accessible storage. The particular combination of storage mount points requested and acquired by control plane 140 may depend on the storage requirement of containerized application 124, and in particular, on the handling of that storage requirement by policy engine 142.

[0028] Once the one or more storage mount points (e.g., 170, 172, 174, 176, or 178) have been acquired by the control plane 140 in accordance with the policy engine 142, the control plane 140 passes the acquired mount points to the data path 132. The control plane 140 may identify the mount points as belonging to a particular request, for example, by associating the mount points with the request identifier. As described, the data path 132 consumes and merges the mount points to create a virtual persistent volume 156, which is passed from a mount point 180 to the requesting containerized application 124. In this way, the allocated storage corresponding to the acquired mount points (e.g., 170, 172, 174, 176, or 178) can be referred to as the underlying storage of the virtual persistent volume 156.The containerized application 124 therefore reads and writes data to the virtual persistent volume 156. Before describing the creation of the virtual persistent volume 156, operational aspects of the data path 132 are first described.

[0029] Data path 132 includes storage virtualizer 136, which maintains an object-based storage virtualization layer 150. One purpose of storage virtualization layer 150 is to decouple where data is stored (i.e., storage allocations accessed via mount points 170, 172, 174, 176, and / or 178) from how the data is presented to a consumer of the data (e.g., a containerized application 124). In this way, data services such as migration, backup, snapshotting, replication, deduplication, compression, and others can be performed on any mix of underlying storage and with reduced, minimal, or even no disruption to a consumer of the data.

[0030] One aspect of the storage virtualization layer 150 is that the storage virtualizer 136 stores data as "objects" in an object store 152. In particular, the object store 152 may store various types of objects, including data objects and metadata objects. Data related to the containerized application 124, including files and / or directories, is formed from one or more data objects. Metadata objects may serve, among other things, to organize the data objects in a meaningful and orderly manner, as described further below. In some implementations, each data object in the object store may be a fixed amount of data, such as 4 or 8 kibibytes of data, and metadata objects may also be a fixed amount of data, such as 1 kibibyte.

[0031] An object-based storage virtualization layer 150 may be distinct from block-level storage (e.g., implemented in a SAN and represented over a storage protocol such as iSCSI or Fibre Channel) and file-level storage (e.g., a file system that manages data in a file hierarchy and represented over a file-level protocol such as NFS or SMB / CIFS), although the object-based storage virtualization layer 150 may underlie block- or file-level storage protocols (e.g., by abstracting over mount points 180, 182, as described below).

[0032] The storage virtualizer 136 maintains an object index 154 that tracks a signature, a physical address, and a reference count for each object (data object and metadata object) in the object store 152. An object's signature may be a cryptographic digest of that object's contents using a hash function such as SHA-1, SHA-256, MD5, etc. Therefore, the signature may also be referred to as a content-based signature. The reference count in the object index 154 refers to the number of times the associated object is referenced across all virtual persistent volumes (including 156, 158) in the storage virtualization layer 150.

[0033] The physical address in object index 154 refers to the actual physical location of the object. While in some examples, object storage 152 may be understood as a storage construct to describe the storage of objects within storage virtualization layer 150, the objects may also be considered to be physically stored on the underlying storage at the physical address. Because data path 132 may utilize multiple storage mount points, the specific mount point may be part of the physical address. Additionally, the physical address may include a location within the memory allocation of a particular mount point. For example, if the mount point refers to physical storage 106, the physical address may include a logical block number.If the mount point refers to a public cloud storage 164, the physical address may contain a reference to a cloud object in the syntax of the corresponding hosting provider. The volume manager 138 is configured to perform data read and write operations for specific physical addresses, e.g., the physical addresses stored in the object index 154.

[0034] In some implementations, the containerized storage virtualization system 130 may employ an additional indirection layer between the object index 154 and the actual physical storage location of the objects. In such implementations, the volume manager 138 may allocate and / or partition the underlying storage allocations into extents (also referred to as mini-volumes) at each mount point provided by the control plane 140. The additional layer of indirection may be implemented by storing a virtual address instead of a physical address in the object index 154 in conjunction with an object signature and maintaining an extent table that maps a given virtual address to an extent and, thus, the corresponding underlying storage.Therefore, to access an object based on a virtual address, the volume manager 138 may first identify the extent targeted by the virtual address using the extent table and a first portion of the virtual address, and then locate the object within the extent using a second portion of the virtual address. In this way, certain data services, such as migration and tiering between extents, may be performed efficiently by updating only the extent identifiers in the extent table, rather than updating a large number of in-memory or persisted references to the data objects (i.e., each affected address in the object index 154) and re-creating various logical addresses, indexes, and other data structures used in managing the storage system.

[0035] Within the storage virtualization layer 150, the storage virtualizer 136 manages one or more virtual persistent volumes secured by the object store 152. In some implementations, a containerized application will be connected to a virtual PV in a one-to-one relationship. For example, in the example illustration of Fig. 1, the containerized application 124 is connected to the virtual PV 156, and the containerized application 126 is connected to the virtual PV 158. In some implementations, each virtual PV is mapped by the container orchestrator 122 to a corresponding persistent orchestrator volume managed by the container orchestrator 122, and the requesting containerized application 124 accesses the storage of the virtual PV 156 through the persistent orchestrator volume.

[0036] In other cases, containerized applications and virtual PVs may be interconnected in one-to-many, many-to-one, or many-to-many relationships. For illustrative purposes only, virtual PVs will now be described with reference to virtual PV 156, although the same description may apply to other virtual PVs, such as virtual PV 158 and other virtual PVs not shown.

[0037] In one implementation, the virtual persistent volume 156 may be an organization of metadata objects and data objects stored in the object store 152, where the organization hierarchically relates the data objects to a root object through associated content-based signatures. In one example, the virtual PV 156 may be a Merkle tree (also referred to as a hash tree) or another hierarchical arrangement (e.g., directed acyclic graphs, etc.). In the case of a hierarchical Merkle tree, data objects may be located at the lowest tree level of each branch (also referred to as the leaf level, which is farthest from the root object), and such data objects may be referred to as leaf data objects. As described above, the data objects constitute the data of the containerized application 124, such as files and directories.

[0038] Within the hierarchical arrangement, a parent object refers to an object that contains the signatures of child objects as its content. For example, a parent object of leaf-level data objects is a metadata object that stores the signatures of its child leaf-level data objects as its content. In this way, the signatures of objects at each level are collected in parent objects at the next higher level in the hierarchical arrangement until the root object is reached. The root object is therefore also a metadata object that stores the signatures of its respective child objects as its content. Viewed from another perspective, the hierarchical arrangement expands in a direction from the root object to the leaf level—a metadata object at any level can extend to a number of child nodes determined by a predefined branching factor.A metadata object may be able to store a set of signatures equal to at least the branching factor of the hierarchical arrangement, so that it can contain the signatures of all child objects.

[0039] Any change in the virtual PV's data (i.e., new data, modified data, deleted data) results in a change in the content of one or more leaf-level data objects, resulting in a change in the content-based signatures of these modified data objects, causing the changes in content and signatures to propagate upward through the parent nodes to the root object. Thus, a virtual PV 156 can be uniquely identified at a given point in time (also called a snapshot) by its root object, specifically by its root object signature.

[0040] Another aspect of the virtual PV 156 is that, in some implementations, a particular file or directory from the containerized application 124 may be stored in a corresponding subtree arrangement within the virtual PV 156. In other words, the virtual PV 156 may be divided into subtrees, each of which corresponds to a corresponding file or directory of the containerized application 124.

[0041] Because files and directories consist of one or more data objects, and these data objects are arranged in the virtual PV 156 and its subtrees by reference to associated data object signatures, in some implementations, each of the data objects may be physically stored only once in the object store 152 and referenced by their respective signatures in multiple metadata objects in the virtual PV 156 or in another virtual PV (e.g., 158) in the storage virtualization layer 150. In this way, the data may be deduplicated by the storage virtualizer 136. Similarly, metadata objects may be stored once and referenced multiple times by corresponding signatures. The number of times a data object or metadata object is referenced in the storage virtualization layer 150 may be recorded in the corresponding reference counts of the object index 154.In some implementations, data deduplication may be performed inline during a write operation, as opposed to post-processed or near-line deduplication, and in this way, data storage may be described as natively deduplicated in the storage virtualization layer 150 and between the virtual PVs 156, 158.

[0042] For use cases where security is a concern, including multi-tenancy scenarios, separate object stores can be used for each sensitive security domain. This allows sensitive data to be isolated in a secured object store without participating in deduplication of other virtual PVs that are not in the security domain.

[0043] To enable the containerized application 124 to access the virtual PV 156, the data path 132 may provide a mount point 180. The data path 132 may provide any of a variety of mount point types, including, but not limited to, a block-type mount point (e.g., iSCSI-compatible), a file-type mount point (e.g., a user-space file system or a FUSE interface, or NFS, SMB, or CIFS-compatible), a key / value share mount point (e.g., a NoSQL volume or an Amazon S3-compatible API), and other mount point types. In this way, the mount point can be understood as contributing to a complete abstraction of storage access, as any type of storage access required by the containerized application 124 (e.g., file protocol or block protocol, etc.) is provided to the containerized application 124.), regardless of the underlying type of storage that makes up the virtual PV 156 (e.g., regardless of software-based or hardware-based, block or file, local or remote, etc.).

[0044] The mount point type, i.e., the type of abstraction, can be selected by the user or predefined according to the containerized application 124 requesting the storage (i.e., based on a class of the containerized application specified by the container orchestrator 122, etc.). The type of abstraction can be indicated to the containerized storage virtualization system 130 via the storage request received at the control plane 140.

[0045] In the example of Fig. 1, the containerized storage virtualization system 130 may also provide a mount point 182 for the containerized application 126 to access a virtual persistent volume 158 created in response to a storage request from the containerized application 126 to the container orchestrator 122, in a manner similar to that described above for the mount point 180 of the virtual PV 156 for the containerized application 124. The virtual PVs 156 and 158 may both contain corresponding Merkle trees for organizing corresponding data sets, while the object store 152 stores the data of both virtual PVs 156 and 158 in a deduplicated manner.

[0046] In operation (i.e., after the virtual PV 156 has been created and the mount point 180 has been made available to the application 124), the storage virtualization system 130 may service input / output (I / O) requests from the containerized application 124 directed to a virtual PV 156 via the mount point 180. For example, to service a read request received via the mount point 180, the storage virtualizer 136 may identify the signatures of data objects in the virtual PV addressed by the read request (i.e., which may involve traversing the Merkle tree structure of the virtual PV based on an address of the read request) and determine the physical addresses of those data object signatures from the object index 154. In some implementations, the physical address of a data object may indicate the mount point of the underlying memory allocation where the data object is stored (e.g.,one or more of mount points 170, 172, 174, 176, or 178). The storage virtualization system can then, via volume manager 138, read the data objects using the physical addresses (or using a virtual address and an extent table, as described above) and return the read data to the containerized application 124 via mount point 180.

[0047] To service a write request, in example implementations, storage virtualizer 136 may receive the data to be written to virtual PV 156 from containerized application 124, check whether the data includes any data objects that are new to object store 152 based on content-based signatures, and write the new data objects to the object store (i.e., data objects that are not already present in the data store). In some implementations, storage virtualizer 136 may compress the new data objects before writing them to object store 152. Specifically, the process of writing to object store 152 may include controlling which underlying memory map (e.g., 106, 108, 160, 162, or 164) the new data objects are written to.In some implementations, the containerized application can specify in the write request which underlying storage map the data should be written to. In some implementations, new data can be written by default to a local storage area of ​​the virtual PV 156, such as the locally attached physical storage 106, which can provide "hot" storage optimized for frequent access. In some implementations, the containerized application can specify a specific policy or service-level agreement for writing the data, and the storage virtualization system can determine which underlying storage map satisfies that policy or SLA. The storage virtualization system then uses the mount point (e.g., 170, 172, 174, 176, or 178) corresponding to that underlying storage map to write the data objects.The storage virtualization system also adds the signature of the data objects to the metadata objects of the virtual PV 156.

[0048] Representing data in a virtual PV 156 that is natively deduplicated and uniquely identified by a root object signature can enable efficient data services, including those provided by the data service provider 134. For example, the data service provider 134 can perform, without limitation, snapshot-based backups, replications, migrations, tiering, redundancy-based data protection (e.g., redundant array of independent nodes, also referred to as RAIN or RAID), or other functions. The data service provider 134 can perform the data services with the virtual PV 156 in a manner that is transparent or non-intrusive to the containerized application 124. For example, the data services can be deployed without changing the mount point 180 and, in the case of some data services, without any input or instruction (e.g., configuration details, setup commands, etc.).) by a user, the containerized application 124, or the container orchestrator 122. Furthermore, in some examples, the data service provider 134 may manipulate data primarily on the storage virtualization layer 150 such that the data services are executed in a manner that is independent of the various underlying storage mounts and the particular composition of the virtual PV 156. In other words, the data service provider 134 may execute a common set of data services consistently and seamlessly, regardless of the type of underlying storage that forms the virtual PV 156. The aforementioned technical advantages may be enabled, for example, by the virtual PV 156 decoupling the underlying storage from the containerized application 124.

[0049] For example, the data service provider 134 may perform an efficient snapshot-based data backup service. In some implementations, the difference between temporal snapshots of a hierarchically arranged virtual PV 156 may be efficiently determined by comparing object signatures in an iterative top-down manner, starting with the root object, to find metadata and data objects that differ. For example, in an operation to back up a current state of the virtual PV 156 (i.e., a current snapshot), the current snapshot may be located on a primary system (e.g., computer system 100), and an older, previously backed up snapshot may already exist on a backup system (e.g., computer system 110).In this example, the difference between the current snapshot and the older snapshot can be determined by comparing signatures of the snapshots in the manner described previously, and the backup system can be searched to determine if the metadata or data objects that differ already exist on the backup system (i.e., in an object store of the backup system). Only the metadata or data objects that do not exist are copied from the primary system to the backup system, thereby reducing data traffic and improving backup times. In other implementations, snapshot-based backups can be performed on the same primary system instead of the backup system, or in addition to it, in a similar manner.

[0050] For example, the snapshot-based backup can be performed on a scheduled basis without interrupting the containerized application 124. Furthermore, the snapshot-based backup can be performed primarily on the software virtualization layer 150, directly avoiding the complexity of managing each underlying storage. Similar to the backup process, a restore process can also be performed by comparing the metadata or data objects to be restored with the objects already present on the restore target and transferring only those data that are not present on the restore target.

[0051] The data service provider 134 may also perform a migration operation. The migration may move data objects between different underlying storages within a virtual PV 156, including between different local storages, between local and remote storages, between different remote storages, between different public cloud storages, between public cloud storages and non-public cloud storages (either local or remote), or between other combinations of underlying storages. The migration is handled at the storage virtualization layer 150 by associating the new physical address of a moved data object with the unchanged content-based signature in the object index 154, making the migration transparent to the containerized application 124.

[0052] As another example, the data service provider 134 may migrate the virtual PV 156 to another computing system. For example, in some cases, it may be beneficial to migrate a virtual PV 156 to be proximate to a workload that uses the virtual PV 156. In an example scenario, the container orchestrator 122 may move the containerized application 124 to another computing system (e.g., from the source computing system 100 to the destination computing system 110) for load balancing or another reason, and the virtual PV 156 may need to be migrated to be proximate the containerized application 124. In some implementations, the storage virtualization system 130 may migrate management of the virtual PV 156 to another storage virtualization system on the destination computing system 110 to which the containerized application 124 has been migrated.The data service provider 134 may also migrate some of the data in the virtual PV 156, such as migrating data objects that were local to the computing system 100 (e.g., on the underlying physical storage 106) to the physical storage of the other computing system 110, which may be useful for maintaining storage locality and other performance characteristics of the virtual PV 156. Such a migration may involve identifying whether the target computing system 110 already has a copy of the metadata or data objects to be migrated and transferring only data objects that do not exist on the target computing system 110. At the same time, the same mount point 180 for the containerized application 124 may be maintained and uninterrupted.

[0053] The data service provider 134 may also perform data tiering within a virtual PV 156, i.e., move data between different types of underlying storage, which may have different characteristics and conform to different storage policies. Tiering may be implemented, for example, by allocating and / or partitioning the constituent storage allocations of the virtual PV 156 across different extents, as previously described. Under certain triggering conditions, the data service provider 134 (in conjunction with the volume manager 138 in some implementations) may move data objects from one extent to another extent and update the extent table accordingly.Example triggering conditions may include an increased security status of data, which may cause the data service provider 134 to move that data from the public cloud storage 164 to non-cloud storage 106, 108, 160, or 162; the aging of data, which may cause the data service provider 134 to move that data to an archival class of storage (e.g., to remote storage 160); recent frequent access to data, which may cause the data service provider 134 to move that data to high-performance storage (e.g., to local physical storage 106); or other types of conditions. By moving and managing data at the storage virtualization layer 150, data tiering can be performed across any type of underlying storage and without disruption to the mount point 180 or the containerized application 124.

[0054] The data service provider 134 may also support redundancy-based data protection. For example, the data service provider 134 may provide RAID data protection. For example, the data service provider 134 (in conjunction with the volume manager 138 in some implementations) may create a RAID set via underlying storage allocations or within an underlying storage allocation (e.g., in cases where the local physical storage 106 comprises a set of drives). The data service provider 134 may include a software RAID controller or may cooperate with a hardware RAID controller to write objects of the virtual PV 156 to the RAID set according to a RAID scheme such as RAID 1, RAID 5, or RAID 6.

[0055] The data service provider 134 may also provide RAIN data protection by replicating or mirroring data of a virtual PV 156 (also referred to as the primary virtual PV for convenience) for data protection and high availability purposes in accordance with the principles of the RAIN architecture. In some implementations, the data service provider 134 may initially replicate or mirror data when the data arrives at the storage virtualization layer 150 as a write request from the containerized application 124. In some implementations, the replicated data may form a virtual PV replica, which may have a similar shape to a virtual PV (e.g., including a Merkle tree) and may be managed by another storage virtualization system on another computing system 110 and local there, relative to the primary virtual PV local to the computing system 100.Additionally or alternatively, the virtual PV replica may consist of underlying storage that is different and / or separate from the underlying storage that comprises the primary virtual PV 156. Thus, if data on the primary virtual PV cannot be recovered, the data can be recovered from the virtual PV replica using a failover procedure.

[0056] In summary, a containerized storage virtualization system 130 that aggregates various storage types into a virtual persistent volume, which can be represented as any number of available storage abstractions, enables the containerized storage datapath to be highly virtualized from end to end. This allows users of containerized applications to obtain a high degree of flexibility in requesting any composition of underlying storage to meet the performance (or other) requirements of an application, while still being able to utilize the storage using any type of abstraction appropriate for the application.Furthermore, a consistent set of storage services can be provided regardless of the composition of the underlying storage and regardless of the type of storage access abstraction used to represent the virtual PV.

[0057] In Fig. 2 shows an example system that includes a policy engine 242. Fig. 2 also shows other elements, including a container orchestrator 222, a container storage interface 223, a containerized application 224, a data path 232, a container storage interface API 244, and one or more storage plugins (e.g., a storage plugin A 228-1, a storage plugin B 228-2, a storage plugin C 228-3). In some implementations, the aforementioned elements may be considered Fig. 2 exist within the same container environment. The policy engine 242 and the elements of Fig. 2 (i.e., 222, 223, 224, 232, 244, 228-1, 228-2, 228-3) may each be implemented as hardware or any combination of hardware and programming to implement their respective functionalities as described herein. The programming may, for example, be executable instructions stored on a non-transitory machine-readable medium, and the hardware may include a processing resource for retrieving and / or executing those instructions. Such a machine-readable medium and processing resource may, for example, be analogous in many respects to the machine-readable medium 104 and processing resource 102, respectively, of Fig. 1. The hardware may also include electronic circuits or logic.

[0058] The container orchestrator 222 may be analogous in many respects to the container orchestrator 122 described above. The container orchestrator 222 may serve as a platform for deploying and managing containerized applications, such as the containerized application 224, which may be similar to the containerized applications 124, 126 described above.

[0059] The container orchestrator 222 may include a container storage interface 223, which may be analogous to the container storage interface 123 described above. The container storage interface 223 may enable a plug-in to share a storage system with the container orchestrator 222 and an associated container environment (i.e., make it available for use and / or consumption). For example, the storage A plug-in 228-1 may make the storage system 229-1 available to the container orchestrator 222, the storage B plug-in 228-2 may make the storage system 229-2 available to the container orchestrator 222, and the storage C plug-in 228-3 may make the storage system 229-3 available to the container orchestrator 222.The plug-ins 228-1, 228-2, and 228-3 (also generally referred to as plug-in(s) 228) may be similar to the plug-ins 128 described above, and each of the storage systems 229-1, 229-2, and 229-3 (also generally referred to as storage, storage type(s), or storage system(s) 229) may be a local storage device (e.g., local storage devices (e.g., locally attached hard disk drives), local or remote storage arrays, software-defined storage, cloud storage, or the like), and may be similar to any of the storage systems or storage types described above, such as the physical storage 106, the local file system 108, the remote storage system 160, the virtualized storage 162, or the public cloud storage 164.

[0060] For a storage plug-in 228 deployed in the container orchestrator 222, the container orchestrator 222 may manage an object called a storage class containing parameters (e.g., in the form of key / value pairs) describing an associated storage 229. A storage class may describe characteristics of the offered storage (e.g., provisionable as a persistent disk via the container storage interface 223), and the described characteristics may include quality of service levels, backup policies, capacity, cost, performance (e.g., IOPS, latency), redundancy, recovery time objective (RTO), recovery point objective (RPO), or other measures. Different storage classes may describe different storage offerings. In some cases, a storage 229 associated with a provider may have multiple storage classes. Different storage 229 from different providers may have different storage classes.Storage classes may not be standardized between vendors or even within a vendor, as different storage classes may use different parameters.

[0061] In Fig. 2, the memory A slot 228-1 may have at least one associated storage class 226-1, the memory B slot 228-2 may have at least one associated storage class 226-2, and the memory C slot 228-3 may have at least one associated storage class 226-3. Storage classes such as memory A storage class 226-1, memory B storage class 226-2, and memory C storage class 226-3 may be referred to herein generally as storage class(es) 226. Although in Fig. 2 three memory slots 228, three memory systems 229 and three memory classes 226 are shown as an example, more or fewer memory slots, memory systems and / or memory classes may be present and available in different implementations.

[0062] In some implementations, the policy engine 242 may form at least a part of a storage virtualization system. For example, the policy engine 242 may be implemented as part of the containerized storage virtualization system 130 of Fig. 1, and in particular, it may serve as part of the control plane 140 and / or the policy engine 142 of Fig. 1 or form part of it. As will be described, the policy engine 242 may be useful, among other things, for virtualizing control plane operations to provide containerized applications (e.g., application 224) with a virtual persistent volume secured by one or more types of available storage to meet a policy objective, even though the available storage types are described by non-standardized storage classes. Furthermore, as will be described, the policy engine 242 may intelligently manage the virtual persistent volume with respect to an associated application to achieve an application-centric storage policy.

[0063] The policy engine 242 may include and / or interact with one or more elements, each of which may be implemented as hardware or as any combination of hardware and programming, to implement their respective functions as described herein. In some implementations, the policy engine 242 may include a scheduler 246, a discovery agent 248, a virtual PV manager 250 including a provisioner 252, a volume extension and replica manager 254, a container storage interface mapper 256, and / or a data path manager 258. In some implementations, the policy engine 242 may communicate (e.g., communicate bidirectionally) with the container storage interface 223 via a container storage interface API 244, which may be analogous to the container storage interface API 144 described above.In some implementations, the policy engine 242 may communicate with a data path 232, which may be analogous to the data path 132 described above, for example, via the data path manager 258.

[0064] The discovery agent 248 can determine which plug-ins 228 and associated storage 229 are available in the container environment. Additionally, the discovery agent 248 can retrieve storage classes 226 corresponding to the discovered plug-ins 228 and storage 229 and register the retrieved storage classes 226 with the policy engine 242. For example, the discovery agent 248 can collect the aforementioned information about plug-ins 228, storage 229, and storage classes 226 by querying the container orchestrator 222 or the container storage interface 223.

[0065] In some implementations, the discovery agent 248 may operate at startup or during an initialization phase of a storage virtualization system to which the policy engine 242 belongs, and / or it may operate periodically while the storage virtualization system is running. For illustrative purposes, a non-limiting example of a container environment may include three to six types of storage plug-ins 228. In various implementations, there may be one or more storage classes for each plug-in 228.

[0066] The storage classes 226 registered by the policy engine 242 may not be standardized, as described above. In other words, different storage classes may use different parameters, or more precisely, different keys and / or different values ​​for parameters defined as key / value pairs. The policy engine 242 may use a common schema 260 to translate each storage class 226 into a corresponding storage profile. For example, storage class 226-1 of storage A may be translated into profile 262-1 of storage A, storage class 226-2 of storage B may be translated into profile 262-2 of storage B, and so on. Storage profiles, such as storage A storage profile 262-1 and storage B storage profile 262-2, may be referred to generally herein as storage profile(s) 262. In some implementations, the common schema 260 may be manually coded, e.g.,based on the administrator's knowledge of the storage classes 226. The common schema 260 can be understood as a mapping between vendor-specific storage class parameters and a common language. In some implementations, as described below, the common language of the common schema 260 can be designed in correlation with the creation of the virtual PV storage class 227. In some implementations, the storage profiles 262 can be static, meaning they are not expected to change after translation.

[0067] To illustrate, the common schema 260 may be useful when an equal performance level in storage class 226-1 of storage A and in storage class 226-1 of storage B has the values ​​A and B, respectively, and is thus translated into an equal performance level value X in profile 262-1 of storage A and 262-2 of storage B. Thus, storage profiles 262-1 and 262-2 (translated from storage classes 226-1 and 226-2, respectively) use a common language to describe storage offerings for storage 229-1 and 229-2, respectively.

[0068] The policy engine 242 may also present or publish one or more virtual PV storage classes 227 to the container orchestrator 222. A virtual PV storage class 227 may be accessible to users alongside the storage classes 226. Similar to the storage classes 226, a virtual PV storage class 227 may describe the characteristics of a mountable persistent volume, particularly a persistent volume mountable by the policy engine 242 or by a storage virtualization system that includes the policy engine 242. In some implementations, a virtual PV storage class 227 and a virtual PV mounted according to the virtual PV storage class 227 may be assumed to have or satisfy a default or initial storage policy.

[0069] Various virtual PV storage classes 227 can be presented or published to the container orchestrator 222, such as storage classes for different service levels, such as Gold, Silver, and Bronze, each with different parameters. For example, each of the various virtual PV storage classes 227 can be designed by an administrator for different virtual storage policy purposes or predefined by a provider providing the policy engine 242. For illustration purposes only: The Fig. The virtual PV storage class 227 shown has the example characteristics of a “golden” storage class with the parameters type: tiered, performance: high and cost: high.

[0070] Unlike storage classes 226, virtual PV storage class 227 is natively compliant with shared schema 260. In other words, the key / value pairs of virtual PV storage class 227 are in the same language as shared schema 260 and do not require further translation. In some implementations, virtual PV storage classes 227 can be defined first, and shared schema 260 can be created based on the parameters of virtual PV storage classes 227.

[0071] The application 224 may request a persistent volume from the container orchestrator 222 either independently or via user input. The request may also include provisioning requirements, such as specifying that the virtual PV storage class 227 is the desired storage class for the requested persistent volume. Other requirements passed with the request may include, for example, a desired capacity. The container orchestrator 222, in turn, may forward a request to create a volume 230 to the policy engine 242, for example, via the container storage interface 223 and the container storage interface API 244. In some implementations, the container orchestrator 222 may extract the parameters of the virtual PV storage class 227 and pass the parameters to the policy engine 242 with the request 230.The request 230 can be received within the policy engine 242 from the Virtual PV Manager 250.

[0072] The virtual PV manager 250 may include various operators for manipulating virtual PVs managed by the policy engine 242, including operators for provisioning new virtual PVs, deleting virtual PVs, handling read / write I / O requests directed to virtual PVs, or other operations. These operators may each be hardware or a combination of hardware and programming to implement their respective functionalities. For example, the virtual PV manager 250 may include an operator 252 for provisioning new virtual PVs. In particular, the virtual PV manager 250 may determine that the request 230 is a request to provision a new virtual PV and cause the provisioner 252 to process the request 230.

[0073] In response to request 230, provisioner 252 begins creating a virtual PV, which is ultimately returned as virtual PV 280 (i.e., including at least one volume identifier) ​​to container orchestrator 222 for consumption or use as persistent storage by containerized application 224. Provisioner 252 determines that storage class 227 of the virtual PV was identified in request 230 as the desired type of virtual PV to be created. Provisioner 252 then evaluates each parameter in the requested virtual PV storage class 227 and determines which storage profile 262 has a corresponding parameter that can satisfy and match the parameter of virtual PV storage class 227.In some implementations, a parameter in the virtual PV storage class 227 is a minimum such that a closest match of the corresponding parameter among the storage profiles 262 exceeds the parameter value in the storage class 227. This match of a parameter of the virtual PV storage class 227 to a parameter of a storage profile 262 is enabled by the fact that the virtual PV storage class 227 and the storage profiles 262 all conform to and are in the language of the common schema 260. For example, if a requested parameter specifies "performance:high," the provisioner 252 would identify a storage profile 262 that also contains "performance:high."

[0074] In some implementations, the provisioner 252 may match the parameters of the virtual PV storage class 227 with the parameters of the storage profiles 262 based on the best match. For example, if no parameters of storage profiles 262 are found that match or exceed the parameters of the virtual PV storage class 227, the closest parameter may be selected. For example, if a requested parameter specifies performance: high, the provisioner 252 may select a storage profile with performance: medium over other storage profiles with performance: low.

[0075] In some implementations, the provisioner 252 may also determine whether assigning the virtual PV storage class 227 to a particular storage profile 262 would violate a resource limit or quota of an associated plug-in 228, and if a violation would occur, avoid assigning it to that plug-in 228. In some implementations, the provisioner 252 may check the remaining capacity of a storage 229 with an out-of-band query command (e.g., not via plug-ins 228, the container storage interface API 244, or the container storage interface 223).

[0076] In some implementations, the provisioner 252 may determine that multiple storage profiles 262 (e.g., more than one) can be combined to satisfy a parameter of the virtual PV storage class 227. For example, a request 230 may request performance:high and 2 terabytes (TB) of capacity. In this figure, however, storage profiles 262-1 and 262-2 with the performance:high parameters are limited to 1 TB volumes, either due to remaining capacity or other resource constraints. In this case, the provisioner 252 may decide to combine 1 TB of capacity from each of the storage profiles 262-1 and 262-2 to satisfy the 2 TB performance:high request.

[0077] In some cases, the provisioner 252 may determine that a single storage profile 262 satisfies all of the parameters of the virtual PV storage class 227. In some cases, the provider 252 may determine that multiple different storage profiles 262 are required to satisfy the parameters of the virtual PV storage class 227. For example, a virtual PV storage class 227 may be of a tiered type and specify high-performance primary storage (e.g., with low latency and / or high IOPS) in a first parameter and archival-class storage (e.g., with low cost and / or high write speed) for backup in a second parameter.In this example, for illustrative purposes, assume that profile 262-1 for storage A is associated with a high-performance storage array 229-1 and profile 262-2 for storage B is associated with an archive / backup storage system, and provisioner 252 may determine that the first parameter is satisfied by profile 262-1 for storage A and the second parameter is satisfied by profile 262-2 for storage B.

[0078] Finally, the provisioner 252 can create a mapping of the parameters of the virtual PV storage class 227 to parameters of one or more storage profiles 262 that serve as proxies for the storage classes 226, such that the parameters of the virtual PV storage class 227 are satisfied. Thus, the provisioner 252 also identifies the plug-ins 228 (corresponding to the mapped storage profiles 262) that should be used to assemble a virtual PV with the properties of the virtual PV storage class 227.

[0079] The provider 252 coordinates with the container storage interface Fig. to provision volumes from each of the storage plug-ins 228 associated with the mapped profiles 262 using their associated storage classes 226. The provisioner 252 informs the container storage interface mapping 256 of the profiles 262 and the plug-ins 228 that the provisioner 252 has identified and mapped to the virtual PV storage class 227, and the container storage interface mapping 256 performs the low-level replacement of each parameter in the virtual PV storage class 227 with one or more of the provisioned volumes using the plug-ins 228 identified by the provisioner 252.In this process, the container storage interface map 256 can translate parameters of the virtual PV storage class 227 into parameters in the storage classes 226 and then provision volumes using the plug-ins 228 and these translated parameters (since the plug-ins 228 are compatible with the storage classes 226). Provisioning volumes using plug-ins 228 can be similar to the process described above in . Fig. 1 described deployment can be carried out using plug-ins 128.

[0080] To illustrate, if the provisioner 252 has determined that the profile 262-1 of Storage A satisfies a first parameter of the virtual PV storage class 227, the container storage interface mapper 256 may translate this first parameter into a corresponding parameter of the storage class 226-1 of Storage A using the common schema 260 and then issue a create volume request 270 specifying the translated parameter of the storage class 226-1 of Storage A (and in some examples, additional parameters such as capacity) to the Storage A plug-in 228-1 via the container storage interface API 244 and the container storage interface 223.In return, the Storage-A plug-in 228-1 provisions storage from the storage system 229-1 according to the request 270 and returns a volume 272, which may contain a volume identifier or a persistent volume object or other control plane means for handling a provisioned volume. The container storage interface mapping 256 may repeat the process sequentially or in parallel for each parameter of the virtual PV storage class 227. Although in . Fig. 2 illustrates a provisioning request with plug-ins 228 for the sake of clarity, it should be understood that in some cases volumes may be provisioned by more than one plug-in 228 in accordance with the allocation determined by the provisioner 252.

[0081] In some implementations, the volume extension and replica manager 254 and / or the data path manager 258 (in conjunction with the data path 232) may provide additional functionality for creating and / or managing the virtual PV. The functionality of the volume extension and replica manager 254 may work in conjunction with the data path manager 258 / data path 232 or may be independent of the data path manager 258 / data path 232.

[0082] In some implementations, the data path manager 258 may send provisioning information 266 to the data path 232. The provisioning information 266 may be sent to the data path 232 while the policy engine 242 creates a virtual PV according to the virtual PV's storage class 227, which may be considered a default storage policy 274, and as such, the provisioning information 266 may be considered initial provisioning information 266, as opposed to the updated provisioning information 286 described further below.

[0083] The initial provisioning information 266 may include information about the volumes provisioned via plug-ins 228 to satisfy the virtual PV storage class 227, such as storage type(s), volume identifiers, etc. (e.g., identifying information about the disk 272). The data path 232 may then consume these provisioned volumes to create the virtual PV in a manner similar to that described above with respect to Fig. 1. In particular, the data path 232 may create the virtual PV as an organization (e.g., a Merkle tree) of metadata objects and data objects stored in the object store, hierarchically linked by associated content-based signatures up to a root object. In some implementations, the hierarchically organized virtual PV backed by the object store is returned to the container orchestrator 222 as a virtual PV 280, and the data path 232 may participate in servicing I / O requests as described above.

[0084] An extent, also called a mini-volume, refers to an organizational unit within the virtual PV. The virtual PV can comprise a single extent (corresponding, for example, to a single provisioned volume) or multiple extents. An extent can target a specific volume provisioned via a plug-in 228 or a subdivision thereof. Dividing the virtual PV into extents can enable I / O to be efficiently directed to the intended destination, for example, in the case of tiered storage with high-performance tier extents for hot data and archiving or backup tier extents for cold data. In this way, a single virtual PV can have automated built-in tiering capabilities, making data tiering simple and invisible to users and administrators.In another example, extents may be useful for managing separate volumes that are to be combined to satisfy a specific parameter of a virtual PV storage class. The volume extent and replica manager 254 may manage extents using an extent table in a similar manner as described above with respect to [ ]. Fig. 1, or outsource this administration in whole or in part to the data path 232.

[0085] In some implementations, the volume extension and replica manager 254 may coordinate a backup policy specified by the storage class 227 of the virtual PV throughout the lifecycle of the virtual PV. During the provisioning phase (i.e., after the provisioner 252 has identified primary storage and backup storage from profiles 262 to satisfy the storage class 227 of the virtual PV), the volume extension and replica manager 254 may coordinate with the container storage interface map 256 to first provision a backup volume (e.g., from a backup storage system 229-2 via plug-in 228-2) and then a primary storage volume (e.g., from a primary storage system 229-1 via plug-in 228-1) using a provisioning request that identifies the backup volume as a replication target. The storage system hosting the primary storage volume (e.g.,Storage system 229-1) can then use its own replication mechanism to back up data to the backup volume identified in the provisioning request (e.g., at storage system 229-2). In some implementations, a scheduler 246 can operate to schedule backups at a specific frequency or according to a specific schedule (e.g., as defined in virtual PV storage class 227). For each scheduled instance, scheduler 246 can instruct the primary storage volume plug-in (e.g., 228-1) to back up data to the target backup volume. In this way, a single virtual PV 280 can contain an automatic backup, simplifying backup configuration and maintenance for users and administrators.

[0086] Once the one or more volumes have been provisioned for the virtual PV via the plug-ins 228 in accordance with the mapping of storage class 227 to storage profile(s) 262, a volume mapping is recorded in the Virtualization Mappings data structure 264 that associates each provisioned volume with the virtual PV to be created. For example, if a VOL(2) was provisioned by the Storage A plug-in 228-1 (e.g., as volume 272) and a VOL(3) was provisioned by the Storage B plug-in 228-2 to satisfy the storage class 227 of the virtual PV, then a mapping of the virtual PV(1), consisting of VOL(2) and VOL(3), is recorded in the virtualization maps 264, as shown in Fig. 2. It should be understood that the foregoing is illustrative and that a virtual PV may consist of more or fewer provisioned volumes. If multiple virtual PVs are requested (e.g., for the same virtual PV storage class or for different virtual PV storage classes), additional mappings are stored in the virtualization map data structure 264. In this sense, the virtualization map data structure 264 can be considered dynamic, since new maps may be added when new virtual PVs are created and existing maps may be deleted when virtual PVs are deleted.

[0087] The policy engine 242 may return the virtual PV 280 (which, in some implementations, may include at least one volume identifier) ​​to the container orchestrator 222 via the container storage interface API 244 and the container storage interface 223, and the container orchestrator 222 may, in turn, provide access to the virtual PV 280 to the containerized application 224. The container orchestrator 222 may attach or associate the volume identifier of the virtual PV 280 in the metadata of the container orchestrator 222 to the containerized application 224. The containerized application 224 may then perform read and / or write operations on the virtual PV 280. The storage virtualization system, including the policy engine 242, may service I / O requests directed to the virtual PV 280 by mediating those I / O requests with the volumes of the storage 229 associated with the virtual PV 280.

[0088] In different implementations and scenarios, the type of virtual PV 280 provisioned by the policy engine 242 may vary. For example, if a single underlying volume is mapped and provisioned for the virtual PV storage class 227, the policy engine 242 may directly provision the provisioned volume (e.g., 272) as a virtual PV 280 without involving the datapath 232. In another example, a single underlying provisioned volume (e.g., 272) may be passed to the datapath 232 and virtualized as described above (e.g., as a hierarchical structure of objects identified by content-based signatures), and this virtualized datapath storage is provisioned as a virtual PV 280.In another example, multiple underlying volumes are mapped and provisioned for the virtual PV storage class 227, these volumes are virtualized via the data path 232 as described above, and this data path virtualized storage is provisioned as virtual PV 280. In . Fig. 2, a virtual PV 280 shown in dashed lines includes, for example, a storage volume 229-1 and a storage volume 229-2 provided by their respective plug-ins 228.

[0089] As previously mentioned, the virtual PV 280, as initially created and provisioned by the policy engine 242 in response to a request to create a volume 230 and in accordance with the virtual PV storage class 227, may be considered to have a default storage policy 274. That is, the virtual PV 280 may initially have properties based on a predefined virtual PV storage class, created, for example, by an administrator. The policy engine 242 may further tune the virtual PV 280 based on information about the requesting containerized application 224 to optimize the virtual PV 280 for that application 224. This tuning may occur before or while the containerized application 224 uses the virtual PV 280 as persistent storage.

[0090] To begin reconciliation, the policy engine 242 may use the volume identifier of the virtual PV 280 to request and read from the container orchestrator 222 an application manifest 282 of the containerized application 224 associated with that volume identifier. For example, a deployed containerized application 224 may be represented by one or more objects (e.g., Kubernetes objects), and these objects may be expressed as one or more JSON (JavaScript Object Notation) or YAML (YAML Ain't Markup Language) files. The manifest 282 is the collection of these objects (e.g., YAML files), and the manifest 282 may be understood as a definition of the containerized application 224.

[0091] The objects of the manifest 282 may each contain a label associated with the containerized application 224. Thus, to locate the manifest 282, the policy engine 242 may use the volume identifier of the virtual PV 280 to locate a persistent volume object with that volume identifier, read the label of that persistent volume object, and locate other objects with the same label.

[0092] The manifest 282 may specify properties such as a name, the image on which the containerized application 224 is based, a number of replicas (e.g., a number of compute replicas, which may relate to scaling), labels, a pod definition of which the containerized application 224 is a member, a reference to another application that uses or is related to the containerized application 224, or other properties. Examples of images may be Postgres, Nginx, MongoDB, MySQL, or other images. Such details may not be communicated with the request to create a volume 230, so the policy engine 242 has no idea what the storage requested via the request to create a volume 230 is intended for or what the type of containerized application 224 is.

[0093] The policy engine 242 may analyze the application manifest 282 by comparing information of the manifest 282 with one or more preconfigured application data store profiles, such as the one in Fig. 2 and the application data store profile B 284-2 (also commonly referred to as application data store profile(s) 284), although more or fewer application data store profiles may be available in various examples. The policy engine 242 may then identify a particular application data store profile that matches the manifest. For example, as discussed further below, each of the application data store profiles 284 is predetermined to provide a tailored store for a corresponding application type. In some implementations, an application data store profile 284 may include metadata for customization criteria that indicates which application type the profile is tailored for, including metadata such as a name, an image name, labels, number of replicas (e.g., compute replicas), and / or other properties.The policy engine 242 may identify a best-fit match between the properties of the application manifest 282 and the metadata information of the application data store profiles 284 (i.e., a best-fit match may be an application data store profile that covers more of the information of the manifest 282 than other application data store profiles). The best-fit match may be based on various logic, rules, filters, weightings, or the like, or any combination thereof (e.g., in some implementations, a match to a container image name may be given the most weight).

[0094] Before further discussing the tuning functionality of the policy engine 242, the application data store profiles 284 will be described in more detail. An application data store profile 284 may be a profile that describes the characteristics of a data store used by a particular application, and in particular, the characteristics may be optimized for the particular application. The application data store profiles 284 may be preconfigured based on an understanding of storage usage by corresponding application types. The application data store profiles 284 may be preconfigured manually by developers or in a semi-automatic or automated manner using machine learning techniques. In some implementations, the application data store profiles 284 may be, for example, human-readable text files.

[0095] The understanding of memory usage on which Profiles 284 are based can be gained in several ways. For example, in some cases, an application type associated with a particular image may be assumed to use a particular memory type based on domain knowledge or analysis of the image's source code (e.g., in the case of open source). In other examples, instances of the application type may be deployed, and while the instances are running, memory-related calls (e.g., at a file system level) may be intercepted or other memory-related behaviors may be observed (e.g., I / O patterns, etc.). These test observations and intercepted calls can be analyzed to infer memory usage.Memory usage determined by any of the aforementioned methods may include caching patterns, I / O patterns, replication behavior, or other characteristics that can be used as a prime indicator of how the corresponding application type utilizes memory. Different application types may be analyzed according to any of the aforementioned methods, and the application types may include different deployment configurations of the same application image (e.g., a fast Postgres instance, a small Postgres instance, a large Postgres instance). In some implementations, the analyzed application types may be selected based on the most common and / or popular types of containerized applications deployed.

[0096] Each application data store profile 284 may be structured to include storage parameters for configuring a virtual PV to match the storage usage determined for the corresponding application type. The storage parameters may specify aspects of a virtual PV, such as a type of underlying storage (e.g., with desired performance or cost characteristics), storage capacity, tiering, a replication policy, a backup policy, a data prioritization policy, or other storage characteristics. The storage usage of different application types and their configurations can vary widely; therefore, the storage parameters specified in the application data store profiles 284 may also vary. Once created, the application data store profiles 284 may be provided to the policy engine 242 to tune virtual PVs based on application manifests.

[0097] In some implementations, the profile engine 242 may receive and store one or more custom application data store profiles 285. For example, a user may create a custom application data store profile 285 to add to the preconfigured application data store profiles 284. For example, custom application data store profiles 285 may be useful for users with specific application and storage deployments that were not contemplated by the preconfigured application data store profiles 284. The author of a custom application data store profile 285 may also provide associated metadata for fitting criteria and a best-fit algorithm.

[0098] Returning to the reconciliation functionality: After the policy engine 242 has identified an application data store profile 284 that matches the application manifest 282, the policy engine 242 may (via the functionality of the virtual PV manager 250, the container storage interface mapper 256, the volume extension and replica manager 254, and / or the data path manager 258 described above) modify the virtual PV 280 according to the storage parameters of the identified application data store profile 284, such that the virtual PV 280 has an updated application-centric storage policy. In many cases, the changes to the virtual PV 280 can be made in a manner that is transparent and non-disruptive to the containerized application 224, particularly when the virtual PV 280 is virtualized and abstracted over the data path 232 and changes to the data path 232 (e.g.Changes to extents or to the object-based hierarchical arrangement of data) or to the underlying stores 229.

[0099] To illustrate, a virtual PV 280 under a default or initial storage policy may include a volume provisioned by storage A plug-in 228-1 and a volume provisioned by storage B plug-in 228-2, and the policy engine 242 may transition the virtual PV 280 into a modified virtual PV 280' that includes the volume provisioned by storage B plug-in 228-2 and a new volume provisioned by storage C plug-in 228-3.

[0100] Any changes to the constituent volumes of the virtual PV 280 (e.g., 280') can be updated in the virtualization cards 264. In some implementations, the policy engine 242 does not need to send updates about the changed virtual PV 280 to the container orchestrator 222 because the volume identifier of the virtual PV 280 remains the same for the virtual PV 280'.

[0101] Many types of changes can be applied to the virtual PV 280 in response to a variety of application data store profiles 284. Various non-limiting examples and use cases are described below. The following examples are not intended to be exhaustive, complete, or mutually exclusive.

[0102] In some implementations, the provisioner 252 may match the storage parameters of the identified application data storage profile 284 with the available storage profiles 262 on a best-fit basis to determine a new mapping to the storage profiles 262, in a manner similar to that described above for matching the parameters of the virtual PV storage class 227 in the first instance. Then, the container storage interface mapping 256 may perform a low-level replacement of the parameters of the identified application data storage profile 284 with parameters of storage classes 226 mapped to the storage profiles 262 in the new mapping, and provision volumes using the plug-ins 228 and the parameters of storage classes 226, replacing the parameters of the identified application data storage profile 284.The container storage interface map 256 may include logic to provision only new volumes that are not already part of the virtual PV manager 250, according to the current mapping of the virtual PV 280 in the virtualization maps 264.

[0103] The newly provisioned volumes and / or the existing volumes retained for the new mapping may be passed to the datapath 232 as an updated provision 286. In some implementations, the datapath 232 processes the updated provision 286 by adding new volumes as new extents to the virtual PV 280, and existing data may be migrated from the datapath 232 to new extents by changing indirections in the extent table and / or object index associated with the virtual PV 280. Extents from the previous mapping that are no longer needed under the new mapping may, in some implementations, be removed after the data is migrated. In this way, an extent composition of the virtual PV 280 may be changed to better match the storage usage of the containerized application 224 without disrupting the application 224.

[0104] The above example implementation of modifying the virtual PV 280 may be useful for scaling storage (e.g., when a manifest 282 indicates high application I / O, increased application performance, increased capacity requirements, increased scale factor, etc.) by, for example, adding additional volumes, larger volumes, and / or volumes with higher-performance media to the virtual PV 280. The above example implementation may also be useful for decreasing capacity (e.g., when a manifest 282 indicates low application I / O, reduced scale factor, etc.) by creating smaller, slower, and / or cheaper disks for the virtual PV 280 and migrating data to those disks.

[0105] In some example use cases, the identified application data store profile 284 may relate to data replication or data backup. For example, the identified application data store profile 284 may specify that the containerized application 224 should have backup and / or replication, even if the initial virtual PV storage class 227 did not request backup and / or replication. To illustrate, an application type that issues a single database may be designated by an administrator via the application data store profile 284 for automatic data protection via backups and / or replication by the policy engine 242.In another illustration, an application type characterized for CI / CD (Continuous Integration / Continuous Delivery) or A / B testing may benefit from data replicas, and therefore the corresponding application data store profiles 284 for these application types may specify one or more replicas. In some implementations, the volume extension and replica manager 254 may coordinate with the container storage interface map 256 to implement a backup policy for the virtual PV 280 in a manner similar to that described above. In some implementations, the container storage interface map 256 may coordinate with the data path manager 258 to implement data replication or mirroring for the virtual PV 280 on the data path 232 in a manner similar to that described above with respect to the data service provider 134 in FIG. Fig. 1 described, for example.

[0106] In another example use case related to replication, the application manifest 282 may have indicated that the application type performs its own replication, which may be referred to as application-level replication. In such cases, the identified application data store profile 284 may indicate that storage-level replication should be disabled. Accordingly, the container storage interface mapping 256 may coordinate with the data path manager 258 to terminate any ongoing data replication for the virtual PV 280 on the data path 232.

[0107] In some example use cases, the identified application data store profile 284 may relate to the caching behavior of the containerized application 224. For example, the manifest 282 may indicate that the containerized application 224 performs caching, the identified application data store profile 284 may specify that caching should be prioritized, and the container storage interface map 256, the data path manager 258, and the data path 232 may coordinate to prioritize cache-related I / O and / or deliver cache-related I / O to a high-speed storage area of ​​the virtual PV 280. To illustrate, when testing a Postgres instance, it may be determined that files are accessed frequently (e.g., log files, certain metadata, etc.).) written to a particular directory in a file system, and accordingly, the corresponding application data store profile may be constructed to indicate that a virtual PV should be represented as a file system mount point and that I / O to that particular directory should be prioritized. In response, the virtual PV manager 250 may coordinate with the data path manager 256 and the data path 232 to identify and prioritize I / O directed to the particular directory.

[0108] The profiles of the application data store 284 may be reviewed from time to time (e.g., periodically, on demand, on a scheduled basis, etc.). Depending on the review, new profiles may be added, outdated profiles removed, and existing profiles modified.

[0109] The policy engine 242 may read the most recent application manifest 282 from the container orchestrator 222 from time to time (e.g., periodically, on demand, on a schedule, etc.) and update the virtual PV 280 according to the current manifest 282 and a matching application data storage profile 284. For example, the manifest 282 may be updated as the containerized application 224 scales, and while there may be some solutions to scale compute performance as the containerized application 224 scales, no solutions are assumed to scale storage accordingly. However, with the policy engine 242 described herein, scaling changes (or any other type of change) in the manifest 282 may cause a responsive modification to an existing virtual PV 280 attached to and used by the containerized application 224 without interruption.Thus, virtual PVs provided by the policy engine 242 can achieve application-centric storage policies.

[0110] In some implementations, a published virtual PV storage class 227 may include a parameter specifying an application type for which that virtual PV storage class 227 is designed (e.g., "Postgres application type"). The policy engine 242 may create a virtual PV 280 in response to a create volume request 230 requesting such a virtual PV storage class 227 using an application data store profile 284 that matches the application type specified in the virtual PV storage class 227. Therefore, in such implementations, the initial virtual PV 280 may be application-specific and optimized to some extent for that specified application type.

[0111] Furthermore, specifying an application type in the virtual PV storage class 227 may be useful for providing a single underlying volume (e.g., 272) that is application-specific and presenting this single volume directly to the container orchestrator 222 as a virtual PV 280 without involving the data path 232. In such cases, it is useful to make the virtual PV 280 application-specific from the start, since otherwise it may be difficult to later change the composition of the virtual PV 280 (which comprises a single volume) without interrupting access, since this virtual PV 280 is not virtualized via the data path 232, as described above.

[0112] Fig. 3 and Fig. 4 are flowcharts illustrating various example methods. In some implementations, one or more blocks of the methods may execute substantially concurrently or in a different order than shown. In some implementations, a method may include more or fewer blocks than shown. In some implementations, one or more of the blocks of a method may be continuous and / or repeating at certain times. In some implementations, the blocks of the methods may be combined.

[0113] The Fig. 3 and Fig. 4 may be implemented in the form of executable instructions stored on a machine-readable medium and executed by a processing resource, and / or in the form of electronic circuits. For example, aspects of the methods may be described below as being executed by a storage virtualization system on a computer system, an example of which may be the containerized storage virtualization system 130 described above on the computer system 100, and may be further associated with a policy engine of such a storage virtualization system, such as the policy engine 242 described above. In addition, other aspects of the methods described below may be described with reference to other Fig. 2 elements shown are described for non-limiting illustration purposes.

[0114] Fig. 3 is a flowchart illustrating an example method 300. The method 300 begins at block 302 and proceeds to block 304, where a processor-based storage virtualization system 130 (or a policy engine 142 or 242 thereof) reads from a container orchestrator 222 a manifest 282 of a containerized application 224 assigned a virtual persistent volume 280.

[0115] In block 306, the storage virtualization system identifies an application data storage profile 284 that matches the information of the manifest 282 from a plurality of application data storage profiles 284. Each of the application data storage profiles 284 includes storage parameters preconfigured for a corresponding application type.

[0116] In block 308, the storage virtualization system modifies the virtual persistent volume 280 according to the application data storage profile 284 identified in block 306. Examples of block 308 may be, as further described below with respect to block 414 of Fig. 4. The method 300 ends in block 310.

[0117] Fig. 4 is a flowchart illustrating an example method 400. The method 400 begins at block 402 and proceeds to block 404, where the storage virtualization system receives a request 230 to create a persistent volume to provide persistent storage to a containerized application 224.

[0118] In block 406, the storage virtualization system creates a virtual persistent volume 280 according to a virtual persistent virtual storage class 227 specified by the "Create Persistent Volume" request 230. The virtual PV 280 thus created is considered to have a default storage policy.

[0119] In block 408, the storage virtualization system initiates autonomous ongoing tuning of the virtual PV 280 for the containerized application 224. The ongoing tuning may include repeating blocks 410, 412, and 414, which will now be described (e.g., periodically, on demand, on schedule, etc.).

[0120] In block 410, the storage virtualization system reads from a container orchestrator 222 a manifest 282 of a containerized application 224 assigned a virtual persistent volume 280. Block 410 may be analogous to block 304 of method 300.

[0121] In block 412, the storage virtualization system identifies an application data storage profile 284 that matches the information of the manifest 282 from a plurality of application data storage profiles 284, each of the application data storage profiles 284 containing storage parameters preconfigured for a respective application type. Block 412 may be analogous to block 306 of method 300.

[0122] In block 414, the storage virtualization system modifies the virtual persistent volume 280 according to the application data storage profile 284 identified in block 412. The modified virtual PV 280 resulting from block 414 is considered to have an updated application-centric storage policy. In some implementations, block 414 includes modifying an extent composition of the virtual PV 280 and migrating data of the virtual PV 280 between the modified extent composition.

[0123] Block 414 further includes some non-limiting, non-mutually exclusive example use cases illustrated in blocks 416, 418, 420, 422, 424, 426. For example, in block 416, the storage virtualization system determines whether a scaling change is indicated by the manifest 282 reflected in the application data store profile 284 identified in block 412. For example, the manifest 282 may specify how many nodes should be deployed with the containerized application 224, and this number may increase or decrease. For example, the application data store profiles 284 may be preconfigured for different sized configurations of an application matching the application type 224.

[0124] In response to the identified application data storage profile 284 matching information from the manifest 282 indicating increased scaling of the containerized application 224, the modification in block 414 may include increasing the size of the virtual PV 280. On the other hand, in response to the identified application data storage profile 284 matching information from the manifest indicating decreased scaling of the containerized application, the modification in block 414 may include shrinking the size of the virtual PV 280. Scaling up or down the virtual PV 280 may include changing the extent composition of the virtual PV 280 to include additional, larger, or faster storage volumes, or fewer, smaller, or cheaper storage volumes, respectively, and migrating data to the changed extent configuration.

[0125] In block 420, the storage virtualization system determines whether a change to data replication or backup should occur according to the application data storage profile 284 identified in block 412. For example, the manifest 282 may relate to data replication or backup for the containerized application 224 by, for example, directly or indirectly indicating that an automatic backup policy is desired for the application 224, that the application 224 has native application-level replication, or that the application 224 is a type that uses replicas (e.g., for CI / CD or A / B testing).

[0126] In response to an affirmative determination in block 420, the storage virtualization system may change a data replication behavior or a data backup behavior of the virtual PV 280 in block 422. In some implementations, the data replication or data backup in block 422 may be implemented at least in part by the volume extension and replica manager 254 as described above. In other implementations, the data replication or data backup may be disabled by the storage virtualization system through block 422, e.g., if the manifest 282 indicates that application-level replication is present.

[0127] In block 424, the storage virtualization system determines whether to perform a change in caching behavior according to the application data storage profile 284 identified in block 412. For example, the manifest 282 may refer to caching behavior by directly or indirectly indicating that the containerized application 224 implements caching, frequently accesses certain data, or the like, which may be implied from an image type in some implementations.

[0128] In response to an affirmative determination in block 424, the storage virtualization system may, in block 426, cause a data path 232 to begin identifying and prioritizing read and write operations of cache data of the containerized application 224. For example, read and write operations to a cache-related directory in a file-oriented virtual PV 280 may be prioritized.

[0129] In block 428, the storage virtualization system determines whether voting should continue. For example, an administrator may disable voting. If voting should continue ("YES" in block 428), method 400 returns and repeats the reading in block 410, the identification in block 412, and the modification in block 414. If voting is canceled ("NO" in block 428), method 400 ends in block 430.

[0130] Fig. 5 shows an example system 500 including a non-transitory, machine-readable medium 504 encoded with example instructions 506, 508, 510 that can be executed by a processing resource 502. In some implementations, the system 500 can be used to implement aspects of the storage virtualization system 130 of Fig. 1 or the Policy Engine 242 from Fig. 2 or to carry out aspects of procedure 300 of Fig. 3 or procedure 400 of Fig. 4 may be useful. For example, instructions 506, 508, 510 in instructions 105 of Fig. 1. In some implementations, the Fig. 2 described functionality may be contained in instructions 506, 508, 510.

[0131] Processing resource 502 may include a microcontroller, a microprocessor, central processing unit core(s), an ASIC, an FPGA, and / or other hardware device suitable for retrieving and / or executing instructions from machine-readable medium 504 to perform functions related to various examples. Additionally or alternatively, processing resource 502 may include or be coupled to electronic circuitry or dedicated logic for performing some or all of the functions of the instructions described herein.

[0132] Machine-readable medium 504 may be any medium suitable for storing executable instructions, such as RAM, ROM, EEPROM, flash memory, a hard disk drive, an optical disk, or the like. In some example implementations, machine-readable medium 504 may be a tangible, non-transferable medium. Machine-readable medium 504 may be located within system 500, where the executable instructions may be considered installed or embedded on system 500. Alternatively, machine-readable medium 504 may be a portable (e.g., external) storage medium and may be part of an installation package.

[0133] As described further below, the machine-readable medium 504 may be encoded with a set of executable instructions 506, 508, 510. It is understood that some or all of the executable instructions and / or electronic circuitry contained in one package may, in alternative implementations, be contained in a different package shown in the figures or in a different package not shown. Some implementations of the system 500 may contain more or fewer instructions than in Fig. 5 are shown.

[0134] Instructions 506, when executed, cause processing resource 502 to read from a container orchestrator (e.g., 222) a manifest (e.g., 282) of a containerized application (e.g., 224) assigned a virtual persistent volume (e.g., 280). Instructions 508, when executed, cause processing resource 502 to identify an application data storage profile that matches information in the manifest from a plurality of application data storage profiles (e.g., 284), each containing preconfigured storage parameters for a respective application type. Instructions 510, when executed, cause processing resource 502 to modify the virtual PV according to the application data storage profile identified by instructions 508.

[0135] Fig. 6 shows an example system 600 including a processing resource 602 coupled to a non-transitory, machine-readable medium 604 encoded with example instructions 606, 608, 610, 612, 614, 616 (collectively, instructions 606-616). The processing resource 602 and the machine-readable medium 604 may be analogous in many respects to the processing resource 502 and the machine-readable medium 504, respectively. In some implementations, the system 600 may be used to implement aspects of the storage virtualization system 130 of Fig. 1 or the Policy Engine 242 from Fig. 2 or to carry out aspects of procedure 300 of Fig. 3 or procedure 400 of Fig. 4 may be useful. For example, instructions 606-616 in instructions 105 of Fig. 1. In some implementations, the Fig.The functionality described in section 2 must be included in instructions 606-616.

[0136] When executed, instructions 606 cause processing resource 602 to receive a persistent volume creation request (e.g., 230) to provide persistent storage to a containerized application (e.g., 224) and to create the virtual PV (e.g., 280) according to a storage class of the virtual PV (e.g., 227) specified by the persistent volume creation request. The virtual PV created by instructions 606 is considered to have a default storage policy.

[0137] Instructions 608, when executed, cause processing resource 602 to read from a container orchestrator (e.g., 222) a manifest (e.g., 282) of a containerized application (e.g., 224) associated with a virtual persistent volume (e.g., 280). Instructions 610, when executed, cause processing resource 502 to identify an application data storage profile matching information in the manifest from a plurality of application data storage profiles (e.g., 284), each containing preconfigured storage parameters for a corresponding application type.

[0138] When executed, instructions 612 cause processing resource 602 to modify the virtual PV according to the application data storage profile identified by instructions 610. The modified virtual PV resulting from the execution of instructions 612 is considered to have an updated application-centric storage policy compared to the virtual PV created by instructions 608.

[0139] For example, if the manifest information indicates an increased scale of the containerized application (e.g., increased scale factor, increased performance, or storage capacity requirement), the application data storage profile identified by instruction 610 adapts to the increased scale, and instructions 612 scale the virtual PV accordingly, for example, by adding additional and / or more powerful underlying storage volumes. In some implementations, if the manifest information indicates a decreased scale of the containerized application (e.g., decreased scale factor, performance, or storage capacity requirement), the application data storage profile identified by instruction 610 adapts to the decreased scale, and instructions 612 scale down the virtual PV accordingly, for example, by migrating data to smaller and / or less expensive storage volumes.

[0140] In another example implementation, when the information of the manifest relates to data replication or data backup for the containerized application, the instructions 612 change a data replication behavior or a data backup behavior of the virtual PV according to the identified application data store.

[0141] In another example implementation, when the manifest information relates to caching behavior of the containerized application, instructions 612 may cause a datapath to identify and prioritize reads and writes of cache data of the containerized application to provide cache prioritization to the virtual PV.

[0142] Instructions 614, when executed, cause processing resource 602 to modify an extent composition of the virtual PV and may cause processing resource 602 to migrate data of the virtual PV under the modified extent composition. Instructions 614 may be included in or called by instructions 612 to perform at least some of the modification of the virtual PV. For example, instructions 614 may be used in increasing or decreasing the size of the virtual PV by changing the extent composition, as described above.

[0143] Instructions 616, when executed, cause processing resource 602 to repeat read instructions 608, identify instructions 610, and modify instructions 612 to provide autonomous ongoing reconciliation of the virtual persistent volume for the containerized application. For example, instructions 616 may be executed automatically, semi-autonomously, or manually, from time to time, periodically, on a scheduled basis, on demand, or the like.

Claims

[1] A non-transitory, machine-readable medium (504) storing instructions for a policy engine (242) of a storage virtualization system (130) that are executable by a processing resource (102; 502), the non-transitory, machine-readable medium (504) comprising: Instructions to read from a container orchestrator (122; 222) a manifest (282) of a containerized application (124; 224) assigned a virtual persistent volume (280); Instructions for identifying an application data storage profile (284) that matches information of the manifest (282) from a plurality of application data storage profiles, each containing preconfigured storage parameters for a respective application type; and Instructions for modifying the virtual persistent volume (280) according to the identified application data storage profile (284), wherein, if the information of the manifest (282) indicates an increased scaling of the containerized application (124; 224), the identifying instructions cause the identification of an application data storage profile (284) that matches the increased scaling, and the modifying instructions cause the virtual persistent volume (280) to be increased in accordance with the application data storage profile (284) that matches the increased scaling, and if the information of the manifest (282) indicates a reduced scaling of the containerized application (124; 224), the instructions for identifying the cause for identifying an application data store profile (284) that matches the reduced scaling, and the instructions for changing the cause for shrinking the virtual persistent volume (280) according to the application data store profile (284) that matches the reduced scaling. [2] The non-transitory machine-readable medium (504) of claim 1, further comprising: Instructions for receiving a request (230) to create a persistent volume for providing persistent storage for the containerized application (124; 224); and Instructions for creating the virtual persistent volume (280) according to a virtual persistent virtual storage class (227) specified by the request (230) to create a persistent volume, wherein it is assumed that the virtual persistent volume (280) thus created has a default storage policy, assuming that the modified virtual persistent volume (280) has an updated application-centric storage policy. [3] The non-transitory machine-readable medium (504) of claim 1 or 2, further comprising: instructions for repeating the instructions for reading, the instructions for identifying, and the instructions for modifying to provide autonomous ongoing tuning of the virtual persistent volume (280) for the containerized application (124; 224). [4] The non-transitory machine-readable medium (504) of claim 1, wherein the information of the manifest (282) relates to data replication or data backup for the containerized application (124; 224) and the instructions for modifying change a data replication behavior or a data backup behavior of the virtual persistent volume (280) according to an application data store identified to match the information related to data replication or data backup. [5] The non-transitory machine-readable medium (504) of claim 1, wherein, when the information of the manifest (282) relates to a caching behavior of the containerized application (124; 224), the instructions for modifying are executed according to an application data storage profile (284) identified to match the information relating to the caching behavior, causing a data path (132; 232) to identify and prioritize read and write operations of cache data of the containerized application (124; 224). [6] The non-transitory machine-readable medium (504) of claim 1, wherein the instructions for modifying comprise: Instructions to modify an extent composition of the virtual persistent volume (280), and Instructions for migrating virtual persistent volume (280) data under modified extent composition. [7] A procedure that includes: Reading, by a processor-based storage virtualization system (130) and by a container orchestrator (122; 222), a manifest (282) of a containerized application (124; 224) assigned a virtual persistent volume (280); Identifying, by the storage virtualization system (130), an application data storage profile (284) that matches information of the manifest (282) from a plurality of application data storage profiles, each containing preconfigured storage parameters for a corresponding application type; and Modifying the virtual persistent volume (280) by the storage virtualization system (130) according to the identified application data storage profile (284), wherein, in response to identifying an application data storage profile (284) that matches information of the manifest (282) indicating increased scaling of the containerized application (124; 224), modifying comprises increasing the size of the virtual persistent volume (280), and in response to identifying an application data storage profile (284) that matches information of the manifest (282) indicating a reduced scale of the containerized application (124; 224), modifying comprises reducing the size of the virtual persistent volume (280). [8] The method of claim 7, further comprising: Receiving a request (230) to create a persistent volume by the storage virtualization system (130) to provide persistent storage to the containerized application (124; 224); and Creating the virtual persistent volume (280) by the storage virtualization system (130) according to a virtual persistent virtual storage class (227) specified by the request (230) to create a persistent volume, wherein the virtual persistent volume (280) thus created is considered to be a default storage policy, assuming that the modified virtual persistent volume (280) has an updated application-centric storage policy. [9] The method of claim 7 or 8, further comprising: repeating the reading, identifying, and modifying as part of an autonomous ongoing reconciliation by the storage virtualization system (130) and the virtual persistent volume (280) for the containerized application (124; 224). [10] The method of any one of claims 7 to 9, wherein modifying comprises modifying an extent composition of the virtual persistent volume (280) and migrating data of the virtual persistent volume (280) under the modified extent composition. [11] The method of claim 7, wherein, in response to the information of the manifest (282) relating to data replication or data backup for the containerized application (124; 224), modifying comprises changing a data replication behavior or a data backup behavior of the virtual persistent volume (280). [12] The method of claim 7, wherein, in response to the information of the manifest (282) relating to a caching behavior of the containerized application (124; 224), modifying the virtual persistent volume (280) comprises causing a data path (132; 232) to identify and prioritize read and write operations of cache data of the containerized application (124; 224). [13] A system consisting of: a processing resource (102; 502); and a non-transitory machine-readable medium (504) storing instructions for a storage virtualization system (130) that, when executed, cause the processing resource (102; 502) to: Creating a virtual persistent volume (280) with a default storage policy in response to a request (230) to provide persistent storage to a containerized application (124; 224), and assigning the virtual persistent volume (280) to the containerized application (124; 224), read from a container orchestrator (122; 222) a manifest (282) of the containerized application (124; 224) to which a virtual persistent volume (280) is assigned, Identifying an application data storage profile (284) that matches information of the manifest (282) from a plurality of application data storage profiles, each containing preconfigured storage parameters for a corresponding application type, and modify the virtual persistent volume (280) according to the identified application data storage profile (284) to cause the virtual persistent volume (280) to have an updated application-centric storage policy, wherein if the information of the manifest (282) indicates an increased scaling of the containerized application (124; 224), the identified application data storage profile (284) matches the increased scaling and the instructions for modifying increase the virtual persistent volume (280), and if the information of the manifest (282) indicates a reduced scaling of the containerized application (124; 224), the identified application data storage profile (284) matches the reduced scaling and the instructions to change reduce the virtual persistent volume (280). [14] The system of claim 13, wherein the instructions that cause the processing resource (102; 502) to modify the virtual persistent volume (280) further cause the processing resource (102; 502): modify an extent composition of the virtual persistent volume (280) according to the identified application data storage profile (284), and Migrate virtual persistent volume data under the changed extent composition. [15] The system of claim 13 or 14, wherein the instructions further cause the processing resource (102; 502) to perform ongoing tuning of the virtual persistent volume (280) for the containerized application (124; 224), wherein the ongoing tuning comprises repeatedly executing a group of commands including the commands that cause the processing resource (102; 502) to read the manifest (282) of the containerized application (124; 224), identify the application data storage profile (284) that matches information from the manifest (282), and modify the virtual persistent volume (280) according to the identified application data storage profile (284). [16] The system of any one of claims 13 to 15, wherein when the information of the manifest (282) relates to data replication or data backup for the containerized application (124; 224), the processing resource (102; 502) modifies the virtual persistent volume (280) by changing a data replication behavior or a data backup behavior of the virtual persistent volume (280) according to an application data store identified to match the information about data replication or data backup. [17] The system of any one of claims 13 to 15, wherein, when the information of the manifest (282) relates to a caching behavior of the containerized application (124; 224), the processing resource (102; 502) modifies the virtual persistent volume by causing a data path (132; 232) to identify and prioritize read and write operations of cache data of the containerized application (124; 224).

Images