Extending file system functionality in container orchestration systems

Abstract

The functional capabilities of a file system in a container orchestration system are extended by associating an extended storage class with a functional extension, and access to a data storage volume belonging to the extended storage class is handled by the extended file system. In response to deployment of an application pod configuration including a definition of an extended volume belonging to the extended storage class, a request is made to mount the extended volume, a staging pod is created for mounting the original data storage volume, and the container orchestration system is caused to deploy the staging pod. Access to data stored in the extended volume by a container in the application pod is handled by the extended file system.

Description

[Technical Field]

[0001] The present invention relates generally to container orchestration systems, and more particularly to extending file system functionality in a container orchestration system without requiring modifications to the container orchestration runtime environment. [Background technology]

[0002] Container orchestration systems automate the deployment, maintenance, and scaling of distributed applications by managing containerized workloads and services, facilitating both declarative configuration and automation. In a containerized system, the kernel enables multiple isolated user-space instances, sometimes called containers, as used, for example, in the well-known container orchestration system Kubernetes. Kubernetes is described at https: / / kubernetes.io, which is incorporated herein by reference. Numerous references also exist, for example, B. Burns, J. Beda, and K. Hightower, Kubernetes Up & Running, O'Reilly (2019), which is incorporated herein by reference.

[0003] However, container orchestration systems such as Kubernetes are limited in their ability to disrupt the state of application deployments. For example, Kubernetes does not support file system layering, and therefore does not provide a mechanism for adding third-party file system protections, such as encryption and access control policies, to applications deployed through the container orchestration system.

[0004] From the above, it is apparent that there is a need for improved ways of providing post-release extensions to applications deployed through container orchestration systems, including providing file system layering in container orchestration systems, thereby providing mechanisms for providing file system protections, such as encryption and access control policies, to applications deployed through container orchestration systems. Summary of the Invention

[0005] The present invention provides a solution to the aforementioned problem by a method for extending the capabilities of a file system in a container orchestration system without making changes to the runtime environment of the container orchestration system according to claim 1 and a container orchestration system according to claim 12. The dependent claims define preferred embodiments of the invention.

[0006] In a first inventive aspect, the present invention provides a method for extending the capabilities of a file system in a container orchestration system without modifying the runtime environment of the container orchestration system, the file system extension providing functionality not otherwise provided by the file system of the container orchestration system, the method comprising: Associating an extended storage class with an extension that handles access to data storage volumes belonging to the extended storage class by the extended file system; -Associating the original volume with a storage class other than the Enhanced storage class, In response to deploying an application pod that is configured to include an extended volume definition that belongs to an extended storage class, Requesting that the extended volume be mounted, Create a staging pod to mount the original data storage volume through the extended file system, and deploy the staging pod in the container orchestration system; Mounting the original data storage volume by the staging pod; Determining a path that identifies the mount of the original data storage volume; Mounting an extended volume belonging to an extended storage class using a path that identifies the mounting of the original data storage volume and using the original data storage volume as a data source; Exposing the mount of the extended volume to the containers in the application pod, and Putting the application pod into the execution phase and handling access to data stored in the extended volume by the containers in the application pod through the extended file system. The present invention provides a method comprising:

[0007] Extending file system functionality in container orchestration systems has proven difficult. Container orchestration systems, such as Kubernetes, operate as state machines. This paradigm limits the changes that can be made to a containerized application after it is deployed. As a result, it is desirable to provide a mechanism that allows for the extension of container orchestration file system functionality while still operating within the container orchestration as a state machine paradigm.

[0008] Some embodiments of the present invention extend the capabilities of a file system in a container orchestration system, such as Kubernetes, without making changes to the runtime environment of the container orchestration system, where the file system extensions provide functionality not otherwise provided by the file system of the container orchestration system.

[0009] The enhancement includes associating the enhancement storage class with an enhancement that handles access to a data storage volume belonging to the enhancement storage class by the enhancement file system; associating an original volume with a storage class other than the enhancement storage class; requesting mounting of the enhancement volume in response to deployment of an application pod configured to include a definition of an enhancement volume belonging to the enhancement storage class; creating a staging pod by the enhancement file system for mounting the original data storage volume; having a container orchestration system deploy the staging pod; and mounting the original data storage volume by the staging pod. This is achieved by determining a path identifying the mount of the original data storage volume, using the path identifying the mount of the original data storage volume to mount an extended volume belonging to an extended storage class using the original data storage volume as a data source, exposing the mount of the extended volume to containers in the application pod, and placing the application pod in the execution phase, whereby access to data stored in the extended volume by containers in the application pod is handled by the extended file system.

[0010] In one embodiment, extending the capabilities of a file system in a container orchestration system includes file system layering, and in another aspect, the extending capability is layering a first volume onto a second volume to provide a combined volume including the first volume and the second volume.

[0011] In some embodiments, the original volume is created as a persistent volume and a corresponding persistent volume request. In alternative embodiments, the original volume is created as a temporary volume and a corresponding temporary volume request.

[0012] In some embodiments, the extended volume is created as a persistent volume and a corresponding persistent volume request. In the alternative, the extended volume is a temporary volume and a corresponding temporary volume request.

[0013] In one embodiment, the extended volume is accessed by an application container, and the access is organized by a driver associated with an extended storage class, which, for example, provides encryption services for data stored on the extended volume and enforces access control policies for data stored on the extended volume.

[0014] In a second inventive aspect, the present invention provides a container orchestration system capable of extending file system capabilities by providing extensions to an original data storage volume provided by the container orchestration system without modifying the runtime environment of the container orchestration system, the file system extension providing functionality not otherwise provided by the file system of the container orchestration system; At least one node having a processor and a memory, the memory being connected to the processor Accepting an association between an extension storage class and a file system extension where access to data storage volumes belonging to the extension storage class is handled by the file system extension's layered file system driver; Accept the association of the original volume with a storage class other than the Enhanced storage class. at least one node storing instructions to cause the The memory further comprises an agent associated with an extended storage class, the agent being associated with the processor. receiving a mount request for an extended volume; Create a staging pod to mount the original data storage volume and have the container orchestration system deploy the staging pod; Determining a path that identifies the mount of the original data storage volume; Mounting an extended volume belonging to an extended storage class using a path that identifies the mounting of the original data storage volume and using the original data storage volume as a data source; Expose the mount of the extended volume to the containers in the application pod and storing an agent that performs the The memory further includes instructions including staging pods, Mounting the original data storage volume and storing an instruction to perform the The memory further stores instructions to cause the processor to enter an execution phase in response to the application receiving notification that the extended volume has been mounted, and access to data stored in the extended volume by a container in the application pod is handled by a driver of the extended file system. Provides a container orchestration system.

[0015] All features described in this specification (including the claims, description and drawings) and / or all steps of the methods described may be combined in any combination, except for such mutually exclusive combinations of features and / or steps. [Brief explanation of the drawings]

[0016] These and other features and advantages of the present invention will be clearly understood from a consideration of the detailed description of the invention which refers to the drawings and which are made clear from preferred embodiments of the invention, given by way of example only and not by way of limitation.

[0017] [Figure 1]Block diagram of a cluster managed using a container orchestration system. [Figure 2] Flow diagram showing the typical flow of a Kubernetes cluster regarding storage, specifically the creation and consumption of persistent storage. [Figure 3] 1 is a high-level block diagram illustrating an example cluster resulting from execution of an example configuration file. [Figure 4] FIG. 1 is a high-level architecture diagram illustrating one embodiment that supports extending file system functionality in a container orchestration system. [Figure 5] 1 a, b, and c are timing sequence diagrams illustrating process steps for providing extension capabilities to application pods in a container orchestration system. [Figure 5a] FIG. 1 is a timing sequence diagram illustrating process steps for providing extensions to application pods in a container orchestration system. [Figure 5b] FIG. 1 is a timing sequence diagram illustrating process steps for providing extensions to application pods in a container orchestration system. [Figure 5c] FIG. 1 is a timing sequence diagram illustrating process steps for providing extensions to application pods in a container orchestration system. [Figure 6] FIG. 6 is a block diagram corresponding to FIG. 3 but showing the result of the flow of FIG. 5. DETAILED DESCRIPTION OF THE INVENTION

[0018] As will be appreciated by those skilled in the art, aspects of the present invention may be embodied in an identification card, an electronic device-based module, or a method for determining the quality of a captured image of an electronic device.

[0019] In the following detailed description, reference is made to the accompanying drawings, which show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It is to be understood that various embodiments of the invention, although different, are not necessarily mutually exclusive. For example, a particular feature, structure, or characteristic described herein in connection with one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. It is also to be understood that the location or arrangement of individual elements within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention is defined solely by the appended claims, appropriately interpreted, along with the full range of equivalents to which such claims are entitled. In the drawings, like numerals refer to the same or similar functionality throughout the several views.

[0020] The following description includes references to various methods that are performed by a processor of an integrated circuit chip. As is common in the art, there may be phrases herein that indicate that these methods or method steps are performed by software instructions or software modules. As those skilled in the art will appreciate, such descriptions should be interpreted to mean that the processor actually executes the methods, software instructions, and software modules, and that such software instructions and modules are stored in an instruction memory.

[0021] The technology described herein provides a mechanism for providing post-publication extensions to applications deployed through a container orchestration system, including providing file system layering to the container orchestration system, thereby enabling the addition of file system encryption and access control policies to applications deployed through a container orchestration system without requiring modifications to the application itself. These file system extensions are provided to containerized applications without requiring modifications to the runtime environment provided by the container orchestration system.

[0022] definition A cluster is a collection of related machines that are collectively managed by a cluster administrator, who, in the case of Kubernetes, might manage the cluster through a command line tool known as kubectl.

[0023] The machines that make up a cluster are called nodes. A node can be either a virtual machine or a physical machine. However, because a virtual machine ultimately runs on one or more physical machines, improving the capabilities of a cluster of nodes improves the capabilities of the physical machines.

[0024] A cluster has at least two types of nodes: worker nodes and master nodes. Containerized applications reside on worker nodes, while Kubernetes services are typically hosted on master nodes.

[0025] A container, or the environment in which a containerized application runs, provides a defined set of resources available to the containerized application running within a given container. A container contains all the definitions a containerized application needs to run: runtime, libraries, and storage definitions. Therefore, a developer of a containerized application can expect their containerized application to run correctly no matter where it is run, because the container definition remains the same regardless of which machine the containerized application is deployed to.

[0026] Kubernetes runs applications by placing containers into pods. A pod is the smallest deployable unit in Kubernetes. So even if a pod contains only one container, that container is still in the pod; the pod provides a shell around the container. The pod definition, or pod configuration (pod configuration file), declares the storage requirements for the pod.

[0027] Figure 1 illustrates some of these concepts as an example. In this case, cluster 101 consists of worker node 103 and master node 105. Worker node 103 contains three pods 107a-107c, which in turn contain containers 109a-109g, respectively.

[0028] The functionality of the Kubernetes system is provided through services located on worker nodes 103 and master nodes 105. Accordingly, worker nodes 103 contain pods called Kubernetes worker node (WN) systems 110. Similarly, master nodes 105 contain pods called Kubernetes master node (MN) systems 112. Details of these relevant to this disclosure are described below.

[0029] The master node 105, which serves applications running on the worker nodes 103, includes several components, including Kubernetes system services located in the Kubernetes master node system 112. These include an application interface server (API) 111 that applications in the worker nodes 103 use to communicate with the cluster. Additionally, the API server 111 serves commands entered into the command line tool kubectl.

[0030] The master node 105 also includes a Container Storage Interface (CSI) server 113, for example within a Kubernetes master node system 112. The role of the CSI server 113 is to manage third-party storage plugins used by containerized applications. These functions are described in more detail below.

[0031] The master node 105 also includes a scheduler 115, e.g., within a Kubernetes master node system 112. The scheduler 115 assigns pods to particular nodes within the cluster depending on the requirements of the various pods running on the cluster 101 and the capabilities of the nodes, e.g., in terms of storage available on a given node.

[0032] Master node 105 includes additional components, although these components are not necessary for this discussion.

[0033] Kubernetes is a state machine in the sense that it attempts to move pods towards a desired state. The pod lifecycle consists of several phases: pending, running, succeeded, and failed. Kubernetes attempts to move pods from pending to one of the termination phases: successful or failed. Of course, for many pods, running may be the phase that the pod is expected to remain in unless it is intentionally shut down or terminated due to some error condition. In general, Kubernetes will attempt to restart pods that have terminated due to an error, moving the cluster towards a desired state.

[0034] In the pending phase, the pod has been accepted into the Kubernetes cluster, but at least one container still requires some form of setup, which may include, for example, storage allocation.

[0035] In the run phase, a pod is assigned to a node, all of the pod's containers are created, and at least one container is running or started.

[0036] When all containers in a pod have terminated normally without errors, the pod is in the Success phase. Pods in the Success phase are not restarted. Conversely, when all containers in a pod have terminated and at least one pod has terminated abnormally, the pod is in the Failed phase. Depending on the pod definition, a pod may be automatically restarted when in the Failed phase.

[0037] When a pod is in one of the various phases, each container is associated with a state. These include waiting, running, and terminated. A container in the waiting state is performing the operations necessary to complete its startup. A running container is a container that is running. A container that has completed or failed is in the terminated state.

[0038] Storage in Kubernetes can be ephemeral or persistent: ephemeral storage expires when the pods using it are not running, whereas persistent storage does not expire when the pods using it terminate.

[0039] Figure 2 shows the typical flow of a Kubernetes cluster for creating and consuming storage, specifically persistent storage. Note that the examples considered here use persistent volumes, but the techniques described are equally applicable to ephemeral volumes.

[0040] The first step is for a cluster administrator to create a persistent volume (step 201), for example using the command line tool kubectl. Table 1 is an example YAML file (YAML is a recursive acronym for YAML Ain't a Markup Language) for creating a persistent volume (nfs-pv.yaml).

[0041] 1 API version: v1 Two types: Persistent Volumes 3 Metadata: 4 Name:nfs-francisco 5. Specifications: 6 Capacity: 7 Storage: 5Gi 8 Access Modes: 9 - Readable and writable from multiple nodes 10 Storage class name: nfs 11 Mounting Options: 12 - Hard 13 -nfsvers=3.0 14 nfs: 15 Path: / home / francisco 16 Server: 10.10.10.10 17 Persistent Volume Reclaim Policy: Keep Table 1: nfs-pv.yaml

[0042] The file nfs-pv.yaml creates a new persistent volume (line 2) named nfs-francisco (line 4), which is a Network File Storage (nfs) volume (defined by the storage class name (line 10)) with a storage capacity of 5 Gigabytes (line 7). The persistent volume creation specifies the location of the volume, in this case on a server named 10.10.10.10 (line 16), and the path on that server, / home / francisco (line 15).

[0043] To use a persistent volume, a namespace manager, for example, requests a persistent volume or a portion thereof (step 203). This may be done using the command line tool kubectl. Such a request is called a persistent volume claim (PVC). Table 2 is an example YAML file for creating a persistent volume claim (PVC).

[0044] 1 API version: v1 Two types: Persistent Volume Claims 3 Metadata: 4 Name:nfs-test-claim 5. Specifications: 6 Storage class name: nfs 7 Access Modes: 8 - Readable and writable from multiple nodes 9 Resources: 10 Request: 11 Storage: 1Gi Table 2: nfs-claim.yaml

[0045] The nfs-claim.yaml file requests 1 Gib (line 11) of persistent (line 2) nfs (line 6) storage. The persistent volume claim is named nfs-test-claim (line 4). Scheduler 115 may assign the persistent volume claim to any persistent volume that can satisfy the requested storage, such as the persistent volume created in step 201, i.e., nfs-francisco in the example of Table 1.

[0046] The persistent volume claim (PVC) is then added to the application pod configuration, i.e., the configuration file that defines the pod (step 205). The pod configuration file defines the directory structure of the containers within that pod. Table 3 is an example pod configuration YAML file that uses the PVC nfs-test-claim.

[0047] 1 API version: v1 2 types: Pods 3 Metadata: 4 Name:nfs-demo 5. Specifications: 6 Volume: 7 - Name:test-vol 8 Persistent Volume Claims: 9 Claim name:nfs-test-claim 10 containers: 11 - Name: ubuntu 12 Image:ubuntu 13 Volume Mount: 14 - Mount path: “ / data” 15 Name:test-vol 16 Commands: 17 - "Sleep" 18 - "604800" 19 Image Pull Policy:IfNotPresent 20 Restart policy: Always Table 3: demo-pod.yaml

[0048] This file configures a pod called nfs-demo (shown in Figure 3 as Nfs-demo pod 305), which includes the use of a persistent volume claim nfs-test-claim (line 9) called test-vol (line 15) in the pod's namespace. It also declares the use of a container called ubuntu (lines 10-19), which should mount the volume test-vol (lines 13-15) to a mount path called / data (line 14).

[0049] Thus, pods within a cluster may have their own namespace within the cluster, allowing functional groups of applications to be separated within the cluster.

[0050] The application pods are then deployed by the Kubernetes cluster (step 207). The details of deploying the application pods are outside the scope of this paper.

[0051] During the startup operation that transitions a pod from suspended to running, the resources required by the pod are configured. During that process, if Kubernetes detects that the pod has one or more volume claims defined, it mounts those volume claims (step 209). The mechanism for mounting the persistent volume corresponding to the volume claim is defined by the driver that supports that volume. For example, if the persistent volume is a host path, a directory in the host is mounted, but if the persistent volume is network storage, the persistent volume definition may include many other parameters such as a server address and an authentication token. In other words, how a persistent volume is mounted is an attribute specific to that persistent volume.

[0052] Once persistent volumes are mounted, these mounts are exposed to all containers in the pod, step 211. Because Kubernetes is a state machine that attempts to maintain a defined state for each pod, once all persistent volumes defined by a pod have been exposed, no more persistent volume claims can be added to the pod.

[0053] Finally, with all persistent volumes defined by the pod configuration mounted, the pod moves to the execution phase (step 213).

[0054] FIG. 3 is a high-level block diagram illustrating a cluster 301 that may result from executing the example configuration files of Tables 1, 2, and 3.

[0055] Cluster 301 is created by a cluster administrator, for example using the command line tool kubectl, and configured through the configuration files in Tables 1, 2, and 3.

[0056] The cluster 301 includes a worker node 303 on which a pod nfs-demo 305 is instantiated. The pod nfs-demo 305 includes a container ubuntu 307 that includes a mount 309 ( / data). Of course, the cluster 301 may include more nodes and more pods, and the pod 305 may include additional containers.

[0057] Mount 309 points to a Kubernetes internal path representation 311, which links mount 309 to persistent volume claim 313 nfs-test-claim, and thus to persistent volume 315 nfs-fransisco, and ultimately to network file storage 317.

[0058] In the embodiments shown in Figures 4-6, the above mechanisms are extended to enable additional functionality beyond that provided by the file system that is required by deployed and exposed pods. For example, the added functionality may be file system layering that provides functionality such as encryption services or access control services. Consider a scenario in which an application intended to consume data stored in a file system requires file system encryption or access control mechanisms. To do so, the embodiments described herein below add a file system layer that provides such encryption services or access control mechanisms.

[0059] 4 is a high-level architectural diagram illustrating an embodiment of file system functionality extensions, such as for supporting file system layering in Kubernetes, specifically for providing a file access control mechanism, including cryptographic services, to containerized applications deployed in a Kubernetes cluster. While file system layering is illustrated in and described below in connection with FIG. 4, this technique may be used to provide other file system extensions, such as file system overlays, to container orchestration systems, where multiple files may be accessed as one.

[0060] The usual Kubernetes services, such as API server 111, CSI server 113, and scheduler 115 (collectively Kubernetes master node (MN) system 411), located on master node 405, which corresponds to master node 105 in Figure 1, remain unchanged.

[0061] Master node 405 further includes an enhanced CSI driver 417. As discussed below, enhanced CSI driver 417 provides persistent volumes on which a layered file system is provided to provide, for example, cryptographic and access control services to stored data.

[0062] Worker nodes 403 of cluster 401, which correspond to worker node 103 in Figure 1, include application pods 407, each having one or more containers 409. Although only one application pod 407 is shown, it should be understood that there is no theoretical limit to the number of pods that a worker node 403 may contain.

[0063] Worker node 403 also includes an extended CSI agent 423 that corresponds to extended CSI driver 417. Together, these constitute an extended CSI system. The role of extended CSI (EXT CSI) agent 423 is to recognize application pods that require services from extended CSI driver 417. Furthermore, extended CSI driver 417 may provide cryptographic services 419, such as file encryption, decryption, key management, and digital signatures. Furthermore, extended CSI driver 417 may (or alternatively) provide access control policy services 421.

[0064] Worker node 403 further includes the worker node portion of the Kubernetes system within Kubernetes worker node (WN) system pod 410.

[0065] 5 is a timing sequence diagram illustrating process steps for providing extended CSI (EXT CSI) services to application pods 407. Many of the steps correspond directly to steps in FIG. 2. These steps are labeled with reference numbers preceded by single quotes. That is, step 201' in FIG. 5 corresponds to step 201 in FIG. 2.

[0066] The first two steps are similar to those in Figure 2. Note that the result of step 203', like step 203, is a persistent volume claim for the given name. This persistent volume claim is referenced in later steps to link the file system layer to the persistent volume claim. In response to the creation of the PV (step 201') and the creation of the PVC (step 203'), the Kubernetes master node system 411 registers the PV (step 501) and PVC (step 503), respectively.

[0067] The following describes a mechanism for adding a file system layer on top of the persistent volume request created in step 201'. That persistent volume request is referred to as the original persistent volume request. In one embodiment, the file system layer added to the original persistent volume request adds encryption services to protect the original persistent volume. In that sense, the original persistent volume may be considered an unprotected volume until the file system layer protects it and a protected volume is created.

[0068] Typically, step 201' is performed by a cluster administrator 601 using the command line tool kubectl. Alternatively, the persistent volume is created automatically. Similarly, the creation of the PVC in step 203' may be performed by a namespace administrator 603 responsible for a particular application pod 407, or in an automated step.

[0069] As a preliminary matter, the storage class of the persistent volume corresponding to the extended CSI is defined by the namespace administrator 603 (step 505). Table 4 shows example YAML code for declaring the storage class of the extended CSI.

[0070] 1 API version: storage.k8s.io / v1 Two types: storage classes 3 Metadata: 4 Name: csi-test-sc 5 Provisioner: ext.csi 6 Reclaim Policy: Retain 7 Volume Binding Mode: Immediate 8 Allow Volume Expansion: True 9 Parameters: 10 cmaddr:192.168.70.1 11 test_parameters: This is a test Table 4: extended-storage.yaml

[0071] The extended-storage.yaml file provides attributes for the extended persistent volume request, including the name of the storage class, csi-test-sc (line 4), in the example of Table 4. Extended CSI driver 417 and extended CSI agent 423 may support multiple types of storage classes, each with different characteristics.

[0072] The extended-storage.yaml file also specifies that the provisioner for the storage class is ext.csi (line 5), which is a link to extended CSI driver 417 and extended CSI agent 423. In other words, data accesses for persistent storage requests that have provisioners corresponding to extended CSI driver 417 and extended CSI agent 423 are directed to extended CSI driver 417 and extended CSI agent 423, collectively the extended CSI system.

[0073] The storage class file also defines some attributes of the parameters associated with the storage class (lines 9-11). These parameters provided in the storage class are used by volumes that use the storage class to provide their volume configuration parameters. These configuration parameters could be any number of things, such as key manager addresses, security policy definitions, encryption keys, authentication tokens, etc.

[0074] When the KubernetesMN system 411 receives a declaration of a storage class for which the provisioner is a provider of extended CSI (EXT CSI), the KubernetesMN system 411 registers the EXT CSI storage class (step 507).

[0075] Next, the namespace administrator 603 creates an extended persistent volume claim (E-PVC) for the previously declared persistent volume claim (step 509). This request may be created by running the YAML file with the command line tool kubectl. Table 5 is an example YAML file for creating an extended persistent volume claim that places file tiering provided by the ext csi system on the file system of the persistent volume claim declared in step 203' for the persistent volume defined in step 201'.

[0076] 1 API version: v1 Two types: Persistent Volume Claims 3 Metadata: 4 Name:extCSI-claim1 5 Comments: 6 extPolicy:Policy_1 7. Specifications: 8 Storage class name: csi-test-sc 9 Access Mode: 10 - Readable and writable by multiple nodes 11 Resources: 12 Request: 13 Storage: 1Gi 14 Data Sources: 15 types: Persistent Volume Claims 16 Name:nfs-test-claim Table 5: ext-csi-claim.yaml

[0077] For example, this yaml file, executed by namespace admin 603 of application pod 407 via the command line tool kubectl, uses the services of the EXT CSI system to create a persistent volume claim using storage class csi-test-sc (line 8), which was configured as the ext-csi storage class in step 505, and is thus associated with the EXT CSI system through the convention given in the storage definition of the csi-test-sc storage class. The storage class name specification has a name corresponding to the name of the storage class (defined in the storage class declaration (step 501)) that the provisioner will place the persistent volume claim within the EXT CSI system.

[0078] The created persistent volume claim is given a name (line 4), in this example extCSI-claim1, and uses an annotation (extPolicy) to define a security policy, in this example policy_1, to be associated with the created persistent volume claim (line 6). Security policies may include encryption, digital signatures, access control, authentication, etc. Note that the use of annotations to specify security policies is provided as an example of how security policies can be added to a layered file system in one embodiment.

[0079] The YAML file that creates an EXT CSI persistent volume claim also defines a data source that corresponds to the EXT CSI persistent volume claim. In a sense, it creates an EXT CSI persistent volume claim for an existing persistent volume claim. In this example, this is defined through the data source field, and in this example, the data source is the persistent volume claim named nfs-test-claim (line 16). Note: This is the name of the persistent volume claim in Table 2 (step 203'). Therefore, the new persistent volume claim, which is an EXT CSI persistent volume claim, uses the previously created persistent volume claim as its data source.

[0080] In response to receiving a persistent volume request with a storage class that is one of the storage classes defined to belong to the EXT CSI system (i.e., the provisioner for the storage class is EXT CSI (row 5 of Table 4)), the KubernetesMN system 411 notifies the EXT CSI driver 417 that an EXT CSI persistent volume request has been created and its definition (step 511). Every persistent volume request requires a persistent volume. Therefore, the EXT CSI driver 417 provides a new virtual persistent volume that corresponds to the received EXT CSI persistent volume request based on the configuration file that defines the storage class of the received persistent volume request (in this example, csi-test-sc) and the underlying persistent volume to which the EXT CSI persistent volume request ultimately references, i.e., the persistent volume defined in step 201', named nfs-francisco in this example (step 513).

[0081] To use the extended persistent volume claim, the namespace administrator 603 of the application pod 407 that is to use the extended services of the EXT CSI system adds the EXT CSI persistent volume claim to the configuration file of the application pod 407 (step 515).

[0082] Table 6 is a YAML file that shows how a namespace administrator 603 can add an EXT CSI persistent volume claim to a pod configuration.

[0083] 1 API version: v1 2 types: Pods 3 Metadata: 4 Name:ext-csi-demo 5. Specifications: 6 Volume: 7 - Name:test-vol 10 Persistent Volume Claims: 11 Claim name: ext-claim1 12 containers: 13 - Name: ubuntu 14 Image:ubuntu 15 Volume Mount: 16 - Mount path: “ / data” 17 Name:test-vol 18 Commands: 19 - "Sleep" 20 - "604800" 21 Image Pull Policy:IfNotPresent 22 Restart policy: Always Table 6: ext-csi-demo-pod.yaml

[0084] For example, the namespace administrator 603 creates this YAML file (ext-csi-demo-pod.yaml) to configure an application pod 407 that will use the services of the EXT CSI system (step 515). In the pod configuration file, e.g., ext-csi-demo-pod.yaml, the pod is given a name such as ext-csi-demo.

[0085] The configuration file defines a volume to use as a persistent volume claim with claim name ext-claim1 (row 11 in Table 6). Because the ext-claim1 persistent volume claim was defined to be an extended CSI persistent volume claim in step 509, the EXT CSI system is invoked when the pod is deployed. Ultimately, this not only enforces the use of the persistent volume defined in step 501, but also the security policy defined for the extended CSI persistent volume claim (policy_1 (row 6 in Table 5)).

[0086] Thus, the claim name definition (line 11) ties together pod 407, the EXT CSI storage claim, the original persistent volume claim, and the original persistent volume.

[0087] After adding the extended persistent volume claim, the namespace administrator 603 requests deployment of the application pod 407 via the command line tool kubectl (step 517), and the KubernetesMN system 411 schedules pod deployment by the Kubernetes worker node 403 by sending a message to the Kubernetes worker node system 410 (step 518). The KubernetesWN system 410 then initiates the actions necessary to deploy the application (step 519).

[0088] The pod configuration file defines the volumes that need to be mounted (lines 15-17). Therefore, the Kubernetes worker node system 410 determines the originator of the storage class and requests the driver associated with that storage class to mount any volumes in the pod configuration (step 521). The driver for the storage class is not shown in Figure 5.

[0089] The mechanism for mounting a volume is within the purview of the volume's originator. Thus, if the mount is for an EXT CSI volume (as in the example in Table 6), KubernetesWN system 410 invokes EXT CSI agent 423 to mount the volume. Thus, EXT CSI agent 423 receives a mount request for one of the volumes for which the EXT CSI system provides file services, such as encryption or access control.

[0090] Contrast the YAML file demo-pod.yaml in Table 3 with the YAML file ext-csi-demo-pod.yaml in Table 5. In the former, the pod accesses the nfs file storage directly. In the latter, the nfs file storage is accessed via the EXT CSI system, so the pod utilizes the security policies that the EXT CSI system associates with the volume through the policy declaration in the EXT persistent volume request.

[0091] In response to receiving a mount request for a volume that belongs to one of the storage classes associated with the EXT CSI system, the EXT CSI agent 423 creates a staging pod: a pod definition for a new pod, staging pod 613 (shown with a dashed line because staging pod 613 does not exist before step 523), that corresponds to the extended CSI persistent volume request, by sending a Kubernetes system-internal configuration file to the Kubernetes master node system 411 (step 523), which then notifies the Kubernetes worker node system 410 of the request to create the staging pod 613. The Kubernetes worker node system 410 deploys the staging pod 613 according to the configuration provided by the EXT CSI agent 423.

[0092] The EXT CSI agent 423 waits for the staging pod 613 to enter the execution phase (step 524).

[0093] On the other hand, the configuration file for staging pod 613, which is a system internal configuration file, has a similar structure to the pod configuration file and may be exported to a YAML file for inspection. Table 7 is an example export file provided to show the contents of the configuration file for staging pod 613.

[0094] 1 Type: Pod 2 API version: v1 3 Metadata: 4 Name:ext-staging-podh6bj9 5 Name generation:ext-staging-pod 6 Namespace: Default 7 uid:8ab1ef8a-d85f-4f03-b547 -48add37b046c 8 Labels: 9 ext-csi:ext-staging-pod 10 ext-volume-1:b1807209-c932-11eb -b4a1-72f29583c971 11 self-link: / api / v1 / namespaces / default / pods / ext-staging-podh6bj9 12 Specs: 13 Volume: 14 - Name:ext-staging-volume-1 15 Persistent Volume Claims: 16 Claim name:nfs-test-claim 17 - Name: default-token-dxh2d 18 Secrets: 19 Secret Name: default-token-dxh2d 20 Default mode: 420 21 containers: 22 - Name: Busybox 23 Image: Busybox 24 Commands: 25 - Tail 26 -'-f' 27 - / dev / null 28 Resources: {} 29 Volume Mount: 30 - Name:ext-staging-volume-1 31 - Mount path: / ext-staging-volume-1 32 - Name: default-token-dxh2d 33 Read Only: True 34 Mount path: / var / run / secrets / kubernetes.io / serviceaccount 35 Image Pull Policy:IfNotPresent 36 Restart Policy: Always 37 End grace period (seconds): 30 38 Node name:ub20-work1 Table 7: Staging Pods

[0095] Figure 6 is a block diagram corresponding to Figure 3, showing the result of the flow in Figure 5. A deployment request for the ext-csi-demo pod results in the instantiation of pod 407. Pod 407 has a container 707 with a mount 709 ( / data). The Kubernetes system recognized the mount as being for extended CSI persistent storage, so it sent a message to the EXT CSI agent 423, alerting the latter to mount a volume with an associated storage class. The EXT CSI agent 423 then created a new pod, Staging pod 613.

[0096] The persistent volume claim in the staging pod points to the original persistent volume claim (in this example, nfs-test-claim) that the extended persistent volume claim is linked to (line 16).

[0097] Like all pods, staging pod 613 includes a container, here staging container 713, with a mount path ( / ext-staging-volume-1) that points to the original persistent volume claim from step 203'.

[0098] In an embodiment such as that shown in FIG. 6, although the cluster may have many nodes, the staging pod 613 is assigned by the EXT CSI agent 423 to the same worker node 303 as the application pod 407.

[0099] The staging pod contains a volume definition for the original persistent volume claim. In this example, lines 14-16 define the volume name ext-staging-volume-1 as the persistent volume claim nfs-test-claim, i.e., the volume on the original persistent volume claim that is required to be protected by the EXT CSI system.

[0100] The staging pod also contains a container (in this example, called busybox (line 22)) that has a volume mount on volume ext-staging-volume-1, so the staging pod requests that volume's driver, the nfs driver in this example, to mount the original volume (step 525).

[0101] Once the persistent volumes required by the staging pod 613 are mounted, the staging pod 613 enters the execution phase (step 535) and notifies the Kubernetes systems 411 and 410 that it has entered the execution phase.

[0102] When the staging pod 613 is executed, the Kubernetes system (411 and 410) notifies the Ext CSI agent 423 that the staging pod is running (step 526).

[0103] EXT CSI agent 423 searches for nodes for a mount of the original persistent volume request (step 527). Once staging pod 613 is created, EXT CSI agent 423 has enough metadata in the cluster to construct a path that indicates where the original data is located on the host.

[0104] In that effort, the EXT CSI agent 423 has some known static information at its disposal. Kubernetes node pod staging directory ( / var / lib / kubelet / pods / ) -Memory obtained from the cluster Metadata fields: Pod node (e.g. ub20-work1) Pod UID (e.g. 8ab1ef8a-d85f-4f03-b547-48add37b046c) Persistent volume name (e.g. nfs-francisco)

[0105] First, the EXT CSI agent 423 performs an initial search path, i.e. (Kubernetes node pod staging) + " / " + (pod UID) " / volumes" ·Example: / var / lib / kubelet / pods / 8ab1ef8a-d85f-4f03-b547-48add37b046c / volumes Generate.

[0106] On the node where the staging pod 613 is created, the EXT CSI agent 423 searches for the persistent volume name (nfs-francisco) in the search path. In this example, the search result is / var / lib / kubelet / pods / 8ab1ef8a-d85f-4f03-b547-48add37b046c / volumes / kubernetes.io~nfs / nfs-francisco This becomes:

[0107] This path contains the mounted volume that corresponds to the original persistent volume request and to which the EXT CSI system's file system tiering will be applied. The lookup performed by the EXT CSI agent 423 results in the volume path of the original persistent volume request, which the EXT CSI agent 423 stores as the data source on which the EXT CSI system's services will be layered.

[0108] Using the path to the original data, the EXT CSI agent 423 mounts the layered file system using the original PVC mount as the data source (step 529). The layer adds some additional functionality to the file system, such as encryption services.

[0109] The mounts associated with the extended persistent volume requests determined in step 527 and mounted in step 529 are exposed by the EXT CSI agent 423 to the containers of the application pod (step 531).

[0110] At this point in the flow of FIG. 5, both the staging pod 613 and the application pod 407 are running (step 533).

[0111] From the above description, it becomes apparent that an efficient and secure mechanism is provided for providing transparent extension of file system capabilities, e.g., file system layering, in conjunction with a container orchestration system, e.g., Kubernetes.

[0112] References to Kubernetes are by way of example only; the techniques described herein are applicable mutatis mutandis to other container orchestration systems.

[0113] Although specific embodiments of the present invention have been described and illustrated, the present invention is not limited to the specific forms or arrangements of parts so described and illustrated. The present invention is limited only by the claims.

Claims

1. 1. A method for extending the capabilities of a file system in a container orchestration system without modifying a runtime environment of the container orchestration system, wherein the file system extension provides functionality not otherwise provided by the file system of the container orchestration system, the method comprising: - associating an extended storage class with an extension that handles access to data storage volumes belonging to said extended storage class by an extended file system; Associating the original volume with a storage class other than the extended storage class; in response to deployment of an application pod configured to include a definition of an extended volume belonging to the extended storage class; - requesting that the extended volume be mounted; creating a staging pod for mounting the original data storage volume by the extended file system, and deploying the staging pod in the container orchestration system; mounting the original data storage volume by the staging pod; determining a path identifying a mount of the original data storage volume; using the path identifying the mount of the original data storage volume and mounting the extended volume belonging to the extended storage class using the original data storage volume as a data source; Exposing a mount of the extended volume to a container in the application pod; and Putting the application pod into an execution phase and handling access to data stored on the extended volume by the containers in the application pod with the extended file system. A method comprising:

2. The method of extending file system capabilities in a container orchestration system of claim 1 , wherein the capabilities being extended are file system layering.

3. 3. The method of extending the capabilities of a file system in a container orchestration system of claim 1 or 2, wherein the extended capabilities are overlaid on a first volume onto a second volume to provide a combined volume including the first volume and the second volume.

4. The method for extending file system capabilities in a container orchestration system according to claim 1 or 2, wherein the container orchestration system is Kubernetes.

5. 2. The method of extending file system capabilities in a container orchestration system of claim 1, comprising creating the original volume as a persistent volume and a corresponding persistent volume request.

6. The method of claim 4 for extending file system capabilities in a container orchestration system, comprising creating the extended volume as a persistent volume and a corresponding persistent volume request.

7. 5. The method of extending file system capabilities in a container orchestration system of claim 4, comprising creating the original volume as a temporary volume and a corresponding temporary volume request.

8. 5. The method of expanding file system capabilities in a container orchestration system of claim 4, comprising creating the expanded volume as a temporary volume and a corresponding temporary volume request.

9. - accessing the extended volume by an application container; Organizing access to the extended volume by the application container using a driver associated with the extended storage class. The method for extending file system capabilities in the container orchestration system of claim 1 or 2, further comprising:

10. The method for extending file system capabilities in a container orchestration system of claim 1 or 2, wherein a driver associated with the extended storage class provides encryption services for data stored in the extended volume.

11. 10. The method of extending file system capabilities in a container orchestration system of claim 9, wherein the driver associated with the extended storage class enforces an access control policy on data stored in the extended volume.

12. A container orchestration system having the ability to extend file system capabilities by providing extensions to an original data storage volume provided by the container orchestration system without modifying a runtime environment of the container orchestration system, the file system extension providing functionality not otherwise provided by the file system of the container orchestration system; - at least one node having a processor and a memory, said memory being connected to said processor; accepting an association between an extended storage class and a file system extension, where access to data storage volumes belonging to the extended storage class is handled by a layered file system driver of the file system extension; Accepting the association of the original volume with a storage class other than the extended storage class at least one node storing instructions to cause the the memory further comprises an agent associated with the extended storage class, the agent being configured to communicate with the processor; - receiving a request to mount an extended volume; creating a staging pod for mounting the original data storage volume and having the container orchestration system deploy the staging pod; determining a path identifying a mount of the original data storage volume; using the path identifying the mount of the original data storage volume and mounting the extended volume belonging to the extended storage class using the original data storage volume as a data source; Exposing the mount of the extended volume to the containers of the application pod and storing an agent that performs the - the memory further includes instructions for the processor to execute the staging pod; Mounting the original data storage volume. and storing an instruction to perform the The memory further stores instructions to the processor to cause the application pod to enter an execution phase in response to the application pod receiving a publication notification of the mounting of the extended volume, and to process access to data stored in the extended volume by the container in the application pod using a driver of the extended file system. Container orchestration system.

13. 13. A container orchestration system capable of extending file system capabilities by providing extensions to original data storage volumes provided by the container orchestration system of claim 12, wherein the extensions are file system layering.

14. 14. A container orchestration system having the ability to extend file system capabilities by providing extensions to original data storage volumes provided by the container orchestration system of claim 12 or 13, wherein the extension is overlaying a first volume onto a second volume to provide a combined volume including the first volume and the second volume.

15. 14. A container orchestration system having the ability to extend file system capabilities by providing functional extensions to the original data storage volume provided by the container orchestration system of claim 12 or 13, wherein the container orchestration system is Kubernetes.

16. 13. A container orchestration system having the ability to extend file system capabilities by providing extensions to original data storage volumes provided by the container orchestration system of claim 12, wherein the original volumes are persistent volumes and corresponding persistent volume requests.

17. 16. A container orchestration system capable of extending file system capabilities by providing extensions to original data storage volumes provided by the container orchestration system of claim 15, wherein the extended volume is a persistent volume and corresponding persistent volume request.

18. 16. A container orchestration system having the ability to extend file system capabilities by providing extensions to original data storage volumes provided by the container orchestration system of claim 15, wherein the original volumes are temporary volumes and corresponding temporary volume requests.

19. 16. A container orchestration system capable of extending file system capabilities by providing functional extensions to original data storage volumes provided by the container orchestration system of claim 15, wherein the extended volume is a temporary volume and corresponding temporary volume request.

20. A container orchestration system capable of extending file system capabilities by providing functional extensions to an original data storage volume provided by the container orchestration system of claim 12 or 13, configured to organize access by application containers to the extended volume through a driver associated with the extended storage class.

21. 14. A container orchestration system having the ability to extend file system capabilities by providing functional extensions to the original data storage volume provided by the container orchestration system of claim 12 or 13, wherein the driver associated with the extended storage class provides encryption services for data stored in the extended volume.

22. 21. A container orchestration system having the ability to extend file system capabilities by providing extensions to original data storage volumes provided by the container orchestration system of claim 20, wherein the driver associated with the extended storage class provides encryption services for data stored in the extended volume.

Images