Cloud native container storage interface plug-in and deployment method across heterogeneous storage backends

By designing a cloud-native container storage interface plug-in across heterogeneous storage backends, a single CSI plug-in is implemented to connect multiple storage protocols, solving the problems of complex deployment, high operation and maintenance costs and low computing resource utilization, and simplifying the Storage management of Kubernetes clusters.

CN118672687BActive Publication Date: 2024-11-0510TH RES INST OF CETC
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411175342.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-05
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

The existing CSI plug-in design goal is to connect to specific storage backends, resulting in the need to deploy multiple sets of dedicated plug-ins in Kubernetes clusters to adapt to heterogeneous storage resources, resulting in complex management, high operation and maintenance costs and low computing resource utilization.

Method used

Design a cloud-native container storage interface plug-in across heterogeneous storage backend, including general components, task distribution components, abstract layer components and storage management components. Through general components, public logic independent of storage protocols is realized, and task distribution components identify storage protocols. Type and call the corresponding storage management component. The abstract layer component defines the data volume's full life cycle management method. The storage management component connects to the storage backend and implements the abstract layer method to realize a single CSI plug-in to connect to multiple storage protocols.

Benefits of technology

It reduces the cost and complexity of Kubernetes cluster management storage resources, improves code reusability and maintainability, simplifies the operation and maintenance costs related to storage management, and improves the utilization rate of cluster computing resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118672687B_ABST
    Figure CN118672687B_ABST
Patent Text Reader

Abstract

The present invention discloses a cloud native container storage interface plug-in and deployment method across heterogeneous storage backends, belonging to the field of cloud native container technology, including: a general component for implementing common logic unrelated to storage protocols; a task distribution component for completing the identification of storage protocol types from CSI request contexts and calling corresponding storage management components; an abstract layer component for defining abstract methods for managing the entire life cycle of data volumes; a storage management component for docking storage backends and implementing abstract layer methods. The present invention achieves the goal of a single CSI plug-in being able to dock with multiple storage protocols, overcomes the current limitation that there must be a one-to-one correspondence between CSI plug-ins and storage backends, and solves the problems of complex deployment process of current dedicated CSI plug-ins, high operation and maintenance costs, and low utilization of cluster computing resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of cloud native container technology, and more specifically, to a cloud native container storage interface plug-in and a deployment method across heterogeneous storage backends. Background Art

[0002] In the era of computing virtualization, container engines represented by Docker use the rootfs mechanism and Linux MountNamespace mechanism to isolate the container from the host file system. When the container wants to call storage resources, it can use the bind mount mechanism to form a mapping relationship between the data volume declared by the container and the specified directory of the host file system, thereby accessing the host's storage resources. However, with the development of container technology, storage resources are no longer just a mapping relationship between data volumes and local directories of the host, but are also regarded as an important resource that can be managed by the container engine, such as the Volume Mount concept proposed by Docker in later iterations.

[0003] Entering the cloud-native era, Kubernetes, as the most popular container orchestration system, has deeply abstracted storage resources in terms of persistent storage capabilities, hiding the specific implementation details of data storage from users. Users only need to describe data storage requirements in the form of YAML files, and Kubernetes will complete the lifecycle management of specific data volumes according to the user's intentions, such as allocation, attachment, mounting, and recycling. Furthermore, in order to avoid the complexity of describing data storage requirements and lower the threshold for using cluster storage resources, Kubernetes abstracts API objects such as PV, PVC, and StorageClass to achieve dynamic provisioning of storage data volumes and separation of responsibilities between cluster users and operation and maintenance personnel.

[0004] In order to more conveniently integrate the remote storage services provided by various cloud storage vendors and realize persistent storage capabilities under different storage media, Kubernetes defines the Container Storage Interface (CSI) specification based on the idea of ​​fully decoupling storage management logic from core scheduling logic. Third-party cloud storage vendors only need to deploy plug-ins that implement the CSI specification together with other sidecar containers officially implemented by Kubernetes to integrate their own storage services into Kubernetes and provide the corresponding storage resources in the form of working data volumes to the application Pods deployed in the Kubernetes cluster.

[0005] However, the current mainstream CSI plug-ins are designed to connect to specific storage backends or storage protocols, resulting in dedicated CSI plug-ins for various storage backends and a lack of unified abstraction at a higher level. For example, the csi-driver-nfs project only connects to the shared file system provided by the NFS server; the csi-driver-nvmf project only connects to the remote NVMe disk provided by the NVMe-oF target. However, in the actual deployment environment of the Kubernetes cluster, the storage resources that can be provided by each node host (including the underlying storage hardware, the host local file system, the distributed storage protocol, etc.) are often heterogeneous, which will result in the need to deploy multiple sets of dedicated CSI plug-ins at the same time in order to adapt to different storage backends when deploying the Kubernetes cluster. This working mode has the following three main disadvantages:

[0006] (1) Multiple sets of dedicated CSI plug-ins are used to support heterogeneous storage resources within the cluster. It is necessary to manage and maintain their corresponding PV, PVC, StorageClass and other objects related to storage management, as well as ServiceAccount, RBAC and other objects related to permission control. This not only leads to a high threshold for users to use cluster storage resources, but also increases the cost of cluster deployment and operation and maintenance.

[0007] (2) The official Kubernetes sidecar container contains some common logic for interacting with the control node, so the dedicated CSI plug-in for each storage backend needs to be deployed in the same Pod as the sidecar container to work properly. However, because there is no difference between the sidecar containers used by different CSI plug-ins, deploying multiple sets of dedicated CSI plug-ins in the cluster means repeatedly deploying multiple sets of the same sidecar container instances, which will reduce the utilization of computing resources in the cluster.

[0008] (3) Although there are significant differences in the specific implementation details of calling the storage service end for dedicated CSI plug-ins for different storage protocols, there is still a large part of public logic in these CSI plug-ins that is unrelated to the specific storage backend, such as the collection of statistical information such as data volume usage, storage service end heartbeat detection, data volume mounting, data volume unmounting and other functions. These public logic codes also need to be repeatedly implemented in the source code of multiple sets of dedicated CSI plug-ins, resulting in reduced code reusability and maintainability. Summary of the invention

[0009] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a cloud-native container storage interface plug-in and deployment method across heterogeneous storage backends, so as to achieve the goal that a single CSI plug-in can connect to multiple storage protocols, overcome the limitation that there must be a one-to-one correspondence between the current CSI plug-in and the storage backend, and solve the problems of complex deployment process of current dedicated CSI plug-ins, high operation and maintenance costs, and low utilization of cluster computing resources.

[0010] The object of the present invention is achieved through the following solutions:

[0011] A cloud-native container storage interface plug-in across heterogeneous storage backends, including:

[0012] Common components, used to implement common logic not related to storage protocols;

[0013] The task distribution component is used to identify the storage protocol type from the CSI request context and call the corresponding storage management component;

[0014] Abstract layer components, used to define abstract methods for managing the entire life cycle of data volumes;

[0015] The storage management component is used to connect to the storage backend and implement abstract layer methods.

[0016] Furthermore, the general component is specifically used to: implement the csi.ControllerServer interface and the csi.NodeServer interface, where csi.ControllerServer represents the controller service and csi.NodeServer represents the node service.

[0017] Furthermore, the common logic unrelated to the storage protocol includes parameter analysis and verification, and data volume indicator collection.

[0018] Furthermore, the identifying of the storage protocol type from the CSI request context and calling the corresponding storage management component specifically includes: when the storage protocol type in the CSI request is unclear, using the general component to cooperate to parse out the storage protocol type and pass it to the task distribution component; then calling the corresponding storage management component according to the storage protocol type obtained by the analysis; when the storage protocol type in the CSI request is clear, the CSI plug-in directly passes the request to the task distribution component.

[0019] Furthermore, the abstract method for defining the full lifecycle management of a data volume specifically includes: associating lower-layer storage resources with upper-layer task distribution components to shield detailed implementation details of data volume management under a specific storage protocol.

[0020] Furthermore, the docking storage backend and implementing the abstract layer method specifically include: one-to-one correspondence between the storage management component and the heterogeneous storage backend, implementing the details of calling the specific storage service end, and cooperating with the abstract layer component to complete the unified management of the heterogeneous storage backend.

[0021] Furthermore, the storage management component is registered with the task distribution component through a hook function during the CSI plug-in startup process. When the task distribution component receives the CSI request, it obtains the specified storage protocol type by parsing the CSI request, or reversely parses the storage protocol type through the data volume id generation rule, and then distributes the task to the corresponding storage management component according to the storage protocol type.

[0022] A method for deploying a cloud native container storage interface plug-in across heterogeneous storage backends includes the following steps:

[0023] Step 1: Deploy the container of the cloud native container storage interface plug-in across heterogeneous storage backends as described in any of the above items and the external-provisioner, external-attacher, and livenessprobe containers provided by Kubernetes as controller services to any working node in the cluster in the form of Deployment, where external-provisioner represents an external storage provider, external-attacher represents an external storage attacher, livenessprobe represents a service probe, and Deployment represents a deployment set;

[0024] Step 2: Deploy the plug-in container and the node-driver-registrar and livenessprobe containers provided by Kubernetes as node services to all working nodes in the cluster in the form of DaemonSet, where DaemonSet represents a daemon service set.

[0025] Step 3: Create a StorageClass object to save the available third-party cloud storage service client information, which includes storage protocol type, IP, port, and data volume related details. StorageClass represents the storage class.

[0026] Step 4: Create a PVC object associated with the StorageClass object. StorageClass represents the storage class.

[0027] Step 5: Declare and deploy containers related to actual business. During the container declaration process, mount the PVC object as a working data volume to a specified path in the application container. PVC represents a persistent data volume declaration.

[0028] A method for deploying a cloud native container storage interface plug-in across heterogeneous storage backends includes the following steps:

[0029] Step 1: During the startup process of the cloud-native container storage interface plug-in across heterogeneous storage backends as described in any of the above items, each storage management component registers the storage protocol it implements to the task distribution component through a hook function;

[0030] Step 2: When the plug-in receives a CSI-related request from the application container, it determines whether the request context carries a clear storage protocol type parameter;

[0031] Step 3: If the CSI request only contains the data volume ID as a call parameter and does not contain a clear storage protocol type parameter, such as the csi.NodeUnpublishVolume method, the general component reversely parses the storage protocol type based on the data volume ID and passes it to the task distribution component; if the CSI request already contains a clear storage protocol type parameter, such as the csi.CreateVolume method, the CSI plug-in directly passes the request to the task distribution component;

[0032] Step 4: The task distribution component sends the task to the corresponding storage management component based on the obtained storage protocol type. The storage management component communicates with the third-party cloud storage service client to complete the management of the storage resources involved in the request, and returns the storage protocol type as a component of the data volume id to the caller. The management of storage resources includes the generation of data volume id, and the allocation and recovery of storage service client resources corresponding to the data volume.

[0033] Furthermore, in the fourth step, in the data volume mounting scenario, the storage management component also performs the following sub-steps: saving the detailed mapping relationship between the data volume and the third-party cloud storage service client in the form of a file in the parent directory of the data volume mounting directory, and when a request to unmount the data volume is subsequently received, the mapping relationship will be obtained from the file.

[0034] Furthermore, in the fourth step, in the CSI request response structure that partially supports setting context information, the storage management component also performs the following sub-steps: returning the cloud storage service end information carried by the corresponding request to the requester as context information for the requester to maintain the data volume context, and carrying this context information in subsequent calls. The cloud storage service end information includes IP, port, and server storage resource mapping information.

[0035] The beneficial effects of the present invention include:

[0036] (1) The present invention abstracts the functions of the CSI plug-in in Kubernetes in a higher dimension, and implements the interaction between the Kubernetes cluster and multiple heterogeneous third-party cloud storage services through a CSI plug-in, so that the Kubernetes cluster can more conveniently call multiple differentiated storage backends, and transforms the existing one-to-one relationship between the dedicated CSI plug-in and the third-party cloud storage protocol into a one-to-many relationship, thereby reducing the cost and complexity of Kubernetes cluster management of related storage resources.

[0037] (2) The present invention adopts a strategy pattern to implement the specific details of the storage protocol as a specific storage strategy and encapsulate it into the storage management component of the CSI plug-in. The task distribution component calls the corresponding storage management component according to the storage protocol type. The code complies with the open-closed principle. When the adopted storage protocol is not included in the current storage management component, the user only needs to implement the storage management component corresponding to the new protocol to support the new storage protocol. The risks introduced by the modification will not affect the existing storage management components. The regression verification cost brought by iterative development is low, and the software maintainability is high.

[0038] (3) The present invention improves the abstract granularity of the CSI plug-in data volume mounting scenario from the storage protocol level to the block storage / file storage level, thereby reducing the repeated implementation of boilerplate code in the existing CSI plug-in working mode and improving code reusability.

[0039] (4) The present invention provides a unified entry point for the full lifecycle management capabilities of container data volumes of a Kubernetes cluster, thereby simplifying the tasks of Kubernetes cluster storage-related indicator monitoring, log collection, status management, etc., and significantly reducing the operation and maintenance costs related to Kubernetes cluster storage management. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1The module schematic and deployment environment diagram of a cloud native container storage interface plug-in across heterogeneous storage backends of the present invention; the external sidecar container refers to a group of containers officially provided by Kubernetes that need to be deployed together with the CSI plug-in of the present invention, including: external storage attacher external-attacher, service liveness probe; external storage provider external-provisioner; node driver registrar node-driver-registar; in addition to working nodes, Kubernetes clusters also need to deploy control nodes separately, and the control nodes need to provide three key services, including: api service kube-apiserve, control management service kube-controller-manager, and scheduler service kube-scheduler; NVMe-oF represents network-based non-volatile memory specification; RBD represents block storage service; iSCSI represents Internet small computer system interface; NFS represents network file system; Lustre, kubelet, and Pod are all self-named terms;

[0042] Figure 2 A schematic diagram of the principle of realizing dynamic provision of PV objects and applying for storage resources from a heterogeneous third-party cloud storage service end by a cloud native container storage interface plug-in across heterogeneous storage backends of the present invention;

[0043] Figure 3 for Figure 2 The operation flow chart of the CSI plug-in in the figure; NVMe-oF represents network-based non-volatile memory specifications; Ceph represents block storage service; StorageClass represents storage class; parameters represents parameters; PVC object represents persistent data volume declaration object; gRPC represents remote procedure call; CreateVolume method represents the "create data volume" method;

[0044] Figure 4 A schematic diagram of the principle of a cloud-native container storage interface plug-in across heterogeneous storage backends of the present invention attaching storage resources provided by third-party cloud storage to working nodes and mounting them as application Pod data volumes;

[0045] Figure 5 for Figure 4The operation flow chart of the CSI plug-in in the figure, where hostpath represents the host path; node-driver-registrar represents the node driver registrar; socket represents the socket; GetPluginInfo interface represents the "get plug-in information" interface; gRPC represents the remote procedure call; kube-scheduler represents the scheduler service; NodeStageVolume method represents the "node temporary data volume" method; mount represents mounting; NodePublishVolume method represents the "node publish data volume" method. DETAILED DESCRIPTION

[0046] All features disclosed in all embodiments in this specification, or steps in all methods or processes implicitly disclosed, except for mutually exclusive features and / or steps, can be combined and / or expanded or replaced in any manner.

[0047] In view of the current situation in the background, the present invention aims to perform a higher-dimensional abstraction of the functions of the CSI plug-in in Kubernetes, so as to achieve the goal that a single CSI plug-in can connect to multiple storage protocols, overcome the limitation that there must be a one-to-one correspondence between the current CSI plug-in and the storage backend, and solve the problems of complex deployment process of the current dedicated CSI plug-in, high operation and maintenance costs, and low utilization of cluster computing resources.

[0048] like Figure 1As shown, the present invention provides a cloud native container storage interface plug-in across heterogeneous storage backends, including four components: a general component, a task distribution component, an abstract layer component, and a storage management component. Among them, the general component mainly implements the csi.ControllerServer interface and the csi.NodeServer interface. In addition, the general component also implements common logic that is not related to the specific storage protocol, such as parameter parsing and verification, and data volume indicator collection. The task distribution component mainly completes two functions: one is to realize the identification of the storage protocol type from the CSI request context. If the storage protocol type in the CSI request is unclear, the cooperation of the general component is also required; the second is to call the corresponding storage management component according to the storage protocol type obtained by parsing. The abstract layer component defines an abstract method for managing data volumes, associates the storage resources of the lower layer with the task distribution component of the upper layer, and shields the detailed implementation details of data volume management under a specific storage protocol. The storage management component corresponds one-to-one with the heterogeneous storage backend, realizes the specific details of calling the specific storage service end, and cooperates with the abstract layer component to complete the unified management of the heterogeneous storage backend. The storage management component is registered with the task distribution component through a hook function during the startup process of the CSI plug-in of the present invention. When the task distribution component receives a CSI request, it obtains the specified storage protocol type by parsing the CSI request, or reversely parses the storage protocol type through the data volume id generation rule, and then distributes the task to the corresponding storage management component according to the storage protocol type.

[0049] Based on the CSI plug-in of the present invention, the present invention also provides a deployment method for a cloud-native container storage interface plug-in across heterogeneous storage backends, and its deployment method is that all components are integrated into the same executable file, that is, the CSI plug-in proposed by the present invention is deployed in a containerized manner and provides a gRPC calling interface to the outside world.

[0050] The CSI plug-in container proposed in the present invention needs to be jointly deployed with the sidecar container officially provided by Kubernetes. The CSI plug-in container proposed in the present invention, the external-provisioner container, the external-attacher container, and the livenessprobe container are deployed in the form of Deployment to any working node in the Kubernetes cluster. The Deployment is called the CSI controller service; the CSI plug-in container proposed in the present invention, the node-driver-registrar container, and the livenessprobe container are deployed in the form of DaemonSet to all working nodes in the Kubernetes cluster. The DaemonSet is called the CSI node service.

[0051] Kubernetes-related control nodes are deployed separately, where the core components related to this embodiment include kube-apiserver, kube-scheduler, and kube-controller-manager, and each working node is deployed with a kubelet process.

[0052] The third-party cloud storage service client program can be deployed on the working node in the Kubernetes cluster, or on other nodes that are not managed by Kubernetes, as long as the network between the node and the Kubernetes cluster is reachable.

[0053] like Figure 2 As shown, based on the plug-in of the present invention, the present invention can also realize the principle of dynamic provision of PV objects and application for storage resources from a heterogeneous third-party cloud storage service client, including the following steps:

[0054] The first step is to create a StorageClass object for each third-party cloud storage service in kube-apiserver, and declare the associated cloud storage service information in the parameters attribute of the StorageClass object in the form of key-value pairs, including storage protocol type, server IP, port, remote storage resource mapping and other information, and the provisioner attribute value of all StorageClass objects is filled in with the CSI plug-in name proposed in the present invention. Then, create a PVC object associated with each StorageClass object in kube-apiserver, and declare the requirements for storage capacity and cross-node read and write control in the PVC object.

[0055] In the second step, deploy the Pod related to the actual business, declare the PVC object created above as a container data volume, and mount it to the working directory in the container. After the external-provisioner container in the CSI controller service listens to the creation event of the PVC object in the kube-apiserver, it constructs the CreateVolumeRequest structure and requests the CreateVolume method of the CSI plug-in container proposed in the present invention in a gRPC manner. The request will first be processed by the general component in the CSI plug-in container. After the general component verifies the parameters, it will parse the corresponding storage protocol type in the CSI request, and save the parameters and storage protocol type and other information into the request context structure. Then the general component will pass the request to the task distribution component. The task distribution component processes the request context structure from the general component, selects the management component that is adapted to the current request storage protocol type from the registered storage management components, and calls the abstract layer method implemented by the storage management component to complete the data volume creation.

[0056] In the third step, after receiving the called request, the storage management component first initiates a storage resource allocation request to the corresponding cloud storage service end. For example, the storage management component needs to call the interface provided by the server and create an internal image when calling the RBD storage protocol type. Then, the storage management component will create a csi.Volume structure based on the request information. The data volume id in the structure, that is, the csi.Volume.VolumeId property is defined as the concatenation of the storage protocol type and the CreateVolumeRequest.Name property with an agreed delimiter; the data volume context structure in the structure, that is, the csi.Volume.VolumeContext property is defined as detailed information of the storage service end, such as IP, port, and other information. Finally, the storage management component returns the data volume id and context information to the external-provisioner container through the CreateVolumeResponse structure.

[0057] In the fourth step, the external-provisioner container constructs a v1.PersistentVolume structure based on the returned information, adds additional information such as the associated PVC object (v1.PersistentVolume.Spec.ClaimRef), and stores the structure in the kube-apiserver.

[0058] like Figure 3 As shown, in an optional implementation, based on the plug-in of the present invention, the present invention can also realize the dynamic supply of PV objects and the operation principle process of applying for storage resources from a heterogeneous third-party cloud storage service client is as follows:

[0059] P1: Deploy the Kubernetes control node and create two working nodes in the Kubernetes cluster;

[0060] P2: Deploy a single copy of the CSI controller service on any worker node, and deploy the CSI node service on each worker node;

[0061] P3: Deploy third-party cloud storage service client processes in containers or directly, including Ceph, NVMe-oF, iSCSI, Lustre, and NFS storage services;

[0062] P4: Create a corresponding StorageClass object for each heterogeneous cloud storage service and save the server information in the form of key-value pairs in its parameters attribute;

[0063] P5: Create a PVC object associated with each StorageClass object and mount the corresponding PVC object in the application Pod;

[0064] P6: After the external-provisioner container in the CSI controller service detects that storage resources need to be managed, it calls the CreateVolume method of the CSI plug-in proposed in the present invention through gRPC;

[0065] P7: The general component completes parameter and storage protocol type parsing according to the CSI request, creates a request context structure, and calls the task distribution component;

[0066] P8: The task distribution component calls the adapted storage management component according to the storage protocol type;

[0067] P9: The storage management component generates a data volume ID by concatenating the storage protocol type and the data volume name;

[0068] P10: The storage management component determines whether the current storage protocol is a block storage type. If it is a block storage type, proceed to step P11; if it is not a block storage type, proceed to step P13;

[0069] P11: The storage management component calls the third-party cloud storage service client to apply for storage resource allocation;

[0070] P12: Reserved storage resources on the third-party cloud storage service side;

[0071] P13: The CSI plug-in proposed in the present invention feeds back response information to the external-provisioner container, including the data volume ID and data volume context information;

[0072] P14: The external-provisioner container builds a PV object based on the response information and stores it in the kube-apiserver.

[0073] like Figure 4 As shown, in an optional implementation, based on the plug-in of the present invention, the present invention can also implement adding storage resources provided by a third-party cloud storage service to a working node, and further mounting it as a container data volume, specifically including the following contents:

[0074] When the CSI node service is started, the host machine's / dev, / var / lib / kubelet / pods, and / var / lib / kubelet / plugins / unistore.csi.com directories will be declared as data volumes in the hostPath mode. After the CSI plug-in container is started, the socket file that provides gRPC services to the outside will be saved in the / var / lib / kubelet / plugins / unistore.csi.com directory. The "unistore.csi.com" directory in its path is the name of the CSI plug-in proposed in the present invention. The node-driver-registrar container will be mounted to the same directory of the host machine, that is, the / var / lib / kubelet / plugins / unistore.csi.com directory, thereby calling the GetPluginInfo interface of the CSI plug-in proposed in the present invention by directly accessing the socket file, and the interface will return the name and version information of the CSI plug-in of the present invention to the caller. After obtaining the CSI plug-in information, the node-driver-registrar container will report the plug-in name, version information, and the path of the socket file to the kubelet process on the working node.

[0075] After kube-scheduler schedules the application Pod to this working node, the VolumeManager component in kubelet calls the NodeStageVolume method of the CSI plug-in container in the CSI node service through a tuning loop. The general component in the CSI plug-in container will process the request, complete the parameter verification, reversely parse the storage protocol type used by the data volume according to the splicing rules of the data volume id, parse the remote storage service's ip, port, remote storage resource metadata and other information from the request data volume context structure attributes (csi.NodeStageVolumeRequest.VolumeContext), and save all this information to the CSI request context structure. The task distribution component processes the CSI request context structure, selects the management component that is adapted to the current request storage protocol type from the registered storage management components, and calls the abstract layer method implemented by the storage management component with the request context structure as a parameter to complete the storage resource attachment operation.

[0076] If the storage protocol type in the CSI request is a block storage type, such as NVMe-oF, RBD, etc., the corresponding storage management component will obtain the remote storage resource metadata from the request context structure, call the block device mapping interface provided by the storage service end, such as the nvme connect command of the NVMe-oF storage protocol type, or the map rbd command of the RBD storage protocol type, to map the remote block device to the current working node. After the storage service end responds normally, the device will be mounted in the host machine's / dev directory in a form similar to an unformatted disk. If the file system format is specified in the request context, the storage management component will format the disk into the specified file system format. Otherwise, the ext4 format will be used by default and mounted to the intermediate path, which is composed of the csi.NodeStageVolumeRequest.StagingTargetPath attribute concatenated with the data volume id. Its paradigm is as follows: / var / lib / kubelet / plugins / kubernetes.io / csi / pv / {pv name} / globalmount / {data volume id}). If the third-party storage service is a file storage type, such as Lustre, NFS, etc., the corresponding storage management component will obtain the call parameters from the request context structure, mainly including the mounted file system type, server IP, port and shared directory path, etc., and execute the mount command to mount the shared directory of the remote file storage service client to the intermediate path of this working node.

[0077] Then, the general component returns the NodeStageVolume method to the caller kubelet to indicate that the execution is complete. At this point, the storage resources provided by the remote storage service are attached to the current working node in the form of block devices or remote file systems. It should be noted that for block storage type data volumes, since the remote storage resource metadata cannot be obtained from the request when unmounting, the storage management component will create an additional file in the parent directory of the intermediate directory to save the metadata of the remote block device. When the data volume needs to be unmounted and the corresponding storage resources need to be removed from this working node, the storage management component will read the relevant information from the file, thereby communicating with the remote block storage service to remove this remote block device.

[0078] Finally, the tuning loop of the VolumeManager component in the kubelet calls the NodePublishVolume method of the CSI plug-in container in the CSI node service. The calling process is similar to the NodeStageVolume method. When the CSI request processing process is executed to the storage management plug-in, the storage plug-in first obtains the target path from the request CSI information, that is, the csi.NodePublishVolumeRequest.TargetPath parameter, and creates this directory in the file system of the worker node host. Then, the intermediate path described in the previous step is mounted to the target path. Since the implementation of this step is independent of the specific storage protocol, the public method implemented by the abstract layer can be used.

[0079] So far, the CSI plug-in proposed in the present invention has realized the attachment of storage resources provided by third-party cloud storage to the working node and mounting them as container data volumes.

[0080] like Figure 5 As shown, in an optional embodiment, based on the plug-in of the present invention, after completing the creation of the PV object and the allocation of storage resources on the server side, the storage resources can be added to the working node and mounted as a container data volume. The specific execution process is as follows:

[0081] Q1: The CSI node service declares the host's agreed path as a data volume in hostPath mode. Specifically, the CSI node service declares the host's / dev, / var / lib / kubelet / pods, and / var / lib / kubelet / plugins / unistore.csi.com directories as data volumes in hostPath mode;

[0082] Q2: The CSI plug-in container of the present invention saves the socket file that provides gRPC services to the outside world in this directory. Specifically, the CSI plug-in container is mounted to the / var / lib / kubelet / plugins / unistore.csi.com directory of the host machine when it is started, and saves the socket file that provides gRPC services to the outside world in this directory;

[0083] Q3: The node-driver-registrar container calls the GetPluginInfo interface of the CSI plug-in of the present invention. Specifically, the node-driver-registrar container is also mounted to the / var / lib / kubelet / plugins / unistore.csi.com directory of the host machine, and the GetPluginInfo interface of the CSI plug-in proposed by the present invention is called by directly accessing the socket file under the directory;

[0084] Q4: The GetPluginInfo interface returns the name and version information of the plug-in of the present invention. Specifically, the GetPluginInfo interface of the CSI plug-in returns the name and version information of the CSI plug-in to the node-driver-registrar container;

[0085] Q5: The node-driver-registrar container reports the name and version of the plug-in to the kubelet process. Specifically, the node-driver-registrar container reports the name and version of the CSI plug-in and the path of the gRPC service socket file to the kubelet process on the working node.

[0086] Q6: After the application pod is scheduled, kubele calls the NodeStageVolume method of the plug-in container of the present invention. Specifically, after kube-scheduler schedules the application pod to the current working node, the VolumeManager component in kubelet calls the NodeStageVolume method of the CSI plug-in container in the CSI node service through a tuning loop;

[0087] Q7: The general component performs parameter verification and parses out the storage protocol type used by the data volume. Specifically, the general component in the CSI plug-in container processes the CSI request, completes parameter verification, and reversely parses out the storage protocol type used by the data volume according to the splicing rules of the data volume ID;

[0088] Q8: The general component parses the storage service end IP, port and other information and saves them in the request context structure. Specifically, the general component parses the remote storage service IP, port and other information from the request data volume context structure attributes, saves all the information in the request context structure, and calls the task distribution component through the CSI request;

[0089] Q9: The task distribution component calls the abstract layer method to complete the storage resource addition operation. Specifically, the task distribution component selects the appropriate management component according to the storage protocol type and calls its corresponding abstract layer method to complete the storage resource addition operation;

[0090] Q10: Determine the storage protocol type. Specifically, the storage management component determines the type of the current storage protocol. If the storage protocol type is a block storage type, Q15 and subsequent steps are executed; if the storage protocol type is a file storage type, Q11 and subsequent steps are executed;

[0091] Q11: The storage management component mounts the shared directory to the intermediate path of the current working node. Specifically, if the storage protocol type is block storage type, the storage management component executes the mount command to mount the shared directory of the remote file storage service client to the intermediate path of the current working node;

[0092] Q12: Return the NodeStageVolume method to kubelet. Specifically, if the storage protocol type is file storage type, the storage management component calls the block device mapping interface provided by the storage service end to map the remote block device to the current working node;

[0093] Q13: kubelet calls the NodePublishVolume method of the CSI plug-in container of the present invention. Specifically, the storage management component will format the disk according to the file system format specified in the request context and mount it to the intermediate path.

[0094] Q14: The storage management component creates the target path and mounts the intermediate path to it. Specifically, the general component returns the NodeStageVolume method to kubelet to indicate that the execution is complete;

[0095] Q15: The storage management component calls the remote block device mapping interface provided by the storage service end. Specifically, the VolumeManager component in the kubelet calls the NodePublishVolume method of the CSI plug-in container in the CSI node service through a tuning loop;

[0096] Q16: The storage management component formats the remote disk and mounts it to the intermediate path. Specifically, the storage plug-in obtains the target path from the call request, creates this directory in the host file system of the working node, and mounts the intermediate path to the target path.

[0097] It should be noted that within the scope of protection defined in the claims of the present invention, the following embodiments can be combined and / or expanded or replaced in any logical way from the above specific implementation methods, such as disclosed technical principles, disclosed technical features or implicitly disclosed technical features.

[0098] In other embodiments, including but not limited to the following embodiments:

[0099] Example 1

[0100] A cloud-native container storage interface plug-in across heterogeneous storage backends, including:

[0101] Common components, used to implement common logic not related to storage protocols;

[0102] The task distribution component is used to identify the storage protocol type from the CSI request context and call the corresponding storage management component;

[0103] Abstract layer components, used to define abstract methods for managing the entire life cycle of data volumes;

[0104] The storage management component is used to connect to the storage backend and implement abstract layer methods.

[0105] Example 2

[0106] On the basis of Example 1, the general component is further used to: implement the csi.ControllerServer interface and the csi.NodeServer interface, where csi.ControllerServer represents a controller service and csi.NodeServer represents a node service.

[0107] Example 3

[0108] Based on Example 1, the common logic unrelated to the storage protocol includes parameter analysis and verification, and data volume indicator collection.

[0109] Example 4

[0110] Based on Example 1, the method of identifying the storage protocol type from the CSI request context and calling the corresponding storage management component specifically includes: when the storage protocol type in the CSI request is unclear, the storage protocol type is parsed out by using the general component and passed to the task distribution component; then the corresponding storage management component is called according to the storage protocol type obtained by the parsing; when the storage protocol type in the CSI request is clear, the CSI plug-in directly passes the request to the task distribution component.

[0111] Example 5

[0112] Based on Example 1, the abstract method for defining the full lifecycle management of a data volume specifically includes: associating lower-layer storage resources with upper-layer task distribution components to shield detailed implementation details of data volume management under a specific storage protocol.

[0113] Example 6

[0114] Based on Example 1, the method of docking the storage backend and implementing the abstract layer specifically includes: one-to-one correspondence between the storage management component and the heterogeneous storage backend, realizing the details of calling the specific storage service end, and cooperating with the abstract layer component to complete the unified management of the heterogeneous storage backend.

[0115] Example 7

[0116] Based on Example 1, the storage management component is registered with the task distribution component through a hook function during the CSI plug-in startup process. When the task distribution component receives the CSI request, it obtains the specified storage protocol type by parsing the CSI request, or reversely parses the storage protocol type through the data volume id generation rule, and then distributes the task to the corresponding storage management component according to the storage protocol type.

[0117] Example 8

[0118] A method for deploying a cloud native container storage interface plug-in across heterogeneous storage backends includes the following steps:

[0119] Step 1: Deploy the container of the cloud native container storage interface plug-in across heterogeneous storage backends described in Example 1 to Example 7 and the external-provisioner, external-attacher, and livenessprobe containers provided by Kubernetes as controller services to any working node in the cluster in the form of Deployment, where external-provisioner represents an external storage provider, external-attacher represents an external storage attacher, livenessprobe represents a service probe, and Deployment represents a deployment set;

[0120] Step 2: Deploy the plug-in container and the node-driver-registrar and livenessprobe containers provided by Kubernetes as node services to all working nodes in the cluster in the form of DaemonSet, where DaemonSet represents a daemon service set.

[0121] Step 3: Create a StorageClass object to save the available third-party cloud storage service client information, which includes storage protocol type, IP, port, and data volume related details. StorageClass represents the storage class.

[0122] Step 4: Create a PVC object associated with the StorageClass object. StorageClass represents the storage class.

[0123] Step 5: Declare and deploy containers related to actual business. During the container declaration process, mount the PVC object as a working data volume to a specified path in the application container. PVC represents a persistent data volume declaration.

[0124] Example 9

[0125] A method for deploying a cloud native container storage interface plug-in across heterogeneous storage backends includes the following steps:

[0126] Step 1: During the startup process of the cloud native container storage interface plug-in across heterogeneous storage backends in Examples 1 to 7, each storage management component registers the storage protocol it implements into the task distribution component through a hook function;

[0127] Step 2: When the plug-in receives a CSI-related request from the application container, it determines whether the request context carries a clear storage protocol type parameter;

[0128] Step 3: If the CSI request only contains the data volume ID as a call parameter and does not contain a clear storage protocol type parameter, such as the csi.NodeUnpublishVolume method, the general component reversely parses the storage protocol type based on the data volume ID and passes it to the task distribution component; if the CSI request already contains a clear storage protocol type parameter, such as the csi.CreateVolume method, the CSI plug-in directly passes the request to the task distribution component;

[0129] Step 4: The task distribution component sends the task to the corresponding storage management component based on the obtained storage protocol type. The storage management component communicates with the third-party cloud storage service client to complete the management of the storage resources involved in the request, and returns the storage protocol type as a component of the data volume id to the caller. The management of storage resources includes the generation of data volume id, and the allocation and recovery of storage service client resources corresponding to the data volume.

[0130] Example 10

[0131] Based on Example 9, in the fourth step, in the data volume mounting scenario, the storage management component also performs the following sub-steps: the detailed mapping relationship between the data volume and the third-party cloud storage service client is saved in the parent directory of the data volume mounting directory in the form of a file, and when a request to unmount the data volume is subsequently received, the mapping relationship will be obtained from the file.

[0132] Embodiment 11

[0133] Based on Example 9, in the fourth step, in the CSI request response structure that partially supports setting context information, the storage management component also performs the following sub-steps: returning the cloud storage service end information carried by the corresponding request to the requester as context information for the requester to maintain the data volume context, and carrying this context information in subsequent calls, wherein the cloud storage service end information includes IP, port, and server storage resource mapping information.

[0134] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0135] According to one aspect of an embodiment of the present invention, a computer program product or a computer program is provided, the computer program product or the computer program includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in the above various optional implementations.

[0136] As another aspect, an embodiment of the present invention further provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

Claims

1. A cloud-native container storage interface plug-in across heterogeneous storage backends, characterized in that: include: Common components, used to implement common logic not related to storage protocols; The task distribution component is used to identify the storage protocol type from the container storage interface CSI request context and call the corresponding storage management component; Abstract layer components, used to define abstract methods for managing the entire life cycle of data volumes; Storage management component, used to connect to the storage backend and implement abstraction layer methods; The identifying the storage protocol type from the container storage interface CSI request context and calling the corresponding storage management component specifically includes: when the storage protocol type in the CSI request is unclear, using the general component to cooperate to parse out the storage protocol type and pass it to the task distribution component; then calling the corresponding storage management component according to the storage protocol type obtained by the parsing; when the storage protocol type in the CSI request is clear, the CSI plug-in directly passes the request to the task distribution component; The method for defining the data volume full life cycle management abstraction specifically includes: associating the storage resources of the lower layer with the task distribution components of the upper layer to shield the detailed implementation details of the data volume management under the specific storage protocol; The method of docking with the storage backend and implementing the abstract layer specifically includes: one-to-one correspondence between the storage management component and the heterogeneous storage backend, realizing the details of calling the specific storage service end, and cooperating with the abstract layer component to complete the unified management of the heterogeneous storage backend.

2. The cloud-native container storage interface plug-in across heterogeneous storage backends according to claim 1, characterized in that: The general component is also specifically used to: implement the csi.ControllerServer interface and the csi.NodeServer interface, where csi.ControllerServer represents the controller service and csi.NodeServer represents the node service.

3. The cloud-native container storage interface plug-in across heterogeneous storage backends according to claim 1, characterized in that: The common logic unrelated to the storage protocol includes parameter analysis and verification, and data volume indicator collection.

4. The cloud-native container storage interface plug-in across heterogeneous storage backends according to claim 1, characterized in that: The storage management component is also used to register with the task distribution component through a hook function during the CSI plug-in startup process. When the task distribution component receives a CSI request, it obtains the specified storage protocol type by parsing the CSI request, or reversely parses the storage protocol type through the data volume id generation rule, and then distributes the task to the corresponding storage management component according to the storage protocol type.

5. A method for deploying a cloud native container storage interface plug-in across heterogeneous storage backends, characterized in that: The following steps are involved: Step 1: Deploy the container of the cloud native container storage interface plug-in across heterogeneous storage backends of any one of claims 1 to 4 and the external-provisioner, external-attacher, and livenessprobe containers provided by Kubernetes as controller services to any working node in the cluster in the form of Deployment, where external-provisioner represents an external storage provider, external-attacher represents an external storage attacher, livenessprobe represents a service probe, and Deployment represents a deployment set; Step 2: Deploy the plug-in container and the node-driver-registrar and livenessprobe containers provided by Kubernetes as node services to all working nodes in the cluster in the form of DaemonSet, where DaemonSet represents a daemon service set. Step 3: Create a StorageClass object to save the available third-party cloud storage service client information, which includes storage protocol type, IP, port, and data volume related details. StorageClass represents the storage class. Step 4: Create a PVC object associated with the StorageClass object. StorageClass represents the storage class. Step 5: Declare and deploy containers related to actual business. During the container declaration process, mount the PVC object as a working data volume to a specified path in the application container. PVC represents a persistent data volume declaration.

6. A method for deploying a cloud native container storage interface plug-in across heterogeneous storage backends, characterized in that: The following steps are involved: Step 1: During the startup process of the cloud-native container storage interface plug-in across heterogeneous storage backends in any one of claims 1 to 4, each storage management component registers the storage protocol it implements into the task distribution component through a hook function; Step 2: When the plug-in receives a CSI-related request from the application container, it determines whether the request context carries a clear storage protocol type parameter; Step 3: If the CSI request only contains the data volume ID as a call parameter and does not contain a clear storage protocol type parameter, the general component reversely parses the storage protocol type based on the data volume ID and passes it to the task distribution component; if the CSI request already contains a clear storage protocol type parameter, the CSI plug-in directly passes the request to the task distribution component; Step 4: The task distribution component sends the task to the corresponding storage management component based on the obtained storage protocol type. The storage management component communicates with the third-party cloud storage service client to complete the management of the storage resources involved in the request, and returns the storage protocol type as a component of the data volume id to the caller. The management of storage resources includes the generation of data volume id, and the allocation and recovery of storage service client resources corresponding to the data volume.

7. The method for deploying a cloud native container storage interface plug-in across heterogeneous storage backends according to claim 6, characterized in that: In the fourth step, in the data volume mounting scenario, the storage management component also performs the following sub-steps: the detailed mapping relationship between the data volume and the third-party cloud storage service client is saved in the parent directory of the data volume mounting directory in the form of a file, and when a request to unmount the data volume is subsequently received, the mapping relationship will be obtained from the file.

8. The method for deploying a cloud native container storage interface plug-in across heterogeneous storage backends according to claim 6, characterized in that: In the fourth step, in the CSI request response structure that partially supports setting context information, the storage management component also performs the following sub-steps: returning the cloud storage service end information carried by the corresponding request to the requester as context information for the requester to maintain the data volume context, and carrying this context information in subsequent calls. The cloud storage service end information includes IP, port, and server storage resource mapping information.