Container-based cloud hard disk management method and device

Abstract

The invention relates to a container-based cloud hard disk management method and device, and the method comprises the steps: carrying out the communication with an ESSD cluster based on a gRPC protocol through a Control Server assembly of a CSI plug-in, and carrying out the management of an ESSD cloud hard disk; the nvmftgt service of the SPDK is operated on the host machine, and interaction with the ESSD cloud hard disk in the ESSD cluster is carried out through an NVMe-oF protocol and an RDMA network, so that a block device BDEV corresponding to the ESSD cloud hard disk is created locally; the method comprises the following steps of: mounting a directory in which an SPDK socket file (spdk.sock) and an SPDK remote procedure call script (rpc.py) on a host machine are located to a Pod of a NodeServer component running a CSI (Channel State Information) plug-in; and calling an rpc.py script in the Pod through a NodeServer component, interacting with the nvmftgt service of the host machine based on an spdk-rpc command, and carrying out mounting or unloading operation on the BDEV. By integrating the SPDK, the NVMe-of and the RDMA, the bottleneck of a kernel can be got rid of, meanwhile, the problem of service interaction between the CSI node component and the host is well solved in a mode of mounting a sock file, and finally containerized efficient use of the high-performance cloud hard disk is realized.

Description

Technical Field

[0001] This application relates to the field of container technology, and in particular to a container-based cloud disk management method and apparatus. Background Technology

[0002] With the rapid development of cloud computing and container technologies, Kubernetes has become the de facto standard in the field of container orchestration. The establishment of its Container Storage Interface (CSI) specification aims to standardize the integration between container orchestration systems and storage backends, enabling storage vendors to provide persistent storage services for containerized applications through a unified interface. Against this backdrop, providing low-latency, high-IOPS (Input / Output Operations Per Second) storage support to high-performance applications running on Kubernetes has become a significant technical challenge.

[0003] Currently, common solutions largely rely on general-purpose distributed storage systems, such as Ceph, and provide storage services through their CSI plugins. Major cloud vendors (such as Huawei and Alibaba Cloud) also offer their own CSI plugins for integrating their cloud storage products. However, these existing solutions have significant limitations: traditional distributed storage systems like Ceph are limited by their architecture and network protocols, resulting in I / O performance bottlenecks that make them unsuitable for applications with extreme performance requirements; while the CSI plugins from major cloud vendors can interface with their own storage services, they are typically based on standard kernel drivers and general network protocols (such as iSCSI), lacking deep optimization for high-performance underlying technologies like NVMe-oF and RDMA (Remote Direct Memory Access), thus failing to fully leverage the potential of next-generation high-performance cloud disks. Therefore, existing technologies lack a solution that can efficiently bridge the Kubernetes container platform with underlying high-performance cloud disks. Summary of the Invention

[0004] This application provides a container-based cloud disk management method and apparatus.

[0005] According to a first aspect of the embodiments of this application, a container-based cloud disk management method is provided, the method comprising: The ControllerServer component of the CSI plugin communicates with the ESSD cluster based on the gRPC protocol and manages the ESSD cloud disks in response to persistent volume declaration (PVC) requests from Kubernetes. The SPDK nvmf_tgt service runs on the host machine and interacts with the ESSD cloud disks in the ESSD cluster through the NVMe-oF protocol and RDMA network to create block devices BDEV corresponding to the ESSD cloud disks locally. Mount the directory containing the SPDK socket file spdk.sock and the SPDK remote procedure call script rpc.py on the host machine to the Pod of the NodeServer component running the CSI plugin; When a Kubernetes request to mount or unmount the ESSD cloud disk is received, the NodeServer component calls the rpc.py script within the Pod and interacts with the nvmf_tgt service of the host machine based on the spdk-rpc command to mount or unmount the block device BDEV.

[0006] According to a second aspect of the embodiments of this application, a container-based cloud disk management device is provided, the device comprising: The management module is used to communicate with the ESSD cluster via the ControllerServer component of the CSI plugin based on the gRPC protocol and to manage the ESSD cloud disks in response to persistent volume declaration (PVC) requests from Kubernetes. The interaction module is used to run the nvmf_tgt service of SPDK on the host machine and interact with the ESSD cloud disks in the ESSD cluster through the NVMe-oF protocol and RDMA network to create block devices BDEV corresponding to the ESSD cloud disks locally. The mounting module is used to mount the directory containing the SPDK socket file spdk.sock and the SPDK remote procedure call script rpc.py on the host machine to the Pod of the NodeServer component running the CSI plugin; The processing module is used to, when receiving a Kubernetes request to mount or unmount the ESSD cloud disk, call the rpc.py script in the Pod through the NodeServer component, and interact with the nvmf_tgt service of the host machine based on the spdk-rpc command to mount or unmount the BDEV (Block Device).

[0007] According to a third aspect of the embodiments of this application, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0008] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the methods described above in this application.

[0009] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods described above in this application.

[0010] The container-based cloud disk management method and apparatus provided in this application integrate SPDK, NVMe-of and RDMA technologies, which can get rid of kernel bottlenecks. At the same time, by mounting sock files, it effectively solves the problem of interaction between CSI node components and host services, and finally realizes the containerized and efficient use of high-performance cloud disks. Attached Figure Description

[0011] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 A schematic diagram of the architecture of a provisioner component provided for an exemplary embodiment of this application; Figure 2 A schematic diagram of a JSON file writing process provided for an exemplary embodiment of this application; Figure 3 A schematic diagram of a JSON file reading process provided for an exemplary embodiment of this application; Figure 4 A schematic diagram illustrating the process of mounting a file system on a nodeserver, as provided in an exemplary embodiment of this application; Figure 5 A flowchart illustrating a container-based cloud disk management method provided for an exemplary embodiment of this application; Figure 6 A schematic block diagram of the functional modules of a container-based cloud disk management device provided for an exemplary embodiment of this application; Figure 7 A structural block diagram of an electronic device provided in an exemplary embodiment of this application; Figure 8 A structural block diagram of a computer system provided for an exemplary embodiment of this application. Detailed Implementation

[0012] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0013] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0014] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0015] It should be noted that the terms "a" and "a plurality of" used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more". The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0016] To address the technical problems existing in related technologies, this application provides a container-based cloud disk management solution that can avoid the overhead of traditional storage stacks. It can directly utilize technologies such as NVMe-oF (NVMe over Fabrics, a network-based NVMe storage access protocol) and RDMA to achieve high-performance, low-latency storage access, while maintaining the standardization and ease of use of the CSI standard.

[0017] This application's embodiments can be used in cloud computing platforms and deployed on physical machines. To support interaction with ESSD (Enhanced Solid State Drive), the embodiments are designed for two main components: a Provisioner and a Node plugin. The Provisioner component is responsible for receiving requests for creating and deleting PVCs (Persistent Volume Claims) and invoking the backend storage system (i.e., ESSD) to manage the volume's lifecycle. The Node plugin is responsible for mounting or unmounting volumes on Kubernetes nodes.

[0018] (1) Design of provisioner architecture.

[0019] The main component in the provisioner is the ControllerServer, which is responsible for control plane operations such as creating, deleting, expanding, or viewing volume information. These operations depend on the backend ESSD cluster.

[0020] Due to the complexity of the ESSD cluster environment and the lack of mature client libraries comparable to Ceph, this implementation uses gRPC (Google Remote Procedure Call) to interact with the backend cluster. gRPC is an open-source Remote Procedure Call (RPC) framework that enables efficient communication between different computers and processes. Using Protocol Buffers as its interface definition language, it automatically generates client and server-side code, greatly simplifying the development of distributed applications. By using gRPC, CSI can easily communicate with the backend storage cluster and also provides a standard CSI interface for Kubernetes clusters, achieving good scalability and maintainability.

[0021] See also Figure 1 As shown, Figure 1 This is a schematic diagram of the architecture of the provisioner component provided in an embodiment of this application. Wherein: ControllerServer11 is a key component of the CSI provisioner10 plugin, used to send volume requests to the ESSD cluster 13. These volume requests can include operations such as creating, deleting, expanding, or querying ESSDs. ControllerServer11 connects to the ESSD cluster 13 via the gRPC protocol to manage the cluster's volume service.

[0022] Kubernetes14 is used to send commands such as creating and deleting PVCs to ControllerServer11.

[0023] ControllerServer11 is the core module within the Provisioner component. It receives relevant instructions from Kubernetes14 and is responsible for translating them into operations on the backend storage.

[0024] In this embodiment, the ESSD cluster 13 includes a gRPC Server, which provides an interface to gRPC servers running on remote ESSD clusters that provide volume management services. ControllerServer 11 also includes a gRPC client, which is a library located within or tightly coupled to ControllerServer 11, and is client code automatically generated through Protocol Buffers definition files.

[0025] In this embodiment, IdentityServer12 is a standard component defined in the CSI specification, used to inform the caller of the CSI plugin's name or version number, etc. Kubernetes first successfully "identifies" and "verifies" the plugin by calling methods such as GetPluginInfo and GetPluginCapabilities of IdentityServer12 before trusting it and sending it more important requests such as CreateVolume or NodeStageVolume. It is the first step in establishing trust and communication between the CSI plugin and Kubernetes. Therefore, IdentityServer12 can handle identification-related requests sent by Kubernetes14, such as obtaining CSI plugin information.

[0026] (2) Design for Node architecture.

[0027] In this embodiment, the main component in Node is NodeServer, which is responsible for node-level operations, such as mounting and unmounting volumes, and is mainly related to the host machine.

[0028] First, the challenge lies in how to read and write high-performance cloud disks on a physical machine. Currently, high-performance cloud disks, primarily ESSD, mostly use NVMe solid-state drives (SSDs) as their underlying storage medium. SPDK (Storage Performance Development Kit) has a natural advantage in NVMe+RDMA scenarios. This application's embodiment connects to remote NVMe-oF targets by installing the SPDK NVMe driver nvmf_tgt on the physical machine, exposes the block devices via RDMA, and interacts with them in the same way as local NVMe SSDs, thus achieving convenient access to high-performance cloud disks.

[0029] Furthermore, the implementation can effectively utilize the SPK high-performance suite's mechanisms such as big pages, polling, core partitioning, and lock-free operation, thereby reducing system overhead caused by CPU (Central Processing Unit) context switching and concurrent mutual exclusion. This can resolve performance bottlenecks in front-end network, back-end network, and disk write operations, improve I / O (Input / Output) response speed, and enhance system performance.

[0030] Secondly, this solution enables communication between the Node plugin and the nvmf_tgt service on the host machine by mounting the SPDK socket file " / var / tmp / spdk.sock" into the pod (KubernetesPod, Kubernetes container group). This is the default listening address used by the RPC client in SPDK. Developers or administrators can connect to this socket to perform various management and control operations on the SPDK application.

[0031] Finally, the embodiment uses the script ". / spdk / scripts / rpc.py" to perform specific operations on the volume. The rpc.py script file is an RPC client tool provided by SPDK, and the RPC functions provided by various modules can be called through this script. Therefore, the embodiment only needs to mount the directory containing rpc.py and spdk.sock into the pod to interact with the host machine's nvmf_tgt via the spdk-rpc command, performing operations such as creating, deleting, expanding, and configuring QoS for block devices. Furthermore, the embodiment can modify the processing logic in the Python files under the ". / spdk / python / spdk / rpc" directory to meet other development and debugging needs, demonstrating a certain degree of extensibility.

[0032] This embodiment utilizes SPDK and NVMe-oF technologies to directly manage block devices on the host machine, fully leveraging the performance advantages of ESSD cloud disks and providing higher storage access performance. The embodiment also eliminates the need for integration and optimization of RDMA support in the source code, as this process is completed on the host machine and relies on the underlying network hardware's RDMA capabilities. Therefore, we only need to correctly configure and use these RDMA network devices. This embodiment simplifies the overall architecture's operation and management, offering a more direct and efficient approach than CephCSI.

[0033] Based on the above embodiments, in order to ensure data reliability, the embodiments also include a persistence design. The embodiments create an essd.volumes dictionary in memory to store ESSD cloud disk volume information, and also persist the volume information to a JSON (JavaScript Object Notation) file on the disk. Each time a volume is created or deleted, the relevant information is written to this JSON file so that the volume information can be recovered from the file when the plugin is restarted.

[0034] The in-memory `essd.volumes` dictionary provides fast access and update capabilities, while the JSON file on disk ensures the persistence and reliability of volume information. This design effectively avoids the loss of volume information due to plugin process restarts or node failures, thereby ensuring the consistency and availability of persistent volumes in the Kubernetes cluster.

[0035] like Figure 2 As shown, Figure 2 The JSON file writing process provided in this application embodiment can specifically include the following steps: S21, Enter volume information.

[0036] In this embodiment, this step is the persistent data preparation phase. After the ControllerServer successfully executes an operation (such as creating or deleting a volume), it retrieves or generates detailed information (metadata) for that volume.

[0037] In this embodiment, volume information is typically organized within a Go language struct. The fields of this struct may include: VolumeID: The unique identifier of the volume (the most critical field).

[0038] Capacity: The volume's capacity (e.g., how many GiB).

[0039] RequestName: The name of the corresponding Kubernetes PVC.

[0040] Status: The current status of the volume (e.g., Created, Deleting).

[0041] Information such as creation time and the cluster to which it belongs.

[0042] Therefore, this step determines what data needs to be saved, which forms the basis for all subsequent operations.

[0043] S22 converts the volume information structure into a JSON string using the json.Marshal interface.

[0044] In this embodiment, this step completes data serialization, which converts the in-memory struct object (Go's native data structure) into a platform-independent, language-independent string representation (JSON format).

[0045] Since serialization is a prerequisite for persistence, data can only be reliably written to a file and correctly read in the future if it is converted to a standard format (JSON).

[0046] S23. Generate the corresponding JSON file path based on the volume ID to ensure uniqueness.

[0047] In this embodiment, this step addresses the organization and indexing of data storage, enabling the allocation of a unique storage location for the metadata of each volume. This ensures that read and write operations on each volume do not interfere with each other. When searching for information about a specific volume, the file path can be constructed directly using the ID for quick location, without traversing all files, thus achieving fast retrieval. Furthermore, it prevents information from different volumes from being incorrectly overwritten.

[0048] S24, writes the encoded JSON data to a file using the ioutil.WriteFile function.

[0049] In this embodiment, this step is the final execution step of data persistence, which permanently saves the serialized JSON data from memory to a specified location on disk. It is a core technical means to achieve high availability and state recovery capabilities. Only by writing the data to non-volatile storage (disk) can it be read again after the plugin restarts (json.Unmarshal), thereby restoring the management state in memory (essd.volumes dictionary) and avoiding information loss.

[0050] The implementation ensures that the CSI plugin's own management state is not lost due to failures such as program restarts or node restarts. If the plugin Pod is scheduled to another node, the newly started instance can quickly take over storage management tasks by reading these JSON files. Furthermore, the implementation utilizes the Go standard library and the universal JSON format to implement a lightweight, highly reliable persistence solution that does not rely on external databases (such as MySQL or Redis), greatly simplifying system architecture and deployment complexity.

[0051] like Figure 3 As shown, Figure 3 The schematic diagram of the JSON file reading process provided in this application embodiment may specifically include the following steps: S31, Enter volume information.

[0052] In this embodiment, this is the trigger and location starting point for data reading. Unlike S21 above, the purpose here is not to input complete volume information, but to input a key identifier on how to find the required volume information.

[0053] The "input" here is usually not manual input, but rather the automatic invocation of program logic. For example: Restoring by traversal: When the component starts, in order to restore the full state, it will attempt to read all JSON files in the persistent directory. At this time, the "input" can be regarded as a directory path (such as / var / lib / essd-csi / ).

[0054] On-demand query: When a request for a specific volume needs to be processed (such as deleting a volume with a known ID), the "input" is the specific volume ID.

[0055] S32 generates the corresponding JSON file path based on the volume ID.

[0056] In this embodiment, this step maps logical identifiers to physical storage locations. It utilizes the exact same algorithm as step S23 described above during writing, deriving the exact location of the corresponding file on the disk from the volume ID. For example, string concatenation is performed using the same rules as in step S23. This design ensures consistency between read and write paths, guaranteeing accurate location of previously written files. Therefore, this step is crucial for achieving reliable persistence. A unique and predictable file path is a guarantee of correct data retrieval.

[0057] S33, use the ioutil,ReadFile function to read the file content.

[0058] The embodiment loads non-volatile data stored on the disk into memory by performing a physical read operation.

[0059] The `ioutil.ReadFile()` function is a convenient Go function for reading the entire contents of a file at once. It takes a file path as an argument and returns a []byte slice of the file contents and an error. If the file does not exist or the read fails, it returns the corresponding error, which the plugin can use to determine if the volume information may have been deleted or if error handling is required.

[0060] S34 is decoded into a recognizable structure using the json.Unmarsha function.

[0061] In this embodiment, this step completes data deserialization, which is the reverse process of step S22 above. It converts the generic, serialized JSON string back into a native data structure (struct) that can be directly understood and manipulated by the Go language program.

[0062] In this embodiment, after decoding, the volume information is no longer a meaningless string of characters, but a three-dimensional program object that restores all its attributes (such as capacity, status, name, etc.). This object is then populated into the essd.volumes dictionary, allowing the CSI plugin to "remember" the status of all managed volumes and continue to function normally and respond to requests.

[0063] Based on the above embodiments, in the embodiments provided in this application, SPDK can provide a high-performance, low-latency raw disk access method, avoiding the overhead of the traditional Linux block device stack. The embodiments can use the SPDK open source library to directly interact with the NVMe-oF protocol of the ESSD cloud disk, thus enabling direct management and mounting of the ESSD cloud disk as a raw disk storage, resulting in higher I / O performance, which is suitable for some application scenarios with high performance requirements.

[0064] Furthermore, since the design of the embodiment is closer to the access method of bare metal storage devices, it can create its own file system, which is compatible with most common file system types, giving the ESSDCSI plugin advantages in terms of performance and flexibility.

[0065] Specifically, such as Figure 4 As shown, Figure 4 A flowchart illustrating the process of mounting a file system on a nodeserver is provided. This process may include the following steps: S41, acquire the mutex lock mutex.Lock0 to ensure thread safety.

[0066] The implementation can prevent multiple mount / unmount requests from operating on the same volume or underlying resource at the same time, avoiding data races and state chaos.

[0067] S42, parse request parameters: volume ID, volume name, volume size, volume type, target path, file system type.

[0068] The example extracts key information from requests sent by Kubernetes. These parameters determine the specific objects and targets of all subsequent operations. For example, which volume to operate on, which file system to use, and where to mount it.

[0069] S43, Create the SPDKController corresponding to the target volume.

[0070] In this embodiment, within the SPDK framework, a logical controller is created for the specified volume, which serves as the entry point for I / O interaction with the NVMe device.

[0071] S44 creates an NVMf target for communication with the SPDKController. It then connects to this target and returns the device path (nvme_path) after a successful connection.

[0072] In this embodiment, the network connects to the remote ESSD cluster (Target) via the NVMe-oF protocol over RDMA, and a corresponding block device file (e.g., / dev / nvme1n1) is generated within the local operating system. This nvme_path is the object of subsequent operations.

[0073] S45, os.Mkdir(mountPath,0750) creates the mount target path.

[0074] In this example, the directory specified by the Pod can be created on the host machine's file system. For example, / var / lib / kubelet / ... / my-volume.

[0075] S46, mount the nvme_path device to the target path.

[0076] In this embodiment, the local block device (nvme_path) obtained in the previous step can be mounted to the created directory (mountPath) by calling a system function (such as mount). At this point, the data channel is fully established.

[0077] Specifically, 1) Create a mount.SafeFormatAndMount instance, which encapsulates mount-related operations diskMounter:=&mount.SafeFormatAndMount{interface:mount.New(""),Exec:utilexec.NewO}.

[0078] 2) Create a file system using diskMounter.Exec.Command0: ext4: mkfs.ext4 -m0 -Enodiscard,lazy itable init=1,lazy journal init=1 $nvme path xfs: mkfs.xfs -K$nvme path.

[0079] 3) Mount the NVMe path to the target path opt := 0string{"netdev"} / / Inform the kernel that this is a file system that requires network support. This ensures that all network devices are ready when the system starts up before mounting the ESSD cloud disk, avoiding mounting failure due to network problems.

[0080] if volFsType == "xfs" { opt = append(opt, "nouuid")} / / XFS file system should not generate UUID (Universally Unique Identifier), which can avoid mount failure due to UUID changes. diskMounter.FormatAndMount(nvme path, mount path, file system type, opt).

[0081] S47 sets the mount point permissions to be readable, writable, and executable by everyone.

[0082] The implementation ensures that container processes within a Pod have sufficient permissions to read and write to mounted directories, thus guaranteeing storage availability.

[0083] S48, Update the volume information in the JSON file.

[0084] The implementation writes the successful result of this mount (e.g., updating the volume's mount status and mount point path) to a persistent JSON file. This ensures that even if the plugin restarts, it will still know which volume is mounted and where.

[0085] S49, release the mutex.

[0086] After the mount operation is completed, the implementation releases the lock acquired at the beginning, allowing the system to process the next storage request for another volume. This releases resources and restores concurrency.

[0087] Based on the above embodiments, this application provides a container-based cloud disk management method, such as... Figure 5 As shown, the method may include the following steps: In step S510, the ControllerServer component of the CSI plugin communicates with the ESSD cluster based on the gRPC protocol and manages the ESSD cloud disks in response to persistent volume declaration (PVC) requests from Kubernetes. This management may include lifecycle management such as creation, deletion, or expansion.

[0088] In this embodiment, the CSI plugin's ControllerServer component acts as the control center, communicating with the remote ESSD cluster management API via the efficient gRPC protocol. It is specifically responsible for responding to Kubernetes cluster-level storage resource requests (PVCs), enabling full lifecycle management of ESSD cloud disks, including creation, deletion, and expansion. This ensures on-demand allocation and elastic scaling of storage resources.

[0089] In step S520, the nvmf_tgt service of SPDK is run on the host machine to interact with the ESSD cloud disks in the ESSD cluster through the NVMe-oF protocol and RDMA network in order to create a block device BDEV corresponding to the ESSD cloud disk locally.

[0090] This embodiment establishes a high-performance data path, enabling the compute nodes (host machines) that need to access storage to establish a low-latency, high-bandwidth network connection with the remote ESSD cluster by running the high-performance service nvmf_tgt of SPDK and utilizing the NVMe-oFover RDMA protocol. The core function of this service is to create a block device (BDEV) within the local operating system that corresponds one-to-one with the remote ESSD cloud disk, providing an object for subsequent mounting operations.

[0091] In step S530, the directory containing the SPDK socket file spdk.sock and the SPDK remote procedure call script rpc.py on the host machine is mounted to the Pod of the NodeServer component running the CSI plugin.

[0092] To enable the NodeServer component running in the Pod to interact with the SPDK service (nvmf_tgt) on the host machine, this example innovatively mounts the directory of the SPDK management socket (spdk.sock) and management script (rpc.py) on the host machine into the Pod. This is equivalent to providing a "control channel" for the program inside the container, allowing it to directly send management commands to the SPDK service on the host machine.

[0093] In step S540, upon receiving a Kubernetes request to mount or unmount the ESSD cloud disk, the rpc.py script is called within the Pod via the NodeServer component, and the nvmf_tgt service on the host machine is interacted with based on the spdk-rpc command to mount or unmount the block device BDEV.

[0094] This embodiment describes the specific mount / unmount execution process. When the NodeServer component receives a mount / unmount request from Kubernetes, it utilizes the "control channel" provided in step S530 to directly call the SPDK RPC script (rpc.py) within the Pod. This interacts with the nvmf_tgt service on the host machine using spdk-rpc commands. This enables the mount or unmount operation of the local block device (BDEV), ultimately completing the entire process of delivering storage resources to applications within the Pod.

[0095] The implementation combines SPDK user-space driver and NVMe-oFoverRDMA technology to eliminate the overhead of traditional kernel storage stack. At the same time, it effectively solves the problem of interaction between CSI node components and host services by mounting sock files, and finally realizes the containerized and efficient use of high-performance cloud disks.

[0096] Based on the above embodiments, in another embodiment provided in this application, when communicating with the ESSD cluster based on the gRPC protocol, the above step S510 may further include the following steps: In step S511, the gRPC service interface is defined using Protocol Buffers.

[0097] In this embodiment, the communication protocol between the client (CSI ControllerServer) and the server (ESSD cluster management API) is explicitly defined using Protocol Buffers (an interface definition language, IDL) developed by Google. This includes specifying the names of callable Remote Procedure Call (RPC) methods, the format of method request parameters, and the format of return responses. This step ensures that both communicating parties have a clear and consistent understanding of the data structures and service methods, which forms the basis for subsequent automated code generation.

[0098] In step S512, client code is generated based on the gRPC service interface, and the generated client code is used to call the volume management service API provided by the ESSD cluster.

[0099] This example utilizes the compilation tools provided by the gRPC framework to automatically generate strongly typed client code (typically in languages ​​like Go or Java) from the .proto interface file defined in the previous step. This eliminates the need to manually write tedious network communication and serialization code, significantly reducing development workload and eliminating human error. Subsequently, the Provisioner component directly uses this automatically generated, type-safe client code to call the specific volume management APIs (such as CreateVolume and DeleteVolume) provided by the ESSD cluster.

[0100] In step S513, communication is established with the ESSD cluster based on the Volume Management Service API.

[0101] Based on the standardized client and defined interface generated in the first two steps, the CSI plugin establishes an efficient and reliable communication channel with the ESSD cluster. All lifecycle management operations (creating, deleting, or expanding volumes) commands and data are transmitted over the network in serialized Protocol Buffers binary format through this channel, ultimately completing the interaction with the backend storage system.

[0102] Therefore, the implementation ensures contract consistency by defining interfaces and using IDL-based design-first approach. Code generation is automated using tools to guarantee correctness and development efficiency. Furthermore, by executing calls, type-safe remote calls can be performed based on the generated code to achieve functionality.

[0103] Based on the above embodiments, in another embodiment provided in this application, during the process of running the SPDK nvmf_tgt service on the host machine and interacting with the ESSD cloud disks in the ESSD cluster through the NVMe-oF protocol and RDMA network, the above step S520 may further include the following steps: Step S521: Load the NVMe-oF protocol and RDMA driver module, and start the nvmf_tgt service.

[0104] On the host machine, the necessary kernel or user-space driver modules are first loaded to prepare the hardware and software environment for using the NVMe-oF protocol and RDMA network. Then, the SPDK's nvmf_tgt service is started. This service is a high-performance user-space target program that acts as the NVMe-oF Initiator, responsible for managing all block device operations performed via the NVMe-oF protocol. This lays the foundation for establishing a high-speed data channel later.

[0105] Step S522: Discover the remote ESSD cloud disk through the RDMA network and establish a connection with the ESSD cloud disk.

[0106] This implementation establishes an end-to-end remote storage access channel. The nvmf_tgt service proactively discovers NVMe-oF targets exposed over the network within the ESSD cluster via a configured RDMA network (such as RoCEv2). Upon discovery, the initiator establishes a connection and performs an NVMe-oF protocol handshake. Once the connection is successful, the remote ESSD cloud disk appears as a standard block device (BDEV) on the local host machine (e.g., / dev / nvme1n1). This process achieves localized abstraction of remote storage, providing objects for subsequent mounting operations.

[0107] This embodiment, by preparing the engine and establishing channels, can configure the environment and start a high-performance user-space service (nvmf_tgt), discover and connect to remote storage resources via RDMA network, and map them as local devices. Compared to traditional methods using kernel-space drivers (such as NVMe-of-TCP kernel drivers) or general network protocols (such as iSCSI), this embodiment utilizes the advanced technology stack of SPDK and NVMe-of-F over RDMA to build data pathways, providing a technical foundation for achieving ultra-low latency and high IOPS performance goals.

[0108] In the embodiments provided in this application, before running the SPDK nvmf_tgt service on the host machine, SPDK can also be configured to use large page memory and bind to a specific CPU core.

[0109] Specifically, in this embodiment, before starting the SPDK's nvmf_tgt service, key configurations of its runtime environment can be performed first: Configure Hugepages: Instruct SPDK to use a large page memory pool instead of the traditional 4KB memory pages. This greatly reduces the number of cache misses in the CPU's TLB (Translation Lookaside Buffer), thereby significantly reducing memory access latency, which is crucial for handling high-frequency I / O requests.

[0110] CPU Pinning: Binds the nvmf_tgt service process or its polling thread to one or more specified CPU physical cores. This avoids the cache invalidation and context switching overhead caused by process scheduling and switching between different cores, ensuring the stability of packet processing and extremely low latency.

[0111] Based on the above embodiments, in another embodiment provided in this application, the method may further include the following steps: In step S550, after creating or deleting the ESSD cloud disk, the volume information of the ESSD cloud disk is persistently stored in a JSON file.

[0112] In this embodiment, whenever the ControllerServer successfully completes an operation that changes the management status of a cloud disk (such as creation or deletion), it immediately writes the latest metadata (such as volume ID, capacity, and status) of the volume involved in the operation to a designated file on the host disk in JSON format. This embodiment ensures that any critical status changes are recorded promptly and persistently on non-volatile storage media, ensuring that status changes are securely saved and thus avoiding memory data loss due to unexpected crashes of plugin processes.

[0113] In step S560, when the Provisioner component or Node component starts, volume information is loaded from a JSON file to restore the CSI plugin's management state of the cloud disk.

[0114] In this embodiment, when the Provisioner or Node component (typically due to a restart or scheduling migration because it runs as a Pod) restarts, it first reads and parses all previously persisted volume information from a JSON file on disk before processing any new requests. Using this information, the component can completely reconstruct its management state (e.g., the essd.volumes dictionary) in memory, thus "remembering" all existing volumes and their attributes. This allows the component to seamlessly take over subsequent management tasks, presenting itself as a reliable service with state recovery capabilities. This ensures service continuity and guarantees that the component can quickly resume operation after a restart.

[0115] Therefore, the implementation can achieve persistence and high availability of CSI plugin management state without relying on any external databases (such as ETCD or MySQL), solely through the file system functions of the operating system itself and application layer encoding (JSON serialization / deserialization). This greatly simplifies the complexity of the system architecture and reduces operation and maintenance costs, while ensuring business continuity.

[0116] Based on the above embodiments, in another embodiment provided in this application, before mounting or unmounting the block device BDEV, the method further includes the following steps: In step S570, the target file system type parameter is received through the NodeServer component.

[0117] This implementation provides an entry point for personalized storage configuration. Before performing the mount operation, the NodeServer component first parses and receives the target file system type (such as xfs, ext4) explicitly specified by the user from the request parameters passed from Kubernetes. This parameter typically originates from the PersistentVolumeClaim (PVC) configuration, reflecting the characteristic of flexibly providing storage services according to application needs.

[0118] The implementation examples demonstrate user-configurable flexibility, enabling them to meet the diverse performance and data management needs of different applications for the file system.

[0119] In step S580, the NodeServer component invokes system commands corresponding to the target file system type on the block device BDEV to autonomously create a file system. The file system can be of type xfs or ext4.

[0120] After obtaining the file system type parameters, the NodeServer component directly calls the corresponding system-level formatting command (such as mkfs.xfs or mkfs.ext4) on the locally mapped block device (BDEV) to create (format) the specified file system. This allows the implementation to operate independently of the backend storage system or the pre-configured file system of the host operating system, providing great flexibility and broad compatibility. The implementation demonstrates environment-independent independence, ensuring that the plugin can autonomously complete the final preparation of the storage medium regardless of the underlying environment.

[0121] The implementation uses standard system commands to build the required file system on demand on a general block device, completely decoupling it from specific backend storage or host environments.

[0122] By dividing each function into corresponding functional modules, this application provides a container-based cloud disk management device, which can be a server, a terminal, or a chip applied to a server. Figure 6 This is a schematic block diagram of the functional modules of a container-based cloud disk management device provided for an exemplary embodiment of this application. Figure 6 As shown, the container-based cloud disk management device includes: The management module 61 is used to communicate with the ESSD cluster via the ControllerServer component of the CSI plugin based on the gRPC protocol, and to manage the lifecycle of ESSD cloud disks by creating, deleting or expanding them in response to persistent volume declaration (PVC) requests from Kubernetes. Interaction module 62 is used to run the nvmf_tgt service of SPDK on the host machine and interact with the ESSD cloud disk in the ESSD cluster through the NVMe-oF protocol and RDMA network to create a block device BDEV corresponding to the ESSD cloud disk locally. Mounting module 63 is used to mount the directory containing the SPDK socket file spdk.sock and the SPDK remote procedure call script rpc.py on the host machine to the Pod of the NodeServer component running the CSI plugin; The processing module 64 is used to, when receiving a Kubernetes request to mount or unmount the ESSD cloud disk, call the rpc.py script in the Pod through the NodeServer component, and interact with the nvmf_tgt service of the host machine based on the spdk-rpc command to mount or unmount the block device BDEV.

[0123] In another embodiment provided in this application, the management module 61 is specifically used for: Define gRPC service interfaces using Protocol Buffers; The client code is generated based on the gRPC service interface, and the generated client code is used to call the volume management service API provided by the ESSD cluster. Communicate with the ESSD cluster based on the volume management service API.

[0124] In another embodiment provided in this application, the interaction module 62 is specifically used for: Load the NVMe-oF protocol and RDMA driver modules, and start the nvmf_tgt service; Discover remote ESSD cloud disks via RDMA network and establish a connection with the ESSD cloud disks.

[0125] In yet another embodiment provided in this application, the apparatus further includes: The configuration module is used to configure SPDK to use massive page memory and bind to specific CPU cores.

[0126] In yet another embodiment provided in this application, the apparatus further includes: The storage module is used to persistently store the volume information of the ESSD cloud disk in a JSON file after the creation or deletion operation of the ESSD cloud disk is performed. The recovery module is used to load the volume information from the JSON file when the Provisioner component or Node component starts, so as to restore the management state of the cloud disk by the CSI plugin.

[0127] In yet another embodiment provided in this application, the apparatus further includes: The parameter receiving module is used to receive parameters of the target file system type through the NodeServer component; The file system creation module is used to autonomously create a file system by calling system commands corresponding to the target file system type on the block device BDEV through the NodeServer component; wherein the file system includes xfs or ext4 type.

[0128] This application also provides an electronic device, including: at least one processor; a memory for storing executable instructions of the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the method disclosed in the embodiments of this application.

[0129] Figure 7 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this application. For example... Figure 7 As shown, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801. The processor 1801 can perform the corresponding steps in the methods disclosed in the embodiments of this application.

[0130] The processor 1801 described above can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this application can be implemented by the integrated logic circuitry in the hardware of the processor 1801 or by instructions in software form. The processor 1801 can be a general-purpose processor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 1802, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 1801 reads information from the memory 1802 and, in conjunction with its hardware, completes the steps of the above method.

[0131] Furthermore, the various operations / processes according to this application, when implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 8 The computer system 1900 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including those described above. Figure 8 A structural block diagram of a computer system provided for an exemplary embodiment of this application.

[0132] Computer System 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present application described and / or claimed herein.

[0133] like Figure 8As shown, the computer system 1900 includes a computing unit 1901, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 into a random access memory (RAM) 1903. The RAM 1903 may also store various programs and data required for the operation of the computer system 1900. The computing unit 1901, ROM 1902, and RAM 1903 are interconnected via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.

[0134] Multiple components in computer system 1900 are connected to I / O interface 1905, including: input unit 1906, output unit 1907, storage unit 1908, and communication unit 1909. Input unit 1906 can be any type of device capable of inputting information into computer system 1900. Input unit 1906 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 1907 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 1908 may include, but is not limited to, hard disks and optical disks. Communication unit 1909 allows computer system 1900 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0135] The computing unit 1901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in the embodiments of this application can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 1908. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 1902 and / or communication unit 1909. In some embodiments, the computing unit 1901 can be configured to perform the methods disclosed in the embodiments of this application by any other suitable means (e.g., by means of firmware).

[0136] This application also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this application.

[0137] The computer-readable storage medium in this application embodiment may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

[0138] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0139] This application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the methods disclosed in the embodiments of this application.

[0140] In embodiments of this application, computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0142] The modules, components, or units described in the embodiments of this application can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0143] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0144] The above description is merely an embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this application.

[0145] While specific embodiments of this application have been described in detail by way of examples, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this application. The scope of this application is defined by the appended claims.

Claims

1. A container-based cloud disk management method, characterized in that, The method includes: The ControllerServer component of the Container Storage Interface (CSI) plugin communicates with the Enhanced Cloud Disk (ESSD) cluster based on the Google Remote Procedure Call (gRPC) protocol and manages the ESSD cloud disks in response to persistent volume declaration (PVC) requests from Kubernetes. The nvmf_tgt service of the Storage Performance Development Kit (SPDK) is run on the host machine. It interacts with the ESSD cloud disks in the ESSD cluster through the network-based NVMe storage access protocol NVMe-oF and the Remote Direct Memory Access (RDMA) network to create block devices (BDEVs) corresponding to the ESSD cloud disks locally. Mount the directory containing the SPDK socket file spdk.sock and the SPDK remote procedure call script rpc.py on the host machine to the Pod of the NodeServer component running the CSI plugin; When a Kubernetes request to mount or unmount the ESSD cloud disk is received, the NodeServer component calls the rpc.py script within the Pod and interacts with the nvmf_tgt service of the host machine based on the spdk-rpc command to mount or unmount the block device BDEV.

2. The method according to claim 1, characterized in that, The communication with the ESSD cluster based on the gRPC protocol includes: Define gRPC service interfaces using Protocol Buffers; The client code is generated based on the gRPC service interface, and the generated client code is used to call the volume management service API provided by the ESSD cluster. Communicate with the ESSD cluster based on the volume management service API.

3. The method according to claim 1, characterized in that, The nvmf_tgt service running SPDK on the host machine interacts with the ESSD cloud disks in the ESSD cluster via the NVMe-oF protocol and RDMA network, including: Load the NVMe-oF protocol and RDMA driver modules, and start the nvmf_tgt service; Discover remote ESSD cloud disks via RDMA network and establish a connection with the ESSD cloud disks.

4. The method according to claim 1, characterized in that, Prior to running the SPDK nvmf_tgt service on the host machine, the method further includes: Configure SPDK to use large page memory and bind to a specific CPU core.

5. The method according to claim 1, characterized in that, The method further includes: After the creation or deletion operation of the ESSD cloud disk is performed, the volume information of the ESSD cloud disk is persistently stored in a JSON file. When the Provisioner component or Node component starts, the volume information is loaded from the JSON file to restore the CSI plugin's management status over the cloud disk.

6. The method according to claim 1, characterized in that, Before performing the mounting or unmounting operation on the block device BDEV, the method further includes: The NodeServer component receives parameters of the target file system type. The NodeServer component, on the block device BDEV, invokes system commands corresponding to the target file system type to autonomously create a file system.

7. The method according to claim 6, characterized in that, The file system includes xfs or ext4 types.

8. The method according to claim 1, characterized in that, The management of ESSD cloud disks includes: The lifecycle management of ESSD cloud disks includes creation, deletion, and expansion.

9. A container-based cloud disk management device, characterized in that, The device includes: The management module is used to communicate with the ESSD cluster via the ControllerServer component of the CSI plugin based on the gRPC protocol and to manage ESSD cloud disks in response to persistent volume declaration (PVC) requests from Kubernetes. The interaction module is used to run the nvmf_tgt service of SPDK on the host machine and interact with the ESSD cloud disks in the ESSD cluster through the NVMe-oF protocol and RDMA network to create block devices BDEV corresponding to the ESSD cloud disks locally. The mounting module is used to mount the directory containing the SPDK socket file spdk.sock and the SPDK remote procedure call script rpc.py on the host machine to the Pod of the NodeServer component running the CSI plugin; The processing module is used to, when receiving a Kubernetes request to mount or unmount the ESSD cloud disk, call the rpc.py script in the Pod through the NodeServer component, and interact with the nvmf_tgt service of the host machine based on the spdk-rpc command to mount or unmount the block device BDEV.

10. An electronic device, characterized in that, include: At least one processor; Memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images