A method and device for safely using distributed storage for bare metal

By establishing cache queues and persistent queues on bare metal servers, and using SPDK and Linux AIO technology, the security and performance problems of bare metal servers when using distributed storage are solved, achieving efficient and secure storage solutions and automatic recovery functions.

CN114610238BActive Publication Date: 2025-04-08SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210246086.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-14
Publication Date
2025-04-08
Estimated Expiration
2042-03-14

AI Technical Summary

Technical Problem

When existing bare metal servers use distributed storage, it is difficult to meet the security and high performance requirements, which poses security risks and long IO paths.

Method used

The controller module receives bare metal connection Ceph storage requests, establishes a cache queue and a persistent queue. The intelligent network card module is based on SPDK and Linux AIO technology to establish a data read and write channel from rbd block devices to bare metal, realizing network isolation and direct mapping of storage protocols.

Benefits of technology

It improves the availability and storage architecture uniformity of bare metal servers, provides network isolation and security, shortens data IO paths, improves the usage performance of distributed storage volumes, and realizes capacitance and automatic recovery.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114610238B_ABST
    Figure CN114610238B_ABST
Patent Text Reader

Abstract

The present invention provides a method and device for bare metal to safely use distributed storage. The method for bare metal to safely use distributed storage includes: receiving a bare metal connection Ceph storage request, establishing a cache queue Q(v), initializing a persistent queue Q(p), judging the status of an agent module, executing a command by an intelligent network card module, mapping an rbd block device to AlO bdev, connecting the AlO bdev to a virtual PC1 device, judging a command execution result, returning a volume identifier and a device index number, completing the connection to the distributed storage, and returning a volume identifier and an empty index number; the device and method provide distributed storage support represented by Ceph, improve the availability of bare metal servers, and the bare metal does not need to be connected to a proprietary storage network, thereby improving the security of a management platform and realizing network isolation between a management side and a tenant side; and after the bare metal is abnormally shut down, it can be ensured that the Ceph storage disk is automatically mounted after restarting, and the drive letter sequence is ensured to be correct.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field related to cloud computing, and in particular to a method and device for securely using distributed storage for bare metal. Background Art

[0002] With the development of cloud computing, more and more usage scenarios require bare metal servers, such as OracleRack, building private clouds, container bare metal clouds, etc. In cloud computing scenarios, bare metal servers are provided to tenants exclusively, and remote storage services are provided for bare metal through distributed storage. However, bare metal servers are limited by hardware flexibility and network security. Existing bare metal solutions using distributed storage cannot meet the requirements of security and high performance, such as:

[0003] The existing solution 1 directly maps the rbd image to the rbd block device in the bare metal through rbd image mapping. This solution is the simplest and most direct, and is suitable for mounting Ceph storage on servers within the platform. However, this solution requires pre-configuration of Ceph-related configurations and requires connecting the server to the storage network. Therefore, there are security risks in using the bare metal solution 1;

[0004] The existing solution is the iSCSI-gateway solution mentioned in "A method for physical bare metal to support Ceph backend volumes". It converts Ceph storage volumes into iSCSI network disks through storage protocols, opens up a dedicated storage network for bare metal, and connects the converted iSCSI network disks to bare metal. This solution has problems such as long IO paths and increased latency due to storage protocol conversion. In addition, it is not suitable for public cloud scenarios to provide a dedicated storage network for a single tenant. Summary of the invention

[0005] The embodiments of the present invention provide a method and device for bare metal to safely use distributed storage, which can improve the security of the management platform and achieve network isolation between the management side and the tenant side.

[0006] A method and device for securely using distributed storage for bare metal includes:

[0007] Receive bare metal connection Ceph storage request;

[0008] Establish a cache queue Q(v);

[0009] Initialize the persistent queue Q(p);

[0010] Determine the status of the proxy module;

[0011] The smart network card module executes commands;

[0012] After mapping the rbd block device to AlObdev, AlObdev is connected to the virtual PCl device;

[0013] Determine the command execution result;

[0014] Returns the volume ID and device index number;

[0015] Connection to distributed storage is complete;

[0016] Returns the volume ID and a null index number.

[0017] Optionally,

[0018] In the process of receiving the bare metal connection Ceph storage request, its controller module receives n distributed storage connection requests, takes out the volume identifier v and puts it into the message cache queue Q(v)={v1,v2,v3,…,vn}, where the identifier v is composed of the storage volume identifier. The message cache queue is used to solve the delay problem of connecting to the Ceph storage volume; the storage volume mounted to the bare metal is put into the persistence queue Q(p)={p1,p2,p3,…,pn} with the identifier p, where the identifier p is composed of the storage volume identifier and the device index number. The device index number is generated after the entire mounting process is completed. The persistence queue is used to solve the problem of storage volume loss caused by module power failure.

[0019] Optionally,

[0020] In the process of establishing the cache queue Q(v), it periodically polls the status of the proxy module, obtains the information reported by the proxy module and determines the status of the proxy module. When an abnormal state occurs, the subsequent specific process is not executed, and the identifier v of the cache queue Q(v) is taken out instead. The identifier v and the empty device index number form the identifier p and are placed in the persistent queue Q(p), and its controller module sequentially dequeues the identifier v[n]. It must dequeue v[n] after obtaining the v[n-1] response message to ensure the sequential consistency of the mount request and the corresponding storage volume.

[0021] Optionally,

[0022] During the process of initializing the persistent queue Q(p), its proxy module verifies the command issued by the controller module, checks the command format, checks the number of mounted rbd block devices, and checks whether the rbd block device to be uninstalled exists; if the command is successfully executed, a mount index number is generated and combined with the volume identifier to form an identifier p, which is placed in the persistent queue Q(p) of the controller module; if the command verification or execution fails, a failure message is directly responded to the controller module.

[0023] Optionally,

[0024] During the execution of the command by the smart network card module, after the verification is correct, the proxy module mounts the rbd image to the smart network card module, and the operating system of the smart network card module loads the rbd kernel driver, maps the rbd image to a local block device, such as / dev / rbd0, and then the smart network card module establishes a data channel from SPDK bdev (Backend Device) to the rbd block device. Based on Linux AIO (Asynchronous IO), a mapping relationship between the SPDK AIO bdev device and the kernel block device / dev / rbd0 is established, and asynchronous IO data reading and writing is performed through the thread's IO channel.

[0025] Optionally,

[0026] After mapping the rbd block device to AlObdev, during the process of AlObdev connecting to the virtual PCl device, its smart network card module establishes a data channel from SPDK AIO bdev to bare metal. Initialize the virtual PCI device, write VIRTIO_F_IOMMU_PLATFORM, VIRTIO_VENDOR_ID, and VIRTIO_DEVICE_ID identifiers to the virtio PCI configuration space, and connect to the hardware PCI bridge of the smart network card module; connect the SPDK AIO bdev device to the virtual PCI device, load the PCI device driver on the bare metal, and read and write data from the SPDK AIO bdev through DMA (Direct Memory Access).

[0027] Optionally,

[0028] In the process of returning the volume identifier and device index number, the mount index number is generated after the mounting is completed, and a success message is responded to the controller. After receiving the message, the controller combines the storage volume identifier and the mount device index number into an identifier p[n] and puts it into the persistent queue Q(p).

[0029] Optionally,

[0030] The device for bare metal securely using distributed storage includes:

[0031] Controller module;

[0032] An agent module, which cooperates with the controller module to implement scheduling and management of the smart network card module;

[0033] Smart network card module;

[0034] The bare metal module cooperates with the smart network card module to establish and control the IO channel of the storage data plane.

[0035] Optionally,

[0036] The use of the bare metal security distributed storage device includes a distributed storage connection process, and a fault-tolerant processing and automatic recovery process when the smart network card module loses power.

[0037] Optionally, the controller module is responsible for receiving a bare metal connection Ceph storage request, establishing a message cache queue Q(v) and a persistence queue Q(p), and periodically polling the proxy module status;

[0038] The proxy module is responsible for receiving commands from the controller module, reporting its own status to the controller, pre-checking the command format and returning the command execution result, and mounting the rbd image to the smart network card module;

[0039] The smart network card module is responsible for storage data plane processing, receiving commands from the proxy module, and establishing a data channel from the rbd block device to the bare metal;

[0040] The bare metal module is responsible for maintaining the power supply of the device and providing tenants with access.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] In the embodiment of the present invention, the device and method provide distributed storage support represented by Ceph, improve the availability of bare metal servers, and unify the storage architecture of virtual machines and bare metal; and compared with directly mounting rbd block devices, the device provides network isolation and security, and users can only see virtio devices in bare metal servers; compared with the iSCSI-gateway solution for mounting Ceph distributed storage, the device does not require iSCSI gateway to convert storage protocols, shortens the data IO path, and improves the performance of distributed storage volumes; and its controller module manages bare metal connected distributed storage through cache queues and persistent queues, provides capacitor error capability, and solves the time delay problem of storage connection process; and the smart network card module is based on SPDK and Linux AIO technology establishes a data read and write channel from the rbd block device to the bare metal. This solution can be expanded to support multiple storage backend types (such as NVME-of), providing a general architecture for bare metal to support distributed storage; and its bare metal does not need to be connected to a proprietary storage network, thereby improving the security of the management platform and achieving network isolation between the management side and the tenant side; and after the bare metal is abnormally shut down, it can ensure that the Ceph storage disk is automatically mounted after restarting, and the drive letter order is correct. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 is a diagram of a device for bare metal supporting Ceph distributed storage according to an embodiment of the present invention;

[0045] Figure 2 It is a flow chart of bare metal connection Ceph distributed storage and capacitor loss error processing according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0047] See also Figure 1 The present invention provides a technical solution: a method and device for bare metal to safely use distributed storage. A method for bare metal to safely use distributed storage includes:

[0048] Receive bare metal connection Ceph storage request;

[0049] Establish a cache queue Q(v);

[0050] Initialize the persistent queue Q(p);

[0051] Determine the status of the proxy module;

[0052] The smart network card module executes commands;

[0053] After mapping the rbd block device to AlObdev, AlObdev is connected to the virtual PCl device;

[0054] Determine the command execution result;

[0055] Returns the volume ID and device index number;

[0056] Connection to distributed storage is complete;

[0057] Returns the volume ID and a null index number.

[0058] The controller module receives n distributed storage connection requests, takes out the volume identifier v and puts it into the message cache queue Q(v) = {v1, v2, v3, ..., vn}, where the identifier v is composed of the storage volume identifier. The message cache queue is used to solve the delay problem of connecting to the Ceph storage volume. The storage volume mounted to the bare metal is put into the persistence queue Q(p) = {p1, p2, p3, ..., pn} with the identifier p. The identifier p is composed of the storage volume identifier and the device index number. The device index number is generated after the entire mounting process is completed. The persistence queue is used to solve the problem of storage volume loss caused by module power failure.

[0059] Poll the proxy module status regularly, obtain the proxy module report information and judge the proxy module status. When the status is abnormal, do not execute the subsequent specific process, instead take out the identifier v of the cache queue Q(v), and the identifier v and the empty device index number form the identifier p, and put it into the persistent queue Q(p).

[0060] The controller module dequeues the identifier v[n] in turn. It must dequeue v[n] after obtaining the v[n-1] response message to ensure the sequential consistency of the mount request and the corresponding storage volume.

[0061] The proxy module verifies the command issued by the controller module, checks the command format, checks the number of mounted rbd block devices, and checks whether the rbd block device to be uninstalled exists; if the command is successfully executed, a mount index number is generated and combined with the volume identifier to form an identifier p, which is placed in the controller module's persistent queue Q(p); if the command verification or execution fails, a failure message is directly responded to the controller module.

[0062] After verification is correct, the proxy module mounts the rbd image to the smart network card module. The operating system of the smart network card module loads the rbd kernel driver and maps the rbd image to a local block device, such as / dev / rbd0.

[0063] The smart network card module establishes a data channel from SPDK bdev (Backend Device) to rbd block device. Based on Linux AIO (Asynchronous IO), a mapping relationship between SPDK AIO bdev device and kernel block device / dev / rbd0 is established, and asynchronous IO data reading and writing is performed through the thread's IO channel.

[0064] The SmartNIC module establishes a data channel from SPDK AIO bdev to bare metal. Initialize the virtual PCI device, write VIRTIO_F_IOMMU_PLATFORM, VIRTIO_VENDOR_ID, and VIRTIO_DEVICE_ID identifiers to the virtio PCI configuration space, and connect to the hardware PCI bridge of the SmartNIC module; connect the SPDK AIO bdev device to the virtual PCI device, load the PCI device driver on bare metal, and read and write data from the SPDK AIO bdev through DMA (Direct Memory Access).

[0065] After the mount is completed, a mount index number is generated and a success message is responded to the controller. After receiving the message, the controller combines the storage volume identifier and the mount device index number into an identifier p[n] and puts it into the persistent queue Q(p).

[0066] The present invention provides a technical solution: a method and device for bare metal to safely use distributed storage, wherein the bare metal device for safely using distributed storage includes: a controller module, an agent module, an intelligent network card module, and a bare metal module. The controller cooperates with the agent module to implement scheduling and management of the intelligent network card module, and the intelligent network card module cooperates with the bare metal module to implement the establishment and control of the IO channel of the storage data plane. The use of the device is divided into two aspects, including the specific process of distributed storage connection, and the fault-tolerant processing and automatic recovery process when the intelligent network card module loses power.

[0067] The controller module is responsible for receiving bare metal connection Ceph storage requests, establishing message cache queue Q(v) and persistence queue Q(p), and periodically polling the agent module status;

[0068] The proxy module is responsible for receiving commands from the controller module, reporting its own status to the controller, pre-checking the command format and returning the command execution result, and mounting the rbd image to the smart network card module;

[0069] The smart network card module is responsible for storage data plane processing, receiving commands from the proxy module, and establishing a data channel from the rbd block device to the bare metal;

[0070] The bare metal module is responsible for maintaining the power supply of the device and providing tenants with access.

[0071] The device usage process includes:

[0072] The controller receives n distributed storage connection requests, takes out the volume identifier v and puts it into the message cache queue Q(v) = {v1, v2, v3, ..., vn}, where the identifier v is composed of the storage volume identifier. The message cache queue is used to solve the delay problem of connecting to the Ceph storage volume. The storage volume mounted to the bare metal is put into the persistence queue Q(p) = {p1, p2, p3, ..., pn} with the identifier p. The identifier p is composed of the storage volume identifier and the device index number. The device index number is generated after the entire mounting process is completed. The persistence queue is used to solve the problem of storage volume loss caused by module power failure.

[0073] Poll the proxy module status regularly, obtain the proxy module report information and judge the proxy module status. When the status is abnormal, do not execute the subsequent specific process, instead take out the identifier v of the cache queue Q(v), and the identifier v and the empty device index number form the identifier p, and put it into the persistent queue Q(p).

[0074] The controller module dequeues the identifier v[n] in turn. It must dequeue v[n] after obtaining the v[n-1] response message to ensure the sequential consistency of the mount request and the corresponding storage volume.

[0075] The proxy module verifies the command issued by the controller module, checks the command format, checks the number of mounted rbd block devices, and checks whether the rbd block device to be uninstalled exists; if the command is successfully executed, a mount index number is generated and combined with the volume identifier to form an identifier p, which is placed in the controller module's persistent queue Q(p); if the command verification or execution fails, a failure message is directly responded to the controller module.

[0076] After verification is correct, the proxy module mounts the rbd image to the smart network card module. The operating system of the smart network card module loads the rbd kernel driver and maps the rbd image to a local block device, such as / dev / rbd0.

[0077] The smart network card module establishes a data channel from SPDK bdev (Backend Device) to rbd block device. Based on Linux AIO (Asynchronous IO), a mapping relationship between SPDK AIO bdev device and kernel block device / dev / rbd0 is established, and asynchronous IO data reading and writing is performed through the thread's IO channel.

[0078] The SmartNIC module establishes a data channel from SPDK AIO bdev to bare metal. Initialize the virtual PCI device, write VIRTIO_F_IOMMU_PLATFORM, VIRTIO_VENDOR_ID, and VIRTIO_DEVICE_ID identifiers to the virtio PCI configuration space, and connect to the hardware PCI bridge of the SmartNIC module; connect the SPDK AIO bdev device to the virtual PCI device, load the PCI device driver on bare metal, and read and write data from the SPDK AIO bdev through DMA (Direct Memory Access).

[0079] After the mount is completed, a mount index number is generated and a success message is responded to the controller. After receiving the message, the controller combines the storage volume identifier and the mount device index number into an identifier p[n] and puts it into the persistent queue Q(p).

[0080] Capacitor drop error and automatic recovery process:

[0081] The smart NIC module is powered by the bare metal module with 12V. When the bare metal server is shut down, the smart NIC module will lose power, resulting in the loss of mounted rbd block device information. When manually remounting, the drive letter sequence will be messed up. At the same time, power failure will cause the proxy module to be in an abnormal state, causing the workflow being executed to fail. Therefore, it is necessary to consider capacitor failure error handling.

[0082] The bare metal module is shut down, causing the smart network card module to lose power. According to the status of the proxy module, the identifier v[n] of Q(v) in the cache queue is dequeued in sequence, without executing the specific task process, and directly put into the persistent queue Q(p);

[0083] When the bare metal module resumes power supply, the proxy module determines that the status is normal. It takes out the elements of the persistent queue Q(p) and puts them at the head of the cache queue Q(v). It then sends the corresponding mount commands to the proxy module in the order of the persistent queue identifier p[n], identifier p[n+1], and cache queue identifier v[n], and regenerates the mount index number to ensure that the storage volume is automatically mounted after the bare metal is powered on and the drive letter order is kept correct.

[0084] The present invention provides distributed storage support represented by Ceph, improves the availability of bare metal servers, and unifies the storage architecture of virtual machines and bare metal; and compared with directly mounting rbd block devices, the device provides network isolation and security, and users can only see virtio devices in bare metal servers; compared with the iSCSI-gateway solution for mounting Ceph distributed storage, the device does not require iSCSI gateway to convert storage protocols, shortens the data IO path, and improves the performance of distributed storage volumes; and its controller module manages bare metal connected distributed storage through cache queues and persistent queues, provides capacitor error capability, and solves the time delay problem of storage connection process; and the smart network card module is based on SPDK and Linux AIO technology establishes a data read and write channel from the rbd block device to the bare metal. This solution can be expanded to support multiple storage backend types (such as NVME-of), providing a general architecture for bare metal to support distributed storage; and its bare metal does not need to be connected to a proprietary storage network, thereby improving the security of the management platform and achieving network isolation between the management side and the tenant side; and after the bare metal is abnormally shut down, it can ensure that the Ceph storage disk is automatically mounted after restarting, and the drive letter order is correct.

[0085] The information interaction, execution process and other contents between the units in the above-mentioned device are based on the same concept as the embodiment of the method of the present invention. For specific contents, please refer to the description in the embodiment of the method of the present invention, and no further description is given here.

[0086] The present invention also provides a rights metadata distributed initialization device, which stores instructions for causing a computer to execute the bare metal security distributed storage method as described herein. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0087] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0088] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0089] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0090] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0091] It should be noted that not all steps and modules in the above-mentioned processes and system structure diagrams are necessary, and some steps or modules can be ignored according to actual needs. The execution order of each step is not fixed and can be adjusted as needed. The system structure described in the above-mentioned embodiments can be a physical structure or a logical structure, that is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or some components in multiple independent devices may be implemented together.

[0092] In the above embodiments, the hardware unit can be realized by mechanical means or electrical means. For example, a hardware unit can include permanent dedicated circuits or logic (such as special processors, FPGA or ASIC) to complete the corresponding operation. The hardware unit can also include programmable logic or circuits (such as general-purpose processors or other programmable processors), which can be temporarily set by software to complete the corresponding operation. Concrete implementation method (mechanical means or dedicated permanent circuits or temporarily set circuits) can be determined based on cost and time considerations.

[0093] The present invention is shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A method for bare metal to safely use distributed storage, characterized in that: The method for bare metal to safely use distributed storage includes: Receive bare metal connection Ceph storage request; Establish a cache queue Q(v); Initialize the persistent queue Q(p); Determine the status of the proxy module; The smart network card module executes commands; After mapping the rbd block device to AlObdev, AlObdev is connected to the virtual PCl device; Determine the command execution result; Returns the volume ID and device index number; Connection to distributed storage is complete; Returns the volume ID and a null index number.

2. A method for bare metal secure use of distributed storage according to claim 1, characterized in that: In the process of receiving the bare metal connection Ceph storage request, its controller module receives n distributed storage connection requests, takes out the volume identifier v and puts it into the message cache queue Q(v)={v1,v2,v3,…,vn}, where the identifier v is composed of the storage volume identifier. The message cache queue is used to solve the delay problem of connecting to the Ceph storage volume; the storage volume mounted to the bare metal is put into the persistence queue Q(p)={p1,p2,p3,…,pn} with the identifier p, where the identifier p is composed of the storage volume identifier and the device index number. The device index number is generated after the entire mounting process is completed. The persistence queue is used to solve the problem of storage volume loss caused by module power failure.

3. A method for bare metal secure use of distributed storage according to claim 1, characterized in that: During the process of establishing the cache queue Q(v), it periodically polls the status of the proxy module, obtains the information reported by the proxy module and determines the status of the proxy module; When an abnormal state occurs, the subsequent specific process is not executed. Instead, the identifier v of the cache queue Q(v) is taken out, and the identifier v and the empty device index number form the identifier p and are placed in the persistent queue Q(p). The controller module dequeues the identifier v[n] in turn, and v[n] must be dequeued after obtaining the v[n-1] response message to ensure the sequential consistency of the mount request and the corresponding storage volume.

4. A method for bare metal secure use of distributed storage according to claim 1, characterized in that: During the process of initializing the persistent queue Q(p), its proxy module verifies the command issued by the controller module, checks the command format, checks the number of mounted rbd block devices, and checks whether the rbd block device to be uninstalled exists; if the command is successfully executed, a mount index number is generated and combined with the volume identifier to form an identifier p, which is placed in the persistent queue Q(p) of the controller module; if the command verification or execution fails, a failure message is directly responded to the controller module.

5. The method for bare metal secure use of distributed storage according to claim 1, characterized in that: During the execution of the command by the smart network card module, after the verification is correct, the proxy module mounts the rbd image to the smart network card module, the operating system of the smart network card module loads the rbd kernel driver, maps the rbd image to a local block device, and then the smart network card module establishes a data channel from SPDK bdev to the rbd block device; Based on Linux AIO, a mapping relationship between the SPDK AIO bdev device and the kernel block device / dev / rbd0 is established, and asynchronous IO data reading and writing is performed through the thread's IO channel.

6. A method for bare metal secure use of distributed storage according to claim 1, characterized in that: After mapping the rbd block device to AlObdev, during the process of AlObdev connecting to the virtual PC1 device, its smart network card module establishes a data channel from SPDK AIO bdev to bare metal; Initialize the virtual PCI device, write VIRTIO_F_IOMMU_PLATFORM, VIRTIO_VENDOR_ID, VIRTIO_DEVICE_ID identifiers to the virtio PCI configuration space, and connect to the hardware PCI bridge of the smart NIC module; Connect the SPDK AIO bdev device to the virtual PCI device, load the PCI device driver on bare metal, and read and write data from the SPDK AIO bdev via DMA.

7. A method for bare metal secure use of distributed storage according to claim 1, characterized in that: In the process of returning the volume identifier and device index number, the mount index number is generated after the mounting is completed, and a success message is responded to the controller. After receiving the message, the controller combines the storage volume identifier and the mount device index number into an identifier p[n] and puts it into the persistent queue Q(p).

Citation Information

Patent Citations

  • Technologies for providing dynamically managed quality of service in a distributed storage system

    CN109196461A

  • Method, device and equipment for creating virtual machine and storage medium

    CN110704167A