A method for implementing multiple connections of a single cloud disk based on SPDK NVMe Over TCP

Abstract

The present invention belongs to the field of storage technology, and specifically relates to a single cloud disk multi-connection implementation method based on SPDK NVMe Over TCP. The invention enables the cloud disk to fully use the queue of the network card when transmitting data, and allows IO to achieve load balancing between TCP links and TCP links between network card queues. At the same time, even if some of the TCP links fail, IO can be transmitted through the existing normal links, so that the performance and robustness of the entire system can reach a relatively ideal state. The number of IO links of a single cloud disk is increased, which is equivalent to providing more parallel data logical paths for a single cloud disk, which helps to improve performance. The number of IO links of a single cloud disk is increased, which is actually equivalent to providing an effective backup path for cloud disk IO, improving the reliability of cloud disk IO. At the qpair layer, for the top-level IO request issued by the VM, different IO requests are hashed to different I / O qpairs through a certain algorithm, which fully guarantees the balance of IO transmission.

Description

Technical Field

[0001] The present invention belongs to the field of storage technology, and specifically relates to a single cloud disk multi-connection implementation method based on SPDK NVMe Over TCP. Background Art

[0002] SPDK is an acceleration library initiated by Intel to accelerate application software that uses NVMe SSD as backend storage. The backend NVMe SSD can be a local disk or a remote cloud disk. For the current SPDK cloud disk technology based on NVMe Over TCP, a corresponding TCP connection needs to be established between the SPDK target and host. Each cloud disk corresponds to each pair of qpairs in the underlying NVMe protocol (composed of a submission queue and a completion queue, which can be an admin qpair or an I / O qpair), and each qpair will be mapped to a TCP connection. Figure 7 , Figure 8 As shown in the figure, if the host maps four cloud disks on the target side, then an admin qpair ( Figure 7 Source port 55376), a qpair for attaching a cloud disk ( Figure 7 source port 55380), and 1 I / O qpair for each cloud disk, a total of 4 I / O qpairs, the entire system will have a total of 6 TCP connections (of which only I / O qpair will be used for business data transmission).

[0003] At the underlying network card level, a TCP link will eventually only choose to use one network card queue (determined by hash algorithms such as RSS based on the five-tuple), and most current network cards support multiple queues and the default RSS hash algorithm. Based on current analysis and experiments, the traditional method of using only one IO TCP link for a cloud disk cannot fully utilize the performance advantages of a multi-queue network card, and a single IO link for a cloud disk is relatively unstable. Summary of the invention

[0004] The purpose of the present invention is to provide a single cloud disk multi-connection implementation method based on SPDK NVMe Over TCP, which can enable the cloud disk to fully utilize the queue of the network card when transmitting data, and realize load balancing of IO between TCP links and TCP links between network card queues. At the same time, even if some of the TCP links fail, IO can be transmitted through the existing normal links, so that the performance and robustness of the entire system can reach a relatively ideal state.

[0005] The technical solution adopted by the present invention is as follows:

[0006] A method for implementing multiple connections of a single cloud disk based on SPDK NVMe Over TCP, comprising:

[0007] Get the RPC command and create a TCP type transmission PORT entity;

[0008] According to the RPC command, create a subsystem and a memory disk, simulate an SSD, store data in NVMe mode, establish a mapping relationship between the SSD and the subsystem, and configure the SSD to belong to the current subsystem;

[0009] Create a TCP listening entity through RPC commands and establish a TCP type port listening;

[0010] Get the configuration file and obtain the number of IO qpairs created by the cloud disk, that is, the number of TCP connections.

[0011] The Host obtains the SSD information on the TGT side according to the RPC command and establishes the TCP links of the admin qpair and ctrl qpair.

[0012] Create an SSD instance locally based on the SSD information on the TGT side, and establish a mapping relationship with the vhost-blk controller. Enable qemu to start the virtual machine, obtain the VM's data disk, and specify the VM's data disk as the vhost-blk controller.

[0013] Get the command that VM sends to vhost to establish IO qpair and create the corresponding TCP link;

[0014] Get the calculated source port and IO qpair quantity, create IO qpair and corresponding TCP link;

[0015] Get the three-way handshake of the TCP link, ensure that the link status is ESTABLISHED, and implement the NVMe Over TCP protocol handshake;

[0016] Get VM to authenticate the disk mapped by vhost-user, issue the identify command, complete the authentication, start the VM, perform IO test, and view the cloud disk.

[0017] In a preferred solution, the steps of creating a subsystem and a memory disk according to an RPC command, simulating an SSD, storing data in an NVMe manner, establishing a mapping relationship between the SSD and the subsystem, and configuring the SSD to belong to the current subsystem include:

[0018] Get the RPC command and create a new subsystem;

[0019] Create a memory disk according to the RPC command;

[0020] Configure the created memory disk to simulate an SSD and simulate the use of the NVMe standard for data storage;

[0021] Associate the identity information of the SSD device with the identity information of the subsystem;

[0022] Configure the SSD's ownership attributes;

[0023] If the attribute of SSD belongs to subsystem, it means that the attribute of this configuration belongs to the scope of data exchange and processing;

[0024] If the SSD attribute does not belong to subsystem, it means that the attribute of this configuration does not belong to the scope of data exchange and processing.

[0025] In a preferred solution, the step of obtaining the configuration file and obtaining the number of IO qpairs created by the cloud disk, that is, the number of TCP links, according to the configuration file includes:

[0026] Obtain a configuration file, which includes information about the number of IO qpairs that need to be created for each cloud disk;

[0027] Create IO qpairs based on the number of IO qpairs required for each cloud disk, which is the number of TCP links.

[0028] In a preferred solution, the step of enabling the Host to obtain the SSD information of the TGT end according to the RPC command and establish the TCP links of adminqpair and ctrl qpair includes:

[0029] Get RPC command;

[0030] The Host obtains the SSD information on the TGT side according to the RPC command;

[0031] Create admin qpair and ctrl qpair based on the obtained TGT SSD information;

[0032] Create TCP links according to the creation of admin qpair and ctrl qpair.

[0033] In a preferred solution, the steps of creating an SSD instance locally according to the SSD information on the TGT side, establishing a mapping relationship with the vhost-blk controller, enabling qemu to start the virtual machine, obtaining the data disk of the VM, and specifying the data disk of the VM as the vhost-blk controller include:

[0034] SSD information obtained from the TGT end;

[0035] Create a simulated SSD instance in the local system based on the SSD information;

[0036] Create a vhost-blk controller based on the created simulated SSD instance;

[0037] Establish the mapping between the SSD instance and the vhost-blk controller;

[0038] Get the qemu command;

[0039] Start the virtual machine according to the qemu command;

[0040] Get the VM's data disk;

[0041] Configure the VM's data disk according to the vhost-blk controller;

[0042] The data disk of the configured VM is the vhost-blk controller.

[0043] In a preferred solution, the step of obtaining a command issued by the VM to the vhost to establish an IO qpair and create a corresponding TCP link includes:

[0044] Get the command generated inside the VM for establishing IO qpair;

[0045] Transfer the command to create IO qpair to vhost;

[0046] The vhost creates a TCP link based on the received command to establish an IO qpair.

[0047] In a preferred solution, the step of obtaining the calculated source port and the number of IO qpairs, and creating an IO qpair and a corresponding TCP link includes:

[0048] Get the calculated source port value and IO qpair number information;

[0049] Create IO qpairs one by one according to the calculated source port and IO qpair quantity;

[0050] For each created IO qpair, a corresponding TCP link is established.

[0051] In a preferred solution, the steps of obtaining a three-way handshake of a TCP link, ensuring that the link state is ESTABLISHED, and implementing the NVMe Over TCP protocol handshake include:

[0052] Get the connection request from the remote device and initiate the three-way handshake;

[0053] Execute the TCP three-way handshake to make the TCP connection status ESTABLISHED;

[0054] According to the NVMe Over TCP protocol requirements, send the protocol handshake signal;

[0055] According to the handshake signal, implement the handshake of the NVMe Over TCP protocol.

[0056] In a preferred solution, the steps of obtaining VM authentication of the disk mapped by vhost-user, issuing an identify command, completing the authentication, starting the VM, performing an IO test, and viewing the cloud disk include:

[0057] Get the vhost-user disk identified inside the VM;

[0058] Issue the identify command to the vhost-user disk to obtain the identification information of the vhost-user disk;

[0059] Verify the authentication of the vhost-user disk to the identify command based on the obtained vhost-user disk identification information;

[0060] If the vhost-user disk is authenticated successfully, the VM is started successfully;

[0061] If the authentication of the vhost-user disk fails, the VM will not start successfully;

[0062] After authentication based on the virtio-blk disk, the VM starts;

[0063] Perform IO verification inside the VM on the virtio-blk disk;

[0064] Check the cloud disk inside the VM to verify whether the authenticated virtio-blk disk is visible in the VM;

[0065] If the authenticated virtio-blk disk is visible in the VM, it is normal;

[0066] If the authenticated virtio-blk disk is not visible in the VM, this indicates an abnormality.

[0067] And, a single cloud disk multi-connection implementation terminal based on SPDK NVMe Over TCP, including:

[0068] one or more processors;

[0069] a storage device having one or more programs stored thereon;

[0070] When the one or more programs are executed by the one or more processors, the one or more processors implement the single cloud disk multi-connection implementation method based on SPDK NVMe Over TCP.

[0071] The technical effects achieved by the present invention are:

[0072] The present invention enables the cloud disk to fully utilize the queue of the network card when transmitting data, and realizes load balancing between IOs among TCP links and TCP links among network card queues. At the same time, even if some of the TCP links fail, IOs can be transmitted through the existing normal links, so that the performance and robustness of the entire system can reach a relatively ideal state.

[0073] The present invention increases the number of IO links of a single cloud disk, which is equivalent to providing more parallel data logical paths for a single cloud disk, which is helpful to improve performance. Increasing the number of IO links of a single cloud disk is actually equivalent to providing an effective backup path for the cloud disk IO, thereby improving the IO reliability of the cloud disk. At the qpair layer, for the top-level IO requests issued by the VM, different IO requests are hashed to different I / Oqpairs through a certain algorithm, which fully guarantees the balance of IO transmission and realizes relatively friendly link-level load balancing. By pre-calculating the source port, load balancing of TCP links between network card queues is realized. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 is a flow chart of the method provided by the present invention;

[0075] Figure 2 This is a distribution diagram of NVMe Over TCP 4 disks with 4 links per disk after optimization provided by the present invention;

[0076] Figure 3 This is a schematic diagram of the optimized NVMe Over TCP single cloud disk IO flow provided by the present invention;

[0077] Figure 4 It is a timing diagram of the operation of the optimized solution provided by the present invention;

[0078] Figure 5 This is a performance comparison chart of IO links of different numbers per disk for multiple disks provided by the present invention;

[0079] Figure 6 This is a performance comparison chart of different numbers of IO links per disk provided by the present invention;

[0080] Figure 7It is a link usage distribution diagram of the traditional NVMe Over TCP disk provided by the present invention;

[0081] Figure 8 This is a schematic diagram of the traditional NVMe Over TCP single cloud disk I / O flow provided by the present invention;

[0082] Fig. 9 This is a conventional NVMe Over TCP runtime diagram provided by the present invention;

[0083] Fig.10 This is the performance comparison of different links of each disk of 4 cloud disks with 512B IO size provided by the present invention;

[0084] Fig.11 This is a comparison of different link performances of each disk of a single cloud disk with 4KB IO size provided by the present invention. DETAILED DESCRIPTION

[0085] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0086] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0087] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure or characteristic that may be included in at least one implementation of the present invention. The phrase "in a preferred embodiment" that appears in different places in this specification does not refer to the same embodiment, nor is it a separate or selective embodiment that is mutually exclusive with other embodiments.

[0088] Secondly, the present invention is described in detail in conjunction with schematic diagrams. When describing the embodiments of the present invention in detail, for the convenience of explanation, the schematic diagrams are only examples and should not limit the scope of protection of the present invention.

[0089] Please refer to the attached Figures 1 to 9 As shown, a single cloud disk multi-connection implementation method based on SPDK NVMe Over TCP is provided, including:

[0090] S1. Get the RPC command and create a TCP type transmission PORT entity;

[0091] S2. Create a subsystem and a memory disk according to the RPC command, simulate the SSD, store data in NVMe mode, establish a mapping relationship between the SSD and the subsystem, and configure the SSD to belong to the current subsystem;

[0092] S3. Create a TCP listening entity through RPC commands and establish a TCP type port listening;

[0093] S4. Obtain the configuration file, and obtain the number of IO qpairs created by the cloud disk, that is, the number of TCP links, according to the configuration file;

[0094] S5. According to the RPC command, the Host obtains the SSD information on the TGT side and establishes the TCP links of admin qpair and ctrlqpair.

[0095] S6. Create an SSD instance locally according to the SSD information on the TGT side, and establish a mapping relationship with the vhost-blk controller, so that qemu starts the virtual machine, obtains the data disk of the VM, and specifies the data disk of the VM as the vhost-blk controller;

[0096] S7, obtain the command sent by VM to vhost to establish IO qpair and create the corresponding TCP link;

[0097] S8. Obtain the calculated source port and IO qpair quantity, and create an IO qpair and a corresponding TCP link;

[0098] S9, obtain the three-way handshake of the TCP link, ensure that the link status is ESTABLISHED, and implement the NVMe Over TCP protocol handshake;

[0099] S10. Obtain VM authentication for the disk mapped by vhost-user, issue an identify command, complete authentication, start the VM, perform IO testing, and check the cloud disk.

[0100] As in the above steps S1 to S10, the RPC command is used for remote calls, which refers to command requests sent from one system to another. The TCP transmission PORT entity is created to establish a TCP-based communication channel for transmitting data and commands between different systems. Subsystem refers to the logical part in the storage field, which may be used to organize and manage storage resources. Simulating SSD is to simulate a storage device similar to a solid-state drive (SSD) through software, store data in NVMe mode, and provide higher performance and low latency. Establishing the mapping relationship between SSD and subsystem and configuring SSD to belong to the current subsystem is to ensure that storage resources are correctly allocated and managed. Creating a TCP listening entity is to listen to TCP connection requests from other systems or devices in order to establish a communication channel. The number of IO qpairs created by the cloud disk is determined through the configuration file in order to optimize system performance and enable the system to process multiple IO requests at the same time. Through the RPC command, the Host system obtains the SSD information on the TGT side and establishes admin qpair and ctrl The TCP link of qpair is used to manage and control the storage device on the target side. In the local system, an SSD instance is created according to the SSD information on the TGT side and mapped to the vhost-blk controller so that the virtual machine (VM) can access and use these storage resources. QEMU starts the virtual machine and specifies the data disk of the VM as the vhost-blk controller so that the virtual machine can read and write these data. The virtual machine (VM) may issue a command to the vhost (virtualized host) to establish an IO qpair and create a corresponding TCP link, which helps to achieve communication between the virtual machine and the storage resources. According to the calculation source port and the number of IO qpairs, an IO qpair and a corresponding TCP link are created to handle more IO requests and improve the concurrent processing capability of the system. The establishment of the TCP link is ensured through a three-way handshake, and the handshake of the NVMe Over TCP protocol is implemented. This is a way to run the NVMe protocol on TCP, which may provide more efficient storage access. The virtual machine authenticates the disk mapped by the vhost-user, which may be to ensure access rights and data security. Performing IO tests and viewing cloud disks may be to verify the performance and reliability of the storage system. By establishing parallel IO Through qpair and TCP links, the system can process multiple IO requests at the same time, improving the concurrent processing capability and performance of the system. By simulating SSDs and establishing subsystems, storage resources are effectively managed and allocated. Through vhost-blk controller and vhost-user mapping, virtual machines are supported to access and use storage resources. The use of NVMe Over TCP protocol may improve the efficiency of storage access. Certification and testing can ensure the security and stability of the storage system.

[0101] According to the RPC command, create a subsystem and a memory disk, simulate an SSD, store data in NVMe mode, establish a mapping relationship between the SSD and the subsystem, and configure the SSD to belong to the current subsystem. The steps include:

[0102] S201, obtain RPC command and create a new subsystem;

[0103] S202, creating a memory disk according to the RPC command;

[0104] S203, configuring the created memory disk to simulate an SSD, and simulating the use of the NVMe standard for data storage;

[0105] S204, associating the identity information of the SSD device with the identity information of the subsystem;

[0106] S205, configuring the ownership attribute of the SSD;

[0107] If the attribute of SSD belongs to subsystem, it means that the attribute of this configuration belongs to the scope of data exchange and processing;

[0108] If the SSD attribute does not belong to subsystem, it means that the attribute of this configuration does not belong to the scope of data exchange and processing.

[0109] As in the above steps S201 to S205, the RPC command may be a communication protocol between remote systems for issuing specific instructions or operation requests. Creating a new subsystem may refer to creating a new logical unit in the storage system for managing and organizing storage resources. A memory disk is a virtual disk stored in memory that can be used to temporarily store data or simulate persistent storage, but the data is usually lost after the system is shut down. Configuring a memory disk to simulate an SSD refers to setting the memory disk to behave like a solid-state drive (SSD), including achieving fast read and write operations and low latency. Using the NVMe standard for data storage refers to simulating the use of NVMe (Non-Volatile Memory SSD is a high-performance data transmission protocol designed for solid-state storage. It associates the identity information of SSD devices with the identity information of subsystems, which may be to associate storage devices with corresponding storage management units in order to manage and allocate storage resources. The attribution attribute of SSD may indicate the logical management unit or storage range to which it belongs. If the attribution attribute of SSD belongs to subsystem, it may indicate that the SSD device is designated as part of a specific subsystem, so that it can participate in the data exchange and processing of the subsystem. If the attribution attribute of SSD does not belong to subsystem, it may indicate that the SSD device does not belong to any specific subsystem and may not participate in the data exchange and processing of a specific subsystem. By creating a new subsy stem, which can better organize and manage storage resources and facilitate the management and allocation of storage. Configuring the memory disk to simulate SSD and using the NVMe standard helps provide higher-performance storage access, including faster data read and write speeds and low latency. Associating SSD devices with subsystems helps ensure that storage devices are properly managed and allocated, and the configuration of attribution attributes can be used to control their participation in specific data processing scopes. Simulating SSDs and adopting the NVMe standard may improve the performance of storage systems, including faster data transfer speeds and lower access latency. Creating subsystems and configuring SSD attribution attributes can provide more flexible storage management, enabling administrators to better control the allocation and use of storage resources. According to the SSD attribution attribute configuration, the scope of data exchange and processing can be clearly divided, which helps optimize data management and processing processes.

[0110] The steps to obtain the configuration file and obtain the number of IO qpairs created by the cloud disk, that is, the number of TCP links, according to the configuration file include:

[0111] S401, obtaining a configuration file, where the configuration file includes information about the number of IO qpairs required to be created for each cloud disk;

[0112] S402. Create IO qpairs according to the information of the number of IO qpairs required to be created for each cloud disk, that is, the number of TCP links.

[0113] As in the above steps S401 to S402, the configuration file may include parameters and information required for system operation or device management, including the number of IO qpairs required to be created for each cloud disk. The configuration file includes information on the number of IO qpairs required for each cloud disk, which will be used to create a corresponding number of IO qpairs. The number of IO qpairs required for each cloud disk may be set according to system requirements, performance optimization or concurrent processing requirements. The configuration file is a file that guides system behavior, which includes specific configuration information of the device or system, such as the number of IO qpairs. According to the information on the number of IO qpairs required for each cloud disk specified in the configuration file, the system will create a corresponding number of IO qpairs on demand to establish a TCP link for each cloud disk. Accurately creating the required number of IO qpairs based on the requirement information in the configuration file helps to optimize the utilization of system resources and avoid excessive or insufficient resource allocation. Creating a corresponding number of IO qpairs based on the requirements of each cloud disk helps to manage the number of concurrent connections, improve the concurrent processing capability of the system, and thus optimize the overall performance. The configuration file provides a flexible way to adjust the number of IO qpairs for each cloud disk as needed to adapt to changes in different workloads and performance requirements.

[0114] The steps to enable the host to obtain the SSD information on the TGT end and establish the TCP links of the admin qpair and ctrl qpair according to the RPC command include:

[0115] S501, obtaining an RPC command;

[0116] S502, enabling the Host to obtain the SSD information of the TGT end according to the RPC command;

[0117] S503, according to the obtained TGT end SSD information, establish admin qpair and ctrl qpair;

[0118] S504. Create TCP links respectively according to the established admin qpair and ctrl qpair.

[0119] As in the above steps S501 to S504, the RPC command may be a remote call mechanism for communication and request between different systems. According to the RPC command, the Host system sends a request to obtain relevant information of the target end (TGT end) SSD, including the identity information, capacity, status, etc. of the SSD. After obtaining the TGT end SSD information, the Host system may establish admin qpair (administrator queue pair) and ctrl qpair (control queue pair) based on this information. These queue pairs may be used to manage and control the target end SSD. The admin qpair is used to manage the queue, and the ctrl qpair is used to control the queue. In order to communicate with the TGT end SSD, the Host system creates TCP links respectively according to the established admin qpair and ctr qpair. The TCP link is used to ensure a stable communication channel between the Host system and the TGT end SSD. Through the RPC command, the Host system sends a request to the TGT end to obtain relevant information of the target end SSD. According to the obtained SSD information, the Host system establishes admin qpair and ctrl qpair. qpair is used to manage and control the access and operation of the target SSD. By using the established admin qpair and ctrl qpair information, a communication connection with the target SSD is established at the TCP level for data transmission and operation management. Through RPC commands and the establishment of queue pairs, the Host system can remotely manage and control the target SSD for data interaction and operation management. Creating a TCP link can ensure stable communication with the target SSD, which helps to ensure the reliability and consistency of data transmission. By using remote commands and communication establishment, the system has flexibility and can dynamically manage and control the target SSD device according to needs.

[0120] The steps of creating an SSD instance locally according to the SSD information on the TGT side, establishing a mapping relationship with the vhost-blk controller, enabling qemu to start the virtual machine, obtaining the data disk of the VM, and specifying the data disk of the VM as the vhost-blk controller include:

[0121] S601, SSD information obtained from the TGT end;

[0122] S602: Create a simulated SSD instance in the local system according to the SSD information;

[0123] S603. Create a vhost-blk controller according to the created simulated SSD instance;

[0124] S604, establish a mapping between the SSD instance and the vhost-blk controller;

[0125] S605, obtaining a qemu command;

[0126] S606, starting the virtual machine according to the qemu command;

[0127] S607, obtaining the data disk of the VM;

[0128] S608, configuring the data disk of the VM according to the vhost-blk controller;

[0129] S609, the data disk of the configured VM is the vhost-blk controller.

[0130] As in the above steps S601 to S609, information about the SSD is obtained from the target end (TGT end), including identity, capacity, status, etc. Based on the SSD information obtained from the TGT end, a simulated SSD instance is created in the local system. The simulated SSD instance may be a simulation of local storage resources to provide a virtual storage device for subsequent operations. According to the simulated SSD instance just created, the system may create a vhost-blk controller. The vhost-blk controller is a virtual controller for block devices in a virtualized environment, which is used to manage and control data transmission between virtual machines and simulated SSD instances. The correct mapping between the simulated SSD instance and the vhost-blk controller is ensured so that the system can identify and use these simulated storage resources. The qemu command for starting the virtual machine is obtained. QEMU is an open source software for virtualization and is used to run virtual machines. The virtual machine is started using the obtained qemu command, which enables the virtual machine to run on the local system. The data disk of the virtual machine is obtained for subsequent configuration and management. According to the previously created vhos Configure the t-blk controller, configure the data disk of the virtual machine accordingly, specify the data disk of the virtual machine as the previously configured vhost-blk controller, so that the virtual machine can access and use the simulated storage resources, create a simulated SSD instance based on the SSD information on the TGT side, and create a vhost-blk controller to manage the storage in the virtualized environment, establish a mapping between the SSD instance and the controller, and configure the vhost-blk controller for the data disk of the virtual machine so that the virtual machine can access the simulated storage resources, use the qemu command to start the virtual machine, and configure it to use the vhost-blk controller as the data disk, so that the virtual machine can use the simulated storage resources. By simulating SSD instances and vhost-blk controllers, virtualized storage resources are provided, the flexibility and utilization of system resources are increased, and the virtual machine data disk is configured as a vhost-blk controller, which simplifies the management and configuration process of virtual machine storage resources, allowing virtual machines to use simulated storage resources and enhancing system availability and data access performance.

[0131] The steps for obtaining the command that the VM sends to the vhost to establish an IO qpair and create a corresponding TCP link include:

[0132] S701, obtaining a command generated inside the VM for establishing an IO qpair;

[0133] S702, transmitting the command for establishing IO qpair to vhost;

[0134] S703. The vhost creates a TCP link according to the received command for establishing an IO qpair.

[0135] As in the above steps S701 to S703, a command for creating an IO qpair is obtained from inside the virtual machine. IO qpair is usually used to establish a communication channel between the virtual machine and the underlying storage. The command for establishing IO qpair obtained is transmitted to the vhost, i.e., the virtualization host, and the transmission is performed through the network or other communication methods. According to the command received from the virtual machine, the vhost performs corresponding operations, which may be to create an IO qpair and establish a TCP link for data transmission and communication between the virtual machine and the vhost. The command for creating IO qpair is obtained from inside the virtual machine, which is a way for the virtual machine to send a request to the underlying storage. The obtained command is transmitted to the vhost, which is completed through the communication mechanism or network protocol of the virtualization platform. According to the received command, the vhost performs corresponding operations, which may be to create an IO qpair and establish a TCP link so as to establish a communication channel between the virtual machine and the underlying storage. By establishing IO qpair and TCP link, communication and data transmission between the virtual machine and the underlying storage resources are realized, and the virtual machine is provided with the ability of storage access, and effective IO The establishment of qpair and TCP links can improve the data transmission efficiency and performance of the system, speed up the data interaction between the virtual machine and the underlying storage, generate commands inside the virtual machine and transmit them to the vhost, simplifying the management and process of communication between the virtual machine and the underlying storage.

[0136] The steps to obtain the calculated source port and IO qpair quantity and create an IO qpair and the corresponding TCP link include:

[0137] S801, obtaining information on the calculated source port value and the number of IO qpairs;

[0138] S802, creating IO qpairs one by one according to the calculated source port and IO qpair quantity;

[0139] S803: For each created IO qpair, establish a corresponding TCP link.

[0140] As in the above steps S801 to S803, the calculated source port value and IO qpair quantity information are obtained. The source port usually refers to the initiating port for data transmission. According to the calculated source port value and IO qpair quantity information, the system creates a corresponding number of IO qpairs one by one. Each IO qpair is usually used to represent a data transmission channel, which can send and receive data. Each created IO qpair will establish a corresponding TCP link to ensure reliable data transmission and communication. The system calculates the source port value and the IO qpair quantity, which may be based on system requirements or performance optimization. According to the calculated information, the system creates the required number of IO qpairs one by one to establish a data transmission channel, and establishes a corresponding TCP link for each IOqpair to ensure reliable data transmission and communication, ensure the stability of communication, accurately create the required number of IO qpairs according to the calculated information, avoid over-allocation or waste of resources, improve the resource utilization of the system, establish a TCP link for each IO qpair, ensure the stability and reliability of the data transmission channel, and help reduce errors and losses in data transmission. Create IO qpairs one by one. qpair and establish corresponding TCP links, which enhances the concurrent processing capability of the system and can handle multiple data transmission requests at the same time.

[0141] Get the three-way handshake of the TCP link, ensure that the link status is ESTABLISHED, and implement the steps of the NVMe Over TCP protocol handshake, including:

[0142] S901, obtain a connection request from a remote device and initiate a three-way handshake;

[0143] S902, perform TCP three-way handshake, and make the TCP connection state ESTABLISHED;

[0144] S903, according to the NVMe Over TCP protocol requirements, send a protocol handshake signal;

[0145] S904: Implement the handshake of the NVMe Over TCP protocol according to the handshake signal.

[0146] As in the above steps S901 to S904, when a remote device (possibly a client) wants to establish a connection with the system, the system receives a connection request and starts the TCP three-way handshake process to establish a connection. The three-way handshake is a standard process for establishing a TCP connection, including the client sending a SYN (synchronization) packet, the server replying a SYN-ACK (synchronization-acknowledgement) packet, and the client sending an ACK (acknowledgement) packet. After the three-way handshake is completed, the state of the TCP connection changes to ESTABLISHED, indicating that the connection has been established and both parties can transmit data. NVMe Over TCP is a specific protocol that requires the sending of a protocol handshake signal when establishing a connection. This signal may contain information for negotiating and confirming protocol-specific parameters. According to the received protocol handshake signal, the system executes the handshake steps of the NVMe Over TCP protocol, including parameter negotiation, authentication, version confirmation and other processes to ensure that the connection complies with the protocol specification. The connection is established through the TCP three-way handshake to ensure that the communication path between the client and the server has been established. According to the requirements of the NVMe Over TCP protocol, after the TCP connection is established, the protocol handshake signal is sent and received to ensure the correctness and security of the connection. According to the received handshake signal, the NVMe Over The steps and specifications required by the TCP protocol to ensure that the connection follows the communication method specified by the protocol, establish a connection in accordance with the requirements of the NVMe Over TCP protocol, ensure that the connection follows the protocol specifications, improve the versatility of the system with other compatible devices, and perform protocol-specific handshake steps, including protocol parameter negotiation and authentication, to help ensure the security and reliability of the connection, ensure that the connection complies with the specifications of the NVMeOver TCP protocol, ensure the consistency of data transmission and communication, and help avoid communication errors and data corruption.

[0147] Obtain the steps for the VM to authenticate the disk mapped by vhost-user, issue the ident fy command, complete the authentication, start the VM, perform IO tests, and view the cloud disk, including:

[0148] S1001, obtain the vhost-user disk identified inside the VM;

[0149] S1002. Issue an identify command to the vhost-user disk to obtain identification information of the vhost-user disk;

[0150] S1003, verifying the authentication of the vhost-user disk to the identify command according to the obtained vhost-user disk identification information;

[0151] If the vhost-user disk is authenticated successfully, the VM is started successfully;

[0152] If the authentication of the vhost-user disk fails, the VM will not start successfully;

[0153] S1004, after authentication of the virtio-blk disk, the VM starts;

[0154] S1005, perform IO verification of virtio-blk disk inside the VM;

[0155] S1006. Check the cloud disk inside the VM to verify whether the authenticated virtio-blk disk is visible in the VM.

[0156] If the authenticated virtio-blk disk is visible in the VM, it is normal;

[0157] If the authenticated virtio-blk disk is not visible in the VM, it indicates an abnormality

[0158] As in the above steps S1001 to S1006, the VM may identify one or more vhost-user disks for communication and data exchange with the host machine, and send an identify command to the vhost-user disk, which is a standard command for requesting the disk to provide its identity information. According to the obtained vhost-user disk identification information, it is authenticated to ensure the identity and legitimacy of the disk. If the authentication is successful, it means that the disk identity authentication is passed and the VM is successfully started; otherwise, the VM startup fails. If the vhost-user disk authentication is successful, the VM is allowed to start, and the virtio-blk disk is authenticated and started. The VM may perform some IO operations to verify the availability and correctness of the authenticated virtio-blk disk and whether the authenticated virtio-blk disk is in the V M is visible inside, ensuring that the authenticated disk can be correctly identified and accessed by the VM. The identify command is sent to obtain the disk's identity information, and the disk is authenticated based on the received information to ensure the disk's legitimacy and availability. After confirming that the authentication is successful, the VM is allowed to start and continue to authenticate and access the virtio-blk disk. IO operations and checks are performed inside the VM to verify whether the authenticated virtio-blk disk is available and normal. The authentication mechanism ensures the legitimacy of the disk's identity, improves the system's security, and prevents unauthorized access. The authentication process ensures the normality and availability of the disk, which helps to stabilize the system and avoid failures or data loss caused by disk problems. Verifying whether the authenticated virtio-blk disk is visible in the VM helps to promptly discover and resolve disk access or configuration problems and ensure the normal operation of the system.

[0159] Compared with the traditional single-disk single-link solution, the single-disk multi-link solution has the main advantage and effect of improving the performance of the cloud disk. The performance data is as attached. Figure 5 and Fig.10 As shown in the figure, the performance comparison data of different IO TCP link numbers of each disk of 4 cloud disks is shown. In the experimental environment, the CX-6 network card is a 25G network card. In the scenario of multiple disks and multiple links per disk, if 4K IO size traffic is used, the network card will be hit to the speed limit, and the hardware will reach the bottleneck first. Therefore, in the verification scheme of this case with multiple disks, the IOsize is adjusted to 512B for relevant comparative experiments, as shown in Figure 5. Fig.10 As shown in the figure, a single disk can perform a 4K IO size performance comparison and verification test, such as Figure 6 , Fig.11 shown.

[0160] And, a single cloud disk multi-connection implementation terminal based on SPDK NVMe Over TCP, including:

[0161] one or more processors;

[0162] a storage device having one or more programs stored thereon;

[0163] When one or more programs are executed by one or more processors, the one or more processors implement a single cloud disk multi-connection implementation method based on SPDK NVMe Over TCP.

[0164] The above is only a preferred embodiment of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention. The structures, devices and operating methods not specifically described and explained in the present invention shall be implemented according to the conventional means in the art unless otherwise specified and limited.

Claims

1. A method for implementing multiple connections of a single cloud disk based on SPDK NVMe Over TCP, characterized in that: include: Get the RPC command and create a TCP type transmission PORT entity; According to the RPC command, create a subsystem and a memory disk, simulate the SSD, store data in NVMe mode, establish a mapping relationship between the SSD and the subsystem, and configure the SSD to belong to the current subsystem; Create a TCP listening entity through RPC commands and establish a TCP type port listening; Get the configuration file and obtain the number of IO qpairs created by the cloud disk, that is, the number of TCP connections. The Host obtains the SSD information on the TGT side according to the RPC command and establishes the TCP links of the admin qpair and ctrl qpair. Create an SSD instance locally based on the SSD information on the TGT side, and establish a mapping relationship with the vhost-blk controller. Enable qemu to start the virtual machine, obtain the VM's data disk, and specify the VM's data disk as the vhost-blk controller. Get the command that VM sends to vhost to establish IO qpair and create the corresponding TCP link; Get the calculated source port and IO qpair quantity, create IO qpair and corresponding TCP link; Get the three-way handshake of the TCP link, ensure that the link status is ESTABLISHED, and implement the NVMe Over TCP protocol handshake; Get VM to authenticate the disk mapped by vhost-user, issue the identify command, complete the authentication, start the VM, perform IO test, and view the cloud disk.

2. According to claim 1, the method for implementing multiple connections of a single cloud disk based on SPDK NVMe Over TCP is characterized in that: The steps of creating a subsystem and a memory disk according to an RPC command, simulating an SSD, storing data in an NVMe manner, establishing a mapping relationship between the SSD and the subsystem, and configuring the SSD to belong to the current subsystem include: Get the RPC command and create a new subsystem; Create a memory disk according to the RPC command; Configure the created memory disk to simulate an SSD and simulate the use of the NVMe standard for data storage; Associate the identity information of the SSD device with the identity information of the subsystem; Configure the SSD's ownership attributes; If the attribute of SSD belongs to subsystem, it means that the attribute of this configuration belongs to the scope of data exchange and processing; If the SSD attribute does not belong to subsystem, it means that the attribute of this configuration does not belong to the scope of data exchange and processing.

3. According to claim 1, the method for implementing multiple connections of a single cloud disk based on SPDK NVMe Over TCP is characterized in that: The step of obtaining the configuration file and obtaining the number of IO qpairs created by the cloud disk, that is, the number of TCP links, according to the configuration file includes: Obtain a configuration file, which includes information about the number of IO qpairs that need to be created for each cloud disk; Create IO qpairs based on the number of IO qpairs required for each cloud disk, which is the number of TCP links.

4. According to claim 1, the method for implementing multiple connections of a single cloud disk based on SPDK NVMe Over TCP is characterized in that: The steps of enabling the Host to obtain the SSD information of the TGT end according to the RPC command and establish the TCP links of the admin qpair and the ctrl qpair include: Get RPC command; The Host obtains the SSD information on the TGT side according to the RPC command; Create admin qpair and ctrl qpair based on the obtained TGT SSD information; Create TCP links according to the creation of admin qpair and ctrl qpair.

5. The method for implementing single cloud disk multi-connection based on SPDK NVMe Over TCP according to claim 1, characterized in that: The steps of creating an SSD instance locally according to the SSD information on the TGT side, establishing a mapping relationship with the vhost-blk controller, enabling qemu to start the virtual machine, obtaining the data disk of the VM, and specifying the data disk of the VM as the vhost-blk controller include: SSD information obtained from the TGT end; Create a simulated SSD instance in the local system based on the SSD information; Create a vhost-blk controller based on the created simulated SSD instance; Establish the mapping between the SSD instance and the vhost-blk controller; Get the qemu command; Start the virtual machine according to the qemu command; Get the VM's data disk; Configure the VM's data disk according to the vhost-blk controller; The data disk of the configured VM is the vhost-blk controller.

6. The method for implementing single cloud disk multi-connection based on SPDK NVMe Over TCP according to claim 1, characterized in that: The step of obtaining a command issued by the VM to the vhost to establish an IO qpair and create a corresponding TCP link includes: Get the command generated inside the VM for establishing IO qpair; Transfer the command to create IO qpair to vhost; The vhost creates a TCP link based on the received command to establish an IO qpair.

7. The method for implementing single cloud disk multi-connection based on SPDK NVMe Over TCP according to claim 1, characterized in that: The step of obtaining the calculated source port and the number of IO qpairs, and creating an IO qpair and a corresponding TCP link includes: Get the calculated source port value and IO qpair number information; Create IO qpairs one by one according to the calculated source port and IO qpair quantity; For each created IO qpair, a corresponding TCP link is established.

8. The method for implementing single cloud disk multi-connection based on SPDK NVMe Over TCP according to claim 1, characterized in that: The steps of obtaining the three-way handshake of the TCP link, ensuring that the link status is ESTABLISHED, and implementing the NVMe Over TCP protocol handshake include: Get the connection request from the remote device and initiate the three-way handshake; Execute the TCP three-way handshake to make the TCP connection status ESTABLISHED; According to the NVMe Over TCP protocol requirements, send the protocol handshake signal; According to the handshake signal, implement the handshake of the NVMe Over TCP protocol.

9. The method for implementing single cloud disk multi-connection based on SPDK NVMe Over TCP according to claim 1, characterized in that: The steps of obtaining VM authentication of the disk mapped by vhost-user, issuing an identify command, completing authentication, starting the VM, performing IO testing, and viewing the cloud disk include: Get the vhost-user disk identified inside the VM; Issue the identify command to the vhost-user disk to obtain the identification information of the vhost-user disk; Verify the authentication of the vhost-user disk to the identify command based on the obtained vhost-user disk identification information; If the vhost-user disk is authenticated successfully, the VM is started successfully; If the authentication of the vhost-user disk fails, the VM will not start successfully; After authentication based on the virtio-blk disk, the VM starts; Perform IO verification inside the VM on the virtio-blk disk; Check the cloud disk inside the VM to verify whether the authenticated virtio-blk disk is visible in the VM; If the authenticated virtio-blk disk is visible in the VM, it is normal; If the authenticated virtio-blk disk is not visible in the VM, this indicates an abnormality.

10. A single cloud disk multi-connection implementation terminal based on SPDK NVMe Over TCP, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the single cloud disk multi-connection implementation method based on SPDK NVMe Over TCP as described in any one of claims 1-9.

Images