Data transmission method and related device

Abstract

The embodiment of the present application provides a data transmission method, wherein the storage node and the computing node communicate based on the RDMA protocol, the storage node and the solid-state hard disk are connected through a PCIe link, the storage node receives a data transmission request from the SQ of the computing node through the RQ of the storage node; the storage node writes the information of the data transmission request into the memory of the solid-state hard disk based on the PCIe link; the storage node receives a data transmission indication from the solid-state hard disk through the PCIe link; the storage node sends a data transmission indication to the RQ of the computing node through the SQ of the storage node; the storage node reads the data to be written from the computing node based on the data transmission indication, and sends it to the solid-state hard disk for writing; or reads the data to be read from the solid-state hard disk, and sends it to the computing node for writing. By adopting the embodiment of the present application, the data transmission efficiency between the computing node and the solid-state hard disk is improved through the PCIe link and the storage node.

Description

Technical Field

[0001] The present application relates to the field of storage technology, and in particular to a data transmission method and related devices. Background Art

[0002] With the rise of big data and cloud computing, more and more scenarios need to process big data in application environments. Therefore, distributed systems represented by computing and storage separation have gradually become an underlying infrastructure for cloud computing. Decoupling storage resources and computing resources facilitates the expansion of storage resources and is conducive to the purpose of storage resource pooling. However, the scale of computing nodes in a computing cluster is usually between tens of thousands and tens of thousands, and the scale of storage nodes in a storage cluster is generally between a few and hundreds, and large Internet manufacturers can even reach millions. Under the above scale, how to improve data transmission efficiency is a technical problem to be solved by technicians in this field. Summary of the invention

[0003] The embodiment of the present application discloses a data transmission method and related devices, which improve the data transmission efficiency between a computing node and a solid-state hard disk through a PCIe link and a storage node.

[0004] In the first aspect, the embodiment of the present application discloses a data transmission method, which is applied to a storage node, wherein the storage node and the computing node communicate based on the remote process direct data access RDMA protocol, and the storage node and the solid-state hard disk are connected through a peripheral device high-speed connection bus PCIe link, wherein: a data transmission request is received from a sending queue SQ of a computing node through a receiving queue RQ of the storage node, and the data transmission request is used to specify data transmission between the solid-state hard disk and the computing node; the information of the data transmission request is written into the memory of the solid-state hard disk based on the PCIe link; a data transmission indication is received from the solid-state hard disk through the PCIe link, and the command type of the data transmission indication is opposite to that of the data transmission request, and the data transmission indication is obtained by the solid-state hard disk according to the information of the data transmission request in the memory of the solid-state hard disk; a data transmission indication is sent to the receiving queue RQ of the computing node through the sending queue SQ of the storage node; based on the data transmission indication, the data to be written is read from the computing node, and sent to the solid-state hard disk for writing; or the data to be read is read from the solid-state hard disk, and sent to the computing node for writing. In other words, the storage node does not register the data transmission request from the computing node in its own memory, but directly registers it in memory on the solid-state hard disk, which reduces the time of memory registration. After the SSD is registered in the memory, it generates a data transmission instruction to control the storage node to realize data transmission between the storage node and the computing node. During the data transmission process, the storage node acts as a transmission medium between the computing node and the SSD, and does not cache data to be written or read, which reduces the time for the storage node to cache data and improves data transmission efficiency.

[0005] In a possible example, the data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, the second command is based on the non-volatile transmission bus NVMe protocol, and the information of the data transmission request is written into the memory of the solid-state drive based on the PCIe link, including: establishing the memory queue of the storage node on the solid-state drive based on the PCIe link; parsing the first command to obtain the solid-state drive corresponding to the data transmission request; writing the information of the second command into the memory queue of the storage node on the solid-state drive, so that the information of the data transmission request is written into the memory of the solid-state drive. It can be understood that the storage node does not register the data transmission request from the computing node in its own memory, but establishes the memory queue of the storage node on the solid-state drive in advance, so that the storage node can directly register the memory on the solid-state drive, reducing the time of memory registration. And after the memory registration, the solid-state drive can allocate memory space for the data to be written in the memory based on the information of the data transmission request, or determine the storage address of the data to be read, so as to generate a command to control the storage node to realize data transmission with the computing node (i.e., the data transmission indication in the embodiment of the present application), thereby improving the data transmission efficiency.

[0006] In a possible example, after reading the data to be written from the computing node based on the data transmission indication and sending it to the solid-state drive for writing; or reading the data to be read from the solid-state drive and sending it to the computing node for writing, the data transmission method also includes: sending a data completion message to the receiving queue RQ of the computing node through the sending queue SQ of the storage node, and the data completion message is used to indicate that the data to be written in the computing node is written to the solid-state drive, or the data to be read in the solid-state drive is written to the computing node. In this way, the data completion message can indicate that the computing node data transmission is completed, and the task queue in the computing node can be cancelled, which is convenient for improving the data transmission efficiency of the distributed system.

[0007] In the second aspect, the embodiment of the present application provides a data transmission method, which is applied to a solid-state hard disk, wherein the storage node and the computing node communicate based on the RDMA protocol, and the storage node and the solid-state hard disk are connected via a PCIe link, wherein: a data transmission indication is generated according to the information of the data transmission request in the memory, the data transmission request is sent by the computing node to the storage node, and is used to specify the data transmission between the solid-state hard disk and the computing node, the information of the data transmission request in the memory is the storage node writing the information of the data transmission request into the memory of the solid-state hard disk based on the PCIe link, and the data transmission indication is used to control the storage node to read the data to be written from the computing node and send it to the solid-state hard disk for writing; or read the data to be read from the solid-state hard disk and send it to the computing node for writing; send the data transmission indication to the storage node via the PCIe link. In other words, the storage node does not register the data transmission request from the computing node in its own memory, but directly registers it in memory on the solid-state hard disk, which reduces the time of memory registration. After the SSD is registered in the memory, it generates a data transmission instruction to control the storage node to realize the data transmission between the computing node and the SSD. During the data transmission process, the storage node acts as the transmission medium between the computing node and the SSD, and does not cache data to be written or read, which reduces the time for the storage node to cache data and improves the data transmission efficiency.

[0008] In a possible example, a data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol. Generating a data transmission instruction based on the information of the data transmission request in the memory includes: parsing the information of the second command to obtain target information; generating a data transmission instruction based on the target information. In this way, the solid-state drive parses the command based on the NVMe protocol to obtain the target information, and the storage node does not need to perform protocol conversion, which can reduce the workload of the storage node. Then, the solid-state drive generates a data transmission instruction based on the target information to control the storage node to read the data to be written, which can improve the storage efficiency of the distributed system.

[0009] In the third aspect, the embodiment of the present application provides a data transmission method, which is applied to a computing node, wherein the storage node and the computing node communicate based on the remote process direct data access RDMA protocol, and the storage node and the solid-state hard disk are connected through a peripheral device high-speed connection bus PCIe link, wherein: a data transmission request is generated, and the data transmission request is used to specify data transmission between the solid-state hard disk and the computing node; a data transmission request is sent to the receiving queue RQ of the storage node through the sending queue SQ of the computing node; a data transmission indication is received from the sending queue RQ of the storage node through the receiving queue RQ of the computing node, and the data transmission indication is obtained by the solid-state hard disk according to the information of the data transmission request written by the storage node in the memory of the solid-state hard disk; the computing node reads the data to be written by the storage node based on the data transmission indication, or writes the data to be read from the solid-state hard disk. In other words, the storage node does not register the data transmission request from the computing node in its own memory, but directly registers it in memory on the solid-state hard disk, which reduces the time of memory registration. And the solid-state drive generates a data transmission instruction after the memory registration to control the storage node to realize the data transmission between the computing node and the solid-state drive. During the data transmission, the storage node acts as a transmission medium between the computing node and the solid-state drive, and does not cache the data to be written or read, which reduces the time for the storage node to cache data and improves the data transmission efficiency. In a possible example, the data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol. In this way, after the computing node sends the data transmission request to the storage node, the storage node can communicate with the computing node based on the RDMA protocol in the first command, and the storage node can also communicate with the solid-state drive based on the NVMe protocol in the second command. And the solid-state drive can parse the data transmission request based on the second command, so as to improve the storage efficiency of the distributed system.

[0010] In the fourth aspect, the embodiment of the present application provides a storage node, the storage node and the computing node communicate based on the RDMA protocol, the storage node and the solid-state hard disk are connected through a PCIe link, and the storage node includes a radio frequency circuit, which is used to receive a data transmission request from the sending queue SQ of the computing node through the receiving queue RQ of the storage node, and the data transmission request is used to specify the data transmission between the solid-state hard disk and the computing node; write the information of the data transmission request into the memory of the solid-state hard disk based on the PCIe link; receive a data transmission indication from the solid-state hard disk through the PCIe link, the data transmission indication and the data transmission request have the opposite command type, and the data transmission indication is obtained by the solid-state hard disk according to the information of the data transmission request in the memory of the solid-state hard disk; send a data transmission indication to the receiving queue RQ of the computing node through the sending queue SQ of the storage node; read the data to be written from the computing node based on the data transmission indication, and send it to the solid-state hard disk for writing; or read the data to be read from the solid-state hard disk and send it to the computing node for writing. In other words, the radio frequency circuit of the storage node does not register the data transmission request from the computing node in the memory of the storage node, but directly registers it in memory on the solid-state hard disk, which reduces the time of memory registration. After the SSD is registered in the memory, it generates a data transmission instruction to control the radio frequency circuit of the storage node to realize data transmission between the storage node and the computing node. During the data transmission process, the storage node acts as a transmission medium between the computing node and the SSD, and does not cache data to be written or read, which reduces the time for the storage node to cache data and improves data transmission efficiency.

[0011] In a possible example, the data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the non-volatile transmission bus NVMe protocol. The radio frequency circuit is specifically used to establish the memory queue of the storage node on the solid state drive based on the PCIe link; the first command is parsed to obtain the solid state drive corresponding to the data transmission request; the information of the second command is written into the memory queue of the storage node on the solid state drive, so that the information of the data transmission request is written into the memory of the solid state drive. It can be understood that the radio frequency circuit of the storage node does not register the data transmission request from the computing node in its own memory, but establishes the memory queue of the storage node on the solid state drive in advance. In this way, the radio frequency circuit of the storage node can directly register the memory on the solid state drive, reducing the time of memory registration. And after the memory registration, the solid state drive can allocate memory space for the data to be written in the memory based on the information of the data transmission request, or determine the storage address of the data to be read, so as to generate a command to control the storage node to realize data transmission with the computing node (i.e., the data transmission indication in the embodiment of the present application), thereby improving the data transmission efficiency.

[0012] In a possible example, the radio frequency circuit is also used to send a data completion message to the receiving queue RQ of the computing node through the sending queue SQ of the storage node, and the data completion message is used to indicate that the data to be written in the computing node is written to the solid-state drive, or the data to be read in the solid-state drive is written to the computing node. In this way, the data completion message can indicate that the data transmission of the computing node is completed, and the task queue in the computing node can be cancelled, which is convenient for improving the data transmission efficiency of the distributed system.

[0013] In the fifth aspect, the embodiment of the present application provides a solid-state hard disk, the storage node and the computing node communicate based on the RDMA protocol, the storage node and the solid-state hard disk are connected through a PCIe link, the solid-state hard disk includes a processor, which is used to generate a data transmission indication according to the information of the data transmission request in the memory, the data transmission request is sent by the computing node to the storage node, and is used to specify the data transmission between the solid-state hard disk and the computing node, the information of the data transmission request in the memory is the storage node writing the information of the data transmission request into the memory of the solid-state hard disk based on the PCIe link, the data transmission indication is used to control the storage node to read the data to be written from the computing node and send it to the solid-state hard disk for writing; or read the data to be read from the solid-state hard disk and send it to the computing node for writing; send the data transmission indication to the storage node through the PCIe link. In other words, the storage node does not register the data transmission request from the computing node in its own memory, but directly registers it in memory on the solid-state hard disk, which reduces the time of memory registration. After the memory is registered, the processor of the solid-state drive generates a data transmission instruction to control the storage node to realize data transmission between the computing node and the solid-state drive. During the data transmission process, the storage node acts as a transmission medium between the computing node and the solid-state drive, and does not cache data to be written or read, which reduces the time for the storage node to cache data and improves data transmission efficiency.

[0014] In a possible example, a data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol. The processor is specifically used to parse the information of the second command to obtain target information; based on the target information, a data transmission instruction is generated. In this way, the processor of the solid-state drive parses the command based on the NVMe protocol to obtain the target information, and the storage node does not need to perform protocol conversion, which can reduce the workload of the storage node. Then, the solid-state drive generates a data transmission instruction based on the target information to control the storage node to read the data to be written, which can improve the storage efficiency of the distributed system.

[0015] In the sixth aspect, the embodiment of the present application provides a computing node, the storage node and the computing node communicate based on the RDMA protocol, the storage node and the solid-state hard disk are connected through a PCIe link, and the computing node includes a processor, a radio frequency circuit and a memory, wherein: the processor is used to generate a data transmission request, the data transmission request is used to specify data transmission between the solid-state hard disk and the computing node; the radio frequency circuit is used to send a data transmission request to the receiving queue RQ of the storage node through the sending queue SQ of the computing node; the receiving queue RQ of the computing node receives a data transmission indication from the sending queue RQ of the storage node, and the data transmission indication is obtained by the solid-state hard disk according to the information of the data transmission request written by the storage node in the memory of the solid-state hard disk; the memory is used to be read by the storage node based on the data transmission indication to be written, or to write the data to be read from the solid-state hard disk. In this way, that is to say, the storage node does not register the data transmission request from the radio frequency circuit of the computing node in its own memory, but directly registers it in memory on the solid-state hard disk, which reduces the time of memory registration. After the SSD is registered in the memory, it generates a data transmission instruction to control the storage node to realize data transmission between the storage node and the computing node. During the data transmission process, the storage node acts as a transmission medium between the computing node and the SSD, and does not cache data to be written or read, which reduces the time for the storage node to cache data and improves data transmission efficiency.

[0016] In a possible example, the data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol. In this way, after the RF circuit of the computing node sends the data transmission request to the storage node, the storage node can communicate with the computing node based on the RDMA protocol in the first command, and the storage node can also communicate with the solid-state drive based on the NVMe protocol in the second command. And the solid-state drive can parse the data transmission request based on the second command, so as to improve the storage efficiency of the distributed system.

[0017] In a seventh aspect, the present application provides a device comprising a processing unit, a storage unit, and a communication unit, for executing instructions for the steps in the method of any of the above aspects.

[0018] In an eighth aspect, the present application provides a device comprising a processor and a memory and a communication interface connected to the processor, wherein the memory is used to store one or more programs and is configured to be executed by the processor, and the program includes instructions for executing the steps in the method of any of the above aspects.

[0019] In a ninth aspect, the present application provides a chip comprising a processor and a memory, wherein the processor is used to call and execute instructions stored in the memory, so that a device equipped with the chip executes the instructions of the steps in the method of any of the above aspects.

[0020] In the tenth aspect, an embodiment of the present application also provides another chip, including: an input interface, an output interface and a processing circuit, the input interface, the output interface and the processing circuit are connected through an internal connection path, and the processing circuit is used to execute instructions of the steps in any of the above methods.

[0021] In the eleventh aspect, an embodiment of the present application also provides another chip, including: an input interface, an output interface, a processor, and optionally, a memory, wherein the input interface, the output interface, the processor and the memory are connected through an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the instructions of the steps in the method of any of the above aspects.

[0022] In the twelfth aspect, an embodiment of the present application also provides a chip system, which includes at least one processor, a memory and an interface circuit, the memory, the transceiver and at least one processor are interconnected through lines, and instructions are stored in at least one memory; when the above instructions are executed by the processor, the method of any one of the above aspects is implemented.

[0023] In a thirteenth aspect, the present application provides a computer-readable storage medium, in which instructions are stored, and when the computer-readable storage medium is run on a computer, the computer executes any of the methods described above.

[0024] In a fourteenth aspect, the present application provides a computer program product, which is used to store a computer program. When the computer program runs on a network device, it enables the computer to execute any of the methods described above.

[0025] In a fifteenth aspect, an embodiment of the present application provides a distributed system, comprising a storage node, a solid state drive, and a computing node described in any of the above aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the architecture of a distributed system provided in an embodiment of the present application;

[0027] Figure 2 It is a structural diagram of a storage node and a solid state drive provided in an embodiment of the present application;

[0028] Figure 3 It is a flow chart of a data transmission method provided by the prior art;

[0029] Figure 4 It is a flowchart of a data transmission method provided by an embodiment of the present application;

[0030] Figure 5It is a flowchart of another data transmission method provided in an embodiment of the present application;

[0031] Figure 6 It is a flowchart of another data transmission method provided in an embodiment of the present application;

[0032] Figure 7 It is a schematic diagram of the architecture of another distributed system provided in an embodiment of the present application;

[0033] Figure 8 It is a structural schematic diagram of a device provided in an embodiment of the present application;

[0034] Fig. 9 It is a structural schematic diagram of another device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0036] Figure 1 Schematic diagram of a distributed system provided in an embodiment of the present application. Figure 1 As shown, the distributed system includes a computing cluster consisting of multiple computing nodes and a storage cluster consisting of multiple storage nodes, and at least one solid-state hard disk connected to each storage node. The computing cluster and the storage cluster communicate based on the remote direct memory access (RDMA) protocol. The RDMA protocol can be at least one of the RDMA over ethernet (RoCE) protocol, InfiniBand (IB), and Internet wide area RDMA protocol, which is not limited here.

[0037] The relevant data operations of RDMA mainly include four types: RDMA send (or receive) operation (send / recv), RDMA read operation (pull), RDMA write operation (push) and RDMA write operation (RDMA write with immediate data) that supports immediate data. Among them, RDMA send ( / receive) is a message-based data transmission protocol of RDMA, and the assembly of all data packets is completed on RDMA hardware, that is to say, the following 4 layers (transport layer, network layer, data link layer, physical layer) in the open system interconnection reference model (open system interconnection reference model, OSI) are all completed on RDMA hardware. The RDMA read operation pulls the data in the remote system memory back to the local system memory. The RDMA write operation pushes the data in the local system memory to the remote system memory. The RDMA write operation that supports immediate data is essentially to push data to the remote system. The data transmission request in the embodiment of the present application includes the above-mentioned RDMA read operation, RDMA write operation and RDMA write operation that supports immediate data, which are specifically used to read the stored data in the solid-state hard disk or write data to the solid-state hard disk.

[0038] RDMA supports three queues in total, including send queue (SQ), receive queue (RQ) and completion queue (CQ). Among them, SQ and RQ are usually created in pairs, collectively referred to as work queue (WQ). SQ is a logical concept, consisting of one or more units, each unit stores a data transfer request, and each data transfer request can be stored in a storage space of up to 64 bytes. SQ corresponds to a ring buffer for storing one or more data transfer requests, which can be specifically represented by a physical region page (PRG) or a scatter gather list (SGL). Each data transfer request (also called a send queue entry (SQE) or a send queue element (SQE)) can be stored in a unit of SQ, which can be called a slot of SQ, and each slot corresponds to two PRGs or one SGL in the buffer. SQ is set with a head pointer and a tail pointer. The head pointer is used to indicate the slot of the SQE that can be taken away at the current moment, and the tail pointer is used to indicate the slot that can store the newly added SQE at the current moment. In the initialization phase, the head pointer = tail pointer = 0. Every time a new SQE is added to the SQ, the tail pointer increases by 1. Every time an SQE is taken away from the SQ, the head pointer increases by 1. The data transmission requests to be executed need to be stored one by one in the slots of the submission queue in the order of reception, and then read one by one in the order of first in first out (FIFO).

[0039] The description of RQ can refer to that of SQ. Each unit stores a data transmission request, which will not be described here.

[0040] CQ is a circular buffer for storing the operation results of completed data transfer requests. Similar to the structure of SQ, the completion queue is also a logical concept, consisting of one or more units, each of which can be called a slot. CQ also corresponds to a circular buffer for storing the operation results of one or more data transfer requests, which can be specifically represented by PRG or SGL. The operation result of each data transfer request can also be called a completion queue entry (CQE) or a completion queue element (CQE). Each SQ corresponds to a CQ, and the same CQ can correspond to multiple SQs. The matching relationship between CQ and SQ is specified by the storage node, and the operation results of the data transfer requests in each SQ will be stored in a specified CQ completion queue.

[0041] RDMA is a message-based transmission protocol, and data transmission is asynchronous. For example, the host is the producer and the RDMA hardware is the consumer. The host produces a work request (WR) and puts it into the WQ; the RDMA hardware consumes the WR in the WQ, and puts the work completion message (WC) into the CQ after consumption; the host receives the content of the WC from the CQ.

[0042] Both the computing node and the storage node may include SQ and RQ, that is, the computing node may send WR (for example, a data transmission command in the prior art) to the storage node through the SQ in the WQ, so that the storage node receives WR through the RQ in the WQ; after the storage node reads the task corresponding to the WR in the RQ, it may cache the data to be written to the storage node by the computing node, or cache the data to be read from the solid-state hard disk by the computing node; then, the storage node generates a WR (for example, a completion message in the prior art) to notify the computing node that the read / write operation is completed. The computing node and the storage node also include a CQ corresponding to the SQ and RQ. For example, after the storage node allocates cache space for the data to be written or the data to be read, the storage node may put a WC (for example, a response message in the prior art) into the CQ to notify the computing node to send the command for the data to be written or the data to be read. After receiving the completion message sent by the storage node, the computing node may put a WC into the CQ to determine that the computing node read / write operation is completed.

[0043] The present application does not limit the types of computing nodes and storage nodes. The computing node can be an independent server with computing functions, for example, the computing node is a virtual block system (VBS), etc. The storage node can be an independent server with a solid-state drive, or an independent SSD box (just a bunch offlash, JBOF), for example, the storage node can be an object-based storage device (OSD), etc. The present application does not limit the number of computing nodes and storage nodes, and one computing node can be connected to multiple storage nodes. It can be understood that the computing nodes in the computing cluster can send data transmission requests to the storage cluster through the network, so that the corresponding storage nodes in the storage cluster execute the data transmission request.

[0044] The solid state drive in the embodiment of the present application may be a solid state drive (SSD), which may include but is not limited to various types of non-volatile memory, such as three-dimensional crosspoint memory, flash memory, ferroelectric memory, silicon oxide-nitride-oxide-silicon (SONOS) memory, polymer memory, nanowire, ferroelectric transistor random access memory (field-effect transistor random access memory, FeTRAM or FeRAM), nanowire or electrically erasable programmable read only memory (EEPROM). The present application does not limit the number of solid state drives connected to the storage node, which may be 12, 24 or 36, etc. The solid state drive may include a connection to the storage node via a high-speed serial computer expansion bus standard (Peripheral Component Interconnect express, PCIe) link.

[0045] The PCIe link is based on the PCIe protocol, which is a serial connection protocol based on data packets and uses an end-to-end connection method. After the storage node and the SSD are connected through the PCIe link, or it can be understood that after the distributed system is powered on, the PCIe link will enumerate the connected devices (storage nodes and SSDs), obtain the topology of the PCIe link, and map the register addresses of the connected devices, so that the memory space of the connected devices can be accessed, so that the storage node can control the SSD and the computing node to achieve end-to-end communication.

[0046] The storage node and the solid-state drive can communicate through the fast non-volatile memory express (NVMe) protocol. The SSD that uses the NVMe protocol for communication can be called NVMe SSD, which is favored by many application fields for its advantages such as low latency, low power consumption, and high read and write speed.

[0047] NVMe is an extensible host-side control interface. This application uses the processor in the storage node as the CPU in the host. Figure 2As shown, the processor uses the root port to connect multiple solid-state drives through a switch (i.e., a switch based on the PCIe protocol, PCIe switch), thereby expanding the storage space of the storage node through the connected solid-state drives. Among them, there is a controller inside each solid-state drive, which can be used to parse the NVMe protocol and process input / output (I / O), etc. It can be understood that the commands received from the computing node in the storage node are converted to the protocol by the solid-state drive, that is, the commands based on the RDMA protocol are converted to commands based on the NVMe protocol, thereby reducing the storage node's work task of converting the protocol, which is convenient for improving the processing efficiency of the storage node.

[0048] In the NVMe protocol, NVMe queues are divided into two types: management queues and I / O queues. The management queue is mainly used for device control and management, such as the creation and deletion of I / O queues. The I / O queue is mainly used for disk access. The submission queue (SQ) and completion queue (CQ) of the I / O queue are not fixed one-to-one correspondence, and can be flexibly configured when creating the SQ. One SQ can correspond to one CQ, or multiple SQs can correspond to one CQ. The description of SQ and CQ can refer to the description of SQ and CQ in the RDMA protocol, which will not be repeated here. It should be noted that in the NVMe protocol, the SQ is sent by the Host to the SSD, and the SSD will not send the SQ to the Host. Therefore, there is a difference in the naming of SQ in the RDMA protocol and the NVMe protocol.

[0049] Since the information (e.g., commands and data, etc.) received by the storage node from the computing node is based on the RDMA protocol, and the information sent by the storage node to the solid-state drive is based on the NVMe protocol, in the prior art, the processor in the storage node is used for protocol conversion, and the memory in the storage node is used for data caching, thereby ensuring the reliability and security of the data.

[0050] For details, please refer to Figure 3 , Figure 3 FIG. 1 is a flow chart of a data transmission method provided in the prior art. Figure 3 As shown, Figure 3 Applied to Figure 1 In the distributed system shown, the method is described by taking the computing node writing data to the solid state drive as an example, and specifically includes the following steps, wherein:

[0051] S301: The computing node sends a first data transmission command to the storage node.

[0052] The first data transmission command is based on the RDMA protocol, and the first data transmission command may be received by the receiving queue RQ of the storage node from the sending queue SQ of the computing node. The method is described by taking the computing node writing data to the solid state drive as an example, and the first data transmission command in step S301 is a write command. The first data transmission command may be stored in a storage space of a maximum size of 64 bytes.

[0053] S302: The storage node performs memory registration based on the first data transmission command to obtain first data information.

[0054] The memory registration of the storage node is used to allocate storage space in the storage node for caching the data to be written. The first data information may include the operation type of the first data transmission command, the source address in the computing node and the data size, and the operation type is a write type.

[0055] S303: The storage node sends a first response message to the computing node.

[0056] The first response message is used to instruct the storage node to complete memory registration, or to instruct the computing node to write data.

[0057] S304: The computing node sends the data to be written to the storage node.

[0058] S305: The storage node caches the data to be written based on the first data information.

[0059] S306: The storage node performs protocol conversion on the first data transmission command based on the first data information to obtain a second data transmission command.

[0060] The second data transmission command is based on the NVMe protocol. In step S306, the command type of the second data transmission command is the same as the first data transmission command, which is also a write command.

[0061] S307: The storage node sends a second data transmission command to the solid state drive through the PCIe link.

[0062] S308: The solid state drive performs memory registration based on the second data transmission command to obtain second data information.

[0063] The memory registration of the solid state drive is used to allocate storage space in the solid state drive for data to be written. Specifically, the second data information may include the operation type of the second data transmission command, the source address in the computing node, the cache address in the storage node, and the data size.

[0064] S309: The solid state drive sends a second response message to the storage node.

[0065] The second response message is used to instruct the solid-state drive to complete memory registration, or to instruct the storage node to transfer the data to be written to the solid-state drive.

[0066] S310: The solid state drive receives data to be written from the computing node through the PCIe link.

[0067] S311: The solid state drive stores the data to be written based on the second data information.

[0068] S312: The solid state drive sends a completion message to the storage node through the PCIe link.

[0069] The completion message is used to indicate that the data to be written has been stored in the solid state drive.

[0070] S313: The storage node sends a completion message from the solid state drive to the computing node.

[0071] The completion message may be sent from the sending queue SQ of the storage node to the receiving queue RQ of the computing node.

[0072] It should be noted that the execution order of step S306 can be the same as that of step S303, and after the storage node obtains the second data transmission command, step S307 can be executed. Both the first data transmission command and the second data transmission command can be stored in a storage space of 64 bytes.

[0073] When the computing node reads data from the solid-state hard disk, the first data information includes at least the operation type of the first data transmission command, the target address in the computing node, the source address in the solid-state hard disk and the data size, etc. At this time, the operation type is a read type; in addition to the first data information, the second data information may also include a cache address in the storage node.

[0074] It can be understood that whether it is data sent by the computing node to the SSD for writing or data read by the computing node from the SSD, it is cached by the memory of the storage node located between the computing node and the SSD. The working efficiency of the memory will directly affect the performance of the entire system.

[0075] Based on this, the present application embodiment proposes a data transmission method, which is applied to Figure 1 The distributed system shown in Figure 1. Figure 4 The method includes the following steps S401 to S406. The method is described by taking the computing node writing data to the solid state drive as an example. That is, the data transmission request sent by the computing node to the storage node is a write command, or it can be understood that the operation type of the data transmission request is a write type. Among them:

[0076] S401: A computing node generates a data transmission request, where the data transmission request is a write command.

[0077] In an embodiment of the present application, a data transmission request is used to indicate data transmission between a solid-state drive and a computing node. The data transmission request may carry data information of an operation type, data to be read or data to be written, such as a data identifier, a data size, a source address or a target address, etc. The present application does not limit the data transmission request and the data information. When the operation type of the data transmission request is a write type, the address in the data transmission request includes the source address in the computing node. When the operation type of the data transmission request is a read type, the address in the data transmission request includes the target address in the computing node and the source address in the solid-state drive. In an embodiment of the present application, the source address refers to the address of the sending end, and the target address refers to the address of the receiving end. The present application does not limit the method for generating a data transmission request by a computing node, and it can be generated according to the information of the source address or the target address, as well as the operation type and data size of the data. In step S401, the data transmission request is a write command, that is, the operation type of the data transmission request is a write type.

[0078] It should be noted that the data transmission request may include two 64-byte storage spaces for storage. The command corresponding to the first 64 bytes may be called the first command, which is described using the RDMA protocol, and the command corresponding to the second 64 bytes may be called the second command, which is described using the NVMe protocol. In this way, the storage node can communicate with the computing node based on the RDMA protocol in the first command, and the storage node can also communicate with the solid-state drive based on the NVMe protocol in the second command.

[0079] S402: The computing node sends a data transmission request to the receiving queue RQ of the storage node through the sending queue SQ of the computing node.

[0080] As mentioned above, the computing node and the storage node can communicate via the RDMA protocol. Therefore, a data transmission request can be sent to the receiving queue RQ of the storage node via the sending queue SQ of the computing node. In other words, the receiving queue of the storage node receives the data transmission request from the sending queue SQ of the computing node.

[0081] S403: The storage node writes the information of the data transmission request into the memory of the solid state drive based on the PCIe link.

[0082] As mentioned above, the storage node can access the SSD through the register address of the SSD in the PCIe link, so that the information of the data transmission request can be written into the memory of the SSD. That is to say, after writing the information of the data transmission request in the SSD, the SSD can control the storage node to read the data to be written from the computing node and send the data to be written to the SSD for storage, or send the data to be read in the SSD to the storage node and control the storage node to write the data to be read to the computing node.

[0083] In a possible example, the data transmission request includes a first command based on the RDMA protocol and a second command based on the NVMe protocol, and step S403 includes the following steps: the storage node establishes a memory queue of the storage node on the solid-state drive based on the PCIe link; the storage node parses the first command to obtain the solid-state drive corresponding to the data transmission request; the storage node writes the information of the second command into the memory queue of the storage node on the solid-state drive, so that the information of the data transmission request is written into the memory of the solid-state drive.

[0084] The information of the data transmission request includes the operation type and the data information of the data to be read or written, which can be referred to the description of step S401 and will not be repeated here. The storage node can obtain the information of the data transmission request by parsing the first command, which includes the solid state drive corresponding to the data transmission request. It can be understood that the data transmission request specifies the data transmission between the solid state drive and the computing node.

[0085] For example, if Figure 5 As shown, the sending queue SQ of the computing node sends a data transmission request to the storage node through the RF circuit of the computing node. The storage node writes the data transmission request into the receiving queue RQ data transmission request through the RF circuit of the storage node. The storage node obtains the register address of the solid-state drive based on the PCIe link and establishes the memory registration (MR) queue of the storage node in the memory of the solid-state drive. Then, the storage node writes the information of the data transmission request into the memory registration queue of the storage node on the solid-state drive through the PCIe link to complete the memory registration of the data transmission request.

[0086] It should be noted that after the storage node and the solid-state drive are connected via a PCIe link, the MR queue of the storage node can be established in the memory of the solid-state drive based on the PCIe link, thereby improving data transmission efficiency. Figure 5 As shown, the storage node can also establish the completion queue CQ of the storage node in the memory of the solid state drive. After the memory registration is completed, the WR is put into the completion queue CQ, so that the storage node can determine that the solid state drive completes the memory registration data transmission request.

[0087] It can be understood that after the memory queue of the storage node is established on the solid-state drive based on the PCIe link, the storage node receives a data transmission request from the computing node, and the storage node can directly register the data transmission request in the memory queue of the storage node in the memory of the solid-state drive, and write the information of the data transmission request. In other words, the storage node does not register the data transmission request from the computing node in its own memory, but establishes the memory queue of the storage node on the solid-state drive in advance, so that the storage node can directly register the memory on the solid-state drive, reducing the time of memory registration. And after the memory registration, the solid-state drive can allocate memory space for the data to be written in the memory based on the information of the data transmission request, or determine the storage address of the data to be read, so as to generate a command to control the storage node to realize data transmission with the computing node (i.e., the data transmission indication in the embodiment of the present application), which improves the efficiency of data transmission.

[0088] S404: The solid state drive generates a data transmission indication according to the information of the data transmission request in the memory, where the data transmission indication is a read command.

[0089] In an embodiment of the present application, a data transmission indication is used to control a storage node to read data to be written from a computing node and send it to a solid-state drive for writing. It should be noted that the command types of a data transmission indication and a data transmission request are opposite, and the data transmission indication is relative to the computing node, and the data transmission request is relative to the solid-state drive. For example, when the data transmission request is a read command, the data transmission indication is a write command, that is, the computing node writes the data to be read in the solid-state drive. When the data transmission request is a write command, the data transmission indication is a read command, that is, the data to be written in the computing node is read by the storage node. And when the data transmission indication is a write command, the data transmission indication includes the target address in the solid-state drive and the source address in the computing node. When the data transmission indication is a read command, the data transmission indication includes the source address in the solid-state drive and the target address in the computing node. It should be noted that the data transmission request is relative to the storage node, and its operation type is a read type, that is, when the data transmission indication is a write command, the storage node reads the data to be read from the solid-state drive and sends it to the computing node for writing. When the data transmission indication is a read command, the storage node reads the data to be written from the computing node and sends it to the solid-state drive for writing.

[0090] The data transmission indication is also used to indicate that the data to be written in the computing node will be read by the storage node. In other words, the computing node will have the data to be written read by the storage node based on the data transmission indication. It can be understood that by sending the data transmission indication to the computing node to indicate that the data transmission request in the computing node is about to be executed, it is convenient to improve the efficiency of task execution in the computing node.

[0091] In a possible example, the data transmission request includes a first command based on the RDMA protocol and a second command based on the NVMe protocol, and step S404 includes: the solid-state drive parses the information of the second command to obtain target information; the solid-state drive generates a data transmission indication based on the target information.

[0092] The target information may include the operation type, data to be read or data to be written, which can be referred to the description of step S401 and will not be repeated here. It should be noted that since the command types of the data transmission indication and the data transmission request are opposite, the source address and the target address in the target information are opposite to the source address or the target address in the data transmission request.

[0093] It can be understood that the SSD first parses the command based on the NVMe protocol to obtain the target information, and the storage node does not need to perform protocol conversion, which can reduce the workload of the storage node. Then the SSD generates a data transmission instruction based on the target information to control the storage node to read the data to be written, which can improve the storage efficiency of the distributed system.

[0094] S405: The solid state drive sends a data transmission instruction to the storage node through the PCIe link.

[0095] Step S405 can also be understood as the storage node receiving a data transmission instruction from the solid state drive via the PCIe connection.

[0096] S406: The storage node sends a data transmission indication to the receiving queue RQ of the computing node through the sending queue SQ of the storage node.

[0097] S407: The storage node reads the data to be written from the computing node based on the data transmission instruction, and sends it to the solid state drive for writing.

[0098] In such Figure 4In the method described, when the data transmission request is a write command, after the storage node receives the data transmission request from the sending queue SQ of the computing node through its RQ queue, the information of the data transmission request is written into the memory of the solid-state drive based on the PCIe link. In this way, the solid-state drive can generate a data transmission indication based on the information in the memory, thereby controlling the storage node to read the data to be written from the computing node according to the data transmission indication, and send it to the solid-state drive for storage. In other words, the storage node does not perform memory registration on the data transmission request from the computing node in its own memory, but directly performs memory registration on the solid-state drive, which reduces the time for memory registration. And the solid-state drive generates a data transmission indication after memory registration to control the storage node to realize data transmission between the computing node and the solid-state drive. During the data transmission process, the storage node acts as a transmission medium between the computing node and the solid-state drive, and does not cache the data to be written, which reduces the time for the storage node to cache data and improves the data transmission efficiency.

[0099] In a possible example, after step S407, the data transmission method further includes: the storage node sends a data completion message to the receiving queue RQ of the computing node through the sending queue SQ of the storage node.

[0100] The data completion message is used to indicate that the data to be written in the computing node is stored in the solid-state drive. That is, after the storage node transfers the data to be written to the solid-state drive, it can send a data completion message to the computing node to indicate that the data transmission of the computing node is completed, and the task queue in the computing node can be cancelled, so as to improve the data transmission efficiency of the distributed system.

[0101] and Figure 4 For details on the embodiments shown in the drawings, please refer to Figure 6 , Figure 6 A data transmission method is provided in an embodiment of the present application. The method includes the following steps S601 to S606, and the method is described by taking a computing node reading data from a solid state drive as an example, wherein:

[0102] S601: The computing node generates a data transmission request, where the data transmission request is a read command.

[0103] S602: The computing node sends a data transmission request to the receiving queue RQ of the storage node through the sending queue SQ of the computing node.

[0104] S603: The storage node writes the information of the data transmission request into the memory of the solid state drive based on the PCIe link.

[0105] S604: The solid state drive generates a data transmission indication according to the information of the data transmission request in the memory, where the data transmission indication is a write command.

[0106] S605: The solid state drive sends a data transmission instruction to the storage node through the PCIe link.

[0107] S606: The storage node sends a data transmission indication to the receiving queue RQ of the computing node through the sending queue SQ of the storage node.

[0108] Among them, steps S601 to S606 can refer to the description of steps S401 to S406, which will not be repeated here. In step S601, the data transmission request is a read command, which can be understood as the operation type of the data transmission request is a read type, that is, the computing node reads data from the solid-state hard disk. The data transmission indication is a write command, which is used to control the storage node to read the data to be read from the solid-state hard disk and send it to the computing node for writing. The data transmission indication is also used to indicate that the data to be read in the solid-state hard disk will be sent to the computing node and written to the computing node. In other words, the computing node receives the data to be read from the solid-state hard disk and writes it.

[0109] S607: The storage node reads the to-be-read data from the solid-state drive based on the data transmission indication, and sends the data to the computing node for writing.

[0110] In such Figure 6 In the method described, when the data transmission request is a read command, after the storage node writes the information of the data transmission request into the memory of the solid-state drive based on the PCIe link, the solid-state drive can generate a data transmission indication based on the information in the memory, thereby controlling the storage node to read the data to be read from the solid-state drive according to the data transmission indication, and write it to the computing node. In other words, the storage node does not perform memory registration on the data transmission request from the computing node in its own memory, but directly performs memory registration on the solid-state drive, which reduces the time for memory registration. And the solid-state drive generates a data transmission indication after memory registration to control the storage node to realize data transmission between the computing node and the solid-state drive. During the data transmission process, the storage node acts as a transmission medium between the computing node and the solid-state drive, and does not cache the data to be read, which reduces the time for the storage node to cache data and improves the data transmission efficiency.

[0111] In a possible example, after step S607, the data transmission method further includes: the storage node sends a data completion message to the receiving queue RQ of the computing node through the sending queue SQ of the storage node.

[0112] The data completion message is used to indicate that the data to be read in the solid-state drive is stored in the computing node. That is, after the storage node stores the data to be read in the computing node, it can send a data completion message to the computing node to indicate that the data transmission of the computing node is completed, and the task queue in the computing node can be cancelled, so as to improve the data transmission efficiency of the distributed system.

[0113] The method of the embodiment of the present application is described above, and the device of the embodiment of the present application is provided below.

[0114] Please refer to Figure 7 , Figure 7 This is a structural diagram of another distributed system provided in an embodiment of the present application, and the distributed system is described with a storage node 20 connected to a second RF circuit 101 of a computing node 10 through a network, and a solid state drive 30 connected to the storage node 20. Figure 7 As shown, the storage node 20 may include a first RF circuit 201 for communicating with the computing node 10. The storage node 20 may also include a first processor 202 connected to the first RF circuit 201 and the solid state drive 30, and a first memory 203 connected to the processor 202.

[0115] The first processor 202 can be connected to the first RF circuit 201, the first memory 203 and the solid state drive 30 through a PCIe link. As described above, after the distributed system is powered on, the first processor 202 can perform device enumeration on the first RF circuit 201, the first memory 203 and the solid state drive 30 (for example, the second memory 301 and the second processor 302), and allocate register addresses for the first RF circuit 201, the first memory 203 and the solid state drive 30, so that the first RF circuit 201 can access the memory space of the first memory 203 and the solid state drive 30 through the register address allocated in the PCIe link.

[0116] The RF circuits described in the embodiments of the present application (for example, the first RF circuit 201 and the second RF circuit 101) may be network interfaces (NICs), etc., which are not limited here. Figure 7 Not shown, it may include a receiver, a signal processing module connected to the receiver, and a transmitter connected to the signal processing module. Among them, the receiver is used to receive information sent by an external or internal processor, the signal processing module is used to process the information received by the transmitter, and the transmitter is used to send the information obtained by the signal processing module. For example, when the operation type of the data transmission request is a write type, the receiver of the first RF circuit 201 is used to receive the data transmission request and the data to be written corresponding to the data transmission request through the transmitter of the second RF circuit 101 of the computing node 10, the signal processing module of the first RF circuit 201 is used to process the data transmission request, obtain the operation type, data size, source address and target address of the data transmission request, and the transmitter of the first RF circuit 201 is used to send the information obtained by the signal processing module to the solid state drive 30.

[0117] The processors (e.g., the first processor 202, the second processor 302, and the third processor 102) and the signal processing modules (e.g., the signal processing modules in the first radio frequency circuit 201) described in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., without limitation. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor may also be a system of chip (SoC) or an embedded processor. Both the processor and the signal processing module may support the transmission of high-speed serial computer expansion bus standard (Peripheral Component Interconnect express, PCIe) messages with a capacity of at least 64 bytes.

[0118] like Figure 7 As shown, the computing node 10 includes a second RF circuit 101 for communicating with the first RF circuit 201; the computing node 10 also includes a third processor 102 connected to the second RF circuit 101, which is used to select the solid-state hard disk that finally stores the data to be written according to the data size of the data to be written and the attribute information of each solid-state hard disk when writing data, and then generate a command (for example, a data transmission instruction in the embodiment of the present application) according to the operation type and data size of the data to be written, the source address of the data to be written stored in the computing node 10, and the target address and other information; it can also be used to determine the target address in the computing node 10 according to the data size of the data to be written when reading data, and then generate a command (for example, a data transmission instruction in the embodiment of the present application) according to the operation type and data size of the data to be written, the source address of the data to be written in the solid-state hard disk, and the target address in the computing node 10. Further, the third processor 102 can also be used to monitor the network status of the RF circuit of the computing node 10 to determine the number of transmission requests executed in the current queue.

[0119] The processor in the solid-state drive 30 can be used to register the instruction in memory based on the instruction sent by the processor of the storage node 20 to determine the target address of the data to be written or the source address of the data to be read. The processor in the solid-state drive 30 can also be used to perform protocol conversion on the instruction. It can be understood that the processor in the solid-state drive 30 converts the command based on the RDMA protocol into a command based on the NVMe protocol, thereby reducing the work task of the storage node to convert the protocol, which is convenient for improving the processing efficiency of the storage node.

[0120] The memory described in the embodiments of the present application (for example, the first memory 203, the second memory 301 and the third memory 103) can be a double data rate (DDR) memory, etc., without limitation. It should be noted that the above-mentioned DDR memory is a general term for memories such as DDR1, DDR2, DDR3, DDR4, etc. The memory in the solid-state drive can also be a DDR memory. In a possible example, the memory in the storage unit can be a host memory buffer (HMB), and the memory in the solid-state drive can be a controller memory buffer (CMB).

[0121] In the embodiment of the present application, the storage node 20 and the computing node 10 communicate based on the RDMA protocol, the storage node 10 and the solid-state drive 30 are connected via a PCIe link, and the computing node 10 includes a third processor 102, a second radio frequency circuit 101, and a third memory 103, wherein:

[0122] The third processor 102 is used to generate a data transmission request, where the data transmission request is used to specify data transmission between the solid state drive 30 and the computing node 10;

[0123] The second radio frequency circuit 101 is used to send a data transmission request to the receiving queue RQ of the storage node 20 through the sending queue SQ of the computing node 10; receive a data transmission indication from the sending queue RQ of the storage node 20 through the receiving queue RQ of the computing node 10, and the data transmission indication is obtained by the solid state drive 30 according to the information of the data transmission request written by the storage node 20 in the memory of the solid state drive 30;

[0124] The third memory 103 is used to read the data to be written by the storage node 20 based on the data transmission indication, or write the data to be read from the solid state drive 30.

[0125] In a possible example, the data transfer request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol.

[0126] In a possible example, the second RF circuit 101 is also used to receive a data completion message from the sending queue SQ of the storage node 20 through the receiving queue RQ of the computing node 10, and the data completion message is used to indicate that the data to be written in the computing node 10 is written to the solid-state drive 30, or the data to be read in the solid-state drive 30 is written to the computing node 10.

[0127] In an embodiment of the present application, the storage node 20 includes a first radio frequency circuit 201 for receiving a data transmission request from a sending queue SQ of the computing node 10 through a receiving queue RQ of the storage node 20, where the data transmission request is used to specify data transmission between the solid-state drive 30 and the computing node 10; writing the information of the data transmission request into the memory of the solid-state drive 30 based on the PCIe link; receiving a data transmission indication from the solid-state drive 30 through the PCIe link, where the command type of the data transmission indication is opposite to that of the data transmission request, and the data transmission indication is obtained by the solid-state drive based on the information of the data transmission request in the memory of the solid-state drive; sending a data transmission indication to a receiving queue RQ of the computing node 10 through the sending queue SQ of the storage node 20; reading the data to be written from the computing node 10 based on the data transmission indication, and sending it to the solid-state drive 30 for writing; or reading the data to be read from the solid-state drive 30, and sending it to the computing node 10 for writing.

[0128] In a possible example, a data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol, and the first RF circuit 201 is specifically used to establish a memory queue of the storage node 20 on the solid-state drive 30 based on the PCIe link; the first command is parsed to obtain the solid-state drive 30 corresponding to the data transmission request; and the information of the second command is written into the memory queue of the storage node 20 on the solid-state drive 30, so that the information of the data transmission request is written into the memory of the solid-state drive 30.

[0129] In a possible example, the first RF circuit 201 is also used to send a data completion message to the receiving queue RQ of the computing node 10 through the sending queue SQ of the storage node 20, and the data completion message is used to indicate that the data to be written in the computing node 10 is written to the solid-state drive 30, or the data to be read in the solid-state drive 30 is written to the computing node 10.

[0130] In the above example, the data transmission process is performed by the first RF circuit 201 of the storage node 20, rather than the first processor 202 of the storage node 20. The first processor 202 acts as a transit node on the PCIe link, which improves the storage efficiency of the storage node. In addition, the first memory 203 of the storage node 20 does not cache data to be written or read, which can avoid being limited by the bandwidth pressure of the first memory 203, thereby improving the bandwidth processing capability of the storage node 20.

[0131] In an embodiment of the present application, the solid-state drive 30 includes a second processor 302, which is used to generate a data transmission indication based on the information of the data transmission request in the memory. The data transmission request is sent by the computing node 10 to the storage node 20, and is used to specify the data transmission between the solid-state drive 30 and the computing node 10. The information of the data transmission request in the memory is that the storage node 20 writes the information of the data transmission request into the memory of the solid-state drive 30 based on the PCIe link. The data transmission indication is used to control the storage node 20 to read the data to be written from the computing node 10 and send it to the solid-state drive 30 for writing; or read the data to be read from the solid-state drive 30 and send it to the computing node 10 for writing; send the data transmission indication to the storage node 20 via the PCIe link.

[0132] In a possible example, a data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol. The second processor 302 is specifically used to parse the information of the second command to obtain target information; and generate a data transmission indication based on the target information.

[0133] Please refer to Figure 8 , Figure 8 A device provided in an embodiment of the present application may include a computing node, a storage node, and a device in a solid-state hard disk. The storage node and the computing node communicate based on the RDMA protocol, and the storage node and the solid-state hard disk are connected via a PCIe link.

[0134] When the device 800 is a computing node, the processing unit 801 is used to generate a data transmission request, and the data transmission request is used to specify data transmission between the solid state drive and the computing node;

[0135] The communication unit 802 is used to send a data transmission request to the receiving queue RQ of the storage node through the sending queue SQ of the computing node; receive a data transmission indication from the sending queue RQ of the storage node through the receiving queue RQ of the computing node, and the data transmission indication is obtained by the solid state drive according to the information of the data transmission request written by the storage node in the memory of the solid state drive;

[0136] The storage unit 803 is used to read the data to be written by the storage node based on the data transmission indication, or write the data to be read from the solid state drive.

[0137] In a possible example, the data transfer request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol.

[0138] In one possible example, the communication unit 802 is also used to receive a data completion message from the sending queue of the storage node through the receiving queue RQ of the computing node, and the data completion message is used to indicate that the data to be written in the computing node is written to the solid-state hard disk, or the data to be read in the solid-state hard disk is written to the computing node.

[0139] When the device 800 is a storage node, the communication unit 802 is used to receive a data transmission request from a sending queue SQ of a computing node through a receiving queue RQ of the storage node, where the data transmission request is used to specify data transmission between the solid state drive and the computing node;

[0140] The processing unit 801 is used to write the information of the data transmission request into the memory of the solid state drive based on the PCIe link;

[0141] The communication unit 802 is used to receive a data transmission indication from the solid-state drive through a PCIe link. The data transmission indication is opposite to the command type of the data transmission request. The data transmission indication is obtained by the solid-state drive according to the information of the data transmission request in the memory of the solid-state drive; the data transmission indication is sent to the receiving queue RQ of the computing node through the sending queue SQ of the storage node;

[0142] The processing unit 801 is also used to read the data to be written from the computing node based on the data transmission indication, and send it to the solid state drive for writing; or read the data to be read from the solid state drive, and send it to the computing node for writing.

[0143] In a possible example, a data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol, and the processing unit 801 is specifically used to establish a memory queue of a storage node on a solid-state drive based on a PCIe link; parse the first command to obtain a solid-state drive corresponding to the data transmission request; write information of the second command into the memory queue of the storage node on the solid-state drive, so that the information of the data transmission request is written into the memory of the solid-state drive.

[0144] In one possible example, the communication unit 802 is also used to send a data completion message to the receiving queue RQ of the computing node through the sending queue SQ of the storage node, and the data completion message is used to indicate that the data to be written in the computing node is written to the solid-state hard disk, or the data to be read in the solid-state hard disk is written to the computing node.

[0145] When the device 800 is a solid-state drive, the processing unit 801 is used to generate a data transmission indication based on the information of the data transmission request in the memory. The data transmission request is sent by the computing node to the storage node and is used to specify data transmission between the solid-state drive and the computing node. The information of the data transmission request in the memory is that the storage node writes the information of the data transmission request into the memory of the solid-state drive based on the PCIe link. The data transmission indication is used to control the storage node to read the data to be written from the computing node and send it to the solid-state drive for writing; or read the data to be read from the solid-state drive and send it to the computing node for writing; send the data transmission indication to the storage node via the PCIe link.

[0146] In a possible example, a data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol, and the processing unit 801 is specifically used to parse the information of the second command to obtain target information; and generate a data transmission indication according to the target information.

[0147] See also Fig. 9 , Fig. 9 A schematic diagram of a device provided in an embodiment of the present application. The device 900 includes at least one processor 901, at least one memory 902, and at least one communication interface 903. The processor 901 and the communication interface 903 are connected via a communication bus 904 and communicate with each other.

[0148] The processor 901 in the device 900 is used to read the computer program code stored in the memory 902. In the embodiment of the present application, the device 900 may include a device in a computing node, a storage node, and a solid-state hard disk. The storage node and the computing node communicate based on the RDMA protocol, and the storage node and the solid-state hard disk are connected through a PCIe link.

[0149] When the device 900 is a computing node, the processor 901 is configured to perform the following operations:

[0150] Generate a data transmission request, where the data transmission request is used to specify data transmission between the solid state drive and the computing node;

[0151] Send a data transmission request to the receiving queue RQ of the storage node through the sending queue SQ of the computing node;

[0152] The receiving queue RQ of the computing node receives the data transmission indication from the sending queue RQ of the storage node. The data transmission indication is obtained by the solid-state drive according to the information of the data transmission request written by the storage node in the memory of the solid-state drive;

[0153] Based on the data transmission instruction, the storage node reads the data to be written, or writes the data to be read from the solid state drive.

[0154] In a possible example, the data transfer request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol.

[0155] In a possible example, the processor 901 is further configured to perform the following operations:

[0156] The data completion message is received from the sending queue SQ of the storage node through the receiving queue RQ of the computing node. The data completion message is used to indicate that the data to be written in the computing node is written to the solid-state drive, or the data to be read in the solid-state drive is written to the computing node.

[0157] When the device 900 is a storage node, the processor 901 is configured to perform the following operations:

[0158] The receiving queue RQ of the storage node receives a data transmission request from the sending queue SQ of the computing node, and the data transmission request is used to specify data transmission between the solid-state drive and the computing node;

[0159] Writing the information of the data transfer request into the memory of the solid-state drive based on the PCIe link;

[0160] Receive a data transfer indication from the solid-state drive through a PCIe link. The data transfer indication is of the opposite command type to the data transfer request. The data transfer indication is obtained by the solid-state drive based on the information of the data transfer request in the memory of the solid-state drive.

[0161] Sending a data transmission indication to a receiving queue RQ of a computing node through a sending queue SQ of a storage node;

[0162] Based on the data transmission indication, the data to be written is read from the computing node and sent to the solid state drive for writing; or the data to be read is read from the solid state drive and sent to the computing node for writing.

[0163] In a possible example, the data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol, and the processor 901 is specifically configured to perform the following operations:

[0164] The memory queue of the storage node is established on the solid-state drive based on the PCIe link;

[0165] Parse the first command to obtain a solid state drive corresponding to the data transmission request;

[0166] The information of the second command is written into the memory queue of the storage node on the solid state drive, so that the information of the data transmission request is written into the memory of the solid state drive.

[0167] In a possible example, the processor 901 is further configured to perform the following operations:

[0168] A data completion message is sent through the sending queue SQ of the storage node to the receiving queue RQ of the computing node. The data completion message is used to indicate that the data to be written in the computing node is written to the solid-state drive, or the data to be read in the solid-state drive is written to the computing node.

[0169] When the device 900 is a solid state drive, the processor 901 is configured to perform the following operations:

[0170] Generate a data transfer indication according to the information of the data transfer request in the memory. The data transfer request is sent by the computing node to the storage node and is used to specify data transfer between the solid-state drive and the computing node. The information of the data transfer request in the memory is written by the storage node into the memory of the solid-state drive based on the PCIe link. The data transfer indication is used to control the storage node to read the data to be written from the computing node and send it to the solid-state drive for writing; or read the data to be read from the solid-state drive and send it to the computing node for writing;

[0171] Send data transfer instructions to the storage node through the PCIe link.

[0172] In a possible example, the data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol, and the processor 901 is specifically configured to perform the following operations:

[0173] Parse the information of the second command to obtain target information;

[0174] Generate a data transmission instruction based on the target information.

[0175] The embodiment of the present application also provides a chip, including a processor and a memory, wherein the processor is used to call and execute instructions stored in the memory from the memory, so that a device equipped with the chip executes Figure 4 and Figure 6 Any of the methods shown.

[0176] The present application also provides another chip, including: an input interface, an output interface and a processing circuit, wherein the input interface, the output interface and the processing circuit are connected through an internal connection path, and the processing circuit is used to execute Figure 4 and Figure 6 Any of the methods shown.

[0177] The embodiment of the present application also provides another chip, including: an input interface, an output interface, a processor, and optionally, a memory, wherein the input interface, the output interface, the processor, and the memory are connected through an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute Figure 4 and Figure 6 Any of the methods shown.

[0178] The embodiment of the present application also provides a chip system, including at least one processor, a memory and an interface circuit, wherein the memory, the transceiver and the at least one processor are interconnected through a line, and a computer program is stored in the at least one memory; when the computer program is executed by the processor, Figure 4 and Figure 6 The method flow shown is realized.

[0179] The present application also provides a computer-readable storage medium in which a computer program is stored. When the computer-readable storage medium is run on a computer, Figure 4 and Figure 6 The method flow shown is realized.

[0180] The present application also provides a computer program product. When the computer program product is run on a computer, Figure 4 and Figure 6 The method flow shown is realized.

[0181] The present application also provides a distributed system, including Figure 4 and Figure 6 The storage node, solid-state drive, and computing node in the method shown.

[0182] In summary, by implementing the embodiments of the present application, after the storage node receives a data transmission request from the sending queue SQ of the computing node through its RQ queue, the information of the data transmission request is written into the memory of the solid-state drive based on the PCIe link. In this way, the solid-state drive can generate a data transmission indication based on the information in the memory, thereby controlling the storage node to read the data to be written from the computing node according to the data transmission indication, and send it to the solid-state drive for storage; or read the data to be read from the solid-state drive and send it to the computing node for writing. In other words, the storage node does not register the data transmission request from the computing node in its own memory, but directly registers it in memory on the solid-state drive, reducing the time for memory registration. And the solid-state drive generates a data transmission indication after the memory registration to control the storage node to realize the data transmission between the computing node and the solid-state drive. During the data transmission process, the storage node acts as a transmission medium between the computing node and the solid-state drive, and does not cache the data to be written or read, which reduces the time for the storage node to cache data and improves the data transmission efficiency.

[0183] It should be understood that the first, second, third, fourth and various digital numbers involved in this document are only distinguished for the convenience of description and are not used to limit the scope of the embodiments of the present application. The digital numbers can be replaced by other digital numbers.

[0184] It should be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0185] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0186] Those of ordinary skill in the art will appreciate that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0187] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0188] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state drive), etc.

[0189] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data transmission method, characterized in that: The storage node and the computing node communicate with each other based on the remote process direct data access (RDMA) protocol, the storage node and the solid state drive are connected via a peripheral device high-speed connection bus PCIe link, and the data transmission method includes: The storage node receives a data transmission request from the sending queue SQ of the computing node through the receiving queue RQ of the storage node, wherein the data transmission request is used to specify data transmission between the solid state drive and the computing node; The storage node writes the information of the data transmission request into the memory of the solid state drive based on the PCIe link; The storage node receives a data transmission indication from the solid-state drive through the PCIe link, wherein the data transmission indication is of a command type opposite to that of the data transmission request, and the data transmission indication is obtained by the solid-state drive according to information of the data transmission request in the memory of the solid-state drive; The storage node sends the data transmission indication to the receiving queue RQ of the computing node through the sending queue SQ of the storage node; The storage node reads the data to be written from the computing node based on the data transmission indication, and sends the data to the solid state drive for writing; or reads the data to be read from the solid state drive, and sends the data to the computing node for writing.

2. The data transmission method according to claim 1, characterized in that: The data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, the second command is based on the non-volatile transmission bus NVMe protocol, and the storage node writes the information of the data transmission request into the memory of the solid-state drive based on the PCIe link, including: The storage node establishes a memory queue of the storage node on the solid state drive based on the PCIe link; The storage node parses the first command to obtain a solid state drive corresponding to the data transmission request; The storage node writes the information of the second command into a memory queue of the storage node on the solid state drive, so that the information of the data transmission request is written into the memory of the solid state drive.

3. The data transmission method according to claim 1 or 2, characterized in that: Reading the data to be written from the computing node at the storage node based on the data transmission indication, and sending the data to the solid state drive for writing; Alternatively, after reading the data to be read from the solid state drive and sending it to the computing node for writing, the method further includes: The storage node sends a data completion message to the receiving queue RQ of the computing node through the sending queue SQ of the storage node, and the data completion message is used to indicate that the data to be written in the computing node is written to the solid-state drive, or the data to be read in the solid-state drive is written to the computing node.

4. The data transmission method according to claim 1 or 2, characterized in that: The storage node includes a radio frequency circuit.

5. A data transmission method, characterized in that: The storage node and the computing node communicate with each other based on the RDMA protocol, the storage node and the solid-state hard disk are connected via a PCIe link, and the data transmission method includes: The solid-state drive generates a data transmission indication according to the information of the data transmission request in the memory, the data transmission request is sent by the computing node to the storage node, and is used to specify data transmission between the solid-state drive and the computing node, the information of the data transmission request in the memory is written by the storage node into the memory of the solid-state drive based on the PCIe link, and the data transmission indication is used to control the storage node to read the data to be written from the computing node and send it to the solid-state drive for writing; or read the data to be read from the solid-state drive and send it to the computing node for writing; The solid state drive sends the data transmission indication to the storage node through the PCIe link.

6. The data transmission method according to claim 5, characterized in that: The data transmission request includes a first command and a second command, the first command is based on the RDMA protocol, the second command is based on the non-volatile transmission bus NVMe protocol, the information of the data transmission request in the memory is information that the storage node writes the information of the second command into the memory queue of the storage node on the solid-state drive, and the solid-state drive generates a data transmission indication according to the information of the data transmission request in the memory, including: The solid state drive parses the information of the second command to obtain target information; The solid state drive generates a data transmission instruction according to the target information.

7. A data transmission method, characterized in that: The storage node and the computing node communicate with each other based on the RDMA protocol, the storage node and the solid-state hard disk are connected via a PCIe link, and the data transmission method includes: The computing node generates a data transmission request, where the data transmission request is used to specify data transmission between the solid state drive and the computing node; The computing node sends the data transmission request to the receiving queue RQ of the storage node through the sending queue SQ of the computing node; The computing node receives a data transmission indication from the sending queue RQ of the storage node through the receiving queue RQ of the computing node, wherein the data transmission indication is obtained by the solid state drive according to information of the data transmission request written by the storage node in the memory of the solid state drive; The computing node reads the to-be-written data from the storage node based on the data transmission indication, or writes the to-be-read data from the solid-state drive.

8. The data transmission method according to claim 7, characterized in that: The data transfer request includes a first command and a second command, wherein the first command is based on the RDMA protocol, and the second command is based on the NVMe protocol.

9. A device, characterized in that: include: A unit corresponding to the data transmission method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The device is used for storing a computer program, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 8.

11. A computer program product, characterized in that The computer program product is used to store a computer program, and when the computer program is run on a computer, the computer is enabled to execute the method according to any one of claims 1 to 8.

12. A distributed system, characterized in that: include: The storage node, solid state drive, and computing node according to any one of claims 1 to 8.

Images