Network adapter with efficient storage protocol emulation

Abstract

The present disclosure relates to network adapters with high performance storage protocol emulation. A network adapter includes a network interface controller and a processor. The network interface controller communicates with a host over a peripheral bus and with a remote storage device over a network. The processor exposes a peripheral bus device that uses a bus storage protocol to communicate with the host on the peripheral bus to receive a first I / O transaction of the bus storage protocol from the host via the exposed peripheral bus device and to complete the first I / O transaction in the remote storage device by (i) translating between the first I / O transaction and a second I / O transaction of a network storage protocol and (ii) performing the second I / O transaction in the remote storage device. To receive and complete the first I / O transaction, the processor causes the network interface controller to transfer data directly between the remote storage device and a memory of the host using zero-copy.

Description

Technical Field

[0001] This invention relates generally to data storage, and more particularly to methods and systems for simulating remote storage. Background Technology

[0002] Various techniques for data storage using network adapters are known in the art. For example, U.S. Patents 9,696,942 and 9,727,503 describe techniques for accessing remote storage devices using a local bus protocol. The disclosed methods include configuring a driver on a host computer to receive commands according to a protocol defined for accessing a local storage device connected to a peripheral component interface bus of the host computer. When the driver receives a storage access command specifying a storage transaction according to the protocol from an application running on the host computer, a remote direct memory access (RDMA) operation is performed by a network interface controller (NIC) connected to the host computer to execute the storage transaction over the network on the remote storage device.

[0003] U.S. Patent 10,657,077 describes a hyperconverged NVMF storage NIC card. A storage communication device inserted into a server includes a circuit board, a bus interface, a media access control (MAC) processor, one or more storage devices, and at least one central processing unit (CPU). The bus interface is configured to connect the device to at least the server's processor. The MAC is mounted on the circuit board and configured to connect to a communication network. The storage devices are mounted on the circuit board and configured to store data. The CPU is mounted on the circuit board and configured to expose the storage devices (i) to the server's processor via the bus interface, and (ii) indirectly to other servers via the communication network. Summary of the Invention

[0004] Embodiments of the present invention described herein provide a network adapter including a network interface controller and a processor. The network interface controller communicates with a host via a peripheral bus and with a remote storage device via a network. The processor exposes a peripheral bus device on the peripheral bus that communicates with the host using a bus storage protocol to receive a first input / output (I / O) transaction of the bus storage protocol from the host via the exposed peripheral bus device, and completes the first I / O transaction of the host in the remote storage device by (i) translating between the first I / O transaction of the bus storage protocol and a second I / O transaction of the network storage protocol, and (ii) executing the second I / O transaction in the remote storage device. To receive and complete the first I / O transaction, the processor enables the network interface controller to directly transfer the data of the first I / O transaction and the second I / O transaction between the memory of the remote storage device and the host using zero-copy transfer.

[0005] In this embodiment, the bus storage protocol is Non-Volatile Memory Fast (NVMe), and the network storage protocol is NVMe-over-TCP (NVMe-based). In the disclosed embodiment, the processor processes the packet headers of the network storage protocol and exchanges the packet headers of the network storage protocol with the remote storage device, while the network interface controller directly exchanges data between the remote storage device and the host's memory. In an example embodiment, to enable the network interface controller to directly transfer data, the processor maps a memory buffer in the host's memory to a Direct Memory Access (DMA) address in the network interface controller.

[0006] In another embodiment, when transmitting a network storage protocol packet to a remote storage device fails, the processor retrieves the packet data from the host's memory and retransmits the packet containing the retrieved data. In yet another embodiment, when transmitting data directly from the host's memory to a remote storage device, the network interface controller calculates a cyclic redundancy check (CRC) on the transmitted data and transmits the CRC along with the data.

[0007] In some embodiments, when data is transferred directly from a remote storage device to the host's memory, the network interface controller verifies the Cyclic Redundancy Check (CRC) received with the data. In an example embodiment, if the CRC cannot be verified, the network interface controller triggers the processor to verify the CRC, and the processor retrieves at least some of the data from the host's memory and uses the retrieved data to verify the CRC.

[0008] According to embodiments of the present invention, a method is also provided in a network adapter including a network interface controller and a processor. The method includes: using the network interface controller of the network adapter to communicate with a host via a peripheral bus and with a remote storage device via a network. The processor using the network adapter (i) exposes a peripheral bus device communicating with the host using a bus storage protocol on the peripheral bus, (ii) receives a first input / output (I / O) transaction of the bus storage protocol from the host via the exposed peripheral bus device, and (iii) completes the first I / O transaction for the host in the remote storage device by (a) translating between the first I / O transaction of the bus storage protocol and a second I / O transaction of the network storage protocol, and (b) executing the second I / O transaction in the remote storage device. Receiving and completing the first I / O transaction includes causing the network interface controller to directly transfer data of the first I / O transaction and the second I / O transaction between the memory of the remote storage device and the host using zero-copy transfer.

[0009] According to embodiments of the present invention, a method for performing input / output (I / O) transactions of a host in a remote storage device using a network adapter is also provided. The method includes: communicating with a host via a peripheral bus and with a remote storage device via a network using a network interface controller in the network adapter. Using a processor in the network adapter, receiving I / O transactions of a bus storage protocol from the host, and completing the I / O transactions for the host in the remote storage device by translating between the bus storage protocol and a network storage protocol. Receiving and completing the I / O transactions includes enabling the network interface controller to transfer the data of the I / O transactions directly between the memory of the remote storage device and the host using zero-copy transfer. Attached Figure Description

[0010] The invention will be more fully understood from the following detailed description of the embodiments in conjunction with the accompanying drawings, in which:

[0011] Figure 1 This is a schematic block diagram illustrating a computing and storage system according to an embodiment of the present invention;

[0012] Figure 2 This schematically illustrates an embodiment of the invention. Figure 1 A simulation diagram of the read command in the system;

[0013] Figure 3 This schematically illustrates an embodiment of the invention. Figure 1 A diagram illustrating write commands in the system; and

[0014] Figure 4 This schematically illustrates an embodiment of the invention. Figure 1 The flowchart shows the method for executing read commands in the system. Detailed Implementation

[0015] Overview

[0016] The embodiments of the invention described herein provide an improved data storage technique in which a network adapter stores and retrieves data for a host in a remote storage device. The network adapter communicates with the host via a peripheral bus (e.g., a Fast Peripheral Component Interconnect (PCIe) bus) and with the remote storage device via a network.

[0017] In the disclosed embodiments, the network device "emulates" a host-based bus storage protocol (e.g., Non-volatile Memory Fast-NVMe) and simultaneously uses a network storage protocol (e.g., NVMe over TCP based on Transmission Control Protocol) to store and retrieve data in a remote storage device. By way of example, the following description will primarily refer to the NVMe and NVMe over TCP protocols.

[0018] From the host's perspective, the network adapter appears and behaves as a local storage device, such as an NVMe drive connected to the host's PCIe bus. However, in reality, the network adapter receives NVMe commands from the host, translates these commands into NVMe-over-TCP commands, and executes them on the remote storage device. (The terms "command" and "I / O transaction," or simply "transaction," are used interchangeably here.)

[0019] In some embodiments, the network adapter is a data processing unit (DPU) that includes a network interface controller (NIC) and a powerful processor (also referred to herein as a "DPU processor" for clarity). In an example embodiment, the processor includes an array of ARM cores.

[0020] In some embodiments, the DPU processor exposes a dedicated PCIe device emulating the NVMe protocol to the host via the PCIe bus. The host issues NVMe commands (e.g., read and / or write commands) and sends them to the dedicated PCIe device, similar to how the host would issue NVMe commands to a local disk. The DPU processor receives NVMe commands from the host via the dedicated PCIe device and completes the host's NVMe commands by (i) translating between NVMe commands and NVMe-over-TCP commands, and (ii) executing NVMe-over-TCP commands on the remote storage device.

[0021] As will be explained in detail below, the NIC in the DPU uses zero-copy transfer, directly transferring NVMe and NVMe-over-TCP transaction data between the remote storage device and the host's memory. In this context, the term "zero-copy" means that data is transferred directly between the host's memory and the remote storage device, without intermediate storage in the DPU. Zero-copy data transfer significantly reduces overall transaction latency and increases achievable throughput. An example of such technology is described in U.S. Patent Application 17 / 338,131, filed June 3, 2021, entitled "Cross Bus Memory Mapping," the disclosure of which is incorporated herein by reference.

[0022] In some embodiments, the DPU processor enables the NIC to perform such direct zero-copy data transfers by mapping a memory buffer in the host memory to a corresponding direct memory access (DMA) address of the NIC. Once the mapping is ready, the NIC can directly access the host's memory using the host's own address space. Such technology is described in U.S. Patent Application 17 / 189,303, filed March 2, 2021, entitled "Cross Address-Space Bridging," the disclosure of which is incorporated herein by reference.

[0023] In some embodiments, the DPU processor runs a TCP protocol stack and establishes a TCP connection with a remote storage device. The DPU then performs NVMe-over-TCP transactions by exchanging TCP packets with the remote storage device via the TCP connection. Internally, the NIC transmits packet data and packet headers via different paths. As described above, using zero-copy transmission, data is transmitted directly from / to host memory via the NIC. Packet headers are transmitted from / to the DPU processor by the NIC for terminating and otherwise processing the TCP connection. This technique avoids the need to copy packet data between the NIC and the DPU processor, thereby reducing latency and increasing throughput.

[0024] When using the disclosed configuration, I / O transaction data typically does not pass through the DPU processor. Therefore, in some embodiments, the NIC is responsible for calculating and verifying the Cyclic Redundancy Check (“CRCs”) associated with the data. During transmission, the NIC typically calculates the CRCs for the data and sends the data and the corresponding CRCs to the remote storage device. During reception, the NIC typically receives the data and the corresponding CRCs from the remote storage device, calculates the CRCs for the received data, and verifies that the calculated CRCs match the received CRCs.

[0025] This document describes several example implementations of the disclosed techniques. Supplementary techniques, such as fallback schemes for the DPU processor to gain access to data in I / O transactions, are also described. Such fallback schemes are used, for example, to implement TCP retransmissions and to recover from CRC failures caused by out-of-order packet arrivals.

[0026] System Description

[0027] Figure 1This is a schematic block diagram of a computing and storage system 20 according to an embodiment of the present invention. System 20 includes a high-performance network adapter 24, also known as a data processing unit (DPU), which serves a host 28. The DPU 24 communicates with the host 28 via a peripheral bus (in this example, a Fast Peripheral Component Interconnect (PCIe) bus 36).

[0028] DPU 24 provides data storage services to host 28 and may perform other tasks, such as in addition to existing networking services. Figure 1 In the example, DPU 24 connects host 28 to network 32 and stores and retrieves data from host 28 in remote storage device 40 via network 32.

[0029] In this example, host 28 includes host CPU 44 and memory 48. Host CPU 44 may run various software applications (“apps”) 52 depending on applicable use cases. In other tasks, application 52 issues input / output (I / O) transactions, such as writing data to or reading data from a file or object, or creating, modifying, or deleting files, directories, objects, etc. Typically, I / O transactions can be issued by any software running on host CPU 44, such as by a virtual machine (VM), process, container, host operating system, or any other software.

[0030] exist Figure 1 In this embodiment, the host CPU 44 runs an NVMe drive 56 that provides data storage services to various types of software running on the host CPU, such as application 52. Drive 56 stores and retrieves data for application 52 by issuing NVMe commands to one or more NVMe devices that can be connected to the PCIe bus 36. Specifically, drive 56 issues NVMe commands to a dedicated NVMe PCIe device 60 exposed by DPU 24, as will be described below.

[0031] For clarity, PCIe device 60 is depicted within the host 28 to highlight the interaction between device 60 and host CPU 44. However, in reality, PCIe device 60 is a logical interface presented to host 28 by DPU 24 via bus 36. Therefore, the terms "PCIe device" and "PCIe interface" are used interchangeably. PCIe device 60 may include PCIe physical or virtual functions. The interaction between host CPU 44 and PCIe device 60 can be implemented using a standard operating system (OS) driver or, depending on the circumstances, a vendor-specific driver.

[0032] like Figure 1As shown, DPU 24 includes a network interface controller (NIC) 64 and a DPU processor 68. In the example embodiment, NIC 64 is implemented using an application-specific integrated circuit (ASIC), and the DPU processor 68 includes multiple interconnected ARM processing cores running suitable software. However, alternatively, any other suitable implementation may be used.

[0033] In some embodiments, DPU 24 emulates the NVMe storage protocol for host 28. However, for the actual execution of I / O transactions, DPU 24 uses NVMe-over-TCP to communicate with remote storage device 40 via network 32. The operation of DPU 24 is described in detail below. Generally, NIC 64 uses zero-copy transfer to directly transfer NVMe and NVMe-over-TCP transaction data between remote storage device 40 and host memory 48. DPU processor 68 enables (or causes) NIC 64 to perform this zero-copy data transfer by mapping memory buffers in host memory 48 to corresponding DMA addresses of NIC 64. The mapping allows NIC 64 to directly read and write packet data in host memory using the host's address space.

[0034] Figure 1 The configuration of system 20 and its components (e.g., DPU 24 and host 28) shown is an example configuration depicted purely for conceptual clarity. Any other suitable configuration may be used in alternative embodiments. The various elements of system 20 and its components (e.g., DPU 24 and host 28) may be implemented using software, using suitable hardware such as in one or more application-specific integrated circuits (ASICs) or field-programmable gate arrays (FPGAs), or using a combination of software and hardware elements.

[0035] Typically, the host CPU 44 and DPU processor 68 include programmable processors that are programmed in software to perform the functions described herein. The software may be downloaded electronically to either of the processors, for example via a network, or it may alternatively or additionally be set up and / or stored on a non-transitory tangible medium, such as magnetic, optical, or electronic memory.

[0036] Storage emulation that eliminates the need for data copying between the NIC and DPU processor

[0037] As described above, DPU 24 receives NVMe commands from host 28 via PCIe device 60, translates the NVMe commands into NVMe-over-TCP commands, and executes the NVMe-over-TCP commands on remote storage device 40. In some embodiments, NIC 64 directly transmits packet data (instead of headers and / or other metadata, as appropriate) between host 28 and remote storage device 40. During normal operation, packet data does not pass through DPU processor 68, thus eliminating the need for copying or otherwise transmitting data between NIC 64 and DPU processor 68. This solution significantly reduces latency during transmission (e.g., writing data from the host to the remote storage device) and reception (e.g., reading data from the remote storage device to the host).

[0038] Figure 2 This diagram schematically illustrates a simulation of a read command in system 20 according to an embodiment of the present invention. The diagram focuses on a specific stage of the execution of the read command—the transfer of the requested data from remote storage device 40 to host 28 via DPU 24. A complete description of the read process, including the preceding stages, is provided below. Figure 4 It is described in the text.

[0039] According to NVMe-over-TCP, remote storage device 40 sends the requested data to DPU 24 in one or more TCP Protocol Data Units (PDUs). (The terms PDU and packet are sometimes used interchangeably herein.) Each PDU includes a packet header and packet data. Additionally, remote storage device 40 generates and sends CRC packets, each including a CRC code calculated for one or more PDUs.

[0040] The path traversed by PDU data, PDU header, and CRC packets in Figure 2 The bold arrows are shown in the diagram. As can be seen, PDU data is received by NIC 64 and transmitted directly to host memory 48 without going through DPU processor 68. PDU headers and CRC packets are received by NIC 64 and transmitted to DPU processor 68.

[0041] DPU processor 68 typically runs the TCP protocol stack and serves as the TCP endpoint for the TCP connection between DPU 24 and remote storage device 40. As part of running the protocol stack, DPU processor 68 terminates TCP connections, maintains the state of TCP connections, and so on, by running appropriate state machines. To perform these tasks, it is sufficient for DPU processor 68 to receive PDU headers and CRCs; it does not need to receive PDU data.

[0042] Figure 3This diagram schematically illustrates a simulation of a write command in system 20 according to an embodiment of the present invention. Here, the diagram also focuses on a specific stage of the execution of the write command—the transfer of data for storage from host 28 to remote storage device 40 via DPU 24.

[0043] exist Figure 3 In the example, DPU 24 sends data to remote storage device 40 in one or more TCP PDUs. Each PDU includes a packet header and packet data. Additionally, DPU 24 generates and sends CRC packets, each including a CRC code calculated for one or more PDUs.

[0044] Figure 3 The thick arrows in the diagram illustrate the different paths traversed by the PDU data, PDU header, and CRC packets. As can be seen, the PDU data is transmitted directly from the host memory 48 to the remote storage device 40 via NIC 64. The PDU header is transmitted from the DPU processor 68 to the remote storage device 40 by NIC 64. The CRC packet is generated in NIC 64 and sent to the remote storage device 40.

[0045] Figure 2 and Figure 3 The example demonstrates that when performing read and write commands using the disclosed technology, PDU data does not pass through the DPU processor 68. This eliminates the need for data copying between the NIC 64 and the DPU processor 68.

[0046] Figure 4 This schematically illustrates an embodiment of the invention. Figure 1 The flowchart illustrates a method for executing a read command in the system. The method begins with the host CPU 44 issuing an NVMe read command in command issuance phase 80. Typically, the NVMe command is sent from the NVMe driver 56 to the NVMe PCIe device 60. Figure 1 This command typically specifies (i) the address of the requested data to be read from it (in the address space of the NVMe device emulated by DPU 24), and (ii) the address of the memory buffer in the host memory 48 where the data will be extracted.

[0047] In mapping phase 84, the DPU processor 68 maps the address of the host memory buffer specified in the NVMe read command to the corresponding direct memory access (DMA) address of the DPU NIC 64. This mapping enables the DPU NIC 64 to directly write data to the appropriate buffer in host memory 48 using the address space of the host CPU 44 itself. In some embodiments, the DPU processor 68 maps the physical address of the memory buffer in host memory 48 to the DMA address of the DPU NIC 64. The mapping can be predefined or performed on demand using a mechanism such as NIC on-demand paging. In this way, the DPU processor 68 can instruct the DPU NIC 64 to write the retrieved data to the appropriate physical address in host memory 44.

[0048] In the protocol translation phase 88, the DPU processor 68 translates NVMe read commands into NVMe-over-TCP read requests. Translation is typically one-to-one, meaning a given NVMe read command is translated into a corresponding NVMe-over-TCP read request, but the disclosed techniques are by no means limited to one-to-one translation. In the request sending phase 92, the DPU NIC 64 sends an NVMe-over-TCP read request to the remote storage device 40 via network 32.

[0049] In response processing phase 96, DPU NIC 64 receives a response packet from remote storage device 40 in response to an NVMe-over-TCP read request. The response packet includes the requested data among other data. DPU NIC 64 writes the data directly to the appropriate address in host memory 48 (the address mapped in phase 84). Neither the host CPU 44 nor the DPU processor 44 is involved in this direct transfer of data.

[0050] Figure 4 The method flow described is an illustrative flow for the sake of conceptual clarity. In alternative embodiments, any other suitable flow may be used to perform the disclosed techniques.

[0051] For example, NVMe write commands can be processed by DPU 24 in a similar manner. In the case of a write command, the command specifies (i) a host memory buffer holding the data to be written, and (ii) the address in which the data is to be written (in the address space of the NVMe device emulated by DPU 24). DPU processor 68 maps the host memory buffer to the corresponding DMA address of DPU NIC 64. Using this mapping, DPU NIC 64 is able to write data directly from the host memory buffer to the remote storage device 40. When processing an NVMe write command, DPU processor 68 translates the NVMe write command into an NVMe-over-TCP write request. The NVMe-over-TCP write request transmits the data to be written. DPU NIC 64 writes the data from the host memory buffer to the remote storage device by sending the write request via network 32.

[0052] Typically, during execution Figure 4 During the process or the NVMe write command process given above, DPU24 uses different paths for packet data and packet headers, such as... Figure 2 and 3 As described in [the text].

[0053] Unloading of CRC operations

[0054] As explained in detail herein, when DPU 24 transfers data between host 28 and remote storage device 40, the data is transmitted via DPU NIC 64 and typically does not reach DPU processor 68. Therefore, in some embodiments, Cyclic Redundancy Check (CRC) calculation and verification (which, by definition, depends on the data) are performed by NIC 64. This solution avoids transferring data from NIC 64 to DPU processor 68 for CRC operations, and the resulting degradation in latency and throughput.

[0055] In this embodiment, NIC 64 calculates CRCs when completing a write command from host 28 in remote storage device 40. Typically, (see...) Figure 3 NIC 64 receives write command data from host 28 and PDU headers from DPU processor 68. NIC 64 assembles NVMe-over-TCP packets based on the data and headers, calculates CRCs for the data in a set of packets, and sends the NVMe-over-TCP packets to the remote storage device.

[0056] Alternatively or concurrently, the NIC 64 verifies the correctness of the CRCs received from the remote storage device as part of completing the read command. Typically (see...) Figure 2NIC 64 receives NVMe-over-TCP packets from remote storage device 40. NVMe-over-TCP packets include PDU data, a PDU header, and CRCs calculated by the remote storage device for the PDU data. NIC 64 calculates the CRCs for the received data and verifies that the calculated CRCs match the received CRCs.

[0057] A mismatch between the received CRC and the corresponding calculated CRC (the CRC calculated by the NIC for the received data) indicates a reception error, or in some cases, out-of-order packet arrival. Typically, when a CRC mismatch (“CRC failure”) is detected, the NIC 64 sends an error message to the DPU processor 68.

[0058] A fallback scenario with data access handled by the DPU processor.

[0059] In the embodiments described herein, PDU data does not reach the DPU processor 68 during normal processing of read and write commands. This feature provides a considerable improvement in throughput and latency because it eliminates the need to copy large amounts of data within the DPU 24, between the NIC 64 and the processor 68. However, in some exceptional scenarios, the DPU processor 68 may temporarily gain access to at least some of the data. Such access is required, for example, when performing TCP-level operations that depend on the data. Such operations are relatively rare, and the resulting latency is acceptable.

[0060] Mitigating CRC failures caused by out-of-order packet arrivals

[0061] One example scenario of this type involves a CRC failure due to out-of-order arrival of packets. Consider a scenario where the DPU24 receives a sequence of NVMe-over-TCP packets and further receives the CRC calculated for that sequence. However, for some reason, some of the packets in the sequence arrive out of order (i.e., in a different order than the order in which the remote storage device sent them). Such an event will result in a CRC mismatch at NIC 64 (because the NIC calculates the CRC based on the packet arrival order, while the remote storage device calculates the CRC based on the packet sending order).

[0062] In principle, DPU 24 could simply treat the entire packet sequence as a failure. However, this solution is very wasteful and requires a lot of data retransmission. Furthermore, CRC failures typically require a full NVMe-over-TCP reconnection, not just retransmission. In some embodiments, DPU 24 mitigates this situation by allowing DPU processor 68 to access data in host memory 48 without retransmission. This solution leverages the fact that in host memory, packet data is organized sequentially according to packet sequence numbers, regardless of whether the packets arrive in order. NIC 64, operating packet by packet and unaware of packet sequence numbers, lacks this ordering information.

[0063] Therefore, in some embodiments, when a CRC failure is detected, NIC 64 triggers DPU processor 68 to attempt and resolve the failure. DPU processor 68 accesses the data in the packet sequence in host memory 48 and recalculates the CRC on that data, but this time knowing that the data is organized in the correct order. If the CRC recalculated by the DPU processor matches the CRC received from the remote storage device, DPU processor 68 asserts that the CRC verification was successful, and the PDU data can be securely provided to the requesting application 52. If the CRC recalculated by the DPU processor still does not match the CRC received from the remote storage device, DPU processor asserts that the CRC mismatch resolution has failed and reverts to NVME-over-TCP reconnection and retransmission.

[0064] TCP packet retransmission

[0065] Another example scenario involves TCP retransmission. Consider a scenario where DPU 24 has already transmitted a sequence of NVMe-over-TCP packets to remote storage device 40, but a given packet was not correctly received at the remote storage device and needs to be retransmitted. For example, the DPU could receive a NACK message indicating that the packet needs to be retransmitted. Typically, NIC 64 cannot retransmit packets autonomously because (i) it does not have the data for the failed packet, and (ii) retransmission not only requires retransmitting the data but also requires rebuilding the TCP connection state when sending the original packet.

[0066] Therefore, in some embodiments, packet retransmission is performed by a DPU processor 68 running the TCP protocol stack as described above. In an embodiment, the DPU processor retrieves the PDU data of the packet from the host memory 48, reconstructs the TCP connection state from its internal protocol stack, and then retransmits the packet in the correct state.

[0067] While the embodiments described herein are primarily for NVMe and NVMe-over-TCP protocols, the methods and systems described herein can also be used with other bus storage protocols and have network variants, such as iSCSI and iSCSI-over-TCP.

[0068] Therefore, it should be understood that the above embodiments are cited by way of example, and the invention is not limited to what is specifically shown and described above. More precisely, the scope of the invention includes combinations and sub-combinations of the various features described above, as well as variations and modifications thereof that would occur to those skilled in the art upon reading the above description and which are not disclosed in the prior art. References to documents incorporated herein by reference are considered part of this application, except where any terms are defined in such incorporated documents in a manner that conflicts with the express or implied definitions in this specification; only the definitions in this specification should be considered.

Claims

1. A network adapter, comprising: A network interface controller is used to communicate with the host via a peripheral bus and with remote storage devices via a network. as well as Processor, used for: Expose peripheral bus devices that communicate with the host using a bus storage protocol on the peripheral bus; Receive the first input / output I / O transaction of the bus storage protocol from the host via the exposed peripheral bus device; as well as The first I / O transaction for the host in the remote storage device is completed by (i) translating between a first I / O transaction of the bus storage protocol and a second I / O transaction of the network storage protocol, and (ii) executing the second I / O transaction in the remote storage device. The second I / O transaction of the network storage protocol includes packets with packet headers and packet data. In order to receive and complete the first I / O transaction, the processor enables the network interface controller to use zero-copy transmission to directly transmit the packet data of the second I / O transaction between the remote storage device and the host's memory, and to transmit the packet header of the second I / O transaction between the remote storage device and the processor.

2. The network adapter of claim 1, wherein the bus storage protocol is non-volatile memory fast NVMe, and wherein the network storage protocol is based on NVMe (NVMe-over-TCP).

3. The network adapter of claim 1, wherein the processor processes the packet headers of the Network Storage Protocol and exchanges the packet headers of the Network Storage Protocol with the remote storage device.

4. The network adapter according to claim 1, wherein, In order for the network interface controller to directly transmit the packet data, the processor maps the memory buffer in the host's memory to the direct memory access (DMA) address in the network interface controller.

5. The network adapter according to claim 1, wherein, When transmission of a packet of the network storage protocol to the remote storage device fails, the processor retrieves the packet data of the packet from the memory of the host and retransmits the packet containing the retrieved packet data.

6. The network adapter according to claim 1, wherein, When transmitting the packet data directly from the host's memory to the remote storage device, the network interface controller calculates a cyclic redundancy check (CRC) on the transmitted packet data and transmits the CRC along with the packet data.

7. The network adapter according to claim 1, wherein, When transmitting the packet data directly from the remote storage device to the host's memory, the network interface controller verifies the Cyclic Redundancy Check (CRC) received along with the packet data.

8. The network adapter according to claim 7, wherein, If the CRC cannot be verified, the network interface controller triggers the processor to verify the CRC, and the processor retrieves at least some of the packet data from the host's memory and uses the retrieved packet data to verify the CRC.

9. A method in a network adapter including a network interface controller and a processor, the method comprising: The network interface controller using the network adapter communicates with the host via a peripheral bus and with a remote storage device via a network; as well as The processor using the network adapter (i) exposes a peripheral bus device on the peripheral bus that communicates with the host using a bus storage protocol, (ii) receives a first input / output I / O transaction of the bus storage protocol from the host via the exposed peripheral bus device, and (iii) completes the first I / O transaction for the host in the remote storage device by (a) translating between the first I / O transaction of the bus storage protocol and a second I / O transaction of the network storage protocol, and (b) executing the second I / O transaction in the remote storage device. The second I / O transaction of the network storage protocol includes packets with packet headers and packet data. Receiving and completing the first I / O transaction includes: enabling the network interface controller to use zero-copy transmission to directly transmit the packet data of the second I / O transaction between the remote storage device and the host's memory, and transmitting the packet header of the second I / O transaction between the remote storage device and the processor.

10. The method of claim 9, wherein the bus storage protocol is non-volatile memory fast NVMe, and wherein the network storage protocol is NVMe-over-TCP based on the transmission control protocol.

11. The method of claim 9, wherein performing the second I / O transaction comprises: The processor is used to process the packet headers of the network storage protocol and exchange the packet headers of the network storage protocol with the remote storage device.

12. The method of claim 9, wherein causing the network interface controller to directly transmit the packet data comprises: Map the memory buffer in the host's memory to the direct memory access (DMA) address in the network interface controller.

13. The method of claim 9, further comprising: If transmitting a packet of the network storage protocol to the remote storage device fails, the packet data is retrieved from the host's memory to the processor, and the packet with the retrieved packet data is retransmitted.

14. The method of claim 9, wherein transferring the packet data directly from the host's memory to the remote storage device comprises: Calculate a Cyclic Redundancy Check (CRC) for the transmitted packet data and transmit the CRC along with the packet data.

15. The method of claim 9, wherein transferring the packet data directly from the remote storage device to the memory of the host comprises: Verify the Cyclic Redundancy Check (CRC) received along with the packet data.

16. The method of claim 15, further comprising: If the CRC cannot be verified, the processor is triggered to verify the CRC, and the processor retrieves at least some of the packet data from the host's memory and uses the retrieved packet data to verify the CRC.

17. A method for performing host-oriented input / output I / O transactions in a remote storage device using a network adapter, the method comprising: Using the network interface controller in the network adapter, it communicates with the host via a peripheral bus and with the remote storage device via a network; Using the processor in the network adapter, I / O transactions of the Bus Storage Protocol are received from the host, and the I / O transactions for the host in the remote storage device are completed by translating between the Bus Storage Protocol I / O transactions and the Network Storage Protocol I / O transactions. The network storage protocol's I / O transactions include packets with packet headers and packet data. The completion of the host's I / O transaction includes: enabling the network interface controller to use zero-copy transmission of the packet data for the I / O transaction of the network storage protocol directly between the remote storage device and the host's memory, and transmitting the packet header of the I / O transaction of the network storage protocol between the remote storage device and the processor.

18. The method of claim 17, wherein the bus storage protocol is non-volatile memory fast NVMe, and wherein the network storage protocol is NVMe-over-TCP based on the transmission control protocol.

Images