NVMeoF response instruction transmission method based on RDMA completion event

Abstract

The present invention discloses a NVMeoF response instruction transmission method based on RDMA completion event. The present invention is oriented to NVMeoF storage network. First, data transmission and NVMeoF response instruction transmission are tightly coupled, and the transmission of NVMeoF response instruction is realized through RDMA completion event, thereby eliminating the two steps of constructing a send request and executing a send operation. Secondly, an asynchronous data memory space release mechanism is adopted, and the task of releasing data memory space is added to a waiting queue for delayed execution, thereby eliminating the time overhead introduced by the operation of releasing data memory space in the read request processing thread. The present invention realizes a streamlined I / O path design for NVMeoF storage network, effectively reduces the number of SEND and RECEIVE bilateral operations caused by read requests in RDMA network, and greatly improves the read request processing efficiency of NVMeoF storage network.

Description

Technical Field

[0001] The present invention relates to the technical field of NVMeoF storage network, and in particular to an NVMeoF response instruction transmission method based on RDMA completion event. Background Art

[0002] RDMA (Remote Direct Memory Access) interconnection network supports computers to directly access data in the memory of other computers in the network. It has the performance advantages of low latency and high concurrency, as well as technical features such as zero memory copy, kernel bypass, and data transmission without CPU intervention. RDMA interconnection network usually consists of RDMA Network Interface Controller (RNIC) and RDMA network switch. RNIC can simultaneously support bilateral SEND and RECEIVE operations, as well as unilateral RDMA PUT and RDMA GET operations. Bilateral operations are usually used to establish connections with remote computers and transmit short messages, and require the CPU to initiate data sending and receiving operations. Unilateral operations do not require CPU intervention, and RNIC implements direct data transmission between local memory and remote memory, thereby enabling computers to directly access the memory of remote computers.

[0003] After completing the access to the remote memory, the RDMA PUT and RDMA GET operations can generate RDMA completion events on the local computer and the remote computer respectively. The CPU confirms whether the RDMA data transmission is completed by monitoring the RDMA completion events. Figure 1 As shown in the figure, the local RDMA completion event is received by the initiator RNIC of the RDMA operation to the event queue (Event queue), and the local computer CPU determines whether the corresponding RDMA operation is completed by monitoring the event queue. The remote RDMA completion event is received by the target RNIC of the RDMA operation to the event queue, and the remote computer CPU determines whether the corresponding RDMA operation is completed by monitoring the event queue. The RDMA completion event can carry a certain size of payload data, which can be used for correctness verification of RDMA data transmission and transmission of additional information. For example, in the Tianhe RDMA interconnection network developed for the Tianhe high-performance computer system in my country, the RDMA completion event can carry a maximum of 64 bytes of data, which can be received by the RNIC event queue after the RDMA data transmission is completed.

[0004] The NVMe (Non-Volatile Memory Express) storage protocol is a host controller interface protocol designed to improve the parallelism of multi-core CPU access to SSDs (Solid State Drives). Based on the NVMe protocol, the NVMeoF (NVMe over Fabrics) storage network implements parallel transmission of instructions and data for remote SSD access. In the NVMeoF storage network, the initiator and processor of the SSD access request are usually two computers interconnected through the RDMA network. The computer that initiates the remote SSD access request is called the NVMoF initiator, and the computer that processes the SSD access request is called the NVMeoF target.

[0005] When the NVMeoF initiator accesses the data in the SSD of the NVMeoF target through the NVMeoF storage network, it is mainly divided into three steps: NVMeoF request instruction transmission, RDMA data transmission, and NVMeoF response instruction transmission. Among them, the size of the NVMeoF request instruction is usually 64 bytes, which is sent and received through the bilateral operation of the RDMA interconnection network; the data size is determined by the amount of data actually accessed by the upper-layer application on the SSD, and the NVMeoF target initiates a unilateral RDMA operation for transmission; the size of the NVMeoF response instruction is usually 16 bytes, which is sent and received through the bilateral operation of the RDMA interconnection network.

[0006] In the existing NVMeoF storage network, the steps of NVMeoF request instruction transmission, RDMA data transmission, and NVMeoF response instruction transmission in the read request processing flow are as follows: Figure 2 In the existing NVMeoF storage network, the transmission and processing flow of the NVMeoF request instruction in the read request processing is as follows: 1) The NVMeoF initiator CPU executes the send (executes SEND) operation to send the NVMeoF request instruction; 2) The NVMeoF target CPU executes the RECEIVE operation to receive the NVMeoF request instruction; 3) The NVMeoF target CPU parses the NVMeoF request instruction to obtain the logical address of the data to be read in the SSD and the NVMeoF initiator memory address where the data needs to be stored; 4) The NVMeoF target CPU applies for temporary memory space and initiates a local SSD read operation to read the data from the SSD into the temporary memory space.

[0007] In the existing NVMeoF storage network, the transmission and processing flow of data and NVMeoF response instructions in read request processing is as follows: Figure 3As shown, the details are as follows: 1) The NVMeoF target CPU constructs an RDMA PUT request, indicating the local temporary memory address where the data has been stored and the NVMeoF initiator memory address where the data needs to be stored; 2) The NVMeoF target CPU constructs an RDMA completion event and initializes the load information in the RDMA completion event; 3) The NVMeoF target RNIC performs an RDMA PUT operation to transfer the data in the temporary memory to the NVMeoF initiator memory; 4) The NVMeoF target RNIC receives the RDMA completion event to confirm that the data transfer is complete; 5) The NVMeoF target CPU releases the temporary memory space; 6) The NVMeoF target CPU constructs a send request (SEND request), which includes the construction and initialization of the NVMeoF response instruction; 7) The NVMeoF target CPU executes a send (executes SEND) operation to send the NVMeoF response instruction; 8) The NVMeoF initiator CPU executes a RECEIVE operation to receive the NVMeoF response instruction. Summary of the invention

[0008] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a NVMeoF response instruction transmission method based on RDMA completion event is provided. The present invention aims to streamline I / O path design for NVMeoF storage network, effectively reduce the number of SEND and RECEIVE bilateral operations caused by read requests in RDMA network, and improve the read request processing efficiency of NVMeoF storage network.

[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0010] A NVMeoF response instruction transmission method based on RDMA completion event includes optimizing the transmission and processing flow of data and NVMeoF response instructions in read request processing for an NVMeoF storage network. The optimization design includes tightly coupling data transmission and NVMeoF response instruction transmission, and realizing the transmission of NVMeoF response instructions through RDMA completion event to eliminate the two steps of constructing a send request and executing a send operation.

[0011] Optionally, tightly coupling the data transmission and the NVMeoF response instruction transmission means encapsulating the NVMeoF response instruction in the payload information of the RDMA completion event when constructing the RDMA completion event, so that when the RDMA PUT operation is performed for data transmission, the transmission of the NVMeoF response instruction is realized through the RDMA completion event.

[0012] Optionally, encapsulating the NVMeoF response instruction in the payload information of the RDMA completion event when constructing the RDMA completion event means that when constructing the RDMA completion event, the construction and initialization of the NVMeoF response instruction are completed, and the 16-byte NVMeoF response instruction is used as the payload information of the RDMA completion event and filled into the data structure corresponding to the RDMA completion event, thereby eliminating the two steps of constructing a send request and executing a send operation in the read request processing flow, and there is no need to transmit the NVMeoF response instruction through a separate SEND operation.

[0013] Optionally, the optimization design also includes adopting an asynchronous data memory space release mechanism, adding the data memory space release task to a waiting queue for delayed execution to eliminate the time overhead introduced by the data memory space release operation in the read request processing thread.

[0014] Optionally, the adding of the task of releasing the data memory space to a waiting queue and delaying its execution means adding the task of releasing the data memory space to a waiting queue and delaying its execution, and scheduling its execution by the CPU when the read request load is low, so that after receiving the interrupt request initiated by the RDMA completion event, the NVMeoF target end CPU can immediately confirm that the NVMeoF response instruction has been sent, complete the processing flow of the read request, and quickly enter the processing flow of the next read request, thereby eliminating the time overhead introduced by the operation of releasing the data memory space in the read request processing thread.

[0015] Optionally, the read request processing flow of the NVMeoF storage network includes:

[0016] Step ①, the NVMeoF initiator encapsulates and sends the NVMeoF request instruction, and transmits the NVMeoF request instruction to the NVMeoF target end;

[0017] Step ②, the NVMeoF target end receives and processes the NVMeoF request instruction, reads data from the NVME solid state drive into the memory as RDMA data, encapsulates the NVMeoF response instruction as the payload information of the RDMA completion event, and then executes the RDMA data and the NVMeoF response instruction to transmit to the NVMeoF initiator end, so that the NVMeoF initiator end receives the RDMA data and parses the NVMeoF response instruction from the received RDMA completion event and processes it.

[0018] Optionally, the method further includes the following steps of transmitting and processing data and NVMeoF response instructions in the read request processing:

[0019] Step ①, the NVMeoF target CPU constructs an RDMA PUT request, indicating the local temporary memory address where the data has been stored and the NVMeoF initiator memory address where the data needs to be stored;

[0020] Step ②, the NVMeoF target CPU constructs an RDMA completion event, initializes the load information in the RDMA completion event, and completes the construction and initialization of the NVMeoF response instruction;

[0021] Step 3: The NVMeoF target RNIC performs an RDMA PUT operation to transfer the data in the temporary memory to the NVMeoF initiator memory.

[0022] Step ④: The NVMeoF target RNIC receives the RDMA completion event and confirms that the data transmission is complete.

[0023] Step ⑤, the NVMeoF initiator RNIC receives the RDMA completion event and parses the NVMeoF response instruction from the RDMA completion event.

[0024] In addition, the present invention also provides an NVMeoF response instruction transmission system based on RDMA completion events, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the NVMeoF response instruction transmission method based on RDMA completion events.

[0025] In addition, the present invention also provides a computer-readable storage medium, in which a computer program or instruction is stored, and the computer program or instruction is programmed or configured to execute the NVMeoF response instruction transmission method based on the RDMA completion event through a processor.

[0026] In addition, the present invention also provides a computer program product, including a computer program or an instruction, which is programmed or configured to execute the NVMeoF response instruction transmission method based on the RDMA completion event through a processor.

[0027] Compared with the prior art, the present invention mainly has the following advantages: For the NVMeoF storage network, the present invention optimizes the design of the transmission and processing flow of data and NVMeoF response instructions in the read request processing. First, the data transmission and the NVMeoF response instruction transmission are tightly coupled, and the transmission of the NVMeoF response instruction is realized through the RDMA completion event to eliminate the two steps of constructing a send request and executing a send operation. Secondly, an asynchronous data memory space release mechanism is adopted, and the task of releasing the data memory space is added to the waiting queue for delayed execution to eliminate the time overhead introduced by the operation of releasing the data memory space in the read request processing thread. The present invention realizes a streamlined I / O path design for the NVMeoF storage network, effectively reduces the number of SEND and RECEIVE bilateral operations caused by read requests in the RDMA network, and greatly improves the read request processing efficiency of the NVMeoF storage network. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 The present invention relates to a process and a method for generating and processing RDMA completion events in the prior art.

[0029] Figure 2 This is a read request processing flow of an NVMeoF storage network in the prior art.

[0030] Figure 3 The invention discloses a transmission and processing flow of request data and NVMeoF response instruction in the prior art.

[0031] Figure 4 This is a read request processing flow of the NVMeoF storage network in an embodiment of the present invention.

[0032] Figure 5 The present invention provides a read request data and NVMeoF response instruction transmission and processing flow. DETAILED DESCRIPTION

[0033] like Figure 4 As shown, the NVMeoF response instruction transmission method based on the RDMA completion event in this embodiment includes optimizing the transmission and processing flow of data and NVMeoF response instructions in read request processing for the NVMeoF storage network, and the optimization design includes tightly coupling the data transmission and the NVMeoF response instruction transmission, and realizing the transmission of the NVMeoF response instruction through the RDMA completion event to eliminate the two steps of constructing a send request (SEND request) and executing a send (execute SEND) operation.

[0034] In this embodiment, the tight coupling of data transmission and NVMeoF response instruction transmission refers to encapsulating the NVMeoF response instruction in the payload information of the RDMA completion event when constructing the RDMA completion event, so that when the RDMA PUT operation is performed for data transmission, the transmission of the NVMeoF response instruction is realized through the RDMA completion event.

[0035] In this embodiment, encapsulating the NVMeoF response instruction in the payload information of the RDMA completion event when constructing the RDMA completion event means that when constructing the RDMA completion event, the construction and initialization of the NVMeoF response instruction are completed, and the 16-byte NVMeoF response instruction is used as the payload information of the RDMA completion event and filled into the data structure corresponding to the RDMA completion event, thereby eliminating the two steps of constructing a send request and executing a send operation in the read request processing flow, and there is no need to transmit the NVMeoF response instruction through a separate SEND operation.

[0036] like Figure 4As shown, the optimized design of the NVMeoF response instruction transmission method based on the RDMA completion event in this embodiment also includes adopting an asynchronous data memory space release mechanism, adding the data memory space release task to a waiting queue for delayed execution to eliminate the time overhead introduced by the data memory space release operation in the read request processing thread.

[0037] In this embodiment, adding the task of releasing data memory space to a waiting queue and delaying its execution means adding the task of releasing data memory space to a waiting queue and delaying its execution, and scheduling its execution by the CPU when the read request load is low, so that after receiving the interrupt request initiated by the RDMA completion event, the NVMeoF target end CPU can immediately confirm that the NVMeoF response instruction has been sent, complete the processing flow of the read request, and quickly enter the processing flow of the next read request, thereby eliminating the time overhead introduced by the operation of releasing data memory space in the read request processing thread.

[0038] like Figure 4 As shown, the read request processing flow of the NVMeoF storage network described in this embodiment includes:

[0039] Step ①, the NVMeoF initiator encapsulates and sends the NVMeoF request instruction, and transmits the NVMeoF request instruction to the NVMeoF target end;

[0040] Step ②, the NVMeoF target end receives and processes the NVMeoF request instruction, reads data from the NVME solid state drive into the memory as RDMA data, encapsulates the NVMeoF response instruction as the payload information of the RDMA completion event, and then executes the RDMA data and the NVMeoF response instruction to transmit to the NVMeoF initiator end, so that the NVMeoF initiator end receives the RDMA data and parses the NVMeoF response instruction from the received RDMA completion event and processes it.

[0041] like Figure 5 As shown, the method of this embodiment also includes the following steps of transmitting and processing data and NVMeoF response instructions in the read request processing:

[0042] Step ①, the NVMeoF target CPU constructs an RDMA PUT request, indicating the local temporary memory address where the data has been stored and the NVMeoF initiator memory address where the data needs to be stored;

[0043] Step ②, the NVMeoF target CPU constructs an RDMA completion event, initializes the load information in the RDMA completion event, and completes the construction and initialization of the NVMeoF response instruction;

[0044] Step 3: The NVMeoF target RNIC performs an RDMA PUT operation to transfer the data in the temporary memory to the NVMeoF initiator memory.

[0045] Step ④: The NVMeoF target RNIC receives the RDMA completion event and confirms that the data transmission is complete.

[0046] Step ⑤, the NVMeoF initiator RNIC receives the RDMA completion event and parses the NVMeoF response instruction from the RDMA completion event.

[0047] The method of this embodiment is oriented to the NVMeoF storage network, and optimizes the design for the three steps of releasing data memory space, constructing a send request, and executing a send operation in the process of processing a read request. (1) Tightly coupling NVMeoF response instruction transmission and data transmission In the NVMeoF storage network, the RDMA PUT operation and SEND operation required for the read request are continuous, and the data transmission and the transmission of the NVMeoF response instruction are completed respectively. To this end, the present invention tightly couples the data transmission and the NVMeoF response instruction transmission. When constructing the RDMA completion event, the NVMeoF response instruction is encapsulated in the payload information of the RDMA completion event, so that when the RDMA PUT operation is executed for data transmission, the transmission of the NVMeoF response instruction is realized through the RDMA completion event. The RDMA completion event can carry a maximum of 64 bytes of data, which meets the 16-byte space requirement for the transmission of the NVMeoF response instruction. When constructing the RDMA completion event, it is necessary to complete the construction and initialization of the NVMeoF response instruction, and fill the 16-byte NVMeoF response instruction as the payload information of the RDMA completion event into the data structure corresponding to the RDMA completion event. This method can eliminate the two steps of constructing a send request and executing a send operation in the read request processing flow, and it is no longer necessary to transmit the NVMeoF response instruction through a separate SEND operation. (2) Asynchronous release of data memory space. In the NVMeoF storage network, only after the NVMeoF target RNIC receives the RDMA completion event can it confirm that the data has been successfully transmitted through RDMA PUT and the data memory space can be released. However, the data memory space does not have to be released immediately after receiving the RDMA completion event. The task of releasing the data memory space can be added to the waiting queue for delayed execution and scheduled by the CPU when the read request load is low. After receiving the interrupt request initiated by the RDMA completion event, the NVMeoF target CPU can immediately confirm that the NVMeoF response instruction has been sent, complete the read request processing flow, and quickly enter the next read request processing flow, thereby eliminating the time overhead introduced by the data memory space release operation in the read request processing thread.

[0048] In summary, the method of this embodiment is oriented to the NVMeoF storage network, including optimizing the transmission and processing flow of data and NVMeoF response instructions in the read request processing: first, the data transmission and the NVMeoF response instruction transmission are tightly coupled, and the transmission of the NVMeoF response instruction is realized through the RDMA completion event, eliminating the two steps of constructing a send request and executing a send operation. Secondly, an asynchronous data memory space release mechanism is adopted, and the task of releasing the data memory space is added to the waiting queue for delayed execution, thereby eliminating the time overhead introduced by the operation of releasing the data memory space in the read request processing thread. This embodiment implements a streamlined I / O path design for the NVMeoF storage network, effectively reducing the number of SEND and RECEIVE bilateral operations caused by read requests in the RDMA network, and greatly improving the read request processing efficiency of the NVMeoF storage network.

[0049] In addition, this embodiment also provides an NVMeoF response instruction transmission system based on RDMA completion events, including a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the NVMeoF response instruction transmission method based on RDMA completion events.

[0050] In addition, this embodiment also provides a computer-readable storage medium, in which a computer program or instruction is stored. The computer program or instruction is programmed or configured to execute the NVMeoF response instruction transmission method based on the RDMA completion event through a processor.

[0051] In addition, this embodiment also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the NVMeoF response instruction transmission method based on the RDMA completion event through a processor.

[0052] Those skilled in the art should understand that the embodiments of the present application may provide a technical solution in the form of a method, system, or computer program product. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0053] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A NVMeoF response instruction transmission method based on RDMA completion event, It is characterized in that The invention comprises optimizing the transmission and processing flow of data and NVMeoF response instructions in the read request processing for NVMeoF storage network, wherein the optimization design comprises tightly coupling the data transmission and the NVMeoF response instruction transmission, realizing the transmission of the NVMeoF response instruction through the RDMA completion event to eliminate the two steps of constructing the sending request and executing the sending operation, wherein the tightly coupling the data transmission and the NVMeoF response instruction transmission refers to encapsulating the NVMeoF response instruction in the load information of the RDMA completion event when constructing the RDMA completion event, thereby executing the RDMA When the PUT operation is used to transmit data, the transmission of the NVMeoF response instruction is realized through the RDMA completion event; the optimization design also includes adopting an asynchronous data memory space release mechanism, adding the data memory space release task to the waiting queue for delayed execution to eliminate the time overhead introduced by the data memory space release operation in the read request processing thread, and adding the data memory space release task to the waiting queue for delayed execution means adding the data memory space release task to the waiting queue for delayed execution, and scheduling the execution by the CPU when the read request load is low, so that after receiving the interrupt request initiated by the RDMA completion event, the NVMeoF target end CPU can immediately confirm that the NVMeoF response instruction has been sent, complete the processing flow of the read request, and quickly enter the processing flow of the next read request, thereby eliminating the time overhead introduced by the data memory space release operation in the read request processing thread.

2. The NVMeoF response instruction transmission method based on RDMA completion event according to claim 1, It is characterized in that The encapsulating the NVMeoF response instruction in the payload information of the RDMA completion event when constructing the RDMA completion event means that when constructing the RDMA completion event, the construction and initialization of the NVMeoF response instruction are completed, and the 16-byte NVMeoF response instruction is used as the payload information of the RDMA completion event and filled into the data structure corresponding to the RDMA completion event, thereby eliminating the two steps of constructing a send request and executing a send operation in the read request processing flow, and no longer needing to transmit the NVMeoF response instruction through a separate SEND operation.

3. The NVMeoF response instruction transmission method based on RDMA completion event according to claim 2, It is characterized in that The read request processing flow of the NVMeoF storage network includes: Step ①, the NVMeoF initiator encapsulates and sends the NVMeoF request instruction, and transmits the NVMeoF request instruction to the NVMeoF target end; Step ②, the NVMeoF target end receives and processes the NVMeoF request instruction, reads data from the NVME solid state drive into the memory as RDMA data, encapsulates the NVMeoF response instruction as the payload information of the RDMA completion event, and then executes the RDMA data and the NVMeoF response instruction to transmit to the NVMeoF initiator end, so that the NVMeoF initiator end receives the RDMA data and parses the NVMeoF response instruction from the received RDMA completion event and processes it.

4. The NVMeoF response instruction transmission method based on RDMA completion event according to claim 3, It is characterized in that The following steps are also included for transmitting and processing data and NVMeoF response instructions in the read request process: Step ①, the NVMeoF target CPU constructs an RDMA PUT request, indicating the local temporary memory address where the data has been stored and the NVMeoF initiator memory address where the data needs to be stored; Step ②, the NVMeoF target CPU constructs an RDMA completion event, initializes the load information in the RDMA completion event, and completes the construction and initialization of the NVMeoF response instruction; Step 3: The NVMeoF target RNIC performs an RDMA PUT operation to transfer the data in the temporary memory to the NVMeoF initiator memory. Step ④: The NVMeoF target RNIC receives the RDMA completion event and confirms that the data transmission is complete. Step ⑤, the NVMeoF initiator RNIC receives the RDMA completion event and parses the NVMeoF response instruction from the RDMA completion event.

5. An NVMeoF response instruction transmission system based on RDMA completion event, comprising a microprocessor and a memory connected to each other, It is characterized in that The microprocessor is programmed or configured to execute the NVMeoF response instruction transmission method based on RDMA completion event as described in any one of claims 1 to 4.

6. A computer-readable storage medium having a computer program or instruction stored therein, It is characterized in that The computer program or instruction is programmed or configured to execute the NVMeoF response instruction transmission method based on the RDMA completion event described in any one of claims 1 to 4 through a processor.

7. A computer program product comprising a computer program or instructions, It is characterized in that The computer program or instruction is programmed or configured to execute the NVMeoF response instruction transmission method based on the RDMA completion event described in any one of claims 1 to 4 through a processor.

Images