Intermediate device, communication method, program, and relay system

Abstract

An intermediate device 10A is provided with: a transfer unit 11 that transfers a request including data to be transmitted from a local 30 to a remote 50; and a generation unit 12 that generates a pseudo response to the request and returns the pseudo response to the local 30. An intermediate device 10B is provided with a cancellation unit 13 that cancels the response to the request from the remote 50. An intermediate device 20B is provided with: a generation unit 22 that generates a pseudo request on the basis of a first request for making a request for transmitting data from the local 30 to the remote 50 and that transmits the pseudo request to the remote 50; and a transfer unit 24 that transfers a response including data to be transmitted from the remote 50 to the local 30. An intermediate device 20A is provided with a cancellation unit 21 that cancels a subsequent request from the local 30.

Description

[Technical field]

[0001] The present invention relates to an intermediate device, a communication method, and a program. [Background technology]

[0002] RDMA (Remote Direct Memory Access), used in the transport layer of InfiniBand (Non-Patent Document 1), is a communication protocol that performs high-speed and highly reliable data transfer between communication terminals located far away from each other. RDMA enables high-speed communication because it directly accesses memory from the memory area of ​​the sending terminal to the memory area of ​​the receiving terminal. RDMA has a credit-based flow control function, and also performs Completion control to confirm the completion of data transfer and proceed with processing, enabling highly reliable communication. RDMA is also used as a transport method for Host-to-Device and Device-to-Device data communication between Solid State Drives (SSDs) and Graphics Processing Units (GPUs).

[0003] As shown in Figure 1, RDMA is a communication model that configures a QP (Queue Pair) between a local server and a remote server and transfers data using the QP. A QP is a set of an SQ (Send Queue) and an RQ (Receive Queue). The communication unit of RDMA is a communication request called a WR (Work Request), which is stacked in the SQ / RQ in units of WQE (Work Queue Element). WR has a Send WR, which is a transmission request, and a Receive WR, which is a reception request. In a Send WR, the memory area of ​​the data to be sent is specified as a WQE and stacked in the SQ. In a Receive WR, the memory area of ​​the data to be received is specified as a WQE and stacked in the RQ. WQEs can be stacked in the SQ / RQ in a FIFO (First-In-First-Out) order up to the queue size of the SQ / RQ. When the WR processing is completed normally between the QPs, a CQE (Completion Queue Entry) indicating normal completion is stacked in the CQ (Completion Queue) corresponding to each SQ / RQ. If the WR processing during the QP ends with an error, a CQE indicating the error is stacked in the CQ. When a normal completion CQE is confirmed, the WQE of the SQ / RQ is deleted and the next WR can be accepted. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] InfiniBand Architecture Specification Volume 1, Release 1.4, April 2020 Summary of the Invention [Problem to be solved by the invention]

[0005] Advances in communication technology may provide networks that connect over long distances with high bandwidth. For example, it is conceivable that transponders installed in current optical transmission systems will be installed in client-server systems, and signals will reach the server of the other party without being converted from electrical to optical during transmission. In addition, technology has been proposed for optical transmission systems that establishes high-speed transmission paths with limited network resources (frequency, etc.) by selecting the optimal transmission mode (modulation method, baud rate, number of carriers, etc.) based on the network conditions (distance, signal quality, etc.). Such technology may enable long-distance, high-speed communication between communication terminals using limited network and terminal resources.

[0006] The problem with RDMA is that the longer the network distance, the worse the transfer performance becomes. This is because longer lines require more transfer time, and in connection-based protocols, the next packet cannot be sent until a packet notifying completion of data transfer is received. In particular, in connection-based service types, in order to complete a WQE on the local side, an ACK must be received from the remote side and a CQE must be issued. The longer it takes for requests and responses between the local and remote sides, the more incomplete WQEs accumulate in the SQ / RQ, and the more WQEs that cannot be queued because they are waiting to be processed are increased, resulting in a decline in transfer performance.

[0007] The present invention has been made in view of the above, and has an object to realize high-bandwidth data transfer even on a network service with a long RTT (Round Trip Time). [Means for solving the problem]

[0008] An intermediate device according to one aspect of the present invention is disposed between a first device and a second device that transfer data using remote direct memory access, and includes a transfer unit that transfers a request including data to be transmitted from the first device to the second device, a generation unit that generates a pseudo response to the request and returns it to the first device, and a discard unit that discards the response to the request from the second device. A table is created that combines the destinations of requests and responses having the same identifier from requests and responses transmitted and received between the first device and the second device, and the generation unit obtains a combination including the destination of the request from the table, and sets a destination corresponding to the destination of the request in the combination as the destination of the pseudo response. .

[0009] An intermediate device of one embodiment of the present invention is an intermediate device disposed between a first device and a second device that transfer data using remote direct memory access, and includes a generation unit that generates a pseudo request requesting data transmission from the first device to the second device based on an initial request requesting data transmission from the first device to the second device and transmits the pseudo request to the second device, a transfer unit that transfers a response including data to be transmitted from the second device to the first device, and a discard unit that discards subsequent requests from the first device.

[0010] A communication method according to one aspect of the present invention is a communication method by an intermediate device disposed between a first device and a second device that transfer data using remote direct memory access, the method comprising: transferring a request including data to be transmitted from the first device to the second device; generating a pseudo response to the request and returning it to the first device; and discarding a response to the request from the second device. A table is created that combines the destinations of requests and responses having the same identifier from among requests and responses transmitted and received between the first device and the second device, and in generating the pseudo response, a combination including the destination of the request is obtained from the table, and a destination corresponding to the destination of the request in the combination is set as the destination of the pseudo response. .

[0011] One embodiment of the communication method of the present invention is a communication method by an intermediate device disposed between a first device and a second device that transfer data using remote direct memory access, which generates a pseudo request requesting data transmission from the first device to the second device based on an initial request requesting data transmission from the first device to the second device and transmits it to the second device, forwards a response including data to be transmitted from the second device to the first device, and discards subsequent requests from the first device. Effect of the Invention

[0012] According to the present invention, high-bandwidth data transfer can be achieved even on a network service with a large RTT. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram for explaining the RDMA communication model. [Diagram 2] FIG. 2 is a diagram for explaining RDMA SEND. [Diagram 3] FIG. 3 is a diagram for explaining RDMA WRITE. [Figure 4] FIG. 4 is a diagram for explaining RDMA WRITE with Immediate. [Diagram 5] FIG. 5 is a diagram for explaining RDMA READ. [Figure 6] FIG. 6 is a diagram for explaining RDMA ATOMIC Operations. [Figure 7] FIG. 7 is a diagram illustrating an example of the configuration of a communication system including an intermediate device of the present embodiment. [Figure 8] FIG. 8 is a sequence diagram showing an example of a process flow of the communication system of FIG. [Figure 9] FIG. 9 is a diagram illustrating an example of the configuration of a communication system including another intermediate device according to the present embodiment. [Figure 10] FIG. 10 is a sequence diagram showing an example of a process flow of the communication system of FIG. [Figure 11] FIG. 11 is a diagram illustrating an example of a method for creating a table to resolve the destination QPN of a response. [Figure 12] FIG. 12 is a diagram illustrating an example of a method for notifying a source QPN and resolving a destination QPN of a response. [Figure 13] FIG. 13 is a diagram illustrating an example in which an intermediate device is configured on a NIC. [Figure 14] FIG. 14 illustrates an example of a hardware configuration of the intermediate device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] [About RDMA] RDMA service types are broadly classified into four types based on the classification of Reliable / Unreliable and Connection / Datagram: RC (Reliable Connection), RD (Reliable Datagram), UC (Unreliable Connection), and UD (Unreliable Datagram). RC and UD are commonly used.

[0015] RC guarantees the order and delivery of messages by checking the success or failure of communication by ACK / NAK and by using a mechanism for retransmission. RC is also connection-oriented, and performs one-to-one communication between local and remote QPs.

[0016] Although UD does not have a mechanism for acknowledgment or retransmission, it enables many-to-many communication, such as sending to multiple QPs and receiving from multiple QPs by specifying the destination for each communication.

[0017] RDMA operation types are broadly divided into four types: SEND, RDMA WRITE (with Immediate), RDMA READ, and ATOMIC Operations. All of these can be used in RC. Only SEND can be used in UD.

[0018] Retransmission control in RDMA is classified into three patterns: when ACK / NAK is not returned, when an RNR (Receiver-Not-Ready) NAK is returned, and when an Out-Of-sequence NAK is returned. If an ACK or NAK is not returned from the remote side within a certain time, the local side will time out and retransmit. The remote side will also return an RNR NAK if a WQE cannot be prepared for the RQ. If an RNR NAK is returned from the remote side, the local side will retransmit after a certain time. The remote side will also return an Out-of-sequence NAK if the PSN (Packet Sequence Number) of the received packet is not in order. If an Out-Of-sequence NAK is returned from the remote side, the local side will retransmit without waiting.

[0019] [RDMA operations] Next, some operations of service type RC are described.

[0020] First, the RDMA operation SEND will be described with reference to Figure 2. SEND is the basic sending and receiving model of RDMA, and transmits data from the local to the remote.

[0021] The local prepares an SQ, the remote prepares an RQ, and the local and remote load WQE into the SQ and RQ respectively.

[0022] When communication is ready, the local sends data with SEND. When the remote receives the data successfully, it places a CQE in the CQ, releases the WQE in the RQ, and returns an ACK to the local. When the local receives an ACK, it places a CQE in the CQ, and releases the WQE in the SQ.

[0023] SEND also provides a special operation, SEND w / Imm (SEND with Immediate). With SEND w / Imm, a special field (imm_data) can be set in the WQE of the local SQ, and imm_data can be sent at the same time as sending data from the local to the remote. When the remote successfully receives the data, it places a CQE containing imm_data in the CQ. The remote can find out the contents of imm_data by referencing the CQE.

[0024] Next, the RDMA operation RDMA WRITE will be described with reference to Figure 3. Like SEND, RDMA WRITE is a method for sending data from a local to a remote, but it differs in that data is transferred directly to a remote memory area.

[0025] The local prepares an SQ and loads a WQE. The WQE contains the memory area of ​​the data to be sent and the remote memory area to be written. The remote secures memory area for RDMA, but there is no need to load a WQE into an RQ.

[0026] When communication is ready, the local sends data with RDMA WRITE. The data is written directly to the remote memory area. When the remote receives the data successfully, it returns an ACK to the local. When the local receives the ACK, it puts a CQE in the CQ and releases the WQE in the SQ.

[0027] Next, referring to Figure 4, we will explain the RDMA operation RDMA WRITE w / Imm (RDMA WRITE with Immediate). RDMA WRITE has the disadvantage that the remote cannot detect changes to the memory area when receiving data. In RDMA WRITE w / Imm, the remote prepares an RQ, loads a WQE into the RQ, and detects changes to the memory area by loading a CQE into the CQ when data is successfully received. The memory area of ​​the data to be sent, the remote memory area to be written, and a special field (imm_data) are set in the WQE of the local SQ. When the remote successfully receives the data, it loads a CQE including imm_data into the CQ, releases the WQE of the RQ, and returns an ACK to the local. The remote can detect changes to any memory area using this CQE.

[0028] Next, the RDMA operation RDMA READ will be described with reference to Figure 5. RDMA READ is a method for pulling data from a remote location to a local location.

[0029] The local prepares an SQ and loads a WQE. The WQE contains the local memory area to receive data from and the remote memory area to read from. The remote allocates memory for RDMA, but does not need to load a WQE into an RQ.

[0030] When communication is ready, the local requests data reading with an RDMA Read Request. When the remote receives the request, it sends the data of the remote memory area directly to the configured local memory area with an RDMA Read Response. The RDMA Read Response includes an ACK extension header. When the local receives this ACK, it puts a CQE in the CQ and releases the WQE in the SQ.

[0031] Next, referring to Figure 6, we will explain the ATOMIC Operations of RDMA operations. ATOMIC Operations are a method of performing memory operations such as FetchAdd (Fetch and Add) or CmpSwap (Compare and Swap) on a remote memory area and reading the memory contents before the operation locally. FetchAdd is an operation that adds the value of an arbitrary 64-bit field to the contents of an arbitrary remote memory address. CmpSwap is an operation that changes the contents of an arbitrary remote memory address to a new 64-bit field value when the contents are the same as the value of a specified 64-bit field.

[0032] The local prepares an SQ and loads a WQE. The WQE contains the local memory area to receive data, the remote memory area to operate, and the operation (FetchAdd or CmpSwap, and its arguments). The remote secures a memory area for RDMA, but there is no need to load a WQE into an RQ.

[0033] When communication is ready, the local sends an ATOMIC Command (FetchAdd or CmpSwap). When the remote receives the command, it performs an ATOMIC operation on the configured local memory area and returns the data before the operation in an ATOMIC ACK. When the local receives this ACK, it puts a CQE in the CQ and releases the WQE in the SQ.

[0034] [Communication system configuration] Next, an example of the configuration of a communication system including intermediate devices 10A and 10B of this embodiment will be described with reference to Fig. 7. Intermediate devices 10A and 10B are placed between local 30 and remote 50, which transfer data using RDMA. More specifically, intermediate device 10A is placed at the front end of a long-distance network on the local 30 side, and intermediate device 10B is placed at the front end of a long-distance network on the remote 50 side. Intermediate device 10A returns a pseudo-response to local 30, and intermediate device 10B discards the response from remote 50.

[0035] Intermediate device 10A includes a transfer unit 11 and a generation unit 12.

[0036] The transfer unit 11 receives a request from the local 30 and transfers it to the remote 50. This request is, for example, the above-mentioned SEND, SEND w / Imm, RDMA WRITE, RDMA WRITE w / Imm, or ATOMIC Command. The request includes data to be sent from the local 30 to the remote 50 or an operation on the data.

[0037] The generation unit 12 picks up a request that is sent from the local 30 and has an Only or Last flag set, and generates a pseudo-response using the PSN included in the request. The generation unit 12 returns the generated pseudo-response to the local 30. Note that the ACK for the request uses the same PSN value as the Only or Last request.

[0038] When the local 30 receives the pseudo response, it recognizes it as a response from the remote 50, puts a CQE in the CQ, and completes the process normally. This allows the local 30 to forcibly release the WQE of the SQ.

[0039] Intermediate device 10B includes forwarding unit 11 and discarding unit 13.

[0040] Transfer unit 11 transfers the request sent by local 30 to remote 50, similar to transfer unit 11 of intermediate device 10A.

[0041] The discard unit 13 discards the true response from the remote 50 to the request. This makes it possible to prevent the local 30 from receiving duplicate responses. Furthermore, since there is a possibility that an RNR or out-of-sequence NAK sent from the remote 50 may cause a malfunction when it arrives at the local 30, the discard unit 13 also discards these NAKs.

[0042] It is also possible for intermediate device 10A to include discarding unit 13, and intermediate device 10B need not be provided.

[0043] [Communication system operation] Next, with reference to a sequence diagram in FIG. 8, an example of a processing flow of a communication system including intermediate devices 10A and 10B will be described.

[0044] In step S11, the local 30 loads the WQE into the SQ and transmits the request to the remote 50. The request is transferred to the remote 50 via intermediate devices 10A and 10B.

[0045] In step S12, intermediate device 10A generates a pseudo response using the PSN included in the request. In step S13, intermediate device 10A returns the pseudo response to local 30. Upon receiving the pseudo response, local 30 loads a CQE into the CQ and releases the WQE of the SQ.

[0046] Thereafter, when the local 30 loads a WQE into an SQ and transmits the request to the remote 50 (step S16), the intermediate device 10A forwards the request, generates a pseudo-response, and returns it to the local 30 (step S17). When the local 30 receives the pseudo-response, it loads a CQE into a CQ, releases the WQE in the SQ, and then loads the WQE into the SQ and transmits the request to the remote 50 (step S18).

[0047] On the other hand, if the remote 50 has successfully received the request (data), it transmits a response to the local 30 in step S14.

[0048] In step S15, intermediate device 10B discards the received response.

[0049] Thereafter, when remote 50 receives a request, it returns a response, and intermediate device 10B discards the response.

[0050] [Another communication system] Next, an example of the configuration of a communication system including intermediate devices 20A and 20B of this embodiment will be described with reference to Fig. 9. Intermediate devices 20A and 20B are placed between local 30 and remote 50, which transfer data using RDMA. More specifically, intermediate device 20A is placed at the front end of a long-distance network on the local 30 side, and intermediate device 20B is placed at the front end of a long-distance network on the remote 50 side. Intermediate device 20A discards requests from local 30, and intermediate device 20B transmits pseudo requests to remote 50.

[0051] Intermediate device 20A includes discard unit 21 and forwarding unit 24.

[0052] The discard unit 21 transfers the first request (request of Only or First) from the Local 30 to the Remote 50, and discards subsequent requests from the Local 30. This makes it possible to prevent the Remote 50 from receiving duplicate requests.

[0053] The transfer unit 24 transfers the response returned from the remote 50 to the local 30. The response includes data to be transmitted from the remote 50 to the local 30.

[0054] Intermediate device 20B includes a generating unit 22, a control unit 23, and a transferring unit 24.

[0055] The generator 22 picks up the first request from the local 30 and generates a pseudo-request using a destination QP Number (QPN) included in an RDMA Extended Transport Header (RETH) and a Base Transport Header (BTH).

[0056] The generation unit 22 generates pseudo requests so that the number of requests sent to the remote 50 minus the number of responses returned from the remote 50 does not exceed the queue size of the SQ of the local 30. The PSN of the pseudo request is calculated by calculating the number of requests for 1 WQE from the DMA Length of the RETH of the request and the PSH of the BTH, and incrementing the PSN by that number.

[0057] When the remote 50 receives the pseudo request, it recognizes it as a request from the local 30, retrieves the data from the memory area, and sends a response to the local 30. This allows the remote 50 to send data without waiting for a request from the local 30.

[0058] The control unit 23 checks the pseudo request sent to the remote 50 and the response returned from the remote 50, inspects whether the response returned is of the expected length and number, and controls the timing of the generation of the pseudo request by the generation unit 22.

[0059] The forwarding unit 24 forwards the response returned from the remote 50 to the local 30, similar to the forwarding unit 24 of the intermediate device 20A.

[0060] It is also possible for intermediate device 20B to include discarding unit 21, and intermediate device 20A need not be provided.

[0061] [Another communication system in action] Next, with reference to a sequence diagram of FIG. 10, an example of a processing flow of a communication system including intermediate devices 20A and 20B will be described.

[0062] In step S21, local 30 loads a WQE into an SQ and transmits the initial request to remote 50. Intermediate device 20A does not discard the initial request, but forwards it to remote 50. Intermediate device 20B obtains the QPN included in the initial request.

[0063] Upon receiving the request, the remote 50 returns a response to the local 30 in step S22. Upon receiving the response, the local 30 places a CQE in the CQ and releases the WQE of the SQ.

[0064] In step S23, the local 30 loads the WQE into the SQ and transmits the subsequent request to the remote 50.

[0065] In step S24, intermediate device 20A discards the subsequent request from local device 30.

[0066] Meanwhile, intermediate device 20B generates a pseudo request in step S25, and transmits the pseudo request to remote device 50 in step S26. Intermediate device 20B controls the timing of generating the pseudo request so that local 30 can correctly receive a response returned from remote device 50.

[0067] When the remote 50 receives the pseudo request, it returns a response including data to the local 30 in step S27.

[0068] Thereafter, when the local 30 receives a response to the pseudo request, it loads a CQE into the CQ, releases the WQE in the SQ, and then loads the WQE into the SQ and transmits a subsequent request to the remote 50 (step S28). The intermediate device 20A discards the subsequent request from the local 30 (step S29).

[0069] Intermediate device 20B generates a pseudo request at a predetermined timing (step S30), and transmits the pseudo request to remote device 50 (step S31).

[0070] [Destination QPN resolution] In the RDMA interface, a QP has a different QPN for each endpoint. The SQ / RQ recognizes the QPN of the other end, and includes the destination QPN in the header when generating an RDMA packet. However, the source QPN is not included in the header. When intermediate device 10A generates a pseudo response, the received request does not contain information indicating the source QPN, so the destination of the pseudo response is unknown. Therefore, in this embodiment, the destination of the pseudo response is specified by the following two methods.

[0071] The first method is to inspect the exchange of the original RDMA request and response, and store the combination of QPNs in a table. The same PSN is used for the Only or Last request and ACK of an RDMA packet. Therefore, the intermediate device 10A inspects the request and response passing through, and adds the destination QPNs of the headers of the Only or Last request and ACK having the same PSN to the table as a combination. In the example of FIG. 11, the destination QPNs of the headers of the request and response having the same PSN are 0x000020 and 0x000010, respectively, so the combination of 0x000010 and 0x000020 is added to the table. In FIG. 11, the local 30 configures a QP between each of the remotes 50A and 50B.

[0072] When intermediate device 10A generates a pseudo response, it obtains from the table a combination of QPNs that includes the destination QPN of the request, and sets the other QPN of the combination as the destination QPN of the pseudo response. For example, when a request with a destination QPN of 0x000020 is received, intermediate device 10A obtains from the table the combination of 0x000010 and 0x000020, which includes 0x000020, and sets the destination QPN of the pseudo response to 0x000010.

[0073] The second method is to put the Source QPN in the RDMA packet. Data ) field exists, and any 32-bit information can be written in the immDt field only for SEND with immediate or RDMA WRITE with immediate. In Fig. 12, the local 30 includes an insertion unit 31, and the insertion unit 31 writes the QPN of the SQ on the local 30 side into the immDt field of the WQE of the SQ on the local 30 side.

[0074] When intermediate device 10A generates a pseudo response, it sets the QPN written in the immDt field of the received request as the destination QPN of the pseudo response.

[0075] [Example] Next, an example in which the intermediate device of this embodiment is applied to each operation of RDMA will be described.

[0076] First, a description will be given of an example in which this embodiment is applied to the SEND shown in Fig. 2. The intermediate devices 10A and 10B shown in Fig. 7 can be used for the SEND.

[0077] When the local 30 sends SEND Only, intermediate device 10A forwards SEND Only to remote 50, and creates a pseudo response (ACK) from the header of SEND Only and returns it to the local 30. When the local 30 receives the pseudo response, it puts a CQE in the CQ and releases the WQE in the SQ.

[0078] If the remote 50 has successfully received the data, it returns an ACK to the local 30. The intermediate device 20B discards the ACK from the remote 50.

[0079] Next, a description will be given of an example in which this embodiment is applied to the RDMA WRITE shown in Fig. 3. For the RDMA WRITE, intermediate devices 10A and 10B in Fig. 7 can be used.

[0080] When the local 30 sends data by RDMA WRITE, the intermediate device 10A transfers the data to the remote 50, and also creates a pseudo response (ACK) from the RDMA WRITE header and returns it to the local 30. When the local 30 receives the pseudo response, it loads a CQE into the CQ and releases the WQE of the SQ.

[0081] If the remote 50 has successfully received the data, it returns an ACK to the local 30. The intermediate device 10B discards the ACK from the remote 50.

[0082] The RDMA WRITE w / Imm shown in Figure 4 can be applied in the same way as RDMA WRITE.

[0083] Next, a description will be given of an example in which this embodiment is applied to the RDMA READ shown in Fig. 5. For the RDMA READ, intermediate devices 20A and 20B in Fig. 9 can be used.

[0084] When the local device 30 requests data reading with an RDMA Read Request, the intermediate device 20A forwards the initial request to the remote device 50 without discarding it.

[0085] When the remote 50 receives the request, it returns a response to the local 30. The intermediate devices 20A and 20B forward the response to the local 30.

[0086] Intermediate device 20B classifies requests as completed or incomplete based on the response status, and estimates the amount of free space in the SQ of local 30. Intermediate device 20B creates new pseudo requests (pseudo RDMA Read Requests) for the amount of free space in the SQ, and sends them to remote 50. Intermediate device 20B repeats creating and sending pseudo requests until the request is classified as completed based on the response status.

[0087] When the local 30 has successfully received the data, it places a CQE in the CQ, releases the WQE in the SQ, places a new WQE in the SQ, and transmits a new request to the remote 50. The intermediate device 20A discards the request from the local 30.

[0088] Next, a description will be given of an example in which this embodiment is applied to the ATOMIC Operations shown in Fig. 6. In the ATOMIC Operations, the operation differs depending on whether the pre-operation data of the remote 50 may be discarded or the local 30 receives the pre-operation data.

[0089] First, a case where pre-operation data may be discarded will be described. When pre-operation data may be discarded in ATOMIC Operations, intermediate devices 10A and 10B in FIG.

[0090] When the local 30 transmits the ATOMIC Command, the intermediate device 10A forwards the ATOMIC Command to the remote 50 side, and also creates a pseudo response (ATOMIC ACK) from the header of the ATOMIC Command and returns it to the local 30. When the local 30 receives the pseudo response, it loads a CQE into the CQ and releases the WQE of the SQ.

[0091] When remote device 50 receives the ATOMIC Command, it performs the ATOMIC operation and returns the pre-operation data in an ATOMIC ACK. Intermediate device 10B discards the ATOMIC ACK from remote device 50.

[0092] Next, a case will be described in which the local 30 receives pre-operation data. When the local 30 receives pre-operation data, it can use the intermediate devices 20A and 20B in FIG.

[0093] When the local 30 transmits the ATOMIC Command, the intermediate devices 20A and 20B transfer the ATOMIC Command to the remote 50.

[0094] When the remote device 50 receives the ATOMIC Command, it performs the ATOMIC operation and returns the pre-operation data in an ATOMIC ACK. The intermediate devices 20A and 20B forward the ATOMIC ACK to the local device 30.

[0095] Intermediate device 20B classifies requests as completed or incomplete based on the response status, and estimates the amount of free space in the SQ of local 30. Intermediate device 20B creates new pseudo requests (pseudo ATOMIC Commands) for the amount of free space in the SQ, and sends them to remote 50. Intermediate device 20B repeats creating and sending pseudo requests until the request is classified as completed based on the response status.

[0096] If the local 30 successfully receives the ATOMIC ACK, it places a CQE in the CQ, releases the WQE in the SQ, places a new WQE in the SQ, and transmits a new ATOMIC Command to the remote 50. The intermediate device 20A discards the ATOMIC Command from the local 30.

[0097] As described above, intermediate device 10A of this embodiment includes a transfer unit 11 that transfers a request including data sent from local 30 to remote 50, and a generation unit 12 that generates a pseudo response to the request and returns it to local 30. Intermediate device 10B includes a discard unit 13 that discards a response to a request from remote 50. Since local 30 releases the WQE of the SQ in response to the pseudo response from intermediate device 10A, high-bandwidth data transfer can be achieved without waiting for a response from remote 50 even when the RTT between local 30 and remote 50 is large.

[0098] Intermediate device 20B of this embodiment includes a generator 22 that generates a pseudo request based on an initial request requesting data transmission from local 30 to remote 50 and transmits it to remote 50, and a transfer unit 24 that transfers a response including data transmitted from remote 50 to local 30. Intermediate device 20A includes a discard unit 21 that discards subsequent requests from local 30. Since remote 50 transmits data in response to the pseudo request from intermediate device 20B, high-bandwidth data transfer can be achieved without waiting for a request from local 30, even if the RTT between local 30 and remote 50 is large.

[0099] In the above, intermediate devices 10A, 10B, 20A, and 20B are installed between local 30 and remote 50. However, as shown in FIG. 13, intermediate devices 10A and 20A may be configured on a NIC (Network Interface Card) of a device in local 30, and intermediate devices 10B and 20B may be configured on a NIC of a device in remote 50.

[0100] Furthermore, intermediate devices 10A, 10B, 20A, and 20B may be configured as physical servers or virtual servers. A network device such as a switch or a router may have the functions of intermediate devices 10A, 10B, 20A, and 20B.

[0101] An intermediate device having the functions of intermediate device 10A and intermediate device 10B may be disposed on the local 30 side, and an intermediate device having the functions of intermediate device 20A and intermediate device 20B may be disposed on the remote 50 side. Alternatively, intermediate device 10A equipped with discard unit 13 may be disposed on the local 30 side, and intermediate device 20B equipped with discard unit 21 may be disposed on the remote 50 side.

[0102] Intermediate devices 10A, 10B, 20A, and 20B described above can be, for example, a general-purpose computer system including a central processing unit (CPU) 901, memory 902, storage 903, communication device 904, input device 905, and output device 906, as shown in Fig. 14. In this computer system, intermediate devices 10A, 10B, 20A, and 20B are realized by CPU 901 executing a predetermined program loaded onto memory 902. This program can be recorded on a computer-readable recording medium such as a magnetic disk, optical disk, or semiconductor memory, or can be distributed via a network. [Explanation of symbols]

[0103] 10A,10B,20A,20B…Intermediate device 11...Transfer section 12...Generation section 13…Destruction section 21…Destruction section 22...Generation section 23...Control section 24...Transfer section 30…Local 50…Remote

Claims

1. 1. An intermediate device disposed between a first device and a second device that transfer data using remote direct memory access, comprising: a forwarding unit for forwarding a request including data to be transmitted from the first device to the second device; a generation unit that generates a pseudo response to the request and returns the pseudo response to the first device; a discard unit that discards a response to the request from the second device; creating a table that combines the destinations of requests and responses having the same identifier from requests and responses transmitted and received between the first device and the second device; The generation unit obtains a combination including the destination of the request from the table, and sets a destination corresponding to the destination of the request in the combination as a destination of the pseudo response. Intermediate device.

2. 2. The intermediate device according to claim 1, The request includes a source of the request; The generation unit sets a transmission source of the request included in the request as a destination of the pseudo response. Intermediate device.

3. 1. An intermediate device disposed between a first device and a second device that transfer data using remote direct memory access, comprising: a generating unit that generates a pseudo request for requesting data transmission from the first device to the second device based on an initial request for requesting data transmission from the first device to the second device, and transmits the pseudo request to the second device; a transfer unit that transfers a response including data to be transmitted from the second device to the first device; a discard unit for discarding subsequent requests from the first device; Intermediate device.

4. 1. A method of communication by an intermediate device disposed between a first device and a second device that transfer data using remote direct memory access, comprising: forwarding a request including data to be transmitted from the first device to the second device; generating and returning a pseudo response to the request to the first device; discarding a response to the request from the second device; creating a table that combines the destinations of requests and responses having the same identifier from requests and responses transmitted and received between the first device and the second device; In generating the pseudo response, a combination including the destination of the request is obtained from the table, and a destination corresponding to the destination of the request in the combination is set as the destination of the pseudo response. Communication methods.

5. 1. A method of communication by an intermediate device disposed between a first device and a second device that transfer data using remote direct memory access, comprising: generating a pseudo request for requesting data transmission from the first device to the second device based on an initial request for requesting data transmission from the first device to the second device, and transmitting the pseudo request to the second device; forwarding a response including data to be sent from the second device to the first device; Discarding subsequent requests from the first device Communication methods.

6. A program for causing a computer to operate as each unit of the intermediate device according to any one of claims 1 to 3.

7. A relay system disposed between a first device and a second device that transfer data using remote direct memory access, comprising: a first intermediate device on the first device side and a second intermediate device on the second device side; The first intermediate device a forwarding unit that forwards a request including data to be transmitted from the first device to the second device to the second intermediate device; a generator that generates a pseudo response to the request and returns the pseudo response to the first device; The second intermediate device: a forwarding unit that forwards the request to the second device; a discard unit that discards a response to the request from the second device; Relay system.

Images