COMMUNICATION SYSTEM, INTERMEDIATE DEVICE, COMMUNICATION METHOD, AND PROGRAM

Abstract

A communication system comprising a first intermediate device 10A and a second intermediate device 10B, wherein: the first intermediate device comprises a buffer management unit 15 for determining a buffer size on the basis of delay information and maintaining a buffer, a transfer unit 11 that, when data to be transferred to a second device 50 is smaller than the credit of the second intermediate device, transfers a request including the data to the second intermediate device, and a generation unit 12A for generating a pseudo response to the request and returning the pseudo request to a first device; and the second intermediate device comprises a buffer management unit 15 for determining a buffer size on the basis of the delay information and maintaining a buffer, a transfer unit 11 that transfers the request to the second device, stores the request in the buffer, and updates the credit of the second intermediate device, and a discarding unit 12B that discards a response received from the second device, deletes the request stored in the buffer, and updates the credit.

Description

[Technical field]

[0001] The present invention relates to a communication system, an intermediate device, a communication method, and a program. [Background technology]

[0002] With the advancement of communication technology, the provision of new networks that connect broadband and long distances is being considered. A system is being considered in which devices such as transponders installed in optical transmission systems are installed in client systems such as servers in the new networks. In such a system, electrical signals reach the communication partner server without undergoing electrical-to-optical conversion during transmission.

[0003] In optical transmission systems, a technology has been proposed that establishes a high-speed transmission path with few network resources (such as frequency) by selecting an optimal communication mode (such as modulation method, baud rate, and number of carriers) based on the network condition (such as distance and signal quality) (see Patent Documents 1 and 2). Such a technology may enable long-distance, high-speed communication between communication devices with few network resources. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 8,964,581 [Patent Document 2] International Publication No. 2020 / 031514 Summary of the Invention [Problem to be solved by the invention]

[0005] RDMA (Remote Direct Memory Access), which is used in the transport layer of Infiniband, is a communication protocol that enables high-speed and reliable data transfer between communication terminals located far away from each other. RDMA enables high-speed communication by directly accessing the memory area of ​​the receiving device from the memory area of ​​the sending device.

[0006] However, the problem with RDMA is that the longer the network, the lower the transfer performance becomes, because longer lines require more transfer time, and connection-based protocols cannot send the next packet until they receive a packet indicating the data transfer is complete.

[0007] The present invention has been made in consideration of the above circumstances, and an object of the present invention is to realize high-speed and highly reliable data transfer even when the RDMA transfer destination is located a long distance. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, one embodiment of the present invention is a communications system comprising a first intermediate device and a second intermediate device, the first intermediate device and the second intermediate device being disposed between a first device and a second device which transfer data using remote direct memory access, the first intermediate device comprising: a buffer management unit which determines a buffer size based on delay information regarding a network delay between the first device and the second intermediate device and secures a buffer of the buffer size; a transfer unit which, when data to be transferred from the first device to the second device is smaller than the credit of the second intermediate device, transfers a request including the data to the second intermediate device; and a generation unit which generates a pseudo response to the request and returns it to the first device, the second intermediate device comprising: a buffer management unit which determines a buffer size based on the delay information and secures a buffer of the buffer size; a transfer unit which transfers the request to the second device, stores the request in the buffer, and updates its own credit; and a discard unit which discards a response to the request received from the second device, deletes the request stored in the buffer, and updates the credit.

[0009] One aspect of the present invention is an intermediate device disposed between a first device and a second device that transfer data using remote direct memory access, the intermediate device comprising: a buffer management unit that determines a buffer size based on network delay information between the first device and the second device and secures a buffer of the buffer size; a transfer unit that, when data to be transferred from the first device to the second device is smaller than the credit of the intermediate device, transfers a request including the data to the intermediate device of the destination; and a generation unit that generates a pseudo-response to the request and returns it to the first device.

[0010] One aspect of the present invention is a communication method performed by a communication system including a first intermediate device and a second intermediate device, the first intermediate device and the second intermediate device being disposed between a first device and a second device that transfer data using remote direct memory access, the first intermediate device performing the steps of: determining a buffer size based on delay information regarding a network delay between the first device and the second intermediate device, and reserving a buffer of the buffer size; if data to be transferred from the first device to the second device is smaller than the credit of the second intermediate device, forwarding a request including the data to the second intermediate device, and generating a pseudo response to the request and returning it to the first device, and the second intermediate device performing the steps of: determining a buffer size based on the delay information and reserving a buffer of the buffer size; forwarding the request to the second device, storing the request in the buffer and updating its own credit; and discarding the response to the request received from the second device, deleting the request stored in the buffer, and updating the credit.

[0011] One aspect of the present invention is a program for causing a computer to function as the intermediate device. Effect of the Invention

[0012] According to the present invention, high-speed and highly reliable data transfer can be achieved even if the RDMA transfer destination is located a long distance away. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram for explaining the RDMA communication model. [Diagram 2] FIG. 2 is a diagram for explaining RDMA SEND. [Diagram 3] FIG. 3 is a diagram illustrating an example of a configuration of a communication system according to the first embodiment. [Figure 4] FIG. 4 illustrates an example of the configuration of the first intermediate device. [Diagram 5] FIG. 5 illustrates an example of the configuration of the second intermediate device. [Figure 6] FIG. 6 is a diagram illustrating an example of a local configuration. [Figure 7] FIG. 7 is a sequence diagram showing an example of a process flow of the communication system shown in FIG. [Figure 8] FIG. 8 is a diagram illustrating an example of a method for creating a table to resolve the destination QPN of a response. [Figure 9] FIG. 9 is a diagram illustrating an example of a method for notifying a source QPN and resolving a destination QPN of a response. [Figure 10] FIG. 10 illustrates an example of a configuration of a communication system according to the second embodiment. [Figure 11] FIG. 11 illustrates an example of the configuration of the orchestrator. [Figure 12] FIG. 12 is a diagram illustrating an example of the configuration of a transmission device. [Figure 13] FIG. 13 is a sequence diagram showing an example of a process flow of the communication system shown in FIG. [Figure 14] FIG. 14 is a diagram illustrating an example of a configuration of a communication system according to the first modification of the second embodiment. [Figure 15] FIG. 15 is a sequence diagram illustrating an example of a processing flow of a communication system according to the second modification of the second embodiment. [Figure 16] FIG. 16 is a diagram illustrating an example of a configuration of a communication system according to the third modification of the second embodiment. [Figure 17] FIG. 17 is a diagram illustrating an example of a configuration of a communication system according to the third embodiment. [Figure 18] FIG. 18 is a diagram illustrating an example of the configuration of a communication system in the case where communication does not go through an intermediate device according to the third embodiment. [Figure 19] FIG. 19 is a diagram illustrating an example of the configuration of another communication system according to the third embodiment. [Figure 20] 2 is an example of a hardware configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0015] 1. About RDMA RDMA is a communication protocol that directly accesses memory from the memory area of ​​a sending device to the memory area of ​​a receiving device. RDMA has a credit-based flow control function, and performs completion control to confirm the completion of data transfer before proceeding with processing, enabling highly reliable communication. RDMA is also used as a transport method for host-to-device and device-to-device data communication between SSDs (Solid State Drives) and GPUs (Graphics Processing Units).

[0016] As shown in Figure 1, RDMA is a communication model that configures a QP (Queue Pair) between a local device and a remote device and transfers data using the QP. A QP is a set of an SQ (Send Queue) and an RQ (Receive Queue). The communication unit of RDMA is a communication request called a WR (Work Request), which is stacked in the SQ / RQ in units of WQE (Work Queue Element). WR includes Send WR, which is a request to send, and Receive WR, which is a request to receive.

[0017] In the Send WR, the memory area for the data to be sent is specified as a WQE and pushed onto the SQ. In the Receive WR, the memory area for receiving data is specified as a WQE and pushed onto the RQ. WQEs can be pushed onto the SQ / RQ in FIFO (First-In-First-Out) order up to the queue size of the SQ / RQ. When WQE processing is completed successfully between QPs, a CQE (Completion Queue Entry) indicating successful completion is pushed onto the CQ (Completion Queue) corresponding to each SQ / RQ. When WQE processing ends with an error between QPs, a CQE indicating the error is pushed onto the CQ. When the WQE is completed successfully, the WQE in the SQ / RQ is deleted, making it possible to accept the next WQE.

[0018] RDMA service types are broadly classified into four types based on the classification of Reliable / Unreliable and Connection / Datagram: RC (Reliable Connection), RD (Reliable Datagram), UC (Unreliable Connection), and UD (Unreliable Datagram). RC and UD are commonly used.

[0019] RC guarantees the order and delivery of messages by checking the success or failure of communication by ACK / NAK and by using a mechanism for retransmission. RC is also connection-oriented, and performs one-to-one communication between local and remote QPs.

[0020] Although UD does not have a mechanism for acknowledgment or retransmission, it enables many-to-many communication, such as sending to multiple QPs and receiving from multiple QPs by specifying the destination for each communication.

[0021] RDMA operation types are broadly divided into four types: SEND, RDMA WRITE (WITH Immediate), RDMA READ, and ATOMIC Operations. All of these can be used in RC. Only SEND can be used in UD.

[0022] Retransmission control in RDMA is classified into three patterns: when ACK / NAK is not returned, when an RNR (Receiver-Not-Ready) NAK is returned, and when an Out-Of-sequence NAK is returned. If an ACK or NAK is not returned from the remote side within a certain time, the local side will time out and retransmit. The remote side will also return an RNR NAK if a WQE cannot be prepared for the RQ. If an RNR NAK is returned from the remote side, the local side will retransmit after a certain time. The remote side will also return an Out-of-sequence NAK if the PSN (Packet Sequence Number) of the received packet is not in order. If an Out-Of-sequence NAK is returned from the remote side, the local side will retransmit without waiting.

[0023] Next, we will explain the operation of the service type RC, taking SEND as an example.

[0024] 2 is an explanatory diagram for explaining SEND of RDMA. SEND is a basic transmission and reception model of RDMA, and transmits data from a local to a remote.

[0025] The local prepares an SQ, the remote prepares an RQ, and the local and remote load WQE into the SQ and RQ respectively.

[0026] When communication is ready, the local sends data with SEND. When the remote receives the data successfully, it places a CQE in the CQ, releases the WQE in the RQ, and returns an ACK to the local. When the local receives an ACK, it places a CQE in the CQ, and releases the WQE in the SQ.

[0027] SEND also provides a special operation, SEND w / Imm (SEND with Immediate). With SEND w / Imm, a special field (imm_data) can be set in the WQE of the local SQ, and imm_data can be sent at the same time as sending data from the local to the remote. When the remote successfully receives the data, it places a CQE containing imm_data in the CQ. The remote can find out the contents of imm_data by referencing the CQE.

[0028] 2. First embodiment <Communication system configuration> Next, with reference to FIG. 3, an example of the configuration of a communication system including intermediate devices 10A and 10B of this embodiment will be described.

[0029] Intermediate devices 10A and 10B are placed between local 30 and remote 50, which transfer data using RDMA. More specifically, intermediate device 10A is placed in front of a long-distance network 9 on the local 30 side, and intermediate device 10B is placed in front of network 9 on the remote 50 side.

[0030] As a data transfer function, intermediate device 10A receives a request (SEND, etc.) from local 30, and returns a pseudo-response to that request to local 30. Intermediate device 10B transfers the request sent from intermediate device 10A to remote 50, and discards the response (ACK, etc.) from remote 50. As a flow control function, intermediate devices 10A and 10B implement credit-based flow control, and periodically transmit the credit (free buffer capacity) of their own device to the other device at a predetermined timing.

[0031] In addition to intermediate devices 10A and 10B, the communication system may include a local 30 and a remote 50 that transfer data using RDMA. The local 30 (first device) is a device that transfers data. The remote 50 (second device) is a device that transfers data to.

[0032] 4 shows an example of the configuration of intermediate device 10A (first intermediate device). Intermediate device 10A includes a forwarding unit 11, a generating unit 12A, a network status measuring unit 13, a buffer 14 (temporary data storage unit), a buffer management unit 15, a credit management unit 16, and a communication unit 17.

[0033] In the illustrated example, forwarding unit 11, generating unit 12A, and network status measuring unit 13 are implemented on software (CPU, memory, storage, etc.), but these may be implemented on a NIC. Also, in the illustrated example, buffer 14, buffer management unit 15, credit management unit 16, and communication unit 17 are implemented on a NIC, but some of these may be implemented on software. Also, intermediate device 10A may be implemented as a virtual machine or a container.

[0034] The transfer unit 11 receives a request from the local 30 and transfers it to the remote 50. This request is, for example, the above-mentioned SEND, SEND w / Imm, RDMA WRITE, RDMA WRITE w / Imm, or ATOMIC Command. The request includes data to be transmitted from the local 30 to the remote 50 or an operation on the data. The transfer unit 11 transmits the request via the communication unit 17.

[0035] In this embodiment, when data (data size) to be transferred from local 30 to remote 50 is smaller than the credit (free size of the buffer) of intermediate device 10B, transfer unit 11 transfers a request including the data to intermediate device 10B. Credit management unit 16 periodically receives credits transmitted from intermediate device 10B.

[0036] In this embodiment, forwarder 11 transmits data smaller than the credit of intermediate device 10B, but, for example, forwarder 11 may estimate the credit of buffer 14 of intermediate device 10B from the credit release notification history transmitted from intermediate device 10B, and transmit data of a size exceeding the credit notified by intermediate device 10B as long as it is within the range of the estimated credit. Further, forwarder 11 may transmit data of a size exceeding the credit notified by intermediate device 10B in cases such as when the retransmission cost is estimated to be small, or when it is estimated that there will be free space in the buffer of intermediate device 10B by the time the communication data reaches intermediate device 10B. This makes it possible to improve the efficiency of data communication in credit-based flow control.

[0037] On the other hand, if the amount of data to be transferred from local 30 to remote 50 is equal to or greater than the credits sent from intermediate device 10B, that is, if the credits are insufficient, transfer unit 11 queues (stores) the request in buffer 14 and waits until the credits of intermediate device 10B increase. In this case, transfer unit 11 updates the credits of its own buffer 14.

[0038] When generating unit 12A transfers the request to intermediate device 10B, it generates a pseudo-response to the request and returns it to local 30. Specifically, generating unit 12A picks up a request that has been sent from local 30 and has an Only or Last flag set, and generates a pseudo-response using the PSN included in the request. Generating unit 12A returns the generated pseudo-response to local 30. Note that the pseudo-response to the request (e.g., pseudo ACK) uses the same PSN value as the Only or Last request.

[0039] When the local 30 receives the pseudo response, it recognizes it as a response from the remote 50, puts a CQE in the CQ, and completes the process normally. This allows the local 30 to forcibly release the WQE of the SQ.

[0040] Network status measurement unit 13 measures the status of network 9 between intermediate devices 10A and 10B. Specifically, network status measurement unit 13 measures delay information related to delays in network 9 by transmitting packets, etc. The delay information includes the transmission delay and transmission capacity of network 9 (the path between intermediate devices 10A and 10B).

[0041] The buffer management unit 15 acquires delay information of the network 9 between the local 30 and the remote 50, determines the buffer size of the buffer 14 based on the delay information, and secures the buffer 14 of the determined size. For example, the buffer management unit 15 may calculate the buffer size as a value that is at least twice the product of the transmission delay of the delay information and the transmission capacity (bandwidth). Data is temporarily stored in the buffer 14.

[0042] Credit management unit 16 performs credit-based flow control. Credit management unit 16 manages data related to flow control such as credits. Credit management unit 16 periodically receives a credit signal from intermediate device 10B via communication unit 17, and updates the credit stored in credit management unit 16 using the credit signal and a predetermined credit update method. The credit signal received from intermediate device 10B includes a credit indicating a free buffer size that can be received by intermediate device 10B (Kung & Morris, Credit-based flow control for ATM networks, IEEE Network, 9(2), 40-48 (1995)). Credit management unit 16 may also periodically transmit the credit (free buffer size) of its own buffer 14 to intermediate device 10B via communication unit 17.

[0043] Communication unit 17 is a network interface for communicating with local 30 and other devices such as intermediate device 10B.

[0044] 5 shows the configuration of intermediate device 10B (second intermediate device). Intermediate device 10B includes a forwarding unit 11, a discarding unit 12B, a network status measuring unit 13, a buffer 14, a buffer management unit 15, a credit management unit 16, and a communication unit 17.

[0045] The transfer unit 11 transfers a request sent by the local 30 via the intermediate device 10A to the remote 50. The transfer unit 11 of this embodiment transfers a request to the remote 50, queues the request in the buffer 14, and updates its own credit.

[0046] The discard unit 12B discards a response to a request received from the remote 50. This makes it possible to prevent the local 30 from receiving duplicate responses. Furthermore, since an RNR or out-of-sequence NAK sent from the remote 50 may cause a malfunction when it arrives at the local 30, the discard unit 12B also discards these NAKs. The discard unit 12B of this embodiment discards a response to the request received from the remote 50, deletes the request queued in the buffer 14, and updates the credit.

[0047] Network status measurement unit 13, buffer 14, buffer management unit 15, credit management unit 16 and communication unit 17 of intermediate device 10B are similar to network status measurement unit 13, buffer 14, buffer management unit 15, credit management unit 16 and communication unit 17 of intermediate device 10A.

[0048] 6 shows the configuration of the local 30. The local 30 includes an application unit 31, a queue management unit 32, a network status management unit 33, a temporary data storage unit 34, a determination unit 35, a packet sorting unit 36, and a communication unit 37.

[0049] The application unit 31 transmits and receives a request including transfer data to and from the remote 50 using RDMA communication.

[0050] The queue management unit 32 manages each queue (CQ, SQ) of the temporary data storage unit 34. The queue management unit 32 may calculate the necessary queue size based on delay information of the network 9 and determine the queue depth (QueueDepth) of the temporary data storage unit 34. The temporary data storage unit 34 of the local 30 has a CQ and an SQ as queues.

[0051] The network status management unit 33 manages the status of the network 9. Specifically, the network status management unit 33 may obtain delay information related to delays in the network 9 from other devices, or may measure the delay information by transmitting packets, etc. The delay information includes transmission delays, transmission capacity, etc.

[0052] The allocator 36 allocates data (packets) received via the communication unit 37 to the corresponding QP in the temporary data storage unit 34. If the received data is delay information, the allocator 36 allocates the data to the network status manager 33 or the queue manager 32.

[0053] Determination unit 35 may determine whether or not to transmit a request via intermediate devices 10A, 10B based on the delay information. When queue management unit 32 determines that the request will not be transmitted via intermediate devices 10A, 10B, it may determine the queue depth of temporary data storage unit 34 based on the delay information. Local 30 of this embodiment may not include network status management unit 33 and determination unit 35.

[0054] Communication unit 37 is a network interface for communicating with other devices such as intermediate device 10A.

[0055] The queue management unit 32 and the network status management unit 33 may be implemented as functions of a NIC. The application unit 31 is not limited to being implemented in the CPU and memory of the local 30, and may be implemented in a hardware accelerator such as a GPU, FPGA, or NIC connected via a separate internal bus.

[0056] The configuration of the remote 50 is similar to that of the local 30 shown in Fig. 6. However, the temporary data storage unit 34 of the remote 50 includes a CQ and an RQ.

[0057] <Communication system operation> An example of the flow of processing in the communication system of this embodiment will be described with reference to the sequence diagram of FIG.

[0058] In step S10, each of intermediate devices 10A and 10B measures delay information (transmission delay, transmission capacity) of network 9 and secures buffer 14 based on the measurement results. Specifically, intermediate devices 10A and 10B measure the state of network 9 by, for example, transmitting packets. Intermediate devices 10A and 10B calculate a required buffer size based on the delay information and secure buffer 14 of that buffer size. For example, intermediate devices 10A and 10B may calculate a value that is at least twice the product of the transmission delay time and the transmission capacity (bandwidth) as the buffer size.

[0059] In step S10, intermediate devices 10A and 10B may exchange communication modes available in their own devices with the other device based on the measurement results in step S10 to determine the communication mode to be used.

[0060] Next, in step S20, local 30 and remote 50 transmit and receive data via intermediate devices 10A and 10B. Specifically, local 30 loads a WQE into an SQ and transmits a request to remote 50 (step S21). Here, a case where a SEND request is transmitted will be described as an example.

[0061] Intermediate device 10A compares the data size of the request received from local 30 with the credit received from intermediate device 10B. If the data size of the request is smaller than the credit of intermediate device 10B, that is, if intermediate device 10A has remaining credit, intermediate device 10A transmits the request to intermediate device 10B (step S22). At this time, intermediate device 10A may queue the request received from local 30 in a buffer and update its own credit.

[0062] The intermediate device 10A generates a pseudo response (pseudo ACK) using the PSN included in the request, and returns the pseudo response to the local 30 (step S23). Upon receiving the pseudo response, the local 30 loads a CQE into the CQ, and releases the WQE of the SQ.

[0063] If no packet loss has occurred in the request sent from intermediate device 10A, intermediate device 10B transmits the request sent from intermediate device 10A to remote 50 (step S24), queues the request in buffer 14, and updates the credit (step S25).

[0064] On the other hand, if packet loss has occurred, intermediate device 10B transmits a NACK to intermediate device 10A (step S26) and receives the resent request from intermediate device 10A (step S27). If packet loss has not occurred in the resent request, intermediate device 10B proceeds to S24.

[0065] Furthermore, if the data size of the request in step S21 is equal to or larger than the credit of intermediate device 10B, that is, if intermediate device 10B has insufficient credit, intermediate device 10A queues the request in buffer 14 and waits until intermediate device 10B's credit increases (step S28). If intermediate device 10B's credit becomes larger than the data size of the request, intermediate device 10A proceeds to S22.

[0066] If remote 50 successfully receives the request in step S24, it transmits an ACK response to intermediate device 10B (step S29). Intermediate device 10B discards the response received from remote 50, releases the request queued in buffer 14 in step S25, and updates the credit (step S30).

[0067] On the other hand, if remote 50 does not normally receive the request of step S24, it transmits a NACK response to intermediate device 10B (step S31) and receives the request resent from intermediate device 10B (step S32). If the resent request is normally received, remote 50 proceeds to step S29, and intermediate device 10B performs step S30.

[0068] Asynchronously with the data transmission and reception of S20, intermediate device 10B periodically transmits its own credit to intermediate device 10A at a predetermined timing using a timer (not shown) or the like (step S30). As a result, intermediate device 10A obtains the credit of intermediate device 10B and updates the credit of intermediate device 10B stored in credit management unit 16.

[0069] <Destination QPN resolution> In the RDMA interface, a QP has a different QPN for each endpoint. The SQ / RQ recognizes the QPN of the other end, and includes the destination QPN in the header when generating an RDMA packet. However, the source QPN is not included in the header. When intermediate device 10A generates a pseudo response, the received request does not contain information indicating the source QPN, so the destination of the pseudo response is unknown. Therefore, in this embodiment, the destination of the pseudo response is specified by the following two methods.

[0070] The first method is to inspect the exchange of the original RDMA request and response, and store the combination of QPNs in a table. The same PSN is used for the Only or Last request and ACK of an RDMA packet. Therefore, the intermediate device 10A inspects the request and response passing through, and adds the destination QPNs of the headers of the Only or Last request and ACK having the same PSN to the table as a combination. In the example of FIG. 8, the destination QPNs of the headers of the request and response having the same PSN are 0x000020 and 0x000010, respectively, so the combination of 0x000010 and 0x000020 is added to the table. In FIG. 8, the local 30 configures a QP between each of the remotes 50A and 50B.

[0071] When intermediate device 10A generates a pseudo response, it obtains from the table a combination of QPNs that includes the destination QPN of the request, and sets the other QPN of the combination as the destination QPN of the pseudo response. For example, when a request with a destination QPN of 0x000020 is received, intermediate device 10A obtains from the table the combination of 0x000010 and 0x000020, which includes 0x000020, and sets the destination QPN of the pseudo response to 0x000010.

[0072] The second method is to put the Source QPN in the RDMA packet. The WQE has a 32-bit immDt (immediate Date) field, and any 32-bit information can be written in the immDt field only for SEND with immediate or RDMA WRITE with immediate. In Fig. 9, the local 30 has an insertion unit 38, which writes the QPN of the SQ on the local 30 side into the immDt field of the WQE of the SQ on the local 30 side.

[0073] When intermediate device 10A generates a pseudo response, it sets the QPN written in the immDt field of the received request as the destination QPN of the pseudo response.

[0074] The present embodiment described above is a communications system including intermediate device 10A and intermediate device 10B, which are disposed between local 30 and remote 50, which transfer data using RDMA, and intermediate device 10A is equipped with a buffer management unit 15 that determines a buffer size based on delay information relating to the delay in network 9 between intermediate device 10B and intermediate device 10A and secures buffer 14 of said buffer size, a transfer unit 11 that, when data to be transferred from local 30 to remote 50 is smaller than the credit of intermediate device 10B, transfers a request including said data to intermediate device 10B, and a generation unit 12A that generates a pseudo-response to the request and returns it to local 30. The intermediate device 10B includes a buffer management unit 15 that determines a buffer size based on delay information and secures a buffer 14 of the buffer size, a transfer unit 11 that transfers the request to the remote 50, stores the request in the buffer 14, and updates its own credit, and a discard unit 12B that discards a response to the request received from the remote 50, deletes the request stored in the buffer 14, and updates the credit.

[0075] In this embodiment, the local 30 releases the WQE of the SQ in response to a pseudo response from the intermediate device 10A, so that high-bandwidth data transfer can be achieved without waiting for a response from the remote 50 even if the RTT (Round Trip Time) between the local 30 and the remote 50 is long.

[0076] Furthermore, in this embodiment, by introducing credit-based flow control to intermediate devices 10A and 10B, it is possible to prevent data from being transmitted that exceeds the performance of intermediate device 10B on the remote side 50. That is, intermediate device 10A transmits data taking into consideration the amount of data that intermediate device 10B can receive. This makes it possible to prevent transmission of data that exceeds the performance of intermediate device 10B, and to avoid data loss on the remote side. Therefore, in this embodiment, even if the RDMA transfer destination is a long distance, high-speed and highly reliable data transfer can be achieved, and communication between intermediate devices 10A and 10B can be guaranteed.

[0077] Furthermore, in this embodiment, by providing intermediate devices 10A and 10B with a retransmission function for NACK, data loss can be handled.

[0078] 3. Second embodiment <Communication system configuration> In the communications system of the first embodiment, when starting data transfer, intermediate devices 10A and 10B measure the state of network 9, so it takes time before data transfer starts. In this embodiment, orchestrator 70 calculates or measures a delay when setting an optical path (transmission line) of network 9, and notifies intermediate devices 10A and 10B of the delay. As a result, in this embodiment, the measurement process of the state of network 9 by intermediate devices 10A and 10B of the first embodiment can be omitted, and the time until data transfer starts can be shortened.

[0079] 10 is a diagram showing an example of a configuration of a communication system according to the present embodiment. The communication system according to the present embodiment includes intermediate devices 10A and 10B and an orchestrator .

[0080] Intermediate devices 10A and 10B of this embodiment are similar to intermediate devices 10A and 10B of the first embodiment (FIGS. 4 and 5). However, network status measurement unit 13 of intermediate devices 10A and 10B of this embodiment acquires delay information related to delay (transmission delay, transmission capacity, optical path information, etc.) from orchestrator 70 without measuring the network 9 status.

[0081] The orchestrator 70 monitors and manages the entire network 9 such as the optical transport network, and controls it centrally. The orchestrator 70 operates between the upper layer computers (local 30, remote 50) and the lower layer optical transport network, and configures and controls optical network devices based on the communication requirements from the computers and the state of the optical transmission paths in the optical transport network, thereby constructing optical paths within the optical transport network. This makes it possible to automatically and optimally control the optical transport network in response to requests from geographically distributed computers without human intervention.

[0082] 9 is a diagram showing an example of the configuration of the orchestrator 70. The orchestrator 70 of this embodiment sets an optical path in the network 9 (optical transport network) in response to a communication requirement from the local 30, and calculates or measures delay information of the optical path.

[0083] The orchestrator 70 shown in the figure includes a communication request receiving unit 71, an ACK sending unit 72, an ACK receiving unit 73, a result output and sending unit 74, a scheduler unit 75, a design unit 76, a topology information storage unit 77, a node information storage unit 78, and a status monitoring and management unit 79.

[0084] The communication request receiving unit 71 receives a communication request including communication requirements related to communication from the local 30 (or remote 50). The communication requirements include the type of application to be communicated, the required bandwidth, the total amount of data, the allowable delay time, the task completion time, the bit error rate (BER), power, etc. The communication request includes completing the task within 10 msec, backing up data, transferring 1 Tbite of data from the local 30 to the remote 50, etc.

[0085] After receiving a setting completion response of the setting information from each of the transmission devices 20A, 20B (optical network devices) in the network 9, the ACK sending unit 72 sends an optical path setting completion notification to the local 30 and the remote 50. In other words, after the establishment of an optical path is completed in the network 9, the ACK sending unit 72 notifies the requesting local 30 of an ACK indicating the completion of the establishment of the optical path as soon as the requesting local 30 and the requesting remote 50 are able to communicate with each other.

[0086] The ACK receiving unit 73 receives, from each of the transmission devices 20A and 20B, a setting completion response of setting information to the optical network device included in the device.

[0087] The result output / transmission unit 74 transmits setting information for the optical network devices of each of the transmission devices 20A and 20B to each of the transmission devices 20A and 20B. In other words, the result output / transmission unit 74 transmits the following setting information designed and selected for optimal control of the network 9 to the optical network devices constituting the network 9.

[0088] Optical paths between computers OEO (optical-electro-optical) conversion point (= DSP insertion point) (i) modulation method, (ii) baud rate, (iii) transmission power, and (iv) FEC (Forward Error Correction) overhead to be set in the transceiver of each of the transmission devices 20A and 20B More specifically, the following setting information is assumed:

[0089] The settings of (i) modulation method, (ii) baud rate, (iii) transmission power, and (iv) FEC (overhead) for the optical paths between computers, OEO conversion points, and optical network devices present in each transmission device 20A and 20B, which are designed and selected to meet the communication requirements Designed to minimize resource usage in the entire optical transport network. The settings of (i) modulation method, (ii) baud rate, (iii) transmission power, and (iv) FEC (overhead) for the selected optical paths between computers, OEO conversion points, and optical network devices in each transmission device 20A and 20B. Designed to minimize the power consumption of the entire optical transport network. The settings of (i) modulation method, (ii) baud rate, (iii) transmission power, and (iv) FEC (overhead) for the selected optical paths between computers, OEO conversion points, and optical network devices in each transmission device 20A and 20B. The result output / transmission unit 74 transmits the setting information to each of the transmission devices 20A and 20B, and causes the optical network devices of the transmission devices 20A and 20B to set the setting information. The optical network devices include transceivers (transponders), optical cross connects (OXCs), reconfigurable optical add / drop multiplexers (ROADMs), amplifiers, etc.

[0090] The scheduler unit 75 manages and schedules communication requests received from the local 30. Since it takes a certain amount of time to complete the design and opening of an optical path, when multiple communication requests are present, the scheduler unit 75 schedules communication requests in order of priority.

[0091] The design unit 76 determines an optical path to be used for communication of the local 30 based on the communication requirements received from the local 30 and the state of the optical transmission path of the network 9, and calculates configuration information to be set in the optical network devices of each of the transmission devices 20A and 20B in order to construct the optical path.

[0092] The design unit 76 includes an optical path design / selection unit 761, a required bandwidth calculation unit 762, a transmission mode selection unit 763, a power calculation unit 764, an OSNR calculation unit 765, a BER calculation unit 766, a multi-flow determination unit 767, a delay calculation unit 768, and a task completion time calculation unit 769.

[0093] The optical path design and selection unit 761 lists a set of candidate optical paths and selects an optimal optical path based on the communication requirements received from the local 30. "Optimal" refers to, for example, optimal delay, task completion time, required bandwidth, resource usage of the entire network 9, and power consumption of the entire network 9.

[0094] To select the optimal optical path, the optical path design and selection unit 761 cooperates with the required bandwidth calculation unit 762 through the delay measurement unit 770 and refers to information on the network 9 stored in the topology information storage unit 77 through the status monitoring and management unit 79. There is a trade-off between transmission capacity and distance, and the reference documents 1 to 3 described below can be used to select the optimal optical transmission mode according to the distance while taking that relationship into account.

[0095] The necessary bandwidth calculation unit 762 calculates a necessary bandwidth based on the communication requirements received from the local 30 in communication between the local 30 and the remote 50 (request source, request destination).

[0096] The transmission mode selection unit 763 lists candidate transmission modes based on the bandwidth and type of application required for communication between the local 30 and the remote 50. The transmission mode selection unit 763 calculates the modulation method, baud rate, FEC, etc. to be set in each transceiver for the candidate transmission mode on a candidate optical path between the local 30 and the remote 50. Note that reference 4 (WO2020 / 031514 A1) can be used for a method of selecting the optimal transmission mode based on these parameters.

[0097] The power calculation unit 764 calculates the appropriate transmit power to be set for each transceiver in a candidate lightpath between the Local 30 and the Remote 50 .

[0098] The OSNR calculation unit 765 calculates the OSNR on a candidate optical path between the local 30 and the remote 50. The OSNR calculation unit 765 uses the network state of an optical path between the local 30 and the remote 50 as an input value, and the OSNR on the input optical path as an output value. Reference 1 can be used for the calculation method of the OSNR.

[0099] Reference 1 is "A. Ferrari et al., "GNPy: an open source application for physical layer aware open optical networks," in Journal of Optical Communications and Networking, vol. 12, no. 6, 2020, pp. C31-C40."

[0100] The BER calculation unit 766 calculates the BER based on the OSNR and margin on a certain optical path. For example, referring to Reference 2, the BER when an optical path p is established with a wavelength λ is p,λ can be calculated using equation (1).

[0101] BER p,λ =ψ(OSNR p,λ -M T (τ)-M d (τ)) (1) Here, ψ(.) is the formula for calculating the BER according to a specific modulation format of a specific optical path. M T (τ) is the system margin that takes into account aging degradation. M d (τ) is the design margin.

[0102] Reference 2 is “P. Soumplis et al., “Multi-period planning with actual physical and traffic conditions”, in IEEE / OSA Journal of Optical Communications and Networking, vol.10, no.1, 2018, p.A144-p.A153”.

[0103] The multi-flow determination unit 767 determines whether or not multi-flow is required on a section constituting a candidate optical path between the local 30 and the remote 50, based on the communication requirements received from the local 30. Regarding technology related to multi-flow communication, Reference 3 can be used.

[0104] Reference 3 and "M. Jinno et al., "Multiflow optical transponder for efficient multilayer optical networking," in IEEE Communications Magazine, vol. 50, no. 5, 2012, pp. 56-65."

[0105] The delay calculation unit 768 calculates the transmission delay time on an optical path between the local 30 and the remote 50. In calculating the transmission delay time, the delay calculation unit 768 takes an optical path between the local 30 and the remote 50 and topology information of the network 9 as input values, and takes the transmission delay on the input optical path as an output value. The transmission delay time is basically determined by the distance between the nodes.

[0106] The task completion time calculation unit 769 calculates the task completion time required to complete a communication task based on the selected candidate optical paths.

[0107] The delay measurement unit 770 may actually measure a transmission delay time on an optical path between the local 30 and the remote 50 using each of the transmission devices 20A and 20B. The transmission delay time may be calculated by the delay calculation unit 768 or may be measured by the delay measurement unit 770.

[0108] The topology information storage unit 77 holds topology information relating to the connection and distance between the transmission devices 20A and 20B. The topology information is necessary for optical path design, multi-flow determination, and the like.

[0109] The node information storage unit 78 holds node information on the type and number of transceivers present in the transmission devices 20A and 20B. The node information is necessary for optical path design because the receiving sensitivity (required OSNR) differs depending on the type of transceiver. The node information storage unit 78 also holds node information on the type and number of amplifiers present in each of the transmission devices 20A and 20B. The node information is necessary because the noise figure (NF) differs depending on the type of amplifier, such as Raman amplification or EDFA (Erbium Doped Fiber Amplifier). The node information storage unit 78 also holds node information on the type and form (number of ports, etc.) of optical nodes (OXC, ROADM) present in each of the transmission devices 20A and 20B.

[0110] The status monitoring and management unit 79 holds monitoring and management information on the usage and status of the amplifiers, transceivers, and optical nodes present in each of the transmission devices 20A and 20B. The status monitoring and management unit 79 also holds monitoring and management information on the usage status of wavelengths in each link, aging deterioration, splices in each link, connector locations, and other optical signal losses. This monitoring and management information is necessary when selecting an optical path (wavelength path) because nonlinear effects (especially XPM (Cross Phase Modulation) and FWM (Four-wave mixing)) affect losses and BER depending on the wavelength usage status. The status monitoring and management unit 79 updates the stored information in response to the addition and deletion of optical paths. The status monitoring and management unit 79 is connected to the optical transmission path measurement unit of each of the transmission devices 20A and 20B, and periodically receives transmission path information on the status and margin of the transmission path, and stores the transmission path information in a manner that allows it to be updated sequentially.

[0111] 12 is a diagram illustrating an example of the configuration of each of the transmission devices 20A and 20B. The transmission devices 20A and 20B are nodes that configure the network 9. Each of the transmission devices 20A and 20B includes a result receiving unit 21, a control unit 22, an optical NW device 23, an ACK sending unit 24, and an optical transmission path measurement unit 25.

[0112] The result receiver 21 receives setting information for the optical NW device 23 of its own node from the orchestrator 70. That is, the result receiver 21 receives setting information to be set in the optical NW device 23 from the orchestrator 70, and passes the setting information to the control unit 22.

[0113] The control unit 22 sets and controls the optical NW device 23 of the own node based on the setting information received from the orchestrator 70. That is, the control unit 22 sets and controls the optical NW device 23 based on the setting information received from the orchestrator 70 as follows.

[0114] -Setting the transmission mode (modulation method, baud rate, FEC, etc.) and transmission power for each transceiver (transponder) -Set wavelengths to be added / dropped / through to ROADM or OXC - Allocation of wavelengths to be amplified to amplifiers Upon completion of the setting and control, the control unit 22 notifies the ACK transmission unit 24 of the completion.

[0115] The optical network device 23 is a device that constitutes the network 9, such as a transceiver, an OXC, a ROADM, an amplifier, and the like.

[0116] After completing the setting of the setting information in the optical NW device 23, the ACK sending unit 24 sends a setting completion response to the orchestrator 70. In other words, as soon as the setting and control of the optical NW device 23 is completed, the ACK sending unit 24 notifies the orchestrator 70 of the completion of the setting and control.

[0117] The optical transmission path measurement unit 25 measures the state and margin of the optical transmission path (optical bus) of the network 9 and transmits the results to the orchestrator 70. In other words, the optical transmission path measurement unit 25 is a measurement device that measures the state and margin of the actual optical transmission path. For example, the optical transmission path measurement unit 25 estimates and calculates the state of the optical transmission path periodically or upon request based on a signal received by a coherent DSP, as described in Reference 5, and transmits state information of the optical transmission path to the orchestrator 70. The transmission path measurement unit 25 may measure delay information related to delays such as the transmission delay time and transmission capacity of the optical transmission path.

[0118] Reference 5 is “T. Sasai et al., “Simultaneous Detection of Anomaly Points and Fiber types in Multi-Span Transmission Links Only by Receiver-side Digital Signal Processing”, in Optical Fiber Communication Conference (pp. Th1F-1), Optical Society of America, 2020.”

[0119] In this embodiment, when communicating with the remote 50, the application unit 31 of the local 30 transmits a communication request including communication requirements related to the communication to the orchestrator 70. In addition, the network status management unit 33 receives, from the orchestrator 70, a notification of completion of setting of an optical path constructed in the network 9 (ACK), and delay information of the set optical path. The delay information includes information on the transmission delay time, transmission capacity, and optical path. The local 30 may be equipped with a wavelength-tunable transceiver capable of dynamically switching wavelengths. The remote 50 is similar to the local 30.

[0120] <Communication system operation> FIG. 13 is a sequence diagram showing the operation of the communication system of the present embodiment.

[0121] To communicate with the remote 50, the local 30 transmits a communication request (path setting request) specifying communication requirements to the orchestrator 70 (step S51). The communication request is the transfer of data or a file, the transmission or distribution of video, etc. The communication requirements include the type of application performing the communication, the required bandwidth, the total amount of data, the allowable delay time, the task completion time, the BER, the power, etc. The local 30 may transmit the communication request to the intermediate device 10A or the transmission device 20A. In that case, the intermediate device 10A or the transmission device 20A transmits the communication request to the orchestrator 70.

[0122] The orchestrator 70 determines an optical path to be used for communication of the local 30 based on the communication requirements from the local 30 and the status of the network 9 previously received from each transmission device disposed in the network 9, and calculates the configuration information to be set in the optical network device 23 of each transmission device 20 to construct the optical path (step S52).

[0123] For example, the orchestrator 70 calculates configuration information regarding (i) modulation method, (ii) symbol rate, (iii) transmission power, and (iv) FEC (overhead) to be set for the optical path between the local 30 and the remote 50, the OEO conversion point, and the transceiver of each transmission device 20.

[0124] At that time, the orchestrator 70 performs calculations to optimize delay, task completion time, required bandwidth, resource usage of the entire network 9, and power consumption of the entire network 9. For example, the orchestrator 70 calculates setting information to satisfy the above communication requirements, and also calculates setting information to minimize resource usage of the entire network 9 and / or calculates setting information to minimize power usage of the entire network 9.

[0125] The orchestrator 70 transmits the above setting information to each of the transmission device 20A and the transmission device 20B, and instructs them to set the optical bus (steps S53 and S54).

[0126] The transmission device 20A and the transmission device 20B each set and control the above setting information in the optical NW device 23 of the own node. For example, the transmission device 20A and the transmission device 20B set the transmission mode (modulation method, symbol rate, FEC, etc.) and transmission power for each transceiver based on the above setting information, set wavelengths to be added / dropped / through in the ROADM and the OXC, and assign wavelengths to be amplified to the amplifiers.

[0127] Upon completing the setting and control of the optical network device 23, the transmission device 20A and the transmission device 20B each notify the orchestrator 70 of the completion of the setting and control (steps S55 and S56).

[0128] Orchestrator 70 transmits delay information including the transmission delay, transmission capacity, and optical path information of the set optical path to intermediate devices 10A and 10B (steps S57 and S58).

[0129] The transmission device 20A and the transmission device 20B calculate a buffer size based on the delay information transmitted from the orchestrator 70, and secure a buffer 14 of the calculated size (steps S59 and S60). For example, the transmission device 20A and the transmission device 20B calculate a value that is at least twice the product of the transmission delay and the transmission capacity as the buffer size.

[0130] After receiving ACK from all the transmission devices 20, the orchestrator 70 notifies the requesting local 30 of the ACK indicating completion of setting of the optical path (step S61). After receiving the ACK from the orchestrator 70, the local 30 starts communication with the remote 50 via the optical path established in the network 9.

[0131] The present embodiment described above includes an orchestrator 70 that calculates or measures delay information and notifies intermediate devices 10A and 10B. In the first embodiment, intermediate devices 10A and 10B actually transmit and receive packets to measure delay information, but in this embodiment, orchestrator 70 calculates or measures delay information when setting an optical path, and notifies intermediate devices 10A and 10B of the delay information when the optical path is established. As a result, in this embodiment, intermediate devices 10A and 10B can reduce the time required to start communication without measuring delay information.

[0132] Orchestrator 70 may read out delay information that has been calculated or measured in advance and notify intermediate devices 10A and 10B of the delay information. Orchestrator 70 may also notify intermediate devices 10A and 10B of the delay information periodically about the state of network 9 or when network 9 is changed.

[0133] <Modification 1 of the second embodiment> 14 is a diagram showing a configuration of Modification 1 of the second embodiment. In the second embodiment, the intermediate devices 10A and 10B obtain the delay information of the optical path from the orchestrator 70, but in Modification 1, the intermediate devices 10A and 10B obtain the delay information of the optical path from the transmission devices 20A and 20B. In this case, the orchestrator 70 may notify the transmission devices 20A and 20B of the calculated delay information in steps S53 and S54 of FIG.

[0134] <Modification 2 of the second embodiment> In the first modification, the transmission devices 20A and 20B notify the intermediate devices 10A and 10B of the delay information of the optical path acquired from the orchestrator 70. In the second modification, the transmission devices 20A and 20B measure the delay information of the optical path set by the orchestrator 70 and notify the intermediate devices 10A and 10B. The configuration of this modification is similar to the configuration of the first modification shown in FIG.

[0135] 15 is a sequence diagram showing the operation of the communication system of Modification 2. Upon receiving a request from the local 30 (or intermediate device 10A), the transmission device 20A cooperates with the transmission device 20B to determine the transmission mode of the optical path set between the local 30 and the remote 50, and measures delay information (step S71). The transmission devices 20A and 20B transmit the measured delay information to the intermediate devices 10A and 10B (steps S72 and S73).

[0136] Intermediate device 10A measures delay information up to transmission device 20A, and intermediate device 10B measures delay information up to transmission device 20B (steps S74, S75).

[0137] Intermediate device 10A calculates a buffer size based on the delay information of steps S71 and S74, and secures buffer 14 of that size (step S76). Similarly, intermediate device 10B calculates a buffer size based on the delay information of steps S71 and S75, and secures buffer 14 of that size (step S77).

[0138] Generally, the distance from intermediate devices 10A, 10B to transmission devices 20A, 20B is relatively short, and the distance between transmission devices 20A, 20B is relatively long. Therefore, most of the transmission delay is the transmission delay between transmission devices 20A, 20B. Therefore, intermediate devices 10A, 10B may omit S74 and S75, calculate a buffer size based on delay information between transmission devices 20A, 20B, and secure buffer 14 of that size.

[0139] <Modification 3 of the second embodiment> 16 is a diagram showing a configuration of Modification 3 of the second embodiment. In Modification 3, an intermediate device has the functions of a transmission device. Specifically, intermediate device 10C is a device in which intermediate device 10A of the second embodiment has the functions of transmission device 20A, and intermediate device 10D is a device in which intermediate device 10B of the second embodiment has the functions of transmission device 20B.

[0140] In this modification, after the orchestrator 70 sets up an optical path, the functions of the transmission devices 20A and 20B implemented in the intermediate devices 10C and 10D also measure delay information when selecting a wavelength in a transceiver (transponder). The intermediate devices 10C and 10D determine the buffer size based on the measured delay information.

[0141] 4. Third embodiment When the delay and transmission capacity are not large, or when the required communication requirements (SLA) are not strict, it may be possible to maintain performance without intermediate devices 10A and 10B by increasing the value of the queue depth (QueueDepth) of temporary data storage units 34 of local 30 and remote 50. For this reason, local 30 may request an optical path that meets the communication requirements from orchestrator 70 and adjust the queue depth of local 30 based on delay information of the optical path obtained from orchestrator 70. This makes it possible to avoid performance degradation due to an increase in transmission delay or transmission capacity by adjusting the queue depth without going through intermediate devices 10A and 10B.

[0142] However, the local 30 and the remote 50 cannot increase the queue size without limit due to restrictions of hardware, protocols, etc. Therefore, the local 30 in this embodiment uses the communication requirements of the application and delay information acquired from the orchestrator 70 to communicate via the intermediate devices 10A and 10B, or to determine whether to adjust the queue depth.

[0143] 17 is a diagram showing an example of a communication system according to the third embodiment. The system according to the present embodiment includes a local 30, a remote 50, intermediate devices 10A and 10B, an orchestrator 70, and network devices 40A and 40B. The intermediate devices 10A and 10B and the orchestrator 70 according to the present embodiment are similar to the intermediate devices 10A and 10B and the orchestrator 70 according to the second embodiment.

[0144] The local 30 and the remote 50 are similar to the local 30 and the remote 50 of the first embodiment. However, the determination unit 35 of the local 30 of the present embodiment determines whether or not to communicate via the intermediate device 10A and the intermediate device 10B based on the delay information. For example, the determination unit 35 determines not to communicate via the intermediate device 10A and the intermediate device 10B when a value equal to or greater than twice the product of the transmission delay and the transmission capacity of the delay information can be secured as the queue size of the temporary data storage unit 34. In this case, the queue management unit 32 of the local 30 determines the queue size and the queue depth of the temporary data storage unit 34 based on the delay information. The queue depth is determined based on the queue size and the packet size.

[0145] Similarly, determination unit 35 of remote 50 also determines whether or not to go through intermediate device 10A and intermediate device 10B based on the delay information, and if the data does not go through intermediate device 10A and intermediate device 10B, queue management unit 32 determines the queue size and queue depth of temporary data storage unit 34 based on the delay information.

[0146] Network device 40A routes the request sent by local device 30 in accordance with instructions from local device 30. Specifically, network device 40A uses a routing table or the like based on the destination information of the request to determine whether the request should be forwarded to intermediate device 10A or network device 40B.

[0147] Similarly, network device 40B, in accordance with instructions from remote 50, determines whether the destination of a response to a request should be intermediate device 10B or network device 40A.

[0148] 18 is a diagram showing a configuration of a communication system when the request does not go through intermediate devices 10A and 10B. Local 30 and remote 50 determine not to go through intermediate devices 10A and 10B based on delay information acquired from orchestrator 70, and adjust the queue depth based on the delay information. Then, local 30 transmits the request to remote 50 via network device 40A (not shown). Remote 50 transmits a response to local 30 via network device 40B (not shown).

[0149] 19, the communication system of the present embodiment may not include the network devices 40A and 40B, and the local 30 and the remote 50 may rewrite the destination of the request or response. Specifically, the determination unit 35 of the local 30 and the remote 50 may control the transfer destination of the request or response by rewriting the destination information in the header.

[0150] In this embodiment, the local 30 and the remote 50 determine whether to communicate via the intermediate devices 10A and 10B based on the delay information. In this way, the local 30 and the remote 50 autonomously select whether to communicate via the intermediate devices 10A and 10B or to adjust the queue depth of the local 30 and the remote 50 in accordance with the delay information, thereby enabling high-speed communication. Therefore, even if the optical path or communication requirements are dynamically changed, it is possible to automatically select the optimal communication system according to the delay information of the changed optical path and communication requirements.

[0151] For example, when the product of the transmission delay of the delay information and the transmission capacity is equal to or greater than a predetermined value, the local 30 and the remote 50 may automatically construct intermediate devices 10A and 10B in the high-functioning parts of the transmission devices 20A and 20B, perform communication via the constructed intermediate devices, and adjust the queue depth when the product of the transmission delay of the delay information and the transmission capacity is smaller than a predetermined value. The settings of the intermediate devices 10A and 10B constructed in the transmission devices 20A and 20B, or the transmission devices 20A and 20B constructing the intermediate devices 10A and 10B, may be specified on the terminal side of the local 30 and the remote 50, or may be determined and determined by the orchestrator 70 based on the conditions such as the distance from the terminal, the route of the optical path, and line congestion.

[0152] In addition, the local 30 and the remote 50 may obtain communication requests and changes to the optical path from the orchestrator, or may monitor them using the application unit 31, or may use another application to monitor delays using ping or the like to determine whether or not to go through the intermediate devices 10A and 10B.

[0153] 5. Hardware Configuration The above-described local 30, remote 50, intermediate devices 10A and 10B, transmission devices 20A and 20B, and orchestrator 70 may each use, for example, a general-purpose computer system as shown in FIG. 20. The illustrated computer system includes a CPU (Central Processing Unit, processor) 901, a memory 902, a storage 903 (HDD: Hard Disk Drive, SSD: Solid State Drive), a communication device 904, an input device 905, and an output device 906. The memory 902 and the storage 903 are storage devices. In this computer system, the CPU 901 executes a predetermined program loaded on the memory 902, thereby realizing the functions of the local 30, remote 50, intermediate devices 10A and 10B, transmission devices 20A and 20B, and orchestrator 70.

[0154] These devices may be implemented in one computer or in multiple computers. These devices may be virtual machines implemented in a computer. The programs of these devices may be stored in a computer-readable recording medium such as a HDD, SSD, a Universal Serial Bus (USB) memory, a Compact Disc (CD), or a Digital Versatile Disc (DVD), or may be distributed via a network.

[0155] The present invention is not limited to the above embodiment, and various modifications are possible within the scope of the present invention. For example, intermediate devices 10A, 10B and / or transmission devices 20A, 20B in the above embodiment may be implemented as a local 30 or remote 50 NIC or application.

[0156] Additionally, local 30 and remote 50 may measure the distance to intermediate devices 10A, 10B, and set the queue depth of the temporary data store according to that distance.

[0157] Furthermore, the present invention may be a combination of at least two of the above first to third embodiments. [Explanation of symbols]

[0158] 10A, 10B, 10C, 10D: Intermediate device 11: Transfer section 12A: Generation section 12B: Destruction section 13: Network status measurement unit 14: Buffer 15: Buffer management section 16: Credit Management Department 17: Communications Department 20A, 20B: Transmission equipment 30: Local 30 31: Application section 32: Queue Management Unit 33: Network status management unit 34: Temporary data storage section 35: Judgment section 36: Packet sorting unit 36 37: Communications Department 40A, 40B: Network device 50: Remote 70: Orchestrator

Claims

1. A communication system including a first intermediate device and a second intermediate device, the first intermediate device and the second intermediate device are disposed between a first device and a second device that transfer data using remote direct memory access; The first intermediate device a buffer management unit that determines a buffer size based on delay information related to a network delay between the second intermediate device and the second intermediate device, and secures a buffer of the determined buffer size; a forwarding unit that forwards a request including data to be forwarded from the first device to the second device to the second intermediate device when the data is smaller than a credit of the second intermediate device; a generator that generates a pseudo response to the request and returns the pseudo response to the first device; The second intermediate device: a buffer management unit that determines a buffer size based on the delay information and secures a buffer of the buffer size; a transfer unit that transfers the request to the second device, stores the request in the buffer, and updates its own credit; a discarding unit that discards a response to the request received from the second device, deletes the request stored in the buffer, and updates the credit. Communication systems.

2. The first intermediate device and the second intermediate device each include a network status measurement unit that measures the delay information.

2. The communication system of claim 1.

3. and an orchestrator that calculates or measures the delay information and notifies the first intermediate device and the second intermediate device.

2. The communication system of claim 1.

4. a buffer management unit of the first intermediate device acquiring the delay information from a first transmission device arranged in the network; A buffer management unit of the second intermediate device acquires the delay information from a second transmission device arranged in the network.

2. The communication system of claim 1.

5. The first device and the second device, Each of the first device and the second device comprises: a determination unit that determines whether or not to communicate via the first intermediate device and the second intermediate device based on the delay information; a queue management unit that determines a queue depth of the device itself based on the delay information when the first intermediate device and the second intermediate device are not passed through. A communication system according to any one of claims 1 to 4.

6. 1. An intermediate device disposed between a first device and a second device that transfer data using remote direct memory access, comprising: a buffer management unit that determines a buffer size based on delay information of a network between the intermediate device and a destination device and secures a buffer of the determined buffer size; a transfer unit that transfers a request including data to be transferred from the first device to the second device to the intermediate device of a transfer destination when the data is smaller than a credit of the intermediate device of the transfer destination; a generating unit that generates a pseudo response to the request and returns the pseudo response to the first device. Intermediate device.

7. A communication method performed by a communication system including a first intermediate device and a second intermediate device, comprising: the first intermediate device and the second intermediate device are disposed between a first device and a second device that transfer data using remote direct memory access; The first intermediate device determining a buffer size based on delay information regarding a network delay between the second intermediate device and the second intermediate device, and reserving a buffer of the buffer size; if the data to be transferred from the first device to the second device is less than the credit of the second intermediate device, transferring a request including the data to the second intermediate device; generating and returning a pseudo response to the request to the first device; The second intermediate device: determining a buffer size based on the delay information and reserving a buffer of the buffer size; forwarding the request to the second device, storing the request in the buffer and updating its own credits; discarding a response to the request received from the second device, deleting the request stored in the buffer, and updating the credit. Communication methods.

8. A program for causing a computer to function as the intermediate device according to claim 6.

9. An intermediate device disposed between a first device and a second device that transfer data using remote direct memory access, comprising: a buffer management unit that determines a buffer size based on delay information of a network between the intermediate device and a transfer source and secures a buffer of the determined buffer size; a transfer unit that transfers a request including data to be transferred from the first device to the second device, the request having been transferred from the source intermediate device, to the second device, stores the request in the buffer, and updates its own credit; a discarding unit that discards a response to the request received from the second device, deletes the request stored in the buffer, and updates the credit. Intermediate device.

10. A program that causes a computer to function as the intermediate device described in claim 9.

Images