Communication control device and communication control method

Abstract

A flow information table (106) stores information about a flow flowing to a first node in a network. If congestion is predicted at a second node through which the flow flows from the first node, a route control unit (105) distributes the flow flowing from the first node to the second node and one or more third nodes on the basis of the information in the flow information table (106).

Description

Communication control device and communication control method

[0001] The present invention relates to a communication control device and a communication control method.

[0002] Conventionally, congestion control techniques have been known for networks in which RDMA (Remote Direct Memory Access) communications are performed.

[0003] For example, PFC (Priority Flow Control) is known, which outputs a pause frame as an Ethernet frame to stop transmission from the opposite device when packets exceed a set threshold in the receive queue of a specific node (see, for example, non-patent document 1).

[0004] Also, for example, there is known an explicit congestion notification (ECN) technique in which the ToS value in the IP header flowing when congestion occurs is rewritten at the forwarding node and sent to the destination server, and the sending server, upon receiving notification of congestion from the destination server, suppresses traffic (see, for example, non-patent document 2).

[0005] IEEE802.1Qbb Priority-based Flow Control, [online], [Retrieved August 19, 2024], Internet (https: / / 1.ieee802.org / dcb / 802-1qbb / #Presentations) RFC3168 The Addition of Explicit Congestion Notification (ECN), [online], [Retrieved August 19, 2024], Internet (https: / / www.rfc-editor.org / rfc / pdfrfc / rfc3168.txt.pdf)

[0006] However, conventional techniques have the problem that they do not allow efficient use of network bandwidth.

[0007] For example, the PFC described in Non-Patent Document 1 has a problem in that if a PFC Storm occurs in which pause frames do not stop, or if PFC pause frames occur on all opposing nodes, a deadlock state occurs and packets cannot be transmitted.

[0008] One way to solve this problem is to use a PFC watchdog, which drops all receive queues with PFC enabled. However, with a PFC watchdog alone, the amount of traffic sent from the server remains unchanged, so even after the receive queues are emptied, they may build up again, causing a PFC.

[0009] The ECN described in Non-Patent Document 2 notifies the occurrence of congestion and suppresses the amount of traffic from the transmission server, which causes a problem that traffic does not flow even when the bandwidth is available across the entire network.

[0010] In order to solve the above-mentioned problems and achieve the objectives, a communication control device is characterized by having a flow information storage unit that stores information on flows flowing to a first node in a network, and a route control unit that, when congestion is predicted at a second node to which a flow flows from the first node, distributes the flow flowing from the first node to the second node and one or more third nodes based on information from the flow information storage unit.

[0011] According to the present invention, network bandwidth can be used efficiently.

[0012] FIG. 1 is a diagram illustrating an example of a network configuration. FIG. 2 is a diagram illustrating an example of a forwarding node configuration. FIG. 3 is a diagram illustrating an example of a flow information table. FIG. 4 is a flowchart illustrating the flow of processing by a forwarding node. FIG. 5 is a diagram illustrating processing according to a first embodiment. FIG. 6 is a diagram illustrating processing according to a second embodiment. FIG. 7 is a diagram illustrating an example of a flow information table according to the second embodiment. FIG. 8 is a diagram illustrating processing according to the second embodiment. FIG. 9 is a diagram illustrating an example of a computer that executes a communication control program.

[0013] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A communication control device and a communication control method according to embodiments of the present invention will be described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the embodiments described below.

[0014] The communication control device of this embodiment performs congestion control in a network. The network may be one in which RDMA communication is performed, or one in which other types of communication are performed.

[0015] Fig. 1 is a diagram showing an example of a network configuration. As shown in Fig. 1, network 1 is a network used in a data center and configured with a leaf and a spine. Network 1 has nodes Spine 11, Spine 12, Leaf 13, Leaf 14, Leaf 15, Leaf 16, Server 17, and Server 18. Each node may be referred to as a forwarding node without distinction. Furthermore, a forwarding node is an example of a communication control device.

[0016] For example, a packet transmitted from a transmitting server passes through a path including a leaf and a spine and is received by a receiving server 18 .

[0017] 2 is a diagram showing an example of the configuration of a forwarding node 10. As shown in FIG. 2, the forwarding node 10 includes a receiving unit 101, a transmitting unit 102, a receiving queue 103, a transmitting queue 104, a route control unit 105, and a flow information table 106.

[0018] The receiving unit 101 receives packets and stores the received packets in a receiving queue 103. The transmitting unit 102 transmits the packets in the receiving queue 103 via a transmitting queue 104 under the control of a route control unit 105. The route control unit 105 refers to a flow information table 106 to control the route for transmitting and receiving packets.

[0019] 3 is a diagram showing an example of a flow information table. Here, it is assumed that the flow information table 106 in FIG. 3 is the flow information table of the Spine 12. Three flows, flow A, flow B, and flow C, exist on the path between the Spine 12 and the Leaf 13. Flow A, flow B, and flow C are all packet flows heading from the Leaf 13 to the Spine 12.

[0020] As shown in FIG. 3, the flow information table 106 has the columns "Flow," "IF," "Mbps," "srcIP," "dstIP," "srcPort," "dstPort," and "Protocol." "Flow" is information that identifies a flow. "IF" is the IF corresponding to the flow (for example, the transmission queue of Leaf 13). "Mbps" is the communication volume of the flow. "srcIP," "dstIP," "srcPort," "dstPort," and "Protocol" are the source IP address, destination IP address, source port number, destination port number, and protocol of the flow, respectively.

[0021] The flow of processing in the forwarding node 10 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of processing in the forwarding node.

[0022] 4, the route control unit 105 determines whether the queue is equal to or larger than a threshold (step S101). That is, the route control unit 105 determines whether the amount of packets accumulated in the receiving queue 103 exceeds a threshold.

[0023] If the queue is not equal to or larger than the threshold (step S101; No), the route control unit 105 ends the process. That is, the forwarding node 10 continues forwarding packets without performing congestion control.

[0024] If the queue is equal to or larger than the threshold (step S101; Yes), the route control unit 105 transmits a congestion notification to the transmitting node (step S102). For example, the route control unit 105 of the leaf 16 transmits the congestion notification to the spine 12, which is the source of the packet. The queue being equal to or larger than the threshold means that congestion is predicted to occur.

[0025] The route control unit 105 of the sending node selects a route from the flow information table 106 (step S103), and then performs load balancing using PBR (Policy-Based Routing) (step S104).

[0026] The forwarding node 10 performs congestion control using PBR. Here, several embodiments using different PBR techniques will be described.

[0027] First Embodiment The route control unit 105 of the leaf 16 transmits a congestion notification to the spine 12, which is the source of the packet. The leaf 16 is a congestion detection node. The spine 12 is a transmission node. For example, the congestion detection node is not a forwarding node where congestion actually occurs, but a forwarding node where congestion is predicted to occur.

[0028] The route control unit 105 allocates the forwarding nodes to which traffic is forwarded using PBR of BGP (Border Gateway Protocol). The route control unit 105 refers to the flow information table 106 and allocates traffic so that the flow volume is evenly distributed to the forwarding nodes to which it is allocated. In addition, to perform PBR, each forwarding node has a different AS, and routes are exchanged using eBGP.

[0029] The operation flow is as follows: When the capacity in the receiving queue 103 of the congestion detection node exceeds a threshold, the congestion detection node notifies the sending node that a possibility of congestion has occurred.

[0030] The route control unit 105 of the sending node refers to the flow information table 106, performs calculations to distribute the packets evenly among the forwarding nodes to which the packets are to be distributed, and selects a route.

[0031] 5 , the 60 Gbps flow A, the 30 Gbps flow B, and the 15 Gbps flow C transmitted from the Spine 12 are all allocated to the Leaf 16. In response to this, the route control unit 105 of the Spine 12 allocates the flows B and C to the Leaf 15 while leaving the flow A allocated to the Leaf 16 so that the flows are distributed equally between the Leaf 15 and the Leaf 16.

[0032] Second Embodiment Fig. 6 is a diagram for explaining the processing of the second embodiment, and Fig. 7 is a diagram showing an example of a flow information table of the second embodiment.

[0033] Assume that there is a 10 Gbps physical link between forwarding nodes between eth in Fig. 6. eth is an interface. As shown in Fig. 7, 5 Gbps flow A, 3 Gbps flow B, 2 Gbps flow C, and 3 Gbps flow D transmitted from Spine 12 are all distributed to Leaf 16. Fig. 7 also shows that flows A, B, C, and D are transmitted from Leaf 13 and Leaf 14 to Spine 12.

[0034] The route control unit 105 of the sending node identifies the available bandwidth by referring to the flow information table 106. The route control unit 105 may also refer to the flow information tables 106 of other forwarding nodes.

[0035] Since the 3-Gbps flow D is allocated to the path from eth12_3 to eth15_2, the route control unit 105 of Spine 12 sets the available bandwidth in the direction of eth15_2 to 7 Gbps, which is obtained by subtracting 3 Gbps from the 10 Gbps of the physical link. On the other hand, for the path to eth16_2 on the congestion detection node side, the route control unit 105 temporarily ignores the allocated flow and sets the available bandwidth to 10 Gbps.

[0036] The route control unit 105 compares routes in descending order of available bandwidth, and distributes flows in descending order of communication capacity.

[0037] First, the available bandwidth of the path between eth12_4 and eth16_2 is 10 Gbps, which is greater than the available bandwidth of the path between eth12_3 and eth15_2, which is 7 Gbps. Therefore, as shown in Fig. 8, the route control unit 105 allocates flow A (5 Gbps), which has the largest communication capacity, to the path between eth12_4 and eth16_2.

[0038] Next, the available bandwidth on the path between eth12_4 and eth16_2 changes to 5 Gbps, while the available bandwidth on the path between eth12_3 and eth15_2, 7 Gbps, becomes larger. Therefore, as shown in Fig. 8, the route control unit 105 allocates flow B (3 Gbps), which has the largest communication capacity among the unallocated flows, to the path between eth12_3 and eth15_2.

[0039] Furthermore, the available bandwidth on the path between eth12_4 and eth16_2 is 5 Gbps, which is greater than the available bandwidth of 4 Gbps on the path between eth12_3 and eth15_2. Therefore, as shown in Fig. 8, the route control unit 105 allocates flow C (2 Gbps), which has the largest communication capacity among the unallocated flows, to the path between eth12_4 and eth16_2.

[0040] As a result, the bandwidth used on the path between eth12_4 and eth16_2 is limited to 8 Gbps out of 10 Gbps, thereby avoiding congestion.

[0041] In this way, the routing control unit 105 preferentially allocates the flow to either the second node or the third node, whichever has the larger available bandwidth of the physical link, thereby making it possible to use the available bandwidth efficiently.

[0042] [Effects of the First Embodiment] As described above, the flow information table 106 stores information about a flow flowing to a first node in the network 1. Furthermore, when congestion is predicted at a second node to which a flow flows from the first node, the route control unit 105 distributes the flow flowing from the first node to the second node and one or more third nodes based on the information in the flow information table 106.

[0043] For example, the Spine 12 is the first node. Also, for example, the Leaf 16 is the second node. Also, for example, the Leaf 15 is the third node. Note that there may be multiple third nodes.

[0044] In this embodiment, a congestion detection node (e.g., Leaf 16) where congestion has occurred notifies a transmitting node (e.g., Spine 12) that is transmitting packets. The transmitting node distributes packets in a load-balanced manner. As a result, according to this embodiment, the bandwidth of the entire network can be used efficiently.

[0045] The threshold value used by the congestion detection node to predict congestion is smaller than the PFC threshold value used to detect actual congestion. Therefore, according to the embodiment, congestion can be avoided before a drop occurs due to the PFC watchdog.

[0046] [System Configuration, etc.] The components of each device shown in the figures are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of the devices can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0047] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0048] [Program] In one embodiment, the forwarding node 10 can be implemented by installing a communication control program that executes the above-described processing as package software or online software on a desired computer. For example, by causing an information processing device to execute the above-described communication control program, the information processing device can function as the forwarding node 10. The information processing device referred to here includes desktop and notebook personal computers. In addition, the information processing device also includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handyphone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants).

[0049] 9 is a diagram showing an example of a computer that executes a communication control program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0050] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0051] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, a program that defines each process of the transferor node 10 is implemented as a program module 1093 in which computer-executable code is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, the program module 1093 for executing the same process as the functional configuration of the transferor node 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD.

[0052] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. The CPU 1020 then reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.

[0053] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0054] 10 forwarding node 101 receiving unit 102 transmitting unit 103 receiving queue 104 transmitting queue 105 route control unit 106 flow information table

Claims

1. A communication control device comprising: a flow information storage unit that stores information on flows flowing to a first node in a network; and a route control unit that, when congestion is predicted at a second node to which a flow flows from the first node, distributes the flow flowing from the first node to the second node and one or more third nodes based on the information in the flow information storage unit.

2. The communication control device according to claim 1, characterized in that the route control unit distributes the flow so that the amount of flow flowing from the first node is equal between the second node and the third node.

3. The communication control device according to claim 1, characterized in that the route control unit preferentially allocates flows to either the second node or the third node, whichever has a larger available bandwidth on the physical link.

4. A communication control method executed by a computer, comprising: a detection step of predicting congestion at a second node to which a flow flows from a first node in a network; and a route control step of, when congestion at the second node is predicted, allocating the flow from the first node to the second node and one or more third nodes based on information about the flow flowing to the first node.

Images