A method for data multipath transmission and congestion control

Abstract

The application provides a data multi-path transmission and congestion control method applied to a leaf-spine structure intelligent center network. The method comprises the following steps: when a sending node generates a data packet, address information of a last-hop leaf switch is obtained from a corresponding multi-path set according to address information of a receiving node and a balancing algorithm, the address information of the last-hop leaf switch is used to modify a destination address of the data packet, and the address information of the receiving node is encapsulated in the data packet. When a leaf switch receives the data packet with the destination address pointing to the leaf switch itself, the destination address information is changed into the address information of the receiving node; when network congestion is detected, a congestion notification packet is sent to the sending node to perform congestion control. The strategy for performing the congestion control is that, first, the flow is transferred to a non-congestion path if there is a non-congestion path, and if all paths have occurred congestion events, a speed reduction operation is performed on the corresponding flow. The application has the advantage that good network bandwidth utilization can be provided.

Description

Technical Field

[0001] This application belongs to the field of network communication technology, specifically relating to a data multipath transmission and congestion control method, which is applied to intelligent computing center networks to improve network bandwidth utilization. Background Technology

[0002] Networks play a crucial role in AI infrastructure. In some cases, network communication time accounts for over 50% of the total training time. This means that for 50% of the time, the GPUs in the servers are idle and unable to work, waiting for the network to complete data transmission. State-of-the-art AI infrastructure, built with investments of millions or billions of yuan, is equipped with powerful GPUs. However, these expensive GPUs spend a significant portion of their time idle and unable to perform effectively. The reason for this is that the network fails to transmit data efficiently, forcing the GPUs to wait and wasting valuable computing resources. Although the network itself may only account for 10% of the total cost, the losses and impacts caused by poor network performance will far exceed that 10%.

[0003] Remote Direct Memory Access (RDMA) technology features high bandwidth, low latency, and no CPU consumption, meeting the demands of large-scale intelligent computing centers and other fields for low-latency and high-throughput data transmission. However, RDMA was originally designed based on single-path transmission, while intelligent computing centers are typically designed with a 1:1 link convergence ratio, meaning the number of flows in the network is roughly equal to the number of paths. Switches using ECMP for multi-path routing are prone to hash collisions and polarization, which means the native RDMA protocol cannot effectively utilize network bandwidth.

[0004] As mentioned above, how to utilize multipath RDMA transmission to improve network bandwidth utilization and thus enhance model training efficiency has become an urgent problem to be solved. Summary of the Invention

[0005] The purpose of this application is to overcome the shortcomings of existing technologies where switches perform multi-path routing via ECMP, which are prone to hash collisions and polarization.

[0006] To achieve the above objectives, this application proposes a data multipath transmission and congestion control method applied to a leaf-ridge structure intelligent computing center network, wherein the intelligent computing center network includes a sending node, a leaf switch, a ridge switch, and a receiving node; the method includes:

[0007] Step S1) The sending node responds to the receiving node's request to transmit data, generates at least one data packet based on the data to be transmitted, obtains the address information of the last hop leaf switch from the multipath set according to the receiving node's address information and the load balancing algorithm, modifies the destination address of the data packet to the address of the last hop leaf switch, encapsulates the receiving node's address information in the data packet, and sends the data packet.

[0008] Step S2) Leaf switches and / or spine switches perform routing and forwarding based on the destination address information in the data packets;

[0009] Step S3) If the leaf switch receives a data packet whose destination address points to itself, it changes the destination address information to the receiving node address information; when network congestion is detected, it sends a congestion notification message to the sending node to perform congestion control.

[0010] Step S4) The receiving node receives the data packet sent by the sending node and parses the data packet back into the data to be transmitted.

[0011] As an improvement to the above method, there is at least one data transmission path between the sending node and the receiving node.

[0012] As an improvement to the above method, the balancing algorithm is a weight-based scheduling algorithm, including a weighted random algorithm, a weighted round-robin algorithm, or a smoothed weighted round-robin algorithm, which selects a path packet by packet according to the weight of each path.

[0013] As an improvement to the above method, when the outgoing port queue of the spine switch is congested, an ECN tag is added to the data packets destined for the congested queue according to the calculated probability.

[0014] As an improvement to the above method, when the outgoing port queue of the leaf switch is congested, the path information in the data packets destined for the congested queue is extracted according to the calculated probability, a congestion notification message is generated, and the leaf switch node congestion identifier is added to the congestion notification message and sent back to the sending node for the sending node to adjust the path weight.

[0015] As an improvement to the above method, when the leaf switch receives a data packet with an ECN tag, it removes the ECN tag from the packet, extracts the path information from the packet, generates a congestion notification packet, adds information about the congestion of the spine switch node to the congestion notification packet, and sends it back to the sending node for the sending node to adjust the path weight.

[0016] As an improvement to the above method, the congestion control process of the sending node includes:

[0017] Step A1) After the sending node receives the congestion notification message, it parses the message path information and congestion location information. If the congestion location is the last hop leaf switch of the transmission path, it jumps to step A4); otherwise, it jumps to step A2.

[0018] Step A2) Obtain the multi-path set based on the path information in the congestion notification message, and determine whether there are any non-congested paths for which path weights can be transferred. If path weights can be transferred, proceed to step A3); otherwise, proceed to step A4.

[0019] Step A3) Randomly transfer some or all of the weights of the congested path to a non-congested path, and timestamp the congested path to mark it as congested for a set period of time. When the path is in a congested state, it will not receive weights transferred from other paths.

[0020] Step A4) Generate congestion control information based on the congestion notification message and reduce the sending rate of the corresponding data stream.

[0021] Compared with existing technologies, the advantages of this application are:

[0022] This invention records the multipath status at the sending node and determines the location of the congestion point based on the congestion notification message returned by the leaf switch. This allows for the modification of the multipath weights to attempt to utilize the bandwidth of other paths. If no other paths are available, a speed reduction operation is then performed. This method is simple to implement in hardware and can provide good network bandwidth utilization. Attached Figure Description

[0023] Figure 1 The diagram shown is an example environment block diagram that enables the implementation of the methods of this application;

[0024] Figure 2 The diagram shown is a schematic block diagram of a topological environment in which the method of this application can be implemented;

[0025] Figure 3 The diagram shows a flowchart of a data multipath transmission and congestion control method. Detailed Implementation

[0026] The technical solution of this application will be described in detail below with reference to the accompanying drawings.

[0027] This invention proposes a data multipath transmission and congestion control method, applied to a leaf-ridge structure intelligent computing center network. The intelligent computing center network includes multiple sending nodes, leaf switches, ridge switches, and receiving nodes. The method includes:

[0028] Step S1) The sending node responds to the receiving node's request to transmit the data to be transmitted, and generates at least one data packet based on the data to be transmitted; according to the receiving node's address information and the load balancing algorithm, it obtains the address information of the last hop leaf switch from the corresponding multipath set, modifies the destination address of the data packet to the address of the last hop leaf switch, encapsulates the receiving node's address information in the data packet, and sends the data packet.

[0029] There should be at least one data transmission path between the sending node and the receiving node.

[0030] The balancing algorithm can be a weighted scheduling algorithm, such as a weighted random algorithm, a weighted round-robin algorithm, or a smoothed weighted round-robin algorithm, which selects a path packet by packet according to the weight of each path.

[0031] Step S2) Leaf switches and / or spine switches perform routing and forwarding based on the destination address information in the data packets;

[0032] Step S3) If the leaf switch receives a data packet whose destination address points to itself, it changes the destination address information to the receiving node address information; when network congestion is detected, it sends a congestion notification message to the sending node to perform congestion control.

[0033] When the egress port queue of a spine switch is congested, an explicit congestion notification flag (ECN flag) is added to the data packets destined for that congested queue according to the calculated probability.

[0034] When the outgoing port queue of a leaf switch (including the first-hop leaf switch and the last-hop leaf switch) is congested, the path information of the data packets destined for the congested queue is extracted according to the calculated probability, a congestion notification message is generated, and the congestion identifier of the leaf switch node is added to the congestion notification message and sent back to the sending node for the sending node to adjust the path weight.

[0035] The probability calculation method can adopt the Random Early Detection (RED) labeling function commonly used in data center switches. The user can configure three parameters according to the actual network, including (ecn-min, ecn-max and probability).

[0036] When a leaf switch receives a data packet with an ECN tag, it removes the ECN tag from the packet, extracts the path information from the packet, generates a congestion notification packet, adds the congestion information of the spine switch node to the congestion notification packet, and sends it back to the sending node for the sending node to adjust the path weight.

[0037] Congestion control at the sending node is performed in steps, including:

[0038] Step A1) After the sending node receives the congestion notification message, it parses the message path information and congestion location information. If the congestion location is the last hop leaf switch of the transmission path, it jumps to step A4); otherwise, it jumps to step A2.

[0039] Step A2) Obtain the multi-path set based on the path information in the congestion notification message, and determine whether there are any non-congested paths for which path weights can be transferred. If path weights can be transferred, proceed to step A3); otherwise, proceed to step A4.

[0040] Step A3) Randomly transfer some or all of the weights of the congested path to a non-congested path, and timestamp the congested path to mark it as congested for a certain period of time. When the path is in a congested state, it will not receive weights transferred from other paths.

[0041] Step A4) Generate congestion control information based on the congestion notification message and transmit it to the data packet generation module to reduce the sending rate of the corresponding data stream.

[0042] Step S4) The receiving node receives the data packet sent by the sending node and parses the data packet back into the data to be transmitted.

[0043] Compared with the prior art, the advantages of the present invention are as follows: The present invention records the multipath status at the sending node and determines the location of the congestion point based on the congestion notification message returned by the leaf switch. Thus, the multipath weight can be modified first to try to utilize the bandwidth of other paths. If no other path is available, the speed reduction operation is then performed. This method is simple to implement in hardware and can provide good network bandwidth utilization.

[0044] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0045] Example 1

[0046] Figure 1 An example environment in which an implementation of this disclosure can be carried out is shown. For example... Figure 1 As shown, instance environment 100 may include host 110 and host 120. Host 110 can be any physical computer, virtual machine, server, etc. Host 110 may include CPU 111, memory 112, RDMA Network Interface Controller (RNIC, also known as RDMA network card) 113, user application ( Figure 1(Not shown) can run on CPU 111. Host 110 can connect to network 130 via RNIC 113. Similarly, host 120 may include CPU 121, memory 122, RDMA Network Interface Controller (RNIC, also known as RDMA network card) 123, user application ( Figure 1 (Not shown) can run on CPU 121. Host 120 can be connected to network 130 via RNIC 123.

[0047] Network 130 serves as the medium for providing a communication link between host 110 and host 120. Network 130 may be Ethernet, such as an Artificial Intelligence Data Center Network (AIDC). In some embodiments, RNIC 113 and RNIC 123 may have multiple network links, each of which may or may not include additional network devices such as switches or routers. In some embodiments, RNIC 113 and RNIC 123 may establish RDMA links via multiple network links to enable RDMA transmission between host 110 and host 120.

[0048] It should be understood that Figure 1 The number of hosts, memory, and RNICs shown is merely illustrative. Depending on implementation needs, the system architecture can have any number of hosts, and each host can have any number of memory and RNICs, etc.

[0049] As mentioned above, single-path RDMA is prone to low network utilization, which reduces system reliability and leads to decreased application performance in modern AIDCs. Therefore, this disclosure provides a data multipath transmission and congestion control method that improves network bandwidth utilization by utilizing multipath RDMA transmission, thereby enhancing AI model training efficiency.

[0050] In modern AIDC, network topology is mostly based on Figure 2 The leaf-spine structure shown is designed such that there are typically several equivalent paths for communication between hosts. The multipath method implemented in this disclosure modifies the destination address in the data packet to the address of the uplink port of the leaf switch, while recording the original destination address in the data packet. When the leaf switch receives a data packet whose destination address points to itself, it parses the data packet, modifies the destination address back to the original address, and then forwards the data packet. The advantage of this approach is that it divides a flow into several sub-flows, increasing the overall number of flows from the switch's perspective, significantly improving hash balancing, and greatly enhancing link utilization.

[0051] To minimize the dependence of this disclosed method on network devices, the spine switch is a data center switch performing general functions and with ECN enabled. Figure 2 In the topology shown, the leaf switch can be understood as a ToR (Top of Rack) switch in the data center. The multipath transmission method and congestion control disclosed in this paper need to be deployed in the leaf switch. Its main functions include: 1. When a data packet is received with the destination IP pointing to the leaf switch itself, the data packet is parsed and replaced with the original destination address to which the data packet was originally sent; 2. When the leaf switch detects congestion, it promptly notifies the host device.

[0052] In this disclosure, the leaf switch can detect congestion in two locations: first, a congestion event occurs in the leaf switch's own outgoing port queue; second, it receives a data packet with an ECN tag, indicating congestion has occurred on the upstream spine switch. When the leaf switch detects a congestion event, it sends a congestion notification message to the sending end, informing either the spine switch or the leaf switch that a congestion event has occurred. The end-side can then adjust the path and rate accordingly based on the congestion control algorithm. Because the leaf switch notifies the sending end of the congestion event in advance, to prevent overreaction from the sending end, the leaf switch will remove the ECN tag from the received ECN-tagged data packet. The leaf switch's advance notification of congestion events in the end-side network helps the end-side perform congestion control more accurately and promptly.

[0053] like Figure 3 The flowchart shown is for congestion control at the endpoint.

[0054] Step A1): When the sending node receives the congestion notification message, it will parse the path information and congestion location information carried in the message. If the congestion location is the last hop leaf switch of the transmission path, it will jump to step A4); otherwise, it will jump to step A2.

[0055] Step A2) Obtain the multi-path set based on the path information in the congestion notification message, and determine whether there are any non-congested paths for which path weights can be transferred. If path weights can be transferred, proceed to step A3); otherwise, proceed to step A4.

[0056] Step A3) Randomly transfer some or all of the weights of the congested path to a non-congested path, and timestamp the congested path to mark it as congested for a certain period of time. When the path is in a congested state, it will not receive weights transferred from other paths.

[0057] Step A4) Generate congestion control information based on the congestion notification message and transmit it to the data packet generation module to reduce the sending rate of the corresponding data stream.

[0058] The principle of congestion control in this disclosed method is as follows: when congestion occurs, traffic is first diverted to non-congested paths if available; if congestion occurs on all paths, the endpoint then performs traffic reduction. Decoupling the multipath algorithm from the congestion control algorithm allows for more efficient use of network bandwidth and significantly reduces the difficulty of algorithm implementation.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of this application do not depart from the spirit and scope of the technical solutions of this application, and should all be covered within the scope of the claims of this application.

Claims

1. A data multipath transmission and congestion control method, applied to a leaf-ridge structured intelligent computing center network, the intelligent computing center network comprising a sending node, a leaf switch, a ridge switch, and a receiving node; the method comprising: Step S1) The sending node responds to the receiving node's request to transmit data and generates at least one data packet based on the data to be transmitted; Based on the receiving node address information and the load balancing algorithm, the address information of the last hop leaf switch is obtained from the multipath set, the destination address of the data packet is modified to the address of the last hop leaf switch, the receiving node address information is encapsulated in the data packet, and the data packet is sent. Step S2) Leaf switches and / or spine switches perform routing and forwarding based on the destination address information in the data packets; Step S3) If the leaf switch receives a data packet whose destination address points to itself, it changes the destination address information to the receiving node address information; when network congestion is detected, it sends a congestion notification message to the sending node to perform congestion control. Step S4) The receiving node receives the data packet sent by the sending node and parses the data packet back into the data to be transmitted.

2. The method according to claim 1, characterized in that, There is at least one data transmission path between the sending node and the receiving node.

3. The method according to claim 1, characterized in that, The balancing algorithm is a weight-based scheduling algorithm, including a weighted random algorithm, a weighted round-robin algorithm, or a smoothed weighted round-robin algorithm, which selects a path packet by packet according to the weight of each path.

4. The method according to claim 1, characterized in that, When the outgoing port queue of the spine switch is congested, an ECN tag is added to the data packets destined for the congested queue according to the calculated probability.

5. The method according to claim 1, characterized in that, When the outgoing port queue of the leaf switch is congested, the path information of the data packets destined for the congested queue is extracted according to the calculated probability, a congestion notification message is generated, and the leaf switch node congestion identifier is added to the congestion notification message and sent back to the sending node for the sending node to adjust the path weight.

6. The method according to claim 1, characterized in that, When the leaf switch receives a data packet with an ECN tag, it removes the ECN tag from the packet, extracts the path information from the packet, generates a congestion notification packet, adds information about the congestion of the spine switch node to the congestion notification packet, and sends it back to the sending node for the sending node to adjust the path weight.

7. The method according to claim 1, characterized in that, The congestion control process of the sending node includes: Step A1) After the sending node receives the congestion notification message, it parses the message path information and congestion location information. If the congestion location is the last hop leaf switch of the transmission path, it jumps to step A4); otherwise, it jumps to step A2. Step A2) Obtain the multi-path set based on the path information in the congestion notification message, and determine whether there are any non-congested paths for which path weights can be transferred. If path weights can be transferred, proceed to step A3); otherwise, proceed to step A4. Step A3) Randomly transfer some or all of the weights of the congested path to a non-congested path, and timestamp the congested path to mark it as congested for a set period of time. When the path is in a congested state, it will not receive weights transferred from other paths. Step A4) Generate congestion control information based on the congestion notification message and reduce the sending rate of the corresponding data stream.

Images