A Centralized RoCEv2 Network Congestion Control Method

Abstract

The present invention provides a centralized RoCEv2 network congestion control method, which relates to the technical field of network congestion control, including: S1, when a data center network is started, a network controller calculates and saves a switch forwarding table and an alternative forwarding table according to the network topology; S2, the network controller determines the network operation state according to link information; S3, if the network operation state is a network congestion state, a flow rate adjustment method is used to perform flow control, and if the congestion situation is improved after flow control, the method goes to step S2; S4, the network controller notifies the congested switch according to the comprehensive link cost to set the alternative forwarding table as the positive forwarding table for data forwarding, and goes to step S2. The present invention generates a global flow control strategy through the network controller, ensures the consistency and effectiveness of the strategy, and automatically switches the forwarding path when the global flow control strategy fails, thereby improving the efficiency and accuracy of congestion control.

Description

Technical Field

[0001] The present invention relates to the technical field of network congestion control, and in particular to a centralized RoCEv2 network congestion control method. Background Art

[0002] With the widespread deployment of applications such as artificial intelligence, cloud computing, and cloud storage, the network bandwidth of data centers has grown to 100G, and the network latency has dropped from milliseconds to microseconds. The traditional TCP / IP network protocol stack cannot meet the network bandwidth and latency requirements. RDMA (Remote Direct Memory Access) technology came into being. It allows user space applications to directly read or write remote memory without kernel interference or memory copying. It is widely used due to its high bandwidth and low latency. RDMA first appeared in the IB (Infiniband) network and was later ported to the traditional Ethernet, which promoted the popularization of RDMA technology. RDMA on Ethernet is divided into two technologies: RoCE (RDMA over Converged Ethernet) and iWARP (RDMA over TCP / IP). RoCE has higher throughput and lower latency than iWARP. RoCE uses the lossless Ethernet protocol as the data link layer, the RoCEv1 network layer uses the IB protocol, and the RoCEv2 network layer uses the IP protocol.

[0003] In order to ensure high-performance data transmission of RDMA, RoCEv2 needs to deploy lossless Ethernet using the priority-based flow control protocol PFC (Priority-based Flow Control) in the data center. The PFC protocol allows congested ports to directly suspend data transmission from upstream source ports, thereby avoiding buffer overflow of congested ports. However, this port-level suspension can lead to performance issues such as congestion diffusion. Therefore, the RoCEv2 network requires a network layer congestion control protocol to alleviate the many problems caused by PFC. The main function of the congestion control algorithm is to detect the data transmission status in the network and adjust the transmission rate accordingly. It is the key to RDMA achieving ultra-low latency, high bandwidth and network stability at high speeds.

[0004] The RoCEv2 congestion control protocol needs to meet the requirements of high bandwidth utilization, fairness, stability, fast convergence, etc. The goals of congestion control are: 1) reduce the flow completion time FCT to ensure low-latency transmission of short flows and high-throughput transmission of long flows; 2) reduce the triggering of the PFC protocol; 3) prevent deadlock.

[0005] According to the role they play in RoCEv2, RoCEv2 congestion control protocols can be divided into four categories: sender-driven, receiver-driven, switch-driven, and protocol improvement. The methods driven by the source include sensing through signals such as ECN and RTT or in-band network telemetry (INT). Such methods include DCQCN (Data Center Quantified Congestion Notification) and its improved algorithm, HPCC (High Precision Congestion Control), Timely, and P4QCN (P4 Quantified Congestion Notification). The receiver driver senses network delay by receiving messages. Such methods include PCN (Photonic Congestion Notification) and RCC (Receiver-driven RDMA congestion control). The switch driver detects congestion at the switch egress port, requiring the switch to have programmable functions. Such methods include RoCC (Robust congestion control) and HierCC (Hierarchical Congestion Control). Protocol improvement requires modifying and improving the RoCEv2 protocol. Such methods include IRN (Improved RoCE NIC) and RoGUE (Rdma over Generic Unconverged Ethernet).

[0006] The above method mainly solves the congestion control problem through heuristic flow rate adjustment, and adopts an "after-the-fact" congestion management mechanism. Before the message is sent, there will be no resource negotiation with the receiver, but the message will be forwarded directly. Only when the port cache of the receiving switch is congested (or is about to be congested), the PFC and ECN protocols will be used to send congestion management messages to let the opposite switch and the opposite network card reduce or suspend the sending of messages. Although this "after-the-fact" processing method can alleviate the impact of congestion to a certain extent, it cannot completely avoid the loss of data packets and network instability.

[0007] In summary, the traditional distributed flow control mechanism faces the following problems: 1. Each node independently makes congestion judgments and traffic adjustments, instead of comprehensively considering the global traffic distribution and the congestion situation of the entire network; 2. It adopts an "after-the-fact" management mechanism, which cannot avoid the loss of data packets and network instability; 3. The congestion identification of a single-flow network cannot well reflect the actual congestion of the network, and the heuristic rate adjustment strategy causes the protocol to perform poorly in terms of convergence, stability and fairness; 4. When a network failure occurs in the network, the network is completely unreachable, and all the above methods fail; 5. The congestion control protocol is based on minor repairs to the protocol and cannot fundamentally solve the problem of network congestion.

[0008] SDN (Software Defined Network) technology makes it possible to centrally control network congestion. It separates the control layer and the data layer. The controller in the control layer collects information from the entire network, deploys network policies, and issues forwarding tables. The switches in the data layer execute the policies issued by the controller and forward data. The control layer and the data layer communicate through the same open interface. Segment routing (SR) has powerful network programmability and can perform explicit path control well, providing a powerful means for centralized control of network congestion.

[0009] However, how to effectively utilize the control concept of SDN and combine it with RoCEv2 is still a problem that needs further study. Based on this background, the present invention proposes a RoCEv2 congestion control method based on centralized control, aiming to give full play to the advantages of the control concept of SDN and overcome the limitations of the existing technology.

[0010] The background description provided herein is for the purpose of generally presenting the context of the present application. Unless otherwise indicated herein, the materials described in this section are not prior art to the claims of this application and are not admitted to be prior art by inclusion in this section. Summary of the invention

[0011] In order to overcome the deficiencies in the background technology, the present invention provides a centrally controlled RoCEv2 network congestion control method.

[0012] In order to achieve the above-mentioned invention object, the present invention adopts the following technical scheme:

[0013] A centrally controlled RoCEv2 network congestion control method is applied to a RoCEv2 data center network, wherein the RoCEv2 data center network includes a network controller and a switch, wherein the network controller is used to monitor messages sent by the switch in real time and send a control command to the switch to request the switch to report link information; the method includes the following steps:

[0014] S1. When the data center network is started, the network controller obtains the network topology through the network topology detection technology, calculates the switch forwarding table and the alternative forwarding table according to the network topology, and saves them;

[0015] S2. The network controller determines the network operation status according to the link information; the network operation status includes network congestion status, congestion warning status and normal status;

[0016] S3. If the network operation state is a network congestion state, first use the flow rate adjustment method to regulate the flow through the congested path. If the congestion situation improves after the flow regulation, go to step S2; otherwise, go to step S4.

[0017] S4. The network controller notifies the congested switch according to the comprehensive link cost to set the alternative forwarding table as the primary forwarding table for data forwarding, and sets the original primary forwarding table as the alternative forwarding table, and then goes to step S2.

[0018] Specifically, the link information includes packet loss rate, forwarding delay, queue length, and link bandwidth capacity; the link bandwidth capacity includes maximum link bandwidth capacity and minimum link bandwidth capacity; the forwarding delay includes minimum forwarding delay and maximum forwarding delay.

[0019] Specifically, step S1 calculates the switch forwarding table and the candidate forwarding table according to the network topology and specifically includes the following steps:

[0020] S11, assign a local identifier LID to each switch SW(i), and initialize the selected forwarding table SW_MAT and the alternative forwarding table SW_MAT_Backup to record the connection relationship between the port of each switch SW(i) and the LID of the directly connected remote switch remote_SW;

[0021] S12, update the switch SW(i)'s forwarding table SW_MAT to record the shortest path hop count from each port Port(j) of SW(i) to other switches SW(m).LID in the subnet; if a shorter path can be found through the connected remote switch remote_SW, update the switch's forwarding table SW_MAT accordingly, and retain a backup forwarding table SW_MAT_Backup;

[0022] S13. Repeat step S12 until the LID matrix of each switch no longer changes, and the forwarding table and the alternative forwarding table of the switch SW(i) are established.

[0023] Specifically, step S1 also includes: setting a control path between all network elements for transmission of control messages and report information.

[0024] Specifically, step S2 is as follows: if the maximum forwarding delay exceeds the preset delay threshold, or the queue length exceeds the preset length threshold, or the minimum link bandwidth capacity is less than the preset bandwidth capacity threshold, the network operation state is determined to be a network congestion state; otherwise, the current network operation state is determined to be a normal state.

[0025] Specifically, step S2 also includes: when congestion occurs in the switch, the switch actively reports the network congestion status to the network controller; when the queue length in the network element exceeds the preset length threshold or the traffic RTT exceeds the preset RTT threshold or an ECN mark is generated, the network element generates a control message report and sends it to the network controller;

[0026] Specifically, the flow rate adjustment method in step S3 includes DCQCN, HPCC, Timely, and IRN.

[0027] Specifically, step S3 also includes, if a congestion warning state occurs, using a flow rate adjustment method to regulate the flow passing through the congested path; the congestion warning state is that the network operation state is normal but the network congestion probability is greater than or equal to a preset congestion probability threshold.

[0028] Specifically, the network controller in step S4 notifies the congested switch according to the comprehensive link cost as follows: the network controller can calculate the comprehensive link cost of each congested link based on the link information, sort them according to the comprehensive link cost, select the one with the largest comprehensive link cost, or select several congested paths from large to small to notify the switches on the path to use the backup routing table for data forwarding.

[0029] Specifically, the network controller includes one or more; if there are multiple network controllers, one of them is a master network controller and the others are slave network controllers, and data synchronization can be performed between the master network controller and the slave network controller; the master network controller and the slave network controller are interchangeable.

[0030] The present invention provides a centrally controlled RoCEv2 network congestion control method, comprising S1, a network controller obtains a network topology by using a network topology detection technology when a data center network is started, and calculates and saves a positive forwarding table and an alternative forwarding table of a switch according to the network topology; S2, a network controller determines a network operation state according to link information; the network operation state includes a network congestion state and a normal state; S3, if the network operation state is a network congestion state, a flow rate adjustment method is used to perform flow control, if the congestion situation is improved after the flow control, the method goes to step S2, otherwise, the method goes to step S4; S4, the network controller notifies a congested switch according to a comprehensive link cost to set the alternative forwarding table as a positive forwarding table for data forwarding, and sets the original positive forwarding table as an alternative forwarding table, and the method goes to step S2.

[0031] The beneficial effects of the present invention are:

[0032] 1. Centralized control: The network controller collects network status information of each node and generates a global traffic control strategy, thus achieving unified management and optimization of network traffic.

[0033] 2. Policy consistency: Each node adjusts traffic according to the global traffic control policy issued by the network controller, ensuring the consistency and effectiveness of the control policy.

[0034] 3. High efficiency: When a network failure occurs, the traditional flow control algorithm basically fails. The present invention can adjust the network flow and solve the congestion problem by adjusting the forwarding path. The network controller can quickly respond to changes in network status and generate adaptive flow control strategies, thereby improving the efficiency and accuracy of congestion control.

[0035] In addition, the present invention also sets up a control path with the highest priority between all network elements, and control messages are forwarded preferentially under any circumstances; even when network congestion occurs, the network controller can send control messages to the switch through the control path or receive report information that the switch wants to pass to the network controller, thereby ensuring the timely transmission of messages and further improving the efficiency of congestion control.

[0036] In addition, the network congestion control framework provided by the present invention abstracts the RoCEv2 network into four major components, which can clarify the role and responsibility of each component in network congestion control, reduce the complexity of network control, and facilitate management and maintenance. The network controller CTLP centrally manages the network topology and forwarding table, which can globally optimize the use of network resources and avoid suboptimal solutions that may be caused by local optimization. It can respond quickly and can be flexibly adjusted, which enhances the scalability of the network, improves the utilization of network resources, and further improves the efficiency and accuracy of congestion control. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 It is a schematic diagram of a centralized RoCEv2 network congestion control method provided by an embodiment of the present invention;

[0039] Figure 2 is a schematic diagram of data synchronization between a master and a slave network controller provided by an embodiment of the present invention;

[0040] Figure 3 is a schematic diagram of a network controller state change diagram provided by an embodiment of the present invention;

[0041] Figure 4 Schematic diagram of the linear forwarding table format provided by an embodiment of the present invention;

[0042] Figure 5 It is a schematic diagram of the ICMP fault reporting process provided by an embodiment of the present invention;

[0043] Figure 6 It is a schematic diagram of the CSP protocol congestion reporting process provided by an embodiment of the present invention;

[0044] Figure 7 Schematic diagram of the CSP message format provided by an embodiment of the present invention;

[0045] Figure 8 is a schematic diagram of the CSP TLV format provided by an embodiment of the present invention;

[0046] Fig. 9 is a schematic diagram of network congestion path switching provided by an embodiment of the present invention;

[0047] Fig.10 is a schematic diagram of active and standby path information of a network controller provided according to an embodiment of the present invention;

[0048] Fig.11 is a schematic diagram of a segment routing message format provided according to an embodiment of the present invention;

[0049] Fig.12 is a schematic diagram of a RoCEv2 congestion control framework provided according to an embodiment of the present invention;

[0050] Fig.13 It is a schematic diagram of the processing process of each network component when congestion occurs in a RoCEv2 network congestion control framework provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention can be explained in detail through the following embodiments. The purpose of providing the present invention is to protect all technical improvements within the scope of the present invention. In the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "front", "back", "left", "right", etc. indicating directions or positional relationships, they only correspond to the drawings of the present application for the convenience of describing the present invention, and do not indicate or imply that the device or element referred to must have a specific direction.

[0052] Embodiment 1.

[0053] refer to Figure 1According to this embodiment, a centrally controlled RoCEv2 network congestion control method is provided, which is applied to a RoCEv2 data center network. The RoCEv2 data center network includes a network controller and a switch. The network controller is used to listen to messages sent by the switch in real time and send a control command to the switch to request the switch to report link information;

[0054] Specifically, the network controller includes one or more network controllers; if there are multiple network controllers, one of them is a master network controller and the others are slave network controllers, and data synchronization can be performed between the master network controller and the slave network controllers.

[0055] To prevent a single point of failure caused by a network controller failure, multiple network controllers can be configured, but only one of them is the master network controller, and the others are slave network controllers. The master and slave network controllers synchronize data through periodic data exchange, that is, the slave network controller periodically requests data from the master network controller, and the master network controller sends back data update information to achieve data synchronization. Figure 2 shown.

[0056] Specifically, the master network controller and the slave network controller can be interchangeable.

[0057] The slave network controller periodically detects the status of the master network controller. If the master network controller is active, it obtains the latest topology information from the master network controller. If the master controller cannot be found, the slave network controller that first discovers this situation is upgraded to the master network controller, and the original master network controller is downgraded to the slave network controller. The status changes of the master and slave network controllers are as follows: Figure 3 shown.

[0058] When the data center is operating normally, the switch will actively report the network status, and the network controller will passively obtain messages sent by the switch through real-time listening to learn about the network operation status. The network controller will also actively send network measurement messages to obtain link information and actively learn about the network status. When it is found that the probability of network congestion exceeds a certain threshold, it will take corresponding measures.

[0059] The link information includes packet loss rate, forwarding delay, queue length, and link bandwidth capacity; the link bandwidth capacity includes maximum link bandwidth capacity and minimum link bandwidth capacity; the forwarding delay includes minimum forwarding delay and maximum forwarding delay.

[0060] The following steps are involved:

[0061] S1. When the data center network is started, the network controller obtains the network topology through the network topology detection technology, calculates the switch forwarding table and the alternative forwarding table according to the network topology, and saves them;

[0062] Specifically, the network topology detection technology includes a breadth-first source routing detection technology or a depth-first source routing detection technology;

[0063] The switch uses a linear forwarding table. The preferred and alternative forwarding paths are in the same table. Its format is as follows: Figure 4 The forwarding table size is determined by the scale and is usually no larger than 48K.

[0064] The network controller uses breadth-first source routing network topology detection technology when the data center starts up to calculate the positive and alternative forwarding tables for each network device;

[0065] Specifically, the following steps are included to calculate the switch forwarding table and the candidate forwarding table according to the network topology:

[0066] S11, assign a local identifier LID to each switch SW(i), and initialize the selected forwarding table SW_MAT and the alternative forwarding table SW_MAT_Backup to record the connection relationship between the port of each switch SW(i) and the LID of the directly connected remote switch remote_SW;

[0067] S12, update the switch SW(i)'s forwarding table SW_MAT to record the shortest path hop count from each port Port(j) of SW(i) to other switches SW(m).LID in the subnet; if a shorter path can be found through the connected remote switch remote_SW, update the switch's forwarding table SW_MAT accordingly, and retain a backup forwarding table SW_MAT_Backup;

[0068] S13. Repeat step S12 until the LID matrix of each switch no longer changes, and the forwarding table and the alternative forwarding table of the switch SW(i) are established.

[0069] The specific algorithm language for implementing the above calculation of the switch's positive forwarding table and the alternative forwarding table based on the network topology is not limited, and all code languages ​​that can implement the above functions are protected.

[0070] Specifically, step S1 further includes: setting a control path between all network elements for transmission of control messages and report information;

[0071] A control path with the highest priority is set between all network elements, and control messages are forwarded first in any case; even when network congestion occurs, the network controller can send control messages to the switch through the control path or receive report information that the switch wants to pass to the network controller.

[0072] Network elements refer to all participants in the network, including network controllers, senders, receivers, and switches.

[0073] S2, the network controller determines the network operation status according to the link information;

[0074] Specifically, the network operation state includes a network congestion state and a normal state; when the data center is operating normally, the network controller actively measures the network operation state, that is, actively sends a network measurement message through the control path to obtain link information, and then determines the network operation state according to the maximum forwarding delay or queue length or minimum link bandwidth capacity in the link information;

[0075] Specifically: if the maximum forwarding delay exceeds the preset delay threshold, or the queue length exceeds the preset length threshold, or the minimum link bandwidth capacity is less than the preset bandwidth capacity threshold, the network operation state is determined to be a network congestion state; otherwise, the current network operation state is determined to be a normal state.

[0076] The preset delay threshold, preset length threshold, and preset bandwidth capacity threshold are set based on experience of application environment requirements.

[0077] Step S2 also includes: actively reporting the network congestion status to the network controller when the switch is congested;

[0078] When the queue length in the network element exceeds a preset length threshold or the traffic RTT exceeds a preset RTT threshold or an ECN mark is generated, the network element generates a control message report and sends it to the network controller.

[0079] The RTT threshold is set according to application environment requirements and experience.

[0080] The queue length exceeds the preset length threshold: Network elements usually have a buffer (queue) to temporarily store packets that are about to be processed and forwarded. If the queue length is too long, it means that the network device may not be able to process all the arriving packets in time, which may cause network performance degradation. The network device refers to the switch.

[0081] Traffic RTT exceeds the preset RTT threshold: RTT (Round-Trip Time) refers to the time it takes for a network data packet to be sent from the source to the destination and received from the destination. It is an important indicator for measuring network latency and is usually used to evaluate the quality and performance of network connections.

[0082] RTT is a measure of network latency. If the RTT is too high, it may be due to congestion at a node or the entire path, or poor link quality. In this case, the response time of network elements is slow, which may affect user experience and service quality.

[0083] Generate ECN (Explicit Congestion Notification) mark:

[0084] ECN is a congestion control mechanism that allows network devices to notify the sender of network congestion by setting a mark on the packet without dropping the packet. When a network element detects congestion and sets the ECN mark, it also generates a control message.

[0085] This embodiment provides a reference for setting preset thresholds in different application environments, as shown in Table 1:

[0086]

[0087] Among them, the preset length threshold = buffer capacity * queue proportion;

[0088] Preset bandwidth capacity threshold = rated bandwidth * capacity ratio;

[0089] This embodiment also provides a congestion reporting protocol and message format:

[0090] This protocol is used by the congestion point switch CP to report to the network controller CTLP. For the convenience of description, we will refer to the congestion reporting protocol as CSP (Congestion Status Protocol). CSP is based on the ICMP protocol extension.

[0091] If an abnormality occurs in the data forwarding path of the original ICMP protocol, the fault information is usually sent only to the data source end. The process is as follows: Figure 5 shown.

[0092] If the network is congested, ICMP messages are not sent under normal circumstances. ICMP messages are sent only when the connection times out or the network is unreachable. We modified this protocol so that when network congestion occurs, the network device can actively send congestion status messages to the data source and the network controller, and the network controller decides whether to use the backup path. The process is as follows: Figure 6 As shown,

[0093] The CSP message format is as follows Figure 7 As shown, it includes: 1 byte for message type field, 1 byte for code field, 2 bytes for checksum field, 4 bytes for network device ID field; 4 bytes for network layer packet header + the first 32 bits of data message; and 4 bytes for link status information.

[0094] The link status information is organized in TLV format to record the status attributes of the interface. Figure 8As shown in the figure, the type field is 1 byte and the length field is 1 byte, indicating that the TLV structure requires 4-byte alignment according to the total length of bytes; the value field is filled with corresponding data according to the type. Link status data can cover the content contained in the in-band network telemetry protocol.

[0095] S3. If the network operation state is a network congestion state, first use the flow rate adjustment method to regulate the flow through the congested path. If the congestion situation improves after the flow regulation, go to step S2; otherwise, go to step S4.

[0096] The flow rate adjustment methods include DCQCN, HPCC, Timely, and IRN;

[0097] DCQCN, HPCC, Timely, IRN, etc. are a series of methods for network congestion control. They aim to avoid or alleviate congestion by adjusting the data transmission rate and the size of the congestion window, thereby improving the transmission efficiency and reliability of the network. This is an existing technology and will not be described in detail here.

[0098] The improvement of the congestion situation includes that the maximum forwarding delay is reduced to below a preset delay threshold and the queue length is reduced to below a preset length threshold;

[0099] After the flow rate adjustment method is used to control the flow, the congestion situation is improved, indicating that the flow rate adjustment method is effective. After adjusting the flow rate, the process returns to step S2 and continues to use the network controller to monitor the link information, and determines the network operation status according to the link information.

[0100] If the congestion situation is not improved after the flow rate adjustment method is used to control the flow, you need to jump to the next step.

[0101] Specifically, the step S3 further includes, if a congestion warning state occurs, using a flow rate adjustment method to regulate the flow through the congested path;

[0102] The congestion warning state is that the network operation state is normal but the network congestion probability is greater than or equal to a preset congestion probability threshold.

[0103] In order to further comprehensively consider the network status according to the whole network situation, make early warning when the network status is normal, and better avoid the occurrence of congestion, this embodiment introduces the concept of network congestion probability, which is the ratio of the network link measurement value divided by the corresponding threshold;

[0104] When the ratio approaches or exceeds the preset congestion probability threshold, it is considered that the probability of congestion is high, and the current quasi-network state is determined to be a congestion warning state.

[0105] The preset congestion probability threshold is selected between [85%, 95%]. Preferably, in this embodiment, the preset congestion probability threshold is 90%.

[0106] For example, if the current maximum forwarding delay is 92ms, the preset delay threshold is 100ms, and the current maximum forwarding delay / preset delay threshold = 92%, which is greater than the preset congestion probability threshold of 90%, then the current network state is considered to be a congestion warning state;

[0107] This embodiment is only one of the network link measurement values ​​to determine the congestion warning state. In other possible implementations, the congestion state can be determined when the ratio of multiple parameter values ​​in the network link measurement value to the corresponding threshold exceeds the preset congestion probability threshold, which is not limited here.

[0108] S4. The network controller notifies the congested switch according to the comprehensive link cost to set the alternative forwarding table as the primary forwarding table for data forwarding, and sets the original primary forwarding table as the alternative forwarding table, and then goes to step S2.

[0109] Specifically, the network controller can calculate the comprehensive link cost of each congested link based on the link information, further determine the congestion of the network based on the comprehensive link cost, and then select one or several congested paths with the largest link cost for adjustment;

[0110] The comprehensive link cost is specifically: the comprehensive link cost is calculated according to a bandwidth cost based on bandwidth, a delay cost based on delay, and a load cost based on load.

[0111] Specifically, the bandwidth-based bandwidth cost = current maximum link bandwidth capacity / reference bandwidth capacity; taking a 1Mbps link as a reference: link cost = 1010 / link bandwidth, such as a 1Gbps link, its link cost = 1010 / 109 = 10, and a 10Gbps link, its link cost = 1010 / 1010 = 1.

[0112] Specifically, the delay cost based on delay = current maximum forwarding delay / reference delay; taking 0.1 millisecond delay as a reference: 1 millisecond delay link cost = 1 / 0.1 = 10, 3 millisecond delay link cost = 3 / 0.1 = 30.

[0113] Specifically, the load-based load cost=current load / reference load (with 1% as a reference), for 50% load, link cost=50% / 1%=50, for 90% load, link cost=90% / 1%=90.

[0114] Specifically, the comprehensive link cost calculation method assigns a weight to each factor and then calculates the weighted average cost. The specific weight is determined according to actual needs:

[0115] In this embodiment, the bandwidth weight is set to 40%, the delay weight is set to 30%, and the load weight is set to 30%. The comprehensive link cost = bandwidth cost*0.4+delay cost*0.3+load cost*0.3. For example, a congested link has a bandwidth of 1 Gbps, a delay of 3 milliseconds, and a load of 50%. The comprehensive link cost = 10*0.4+30*0.3+50*0.3=28.

[0116] Sort by comprehensive link cost, select the one with the largest comprehensive link cost or select several congested paths from large to small to notify the switches on the path to use the backup forwarding table for data forwarding. The number of paths selected in this embodiment is determined according to the specific situation, preferably, the number of paths that is rounded to one third of the number of congested paths is selected.

[0117] This embodiment collects network status information of each node from the network controller, calculates the comprehensive link cost, and generates a global flow control strategy, thereby achieving unified management and optimization of network flow.

[0118] This embodiment also provides a possible implementation method for path switching when the network is congested. Assume that the data center network topology is as follows: Fig. 9 As shown, the network core layer switches are A, B, and C, the aggregation layer switches are D, E, F, and G, and the edge layer switches are H, I~K. The working process of the present invention is as follows:

[0119] 1) The network controller CTLP collects network topology information when the data center network is started. The optional methods include breadth-first source routing detection and depth-first source routing detection.

[0120] 2) After path detection, the segment routing path from source S to destination Y is as follows Fig.10 As shown, it includes a primary path and a backup path from the source S to the destination Y; it can be understood that the primary and backup paths are seen by the controller from a global perspective, and the primary and backup forwarding tables are seen by the switch from a local perspective.

[0121] The format of segment routing is as follows Fig.11 As shown;

[0122] 3) When switch F on the path fails, switch F or its predecessor switch A will sense the link failure and report this to the network controller CTLP;

[0123] 4) The network controller CTLP switches the forwarding path from S to Y to the backup path based on the saved topology and backup path information, thereby effectively avoiding the faulty path.

[0124] The present embodiment provides a centrally controlled RoCEv2 network congestion control method, comprising the following steps: S1, the network controller obtains the network topology through the network topology detection technology when the data center network is started, calculates the switch's active forwarding table and alternative forwarding table according to the network topology and saves them; S2, the network controller determines the network operation status according to the link information; the network operation status includes the network congestion status and the normal status; S3, if the network operation status is the network congestion status, use the flow rate adjustment method to perform flow regulation, if the congestion situation improves after flow regulation, go to step S2, otherwise go to step S4; S4, the network controller notifies the switch where congestion occurs according to the comprehensive link cost to set the alternative forwarding table as the active forwarding table for data forwarding, and sets the original active forwarding table as the alternative forwarding table, and go to step S2.

[0125] The beneficial effects of this embodiment are:

[0126] 1. Centralized control: The network controller collects network status information of each node and generates a global traffic control strategy, thus achieving unified management and optimization of network traffic.

[0127] 2. Policy consistency: Each node adjusts traffic according to the global traffic control policy issued by the network controller, ensuring the consistency and effectiveness of the control policy.

[0128] 3. High efficiency: When a network failure occurs, the traditional flow control algorithm basically fails. The present invention can adjust the network flow and solve the congestion problem by adjusting the forwarding path. The network controller can quickly respond to changes in network status and generate adaptive flow control strategies, thereby improving the efficiency and accuracy of congestion control.

[0129] In addition, the present invention also sets up a control path with the highest priority between all network elements, and control messages are forwarded preferentially under any circumstances; even when network congestion occurs, the network controller can send control messages to the switch through the control path or receive report information that the switch wants to pass to the network controller, thereby ensuring the timely transmission of messages and further improving the efficiency of congestion control.

[0130] refer to Fig.12 This embodiment also provides a centrally controlled RoCEv2 network congestion control framework, which abstracts the RoCEv2 network into four major components: a network controller CTLP (ConTroL Point), a sender RP (Reaction Point), a switch CP (Congestion Point) and a receiver NP (Notification Point).

[0131] A control path with the highest priority is set between the network controller CTLP and the switch CP, and the control flow and the report flow follow the control path; the network controller CTLP stores the network topology of the entire network, as well as the positive forwarding table and backup path forwarding table of each switch CP, and monitors the network congestion of the entire network in an active or passive manner; the switch CP detects the occurrence of network congestion in a specific way and reports it to the network controller CTLP; and the receiver NP receives the network congestion signal and piggybacks it in the feedback flow to notify the sender RP of the occurrence of network congestion; after receiving the feedback, the sender RP adjusts the sending rate through a specific algorithm; when the network controller CTLP believes that the congestion problem cannot be solved by adjusting the flow rate alone, the network controller CTLP can choose to send a control message to the switch to adjust the forwarding path. After receiving the control message, the switch CP performs the path adjustment operation and feeds back the completion information to the controller CTLP, or directly sends the alternative path to the remote end of the data to perform segment routing programming to avoid the congested path.

[0132] This embodiment provides a data processing process of each network component in a congestion control framework when congestion occurs in a RoCEv2 network, such as Fig.13 As shown, specifically including:

[0133] ① The switch CP monitors the queue length and other information when forwarding data, and sets the data flow ECN mark if congestion occurs;

[0134] ② The receiving NP receives the data stream with ECN mark from the network and piggybacks it into the feedback stream;

[0135] ③ After receiving the feedback, the sender RP adjusts the sending rate through a specific algorithm (such as DCQCN, HPCC, Timely, IRN, etc.);

[0136] ④ If RP finds that the situation has improved (queue length and delay have decreased), it goes to step ①, otherwise the switch CP sends the congested flow information to the controller CTLP;

[0137] ⑤ The controller CTLP considers the comprehensive link cost based on the overall network situation to determine whether to adjust the path;

[0138] ⑥ If the controller CTLP decides to enable the alternative path, it will issue a command to the RP and CP to enable the alternative path;

[0139] ⑦ The RP and CP enable the alternative path and make it the primary path, and set the original selected path as the alternative path.

[0140] The control flow corresponds to the control message, and the report flow corresponds to the report information, which are transmitted through the control path; the feedback flow and the data flow are transmitted through the main path.

[0141] The network congestion control framework of this embodiment abstracts the RoCEv2 network into four major components, which can clarify the role and responsibility of each component in network congestion control, reduce the complexity of network control, and facilitate management and maintenance. The network controller CTLP centrally manages the network topology and forwarding table, which can globally optimize the use of network resources and avoid suboptimal solutions that may be caused by local optimization. It can respond quickly and can be flexibly adjusted, which enhances the scalability of the network, improves the utilization of network resources, and further improves the efficiency and accuracy of congestion control.

[0142] The parts of the present invention that are not described in detail are prior art. It is obvious to those skilled in the art that the present invention is not limited to the details of the above-mentioned exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention; therefore, no matter from which point of view, the embodiments should be regarded as exemplary and non-restrictive, and it is intended that all changes that fall within the meaning and scope of equivalent elements are included in the present invention.

Claims

1. A centralized control RoCEv2 network congestion control method, which is applied to a RoCEv2 data center network, wherein the RoCEv2 data center network includes a network controller and a switch, wherein the network controller is used to monitor messages sent by the switch in real time and send a control command to the switch to request the switch to report link information; characterized in that: The following steps are involved: S1. When the data center network is started, the network controller obtains the network topology through the network topology detection technology, calculates and saves the switch forwarding table and the alternative forwarding table according to the network topology; Step S1 also includes: setting the highest priority control path between all network elements for the transmission of control messages and report information; S2, the network controller determines the network operation status according to the link information; the network operation status includes network congestion status, congestion warning status and normal status; step S2 also includes: when congestion occurs, the switch actively reports the network congestion status to the network controller; S3. If the network operation state is a network congestion state or a congestion warning state, first use the flow rate adjustment method to regulate the flow through the congested path. If the congestion situation improves after the flow regulation, go to step S2, otherwise go to step S4; the congestion warning state is that the network operation state is normal but the network congestion probability is greater than or equal to the preset congestion probability threshold; S4, the network controller notifies the congested switch according to the comprehensive link cost to set the alternative forwarding table as the main forwarding table for data forwarding, and sets the original main forwarding table as the alternative forwarding table, and then goes to step S2; The network controller calculates the comprehensive link cost of each congested link based on the link information, sorts them according to the comprehensive link cost, selects the one with the largest comprehensive link cost or selects several congested paths from large to small to notify the switches on the path to use the backup forwarding table for data forwarding.

2. The method according to claim 1, characterized in that The link information includes packet loss rate, forwarding delay, queue length, and link bandwidth capacity; the link bandwidth capacity includes maximum link bandwidth capacity and minimum link bandwidth capacity; the forwarding delay includes minimum forwarding delay and maximum forwarding delay.

3. The method according to claim 1, characterized in that: Step S1 calculates the switch forwarding table and the candidate forwarding table according to the network topology, and specifically includes the following steps: S11, assign a local identifier LID to each switch SW(i), and initialize the selected forwarding table SW_MAT and the alternative forwarding table SW_MAT_Backup to record the connection relationship between the port of each switch SW(i) and the LID of the directly connected remote switch remote_SW; S12, update the switch SW(i)'s forwarding table SW_MAT to record the shortest path hop count from each port Port(j) of SW(i) to other switches SW(m).LID in the subnet; if a shorter path is found through the connected remote switch remote_SW, update the switch's forwarding table SW_MAT accordingly, and retain a backup forwarding table SW_MAT_Backup; S13. Repeat step S12 until the LID matrix of each switch no longer changes, and the forwarding table and the alternative forwarding table of the switch SW(i) are established.

4. The method according to claim 1, characterized in that Step S2 is specifically as follows: if the maximum forwarding delay exceeds the preset delay threshold, or the queue length exceeds the preset length threshold, or the minimum link bandwidth capacity is less than the preset bandwidth capacity threshold, the network operation state is determined to be a network congestion state; otherwise, the current network operation state is determined to be a normal state.

5. The method according to claim 4, characterized in that Step S2 also includes: when the queue length in the network element exceeds a preset length threshold or the traffic RTT exceeds a preset RTT threshold or an ECN mark is generated, the network element generates a control message report and sends it to the network controller.

6. The method according to claim 1, characterized in that The flow rate adjustment methods in step S3 include DCQCN, HPCC, Timely, and IRN.

7. The method according to claim 1, characterized in that The network controller includes one or more network controllers; if there are multiple network controllers, one of them is a master network controller and the others are slave network controllers, and data synchronization can be performed between the master network controller and the slave network controllers; the master network controller and the slave network controllers are interchangeable.

Images