Flow congestion control system and method

Abstract

The invention provides a flow congestion control system and method. The system comprises an end side sending device, an intermediate routing device and an end side receiving device. The end side sending equipment is used for sending a data message to the end side receiving equipment through the intermediate routing equipment; the intermediate routing equipment is used for detecting whether traffic congestion occurs at the side of the intermediate routing equipment after receiving a data message, carrying a congestion mark in the data message after detecting the traffic congestion, and forwarding the data message to the end-side receiving equipment; the end-side receiving device is used for analyzing whether a congestion mark is carried in a data message after receiving the data message, detecting whether traffic congestion occurs at the end-side receiving device side, and sending the data message to the end-side receiving device side after analyzing that the congestion mark is carried in the data message or detecting that traffic congestion occurs at the end-side receiving device side. And constructing a congestion notification message, and sending the congestion notification message to the end side sending equipment through the intermediate routing equipment.

Description

Technical Field

[0001] This application belongs to the technical field of network traffic congestion control, and particularly relates to a traffic congestion control system and method. Background Art

[0002] The Remote Direct Memory Access (RDMA) protocol provides the high throughput, ultra-low latency, and low CPU overhead required by data center applications. RDMA is deployed using the RoCEv2 protocol, which relies on priority-based flow control (PFC) to achieve a packet-drop-free network. Data Center Quantized Congestion Notification (DCQCN) is an end-to-end congestion control solution for RoCEv2. Summary of the Invention

[0003] The purpose of this application is to provide a traffic congestion control system and method, aiming to solve the problem in the related art that the network transmission system cannot identify traffic congestion occurring at the end-side receiving layer in the RDMA protocol.

[0004] According to the first aspect of this application, a traffic congestion control system is provided, including: an end-side sending device, an intermediate routing device, and an end-side receiving device;

[0005] The end-side sending device is configured to send data packets to the end-side receiving device via the intermediate routing device;

[0006] The intermediate routing device is configured to detect whether traffic congestion occurs on the intermediate routing device side after receiving the data packet, and forward the congestion mark carried after the data packet to the end-side receiving device after detecting traffic congestion;

[0007] The end-side receiving device is configured to parse whether a congestion mark is carried in the data packet after receiving the data packet, and detect whether traffic congestion occurs on the end-side receiving device side, and construct a congestion notification packet and send the congestion notification packet to the end-side sending device via the intermediate routing device after parsing that the data packet carries a congestion mark or detecting that traffic congestion occurs on the end-side receiving device side.

[0008] In this embodiment, on the transmission path of the data packet, by detecting two nodes where traffic congestion may occur, namely at the intermediate routing device and the end-side receiving device, and then, according to the detection result, the end-side receiving device constructs a congestion notification packet and returns it to the end-side sending device to notify the end-side sending device that traffic congestion has occurred on this network transmission path. In this way, in the Incast multi-sender scenario, since the traffic congestion detection covers various possible situations, the network performance will not degrade due to the failure to mark the occurrence of traffic congestion, and ultimately, the situation of packet loss at the end side will not occur. Additionally, in this embodiment, under the RDMA protocol, the existing DCQCN congestion algorithm is extended, but the current standard protocol is not modified. The format of the congestion notification packet adopted and the speed regulation algorithm implemented by the end-side sending device after receiving the congestion notification packet both conform to the definition of the protocol standard.

[0009] In an alternative embodiment, the end-side sending device is further configured to reduce the speed of subsequent data packets sent to the end-side receiving device after receiving the congestion notification packet.

[0010] The speed reduction processing algorithm implemented by the end-side sending device after receiving the congestion notification packet can still adopt the standard algorithm that conforms to the RDMA protocol definition. Therefore, the processing process of the end-side sending device does not need to be modified due to the implementation of this solution.

[0011] In an alternative embodiment, after receiving the data packet, the intermediate routing device determines whether traffic congestion occurs by detecting the depth status of the packet buffer queue on the intermediate routing device side;

[0012] After receiving the data packet, the end-side receiving device determines whether traffic congestion occurs by detecting the depth status of the packet buffer queue on the end-side receiving device side.

[0013] In this way, in the Incast multi-sender scenario, since the traffic congestion detection covers various possible situations, the network performance will not degrade due to the failure to mark the occurrence of traffic congestion, and ultimately, the situation of packet loss at the end side will not occur.

[0014] In an alternative embodiment, if the end-side receiving device parses that the data packet carries a congestion mark, it constructs a congestion notification packet;

[0015] If the end-side receiving device parses that the data packet does not carry a congestion mark but detects traffic congestion on the end-side receiving device side, it constructs a congestion notification packet.

[0016] In this way, in the case of multiple Incast scenarios, since the traffic congestion detection covers various possible situations, the network performance will not degrade due to the failure to mark the occurrence of traffic congestion, and finally, packet loss on the end side will not occur.

[0017] In an alternative embodiment, the end-side sending device and the end-side receiving device are network cards.

[0018] According to a second aspect of the present application, there is provided a traffic congestion control method, including:

[0019] The end-side sending device sends a data packet to the end-side receiving device via an intermediate routing device;

[0020] After receiving the data packet, the intermediate routing device detects whether traffic congestion occurs on the intermediate routing device side, and after detecting traffic congestion, forwards the congestion mark carried after the data packet to the end-side receiving device;

[0021] After receiving the data packet, the end-side receiving device parses whether the congestion mark is carried in the data packet, and detects whether traffic congestion occurs on the end-side receiving device side;

[0022] After parsing that the data packet carries the congestion mark or detecting that traffic congestion occurs on the end-side receiving device side, the end-side receiving device constructs a congestion notification packet and sends the congestion notification packet to the end-side sending device via the intermediate routing device.

[0023] In an alternative embodiment, the method further includes:

[0024] After receiving the congestion notification packet, the end-side sending device reduces the speed of subsequent data packets sent to the end-side receiving device.

[0025] In an alternative embodiment, after receiving the data packet, the intermediate routing device detects whether traffic congestion occurs on the intermediate routing device side, including:

[0026] After receiving the data packet, the intermediate routing device determines whether traffic congestion occurs by detecting the depth state of the packet buffer queue on the intermediate routing device side;

[0027] Detecting whether traffic congestion occurs on the end-side receiving device side includes:

[0028] After receiving the data packet, the end-side receiving device determines whether traffic congestion occurs by detecting the depth state of the packet buffer queue on the end-side receiving device side.

[0029] In an alternative embodiment, after the end - side receiving device parses that the data packet carries a congestion mark or detects traffic congestion on the end - side receiving device side, it constructs a congestion notification packet, including:

[0030] If the end - side receiving device parses that the data packet carries a congestion mark, it constructs a congestion notification packet;

[0031] If the end - side receiving device parses that the data packet does not carry a congestion mark but detects traffic congestion on the end - side receiving device side, it constructs a congestion notification packet.

[0032] In an alternative embodiment, the end - side sending device and the end - side receiving device are network cards.

[0033] Other features and advantages of the present application will be described in the subsequent specification, and will be partially obvious from the specification, or understood by implementing the present application. The objectives and other advantages of the present application can be achieved and obtained through the structures and processes pointed out in the specification and the drawings. Brief Description of the Drawings

[0034] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the drawings in the following description are certain embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0035] Figure 1 It is a schematic diagram of the implementation process of the DCQCN algorithm according to related technologies.

[0036] Figure 2 It is a schematic diagram of the congestion occurrence effect in the Incast multi - to - one traffic scenario according to related technologies.

[0037] Figure 3 It is a schematic diagram of the framework structure of the traffic congestion control system according to an exemplary embodiment of the present application.

[0038] Figure 4 It is a schematic diagram of the congestion marking process of the traffic congestion control system according to an exemplary embodiment of the present application.

[0039] Figure 5 It is a schematic diagram of the traffic congestion control method process according to an exemplary embodiment of the present application. Detailed Embodiments

[0040] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are some, but not all, of the embodiments of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in this application without creative efforts fall within the scope of protection of this application.

[0041] The implementation of the DCQCN (Data Center Quantized Congestion Notifcation) algorithm is as Figure 1 shown: The data packets sent by the NIC0 on the end side pass through the switch to the NIC1 on the end side for reception. When the data packets pass through the egress queue of the switch, the switch will monitor the queue depth status in real time. According to the predefined congestion threshold parameters (Kmin is the initial congestion trigger threshold, and Kmax is the full congestion threshold), the switch performs the following processing:

[0042] If the queue depth is in the interval [Kmin, Kmax), enable the probabilistic ECN marking mechanism, and the marking probability increases linearly with the queue depth;

[0043] If the queue depth ≥ Kmax, force ECN marking for all packets. This mechanism transmits the congestion status to the terminal node through the ECN field (Explicit Congestion Notification) in the IP header.

[0044] The NIC1 on the end side receives the packet and the hardware acceleration engine parses the ECN status of the packet in real time. After detecting the ECN mark, the NP module (Notification Point) constructs a congestion notification packet (CNP), and its characteristics include:

[0045] Adopt the CNP control packet format extended by the RoCEv2 protocol, and carry metadata such as flow identifiers (QP sequence numbers) and congestion quantization levels.

[0046] Control the CNP sending frequency based on a throttling mechanism (typically set to generate one CNP every 50 μs) to avoid feedback storms.

[0047] NIC0 performs speed reduction processing according to the received CNP packets.

[0048] The above protocol implementation method will have the following problems in the Incast many-to-one traffic scenario as Figure 2 shown:

[0049] Under the CLOS multi - level switching architecture, although the overall network achieves 1:1 non - blocking convergence, the inbound port bandwidth of the end - side receiving network interface card (NIC) still constitutes a physical convergence bottleneck (for example, a 100Gbps NIC receives aggregated traffic from multiple 400Gbps links). At this time, the congestion core area appears not only in the switch queue but also in the RX queue buffer of the terminal receiving NIC. In related technologies, the ECN marking mechanism based on the switch egress queue (such as the Kmin / Kmax waterline detection of DCQCN) cannot identify the congestion of the terminal NIC RX queue buffer, resulting in the inability to achieve speed regulation.

[0050] Based on the above analysis, see Figure 3 As shown, this application exemplarily proposes a traffic congestion control system, including: an end - side sending device, an intermediate routing device, and an end - side receiving device;

[0051] The end - side sending device is used to send data packets to the end - side receiving device via the intermediate routing device;

[0052] The intermediate routing device is used to detect whether traffic congestion occurs on the intermediate routing device side after receiving the data packet, and forward the congestion mark carried after the data packet to the end - side receiving device after detecting traffic congestion;

[0053] The end - side receiving device is used to parse whether the congestion mark is carried in the data packet after receiving the data packet, and detect whether traffic congestion occurs on the end - side receiving device side. After parsing that the data packet carries the congestion mark or detecting that traffic congestion occurs on the end - side receiving device side, construct a congestion notification packet, and send the congestion notification packet to the end - side sending device via the intermediate routing device.

[0054] Exemplarily, there are two congestion nodes focused on detection in this embodiment: the intermediate routing layer cache and the end - side receiving layer cache; when traffic congestion occurs in these two places, the traffic congestion control system will feedback congestion information to the sending side for speed regulation processing.

[0055] Exemplarily, the traffic congestion control system is applicable to the RDMA protocol. The end - side sending device can send data packets to the end - side receiving device via the intermediate routing device. After receiving the data packet, the intermediate routing device detects whether there is traffic congestion on its own side. If there is traffic congestion, carry the congestion mark in the data packet and send it to the end - side receiving device. If there is no traffic congestion, do not carry the congestion mark in the data packet, and forward the data packet without the congestion mark to the end - side receiving device. In some embodiments, the congestion mark can be carried through the ECN field (Explicit Congestion Notification) of the IP header of the data packet.

[0056] Exemplarily, after receiving a data packet, the end-side receiving device constructs a congestion notification packet according to two situations. These two situations are: parsing a congestion mark carried in the data packet, or detecting traffic congestion at the local end. When either of these two situations occurs, the end-side receiving device constructs a congestion notification packet and returns the congestion notification packet to the end-side sending device via an intermediate routing device to notify the end-side sending device that there is traffic congestion in the network transmission path leading to the end-side receiving device. Among them, the congestion notification packet can adopt the CNP control packet format extended by the RoCEv2 protocol and carry metadata such as a flow identifier (QP sequence number) and a congestion quantization level; the end-side receiving device can also control the CNP sending frequency based on a throttling mechanism (for example, generating a CNP every 50 μs) to avoid feedback storms.

[0057] In this embodiment, on the transmission path of the data packet, by detecting two nodes where traffic congestion may occur, that is, at the intermediate routing device and the end-side receiving device, the end-side receiving device constructs a congestion notification packet according to the detection result and returns it to the end-side sending device to notify the end-side sending device that there is traffic congestion on this network transmission path. In this way, in the Incast multi-send scenario, since the traffic congestion detection covers various possible situations, the network performance will not decline due to the failure to mark the traffic congestion situation, and finally the situation of end-side packet loss will not occur. In addition, in this embodiment, under the RDMA protocol, the existing DCQCN congestion algorithm is extended, but the current standard protocol is not modified. The congestion notification packet format adopted and the speed regulation algorithm implemented by the end-side sending device after receiving the congestion notification packet both conform to the definition of the protocol standard.

[0058] In some optional implementation manners, the end-side sending device is further configured to reduce the speed of subsequent data packets sent to the end-side receiving device after receiving the congestion notification packet.

[0059] Exemplarily, after receiving the congestion notification packet, the end-side sending device can implement a speed reduction processing algorithm to reduce the speed of the data packets sent to the end-side receiving device and relieve the traffic congestion of the intermediate routing device and / or the end-side receiving device.

[0060] The speed reduction processing algorithm implemented by the end-side sending device after receiving the congestion notification packet can still adopt the standard algorithm that conforms to the RDMA protocol definition, so the processing process of the end-side sending device does not need to be modified due to the implementation of this solution.

[0061] In some optional implementation manners, after receiving a data packet, the intermediate routing device determines whether there is traffic congestion by detecting the depth state of the packet cache queue on the intermediate routing device side;

[0062] After the end - side receiving device receives a data packet, it determines whether traffic congestion occurs by detecting the depth status of the packet buffer queue on the end - side receiving device side.

[0063] Exemplarily, traffic congestion on the intermediate routing device side mainly occurs at the packet buffer queue of the data packet. Therefore, the intermediate routing device can detect the depth status of the packet buffer queue of the data packet. If the depth of the buffer queue is relatively deep, that is, there are more buffered data packets, it can be considered that there is a traffic congestion situation. Then, a congestion mark is carried in the data packet. For example, after setting the ECN field in the IP header of the data packet to the corresponding value, it is forwarded to the end - side receiving device. If the depth of the packet buffer queue of the data packet is relatively shallow, that is, there are fewer buffered data packets, it can be considered that there is no traffic congestion situation, and thus no congestion mark is carried in the data packet, but it is directly forwarded to the end - side receiving device.

[0064] Exemplarily, traffic congestion may also occur at the end - side receiving device, especially in the Incast multi - send scenario. Therefore, in this embodiment, the end - side receiving device will also detect the depth of the packet buffer queue for caching the received data packets in real time. If the depth of the packet buffer queue is relatively deep, that is, there are more buffered data packets, it can be considered that there is a traffic congestion situation. If the depth of the packet buffer queue of the data packet is relatively shallow, that is, there are fewer buffered data packets, it can be considered that there is no traffic congestion situation.

[0065] In this way, in the Incast multi - send scenario, since traffic congestion detection covers various possible situations, the network performance will not decrease due to the failure to mark the occurrence of traffic congestion, and finally, the situation of end - side packet loss will not occur.

[0066] In some alternative implementation manners, if the end - side receiving device parses that the data packet carries a congestion mark, it constructs a congestion notification packet;

[0067] If the end - side receiving device parses that the data packet does not carry a congestion mark, but detects that traffic congestion occurs on the end - side receiving device side, it constructs a congestion notification packet.

[0068] Exemplarily, as shown in Figure 4 This traffic congestion control system is divided into three layers, namely the end - side sending layer, the end - side receiving layer, and the intermediate routing layer. The congestion judgment and processing process is as follows:

[0069] 1. The end - side sending device NIC0 in the end - side sending layer sends a data packet to the end - side receiving device NIC3 in the end - side receiving layer. When there is a queue congestion at the intermediate routing device Switch0 in the intermediate routing layer, Switch0 will mark the data packet with an ECN and send it to NIC3. If NIC3 detects that the data packet has an ECN mark, it will send a standard CNP packet to NIC0. NIC0 parses the CPN packet to determine that the path is congested and performs speed - adjustment processing.

[0070] 2. NIC0 sends a data packet to NIC3. When there is no queue congestion at Switch0, Switch0 normally transmits the data packet to NIC3. When NIC3 receives the data packet and detects congestion in the end - side receiving buffer, it will send a standard CNP packet to NIC0. NIC0 parses the CPN packet to determine that the path is congested and performs speed - adjustment processing.

[0071] In this way, in the Incast multi - hit scenario, since the traffic congestion detection covers various possible situations, the network performance will not degrade due to the failure to mark the traffic congestion situation, and finally there will be no end - side packet loss.

[0072] In some alternative implementation manners, the end - side sending device and the end - side receiving device are network cards.

[0073] Exemplarily, the intermediate routing device can be an intermediate switch, and there can be multiple intermediate routing devices. After any intermediate routing device on the transmission path of the data packet detects traffic congestion, it can carry a congestion mark in the data packet.

[0074] See Figure 5 the flowchart of

[0075] Step S501: The end - side sending device sends a data packet to the end - side receiving device via the intermediate routing device.

[0076] Step S502: After receiving the data packet, the intermediate routing device detects whether there is traffic congestion on the intermediate routing device side, and after detecting traffic congestion, it forwards the data packet with a congestion mark carried to the end - side receiving device.

[0077] Step S503: After receiving the data packet, the end - side receiving device parses whether there is a congestion mark in the data packet and detects whether there is traffic congestion on the end - side receiving device side.

[0078] Step S504: After the end - side receiving device parses that the data packet carries a congestion mark or detects traffic congestion on the end - side receiving device side, it constructs a congestion notification packet and sends the congestion notification packet to the end - side sending device via the intermediate routing device.

[0079] In some alternative implementations, the method further includes:

[0080] After receiving the congestion notification message, the sending device on the terminal side reduces the speed of subsequent data messages sent to the receiving device on the terminal side.

[0081] In some alternative implementations, after receiving a data message, the intermediate routing device detects whether traffic congestion occurs on the side of the intermediate routing device, including:

[0082] After receiving the data message, the intermediate routing device determines whether traffic congestion occurs by detecting the depth status of the message buffer queue on the side of the intermediate routing device;

[0083] Detecting whether traffic congestion occurs on the side of the receiving device on the terminal side, including:

[0084] After receiving the data message, the receiving device on the terminal side determines whether traffic congestion occurs by detecting the depth status of the message buffer queue on the side of the receiving device on the terminal side.

[0085] In some alternative implementations, after parsing out the congestion mark carried in the data message or detecting traffic congestion on the side of the receiving device on the terminal side, the receiving device on the terminal side constructs a congestion notification message, including:

[0086] If the receiving device on the terminal side parses out the congestion mark carried in the data message, it constructs a congestion notification message;

[0087] If the receiving device on the terminal side parses out that the data message does not carry the congestion mark, but detects traffic congestion on the side of the receiving device on the terminal side, it constructs a congestion notification message.

[0088] In some alternative implementations, the sending device on the terminal side and the receiving device on the terminal side are network cards.

[0089] The above method can be implemented by the traffic congestion control system provided in the above embodiments. For specific implementation manners, reference can be made to the description of the traffic congestion control system in the above embodiments, which will not be elaborated here.

[0090] It can be understood that the circuit structures, names, and parameters described in the above embodiments are only examples. Those skilled in the art can also make easily conceivable combinations and adjustments to the structural features of the above multiple embodiments according to the usage requirements, and should not limit the concept of the present application to the specific details of the above examples.

[0091] Although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A traffic congestion control system, characterized in that, Including: An end - side sending device, an intermediate routing device, and an end - side receiving device; The end - side sending device is used to send data packets to the end - side receiving device via the intermediate routing device; The intermediate routing device is used to detect whether traffic congestion occurs on the side of the intermediate routing device after receiving the data packet, and carry a congestion mark after the data packet and forward it to the end - side receiving device after detecting traffic congestion; The end - side receiving device is used to parse whether a congestion mark is carried in the data packet after receiving the data packet, and detect whether traffic congestion occurs on the side of the end - side receiving device, and construct a congestion notification packet and send the congestion notification packet to the end - side sending device via the intermediate routing device after parsing that the data packet carries a congestion mark or detecting that traffic congestion occurs on the side of the end - side receiving device.

2. The traffic congestion control system according to claim 1, wherein The end - side sending device is further used to reduce the speed of subsequent data packets sent to the end - side receiving device after receiving the congestion notification packet.

3. The traffic congestion control system according to claim 1 or 2, characterized in that, After receiving the data packet, the intermediate routing device determines whether traffic congestion occurs by detecting the depth status of the packet buffer queue on the side of the intermediate routing device; After receiving the data packet, the end - side receiving device determines whether traffic congestion occurs by detecting the depth status of the packet buffer queue on the side of the end - side receiving device.

4. The traffic congestion control system according to claim 1 or 2, characterized in that, If the end - side receiving device parses that the data packet carries a congestion mark, it constructs a congestion notification packet; If the end - side receiving device parses that the data packet does not carry a congestion mark but detects that traffic congestion occurs on the side of the end - side receiving device, it constructs a congestion notification packet.

5. The traffic congestion control system according to claim 1 or 2, characterized in that, The end - side sending device and the end - side receiving device are network cards.

6. A traffic congestion control method, characterized in that, Including: The end - side sending device sends data packets to the end - side receiving device via the intermediate routing device; After receiving the data packet, the intermediate routing device detects whether traffic congestion occurs on the side of the intermediate routing device, and carries a congestion mark after the data packet and forwards it to the end - side receiving device after detecting traffic congestion; After receiving the data packet, the end - side receiving device parses whether a congestion mark is carried in the data packet, and detects whether traffic congestion occurs on the side of the end - side receiving device; After the end - side receiving device parses that the data packet carries a congestion mark or detects that traffic congestion occurs on the side of the end - side receiving device, it constructs a congestion notification packet and sends the congestion notification packet to the end - side sending device via the intermediate routing device.

7. The traffic congestion control method according to claim 6, characterized in that, The method further includes: The end - side sending device reduces the speed of subsequent data packets sent to the end - side receiving device after receiving the congestion notification packet.

8. The traffic congestion control method according to claim 6 or 7, characterized in that, After receiving the data packet, the intermediate routing device detects whether traffic congestion occurs on the side of the intermediate routing device, including: After receiving the data packet, the intermediate routing device determines whether traffic congestion occurs by detecting the depth status of the packet buffer queue on the side of the intermediate routing device; Detecting whether traffic congestion occurs on the side of the end - side receiving device includes: After receiving the data packet, the end-side receiving device determines whether traffic congestion occurs by detecting the depth status of the packet cache queue on the end-side receiving device side.

9. The traffic congestion control method according to claim 6 or 7, characterized in that After parsing the data packet to carry a congestion mark or detecting traffic congestion on the end-side receiving device side, the end-side receiving device constructs a congestion notification packet, including: If the end-side receiving device parses that the data packet carries a congestion mark, it constructs a congestion notification packet; If the end-side receiving device parses that the data packet does not carry a congestion mark but detects traffic congestion on the end-side receiving device side, it constructs a congestion notification packet.

10. The traffic congestion control method according to claim 6 or 7, characterized in that, The end-side sending device and the end-side receiving device are network cards.

Images