Congestion control method based on time delay

Abstract

The invention discloses a time delay-based congestion control method, which comprises the following steps that: a sender simultaneously sends a current data packet group at a set rate and time, and when a required confirmation message is received, according to a congestion result, a current rate, the number of data packets which are still unconfirmed in the current data packet group and whether the current data packet group is waiting for transmission adjustment or not, the current data packet group is subjected to transmission adjustment; and determining whether to wait for subsequent confirmation or immediately adjust the sending rate and the sending time of the subsequent next data packet group. When a subsequent data packet needs to be transmitted at a reduced speed, selecting a aggressive or conservative adjustment scheme according to whether the queue is continuously increased at the moment; and when a subsequent data packet needs to be transmitted in a speed-up manner, the speed-up step length is dynamically adjusted. Compared with an existing algorithm which only adjusts the transmission rate, the method has the advantages that the network congestion can be processed more quickly, the performance loss caused by excessive response or slow response is reduced, the convergence speed is improved, and the method can be better applied to a high-speed data center network environment.

Description

Technical Field

[0001] The present invention relates to the field of data center network transmission, and particularly to a congestion control method based on delay. Background Art

[0002] Congestion control is one of the core technologies of high-performance data center networks. Modern data center networks usually have the characteristics of high bandwidth (such as 400G / 800G ports), low latency (in the order of microseconds), and ultra-large scale (interconnection of tens of thousands of nodes), and carry diverse traffic such as real-time video stream processing, distributed storage systems, and large-scale artificial intelligence training parameter synchronization. The traffic in these business scenarios often has the characteristic of suddenness. For example, in distributed machine learning training, the parameter server may receive gradient update requests from hundreds of worker nodes simultaneously, forming an Incast traffic storm.

[0003] Currently, existing congestion control schemes aim to improve performance through more accurate congestion signals and control laws. For example, through network switch devices that support In-Network Telemetry (INT) technology, congestion control can obtain the cumulative amount of data sent by each switch port and the current queue length when a data packet passes through each switch port. Compared with the traditional single-bit ECN marking method, INT technology can provide more fine-grained network status information.

[0004] However, the existing congestion control methods control the transmission in an inflexible manner and are difficult to cope with the diverse application requirements and dynamic traffic patterns in the data center, manifested as overreacting or reacting sluggishly when dealing with congestion. Overreacting will cause the bandwidth not to be fully utilized after dealing with congestion, thereby affecting the completion time of long flows; reacting sluggishly will cause congestion to last longer, and the network will have a higher average queue, which will seriously affect the performance of delay-sensitive type traffic, especially short flows. Summary of the Invention

[0005] In order to solve the problems in the background art, the present invention proposes a congestion control method based on delay, which realizes more flexible data packet transmission control to improve the problem of overreacting or reacting sluggishly when dealing with congestion, improves the convergence speed, and processes network congestion faster. The present invention includes the following steps:

[0006] (1) The sender device sends the current data packet group at the current rate R and time T. The sender device records the sequence numbers and sending times of the data packets in the data packet group, and sets the required acknowledgment packet sequence number expseq as the sequence number of the first data packet in the current data packet group;

[0007] (2) Forward each data packet in the data packet group to the receiving device. For each data packet received by the receiving device, return an acknowledgment message with the same sequence number as the data packet to the sending device;

[0008] (3) When the sending device receives the acknowledgment message with the sequence number expseq in the data packet group, determine the reception time, and determine the round-trip delay based on the transmission time T s of the data packet corresponding to the current acknowledgment message and the reception time, and calculate the total queuing delay Q during the forwarding process based on the round-trip delay;

[0009] (4) Determine whether there is congestion based on the total queuing delay Q during the forwarding process to obtain the congestion result;

[0010] (5) According to the congestion result, the current rate R, the number of unacknowledged data packets in the current data packet group, and whether the current data packet group is waiting for transmission adjustment, when the following three conditions (a), (b), and (c) are all satisfied, still wait for the acknowledgment message of the unacknowledged data packet and execute step (7); otherwise, directly adjust the transmission of the subsequent data packet group and execute step (6):

[0011] In step (5), the three conditions are (a), (b), and (c):

[0012] (a) The number of unacknowledged data packets in the current data packet group is greater than or equal to 2;

[0013] (b) The congestion result holds, or the congestion result does not hold but the current rate R exceeds the target rate target;

[0014] (c) The current data packet group is not waiting for transmission adjustment;

[0015] (6) Adjust the start time T next and the rate R next of the subsequent data packet according to the congestion result obtained in (4), return to step (1), and the sending device updates T slast to the transmission time T of the data packet corresponding to the current acknowledgment message s , update Q last to the total queuing delay Q, and the sending device sends the next data packet group at the rate R next and the time T next ;

[0016] (7) In the acknowledgment message corresponding to the current data packet group, set the sequence number expseq of the next required acknowledgment message. When the acknowledgment message with the sequence number expseq is received, return to step (3), update the acknowledgment message of the first data packet to the acknowledgment message with expseq, and at the same time the sending device updates T slast to the transmission time T of the corresponding data packet s , update Qlast is the total queuing delay Q.

[0017] The main idea of the present invention is to introduce T next to solve the conflict problem between throughput and delay performance when dealing with congestion. T next is set to the time when the current congestion is expected to end. After T next , the next packet group is sent. Theoretically, it is ensured that the next packet group will not further extend the duration of the current congestion, allowing the current congestion to be alleviated at the fastest speed. After T next , sending according to R next guarantees high throughput in the subsequent network. In summary, by simultaneously controlling the sending rate R next and the sending time T next of each packet group, the present invention achieves high throughput and low delay simultaneously when dealing with congestion, and optimizes the performance loss in the existing scheme.

[0018] In step (4), it is determined whether there is congestion according to the total queuing delay Q in the forwarding process, specifically including:

[0019] When the following formula (1) is satisfied, it is determined that there is congestion;

[0020]

[0021] where R is the current rate, C is the link rate, and Q th is 25% of the round-trip delay without queuing.

[0022] In step (6), the start sending time T next and the rate R next of the subsequent packets are adjusted according to the congestion result obtained in (4), specifically including:

[0023] (6.1) If the congestion result does not hold, the start sending time T next and the rate R next of the subsequent packets are adjusted according to formulas (2) and (3):

[0024] T next = max(now, T latest + MTU / R next ) (2)

[0025]

[0026] where T latest is the sending time of the last packet sent in the current packet group, now is the current time, MTU is the packet length, target is the target rate, num is the current consecutive speed increase times, inc_step = 0.02% C is the speed increase step based on the benchmark, and R and Rlast They are the current rate and the rate at the last update respectively;

[0027] (6.2) If the congestion result holds, calculate the delay gradient G based on the total queuing delay in the forwarding process, the transmission time of the data packet, the total queuing delay in the forwarding process calculated last time, and the transmission time of the corresponding data packet at the last calculation; according to the delay gradient G, determine whether the current queue is continuously growing, and according to whether the current queue is continuously growing, set the target rate target to the current rate R and adjust the start time T of the subsequent data packets to be sent next and the rate R next .

[0028] In step (6.2), the delay gradient G is calculated according to the following formula (4):

[0029] G = (Q - Q last ) / (T s - T slast ) (4)

[0030] where Q is the total queuing delay, Q last is the total queuing delay in the forwarding process calculated last time, T s is the transmission time of the data packet corresponding to the current acknowledgment message, and T slast is the transmission time of the corresponding data packet at the last calculation.

[0031] In step (6.2), according to the delay gradient G, determine whether the current queue is continuously growing, specifically including:

[0032] When the following two conditions are both satisfied, it is determined that the current queue is continuously growing:

[0033] (d) The number of unacknowledged data packets in the current data packet group is greater than or equal to 2;

[0034] (e) The delay gradient G ≥ 0.9 or the current transmission rate R ≥ C / 8, where C is the link rate.

[0035] In step (6.2), according to whether the current queue is continuously growing, set the target rate target to the current rate R and adjust the start time T of the subsequent data packets to be sent next and the rate R next , specifically including:

[0036] (6.2.1) If the current queue is continuously growing, adjust the start time T of the subsequent data packets to be sent according to formulas (5) and (6) next and the rate R next :

[0037] T next = T latest+Q+G(T latest -T s ) (5)

[0038] R next = 0.95R / (1 + G) (6)

[0039] where Q is the total queuing delay, G is the delay gradient, T latest is the transmission time of the last packet sent in the current packet group, T s is the transmission time of the packet corresponding to the current acknowledgment message, and R is the current rate;

[0040] (6.2.2) If the current queue does not grow continuously, adjust the start transmission time T next of the subsequent packets and the rate R next according to formulas (7) and (8):

[0041]

[0042] R next = 0.95R (8).

[0043] where T latest is the transmission time of the last packet sent in the current packet group, Q is the total queuing delay, Q last is the total queuing delay in the previous calculated forwarding process, Q th is 25% of the round-trip delay without queuing, and R is the current rate.

[0044] In step (7), expseq is set according to the following principle:

[0045] (7.1) If the current rate is the link rate, expseq is the sequence number of the next packet in the packet group; otherwise, expseq is the average of the current sequence number and the sequence number of the first packet in the next packet group.

[0046] Compared with the prior art, the present invention has the following advantages:

[0047] The present invention designs a more fine-grained congestion control method based on delay. Compared with the existing solutions that only adjust the transmission rate, the present invention adjusts the subsequent transmission more flexibly, adjusts both the start transmission time and the rate of the subsequent packets simultaneously, can handle congestion in high-speed networks above 100 Gbps faster, optimizes the performance loss during congestion handling, and can meet the different performance requirements of various applications in the data center.

[0048] Moreover, the solution of the present invention does not require new switch features, only requires very little space and computational overhead, and can improve the performance loss during congestion handling. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is the algorithm flowchart of the method of the present invention.

[0050] Figure 2 It is a schematic diagram of an embodiment in the synchronization of artificial intelligence training parameters of the present invention.

[0051] Figure 3 It is in the present invention Figure 2 In the scenario, compared with not using T next The experimental results of network throughput. Specific implementation manners

[0052] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0053] The present invention proposes a congestion control method based on delay, including the following steps:

[0054] The present invention includes the following steps:

[0055] (1) The sender device sends the current data packet group at the current rate R and time T. The sender device records the sequence numbers and sending times of the data packets in the data packet group, and sets the required acknowledgment packet sequence number expseq as the sequence number of the first data packet in the current data packet group;

[0056] (2) Forward each data packet in the data packet group to the receiver device. Each time the receiver device receives a data packet, it returns an acknowledgment packet consistent with the data packet sequence number to the sender device;

[0057] (3) When the sender device receives the acknowledgment packet with the sequence number expseq in the data packet group, it determines the reception time, calculates the round-trip delay according to the sending time T s of the data packet corresponding to the current acknowledgment packet and the reception time, and calculates the total queuing delay Q in the forwarding process according to the round-trip delay;

[0058] (4) Determine whether there is congestion according to the total queuing delay Q in the forwarding process, and obtain the congestion result;

[0059] (5) According to the congestion result, the current rate R, the number of unacknowledged data packets in the current data packet group, and whether the current data packet group is waiting for transmission adjustment, when the following three conditions (a), (b), and (c) are simultaneously satisfied, still wait for the acknowledgment packet of the unacknowledged data packet, and execute step (7), otherwise directly adjust the transmission of the subsequent data packet group and execute step (6):

[0060] In step (5), the three conditions are (a), (b), and (c):

[0061] (a) The number of unacknowledged data packets in the current data packet group is greater than or equal to 2;

[0062] (b) The congestion result holds, or the congestion result does not hold but the current rate R exceeds the target rate target;

[0063] (c) The current data packet group is not waiting for transmission adjustment;

[0064] (6) Adjust the start time T of subsequent data packets according to the congestion result obtained in (4) next and the rate R next , go back to step (1), and the sender device updates T slast to the transmission time T of the data packet corresponding to the current acknowledgment message s , update Q last to Q, and the sender device sends the next data packet group at the rate R next and the time T next ;

[0065] (7) In the acknowledgment message corresponding to the current data packet group, set the sequence number expseq of the next required acknowledgment message. When the acknowledgment message with the sequence number expseq is received, go back to step (3), update the acknowledgment message of the first data packet to the acknowledgment message of expseq, and at the same time the sender device updates T slast to the transmission time T of the data packet corresponding to the current acknowledgment message s , update Q last to Q.

[0066] Compared with the existing method that only adjusts the transmission rate of subsequent data packets, the present invention realizes more flexible data packet transmission control by adjusting both the start time and rate of subsequent data packets to improve the problems of overreaction or slow reaction when dealing with congestion.

[0067] Figure 2 An embodiment of the present invention in artificial intelligence training parameter synchronization is given. The sender includes 8 worker nodes, and the receiver is a parameter server. After each node completes the gradient calculation of this round, it needs to push 200MB of gradient tensors to the parameter server simultaneously. The worker nodes first send the first data packet group at the initial link rate. The number of data packets in the first data packet group is the bandwidth-delay product, about several hundred KB. After the parameter server returns the acknowledgment message, the congestion control method described in the present invention is executed. Specifically, each worker node executes the congestion control method described in the present invention according to the Figure 1 algorithm flow shown to transmit subsequent data packet groups until the gradient parameter transmission ends.

[0068] As Figure 1As described above, first, when receiving an acknowledgment message, it is determined whether it is expseq. If so, the end-to-end delay and queue delay are measured, and based on the current delay threshold, it is determined whether congestion occurs at this time. Subsequently, it is determined whether to wait. This condition specifically includes the following three parts: ① The number of unacknowledged packets is not less than 2; ② The congestion determination condition does not hold and the condition of R < target are not both satisfied; ③ When receiving the acknowledgment message with the sequence number expseq last time, the start time and rate of sending subsequent packets are adjusted, which is recorded as the flag in Figure 2 . Among them, the first condition is used to directly adjust according to the congestion situation in a timely manner without the need to pause and wait for the second measurement additionally; the second condition indicates whether it is the previous speed increase. If so, there is no need to pause and wait for the second measurement additionally. Otherwise, it will slow down the speed increase instead, reducing the throughput. Only when all three conditions are satisfied, expseq is directly updated and waiting for the second measurement. And if there is congestion, the transmission is immediately temporarily paused.

[0069] When receiving the second measurement result or not needing to wait, it is further determined whether there is congestion at this time. When there is congestion, it is determined whether it is a scenario of continuous queue growth based on whether the current window is greater than 2 packets and whether one of the conditions of G ≥ 0.9 or R ≥ C / 8 is satisfied, so as to decide which congestion response method to use: If it is a scenario of continuous queue growth, the start time and rate of sending subsequent packets are adjusted according to formulas (5) and (6). Otherwise, they are adjusted according to formulas (7) and (8). Both methods are designed according to the following principles: On the one hand, the setting of the sending time should ensure that when the subsequent packets arrive at the queue, all the packets accumulated in the queue during the current congestion event are just sent out, that is, the network is no longer congested at this time, and the sending of subsequent packets is paused before that. This can theoretically eliminate congestion fastest because from the determination of congestion to the set transmission time, the congestion response module only needs to pause packet sending, and the queue length can decrease at full speed; On the other hand, the rate of starting to send subsequent packets can ensure that the sum of unacknowledged packets of each flow matches the network capacity. Combining the start time, theoretically, after the current congestion ends, the network can achieve high throughput and no longer be congested at the same time.

[0070] In addition, when there is no congestion, it speeds up rapidly with a variable step size according to formulas (2) and (3). The specific principle is as follows: At the beginning of speeding up, the current rate is less than the rate before deceleration. At this time, it is less likely to have congestion, and the step size of each speed increase can be increased aggressively; when it exceeds the rate before deceleration, the step size should increase in a conservative strategy to avoid excessive congestion caused by too large a step size. In addition, if there is a large amount of idle bandwidth due to reasons such as flow leaving at this time, the fast speed-up module needs to switch to a more aggressive speed-up mode after speeding up for a period of time to make full use of the bandwidth.

[0071] Finally, after setting the time and rate for the subsequent data packets to start sending, modify the flag and also update expseq. The number of data packets in the next packet group, i.e., the congestion window, is determined by multiplying the current rate R by the round-trip delay baseRTT without queuing.

[0072] To verify the effectiveness of the present invention, Figure 3 the experimental results of the network throughput of the present invention in Figure 2 the scenario compared with not using T next are given. The network interface rates shown in the figure are all 200 Gbps. The network throughput is the forwarding rate of the last-hop switch connected to the receiving terminal device per unit time. The greater the throughput loss, the longer the transmission time. The results show that when not using T next only adjusts the sending rate of the subsequent data packets, that is, when adopting a similar method to the existing method, an overreaction phenomenon will occur, and there will be a throughput loss at the end of the current congestion event, thus affecting the transmission time.

[0073] In contrast, the present invention realizes more flexible data packet transmission control and better performance by simultaneously adjusting the time and rate for the subsequent data packets to start sending, without the overreaction phenomenon, and can handle network congestion faster. In addition, the present invention also has good generality and deployment flexibility and does not rely on new network technologies dedicated to specific devices such as INT. In summary, the present invention can be better applied to the high-speed data center network environment.

[0074] The above specific embodiments are used to explain and illustrate the present invention, rather than limiting the present invention. Any modifications and changes made to the present invention within the spirit and scope of the claims of the present invention fall within the protection scope of the present invention.

Claims

1. A delay-based congestion control method, characterized in that: The following steps are involved: (1) The sending device sends the current data packet group at the current rate R and time T. The sending device records the sequence number and sending time of each data packet in the data packet group, and sets the required confirmation message sequence number expseq to the sequence number of the first data packet in the current data packet group; (2) forwarding each data packet in the data packet group to the receiving device. Each time the receiving device receives a data packet, it returns a confirmation message consistent with the sequence number of the data packet to the sending device; (3) When the sending device receives the confirmation message with the sequence number expseq in the data packet group, it determines the receiving time and the sending time T of the data packet corresponding to the current confirmation message. s The round trip delay is determined by the receiving time, and the total queuing delay Q in the forwarding process is calculated based on the round trip delay; (4) Determine whether there is congestion based on the total queuing delay Q in the forwarding process and obtain the congestion result; (5) According to the congestion result, the current rate R, the number of unconfirmed data packets in the current data packet group, and whether the current data packet group is waiting for transmission adjustment, when the following three conditions (a), (b) and (c) are met at the same time, it is still necessary to wait for the confirmation message of the unconfirmed data packet, and execute step (7); otherwise, directly adjust the transmission of the subsequent data packet group and execute step (6): In step (5), the three conditions are (a), (b) and (c): (a) The number of unconfirmed packets in the current packet group is greater than or equal to 2; (b) The congestion result is established, or the congestion result is not established but the current rate R exceeds the target rate target; (c) The current packet group is not waiting for transmission adjustment; (6) According to the congestion result obtained in (4), adjust the time T at which subsequent data packets start to be sent next and rate R next , return to step (1), the sending device updates the sending time of the last calculated queuing delay packet The sending time T of the data packet corresponding to the current confirmation message s , update the total queuing delay Q in the forwarding process calculated last time last is the current total queuing delay Q, and then according to the rate R next and time T next Send the next packet group; (7) In the confirmation message corresponding to the current data packet group, set the sequence number expseq of the next required confirmation message. When receiving the confirmation message with the sequence number expseq, return to step (3) and the sending device updates The sending time T of the data packet corresponding to the current confirmation message s , update Q last is the current total queuing delay Q.

2. The delay-based congestion control method according to claim 1, characterized in that: In step (4), whether there is congestion is determined based on the total queuing delay Q in the forwarding process, which specifically includes: When the following formula (1) is satisfied, congestion is determined; Where R is the current rate, C is the link rate, Q th It is 25% of the round-trip delay when there is no queuing.

3. The delay-based congestion control method according to claim 1, characterized in that: In step (6), the time T at which subsequent data packets start to be sent is adjusted according to the congestion result obtained in step (4). next and rate R next , specifically including: (6.1) If the congestion result is not established, adjust the time T at which subsequent data packets start to be sent according to formulas (2) and (3) next and rate R next : T next =max(now,T latest +MTU / R next ) (2) Where T latest is the sending time of the last data packet sent in the current data packet group, now is the current time, MTU is the data packet length, target is the target rate, num is the current number of continuous speed increases, inc_step = 0.02% C is the base speed increase step, R and R last They are the current rate and the rate of the last update respectively; (6.2) If the congestion result is established, calculate the delay gradient G according to the total queuing delay in the forwarding process, the sending time of the data packet, the total queuing delay in the forwarding process calculated last time, and the sending time of the corresponding data packet in the last calculation; according to the delay gradient G, determine whether the current queue continues to grow. According to whether the current queue continues to grow, set the target rate target to the current rate R, and adjust the time T when the subsequent data packets start to be sent. next and rate R next .

4. The delay-based congestion control method according to claim 1, characterized in that: In step (6.2), the time delay gradient G is calculated according to the following formula (4): Among them, Q last is the total queuing delay in the forwarding process calculated last time, It is the sending time of the corresponding data packet in the last calculation.

5. The delay-based congestion control method according to claim 1, characterized in that: In step (6.2), based on the delay gradient G, it is determined whether the current queue continues to grow, specifically including: When the following two conditions are met at the same time, the current queue is determined to be continuously growing: (d) The number of unconfirmed data packets in the current data packet group is greater than or equal to 2; (e)Satisfy G ≥ 0.9 or R ≥ C / 8.

6. The delay-based congestion control method according to claim 1, characterized in that: In step (6.2), according to whether the current queue continues to grow, the target rate target is set to the current rate R, and the time T at which subsequent data packets start to be sent is adjusted. next and rate R next , specifically including: (6.2.1) If the current queue is growing continuously, adjust the time T at which subsequent data packets start to be sent according to formulas (5) and (6) next and rate R next : T next =T latest +Q+G(T latest -T s ) (5)R next =0.95R / (1+G) (6) Among them, T latest is the sending time of the last data packet sent in the current data packet group, T s The sending time of the data packet corresponding to the current confirmation message; (6.2.2) If the current queue is not growing continuously, adjust the time T at which the subsequent data packets start to be sent according to formulas (7) and (8) next and rate R next : R next =0.95R (8)。 7. The delay-based congestion control method according to claim 1, characterized in that: In step (7), expseq is set according to the following principles: (7.1) If the current rate is the link rate, expseq is the sequence number of the next packet in the packet group; otherwise, expseq is the average of the current sequence number and the sequence number of the first packet in the next packet group.

Images