Congestion management method based on network congestion notification and rate adjustment strategy

Abstract

The application relates to the technical field of data transmission, and discloses a congestion management method based on a novel network congestion announcement and rate adjustment strategy, which comprises the following steps: S1: constructing a flow statistical system for counting receiving rates and generating accurate congestion announcement data packets at a receiving end; S2: judging whether accurate congestion announcement data packets are generated for a specified queue pair according to statistical information of the system, and sending the accurate congestion announcement data packets to a specified sending end; and S3: constructing a rate adjustment system at the sending end, wherein the flow statistical system comprises a queue pair rate statistical module and an RDMA rate statistical module, and each RDMA network card port needs to contain an RDMA rate statistical module. In the application, the calculated statistical rate is carried in a 16-byte reserved field of a standard congestion announcement data packet, which is compatible with an existing RoCEv2 congestion management architecture, does not need an additional detection marking function provided by a data center switch, and is easy to deploy and expand.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data transmission, and particularly relates to a congestion management method based on network congestion notification and rate adjustment strategy. BACKGROUND

[0002] The traditional TCP / IP network stack cannot meet the requirements of high bandwidth, low latency and low CPU utilization of data center networks, and the RDMA (Remote Direct Memory Access) technology supports the direct access of a local host to a remote host memory, and the data access operation is only executed by the network card (NIC) hardware without the intervention of the host system kernel, which avoids additional data copy operations, so the RDMA technology has the characteristics of kernel bypass and zero copy, and meets the requirements of high-speed networks of data centers. In order to deploy the RDMA network on the existing Ethernet architecture, the large cloud vendors mainly use the RoCEv2 (RDMA on Converged Ethernet Version 2) technology in the data center, and the RoCEv2 technology needs to use the priority-based flow control protocol (PFC) to ensure zero packet loss of RDMA data transmission, and the PFC protocol will cause performance problems such as head-of-line blocking and congestion diffusion, which need to be solved by an effective congestion control mechanism.

[0003] The commercial RDMA network card in the existing data center mainly uses the DCQCN protocol as the solution of RoCEv2 congestion control, which is based on the standard RoCEv2 congestion management (RCM) framework, uses ECN (Explicit Congestion Notification) as the congestion signal, and is prone to frequent triggering of PFC in large-scale operation, resulting in poor overall network throughput and average latency, and the latest RoCEv2 congestion control schemes such as HPCC, PCN and the like are incompatible with most existing data center RCM architectures, and need to provide additional detection marking functions of the switch.

[0004] Therefore, there is an urgent need for a congestion management method based on network congestion notification and rate adjustment strategy. SUMMARY

[0005] The present application aims to provide a congestion management method based on network congestion notification and rate adjustment strategy, which solves the problems proposed in the background.

[0006] To achieve the above purpose, the present application provides the following technical scheme: a congestion management method based on network congestion notification and rate adjustment strategy, comprising the following steps:

[0007] S1: constructing a flow statistics system for statistically receiving a rate and generating an accurate congestion notification data packet at a receiving end;

[0008] S2: The receiving end determines whether to generate a precise congestion notification data packet for a specified queue pair according to the statistical information of the system, and sends the precise congestion notification data packet to the specified sending end.

[0009] S3: A rate adjustment system is constructed at the sending end.

[0010] Preferably, the traffic statistical system comprises a queue pair rate statistical module and an RDMA rate statistical module.

[0011] Preferably, each RDMA network card port comprises an RDMA rate statistical module.

[0012] Preferably, the queue pair rate statistical module comprises a plurality of sub queue pair rate statistical modules.

[0013] Preferably, each active RDMA queue pair corresponds to a unique sub queue pair rate statistical module.

[0014] Preferably, the rate adjustment system comprises a unique sub rate adjustment module corresponding to each active queue pair.

[0015] Preferably, the sub rate adjustment module is responsible for controlling the sending rate of the specified queue pair.

[0016] The present application provides a congestion management method based on network congestion notification and rate adjustment strategy.

[0017] (1) The traffic statistical system of the receiving end can calculate the RDMA total receiving rate and the receiving rate of the specified queue pair, and the sending end can use the statistical rate to achieve more precise rate adjustment.

[0018] (2) The rate adjustment of the sending end is only based on the statistical rate of the receiving end, and there is no fixed rate increment, which can make the rate adjustment of the sending end more rapid and accurate, and the fluctuation is smaller.

[0019] (3) The receiving end carries the calculated statistical rate in the 16-byte reserved field of the standard congestion notification data packet, which is compatible with the existing RoCEv2 congestion management architecture, does not require additional detection marking function of the data center switch, and is easy to deploy and expand. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 : The overall congestion management architecture of the present application;

[0021] Figure 2 : The precise congestion notification data packet format of the present application;

[0022] Figure 3 Queue pair rate statistics module flow chart of the receiving end flow statistics system of the application;

[0023] Figure 4 Sub queue pair rate statistics module flow chart of the receiving end flow statistics system of the application;

[0024] Figure 5 RDMA rate statistics module flow chart of the receiving end flow statistics system of the application;

[0025] Figure 6 Sending end rate adjustment system flow chart of the application;

[0026] Figure 7 Sub rate adjustment module flow chart of the sending end rate adjustment system of the application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the application will be apparently and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application.

[0028] Examples of the described embodiments are shown in the drawings, in which the same or similar notations represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by reference to the drawings are exemplary and are intended to explain the application, and cannot be understood as a limitation of the application.

[0029] The application provides a technical solution: a congestion management method based on network congestion notification and rate adjustment strategy, comprising the following steps:

[0030] S1: constructing a flow statistics system for statistically receiving rate and generating accurate congestion notification data packets at a receiving end, the flow statistics system comprising a queue pair (Queue Pair) rate statistics module and an RDMA rate statistics module, each RDMA network card port comprising an RDMA rate statistics module, and the queue pair (Queue Pair) rate statistics module comprising a plurality of sub queue pair rate statistics modules, each active RDMA queue pair uniquely corresponding to a sub queue pair rate statistics module;

[0031] 1.1, as shown in Figure 5 When a RoCEv2 data packet enters the receiving end flow statistics system, it first enters the RDMA rate statistics module, which is processed by the queue pair rate statistics module, and finally processed by the RDMA rate statistics module;

[0032] 1.2、RDMA rate statistics module mainly maintains two statistics items, RDMA total byte statistics item ALL_Bytes and queue pair connection number statistics item Num_of_QP, and all sub-queue pair rate statistics modules can obtain the values of the above statistics items. When the RDMA rate statistics module is started or restarted, ALL_Bytes and Num_of_QP are set to 0;

[0033] 1.3、When a RoCEv2 data packet finally enters the RDMA rate statistics module, update the RDMA total byte statistics item ALL_Bytes = ALL_Bytes + Pkt.Size, where Pkt.Size is the packet size of the triggering data packet; if the data packet is the first data packet of the queue pair, increase the queue pair connection number statistics item Num_of_QP = Num_of_QP + 1; if the data packet is the last data packet of the queue pair, decrease the queue pair connection number statistics item Num_of_QP = Num_of_QP - 1;

[0034] 1.4、As shown in Figure 3 , when a RoCEv2 data packet enters the queue pair rate statistics module, first, the queue pair rate statistics module obtains the corresponding source queue pair number according to the header information of the data packet, and checks whether there is a sub-queue pair rate statistics module corresponding to the source queue pair number. If there is, the data packet is passed to the sub-module for processing; if not, a new sub-queue pair rate statistics module needs to be created for the source queue pair number;

[0035] 1.5、As shown in Figure 4 , when a RoCEv2 data packet triggers the creation of a new sub-queue pair rate statistics module, the statistical parameter values of the new module need to be initialized according to the RoCEv2 data packet, including the received data packet count value Pkt_Count, the initial value of which is 1; the received data packet ECN count value Pkt_Ecn, which is set to 1 when the two-bit IP ECN field of the RoCEv2 data packet is 0b11, otherwise Pkt_Ecn is set to 0; the received byte count Pkt_Size, the initial value of which is the packet size of the triggering data packet; the statistical time count value Num_of_T, the initial value of which is 1; the previous RDMA total byte count Pre_AllBytes, the initial value of which is the value of the RDMA total byte statistics item ALL_Bytes when the sub-module is created. After the initialization of the sub-module parameters is completed, the timer NPTimer of the sub-module is reset, and the timing period is T, the default value of which is pre-configured by the user;

[0036] 1.6、When a RoCEv2 packet enters the initialized subqueue pair rate statistics module, the statistics of the subqueue pair rate statistics module are updated at this time, including the received packet count value update Pkt_Count = Pkt_Count + 1; the received packet ECN count value update Pkt_Ecn = Pkt_Ecn + (Pkt.Ecn == 0b11?1:0), wherein Pkt.Ecn is the two-bit IP ECN value of the triggering packet; the received byte number update Pkt_Size = Pkt_Size + Pkt.Size, wherein Pkt.Size represents the packet size of the triggering packet;

[0037] S2: The receiving end determines whether to generate a precise congestion notification packet for a specified queue pair according to the statistical information of the system, and sends it to the specified sending end;

[0038] 2.1、As shown in Figure 4 , when the NPTimer timer of a subqueue pair rate statistics module in the traffic statistics system expires, it is detected whether the Pkt_Count statistical value of the sub-module is greater than 0, if Pkt_Count is greater than 0, it is considered that the sub-module has received a packet in the statistical period, otherwise it is considered that the sub-module has not received a packet;

[0039] 2.2、When the subqueue pair rate statistics module does not receive a packet in the statistical period, the statistical time count value Num_of_T = Num_of_T + 1 of the sub-module is updated, and the timer NPTimer of the sub-module is reset, and the timing period is still T; 2.3、When the subqueue pair rate statistics module receives a packet in the statistical period, the proportion of packets with ECN marks in the received packets in the statistical period is calculated ECNRadio = Pkt_Ecn / Pkt_Count, when the ECNRadio value is greater than MaxRadio, it is considered that the queue pair corresponding to the sub-module has occurred congestion, otherwise it is considered that the corresponding queue pair has not occurred congestion, wherein MaxRadio represents the maximum congestion ratio, which is pre-configured by the user;

[0040] 2.4、When the queue pair corresponding to the subqueue pair rate statistics module occurs congestion, the target rate TarRate = B / Num_of_QP is calculated, wherein B represents the bandwidth of the RDMA network card port corresponding to the sub-module, the receiving rate RecRate = Pkt_Size / (Num_of_T*T), and the remaining receiving rate ResRate is set to 0;

[0041] 2.5、When the queue pair corresponding to the sub queue pair rate statistics module is not congested, first, the target rate TarRate = B / Num_of_QP is calculated, and the total receiving rate AllRecRate = (ALL_Bytes-Pre_AllBytes) / (Num_of_T*T) is also calculated, then the remaining receiving rate ResRate = B-AllRecRate, wherein B represents the bandwidth of the RDMA network card port corresponding to the sub module, and the receiving rate RecRate is set to 0;

[0042] 2.6、After the three statistical rates-target rate TarRate, receiving rate RecRate, and remaining receiving rate ResRate are calculated, the sub queue pair rate statistics module needs to generate a precise congestion notification data packet for the queue pair corresponding to it, and fill the 16-byte reserved field with the three statistical rate values in turn, as shown in the format Figure 2 , each statistical rate occupies 4 bytes. If the corresponding queue pair is congested, the IP ECN field of the precise congestion notification data packet is set to 0b11, otherwise it is set to 0b10; finally, the precise congestion notification data packet is sent to the specified sending end;

[0043] 2.7、After sending the precise congestion notification data packet, the statistical items in the sub queue pair rate statistics module are reset, Pkt_Count, Pkt_Ecn and Pkt_Size are all set to 0, Num_of_T is set to 1, and Pre_AllBytes is set to the value of ALL_Bytes when the precise congestion notification data packet is generated;

[0044] S3: constructing a rate adjustment system at the sending end;

[0045] 3.1、As shown in Figure 6 , whenever the rate adjustment system at the sending end receives a precise congestion notification data packet, first, the destination queue pair number contained in the data packet header information is obtained, and it is checked whether there is a sub rate adjustment module corresponding to the destination queue pair number, if there is, the data packet is handed over to the corresponding sub rate adjustment module for processing; if not, a new sub rate adjustment module needs to be created for the destination queue pair number;

[0046] 3.2、As shown in Figure 7 , each sub rate adjustment module first sets the sending rate SendRate of the specified queue pair to the bandwidth of the network card port where the queue pair is located during the initialization stage; it also needs to save the receiving end network card port bandwidth Brec, which is actively informed by the receiving end to the sending end during the early connection establishment process; finally, the current speed-up stage Incstage is set to 0, which is used for subsequent speed-up stages;

[0047] 3.3 When the sub-rate adjustment module receives a precise congestion notice data packet, it first needs to extract the value of the ECN field from the IP header field of the data packet, and then extract three statistical rates from the reserved field of the data packet: target rate TarRate, receive rate RecRate, and remaining receive rate ResRate. Then, it executes the following algorithm, the specific method of which is as follows:

[0048] 3.3.1 Calculate the number of receiver queue pairs N = Brec / TarRate;

[0049] 3.3.2 When the ECN field value of the Precise Congestion Notification data packet is 0b11, the logic for the deceleration phase is entered, proceed to 3.3.3; otherwise, the logic for the acceleration phase is entered, proceed to 3.5.7.

[0050] 3.3.3 Calculate the target deceleration rate DecTar = min{RecRate, TarRate};

[0051] 3.3.4 Calculate the reduction factor w = tanh(N / P), where P is the scaling factor, which is pre-configured by the user;

[0052] 3.3.5 Calculate the new transmission rate SendRate = min{DecTar*(1-w),SendRate};

[0053] 3.3.6 Set the current growth stage Incstage to 0 and proceed to 3.5.14;

[0054] 3.5.7 Calculate the target growth rate TarVal = (SendRate) <TarRate?TarRate-SendRate,0);

[0055] 3.5.8 Calculate the growth rate IncTar = min{TarVal, ResRate};

[0056] 3.5.9 If the current growth stage Incstage is less than MaxStage, proceed to 3.5.10; otherwise, proceed to 3.5.12. MaxStage is the maximum fair growth stage value, which is pre-configured by the user.

[0057] 3.5.10 Calculate the new transmission rate SendRate = SendRate + min{IncTar / N, SendRate};

[0058] 3.5.11 Update the current growth stage Incstage = Incstage + 1, and go to 3.5.14;

[0059] 3.5.12, calculate a new sending rate SendRate = SendRate + min{IncTar, SendRate};

[0060] 3.5.13, reset the current speed-up stage Incstage = 0, and go to 3.5.14;

[0061] 3.5.14, update the sending rate of the specified queue pair to the calculated SendRate, and return to 3.3.

[0062] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, alternatives and variations can be made to the embodiments without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

[0063] The above description is only the preferred embodiment of the present application, but the protection scope of the present application is not limited to this, any person skilled in the art can make equivalent replacements or changes according to the technical solution and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A congestion management method based on network congestion announcement and rate adjustment strategies, characterized in that: Includes the following steps: S1: Build a traffic statistics system at the receiving end to count the receiving rate and generate accurate congestion notification packets; S2: Based on the statistical information of the above system, the receiving end determines whether it is a specified queue pair, generates a precise congestion notification data packet, and sends it to the specified sending end; S3: Build a rate adjustment system at the transmitting end; The traffic statistics system includes a queue pair rate statistics module and an RDMA rate statistics module; The queue pair rate statistics module contains multiple sub-queue pair rate statistics modules; After calculating the target rate TarRate, the receiving rate RecRate, and the remaining receiving rate ResRate, the sub-queue pair rate statistics module needs to generate a precise congestion notification data packet for its corresponding queue pair and fill in the three statistical rate values ​​in sequence in its 16-byte reserved field.

2. The congestion management method based on network congestion notification and rate adjustment strategy according to claim 1, characterized in that: Each RDMA network card port must include an RDMA rate statistics module.

3. The congestion management method based on network congestion notification and rate adjustment strategy according to claim 2, characterized in that: Each active RDMA queue pair must uniquely correspond to a sub-queue pair rate statistics module.

4. The congestion management method based on network congestion notification and rate adjustment strategy according to claim 1, characterized in that: The rate adjustment system has a unique sub-rate adjustment module for each active queue pair.

5. The congestion management method based on network congestion notification and rate adjustment strategy according to claim 4, characterized in that: The sub-rate adjustment module is responsible for controlling the transmission rate of a specified queue pair.

Images