Switch cache management policy for data center network hybrid flows

Abstract

The application relates to the field of computer networks, in particular to a switch cache management strategy for a data center network hybrid flow, which comprises a flow statistics module, a cache pool division module and a threshold calculation module; the flow statistics module is used for monitoring the queue transmission rate and length of each type of flow in the switch, the TCP flow comprises TCP packets, the RDMA flow comprises RDMA, PFC and RESUME packets, the queue length of the flow statistics collection is continuously compared with a long queue threshold defined by the product of the basic bandwidth delay of each flow and 2, the switch RDMA and TCP queues are judged whether in a continuous long queue state, the running state of each queue in the switch is collected in real time, and the data is respectively sent to the cache pool division module and the threshold calculation module; under the premise of not affecting the global throughput of the data center network, the absorption capacity of the switch cache for small flows and burst flows is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer networks, and in particular to a switch buffer management policy for data center network mixed flows. BACKGROUND

[0002] In recent years, the ultra-low latency and high bandwidth requirements of data center applications on the network are increasingly high. In order to meet the growing SLA requirements and reduce kernel overhead, data center providers such as Amazon, Microsoft, Alibaba, etc. are gradually using remote direct memory access (RDMA) to offload the traffic transmission of specific services from the kernel to the network card. Although RDMA has developed very rapidly and has shown very excellent performance, the kernel-based TCP protocol stack is still widely used for traditional applications, inter-data center transmission, and as a backup solution for RDMA transmission. According to Microsoft's large-scale traffic statistics on its data center, about 70% of the total traffic is RDMA traffic and the other is TCP traffic. Therefore, the coexistence of RDMA and TCP in data centers is widespread.

[0003] Currently, the buffer management of data center RDMA / TCP mixed flows is usually treated as traditional multi-type traffic. One representative work is to partition the switch buffer, i.e., the switch allocates a fixed size for different traffic types to limit their maximum buffer occupancy. This strategy effectively achieves performance isolation between different traffic types, however, such fixed pre-defined settings cannot meet the dynamic fluctuations of traffic. In addition, since the fixed partition method still uses a multi-port shared buffer pool, competition still exists between ports of the same traffic type. For this reason, another representative work DT and its variants use a dynamic threshold scheme to improve the unfairness between different ports of the switch. DT uses pre-defined weighting parameters and real-time remaining buffer size to dynamically calculate the threshold for each queue on all ports. However, since this threshold is proportional to the remaining buffer size, and DT does not constrain how much buffer different traffic types can get at most. With the change of the remaining buffer size, most of the early-arriving traffic types will still be unfair to other late-arriving traffic.

[0004] In addition to the above typical shortcomings in the multi-type traffic scenario, the other two characteristics of the RDMA / TCP mixed flow further restrict the performance of the cache allocation. First, the significant difference in network transmission between RDMA and TCP traffic leads to different sizes of demand for switch cache, although DT can distinguish the difference by setting different parameters for the queue, but it cannot avoid the excessive occupation of the cache area by a few congested queues. Second, in order to achieve the lowest possible transmission delay, the existing transmission strategy usually sends the first RTTBytes packet of each flow to the network at a line speed. As the cache area of commercial switches becomes shallower and the trend of ultra-low delay requirement of application programs, it becomes more urgent to find an effective way to optimize the cache area management performance of RDMA / TCP mixed flow in DCN.

[0005] In view of the above technical defects, a solution is proposed. SUMMARY

[0006] The purpose of the present application is to provide a switch cache management strategy for data center network mixed flow to solve the above technical defects. The present application considers the different transmission characteristics of different traffic when allocating cache in the RDMA / TCP mixed flow coexistence scenario of the switch, and tries to reduce the impact of high cache demand traffic type on the transmission performance of other traffic types due to excessive cache occupation. Without affecting the overall transmission throughput, improve the absorption capacity of cache for burst flow and reduce the network transmission delay of small flow.

[0007] The purpose of the present application can be realized by the following technical scheme: a switch cache management strategy for data center network mixed flow, comprising a traffic statistics module, a cache pool division module and a threshold calculation module;

[0008] The traffic statistics module is used to monitor the queue transmission rate and length of each type of traffic in the switch. The TCP traffic includes TCP packets, and the RDMA traffic includes RDMA packets, PFC packets and RESUME packets. The queue length collected by traffic statistics is compared with the long queue threshold defined by twice the basic bandwidth delay product of each traffic. The basic bandwidth delay product refers to the amount of data sent on the link within the round-trip transmission delay, and whether the queue length of each port is greater than the long queue threshold in Determine whether the RDMA and TCP queues of the switch are in a continuous long queue state, and collect the running state of each queue in the switch in real time, and send the data to the cache pool division module and the threshold calculation module respectively;

[0009] The cache pool division module is used to calculate the total cache size allocated to RDMA and TCP according to the total queue transmission rate and the number of continuous long queues of each type of traffic, and send the data to the threshold calculation module;

[0010] The threshold calculation module is configured to calculate the buffer threshold of each queue of RDMA and TCP respectively according to the remaining size of the buffer pool of RDMA and TCP respectively and the normalized dequeue rate of each queue, and to make permission judgment.

[0011] Preferably, the statistical process of the flow statistics module is as follows:

[0012] For TCP flow, the instantaneous queue length and dequeue rate of each TCP queue are monitored at the out port of the switch; for RDMA flow, the instantaneous queue length and enqueue rate of each RDMA queue are monitored at the in port of the switch, the queue length is the instantaneous value at the sampling time, and the transmission rate is the average rate of the previous sampling interval. The active flow queue length of the switch is compared with the threshold , and the total number of long queues of each type of flow is counted, wherein tp represents the type of flow, and represent the long queue threshold of RDMA and TCP respectively.

[0013] Preferably, the division process in the buffer pool division module is as follows:

[0014] The buffer size allocated to each type of flow is calculated according to the total number of long queues and the total dequeue rate of RDMA and TCP flows at all ports of the switch, and the normalized dequeue rate is calculated according to the ratio of the dequeue speed of each queue to the maximum bandwidth of the switch port , the total dequeue rate of each queue of RDMA and TCP is calculated, and a configurable parameter is used to control the aggressiveness of queue buffer allocation. The buffer area division coefficient is positively correlated with and the total dequeue rate, and negatively correlated with the total number of long queues, and the buffer area division coefficient is calculated according to the product of the ratio of the parameters , and the pseudo code formula is as follows:

[0015]

[0016] , wherein, and are configurable parameters of the tp queue, and are the number of long queues in the tp type queue, pmax is the number of switch ports, k is the priority number, and the value is in the range of the RDMA queue set or the TCP queue set , and and are used to calculate the total allocated buffer area of RDMA and TCP flow respectively, and an exponentially weighted moving average parameter The latest value and the previous value are reconciled to reduce the impact of the dramatic fluctuation of cache requirement, while the total allocated cache size of each traffic type is substantially positively correlated with its total transmission rate and negatively correlated with the number of long queues, and the allocation according to the characteristics of traffic cache requirement is ensured, while the meaningless long queue cache occupation is reduced, and the minimum cache area of each type of traffic is allocated.

[0017] Preferably, the calculation process of the threshold calculation module is as follows:

[0018] The traffic statistics module obtains the total allocated cache size of RDMA and TCP traffic from the cache pool division module, and obtains the dequeue rate of each queue from the traffic statistics module. The traffic statistics module calculates the cache threshold for each queue according to the remaining cache size of RDMA and TCP respectively and the dequeue rate of the queue granularity, and uses a configurable parameter similar to DT to manipulate the size of the queue threshold. For a TCP queue with priority k on port i at time t, the cache threshold of the queue granularity is calculated by the formula:

[0019]

[0020] Among them, is the total allocated cache size of TCP traffic calculated, is the normalized average dequeue rate of queue k on port i in the last time interval, is the remaining unused cache size of TCP traffic in the switch at time t, is a configurable parameter used to calculate the threshold of this queue If the length of the queue exceeds its corresponding threshold, the data packet will be discarded directly.

[0021] For an RDMA queue with priority k on port i at time t, the cache threshold of the queue granularity is calculated by the formula:

[0022]

[0023] Similarly, is the total cache size allocated to the RDMA queue, is the total size allocated to the Headroom part, is a configurable parameter used to constrain the maximum usage of the RDMA traffic queue, is the remaining unused cache size of all RDMA queues at time t, and the threshold of the queue is obtained by the formula If the length of the queue exceeds its corresponding threshold, the data packet will be discarded directly.

[0024] The beneficial effects of the present application are as follows:​

[0025] The application proposes a new cache management scheme to solve the performance degradation problem caused by cache preemption when RDMA / TCP coexists in data center networks. The application limits the cache area occupation of long queues to reserve more available caches in the switch, thereby minimizing the performance impact of different flows due to differentiated cache requirements. Finally, the application effectively improves the absorption capacity of switch cache for small flows and burst flows without affecting the global throughput of data center networks. BRIEF DESCRIPTION OF DRAWINGS

[0026] The application will be further described below with reference to the accompanying drawings;

[0027] Fig. 1 is a system architecture diagram of the application;

[0028] Fig. 2 is a parameter and variable related to the application;

[0029] Fig. 3 is a cache pool partitioning algorithm pseudo code of the application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the application.

[0031] Embodiment one:

[0032] As shown in Figs. 1-3 , it is a structure schematic diagram of a smart energy storage control system according to an embodiment of the application, which comprises:

[0033] a flow statistics module, a cache pool partitioning module, and a threshold calculation module;

[0034] The flow statistics module is used to monitor the queue transmission rate and length of each type of flow in the switch, including TCP, RDMA, PFC, and RESUME. The queue length obtained by flow statistics collection is used as a window for cache pool partitioning with a long queue threshold defined by twice the basic bandwidth delay product of each flow. It is determined whether the RDMA and TCP queues of the switch are in a persistent long queue state, and the running state of each queue in the switch is collected in real time. The data is sent to the cache pool partitioning module and the threshold calculation module, respectively.

[0035] The cache pool division module is configured to calculate the total cache size allocated to RDMA and TCP according to the total queue transmission rate and the number of long queues of each type of traffic, and send data to the threshold calculation module;

[0036] The threshold calculation module is configured to calculate the cache threshold of each queue of different RDMA and TCP according to the remaining size of the cache pool of RDMA and TCP and the normalized dequeue rate of each queue, and perform permission judgment;

[0037] The statistical process of the traffic statistics module is as follows:

[0038] For TCP traffic, the instantaneous queue length and dequeue rate of each TCP queue are monitored at the out-port of the switch; for RDMA traffic, the instantaneous queue length and enqueue rate of each RDMA queue are monitored at the in-port of the switch, the queue length is the instantaneous value at the sampling time, and the transmission rate is the average rate in the previous sampling interval. The active traffic queue length of the switch is compared with the threshold , and the total number of long queues of each type of traffic is counted, wherein and represent the long queue thresholds of RDMA and TCP traffic, respectively;

[0039] The division process in the cache pool division module is as follows:

[0040] The cache size allocated to each type of traffic is calculated according to the total number of long queues and the total dequeue rate of RDMA and TCP traffic at all ports of the switch, and the normalized dequeue rate is calculated according to the dequeue speed of each queue and the maximum bandwidth ratio of the switch port , the total dequeue rate of each queue of RDMA and TCP is calculated, and the configurable parameter is used to control the aggressiveness of queue cache allocation, the cache area division coefficient is positively correlated with and the total dequeue rate, and negatively correlated with the total number of long queues, and the cache area division coefficient is calculated according to the product of the parameter ratio , and the pseudo code formula is as follows:

[0041]

[0042] wherein, and are configurable parameters of the tp queue, and are the number of long queues in the tp queue, pmax is the number of switch ports, k is the priority number, and the value is in the range of the RDMA queue set or the TCP queue set , and and The total allocated buffer size of RDMA and TCP traffic is calculated respectively using an exponentially weighted moving average parameter to reconcile the latest value with the previous value to reduce the impact of the dramatic fluctuations in buffer demand.

[0043] In this embodiment, the active traffic queue length of the switch is compared with the threshold value , and the total number of long queues of each type of traffic is counted; here and represent values related to the BDP size of RDMA / TCP traffic respectively;

[0044] The normalized dequeue speed is calculated according to the ratio of the dequeue speed of each queue to the maximum bandwidth of the switch port , and the total dequeue rate of RDMA / TCP queues is obtained; a configurable parameter is used to control the aggressiveness of the allocation of queue buffer, and the buffer division coefficient is positively correlated with and the total dequeue rate, and negatively correlated with the total number of long queues; the buffer division coefficient is calculated according to the product of the ratio of the above parameters;

[0045] The total allocated buffer size of RDMA and TCP traffic is calculated respectively using and , and an exponentially weighted moving average parameter is used to reconcile the latest value with the previous value to reduce the impact of the dramatic fluctuations in buffer demand.

[0046] Embodiment Two:

[0047] As shown in Figs. 1-3 , the calculation process of the threshold value calculation module is as follows:

[0048] The traffic statistics module obtains the total allocated buffer size of RDMA and TCP traffic from the buffer pool division module, and obtains the dequeue rate of each queue from the traffic statistics module; the traffic statistics module calculates the buffer threshold value for each queue according to the respective remaining buffer size of RDMA and TCP and the dequeue rate of the queue granularity, and uses a configurable parameter similar to that in DT to manipulate the size of the queue threshold value; for a TCP queue with priority k on port i at time t, the buffer threshold value of the queue granularity is calculated by the formula:

[0049]

[0050] wherein is the total allocated buffer size of TCP traffic calculated, is the normalized average dequeue rate of queue k on port i in the previous time interval, ​is the remaining unused buffer size of TCP traffic in the switch at time t, is a configurable parameter, and the threshold of this queue is calculated by If the length of the queue exceeds its corresponding threshold, the packet will be directly discarded.

[0051] For the RDMA queue of priority k on port i at time t, the buffer threshold of its queue granularity is calculated by

[0052]

[0053] Similarly, is the total number of buffers allocated for the RDMA queue, is the total size of the allocated Headroom part, is a configurable parameter, which is used to constrain the maximum usage of the RDMA traffic queue, is the remaining unused buffer size of all RDMA queues at time t, and the threshold of the queue is calculated by If the length of the queue exceeds its corresponding threshold, the packet will be directly discarded.

[0054] In this embodiment, for the TCP queue of priority k on port i at time t, the buffer threshold of its queue granularity is calculated by

[0055]

[0056] wherein, is the total allocated buffer of TCP traffic obtained from the stage calculation, is the normalized average dequeue rate of queue k on port i in the last time interval, is the remaining unused buffer size of TCP traffic in the switch at time t, is a configurable parameter, which is used to constrain the maximum usage of the TCP traffic queue; and the threshold of this queue is finally calculated based on the above calculation If the length of the queue exceeds its corresponding threshold, the packet will be directly discarded.

[0057] The threshold calculation method of the RDMA queue is similar to that of TCP, but the difference is that the RDMA traffic relies on the flow control mechanism to achieve its lossless transmission on Ethernet, and it is necessary to additionally reserve the Headroom buffer for the RDMA queue. The present application adopts a method similar to that of a commercial switch to allocate a fixed size of Headroom buffer for each active RDMA queue. For the RDMA queue of priority k on port i at time t, the buffer threshold of its queue granularity is calculated by

[0058]

[0059] Also, is the total number of allocated cache for RDMA queues, is the total size allocated to Headroom part, is a configurable parameter to constrain the maximum usage of RDMA traffic queues; therefore, is the size of unused cache remaining in all RDMA queues at time t; based on the above calculation, the threshold of this queue is finally obtained If the queue length of the data packet arriving at the queue exceeds its corresponding threshold, it will be directly discarded.

[0060] The above formulas are obtained by collecting a large amount of data for software simulation and selecting a formula close to the true value. The coefficients in the formula are set by those skilled in the art according to the actual situation. The above is only a preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art, according to the technical solution and the inventive concept of the present application, makes equivalent replacement or change within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A switch cache management policy for data center network hybrid flows, characterized in that, The method comprises a traffic statistics module, a cache pool division module and a threshold calculation module; The flow statistics module is used for monitoring the queue transmission rate and length of each type of flow in the switch, wherein the TCP flow includes TCP packets, the RDMA flow includes RDMA packets, PFC packets and RESUME packets, the queue length of the flow statistics collection is compared with a long queue threshold defined by twice the basic bandwidth-delay product of each flow, the basic bandwidth-delay product refers to the data amount sent by the link within the round-trip transmission delay, and whether the queue length of each port is greater than the long queue threshold is used to judge whether the RDMA and TCP queues of the switch are in a sustained long queue state, and the running state of each queue in the switch is collected in real time, and the data is respectively sent to the cache pool division module and the threshold calculation module. The flow statistics module is used for monitoring the queue transmission rate and length of each type of flow in the switch, wherein the TCP flow includes TCP packets, the RDMA flow includes RDMA packets, PFC packets and RESUME packets, the queue length of the flow statistics collection is compared with a long queue threshold defined by twice the basic bandwidth-delay product of each flow, the basic bandwidth-delay product refers to the data amount sent by the link within the round-trip transmission delay, and whether the queue length of each port is greater than the long queue threshold is used to judge whether the RDMA and TCP queues of the switch are in a sustained long queue state, and the running state of each queue in the switch is collected in real time, and the data is respectively sent to the cache pool division module and the threshold calculation module. The cache pool division module is used for calculating the total cache size of RDMA and TCP according to the total queue transmission rate and the number of long queues of each type of traffic, and sending data to the threshold calculation module; The threshold calculation module is used for calculating the cache threshold of each RDMA and TCP queue according to the remaining size of the cache pool of RDMA and TCP and the normalized dequeue rate of each queue, and performing permission judgment.

2. The data center network mixed flow facing switch cache management policy of claim 1, wherein, The statistics process of the traffic statistics module is as follows: For TCP traffic, the instantaneous queue length and dequeue rate of each TCP queue are monitored at the switch egress port; for RDMA traffic, the instantaneous queue length and enqueue rate of each RDMA queue are monitored at the switch ingress port. The queue length is the instantaneous value at the time of sampling, and the transmission rate is the average rate of the previous sampling interval. The length of the switch active traffic queue is compared with the threshold. For comparison, count the total number of persistent long queues for each type of traffic, where tp represents the traffic type. and They represent the long queue thresholds for RDMA and TCP traffic respectively.

3. The switch cache management strategy for data center network hybrid flows according to claim 1, characterized in that: The division process in the cache pool division module is as follows: The cache size allocated to each type of traffic is calculated according to the total number of persistent long queues and the total dequeue rate of RDMA and TCP traffic on all ports of the switch, and the normalized dequeue rate is calculated according to the ratio of the dequeue rate of each queue to the maximum bandwidth of the switch port The total dequeue rate of each queue of RDMA and TCP is calculated, and a configurable parameter is used to control the aggressiveness of the allocation of the queue cache area. The cache area division coefficient is positively correlated with and the total dequeue rate, and is negatively correlated with the total number of persistent long queues. The cache area division coefficient is calculated according to the product of the ratio of the parameters , and the pseudo-code formula is as follows: where, and are configurable parameters for the tp queue, and is the number of long queues in the tp class queue, pmax is the number of switch ports, k is the number of priority classes, and is in the range of the RDMA queue set or TCP queue set , the total allocated buffers for RDMA and TCP traffic are calculated using and respectively, and the exponential weighted moving average parameter is used to reconcile the latest value with previous values to reduce the impact of sharp fluctuations in buffer demand.

4. The data center network mixed flow facing switch cache management policy of claim 3, wherein, The calculation process of the threshold calculation module is as follows: The traffic statistics module obtains the total allocated buffer size of RDMA and TCP traffic from the buffer pool partition module, obtains the dequeue rate of each queue from the traffic statistics module, and calculates the buffer threshold for each queue based on the remaining buffer size of RDMA and TCP and the dequeue rate of the queue granularity, and uses configurable parameters similar to those in DT. To manipulate the size of the queue threshold, for the TCP queue with priority k on port i at time t, the cache threshold of its queue granularity is calculated by the formula: wherein, is the total allocated buffer for the TCP flow computed, is the normalized average dequeue rate of queue k on port i in the last time interval, is the remaining unused buffer size of TCP flow in the switch at time t, is a configurable parameter, the threshold of this queue is computed by if the queue length of the packet arrival exceeds its corresponding threshold, the packet will be directly dropped. For the RDMA queue with priority k on port i at time t, the cache threshold of the queue granularity is calculated by the formula: Likewise, is the total number of allocated buffers for RDMA queues, is the total size allocated for the Headroom part, is a configurable parameter to constrain the maximum usage of RDMA traffic queues, is the remaining unused buffer size in all RDMA queues at time t, the threshold of a queue is obtained by formula If the queue length of a data packet exceeds its corresponding threshold, the data packet will be directly discarded.

Images