Network switch port WRR scheduling resource dynamic optimization method based on convex optimization modeling

Abstract

The invention discloses a dynamic optimization method for WRR scheduling resources of a network switch port based on convex optimization modeling, which is applied to the field of switching equipment scheduling control and aims to solve the problem that switching equipment cannot provide stable forwarding performance due to the fact that the existing scheduling weight configuration condition cannot adapt to environment change. According to the method, a constraint function is established based on port bandwidth, token resources and the number of queues, arrival probability distribution of each queue data packet in a switch port is acquired to establish a distribution model, and a convex optimization problem of switch port queue resource allocation is established in combination with the constraint function and the acquisition model. And an optimization problem target is defined as the sum of the minimum queuing probabilities of the switch queues. Solving a problem result by using an existing convex optimization solver, and configuring the solved result as the weight of each port queue based on a weight rotation (WRR) scheduling algorithm; according to the invention, the problem of self-adaptive allocation of resources in the dynamic network flow exchange process is solved, so that the network exchange equipment provides stable network performance under the condition of bearing different services.

Description

Technical Field

[0001] The present invention belongs to the field of scheduling control of switching devices, and particularly relates to a resource scheduling technology in the process of dynamic network traffic switching. Background Art

[0002] Network packet scheduling algorithms emerged in the context of the development of the Internet, aiming to solve problems such as limited network resources, different quality-of-service requirements for different application services, and network congestion. From the early first-in-first-out algorithm to priority scheduling, fair queue algorithms, and then to scheduling algorithms based on new technologies such as software-defined networks, they have been continuously developed and improved. Related technologies include traffic classification, queue management, and bandwidth allocation, etc., which jointly support the realization of efficient network packet scheduling. These algorithms and technologies are of great significance for improving network performance, meeting the quality-of-service requirements of different applications, and ensuring the stability and reliability of the network. With the rapid development of the Internet and the continuous emergence of various new network applications, the data traffic in the network has increased explosively. In this context, how to efficiently manage and schedule network packets has become a crucial issue. Different types of network applications have different quality-of-service requirements. For example, real-time video conferencing requires low latency and stable bandwidth, while file downloading pays more attention to high throughput. In today's data center network scenario, numerous upper-layer applications and underlying hardware have strict requirements for the stability of network transmission, especially in terms of packet loss performance, almost reaching the level of zero tolerance. This is because once network packet loss occurs, whether it is an online video conferencing with extremely high real-time requirements, a financial high-frequency trading system, or upper-layer applications such as large-scale distributed storage and cloud computing management platforms that are extremely sensitive to data integrity, they will all suffer serious performance impacts, such as freezing, soaring latency, or even transaction errors. For the underlying hardware, devices such as ultra-high-speed solid-state drive arrays and low-latency network cards, packet loss may cause data write or read verification failures, thereby affecting the reliability of the entire system. Usually, in order to meet this almost demanding lossless transmission requirement, data centers will adopt the PFC (Priority-based Flow Control) protocol. This protocol can finely regulate network congestion according to the priority of data traffic. When network congestion signs appear, it can suspend the transmission of low-priority traffic and give priority to ensuring the smooth passage of high-priority critical service data, so as to achieve the goal of lossless transmission. However, the PFC protocol brings a difficult problem in actual application, that is, the network throughput performance will become very unsatisfactory. This is mainly due to the inherent characteristics of the PFC protocol's working mechanism. When it frequently suspends and starts various priority traffic, the network link is difficult to maintain an efficient and continuous data stream transmission state, resulting in a significant decline in the overall throughput.

[0003] In view of this, optimizing the scheduling algorithm of network switching devices has become an urgent task. Summary of the Invention

[0004] To solve the above technical problems, the present invention proposes a method for dynamically optimizing the WRR scheduling resources of network switch ports based on convex optimization modeling.

[0005] The technical solution adopted by the present invention is as follows: A method for dynamically optimizing the WRR scheduling resources of network switch ports based on convex optimization modeling, including:

[0006] S1. Determine various parameters required by the algorithm based on the switch port configuration information, where the parameters include: the number of port queues, the total port bandwidth, the initial size of the queue token, and the slot length;

[0007] S2. Collect the probability distribution of packet arrivals in each queue within the switch port, and use the Gaussian kernel density estimation function to convert the statistical mean of the arrival probability distribution into a probability density function of a continuous distribution;

[0008] S3. Establish a constraint objective function based on the queue length;

[0009] S4. Combine the objective constraint function and the continuous distribution probability density function to form a convex optimization problem;

[0010] S5. Use a solver to calculate the optimal solution of the convex optimization problem, and convert the calculation result into the weights of each queue within the port;

[0011] S6. Use the weighted fair queue WRR algorithm to perform packet transmission scheduling based on the weights of each queue.

[0012] Advantages of the present invention: The method of the present invention has the following advantages:

[0013] 1. A dynamic optimization algorithm for network switch port scheduling resources of the present invention is modeled based on a convex optimization algorithm, collects the status information of the switch sending port, combines it into a constraint problem based on packet loss, and calculates the optimal bandwidth allocation strategy through a solver; achieving the lowest port packet loss rate;

[0014] 2. The present invention converts the discrete network state distribution into a continuous distribution through the Gaussian density estimation algorithm, improving the calculation accuracy; at the same time, the continuousization of discrete values can make the data change smoother, effectively reducing the influence of special values on the calculation results;

[0015] 3. The present invention configures the final calculation result onto the switch port as the weights of each port queue in the WRR scheduling algorithm; no modification to the switch hardware is required, reducing the complexity of algorithm implementation, and only need to implement calculation and configuration at the software level, improving the scalability of the algorithm. Brief Description of the Drawings

[0016] Figure 1 is the flowchart of the method of the present invention;

[0017] Figure 2 is an example diagram of Gaussian kernel density estimation;

[0018] Figure 3 is a schematic diagram of the WRR scheduling algorithm. Detailed Implementation Manner

[0019] In order to solve the problems existing in the prior art, the present invention deeply studies the known network operating environment, including factors such as the type distribution of traffic, the traffic rules during peak and off-peak periods, and the dynamic change of the priority of different service applications. On this basis, a set of intelligent and flexible scheduling algorithms is constructed. The algorithm proposed by the present invention can accurately predict the trend of network congestion, and take strategies such as traffic diversion, caching or priority adjustment in advance to avoid the situation of relying too much on the PFC protocol to pause traffic. Through optimization, even under complex, changeable and high-load network conditions, the scheduling effect of minimizing packet loss can be achieved, ensuring that the key service data "does not fall behind", and at the same time ensuring that the network throughput is always stable at a relatively high level, meeting the requirements of the overall efficient operation of the data center, and providing a solid and reliable network support for various upper-layer applications and underlying hardware.

[0020] Meanwhile, considering that network resources are limited, link bandwidth, buffer size, etc. in the network are all limited resources that need to be reasonably allocated to meet the requirements of different applications. In addition, congestion phenomena frequently occur in the network. When the number of data packets in the network exceeds the processing capacity of the network, congestion will occur, reducing network performance. To solve these problems, network packet scheduling algorithms have emerged. In the prior art, early network packet scheduling algorithms were mainly simple First-In-First-Out (FIFO) algorithms, and all data packets were processed in the order of arrival. However, this algorithm cannot distinguish different types of data packets and cannot meet the quality-of-service requirements of different applications. With the development of network technology, priority scheduling algorithms have emerged, which process data packets according to their priorities, and high-priority data packets are served first. For example, in the network, data packets with high real-time requirements can be set to high priority to ensure their timely transmission. Later, the Fair Queuing algorithm was proposed, aiming to achieve fairness between different data streams and avoid a certain data stream occupying too many network resources. The Weighted Fair Queuing algorithm further considers the importance of different data streams and realizes more flexible resource allocation by assigning different weights to different data streams. In recent years, with the emergence of new technologies such as Software-Defined Network (SDN) and Network Function Virtualization (NFV), network packet scheduling algorithms have been further developed. For example, the packet scheduling algorithm based on SDN can utilize the centralized control and programmable characteristics of SDN to achieve more flexible and efficient packet scheduling.

[0021] To achieve effective packet scheduling, it is necessary to classify the traffic in the network. Traffic classification can be based on information such as the source address, destination address, port number, and protocol type of the data packet. Through traffic classification, different types of data packets can be distinguished, and different quality-of-service can be provided for different data packets. Packet scheduling algorithms usually need to be combined with queue management techniques. Queue management techniques are used to manage the data packet queues in the network. When the queue is full, it is necessary to decide which data packets can enter the queue and which data packets need to be discarded. Common queue management techniques include TailDrop, Random Early Detection (RED), etc. To meet the quality-of-service requirements of different applications, it is necessary to reasonably allocate the bandwidth resources in the network. Bandwidth allocation can be based on different strategies, such as fixed bandwidth allocation, dynamic bandwidth allocation, etc. Dynamic bandwidth allocation can adjust the bandwidth allocation in real time according to the traffic conditions in the network and the requirements of applications, improving the utilization rate of network resources.

[0022] The following further explains the content of the present invention with reference to the accompanying drawings.

[0023] AsFigure 1 As shown in the figure, the method of the present invention includes the following steps:

[0024] S1. Determine various parameters required by the algorithm based on the switch port configuration information. The parameters include: the number of port queues, the total port bandwidth, the initial size of the queue token, and the slot length. Step S1 also includes preprocessing the collected parameters:

[0025] A1. Collect the total bandwidth U of the port, the number of queues N in the port, and count the initial token number token of each queue i , i = 1, 2,..., N, and determine that the slot size is T, that is, the algorithm will be executed every T time length hereafter;

[0026] A2. Collect the number of bits b reached by all queue data packets in the port within each slot i , i = 1, 2,..., N, and calculate the slot arrival rate using the following formula:

[0027]

[0028] S2. Collect the statistical distribution of the packet arrival rates of each queue in the port, and use the Gaussian kernel density estimation function to convert the statistical mean of the arrival rate distribution into a probability density function of a continuous distribution. Specifically, it includes the following sub-steps:

[0029] S21. As Figure 2 shown in the figure, use the Gaussian kernel density estimation algorithm (gaussian_kde) to calculate the continuous distribution probability density p of the statistical mean of the arrival rate within the slot according to the statistical distribution of the arrival rate i,t , i represents the i-th queue, and t represents the t-th slot;

[0030] S22. Integrate the continuous distribution probability density calculated in step S21, and the integration range is [0, u i + token i / T], where u i is a variable representing the bandwidth allocated to queue i; obtain the cumulative distribution function (CDF) P i,t with respect to u i , and the cumulative distribution function P i,t is expressed as:

[0031]

[0032] S3. Establish a constraint objective function based on the total port bandwidth, that is, the sum of the bandwidths u i of all queues in the port should be equal to the total bandwidth U;

[0033]

[0034] S4. Form a convex optimization problem by combining the target constraint function and the continuous distribution probability density function;

[0035] S41. Construct an optimization objective. The dynamic bandwidth allocation tries to ensure the bandwidth and the number of tokens (token) within a time slot i meet the scheduling requirements within the time slot, that is, the allocated bandwidth size and the number of tokens meet the amount of data packet bits forwarded within the time slot:

[0036] P i,t = P(u i ≥ λ i ) + P(u i < λ i ) * P((λ i - u i )T < token i )

[0037] The packet scheduling weight of the switch port is implemented based on the principle of the token bucket. When a request arrives, for example, a network data packet needs to be sent, a token needs to be obtained from the bucket to represent that the request is allowed to be processed. If there are enough tokens in the bucket, the request can be processed immediately, and the number of tokens in the bucket is reduced by one. But if there are no tokens in the bucket, then this request is either rejected or put into a queue to wait until new tokens are generated. The advantage of this algorithm is that it can smooth out bursty traffic. Compared with the simple leaky bucket algorithm, the token bucket algorithm allows a certain degree of bursty traffic. For example, during periods of network idleness, a large number of tokens accumulate in the bucket. When a bursty traffic peak appears, as long as there are enough tokens in the bucket, these bursty requests can be processed quickly without being immediately restricted. This makes it widely used in many fields such as network bandwidth control and server request processing, effectively balancing the burstiness of traffic and the control of the long-term average rate, and ensuring the efficient operation of system resources within a reasonable load range.

[0038] Therefore, the allocated bandwidth size and the number of tokens to meet the amount of data packet bits forwarded within the time slot need to be considered from two perspectives. One is that the currently allocated bandwidth u i exceeds the current time slot data packet arrival rate λ i , and the other is that the currently allocated bandwidth u i is insufficient, but due to the existence of tokens, multiple tokens can be used to process the data volume arriving in the current time slot, and this situation can also meet the resource requirements of the scheduling algorithm within the time slot.

[0039] S42. Based on the expression in step S41, combine with the continuous probability density function to construct a convex optimization objective:

[0040]

[0041] The optimization objective is to maximize the sum of the resource satisfaction probabilities of N queues in a port, where P i,t is a function of u i The sum of the bandwidth ratios of the N queues should be equal to the total bandwidth of the port. The optimization objective is to find a set of queue bandwidths that maximize the probability of meeting the port resource requirements. At the same time, all bandwidth resources should be allocated to each queue to prevent bandwidth waste.

[0042] S43. Based on the constructed optimization objective, use the maximize function in the scipy.optimize library in Python as the solver to find the optimal bandwidth allocation strategy under the convex optimization problem. The solver needs to input four parameters, namely the objective formula, the initial value of the solution matrix, the constraint conditions, and the selection of the solution method of the solver. The objective formula and the constraint conditions use the two formulas in S42. The initial value of the matrix is default set to the zero matrix. The solution method selects the BFGS (Broyden Fletcher Goldfarb Shanno, quasi-Newton method) method provided by the library;

[0043] S5. Use the existing solver to calculate the optimal bandwidth allocation strategy for the convex optimization problem. The solver will calculate the u i value of each queue in the port. Multiply the u i value by the time slot size T to calculate the corresponding number of bytes of the queue, which is used as the weight of each queue in the port;

[0044] S6. Use the weighted fair queue WRR algorithm to perform packet sending scheduling based on the weights of each queue. The specific process is as follows: The WRR algorithm sends according to the queue number order. When the number of bytes sent by a queue is equal to the weight of the queue, the next queue is sent until the last queue finishes sending. Then, reset the number of bytes sent and the weight of each queue and start sending from the first queue again to form a cycle. For example Figure 3 As shown, the weights of the three queues are all 1500 bytes, and the sizes of packets 1-9 are the same, which is 1500 bytes. The WRR algorithm starts from queue 1 and sends packet 1 to the rightmost. At this time, the number of bytes sent by queue 1 is equal to the weight of the queue, and the next queue (queue 2) is sent. After queue 2 sends packet 4, the number of bytes sent by queue 2 is also equal to the weight of the queue, and queue 3 is sent. After queue 3 sends packet 7, this cycle ends; The WRR enters the next sending cycle, and so on. Finally, the packet sending order is as shown Figure 3 on the right (the packet order from right to left represents the sending order, that is, packet 1 is sent first, then packet 4, packet 7...).

[0045] Packet loss will trigger the PFC protocol to send pause frame data packets to the sender. The stop frame data packets cause the suspension of data transmission, resulting in a significant drop in throughput. The greater the packet loss rate, the more frequent the trigger and the lower the throughput. The technical solution of the present invention achieves the lowest theoretical port packet loss rate and the lowest PFC trigger frequency, solving the problem of a significant decline in throughput.

[0046] The theoretical analysis of the lowest port packet loss rate is based on the convex optimization modeling of the present invention. The goal of the model is to solve the maximum max∑ i∈[1,N] P i,t , where the cumulative distribution function P i,t The physical meaning is: the probability that the arrival rate is less than the distribution bandwidth plus the probability that the arrival rate is greater than the distribution bandwidth and the number of tokens consumed within the time slot is less than the initial number of tokens in the time slot. This is exactly the sum of the probabilities of the two scenarios without packet loss. Therefore, the final solution of the model is the maximum probability of no packet loss, corresponding to the theoretical optimal value of the minimum packet loss rate. Based on the positive correlation between the PFC trigger frequency and the packet loss rate, and the negative correlation with the throughput, the greater the packet loss rate, the more frequent the PFC trigger and the lower the throughput. By using the method of the present invention, the lowest theoretical port packet loss rate can be achieved, thus solving the problem of a significant decline in throughput in the prior art.

[0047] Those of ordinary skill in the art will realize that the embodiments described herein are for helping readers understand the principles of the present invention and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. For those skilled in the art, various changes and modifications can be made to the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included within the scope of the claims of the present invention.

Claims

1. A dynamic optimization method for network switch port WRR scheduling resources based on convex optimization modeling, characterized in that, Including: S1. Determine various parameters required by the algorithm based on the switch port configuration information. The parameters include: the number of port queues, the total port bandwidth, the initial size of the queue token, and the slot length; S2. Collect the packet arrival probability distribution of each queue in the switch port, and use the Gaussian kernel density estimation function to convert the statistical mean of the arrival probability distribution into a probability density function of a continuous distribution; S3. Establish a constraint objective function based on the queue length; S4. Combine the objective constraint function and the continuous distribution probability density function to form a convex optimization problem; S5. Use a solver to calculate the optimal solution of the convex optimization problem, and convert the calculation result into the weights of each queue in the port; S6. Use the weighted fair queue WRR algorithm to perform packet transmission scheduling based on the weights of each queue.

2. A dynamic optimization method for network switch port WRR scheduling resources based on convex optimization modeling according to claim 1, characterized in that Step S1 further includes preprocessing each parameter, specifically: The number of bits b reached by all queue data packets in the collection port within each time slot i , where i = 1, 2, …, N; Calculate the slot arrival rate: where N represents the number of port queues, T represents the slot length, and λ i represents the slot arrival rate of the i-th queue.

3. A dynamic optimization method for network switch port WRR scheduling resources based on convex optimization modeling according to claim 2, characterized in that, The implementation process of step S2 is: Using the Gaussian kernel density estimation algorithm, calculate the continuous distribution probability density p of the statistical mean of the arrival rate within a time slot according to the statistical distribution of the arrival rate i,t , where i represents the i-th queue and t represents the t-th time slot; Integrate the continuous distribution probability density p i,t to obtain the cumulative distribution function P i,t .

4. A dynamic optimization method for network switch port WRR scheduling resources based on convex optimization modeling according to claim 3, characterized in that The constraint described in step S3 is specifically that the sum of the bandwidths u of all queues within the port i is equal to the total bandwidth U.

5. A dynamic optimization method for network switch port WRR scheduling resources based on convex optimization modeling according to claim 4, characterized in that, The convex optimization problem described in step S4 is expressed as: where, P i,t = P(u i ≥ λ i ) + P(u i < λ i ) * P((λ i - u i )T < token i ), token i represents the initial size of the token of the i-th queue.

6. A dynamic optimization method for network switch port WRR scheduling resources based on convex optimization modeling according to claim 5, characterized in that The solution process of step S5 is: Using the maximize function in the scipy.optimize library in Python as the solver to find the optimal bandwidth allocation strategy under convex optimization problems. The input of the solver consists of four parameters, namely: the objective function, the constraints, the initial value of the solution matrix, and the solution method. Among them, the objective function is max∑ i∈[1,N] P i,t , t = 1, 2, 3, and the constraint is ∑ i∈[1,N] u i = U. The initial value of the solution matrix is a zero matrix, and the solution method is the BFGS method; The solver output is the u value of each queue within the port i value.

7. A dynamic optimization method for network switch port WRR scheduling resources based on convex optimization modeling according to claim 6, characterized in that The weight calculation process for each queue in step S5 is as follows: Multiply the u value of each queue by the time slot size T to calculate the number of bytes corresponding to each queue, and use the calculated number of bytes of each queue as the weight of each queue within the port. i value and the time slot size T to calculate the number of bytes corresponding to each queue, and use the calculated number of bytes of each queue as the weight of each queue within the port.

Images