A network switch port WRR scheduling resource dynamic optimization method based on convex optimization modeling

Abstract

This invention discloses a dynamic optimization method for network switch port WRR scheduling resources based on convex optimization modeling. Applied to the field of switching equipment scheduling and control, it addresses the problem that existing scheduling weight configurations cannot adapt to environmental changes, resulting in unstable forwarding performance of switching equipment. This invention establishes constraint functions based on port bandwidth, token resources, and queue number, collects the arrival probability distribution of data packets in each queue within the switch port to establish a distribution model, and combines the constraint functions and the collected model to construct a convex optimization problem for switch port queue resource allocation. The objective of the optimization problem is defined as minimizing the sum of queuing probabilities of the switch queues. The problem is solved using an existing convex optimization solver, and the results are configured as the weights of each port queue based on the Weighted Round Robin (WRR) scheduling algorithm. This invention solves the problem of adaptive resource allocation during dynamic network traffic exchange, enabling network switching equipment to provide stable network performance under different service loads.

Description

Technical Field

[0001] This invention belongs to the field of switching equipment scheduling and control, and specifically relates to a resource scheduling technology in the process of dynamic network traffic switching. Background Technology

[0002] Network packet scheduling algorithms emerged in response to the development of the Internet, aiming to solve problems such as limited network resources, different service quality requirements of various applications, and network congestion. From early First-In-First-Out (FIFO) algorithms to priority scheduling, fair queueing algorithms, and scheduling algorithms based on new technologies such as software-defined networking, they have continuously evolved and improved. Related technologies include traffic classification, queue management, and bandwidth allocation, all contributing to efficient network packet scheduling. These algorithms and technologies are of great significance for improving network performance, meeting the service quality requirements of different applications, and ensuring network stability and reliability. With the rapid development of the Internet and the continuous emergence of various new network applications, data traffic in networks has exploded. In this context, how to efficiently manage and schedule network packets has become a crucial issue. Different types of network applications have different service quality requirements; for example, real-time video conferencing requires low latency and stable bandwidth, while file downloads prioritize high throughput. In today's data center network scenarios, many upper-layer applications and underlying hardware have stringent requirements for network transmission stability, especially in terms of packet loss performance, which is almost zero-tolerance. This is because once network packet loss occurs, all applications, from high-reality online video conferencing and high-frequency financial trading systems to large-scale distributed storage and cloud computing management platforms that are extremely sensitive to data integrity, will suffer severe performance impacts, resulting in stuttering, spiked latency, and even transaction errors. For underlying hardware, devices like ultra-high-speed solid-state drive arrays and low-latency network cards may experience data write or read verification failures due to packet loss, thus affecting the reliability of the entire system. Typically, to meet this near-stringent lossless transmission requirement, data centers employ the Priority-based Flow Control (PFC) protocol. This protocol can fine-tune network congestion based on data traffic priority. When network congestion is detected, it can suspend the transmission of low-priority traffic, prioritizing the smooth passage of high-priority critical business data, thereby achieving the goal of lossless transmission. However, in practical applications, the PFC protocol presents a significant challenge: network throughput performance becomes extremely poor. This is mainly due to the inherent characteristics of the PFC protocol's working mechanism. When it frequently pauses and starts various priority traffic, the network link struggles to maintain an efficient and continuous data flow transmission state, leading to a significant drop in overall throughput.

[0003] Therefore, optimizing the scheduling algorithm of network switching equipment has become an urgent task. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a dynamic optimization method for WRR scheduling resources of network switch ports based on convex optimization modeling.

[0005] The technical solution adopted in this invention is: a dynamic optimization method for network switch port WRR scheduling resources based on convex optimization modeling, comprising:

[0006] S1. Determine the various parameters required for the algorithm based on the switch port configuration information. The parameters include: number of port queues, total port bandwidth, initial size of queue tokens, and time slot length.

[0007] S2. Collect the arrival probability distribution of data packets in each queue within the switch port, and use the Gaussian kernel density estimation function to transform the statistical mean of the arrival probability distribution into a continuous probability density function.

[0008] S3. Establish a constraint objective function based on queue length;

[0009] S4. Combine the objective constraint function and the continuous probability density function to form a convex optimization problem;

[0010] S5. Use the solver to calculate the optimal solution to the convex optimization problem and convert the calculation results into the weights of each queue within the port;

[0011] S6. Use the Weighted Fair Queuing (WRR) algorithm to schedule the transmission of data packets based on the weights of each queue.

[0012] The beneficial effects of the present invention: The method of the present invention has the following advantages:

[0013] 1. The present invention provides a dynamic optimization algorithm for network switch port scheduling resources, which is based on convex optimization algorithm for modeling, collects the status information of the switch's sending ports, combines them into a constraint problem based on packet loss, and calculates the optimal bandwidth allocation strategy through a solver; thereby achieving the lowest port packet loss rate.

[0014] 2. This invention transforms the discretized network state distribution into a continuous distribution using a Gaussian density estimation algorithm, thereby improving the accuracy of the calculation. At the same time, the continuous nature of the discrete values ​​makes the data changes smoother and effectively reduces the impact of special values ​​on the calculation results.

[0015] 3. This invention configures the final calculation results onto the switch ports as the weights of each port queue in the WRR scheduling algorithm; it does not require modification of the switch hardware, reducing the complexity of the algorithm implementation, and only requires calculation and configuration at the software level, thus improving the algorithm's scalability. Attached Figure Description

[0016] Figure 1 This is a flowchart of the method of the present invention;

[0017] Figure 2 Example plot of Gaussian kernel density estimation;

[0018] Figure 3 This is a schematic diagram of the WRR scheduling algorithm. Detailed Implementation

[0019] To address the problems existing in current technologies, this invention delves into known network operating environments, including traffic type distribution, traffic patterns during peak and off-peak periods, and dynamic changes in the priorities of different business applications. Based on this, an intelligent and flexible scheduling algorithm is constructed. The algorithm proposed in this invention can accurately predict network congestion trends and proactively implement strategies such as traffic offloading, caching, or priority adjustment to avoid over-reliance on the PFC protocol to pause traffic. Through optimization, it achieves minimal packet loss even under complex, variable, and high-load network conditions, ensuring that critical business data is not lost, while also maintaining a consistently high level of network throughput to meet the overall high-efficiency operation requirements of the data center, providing robust and reliable network support for various upper-layer applications and underlying hardware.

[0020] Meanwhile, considering the finite nature of network resources—link bandwidth, buffer size, etc.—requires reasonable allocation to meet the needs of different applications. Furthermore, network congestion is frequent; when the number of data packets exceeds the network's processing capacity, congestion occurs, degrading network performance. To address these issues, network packet scheduling algorithms have emerged. Early network packet scheduling algorithms primarily used simple First-In-First-Out (FIFO) algorithms, processing all packets in the order of arrival. However, this algorithm cannot distinguish between different types of packets or meet the service quality requirements of different applications. With the development of network technology, priority scheduling algorithms emerged, processing packets based on their priority, with higher-priority packets receiving priority service. For example, packets with high real-time requirements can be set as high priority to ensure timely transmission. Later, the Fair Queuing algorithm was proposed to achieve fairness among different data flows, preventing any single data flow from consuming excessive network resources. The Weighted Fair Queuing algorithm further considers the importance of different data flows, achieving more flexible resource allocation by assigning different weights to different data flows. In recent years, with the emergence of new technologies such as Software-Defined Networking (SDN) and Network Functions Virtualization (NFV), network packet scheduling algorithms have also been further developed. For example, SDN-based packet scheduling algorithms can leverage the centralized control and programmability of SDN to achieve more flexible and efficient packet scheduling.

[0021] To achieve effective packet scheduling, network traffic needs to be classified. Traffic classification can be based on information such as the source address, destination address, port number, and protocol type of the packets. By classifying traffic, different types of packets can be distinguished, and different qualities of service (QoS) can be provided to different packets. Packet scheduling algorithms are often combined with queue management techniques. Queue management techniques are used to manage packet queues in the network. When a queue is full, it is necessary to decide which packets can enter the queue and which packets should be dropped. Common queue management techniques include TailDrop and Random Early Detection (RED). To meet the QoS requirements of different applications, network bandwidth resources need to be allocated rationally. Bandwidth allocation can be based on different strategies, such as fixed bandwidth allocation and dynamic bandwidth allocation. Dynamic bandwidth allocation can adjust bandwidth allocation in real time according to network traffic conditions and application needs, improving the utilization of network resources.

[0022] The invention will be further explained below with reference to the accompanying drawings.

[0023] like Figure 1 As shown, the method of the present invention includes the following steps:

[0024] S1. Determine various parameters required for the algorithm based on the switch port configuration information, including: number of port queues, total port bandwidth, initial size of queue tokens, and time slot length; Step S1 also includes preprocessing the collected parameters:

[0025] A1. Total bandwidth U of the acquisition port, number of queues N within the port, and initial token count for each queue. i Let i = 1, 2, ..., N, and determine the time slot size as T, meaning that the algorithm will be executed once every T time slots thereafter.

[0026] A2. The number of bits reached by all queued data packets within each time slot in the acquisition port (b) i For i = 1, 2, ..., N, calculate the time slot arrival rate using the following formula:

[0027]

[0028] S2. Collect the statistical distribution of data packet arrival rates for each queue within the port, and use the Gaussian kernel density estimation function to transform the statistical mean of the arrival rate distribution into a probability density function of a continuous distribution; specifically including the following steps:

[0029] S21, such as Figure 2 As shown, the Gaussian kernel density estimation algorithm (gaussian_kde) is used to calculate the continuous probability density p of the statistical mean of the arrival rate within a time slot based on the statistical distribution of the arrival rate. i,t , where i represents the i-th queue and t represents the t-th time slot;

[0030] S22. Integrate the continuous probability density calculated in step S21. The range of the integral is [0, u]. i +token i / T], where u i It is a variable representing the bandwidth allocated to queue i; obtain the cumulative distribution function (CDF) P. i,t About u i The function, cumulative distribution function P i,t Represented as:

[0031]

[0032] S3. Establish a constraint objective function based on the total port bandwidth, i.e., the bandwidth u of all queues within the port. i The sum should equal the total bandwidth U;

[0033]

[0034] S4. Combine the objective constraint function and the continuous probability density function to form a convex optimization problem;

[0035] S41. Construct optimization goals and dynamically allocate bandwidth to ensure the maximum possible bandwidth and token count within a time slot. i To meet the scheduling requirements within a time slot, the allocated bandwidth and number of tokens must satisfy the packet bit forwarding volume within the time slot.

[0036] P i,t =P(u i ≥λ i )+P(u i <λ i )*P((λ i -u i )T <token i )

[0037] The packet scheduling weighting algorithm for switch ports is based on the token bucket principle. When a request arrives, such as a network packet to be sent, a token is retrieved from the bucket to indicate that the request is allowed to be processed. If there are enough tokens in the bucket, the request can be processed immediately, and the number of tokens in the bucket decreases by one. However, if there are no tokens in the bucket, the request is either rejected or placed in a queue to wait until a new token is generated. The advantage of this algorithm is that it can smooth out bursts of traffic. Compared with the simple leaky bucket algorithm, the token bucket algorithm allows for a certain degree of bursty traffic. For example, during periods of network idle time, a large number of tokens accumulate in the bucket. When a sudden traffic surge occurs, as long as there are enough tokens in the bucket, these bursty requests can be processed quickly without being immediately restricted. This makes it widely used in various fields such as network bandwidth control and server request processing, effectively balancing the burstiness of traffic and the control of long-term average rates, ensuring that system resources operate efficiently within a reasonable load range.

[0038] Therefore, the allocated bandwidth and token count to meet the packet bit forwarding volume within a time slot need to be considered from two perspectives: one is the currently allocated bandwidth u. i The current time slot packet arrival rate λ has been exceeded. i Another option is the currently allocated bandwidth u i While this is insufficient, the existence of tokens allows multiple tokens to be used to handle the amount of data arriving in the current time slot, which can also meet the resource requirements of the time slot scheduling algorithm.

[0039] S42. Construct a convex optimization objective based on the expression in step S41 and the continuous probability density function:

[0040]

[0041] The optimization objective is to maximize the sum of the probabilities of resource satisfaction for N queues within a single port, where P i,t It's about u i The function is such that the sum of the bandwidth ratios of N queues should equal the total bandwidth of the port. The optimization goal is to find a set of queues whose bandwidth maximizes the probability of meeting the port's resource requirements, while allocating all bandwidth resources to each queue to prevent bandwidth waste.

[0042] S43. Based on the constructed optimization objective, the maximize function of the scipy.optimize library in Python is used as the solver to find the optimal bandwidth allocation strategy under the convex optimization problem. The solver requires four input parameters: the objective formula, the initial value of the solution matrix, the constraints, and the solution method selection. The objective formula and the constraints use the two formulas in S42. The initial value of the matrix is ​​set to the zero matrix by default. The solution method is the BFGS (Broyden Fletcher Goldfarb Shanno, quasi-Newton method) method provided by the library.

[0043] S5. Using the existing solver, calculate the optimal bandwidth allocation strategy for the convex optimization problem. The solver will calculate u for each queue within the port. i Value, based on the occupancy of u i Multiply by the time slot size T to calculate the number of bytes corresponding to the queue, which is used as the weight of each queue within the port;

[0044] S6. The Weighted Fair Queuing (WRR) algorithm is used to schedule packet transmission based on the weights of each queue. The specific process is as follows: The WRR algorithm sends packets sequentially according to queue number. When the number of bytes sent by a queue equals its weight, the next queue is moved on, until the last queue has finished sending. Then, the number of bytes sent and the weight of each queue are reset, and transmission restarts from the first queue, forming a loop. For example... Figure 3 As shown, the weight of each of the three queues is 1500 bytes, and the size of messages 1-9 is the same, also 1500 bytes. The WRR algorithm starts from queue 1, sending message 1 to the rightmost position. At this point, the number of bytes sent by queue 1 equals its weight, and the algorithm sends to the next queue (queue 2). After queue 2 sends message 4, the number of bytes sent by queue 2 also equals its weight, and the algorithm sends to queue 3. After queue 3 sends message 7, the loop ends; WRR enters the next sending cycle, and so on, until the final message sending order is as follows. Figure 3 The right side shows the message order from right to left, indicating the order in which they were sent, i.e., message 1 was sent first, followed by message 4, message 7, and so on.

[0045] Packet loss triggers the PFC protocol to send a pause frame data packet to the sender. This pause frame data packet causes a halt in data transmission, significantly reducing throughput. The higher the packet loss rate, the more frequently PFC is triggered, and the lower the throughput. The technical solution of this invention theoretically achieves the lowest port packet loss rate and the lowest PFC trigger frequency, thus solving the problem of a significant drop in throughput.

[0046] The theoretical analysis of minimizing port packet loss rate is based on the convex optimization modeling of this invention. The goal of the model is to solve for the maximum max∑ i∈[1,N] P i,t Here, the cumulative distribution function P i,t The physical meaning of this is: the probability that the arrival rate is less than the distributed bandwidth plus the probability that the arrival rate is greater than the distributed bandwidth and the token book consumed within the time slot is less than the initial number of tokens in the time slot. This is precisely the sum of the probabilities of the two scenarios with no packet loss. Therefore, the final model solution is the maximum probability of no packet loss, corresponding to the theoretically optimal value of the minimum packet loss rate. Since the PFC triggering frequency is positively correlated with the packet loss rate and negatively correlated with throughput, the higher the packet loss rate, the more frequently PFC is triggered, and the lower the throughput. The method of this invention can theoretically achieve the lowest port packet loss rate, thereby solving the problem of a significant drop in throughput in existing technologies.

[0047] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. A network switch port WRR scheduling resource dynamic optimization method based on convex optimization modeling, characterized in that, include: S1. Determine the various parameters required for the algorithm based on the switch port configuration information. The parameters include: number of port queues, total port bandwidth, initial size of queue tokens, and time slot length. S2. Collect the arrival probability distribution of data packets in each queue within the switch port, and use the Gaussian kernel density estimation function to transform the statistical mean of the arrival probability distribution into a continuous probability density function. S3. Establish a constraint objective function based on queue length; S4. Combining the objective constraint function and the continuous probability density function to form a convex optimization problem; specifically including: S41. Construct optimization goals and dynamically allocate bandwidth to ensure the maximum possible bandwidth and token count within a time slot. To meet the scheduling requirements within a time slot, the allocated bandwidth and number of tokens must satisfy the packet bit forwarding volume within the time slot. ； wherein, denotes the initial size of the token for the queue, u i is the queue bandwidth, denotes the arrival rate of the time slots for the queue, denotes the length of the time slots; S42. Construct a convex optimization objective based on the expression in step S41 and the continuous probability density function: ； ； S5. Use the solver to calculate the optimal solution to the convex optimization problem and convert the calculation results into the weights of each queue within the port; S6. Use the weighted round-robin (WRR) algorithm to schedule the sending of data packets based on the weight of each queue.

2. The method of claim 1, wherein the method is based on convex optimization modeling of network switch port WRR scheduling resource dynamic optimization. Step S1 also includes preprocessing the parameters, specifically: the number of bits reached by all the queue data packets in the acquisition port in each time slot , ; Calculate the time slot arrival rate: ； wherein, represents the number of port queues.

3. The method of claim 2, wherein the method is based on convex optimization modeling of network switch port WRR scheduling resource dynamic optimization. The implementation process of step S2 is as follows: A continuous distribution probability density of the statistical mean of the arrival rate in a time slot is calculated from the statistical distribution of the arrival rate using a Gaussian kernel density estimation algorithm where i denotes the i-th queue and t denotes the t-th time slot. for a continuous distribution of probability density Integrating, the cumulative distribution function is obtained .

4. The method of claim 3, wherein the method further comprises: The constraint described in step S3 is that the sum of all queue bandwidths u i within a port equals the total bandwidth U.

5. The method of claim 4, wherein the method further comprises: The solution process for step S5 is as follows: The `maximize` function from the `scipy.optimize` library in Python is used as the solver to find the optimal bandwidth allocation strategy for convex optimization problems. The solver's input includes four parameters: the objective function, constraints, initial values ​​of the solution matrix, and the solution method. The objective function is The constraint condition is The initialized value of the solution matrix is a zero matrix, and the solution method is the quasi-Newton BFGS method. The solver outputs the u values for each queue within the port i values.

6. The method of claim 5, wherein the method is based on convex optimization modeling of network switch port WRR scheduling resource dynamic optimization. The weight calculation process for each queue in step S5 is as follows: The weights of each queue are calculated as follows: u... i The value is multiplied by the time slot size T to calculate the number of bytes corresponding to each queue, and the calculated number of bytes for each queue is used as the weight of each queue within the port.

Images