WFQ multi-queue scheduling system and method based on FPGA
By designing a WFQ multi-queue scheduling system on FPGA, using parameter cache, label calculation and label scheduling modules, the problem of poor integer division operation accuracy is solved, and high-precision bandwidth allocation and minimum bandwidth guarantee are achieved.
Patent Information
- Application Number
- CN202311508327.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-13
- Publication Date
- 2025-05-13
AI Technical Summary
When implementing WFQ multi-queue scheduling on FPGA, the accuracy of integer division operation is poor, resulting in inaccurate bandwidth allocation.
A WFQ multi-queue scheduling system based on FPGA is designed, including parameter caching module, tag calculation module and tag scheduling module. By rounding up and adjusting the integer division results, calculate the new completion time and deviation values, and select the queue corresponding to the minimum tag for scheduling when scheduling the first packet of the queue.
High-precision bandwidth allocation for WFQ multi-queue scheduling on FPGA is realized, ensuring the minimum bandwidth guarantee for each queue in the case of congestion.
Smart Images
Figure CN119996339A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to network data packet scheduling and SDN hardware, and in particular to a WFQ multi-queue scheduling system and method based on FPGA. Background Art
[0002] Network forwarding equipment needs to provide effective bandwidth resource allocation and management for different network links to improve the utilization efficiency of bandwidth resources. At the forwarding node, different network flows are assigned to different queues, and the switch then schedules the data packets in different queues according to a certain scheduling algorithm.
[0003] The Weighted Fair Queueing (WFQ) algorithm assigns a weight to each queue, and then schedules the packets in each queue based on the weight of the queue where each packet is located and the packet length of the packet. Each queue has a virtual clock, and the packets are scheduled in the order of the clock. When a packet in a queue is sent out, the clock of the queue moves forward to simulate the bandwidth ratio allocated to the queue. WFQ is a scheduling algorithm that takes the length of the packet into consideration when scheduling. It ensures that each queue has a fair chance to use the network bandwidth at any given time, and can assign different values to the weight of each queue to meet the quality of service requirements of different queues.
[0004] FPGA stands for Field Programmable Gate Array. In high-speed networks, offloading the functions of processing and forwarding network data packets to the FPGA can save CPU computing resources. The data plane offloaded to the FPGA still needs to provide different bandwidth guarantees for different queues. However, when integer division operations are implemented on the FPGA, the result is expressed in the form of a quotient and a remainder. The existing WFQ label update algorithm does not take the remainder into account, so the accuracy of implementing WFQ directly on the FPGA according to the existing algorithm is poor. Summary of the invention
[0005] The purpose of the present invention is to overcome the defects of the prior art and propose a WFQ multi-queue scheduling system and method based on FPGA.
[0006] In order to achieve the above object, the present invention proposes a WFQ multi-queue scheduling system based on FPGA, the system comprising: a parameter cache module, a label calculation module and a label scheduling module; wherein,
[0007] The parameter cache module is used to cache the parameters of each WFQ queue and record the deviation value caused by each label calculation;
[0008] The label calculation module is used to obtain parameters from the parameter cache module, first adjust the relevant parameters in the opposite direction according to the deviation value of the previous data packet of the same queue to eliminate the error, then calculate the new completion time and deviation value, send them to the parameter cache module, and send the new completion time as the head data packet label of the queue to the label scheduling module;
[0009] The label scheduling module is used to receive and cache the label of the first data packet of the queue, and is also used to schedule according to the external signal, select the smallest label from the cached labels, send it to the parameter cache module, delete the label of the first data packet of the queue, and schedule the first data packet of the corresponding queue to be dequeued.
[0010] Preferably, the parameters of each WFQ queue include weight, completion time, deviation value and virtual time shared by all queues.
[0011] Preferably, the weight is a positive integer configured by the host or controller.
[0012] Preferably, the virtual time shared by all queues is the tag of the first data packet of the previously scheduled queue, and is kept until a new queue is scheduled, and is replaced by the tag of the first data packet of the new scheduled queue.
[0013] Preferably, the new completion time and deviation value are obtained by the following steps:
[0014] Select the larger value between the completion time and the virtual time as the start time;
[0015] Calculate the difference between the packet length and the deviation value as the corrected packet length;
[0016] Calculate the integer division of the modified packet length and the weight to obtain the quotient and remainder;
[0017] Calculate the sum of the start time and the quotient, then add one to get the new finish time;
[0018] Calculate the difference between the weight and the remainder as the new bias value.
[0019] Preferably, the parameter cache module, label calculation module and label scheduling module are all implemented by FPGA-based hardware circuits.
[0020] On the other hand, the present invention proposes a WFQ multi-queue scheduling method based on FPGA, which is implemented based on the above system. The method includes:
[0021] The label calculation module obtains parameters from the parameter cache module, first adjusts the relevant parameters in the opposite direction according to the deviation value of the previous data packet of the same WFQ queue to eliminate the error, then calculates the new completion time and deviation value, and sends them to the parameter cache module; the new completion time is sent to the label scheduling module as the label of the first data packet of the queue;
[0022] The parameter cache module caches the received parameters of each WFQ queue to record the deviation value caused by each label calculation;
[0023] The label scheduling module receives and caches the head-of-queue packet label of each WFQ queue;
[0024] When receiving an external signal, the label scheduling module selects the smallest label from the cached labels, sends it to the parameter cache module, deletes the label of the first packet of the queue, and schedules the first packet of the corresponding queue to be dequeued.
[0025] Compared with the prior art, the advantages of the present invention are:
[0026] In the process of performing WFQ multi-queue scheduling and calculating the completion label of the data packet, division operations are involved. If software calculation is used, the result of the floating-point division operation can be directly used in subsequent operations without considering the integer division problem and precision problem. However, it is difficult to perform floating-point operations on FPGA, and only integer division can be used for division operations. The result obtained deviates from the precise result. Using this result to participate in subsequent operations will cause errors in the calculated completion label, and ultimately cause deviations in the bandwidth allocated to each queue. The WFQ multi-queue scheduling algorithm implemented by the present invention can provide a more accurate minimum bandwidth guarantee for each queue when congestion occurs. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 It is a schematic diagram of the system scheduling multiple queue head data packets of the present invention;
[0028] Figure 2 It is the overall architecture diagram of the WFQ multi-queue scheduling system based on FPGA implementation of the present invention;
[0029] Figure 3 It is the hardware implementation label calculation circuit diagram of the present invention. DETAILED DESCRIPTION
[0030] In order to solve the problem of poor bandwidth allocation accuracy when implementing the WFQ scheduling algorithm on an FPGA switch, this application proposes a method and system for implementing WFQ multi-queue scheduling based on FPGA, which achieves high-precision WFQ scheduling by calculating and scheduling labels for the data packets at the head of each queue.
[0031] The idea is to calculate and schedule labels for the head packets of multiple queues. When calculating labels, round up the integer division result, record the deviation caused by rounding up, and make a reverse adjustment for the deviation of the previous packet before calculating the next packet of the same queue. When scheduling the head packet, select the queue corresponding to the smallest label for scheduling, and the head packet of the queue is scheduled out of the queue.
[0032] The system implemented by the present application calculates and schedules labels for the data packets at the head of each queue. When there is a data packet cached at the head of an empty queue, or when a data packet at the head of a queue is scheduled out of the queue and the queue is still a non-empty queue, the system implemented by the present application calculates the label for the data packet at the head of the queue at this time, caches the label, and when it is necessary to schedule a data packet out of the queue, selects the queue corresponding to the smallest queue label that has been cached, and schedules the data packet at the head of the queue.
[0033] The system implemented in the present application includes a parameter cache module, a label calculation module and a label scheduling module. The parameter cache module caches parameters such as weight, completion time, deviation value, etc. for each queue. Among them, the weight of each queue is configured by the host or controller and is a positive integer. When it is necessary to calculate the label of the first data packet of a queue, all parameters cached in the parameter cache module of the queue are read out, and the label is calculated together with the packet length and virtual time of the first data packet of the queue. The label calculation is implemented by the label calculation module through a hardware circuit. According to the deviation value calculated last time provided by the parameter cache module, before calculating the label of the next data packet of the same queue, it is adjusted in the opposite direction to eliminate the error. Then use the new completion time and new deviation value calculated by the label to update the corresponding parameters of the parameter cache module. The label scheduling module receives and caches the queue labels calculated by all label calculation modules. When it is necessary to schedule a data packet out of the queue, the label scheduling module selects the smallest queue label that has been cached, deletes the label, and the first data packet of the corresponding queue is scheduled out of the queue.
[0034] All queues share a virtual time. When a packet is scheduled, the virtual time is updated to the label of the packet. When no queue is scheduled for a period of time, the virtual time remains unchanged. Once a queue is scheduled, the virtual time is updated to the label of the first packet in the queue.
[0035] The algorithm for label calculation is as follows: first, select the larger value between the completion time and the virtual time as the start time; at the same time, calculate the difference between the packet length and the deviation value as the corrected packet length; then calculate the integer division of the corrected packet length and the weight to obtain the quotient and remainder; finally, calculate the sum of the start time and the quotient, and add one to the result as the new completion time. Calculate the difference between the weight and the remainder as the new deviation value. Use the new completion time as the label of the first packet in the queue.
[0036] Let the weight of queue i be w i , the jth packet in queue i is recorded as Will The length of the data packet is recorded as Will The time of arrival is recorded as Will The completion time is recorded as Will The deviation value is denoted as T i j , let the virtual time at time t be v(t), then the label calculation algorithm can be expressed as:
[0037]
[0038]
[0039] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.
[0040] Example 1
[0041] Embodiment 1 of the present invention proposes a WFQ multi-queue scheduling system based on FPGA.
[0042] Figure 1 The system schedules multiple queues with the first-line data packets. The arriving data packets are assigned to different cache queues according to their queue numbers. The system involved in the present invention, as a WFQ scheduler, schedules the data packets at the head of each queue. In each queue, the time when the data packets that enter the queue later are scheduled must be later than the data packets that enter the queue earlier.
[0043] Figure 2is the overall architecture diagram of the system. The system includes a parameter cache module, a label calculation module and a label scheduling module. In the figure, ① indicates a request sent to the parameter cache module when the head packet of a queue needs to be label calculated. When a packet is cached at the head of an empty queue, or the queue is still a non-empty queue after a head packet of a queue is scheduled out of the queue, the head packet of the queue needs to be label calculated. ② indicates that the parameter cache module sends the weight, completion time, deviation value, and virtual time shared by all queues of the queue obtained by query to the label calculation module. ③ indicates that the label calculation module sends the calculated new completion time and deviation value of the queue back to the parameter cache module. ④ indicates that the label calculation module sends the calculated label of the head packet of the queue to the label scheduling module. ⑤ indicates that when a packet is scheduled, the label scheduling module sends the label of the packet to the parameter cache module for updating the virtual time. ⑥ indicates the input signal from outside the system when a packet needs to be scheduled. ⑦ indicates that the label scheduling module selects the queue number to be scheduled, sends the queue number to the outside world, and then the head packet of the queue is scheduled.
[0044] Figure 3 This is the circuit diagram of the hardware implementation of label calculation. The rectangles in the figure represent the values of the corresponding parameters, and the diamonds represent the calculations between the corresponding parameters. Each parameter in the figure corresponds to the implementation of the corresponding register in the FPGA. The parameters in the same column, and their corresponding registers in the FPGA are updated to new values in the same clock cycle. In the same clock cycle, all parameters are input at the same time, and accordingly, all output parameters are calculated at the same time in the same clock cycle.
[0045] The following example illustrates the working principle of the hardware implementation of the label calculation circuit diagram. Assume that the weight of a queue is 7. Assume that at a certain moment, the completion label of the data packet at the head of the queue is calculated, and the length of the data packet is 100B. The completion time of the previous data packet of the queue read from the parameter cache module is 123, the virtual time is 150, and the deviation value is 4. First, the maximum value between the completion time and the virtual time is calculated as the start time. In this example, the start time is updated to 150. At the same time, the difference between the packet length and the deviation value is calculated to obtain the corrected packet length. In this example, the corrected packet length is updated to 96. Then, the division of the corrected packet length and the weight is calculated, and the quotient is 13 and the remainder is 5. Then, the sum of the start time and the quotient is calculated, and the result is added by one as the new completion time. In this example, the new completion time is equal to 150 plus 13 plus 1, that is, 164. At the same time, the difference between the weight and the remainder is calculated as the new deviation value. In this example, the new deviation value is equal to 7 minus 5, that is, 2. Finally, the new completion time is used as the label of the first packet in the queue and is sent to the label scheduling module; the new completion time and the new deviation value are updated back to the corresponding position of the parameter cache module. When the next packet in the queue needs to calculate the label, the completion time and the deviation value read from the parameter cache module will be equal to 164 and 2 respectively.
[0046] Using the above technical solution, WFQ multi-queue scheduling is implemented on the FPGA board of Xilinx Kintex-7 series model xc7k325tffg900-2L. The total bandwidth of the limited port is 5Gbps, and a total of 6 WFQ queues are configured on the port. When the weights of these 6 queues are configured as 1:2:3:4:5:6, the actual bandwidth allocation ratio can reach 1.00000:2.00001:3.00002:4.00001:4.99997:5.99999; when the weights of these 6 queues are configured as 1:51:135:256:521:999, the actual bandwidth allocation ratio can reach 1.00001:51.00056:134.99994:255.99901:521.00132:998.99916.
[0047] Example 2
[0048] Embodiment 2 of the present invention proposes a WFQ multi-queue scheduling method based on FPGA implementation, which is based on the system implementation of embodiment 1.
[0049] The process is:
[0050] Step 1: The parameter cache module provides the parameters of the queue and the deviation generated by the previous label calculation of the queue;
[0051] Step 2: The label calculation module adjusts the corresponding parameters in the opposite direction according to the provided deviation;
[0052] Step 3: Label calculation is performed using the adjusted parameters to obtain a new completed label. This calculation also introduces new deviations.
[0053] Step 4: This deviation is cached in the parameter cache module until the next packet in the queue is to be labeled and step 1 is repeated.
[0054] The present invention implements a method and system for implementing WFQ multi-queue scheduling based on FPGA, and performs label calculation and scheduling on the head data packets of multiple queues. When performing label calculation, the integer division result is rounded up, the deviation caused by rounding up is recorded, and before calculating the next data packet of the same queue, the deviation of the previous data packet is reversely adjusted. When scheduling the head data packet, the queue corresponding to the smallest label is selected for scheduling, and the data packet at the head of the queue is scheduled out of the queue. The WFQ multi-queue scheduling algorithm implemented by the present invention realizes more fair and accurate bandwidth allocation for each queue.
[0055] Deploying a WFQ scheduling algorithm for scheduling different queues on a switch is beneficial to bandwidth allocation and isolation between different queues. However, due to the poor precision of division operations on FPGA, the bandwidth allocation precision between queues implemented by implementing the WFQ scheduling algorithm on an FPGA switch is poor. The present invention improves the bandwidth allocation precision of the WFQ scheduling algorithm implemented on an FPGA switch by improving the algorithm for calculating data packet labels.
[0056] The system of the present invention can be applied to scenarios such as data centers where the data plane is offloaded to FPGAs, and WFQ scheduling is provided for different queues of the data plane offloaded to the FPGAs. The WFQ scheduling algorithm implemented on the FPGA by the present method can provide more accurate minimum bandwidth guarantees for each queue when congestion occurs.
[0057] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention and should be included in the scope of the claims of the present invention.
Claims
1. A WFQ multi-queue scheduling system based on FPGA, characterized in that: The system includes: a parameter cache module, a label calculation module and a label scheduling module; wherein, The parameter cache module is used to cache the parameters of each WFQ queue and record the deviation value caused by each label calculation; The label calculation module is used to obtain parameters from the parameter cache module, first adjust the relevant parameters in the opposite direction according to the deviation value of the previous data packet of the same queue to eliminate the error, then calculate the new completion time and deviation value, send them to the parameter cache module, and send the new completion time as the head data packet label of the queue to the label scheduling module; The label scheduling module is used to receive and cache the label of the first data packet of the queue, and is also used to schedule according to the external signal, select the smallest label from the cached labels, send it to the parameter cache module, delete the label of the first data packet of the queue, and schedule the first data packet of the corresponding queue to be dequeued.
2. The WFQ multi-queue scheduling system based on FPGA implementation according to claim 1 is characterized in that: The parameters of each WFQ queue include weight, completion time, deviation value, and virtual time shared by all queues.
3. The WFQ multi-queue scheduling system based on FPGA implementation according to claim 2 is characterized in that: The weight is a positive integer configured by the host or controller.
4. The WFQ multi-queue scheduling system based on FPGA implementation according to claim 2 is characterized in that: The virtual time shared by all queues is the tag of the first data packet of the last scheduled queue, and is kept until a new queue is scheduled, and is replaced by the tag of the first data packet of the new scheduled queue.
5. The WFQ multi-queue scheduling system based on FPGA implementation according to claim 2 is characterized in that: The new completion time and deviation value are obtained by the following steps: Select the larger value between the completion time and the virtual time as the start time; Calculate the difference between the packet length and the deviation value as the corrected packet length; Calculate the integer division of the modified packet length and the weight to obtain the quotient and remainder; Calculate the sum of the start time and the quotient, then add one to get the new finish time; Calculate the difference between the weight and the remainder as the new bias value.
6. The WFQ multi-queue scheduling system based on FPGA implementation according to claim 1, characterized in that: The parameter cache module, label calculation module and label scheduling module are all implemented by FPGA-based hardware circuits.
7. A WFQ multi-queue scheduling method based on FPGA, implemented based on the system according to any one of claims 1 to 6, the method comprising: The label calculation module obtains parameters from the parameter cache module, first adjusts the relevant parameters in the opposite direction according to the deviation value of the previous data packet of the same WFQ queue to eliminate the error, then calculates the new completion time and deviation value, and sends them to the parameter cache module; the new completion time is sent to the label scheduling module as the label of the first data packet of the queue; The parameter cache module caches the received parameters of each WFQ queue to record the deviation value caused by each label calculation; The label scheduling module receives and caches the head-of-queue packet label of each WFQ queue; When receiving an external signal, the label scheduling module selects the smallest label from the cached labels, sends it to the parameter cache module, deletes the label of the first packet of the queue, and schedules the first packet of the corresponding queue to be dequeued.