A token bucket algorithm current limiting system for DPU based on FPGA

Abstract

The present invention discloses a token bucket algorithm current limiting system for DPU based on FPGA, which is directly deployed on FPGA and connected to the network card of the host through PCI-e interface; the system includes system configuration module, fast memory access module, Ethernet media access controller module and two groups of user logic frame modules. Aiming at the problem of traffic burst in system link, the present invention designs a flow control method according to the token bucket principle, so that the traffic in the link will not exceed the limit of the system transceiver, and can provide different strategies according to the type of data to meet user requirements, on the basis of ensuring the safe operation of the system, without wasting the original bandwidth, solving the instability factor brought by traffic burst to DPU; it has the advantages of realizing multi-path and different data packet current limiting strategies, and there is no other dependent hardware, algorithm, protocol resources and requirements in deployment, and it can be directly used on FPGA.

Description

Technical Field

[0001] The invention belongs to the technical field of data processors, and in particular relates to a token bucket algorithm current limiting system for a DPU based on FPGA. Background Art

[0002] The Data Processing Unit (DPU) was proposed to meet the needs of data centers. The contradiction between the difficulty of increasing the performance of the Central Processing Unit (CPU) and the slowdown of Moore's Law has led to a rapid increase in the marginal cost of CPU performance growth, and the CPU has gradually become unable to meet the data processing needs in data center networks. The DPU has successfully achieved better processing capabilities of the original CPU at a lower price with a new processing method and computing structure that is independent of the CPU architecture.

[0003] However, in the process of DPU processing data transmission and reception, the traffic volume at different times is different. The sudden growth and drop of actual network traffic are unpredictable, and there is no necessary regularity at any time and space scale. In order to ensure the stability of system operation, it is necessary to clear and limit the sudden growth of traffic so that the DPU itself can maintain high throughput, low latency and load balancing, and realize effective resource allocation and rapid deployment. The present invention is a study on the data transmission and reception flow control of DPU.

[0004] Most of the existing flow control algorithms are implemented purely in software. There are two ways to handle traffic that exceeds the limit. One is to send a signal to the source address of the traffic to control the rate at which the source address sends it, thereby reducing the amount received in a recent period of time. The other is internal processing within the system, for example, using a cache area to temporarily cache the excess traffic and send it when the path traffic is idle.

[0005] As a new processor after the CPU, the existing architecture of DPU generally integrates a set of network card functions. The network card needs to implement the network-related functions mentioned above to ensure smooth and efficient data transmission. Therefore, integrating the flow control algorithm directly on its network card hardware can release the processing resources of its control unit and optimize the performance of DPU. In addition, the Field Programmable Gate Array (FPGA) shows strong performance and high flexibility in data transmission and reception control, with short development time and good cost performance. Some researchers point out that FPGA is highly competitive with microprocessors in terms of peak performance and sustained performance.

[0006] In addition to high-performance computing, current FPGA structures also provide a large amount of on-chip memory and abundant I / O pins. These features enable FPGA designs to provide high on-chip and off-chip bandwidth through the use of I / O interfaces. Therefore, FPGA-based designs can enable flow control algorithms to have great bandwidth control capabilities.

[0007] Among the existing FPGA-based flow control algorithms, the better ones include MP-RDMA proposed by Y. Lu et al. in "Multi-path transport for RDMA in datacenters" and Improved RDMA over converged Ethernet NIC (IRN) proposed by R. Mittal et al. in "Revisiting network support for RDMA". These algorithms maintain the congestion window to enable the sender to change the sending rate, and improve the retransmission mechanism when data packets are lost or received out of order to improve efficiency.

[0008] However, there are some problems with the existing technology: the above-mentioned existing technology needs to rely on a lot of additional technical costs during the deployment process, such as the need for priority flow control technology support, design focus on lossy networks, and in its specific deployment, additional resources for controlling data packet acks are required to make the algorithm adapt to the deployment environment. These features are very hardware-cost-intensive when deployed in a data center.

[0009] Therefore, the present invention proposes a token bucket algorithm current limiting system for DPU based on FPGA. Summary of the invention

[0010] In view of the problems existing in the prior art, the present invention provides a token bucket algorithm current limiting system for DPU based on FPGA; the system adopts the token technology in the software-side flow control algorithm, and uses a virtual queue to implement a flow limiting strategy for multiple paths and different data packets, and has no other dependent hardware, algorithms, protocol resources and requirements in deployment, and can be used directly on FPGA, thus solving the problem that the existing FPGA-based flow control system consumes a lot of hardware costs during the deployment process.

[0011] The present invention is implemented as follows: a token bucket algorithm current limiting system for DPU based on FPGA is arranged on the link between the host end and the network end. The system is directly deployed on the FPGA, and the system is connected to the network card of the host end through the PCI-e interface;

[0012] The system includes a system configuration module, a fast memory access module, an Ethernet media access controller module and two sets of user logic box modules: a User Logic Box@322MHz module and a User Logic Box@250MHz module;

[0013] The system configuration module, fast memory access module and Ethernet media access controller module all follow the framework design criteria provided by Open-NIC-Shell;

[0014] The system configuration module is used to store registers related to system configuration, and the system configuration module is used to store register values ​​corresponding to register modification instructions sent from the host end;

[0015] The fast memory access module is used to capture data packets sent by the host end through the PCI-e interface, and the fast memory access module is also used to send upload data packets sent by the network end to the host end;

[0016] The Ethernet media access controller module is configured as an access controller integrating 100G Ethernet media, and the Ethernet media access controller module is used to capture data packets from the network end or send data packets to the network end, thereby providing the system with broadband for sending and receiving data;

[0017] The two groups of user logic frame modules are used for transmission and control processing of data packet traffic; wherein:

[0018] The User Logic Box@322MHz module is responsible for the transmission of data packet traffic, and is automatically formed by the network card engineering after being instantiated in a default manner;

[0019] The User Logic Box@250MHz module is a module that provides core logic and is used for controlling and processing data packet traffic; the fast memory access module establishes a connection with the User Logic Box@250MHz module using the AXI-Stream protocol to send or receive data packets. The specific process is as follows:

[0020] Parse the data packets on the acquired link to identify the category and obtain the transmission path, filter and divert the classified data packets according to the data packet type, and then based on the single-rate three-color token bucket algorithm, enter the IDLE mode when the link resources are sufficient, that is, Te≥EBS_1_x, and directly forward the data packet without judging the length of the data packet. When the link resources are low to a certain threshold, that is, Te<EBS_1_x, enter the non-IDLE mode. When the length of the data packet and the remaining resources in the link meet Tc≥CBS_x_1 and Te=EBS, return to the IDLE mode without forwarding the data packet. The length of the packet is directly forwarded; where: Te represents the number of token resources that can be used for burst data flow in the current transmission path, EBS_1_x represents the lower limit of tokens for opening the current limiting mode (judgment threshold), Tc represents the number of token resources remaining in the current transmission path, CBS_x_1 represents the upper limit of tokens for closing the current limiting mode (judgment threshold), and EBS represents the excess burst size of the token bucket; Tc and Te add up to the total number of link resources, and the resources of the link are always consumed by Tc first and replenished by Te first. CBS_x_1 and EBS_1_x are user-defined parameters.

[0021] As a preferred embodiment of the present invention, the User Logic Box@250MHz module includes a data packet parsing submodule, a data path submodule, a data packet sending request judgment submodule, a protocol register submodule, a timing submodule and a token generator;

[0022] The User Logic Box@250MHz module also includes a register, in which a token bucket is provided for storing the number of tokens;

[0023] The timing submodule and the token generator are independent of the other four submodules, and the timing submodule and the token generator maintain registers according to the rules of the single-rate three-color token bucket;

[0024] The data packet parsing submodule is connected to the fast memory access module to send or receive data packets and confirm the protocol and category of the data packets, and input the category to the data packet sending request judgment submodule;

[0025] The data packet sending request judgment submodule is used to obtain the number of tokens in the register maintained in the token generator, and confirm whether the data can be sent according to the number of tokens, and give a signal to the data path submodule to allow the data to be sent or blocked;

[0026] The data path submodule is used to receive the signal of the data packet sending request judgment submodule, the data path submodule temporarily buffers the data packet in the register of the corresponding category, and transmits it to the protocol register slice submodule when it can be sent;

[0027] The protocol register slice submodule is used to couple the AXI-stream signal of the data path submodule and the AXI-stream signal of the CMACSubsystem module, and send the coupled signal to the host end or the network end.

[0028] As a preferred embodiment of the present invention, the timing submodule is used to pre-set a timer value, and when the timer reaches the value, the timer timer_pulse signal is pulled high for one cycle and the counting is restarted, and an operation of adding a token bucket token is set between two timer_pulse signals;

[0029] The timer_pulse signal is used to pull high the timer handshake signal timer_pulse_req facing the token generator, and the timing submodule handshake signal timer_pulse_req waits for the token generator to pull high the timer_pulse_ack signal to form a handshake.

[0030] As a preferred embodiment of the present invention, the data packet sending request judgment submodule includes IDLE mode, CPR mode, WAIT mode and ASSERT mode;

[0031] When the system is initialized or Tc≥CBS_x_1 and Te=EBS are detected in CPR mode, the data packet sending request judgment submodule enters IDLE mode, does not judge the length of the data packet, and directly forwards the data packet; wherein: Tc represents the number of token resources remaining in the current transmission path, Te represents the number of token resources available for burst data flow in the current transmission path, which does not exceed the excess burst size of the token bucket, CBS_x_1 represents the token upper limit of the token bucket in the current limiting mode, and EBS represents the excess burst size of the token bucket;

[0032] In the CPR mode, if the condition of returning to the IDLE mode (CBS_x_1&&Te==EBS) is not met, when the raised tvalid is received, it is determined whether the data packet can obtain the corresponding number of tokens. If yes, it enters the ASSERT mode and updates the value of Tc according to tuser_size, otherwise it enters the WAIT state; where: tuser_size is the length value of the data packet to be sent to the User Logic Box@250MHz module;

[0033] The WAIT mode is entered when a data packet cannot be sent in the CPR mode. In the WAIT mode, the register channel is closed to block the interaction between the master and slave machines until the token generator is detected to have completed a token number refresh. Then, the CPR state is re-entered to prepare to determine again whether the current data packet can obtain the corresponding token number.

[0034] In the ASSERT state, data packets begin to be sent. When the tlast signal is received as a high level, tvalid and tready on the host side are both high levels, returning to CPR mode.

[0035] As a preferred embodiment of the present invention, when the CPR mode confirms whether a data packet can obtain a corresponding number of tokens, it is set to a token lending mode;

[0036] The token lending mode includes: when the token bucket is not empty, the tokens in the bucket are emptied and the number of outstanding loans is recorded, and the current data packet is sent in advance; and when the token bucket is replenished with tokens, the outstanding loan number is checked first, and the replenished token number is used to repay the outstanding loan number first. That is, the rule for the data packet sending request judgment submodule to confirm whether the data can be sent according to the number of tokens is: when the current mode is not IDLE and the register value (token number) is in the outstanding loan state, that is, 0 or less, it is judged as not sendable, and in other cases it is judged as sendable, and the register value is updated, that is, the value of the corresponding data packet length is subtracted.

[0037] As a preferred embodiment of the present invention, the token lending mode includes: when the CPR mode confirms the judgment that the token needs to be reduced, checking the status of the token generator, if the token generator needs to add tokens, leaving the operation of reducing tokens to the ASSERT mode.

[0038] As a preferred embodiment of the present invention, the token generator manages the increase of tokens according to the handshake signal timer_pulse_req of the timing submodule;

[0039] When receiving the handshake signal timer_pulse_req from the timing submodule, the token generator pulls up the timer_pulse_ack signal to form a handshake, and runs the logic of token generation to maintain the number of tokens, while preventing the data packet sending request judgment submodule from maintaining the number of tokens to prevent conflicts.

[0040] As a preferred embodiment of the present invention, the data path submodule is used to store the data packet sent by the host for one cycle and then send it to the Register Slice submodule, thereby providing a one-cycle delay for data forwarding; the transmission data of the data path submodule is set to adopt the AXI4-stream protocol.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] The present invention aims at the problem of traffic burst in the system link and designs a traffic control method according to the token bucket principle. When the link resources are insufficient, the data packet sending of the current link is temporarily stopped and the resources are waited for recovery, so that the traffic in the link does not exceed the limit of the system transceiver, and different strategies can be provided according to the type of data to meet the user requirements. On the basis of ensuring the safe operation of the system, the data packet sending request judgment submodule is selected to enter the IDLE mode when the resource link is sufficient, so as not to waste the original bandwidth, thereby solving the instability factor brought by the traffic burst to the DPU; the token technology in the software-side traffic control algorithm is adopted, and the virtual queue method is adopted to realize the flow limiting strategy of multi-path and different data packets, and there is no other dependent hardware, algorithm, protocol resources and requirements in the deployment, and it can be directly used on the FPGA. In addition, the present invention adjusts the startup strategy of the flow limiting algorithm in a targeted manner, and can reduce the flow delay time while making the best use of the original bandwidth. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic block diagram of the system architecture provided by the present invention.

[0044] Figure 2 It is a schematic diagram of the structure of the User Logic Box@250MHz module provided in an embodiment of the present invention.

[0045] Figure 3 It is a flow chart of a token bucket algorithm provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to further understand the content, features and effects of the present invention, the following embodiments are given as examples and described in detail with reference to the accompanying drawings.

[0047] The structure of the present invention is described in detail below in conjunction with the accompanying drawings.

[0048] like Figures 1 to 3 As shown, the embodiment of the present invention provides a token bucket algorithm current limiting system for DPU based on FPGA, which is implemented at the algorithm level through Verilog language and finally completed at the hardware level;

[0049] The system includes a System Config module, a QDMA Subsystem module, a CMAC Subsystem module, and two sets of User Logic Box modules.

[0050] The System Config module, QDMA Subsystem module, and CMAC Subsystem module all follow the framework design guidelines provided by Open-NIC-Shell;

[0051] The System Config module is used to store registers related to the system configuration, and the System Config module is also used to store register values ​​corresponding to register modification instructions sent from the host side;

[0052] The QDMA Subsystem module is used to capture data packets sent by the host through the PCI-e interface. The QDMA Subsystem module is also used to send upload data packets sent by the network to the host;

[0053] The CMAC Subsystem module is set as an access controller integrated with 100G Ethernet media. The CMAC Subsystem module is used to capture data packets from the network end or send data packets to the network end, thereby providing the system with broadband for sending and receiving data;

[0054] Two sets of User Logic Box modules are used for transmission and control processing of data packet traffic.

[0055] The above technical solution is adopted: a flow control processing module of a single-rate three-color token bucket is implemented in hardware. The module is implemented in User Logic Box@250MHz and integrated in the open source project Open-NIC-Shell. It can be directly deployed in FPGA for use without other technical dependencies and is easy to deploy. After the network card obtains the data packet on the network, the data packet parsing submodule parses the data packet to identify the category and obtain the transmission path, and filters and diverts the data packets of the identified data packet type. When the link resources are sufficient, the data packet is directly forwarded without length judgment. When the link resources are low to a certain threshold, the data packet sending request judgment submodule is started in non-IDLE mode, and the data packet can be sent according to the length of the data packet and the remaining resources in the link. The idea is that the flow limiting process will definitely consume time and affect the delay and bandwidth. Direct forwarding when resources are sufficient can improve the quality of data packet transmission, so that the system can achieve stable link data packet flow and send data packets with low latency when link resources are sufficient.

[0056] The hardware architecture of the present invention is designed as follows Figure 1 As shown, the User Logic Box@250MHz is a module that provides core logic and is responsible for current limiting. Specifically:

[0057] The System Config module is responsible for storing registers related to system configuration and register values ​​of corresponding register modification instructions sent from the host side (PCIe side), and uses the AXI-lite protocol to communicate with the remaining four modules of Open-NIC (i.e., QDMA, CMAC, User Logic Box@322MHz and User Logic Box@250MHz) to change the parameters of the deployed network card. This module allows users to control the flow rate of the network card on the host after the network card is deployed;

[0058] QDMA is a DMA intellectual property core (IP) designed based on the queue concept. It can capture data packets sent by the host through the PCI-e interface, and can also send upload data packets sent by the network side to the host. In addition, the module also sends data packets to the User Logic Box@250MHz module in the AXI-Stream protocol, or receives data in this protocol;

[0059] CMAC is a very large scale integrated 100G Ethernet media access controller. This module captures data packets from the network or sends data packets to the network. This module enables the entire system to send and receive data with ultra-high bandwidth;

[0060] The User Logic Box@322MHz module is automatically formed by the network card project after being instantiated in the default way. It has no special functions and is responsible for the transmission of data streams and is coupled with the CMAC wrapper.

[0061] User Logic Box@250MHz module internal design Figure 2 , the figure shows the architecture of the host sending data packets to the network, which is applied to the QDMA send port to send to the User Logic Box@250MHz module and then to the send adapter in the CMAC (see Figure 1 ), and also shows the architecture of the host receiving data packets from the network end. This is a process of corresponding interface mirroring, which is achieved by connecting the part of the sending architecture facing the QDMA module to the receiving adapter in the CMAC, and the part facing the sending adapter in the CMAC to the receiving port of the QDMA.

[0062] The User Logic Box@250MHz module includes a data packet parsing submodule, a data path submodule, a data packet sending request judgment submodule, a Register Slice (protocol register slice) submodule, a timing submodule and a token generator;

[0063] The User Logic Box@250MHz module also includes a register, in which a token bucket is provided for storing the number of tokens;

[0064] The QDMA module sends data to the module according to the AXI-stream protocol. It will first pass through the data packet parsing submodule to confirm the data packet protocol and category to determine the transmission path, and then input the category to the data packet sending request judgment submodule. The data packet sending request judgment submodule will obtain the corresponding token number in the register maintained by the token generator to confirm whether the data can be sent, and give the data path submodule a signal to send or block the data. After receiving the signal from the data packet sending request judgment submodule, the data path submodule temporarily caches the data packet in the register of the corresponding category, and transmits it to the Register Slice (protocol register slice) submodule when it can be sent. The Register Slice submodule then couples the AXI-stream of the data path submodule and the AXI-stream signal of the next module (CMAC) and sends it. The timing submodule and token generator are independent of the other four submodules. They strictly maintain the registers storing the number of tokens according to the rules of the single-rate three-color tag token bucket.

[0065] In the algorithm design of the present invention, first, for each type of data packet, the parameters are defined as shown in Table 1:

[0066] Table 1

[0067]

[0068]

[0069] Where n is the total number of defined packet types.

[0070] like Figure 3 As shown in FIG. 1 , the token bucket algorithm process is implemented by the state machine of the data packet sending request submodule, where:

[0071] About the working principle and process of the timing module:

[0072] The timing module needs to set the timer value in advance. When the timer reaches this value, it will pull up the timer_pulse signal for one cycle and restart the counting. This signal will pull up the timer handshake signal timer_pulse_req facing the token generator, and it is necessary to wait for the token generator to pull up the timer_pulse_ack signal to form a handshake. The handshake mechanism is adopted because the core current limiting method of the token bucket algorithm lies in the addition of tokens to the token bucket, which will be introduced in the token generator section. In order for the token bucket algorithm to work properly, there must be an addition operation of the token bucket token between two timer_pulses, otherwise it will cause the possibility of missed detection of the added signal, resulting in inaccurate current limiting.

[0073] About the working principle and process of the data packet sending request submodule:

[0074] This module is written in a state machine and is divided into four modes: IDLE, CPR, WAIT, and ASSERT. After the system is initialized or when Tc ≥ CBS_x_1 and Te = EBS are detected in the CPR state, the module enters the IDLE state. In theory, the module does not need to do anything in this state, but this patent assigns it similar work as that during initialization to manage the registers related to the module to ensure state stability. In this state, the operation of updating the number of tokens is handed over to the data path submodule. Specifically, in IDLE mode, the data packet sending request module updates the number of tokens according to the value of tuser_size (reduces the number of tokens). In non-IDLE mode, it is handed over to the data path submodule to update the number of tokens (reducing the number of tokens by 1 per cycle).

[0075] In the CPR state, if the conditions for returning to the IDLE state are not met, when the high tvalid is received, it is determined whether the data packet can obtain the corresponding number of tokens. If so, it enters the ASSERT state and updates the value of Tc according to tuser_size, otherwise it enters the WAIT state.

[0076] The WAIT state is entered when a data packet cannot be sent in CPR. In this state, the register channel will be closed, blocking the interaction between the master and slave machines. It will not re-enter the CPR state until it detects that the token generator has completed a token number refresh and prepares to determine again whether the current data packet can obtain the corresponding token number.

[0077] In the ASSERT state, the data packet starts to be sent until the tlast signal is received as a high level, indicating that the last beat of the current data packet has arrived. At this time, if the tvalid and tready of the host are both high levels, the data of the last beat will also be stored by the register transmission path and return to the CPR state. In addition, in the ASSERT state, some token number update operations that were not completed in the CPR state may be completed.

[0078] When CPR confirms whether a data packet can pass, a token lending mode is used. Because the original processing mechanism uses a flow limiting strategy when it detects that the number of tokens in the CBS and EBS buckets is less than tuser_size, this method cannot fully utilize the remaining tokens in the CBS bucket. If there is a burst of traffic or high-speed traffic, the data packet may be "pre-limited". The lending mechanism allows the tokens in the bucket to be emptied and the number of outstanding loans to be recorded when the CBS bucket is not empty, and the current data packet is sent in advance. Compared with the original judgment method, this method does not affect the implementation of the flow limiting function, and can send data packets in advance for subsequent module processing. When replenishing the number of tokens, it will first check whether there is an outstanding loan, and repay the outstanding loan first.

[0079] Exemplarily, the token consumption pseudo code of this embodiment is shown in Table 2:

[0080] Table 2

[0081]

[0082]

[0083] The maintenance of the number of tokens involves a problem, that is, both the packet sending request submodule and the token generator will change the number of tokens. This patent adopts a method of specifying priorities, stipulating that adding tokens always takes precedence over subtracting tokens. When the module is in the CPR state and it is determined that tokens need to be reduced, the state of the token adding module is checked. If it needs to add a module, the operation of subtracting tokens is left to the ASSERT module. This method effectively solves the problem of addition and subtraction conflicts.

[0084] About the working principle and process of the token generator:

[0085] The token unit in the token bucket corresponds to tuser_size, which is set to B. The token generator manages the increase of tokens according to the handshake signal of the timer module. Set CIR_VALUE to the number of tokens added every timer_th clock cycle, then the two values ​​determine the allowed transmission bandwidth of the token bucket algorithm in the user logic box. When the timer_pulse_req of the timing module is set to 1, timer_pulse_ack is returned to form a handshake, and then the logic of token generation is run to maintain the number of tokens, while preventing other modules from maintaining the number of tokens to prevent conflicts. The logic of token generation is the reverse operation of Algorithm 1. In addition, the CIR_VALUE of each different data packet type is different. Each time an update is triggered, the token bucket of the next data packet category needs to be polled after the value update is completed.

[0086] In this system, the bandwidth BW (bits per second) after shaping satisfies:

[0087]

[0088] Among them, 512 is the bit width of tada, f clk is the system clock frequency, which is set to 250MHz. It is worth noting that CIR_VALUE≤timer_th should strictly hold true, and the overall allowed bandwidth is strictly less than or equal to the full bandwidth of the system. When CIR_VALUE=timer_th, in theory, it is as if there is no current limiting measure, and the actual limit bandwidth cannot reach the theoretical upper limit due to the influence of the data packet sending request judgment module.

[0089] About the working principle and process of the data path submodule:

[0090] This module implements the function of storing the data packet sent by the host for one cycle and then sending it to the protocol register slice. It provides at least one cycle of delay from the data received by the data packet parsing submodule to the protocol register slice. At the same time, the data path submodule uses a register with a depth of 2, in which the register slice using the full mode will provide an unfixed delay, and the entire module does not affect the continuity of the data.

[0091] The data transmission principle of this module complies with the AXI4-stream protocol. When the receiver (here is the protocol register slice) can receive data or its own primary register has no valid value, it pulls up the i_tready of the sender to refresh its own register value. The register value is directly connected to the sending port. When the value is valid, it pulls up r_tvalid to wait for the receiver to receive.

[0092] This channel is divided into three states, master_1, RST and master_x. In the master_1 state, the packet sending request judgment submodule enters the IDLE state. This module directly couples the sender and the protocol register chip. Each time it receives a beat of data, it updates the token number (following the priority system of addition and subtraction) until the packet sending request judgment submodule leaves the IDLE state. When the condition is met for the packet sending request judgment submodule to leave the IDLE state, it still sends the current data packet first to avoid triggering the current limiting judgment in the middle of the data packet and causing errors, and then enters the RST state. In the RST state, after detecting that a data packet has been sent, r_tvalid will be pulled down to stop sending, waiting for the next data packet to arrive, and entering the master_x or master_1 state according to the result of the CPR state of the data packet sending request judgment submodule. Master_x sends an entire data packet and enters the RST state at the last beat of the data packet.

[0093] About the working principle and process of the protocol register slice (Register Slice):

[0094] In this article, the module directly instantiates the IP core AXI4-stream Register Slice provided by AMD. After selecting the interface, adjust the working mode to auto-pipelined. The configuration of each port is shown in Table 3:

[0095] Table 3

[0096] Port Function Is it enabled? Bit width / mode TREADY yes 1 TDATA yes 512 TSTRB no NA TKEEP yes 64 TLAST yes 1 TID no NA IDEST no NA TUSER yes 50 ACLKEN no NA Pipeline type yes auto-pipelined

[0097] The present invention obtains data packets on the link through the User Logic Box@250MHz module, and the data packet parsing submodule parses the data packet to identify the category and obtain the transmission path, and screens and diverts the data packets whose data packet types have been identified. Then, based on the single-rate three-color token bucket algorithm, when the link resources are sufficient (Te≥EBS_1_x), the IDLE mode of the data packet sending request judgment submodule is entered without length judgment and the data packet is directly forwarded. When the link resources are lower than a certain threshold (Te<EBS_1_x), the IDLE mode is exited, and the non-IDLE mode of the data packet sending request judgment submodule is started. The judgment is made according to the length of the data packet and the remaining resources in the link. When Tc≥CBS_x_1 and Te=EBS are satisfied, the IDLE mode is returned to the IDLE mode, and the length of the data packet is directly judged without the data packet. The forwarded data packet is sent by the data path submodule and the protocol register slice submodule.

[0098] In summary: the present invention implements a flow control processing module of a single-rate three-color token bucket in hardware, which is integrated in the open source project Open-NIC-Shell and can be directly deployed in FPGA for use, without relying on other technologies, and is easy to deploy. The present invention adopts the token technology in the software-side flow control algorithm, which adopts the virtual queue method to implement the flow limiting strategy of multiple paths and different data packets, and has no other dependent hardware, algorithms, protocol resources and requirements in deployment, and can be used directly on FPGA. In addition, the patent specifically adjusts the startup strategy of the flow limiting algorithm, which can reduce the flow delay time while making the best use of the original bandwidth.

[0099] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0100] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A token bucket algorithm current limiting system for DPU based on FPGA, arranged on the link between the host end and the network end, characterized in that: The system is directly deployed on the FPGA, and the system is connected to the host-side network card through the PCI-e interface; The system includes a system configuration module, a fast memory access module, an Ethernet media access controller module and two sets of user logic box modules: a User Logic Box@322MHz module and a User Logic Box@250MHz module; the system configuration module is used to store registers related to system configuration, and to store register values ​​of corresponding register modification instructions sent from the host end, and to communicate with the other four modules using the AXI-lite protocol to change the parameters of the deployed network card, so that after the network card is deployed, the user can control the flow rate of the network card current limiting on the host; The fast memory access module is used to capture data packets sent by the host through the PCI-e interface, and to send upload data packets sent by the network to the host. At the same time, it establishes a connection with the User Logic Box@250MHz module through the AXI-Stream protocol to send or receive data packets. The Ethernet media access controller module is used to capture data packets from the network or send data packets to the network, thereby providing the system with broadband for sending and receiving data. Among them: The User Logic Box@322MHz module is responsible for the transmission of data packet traffic, and is automatically formed by the network card engineering after being instantiated in a default manner; The User Logic Box@250MHz module is used for controlling the data packet flow. The specific process is as follows: Parse the acquired data packets on the link to identify the category and obtain the transmission path, filter and divert the classified data packets according to the data packet type, and then based on the single-rate three-color token bucket algorithm, enter the IDLE mode when the link resources are sufficient, that is, Te≥EBS_1_x, and directly forward the data packets without judging the length of the data packets. When the link resources are lower than a certain threshold, that is, Te<EBS_1_x, enter the non-IDLE mode, judge according to the length of the data packet and the remaining resources in the link, and return to the IDLE mode when Tc≥CBS_x_1 and Te=EBS, and directly forward the data packets without judging the length of the data packets; where: Te represents the number of token resources that can be used for burst data flow in the current transmission path, EBS_1_x represents the lower limit of tokens for turning on the current limiting mode, Tc represents the number of token resources remaining in the current transmission path, CBS_x_1 represents the upper limit of tokens for turning off the current limiting mode, and EBS represents the excess burst size of the token bucket.

2. The token bucket algorithm current limiting system based on FPGA and oriented to DPU according to claim 1 is characterized in that: The User Logic Box@250MHz module includes a data packet parsing submodule, a data path submodule, a data packet sending request determination submodule, a protocol register slice submodule, a timing submodule and a token generator; The User Logic Box@250MHz module also includes a register, in which a token bucket is provided for storing the number of tokens; the timing submodule and the token generator maintain the register according to the rules of the single-rate three-color marking token bucket; The data packet parsing submodule is connected to the fast memory access module to send or receive data packets and confirm the protocol and category of the data packets to obtain the transmission path, filter and divert the data packets of the identified category according to the data packet type, and input the category to the data packet sending request judgment submodule; The data packet sending request judgment submodule is used to obtain the number of tokens in the register maintained by the token generator, and confirm whether the data can be sent according to the number of tokens; The data path submodule receives the signal of the data packet sending request judgment submodule, temporarily buffers the data packet in the register of the corresponding category, and transmits it to the protocol register slice submodule when it can be sent; The protocol register slice submodule is used to couple the AXI-stream signal of the data path submodule and the AXI-stream signal of the Ethernet media access controller module, and send the coupled signal to the host end or the network end.

3. The token bucket algorithm current limiting system based on FPGA and oriented to DPU as claimed in claim 2, characterized in that: The timing submodule pre-sets a timer value, and when the timer reaches the value, it pulls up the timer timer_pulse signal for one cycle and restarts counting. An operation of adding tokens to the token bucket is set between two timer_pulse signals; The timer_pulse signal is used to pull high the timer handshake signal timer_pulse_req facing the token generator, and the timing submodule handshake signal timer_pulse_req waits for the token generator to pull high the timer_pulse_ack signal to form a handshake.

4. The token bucket algorithm current limiting system based on FPGA and oriented to DPU as claimed in claim 2, characterized in that: The data packet sending request judgment submodule includes IDLE mode, CPR mode, WAIT mode and ASSERT mode; When the system is initialized or Tc≥CBS_x_1 and Te=EBS is detected in CPR mode, the data packet sending request judgment submodule enters IDLE mode, does not judge the length of the data packet, and directly forwards the data packet; where: Tc represents the number of token resources remaining in the current transmission path, Te represents the number of token resources available for burst data flow in the current transmission path, which does not exceed the excess burst size of the token bucket, CBS_x_1 represents the token upper limit of the token bucket in the current limiting mode, and EBS represents the excess burst size of the token bucket; In the CPR mode, if the condition for returning to the IDLE mode is not met, when the raised tvalid is received, it is determined whether the data packet can obtain the corresponding number of tokens. If yes, it enters the ASSERT mode and updates the value of Tc according to tuser_size, otherwise it enters the WAIT state; where: tuser_size is the length value of the data packet to be sent to the User Logic Box@250MHz module; The WAIT mode is entered when a data packet cannot be sent in the CPR mode. In the WAIT mode, the register channel is first closed to block the interaction between the master and slave machines, and the CPR state is re-entered to prepare to determine again whether the current data packet can obtain the corresponding token number until the token generator is detected to have completed a token number refresh. In the ASSERT state, data packets begin to be sent. When the tlast signal is received as a high level, tvalid and tready on the host side are both high levels, returning to CPR mode.

5. The token bucket algorithm current limiting system based on FPGA and oriented to DPU as claimed in claim 4, characterized in that: The CPR mode is set to token lending mode when confirming whether the data packet can obtain the corresponding number of tokens; The token lending mode includes: when the token bucket is not empty, clearing the tokens in the bucket and recording the amount of outstanding loans, and sending the current data packet in advance; When replenishing the token number in the token bucket, the outstanding number is checked first, and the replenished token number is used to repay the outstanding number first.

6. The token bucket algorithm current limiting system based on FPGA and oriented to DPU as claimed in claim 5, characterized in that: The token lending mode includes: when the CPR mode confirms the judgment that the token needs to be reduced, checking the status of the token generator, and if the token generator needs to add tokens, leaving the operation of reducing tokens to the ASSERT mode.

7. The token bucket algorithm current limiting system for DPU based on FPGA as claimed in claim 2, characterized in that: The token generator manages the increase of tokens according to the handshake signal timer_pulse_req of the timing submodule; When receiving the handshake signal timer_pulse_req from the timing submodule, the token generator pulls up the timer_pulse_ack signal to form a handshake, and runs the logic of token generation to maintain the number of tokens, while preventing the data packet sending request judgment submodule from maintaining the number of tokens to prevent conflicts.

8. The token bucket algorithm current limiting system for DPU based on FPGA as claimed in claim 2, characterized in that: The data path submodule is used to store the data packet sent by the host for one cycle and then send it to the protocol register chip submodule, thereby providing a one-cycle delay for data forwarding; the transmission data of the data path submodule is set to adopt the AXI4-stream protocol.

Images