A network load tester for assessing the effectiveness of congestion control deployment

Abstract

The application provides a network load tester for evaluating congestion control deployment effect, and relates to the field of network data monitoring or testing. The network load tester is realized by cooperation of FPGA and programmable switch. The FPGA network card is responsible for calculation of the congestion control algorithm, generates a scheduling message, and controls the programmable switch to generate a data message. The programmable switch accepts an acknowledgement message, generates a feedback message after processing, and sends the feedback message to the FPGA, so that processing and generation of a predetermined scale of flow are realized. The application simultaneously plays the high-density advantage of the programmable switch and the high processing capacity of the FPGA, amplifies the processing capacity of the FPGA through the programmable switch, and realizes low-cost Tbps-level high-density load testing. The application proposes a new three-stage rate and burst control design on the programmable switch, realizes high-precision rate burst control, and realizes high-precision real-time measurement without additional message headers.

Description

Technical Field

[0001] This invention relates to the field of network data monitoring or testing, and more particularly to a network load tester for evaluating the effectiveness of congestion control deployment. Background Technology

[0002] Test instruments used in large-scale testing scenarios must achieve high density. Common devices under test in data centers, such as switches and routers, have Tbps-level forwarding capabilities, and test instruments must generate traffic of the same level to perform effective load testing. Low-cost testing solutions are beneficial for increasing test scale and accelerating test solution iteration. Existing test instrument solutions cannot simultaneously meet the requirements of low cost and high density. Commercial test instruments have high density but are expensive, while software solutions offer no advantage in terms of density or cost.

[0003] Congestion control algorithms are the cornerstone of ensuring cloud service quality. These algorithms control the rate and characteristics of packets entering the network link. In recent years, more and more congestion control algorithms have been proposed to meet the different needs of various application scenarios. Network service providers need to select from numerous congestion control algorithms and adjust their parameters according to the actual topology. Congestion control algorithms iterate frequently, and deployment and optimization may involve modifications to the algorithms. Therefore, testing equipment should have the ability to customize congestion control algorithms, not just their parameters.

[0004] High-precision rate and burst control are fundamental to simulating real-world traffic characteristics. Traffic patterns reflect the load conditions of a data center and can test the performance of the device under test (DUT) under different stress levels, such as flow completion time, queue depth, and fairness. Burst traffic is a key characteristic of traffic patterns; burst traffic fills the bandwidth in a short period, potentially causing queue overflow in intermediate devices, leading to latency, packet loss, or even crashes. Measurement accuracy is one of the basic performance indicators of a test instrument. Measurements should not affect the structure of service packets; otherwise, it will affect the behavior of devices within the network and the statistical packet length. Summary of the Invention

[0005] Purpose of the invention: To propose a network load tester for evaluating the effectiveness of congestion control deployment, in order to solve the above-mentioned problems existing in the prior art.

[0006] This invention proposes a network load tester for evaluating the effectiveness of congestion control deployment. This tester utilizes a collaborative implementation of an FPGA and a programmable switch. The FPGA network interface card (NIC) is responsible for calculating the congestion control algorithm, generating scheduling messages, and controlling the programmable switch to generate data packets. The programmable switch receives acknowledgment messages, processes them, and generates feedback messages to send to the FPGA, thus achieving the processing and generation of a predetermined scale of traffic.

[0007] In a further embodiment, the data packet enters the programmable switch, is truncated to a length of 64 bytes by mirroring in traffic management, and is sent to the receiving logic to generate an acknowledgment packet.

[0008] After the confirmation message is processed by the programmable switch to collect statistical data, its format is modified to generate a feedback message, which is then sent to the corresponding port of the FPGA.

[0009] In a further embodiment, the scheduling message is sent in through the FPGA port, and the scheduling information in the scheduling message is stored in a queue in the outgoing direction, with each port corresponding to one queue.

[0010] Template messages are sent from the control plane to the data plane, occupying one loopback port for loopback, and multicasting line-rate traffic to all data packet output ports.

[0011] In a further embodiment, after multicasting, the message reads the queue in the outgoing direction, extracts the scheduling information, modifies the message fields, records statistical information, marks it, and then sends it out.

[0012] In a further embodiment, the FPGA allocates a stream processing core to each port of the programmable switch, and the mapping between the stream sequence number of the packet and the stream processing core is realized through the distributor and the egress multiplexer; the FPGA side includes a timeout judgment module and a congestion control algorithm module.

[0013] In a further embodiment, the FPGA side includes the following processing stages:

[0014] The receive queue generates data packet events to the congestion control algorithm module, and the timeout judgment module generates timeout events to the congestion control algorithm module.

[0015] Upon receiving a data packet event and / or a timeout event, the congestion control algorithm module reads the parameters of the corresponding flow from the congestion control algorithm parameter storage module, calculates the scheduling information through a finite state machine, updates the parameters, and issues a scheduling request.

[0016] Using an FPGA inner loop, flows with unfilled sending windows are passed through multiplexers and re-entered into the scheduling queue. The scheduler retrieves the flows to be scheduled from the scheduling queue, calculates the sequence number, generates a scheduling message, and writes the congestion control algorithm parameters back to the storage module. The rescheduling request passes through the multiplexer and the scheduling sequence number is placed into the scheduling queue.

[0017] In a further embodiment, each stream processing kernel has a receive timer and a send timer, which control the reception or transmission of a packet every certain number of clock cycles, respectively. The receive timer is used to avoid read / write conflicts caused by excessively short intervals; the send timer is used to avoid overflow of the switch scheduling queue caused by excessively rapid generation of scheduling packets. The distributor is responsible for distributing the received packets to one of the stream processing kernels, and the multiplexer is responsible for sending the packets generated by all stream processing kernels sequentially.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] (1) This invention leverages the high density advantage of programmable switches and the high processing power of FPGAs, amplifying the processing power of FPGAs through programmable switches to achieve low-cost Tbps-level high-density load testing.

[0020] (2) In the joint test system, the ability to customize message structure and fields is placed on a programmable switch that can be rapidly iterated, while relatively stable functions such as protocol processing are placed on the FPGA. Through the collaboration of the built-in communication architecture, high flexibility and rapid iteration are achieved.

[0021] (3) This invention separates traffic generation measurement and protocol processing, and deploys the congestion control algorithm on FPGA. In this process, it overcomes the problems of mismatch between switch and network card speed and frequency mismatch, realizes traffic amplification, overcomes the problem of fair scheduling of tens of thousands of flow lines, overcomes the problem of congestion control parameter read and write conflict, overcomes the problem of algorithm timing limitation, and realizes the deployment and testing of highly scalable custom congestion control protocol.

[0022] (4) This invention proposes a novel three-stage rate and burst control design for programmable switches, which realizes high-precision rate burst control and achieves high-precision real-time measurement without adding extra message headers. Attached Figure Description

[0023] Figure 1 This is an overall architecture diagram of the network load tester in this embodiment of the invention.

[0024] Figure 2 This is a schematic diagram of the various modules of the programmable switch in a network load tester.

[0025] Figure 3 This is a schematic diagram of the FPGA modules in a network load tester. Detailed Implementation

[0026] In the following description, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be practiced without one or more of these details. In other instances, certain technical features well-known in the art have not been described in order to avoid obscuring the invention.

[0027] The programmable switch used in the implementation example is a Tofino BFN-T10-032Q, which has a management port and 32 100GE panel ports. Unless otherwise specified, all ports mentioned below are 100GE panel ports. The FPGA card is a Xilinx Alevo U280, which has two 100GE ports.

[0028] This embodiment presents a network load tester for evaluating the effectiveness of congestion control deployments. The overall architecture is described in [link to documentation]. Figure 1 As shown, this method utilizes the collaboration of an FPGA and a programmable switch. The FPGA network card is responsible for calculating the congestion control algorithm, generating scheduling messages, controlling the programmable switch to generate data messages, receiving acknowledgment messages, processing them, generating feedback messages, and sending them to the FPGA, thus realizing the processing and generation of large-scale traffic.

[0029] See the internal architecture of the switch. Figure 2 :

[0030] 1. Data Packet: After the data packet arrives, it is truncated to 64 bytes in length through mirroring in traffic management, and then sent to the receiving end logic to generate an acknowledgment packet.

[0031] 2. Acknowledgment Message: After the switch processes and collects statistical data, the acknowledgment message is formatted to generate a feedback message, which is then sent to the corresponding port on the FPGA.

[0032] 3. Scheduling message: Sent in by the FPGA port, the scheduling information in the scheduling message is stored in the queue on the outgoing side, with one queue corresponding to each port.

[0033] 4. Template Message: Sent from the control plane to the data plane, occupying one loopback port for loopback, and multicasting line-rate traffic to all data packet output ports. After multicasting, the message reads the queue in the outgoing direction, extracts scheduling information, modifies message fields, records statistics, marks it, and then sends it out.

[0034] See FPGA architecture Figure 3The FPGA uses the OpenNIC open-source architecture, implementing its functionality in a 322MHz user-programmable module. Furthermore, a QDMA subsystem is used for PCIe communication with the board, a CMAC submodule for sending and receiving data packets, and a system configuration module for reset. The parser and de-parser are responsible for parsing and generating the AXI-stream protocol. The FPGA allocates a stream processing core to each port of the programmable switch, and the mapping between the packet stream sequence number and the stream processing core is implemented through a distributor and an egress multiplexer, involving three processing stages:

[0035] Phase 1: The receive queue and timeout judgment generate received packet events and timeout events respectively to the congestion control algorithm module. Upon receiving an event, the congestion control algorithm reads the parameters of the corresponding stream from the congestion control algorithm parameter storage module, calculates the scheduling information using a finite state machine, updates the parameters, and issues a scheduling request. The congestion control algorithm is a customizable module; any congestion control algorithm that meets the timing requirements and interface format can be replaced. Custom congestion control algorithm modules can be developed using C++ instead of Verilog based on high-level synthesis, reducing development iteration costs.

[0036] Phase Two: To achieve rapid transmission of packets within the congestion window, an FPGA-based internal loop is used. Packets with unfilled transmission windows pass through the multiplexer and are then re-placed into the scheduling queue. The scheduler retrieves the streams to be scheduled from the queue, calculates their sequence numbers, generates scheduling messages, and simultaneously writes the congestion control algorithm parameters back to the storage module. Rescheduling requests pass through the multiplexer, and the scheduling sequence number is placed into the scheduling queue.

[0037] Phase 3: Each stream processing kernel has a receive and send timer, controlling the reception or transmission of a packet every certain number of clock cycles. The former avoids read / write conflicts caused by too small intervals, while the latter prevents the switch scheduling queue from overflowing due to excessively rapid packet generation. The distributor is responsible for distributing the received packets to one of the stream processing kernels, and the multiplexer is responsible for sending the packets generated by all stream processing kernels sequentially.

[0038] The network load tester disclosed in this embodiment for evaluating the effectiveness of congestion control deployment can be used to evaluate the effectiveness of TCP Reno congestion control deployment. The operation method is as follows:

[0039] S1. Environment Preparation. Load the data plane program of this invention onto the programmable switch, and copy the control plane program of this invention onto the programmable switch. Connect the 12 ports (c1 to c12) of the tester to the transmitting side port of the device under test (DUT) using cables, and connect the 12 ports (s1 to s12) of the tester to the receiving side port of the DUT using cables. Connect port 29 of the switch to port 0 of the FPGA card. The programmable switch and FPGA as a whole are referred to as the tester in the following text. Configure ports 1 and 2 as loopback. Initialize the DUT, run the tester's control plane program. The control plane program will read the configuration file and set the data plane action matching table according to the parameters in the configuration file to complete the port initialization. 65536 flows will be pre-divided into 12 groups, each group corresponding to a queue. The data plane has 256*16 registers for storing queue elements. The control plane assigns a queue number to each flow, calculates the base address as 256 times the sequence number, and stores the binding relationship between flow, port, base address, and queue number in the action matching table.

[0040] S2. The control plane sends 32 control plane template messages to the tester through an internal port. Each control plane template message consists of an Ethernet header, IP header, template header, TCP header, and payload, with a total length of 1024 bytes. After entering the data plane, the control plane template message is classified as a template message in the classification module based on the IP protocol field. In the Ingress, the template header is parsed and removed, the protocol in the IP header is modified to TCP, the type in the template header is sent to the action matching table, and the outgoing port is set (at this point, the message is renamed a template message). It then enters the egress of the specified outgoing port. After exiting the egress, it loops back into the ingress of that port. In the ingress, the outgoing port is set to the same port, and it enters the egress of that port again. This cycle continues, ensuring that the first loopback port always has a line-speed template message looping back. Then, the FPGA card driver is loaded, and initialization is completed.

[0041] S3. When the first timeout event occurs, the FPGA card will abbreviate the event to the congestion control algorithm module for processing. The sequence number of flow 0 (and flows 1 to 65534 similarly) will be added to the scheduling queue. The scheduler will retrieve the sequence number from the queue, read and write the parameter BRAM, obtain the packet sequence number, and wait for the corresponding timer (12 in total) for that flow to reach the next transmission time. The timer interval is the arrival interval of a 1024B packet at a 100G rate. A scheduling message will be generated and sent from FPGA port 0 to switch port 29. After scheduling is completed, based on the congestion window and the current sequence number information, it will determine whether the flow can continue to be transmitted. If it can, the sequence number will be added to the scheduling queue; otherwise, it will be discarded.

[0042] S4. The scheduling message enters port 10 (ingress), sets the outgoing port through the action matching table, and enters port 2 (egress). Then, through the action matching table, the base address and queue number are obtained using the flow sequence number. The queue number is used to query the length register group. If the length is not equal to the maximum value, the element number is obtained by incrementing the header pointer register group's sequence number position value. The actual index of the element in the element register group is calculated using the base address and element number, and the scheduling flow number and message sequence number are stored in this index. The scheduling message is finally discarded.

[0043] S5. When a template message enters the ingress, a multicast group is set up and sent to s1 to s12. Multicast messages arriving at the corresponding port's egress are called pre-message messages. The pre-message message enters the egress, passes through the action matching table, and obtains the base address and queue sequence number using the port number. The queue sequence number is used to query the length register group. If the length is not equal to 0, the element sequence number is obtained by incrementing the tail pointer register group's sequence number position value. The actual index of the element in the element register group is calculated using the base address and element sequence number. The scheduling flow number and message sequence number at this index are read, and the corresponding TCP fields are rewritten. Other message fields are set according to the standard, and the message message is sent out. Before being sent, the message message enters the statistics module. Based on the flow sequence number and port sequence number, the number of bytes is accumulated into the flow throughput statistics and port throughput statistics registers. The 20th to 32nd bits of the timestamp are used as an index to accumulate the number of bytes into one of 4096 precise throughput statistics registers.

[0044] S6. After the message packet passes through the device under test (DUT) and arrives at the receiving port, it enters the ingress. The mirror number is obtained and set using the state matching table. The control plane pre-sets the target port for this mirror number to the receiving port, and during mirroring, it is clipped to 64 bytes. The original packet is discarded, and the mirror packet enters the port's egress. The expected sequence number register is queried and updated using the flow sequence number as the index. The TCP identifier and acknowledgment number are modified according to the TCP cumulative acknowledgment mechanism, resulting in an acknowledgment packet being sent from this port. After passing through the DUT, the acknowledgment packet arrives at the sending port, where the protocol is modified to the internal message protocol and forwarded to port 29, entering the FPGA.

[0045] S7. The FPGA receives packets through the OpenNIC framework, sends them to the receive queue, retrieves packets from the queue according to the frequency of the receive timer, parses out the flow sequence number and packet sequence number, and sends them to the congestion control algorithm module. Internally, the module processes and updates the congestion control parameters according to the algorithm specified by the Reno standard, and generates scheduling events as needed, modifies the scheduling management parameters, and adds the flow sequence number to the scheduling queue. The FPGA continuously generates scheduling, triggering the switch to send packets, repeating the above steps. The control plane periodically reads all statistical registers, calculates statistical data, and for precise throughput statistical registers, reads all registers at intervals of less than 4 seconds, calculates the difference in the values ​​stored in registers with different timestamps, and obtains the throughput within the corresponding millisecond interval.

[0046] This invention proposes a collaborative architecture between a programmable switch and an FPGA, enabling the deployment and testing of the Reno congestion control algorithm with a test throughput of 1.2 Tbps. It supports replacing any custom congestion control algorithm that meets timing constraints, and introduces high-level synthesis for the development of custom congestion control algorithm modules, significantly reducing iteration costs. Rate control based on cascaded meters and counters can precisely control traffic rate and burst modes, supporting packet-level burst control. Compared to existing switch solutions, throughput control error is reduced by more than tenfold. Throughput measurement achieves microsecond-level granularity, and delay measurement error is less than 4%.

[0047] As described above, although the invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the invention itself. Various changes in form and detail may be made without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A network load tester for evaluating the effectiveness of congestion control deployment, characterized in that, Implemented using FPGA and programmable switch in tandem: The FPGA network interface card (NIC) is responsible for calculating the congestion control algorithm, generating scheduling messages, and controlling the programmable switch to generate data packets. Specifically, this includes: The FPGA allocates a stream processing core to each port of the programmable switch, and implements the mapping between the stream sequence number of the packet and the stream processing core through the distributor and the egress multiplexer; the FPGA side includes a timeout judgment module and a congestion control algorithm module. The FPGA side includes the following processing stages: The receive queue generates a data packet event to the congestion control algorithm module, and the timeout judgment module generates a timeout event to the congestion control algorithm module. After receiving a data packet event and / or a timeout event, the congestion control algorithm module reads the parameters of the corresponding flow from the congestion control algorithm parameter storage module, calculates the scheduling information through a finite state machine, updates the parameters, and issues a scheduling request. Using an FPGA inner loop, flows with unfilled sending windows are passed through multiplexers and re-entered into the scheduling queue. The scheduler retrieves the flows to be scheduled from the scheduling queue, calculates the sequence number, generates a scheduling message, and writes the congestion control algorithm parameters back to the storage module. The rescheduling request passes through the multiplexer and the scheduling sequence number is placed into the scheduling queue. Each stream processing kernel has a receive timer and a send timer, which control the reception or transmission of a packet every certain number of clock cycles, respectively. The receive timer is used to avoid read-write conflicts caused by too small intervals, and the send timer is used to avoid the overflow of the switch scheduling queue caused by the generation of scheduling packets too quickly. The distributor is responsible for distributing the received packets to one of the stream processing kernels, while the multiplexer is responsible for sending the packets generated by all the stream processing kernels in sequence. The programmable switch receives acknowledgment messages, processes them, generates feedback messages, and sends them to the FPGA to process and generate traffic of a predetermined scale. Specifically, this includes: When a data packet enters the programmable switch, it is truncated to a length of 64 bytes through the mirroring operation in traffic management and sent to the receiving end logic to generate an acknowledgment packet. After the confirmation message is processed by the programmable switch to collect statistical data, the format is modified to generate a feedback message, which is then sent to the corresponding port of the FPGA. The scheduling message is sent in through the FPGA port, and the scheduling information in the scheduling message is stored in the queue in the outgoing direction. Each port corresponds to one queue. Template messages are sent from the control plane to the data plane, occupying one loopback port for loopback, and multicasting line-rate traffic to all data packet output ports.

2. The network load tester for evaluating the effectiveness of congestion control deployment according to claim 1, characterized in that: After multicasting, the message reads the queue in the outgoing direction, extracts the scheduling information, modifies the message fields, records statistical information, marks it, and then sends it out.

Images