Network load tester for evaluating congestion control deployment effect

Abstract

The invention provides a network load tester for evaluating a congestion control deployment effect, and relates to the field of network data monitoring or testing. The network load tester is realized through cooperation of an FPGA and a programmable switch, an FPGA network card is responsible for calculation of a congestion control algorithm, a scheduling message is generated, and the programmable switch is controlled to generate a data message; and the programmable switch receives the acknowledgement message, processes the acknowledgement message, generates a feedback message and sends the feedback message to the FPGA, so that processing and generation of the preset-scale traffic are realized. According to the invention, the high-density advantage of the programmable switch and the high processing capability of the FPGA are exerted at the same time, the processing capability of the FPGA is amplified through the programmable switch, and the low-cost Tbps-level high-density load test is realized. According to the invention, a novel three-stage rate and burst control design on the programmable switch is provided, high-precision rate burst control is realized, and high-precision real-time measurement is realized without adding an additional message header.

Description

Technical Field

[0001] The invention relates to the field of network data monitoring or testing, and in particular to a network load tester for evaluating the deployment effect of congestion control. Background Art

[0002] Testers used in large-scale testing scenarios must achieve high density. Common devices under test in data centers, such as switches and routers, have Tbps-level forwarding capabilities. Testers must generate the same level of traffic to perform effective load testing. Low-cost testing solutions are conducive to increasing the scale of testing and accelerating the iteration of testing solutions. Existing tester solutions cannot meet both low cost and high density. Commercial testers have high density but high cost, and software solutions have no advantages in density and cost.

[0003] Congestion control algorithms are the cornerstone of ensuring the quality of cloud services. Congestion control algorithms control the rate and characteristics of packets entering network links. In recent years, more and more congestion control algorithms have been proposed to meet different needs in different application scenarios. Network service providers need to choose from a variety of congestion control algorithms and adjust the congestion control algorithm parameters based on the actual topology. Congestion control algorithms are frequently iterated, and deployment and tuning may be accompanied by algorithm modifications. Testers should have the ability to customize congestion control algorithms rather than just parameters.

[0004] High-precision rate and burst control are the basis for simulating real traffic characteristics. The traffic pattern reflects the load of the data center and can test the performance of the device under different pressure scales, such as flow completion time, queue depth, fairness, etc. Burst traffic is an important feature of the traffic pattern. Burst traffic fills up the bandwidth in a short time, which may cause queue overflow of intermediate devices, causing delays, packet loss, or even crashes. Measurement accuracy is one of the basic indicators of the tester. The measurement should not affect the structure of the service message, otherwise it will affect the behavior of the network device and the statistical message length. Summary of the invention

[0005] Purpose of the invention: To propose a network load tester for evaluating the effect of congestion control deployment, so as to solve the above problems existing in the prior art.

[0006] The present invention proposes a network load tester for evaluating the deployment effect of congestion control, which is implemented by using FPGA and programmable switch in collaboration. The FPGA network card is responsible for calculating the congestion control algorithm, generating scheduling messages, and controlling the programmable switch to generate data messages; the programmable switch receives the confirmation message, generates a feedback message after processing, and sends it to the FPGA to realize the processing and generation of the predetermined scale of traffic.

[0007] In a further embodiment, the data message enters the programmable switch, is truncated to 64B length through the mirroring operation in the traffic management, is sent to the receiving end logic, and generates an acknowledgment message; After the confirmation message is processed by the programmable switch to collect statistical data, the format is modified to generate a feedback message and sent to the corresponding port of the FPGA.

[0008] In a further embodiment, the scheduling message is sent in by the FPGA port, and the scheduling information in the scheduling message is stored in a queue in the outgoing direction, and each port corresponds to a queue; The template message is sent from the control plane to the data plane, occupies a loopback port for loopback, and sends line-speed traffic to all data message outbound ports through multicast.

[0009] In a further embodiment, the multicast message reads the queue in the outgoing direction, extracts the scheduling information, modifies the message field, records the statistical information, and sends it out after marking.

[0010] In a further embodiment, the FPGA allocates a flow processing core to each port of the programmable switch, and realizes the mapping between the flow sequence number of the message and the flow processing core through the distributor and the export multiplexer; the FPGA side includes a timeout judgment module and a congestion control algorithm module.

[0011] In a further embodiment, the FPGA side includes the following processing stages: The receiving queue generates a data packet event to the congestion control algorithm module, and the timeout judgment module generates a timeout event to the congestion control algorithm module.

[0012] After receiving a data packet event and / or a timeout event, the congestion control algorithm module reads the parameters of the corresponding flow from the congestion control algorithm parameter storage module, calculates the information to be scheduled through a finite state machine, updates the parameters, and issues a scheduling request.

[0013] Using the FPGA internal loop method, the flow with an unfilled sending window passes through the multi-way selector and is put back into the scheduling queue; the scheduler takes the flow to be scheduled from the scheduling queue, calculates the sequence number, generates a scheduling message, and writes the congestion control algorithm parameters back to the storage module; the rescheduling request passes through the multi-way selector and puts the scheduling sequence number into the scheduling queue.

[0014] In a further embodiment, each flow processing core has a receiving timer and a sending timer, which respectively control the receiving or sending of a message every several clocks; the receiving timer is used to avoid read-write conflicts caused by too small intervals; the sending timer is used to avoid the scheduling message generation too fast causing the switch scheduling queue to overflow. The distributor is responsible for distributing the received message to one of the flow processing cores, and the multiplexer is responsible for sending the messages generated by all flow processing cores in sequence.

[0015] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention takes advantage of both the high density of programmable switches and the high processing capability of FPGAs, amplifies the processing capability of FPGAs through programmable switches, and implements low-cost Tbps-level high-density load testing.

[0016] (2) In the joint test system, the customization capabilities of message structure and fields are placed on a programmable switch that can be quickly iterated, and relatively stable functions such as protocol processing are placed on the FPGA. Through the collaboration of the built-in communication architecture, high flexibility and rapid iteration are achieved.

[0017] (3) The present invention separates traffic generation measurement and protocol processing, and deploys the congestion control algorithm on the FPGA. In this process, it overcomes the rate mismatch and frequency mismatch problems between the switch and the network card, realizes traffic amplification, overcomes the problem of fair scheduling of 10,000 streams, overcomes the problem of congestion control parameter reading and writing conflicts, overcomes the algorithm timing limitation problem, and realizes a highly scalable custom congestion control protocol deployment test.

[0018] (4) The present invention proposes a novel three-stage rate and burst control design on a programmable switch, which realizes high-precision rate burst control and achieves high-precision real-time measurement without adding additional message headers. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is an overall architecture diagram of a network load tester in an embodiment of the present invention.

[0020] Figure 2 This is a schematic diagram of the modules of the programmable switch in the network load tester.

[0021] Figure 3 This is a schematic diagram of the FPGA modules in the network load tester. DETAILED DESCRIPTION

[0022] In the following description, a large number of specific details are provided to provide a more thorough understanding of the present invention. However, it is apparent to those skilled in the art that the present invention can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present invention, some technical features known in the art are not described.

[0023] The programmable switch used in the implementation example is Tofino BFN-T10-032Q, which has a management port and 32 100GE panel ports. The ports mentioned later are 100GE panel ports unless otherwise specified as management ports. The FPGA card is Xilinx Alevo U280, which has 2 100GE ports.

[0024] This embodiment proposes a network load tester for evaluating the effect of congestion control deployment. The overall architecture is shown in Figure 1 As shown, FPGA and programmable switch are used to achieve the collaboration. The FPGA network card is responsible for calculating the congestion control algorithm, generating scheduling messages, and controlling the programmable switch to generate data messages. The programmable switch receives confirmation messages, processes them, and generates feedback messages to send to the FPGA, thereby realizing the processing and generation of large-scale traffic.

[0025] The internal architecture of the switch is shown in Figure 2 : 1. Data message: After the data message enters, it is truncated to 64B length through the mirroring operation in traffic management, sent to the receiving end logic, and generates a confirmation message 2. Confirmation message: After the switch processes the confirmation message and collects statistical data, it modifies the format to generate a feedback message and sends it to the corresponding port of the FPGA 3. Scheduling message: sent in by the FPGA port, and the scheduling information in the scheduling message is stored in the queue in the outbound direction. Each port corresponds to a queue.

[0026] 4. Template message: sent from the control plane to the data plane, occupying a loopback port for loopback, and sending line-speed traffic to all data message outbound ports through multicast. The multicast message reads the queue in the outbound direction, extracts the scheduling information, modifies the message field, records the statistical information, and marks it before sending it out.

[0027] The architecture of FPGA is shown in Figure 3 , FPGA uses the OpenNIC open source architecture and implements its functions in a 322MHz user programmable module. In addition, a QDMA subsystem is used to implement communication between PCIe and the board, a CMAC submodule is used to send and receive data packets, and a system configuration module is used for reset. The parser and deparser are responsible for parsing and generating the AXI-stream protocol. The FPGA assigns a stream processing core to each port of the programmable switch, and the mapping of the message stream sequence number and the stream processing core is realized through the distributor and the export multiplexer. There are three processing stages: Phase 1: The receiving queue and timeout judgment generate received data packet events and timeout events to the congestion control algorithm module respectively. After receiving the event, the congestion control algorithm reads the parameters of the corresponding flow from the congestion control algorithm parameter storage module, calculates the information to be scheduled through the finite state machine, updates the parameters, and issues a scheduling request. The congestion control algorithm is a customizable module. Any congestion control algorithm that meets the timing requirements and interface format can be replaced. Based on high-level synthesis, a custom congestion control algorithm module can be developed in C++ instead of Verilog to reduce the cost of development iterations.

[0028] Phase 2: In order to quickly send messages within the congestion window, the FPGA internal loop is used to put the flow whose sending window is not full through the multiplexer and put it back into the scheduling queue. The scheduler takes the flow to be scheduled from the scheduling queue, calculates the sequence number, generates the scheduling message, and writes the congestion control algorithm parameters back to the storage module. The rescheduling request passes through the multiplexer and puts the scheduling sequence number into the scheduling queue.

[0029] Phase 3: Each flow processing core has a receiving and sending timer, which controls the receiving or sending of a message every several clocks. The former avoids read-write conflicts caused by too small intervals, and the latter avoids the overflow of the switch scheduling queue caused by too fast generation of scheduling messages. The distributor is responsible for distributing the received messages to one of the flow processing cores, and the multiplexer is responsible for sending the messages generated by all flow processing cores in sequence.

[0030] The network load tester for evaluating the deployment effect of congestion control disclosed in this embodiment can be used to evaluate the deployment effect of TCP Reno congestion control. The operation method is as follows: S1. Prepare the environment. Load the data plane program of the present invention onto the programmable switch, and copy the control plane program of the present invention onto the programmable switch. Connect the 12 ports (denoted as c1 to c12) of the tester to the sending side port of the device under test with cables, and connect the 12 ports (denoted as s1 to s12) to the receiving side port of the device under test with cables, connect the switch port 29 to the FPGA card port 0, and the programmable switch and FPGA as a whole are referred to as the tester below. Configure ports 1 and 2 as loopbacks. Initialize the device under test, run the control plane program of the tester, the control plane program will read the configuration file, and set the data plane action matching table according to the parameters in the configuration file to complete the port initialization. The 65536 flows will be divided into 12 groups in advance, each group corresponds to a queue, the data plane has 256*16 registers for storing queue elements, the control plane assigns a queue number to each flow, calculates the base address as 256 times the sequence number, and stores the binding relationship between the flow, port, base address, and queue number in the action matching table.

[0031] S2. The control plane sends 32 control plane template messages to the tester through the internal port. The control plane template message consists of an Ethernet header, an IP header, a template header, a TCP header, and a payload, with a total length of 1024B. After the control plane template message enters the data plane, it is classified as a template message in the classification module according to the IP protocol field. The template header is parsed and removed in Ingress, the protocol in the IP header is modified to TCP, the type in the template header is sent to the action matching table, the egress port is set (the message is now called a template message), and it enters the egress of the specified egress port. After exiting egress, it loops back to enter the port ingress, sets the egress port in ingress to the same port, and enters the port egress again. In this cycle, the template message at line speed will be looped back in the loopback port No. 1. Then load the driver of the FPGA card to complete the initialization.

[0032] S3. When the first timeout event occurs, the FPGA card will shrink the event into the congestion control algorithm module for processing, add the sequence number of stream 0 (same for streams 1 to 65534), and the scheduler will take the sequence number from the queue, read and write the parameter BRAM, obtain the message sequence number, wait for the timer corresponding to the stream (a total of 12) to the next sending time, and the timer interval is the arrival interval of 1024B messages at 100G rate, generate a scheduling message, send it from FPGA port 0, and enter switch port 29. After the scheduling is completed, it is determined whether the stream can continue to be sent based on the congestion window and the current sequence number information. If it can, the sequence number is put into the scheduling queue, otherwise it is discarded.

[0033] S4. The dispatched message enters port 10 ingress, sets the egress port through the action matching table, and enters port 2 egress. Then, through the action matching table, the flow sequence number is used to obtain the base address and queue sequence number, and the queue sequence number is used to query the length register group. If the length is not equal to the maximum value, the sequence number is used to increment the head pointer register group sequence number position value to obtain the element sequence number, and the base address and element sequence number are used to calculate the actual index of the element in the element register group, and the dispatched flow number and message sequence number are stored in this index. The dispatched message is finally discarded.

[0034] S5. When the template message enters ingress, a multicast group will be set and sent to s1 to s12. After the multicast message reaches the egress of the corresponding port, it is called a pre-message message. The pre-message message enters egress, passes through the action matching table, and uses the port number to obtain the base address and queue number. The queue number is used to query the length register group. If the length is not equal to 0, the tail pointer register group number position value is incremented by the number to obtain the element number. The base address and element number are used to calculate the actual index of the element in the element register group. The scheduling flow number and message number at this index are read, and the corresponding TCP field is rewritten. Other message fields are set according to the standard to obtain the message message and send it out. Before sending, the message message will enter the statistics module, and the number of bytes will be accumulated into the flow throughput statistics and port throughput statistics registers according to the flow number and port number, and the 20th to 32nd bits of the timestamp will be used as the index to accumulate the number of bytes into one of the 4096 precise throughput statistics registers.

[0035] S6. After passing through the device under test, the message reaches the receiving port and enters ingress. The mirror number is obtained and set through the state matching table. The control plane will set the target port of the mirror number to the receiving port in advance, and cut it to 64B when setting the mirror. The original message is discarded, and the mirror message enters the port egress. The expected sequence number register is queried and updated using the flow sequence number as the index. The TCP identifier and response sequence number are modified according to the TCP cumulative confirmation mechanism, and the response message is sent from the port. After passing through the device under test, the response message reaches the sending port, the protocol is modified to the internal message protocol, forwarded to port 29, and enters the FPGA.

[0036] S7. FPGA receives the message through the OpenNIC framework, sends it to the receiving queue, takes out the message according to the frequency of the receiving timer, parses the flow number and message number, and sends it to the congestion control algorithm module. It processes and updates the congestion control parameters according to the algorithm specified in the Reno standard, and generates scheduling events according to the situation, modifies the scheduling management parameters, and puts the flow number into the scheduling queue. FPGA continuously generates scheduling, triggers the switch to send messages, and repeats the above steps. The control plane reads all statistical registers regularly and calculates the statistical data. For the precise throughput statistical register, it is necessary to read all registers at intervals of less than 4s, calculate the difference between the values ​​stored in the registers with different timestamps, and obtain the throughput in the corresponding millisecond interval.

[0037] The present invention proposes a collaborative architecture of programmable switches and FPGAs, and implements the deployment test of the Reno congestion control algorithm, with a test traffic of 1.2Tbps. It supports the replacement of any custom congestion control algorithm that meets the timing constraints, and introduces high-level synthesis to develop custom congestion control algorithm modules, which significantly reduces the iteration cost. The rate control based on cascade meters and counters can accurately control the traffic rate and burst mode, support packet-level burst control, and reduce the throughput control error by more than ten times compared to existing switch solutions. The throughput measurement reaches microsecond granularity, and the delay measurement error is less than 4%.

[0038] As described above, although the present invention has been shown and described with reference to specific preferred embodiments, it should not be construed as limiting the present invention itself. Various changes in form and details may be made without departing from the spirit and scope of the present invention as defined in the appended claims.

Claims

1. A network load tester for evaluating the effect of congestion control deployment, characterized in that: Utilize FPGA and programmable switches to achieve: The FPGA network card is responsible for calculating the congestion control algorithm, generating scheduling messages, and controlling the programmable switch to generate data messages; the programmable switch receives the confirmation message, processes it, and generates a feedback message to send to the FPGA to realize the processing and generation of predetermined scale traffic.

2. The network load tester for evaluating the deployment effect of congestion control according to claim 1, characterized in that: The data message enters the programmable switch, is truncated to 64B in length through the mirroring operation in traffic management, and is sent to the receiving end logic to generate a confirmation message; After the confirmation message is processed by the programmable switch to collect statistical data, the format is modified to generate a feedback message and sent to the corresponding port of the FPGA.

3. The network load tester for evaluating the deployment effect of congestion control according to claim 2, characterized in that: The scheduling message is sent in by the FPGA port, and the scheduling information in the scheduling message is stored in the queue in the outbound direction. Each port corresponds to a queue; The template message is sent from the control plane to the data plane, occupies a loopback port for loopback, and sends line-speed traffic to all data message outbound ports through multicast.

4. The network load tester for evaluating the deployment effect of congestion control according to claim 3, characterized in that: The multicast message reads the queue in the outbound direction, extracts the scheduling information, modifies the message field, records the statistical information, and sends it out after marking.

5. The network load tester for evaluating the deployment effect of congestion control according to claim 1, characterized in that: FPGA allocates a flow processing core to each port of the programmable switch, and realizes the mapping between the flow sequence number of the message and the flow processing core through the distributor and the export multiplexer; the FPGA side includes a timeout judgment module and a congestion control algorithm module.

6. The network load tester for evaluating the deployment effect of congestion control according to claim 5, characterized in that: The FPGA side includes the following processing stages: The receiving queue generates a data packet event to the congestion control algorithm module, and the timeout judgment module generates a timeout event to the congestion control algorithm module; After receiving a data packet event and / or a timeout event, the congestion control algorithm module reads the parameters of the corresponding flow from the congestion control algorithm parameter storage module, calculates the information to be scheduled through a finite state machine, updates the parameters, and issues a scheduling request.

7. The network load tester for evaluating the deployment effect of congestion control according to claim 5, characterized in that: The FPGA side includes the following processing stages: Using the FPGA internal loop method, the flow with an unfilled sending window passes through the multi-way selector and is put back into the scheduling queue; the scheduler takes the flow to be scheduled from the scheduling queue, calculates the sequence number, generates a scheduling message, and writes the congestion control algorithm parameters back to the storage module; the rescheduling request passes through the multi-way selector and puts the scheduling sequence number into the scheduling queue.

8. The network load tester for evaluating the deployment effect of congestion control according to claim 6 or 7, characterized in that: Each flow processing core has a receiving timer and a sending timer, which respectively control the receiving or sending of a message every several clocks; the receiving timer is used to avoid read-write conflicts caused by too small intervals; the sending timer is used to avoid the overflow of the switch scheduling queue caused by too fast generation of scheduling messages; The distributor is responsible for distributing the received messages to one of the stream processing cores, and the multiplexer is responsible for sending the messages generated by all stream processing cores in sequence.

Images