High-speed network data packet processing framework based on graph node polling scheduling

Abstract

The invention discloses a high-speed network data packet processing framework based on graph node polling scheduling. The invention provides a high-speed network data packet processing framework based on graph node polling scheduling. Rapid collaborative development can be carried out based on the framework in the process of constructing a network data packet processing system. According to the invention, the user mode network drive is utilized to transmit and receive the network data packet, and the built-in protocol analyzer is utilized to automatically complete the analysis of the network data packet; a user is allowed to divide data packet processing logic into a plurality of independent processing logic units according to functions, and the independent processing logic units allow network data packet transmission through a context container and form a processing node graph. And the framework schedules each independent processing logic unit through polling to complete the overall business logic processing.

Description

Technical Field

[0001] This invention relates to the field of electronic information technology. Background Technology

[0002] When network developers use user-space network packet transceiver drivers, they often need to process network packets according to business logic. When the business logic is complex and there are many developers, problems arise such as inconsistent development styles, unclear packet flow paths, and difficulties in collaborative development. Therefore, creating a programming framework that can solve these existing technical problems becomes a pressing technical issue. Summary of the Invention

[0003] This invention addresses the aforementioned technical problems by proposing a high-speed network packet processing framework based on graph node round-robin scheduling. This framework requires administrators to divide complex processing flows into simpler processing units, i.e., graph nodes. Developers then write processing logic within their assigned graph nodes according to their functions. Packets requiring further processing are sent to designated graph nodes using the framework's built-in context container. All graph nodes together form a complex packet flow processing graph, thus completing the entire development task.

[0004] This invention proposes a high-speed network packet processing framework based on graph node round-robin scheduling, comprising the following execution steps:

[0005] 1) Establish a user-space network data transmission and reception driver model to form a unified high-speed data packet transmission and reception interface for different drivers.

[0006] 2) Establish a network packet protocol parser to parse the received network packets according to the protocol, so that users can use it directly.

[0007] 3) Establish a graph node repository to store built-in and user-defined graph node information.

[0008] 4) Create a graph node context container to cache network data packets sent from the source node to the target node.

[0009] 5) Establish a graph node scheduling thread to schedule built-in and user-defined graph nodes.

[0010] 6) In the graph node scheduling thread, the built-in and user-defined graph node processing logic is executed sequentially using the round-robin scheduling algorithm.

[0011] 7) When scheduling to different node types, built-in nodes will be processed according to the framework's fixed scheduling process, while user-defined nodes will be processed according to the user-written scheduling process.

[0012] Furthermore, a unified data packet sending and receiving interface is defined, including: an interface for batch receiving data packets and returning the number of packets received; and an interface for batch sending data packets and returning the actual number of packets sent. Global graph node types are defined, including: INPUT, PROTOCOL, OUTPUT, PROCESS, and DROP. Built-in node types include: INPUT, PROTOCOL, OUTPUT, and DROP; user-defined graph node types include PROCESS.

[0013] Subsequently, different functional responsibilities are defined according to the different node types. In the graph node types, the I NPUT node is a packet receiving node, which is specifically used to obtain data packets from the network data packet sending and receiving driver; the PROTOCOL node is a protocol node, and the data packets received from the I NPUT node will be automatically sent to the corresponding protocol node; the OUTPUT node is a packet sending node, which is specifically used to send the assembled data packets through the network data packet sending and receiving driver; the PROCESS node is a user-defined processing node, which is used to process the data packet logic written by the user. This type of node can inherit from a PROTOCOL node, thereby diverting data packets of a specific protocol to this node; the DROP node is a packet dropping node, which is specifically used to discard data packets that do not need to be processed.

[0014] Finally, the built-in nodes and user-defined nodes form a logical execution graph, which is then scheduled using the following algorithm:

[0015] ① Traverse the nodes sequentially according to the order in which they were added;

[0016] ② Obtain the type of the traversed nodes;

[0017] ③ If it is an IN NPUT or OUTPUT type node, the processing logic is executed directly;

[0018] ④ If it is another node, then get the number of cached items in the context container of that node;

[0019] ⑤ If the number of cached items is 0, skip the execution of the processing logic for that node;

[0020] ⑥ If the number of cached items is not 0, then execute the processing logic for that node;

[0021] ⑦ Clear the context container cache of the scheduled node.

[0022] This invention proposes a high-speed network packet processing framework based on graph node polling scheduling. Compared with the prior art, this invention uses a unified packet sending and receiving interface, shielding the diversity of packet sending and receiving methods and packet structures driven by different user spaces. By following a unified programming method, it makes the development style of developers more consistent, the packet flow path clearer, and collaborative development easier. Attached Figure Description

[0023] Figure 1 shows a schematic diagram of the overall structure of the present invention;

[0024] The graph includes a user-mode driven abstraction layer, a graph node processing layer, and a graph node construction layer.

[0025] The user-space driver abstraction layer is responsible for abstracting user-space packet sending and receiving drivers, such as DPDK, TAP, and TUN, and providing a unified packet sending and receiving interface.

[0026] The graph node construction layer is responsible for forming a unified node graph structure from the system's built-in graph nodes and user-defined nodes.

[0027] The graph node processing layer has built-in auxiliary libraries, including components for protocol parsing, data packet conversion, polling scheduling strategies, memory management, and transmission management. It is responsible for receiving data packets and performing business processing according to the node graph structure.

[0028] Figure 2 shows the graph node names and their connection relationships in a specific implementation example:

[0029] The custom graph nodes include ARP LEARN, GLOBAL, FORWARD, and BROADCAST nodes; the framework's built-in graph nodes include DPDK I NPUT, ETHERNET, IPv4, IPv6, ARP, TCP, UDP, I CMP, DROP, and DPDK OUTPUT nodes.

[0030] The overall process involves the DPDK I NPUT node receiving data packets, which are then parsed and processed by various protocol nodes. ARP protocol data packets are sent to the ARP LEARN node, while other protocol data packets are sent to the GLOBAL node. Data packets received by the GLOBAL node are determined to be broadcast packets; broadcast packets are sent to the BROADCAST node, and non-broadcast packets are sent to the FORWARD node. Finally, packets that need to be discarded are sent to the DROP node, and packets that need to be sent are sent to the DPDK OUTPUT node.

[0031] Figure 3 shows the test results for a specific embodiment:

[0032] With a packet size of 64 bytes, the packet forwarding rate of a specific implementation program written using this framework can reach 36.02 Mpps.

[0033] With a packet size of 128 bytes, the packet forwarding rate of a specific implementation program written using this framework can reach 33.72 Mpps.

[0034] With a packet size of 256 bytes, the packet forwarding rate of a specific implementation program written using this framework can reach 18.11 Mpps.

[0035] With a packet size of 512 bytes, the packet forwarding rate of a specific implementation program written using this framework can reach 9.65 Mpps. Detailed Implementation

[0036] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0037] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the contents disclosed below.

[0038] The technical solution of the present invention will be further described below with reference to Figures 2 and 3.

[0039] The first embodiment discloses a Layer 2 switch implemented using a high-speed network packet processing framework based on a graph node round-robin scheduling algorithm. The implementation process includes:

[0040] As shown in Figure 2, custom graph nodes are divided according to the functions of the Layer 2 switch, including ARP LEARN node, GLOBAL node, FORWARD node, and BROADCAST node.

[0041] The ARP LEARN node receives all ARP protocol packets and parses them to learn the mapping relationship between MAC addresses and ports, forming a MAC table.

[0042] The GLOBAL node receives TCP, UDP, and I CMP protocol data packets, determines whether the protocol data packet is a broadcast data packet or a unicast data packet, and sends the data packet to the BROADCAST node if it is a broadcast data packet, and sends the data packet to the FORWARD node if it is a unicast data packet.

[0043] The BROADCAST node receives all broadcast packets and sends them to various ports.

[0044] The FORWARD node queries the MAC table to find the correspondence between the destination MAC and the destination port. If a match is found, the request is forwarded to the corresponding port. If no match is found, the node actively initiates an ARP broadcast request to each port.

[0045] As shown in Figure 3, when using an Intel X710-DA4 (4x10G) network card for packet forwarding tests, with a packet size of 64 bytes, the packet forwarding rate of the Layer 2 switch written using this framework can reach 36.02 Mpps, which meets expectations.

[0046] Through the cooperation of the above custom nodes, each data packet will be executed according to the predetermined logic through the node graph. At the same time, each custom node can be developed and tested independently, the data packet flow path is relatively clear, and collaborative development is relatively easy.

[0047] During use:

[0048] A Layer 2 switch implemented using a high-speed network packet processing framework based on a graph node round-robin scheduling algorithm can correctly forward packets at Layer 2 according to a custom graph node logic.

[0049] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A high-speed network packet processing framework based on graph node round-robin scheduling, characterized by: The execution steps include the following: 1) Establish a user-space network data transmission and reception driver model to form a unified high-speed data packet transmission and reception interface for different drivers. 2) Establish a network packet protocol parser to parse the received network packets according to the protocol, so that users can use it directly. 3) Establish a graph node repository to store built-in and user-defined graph node information. 4) Create a graph node context container to cache network data packets sent from the source node to the target node. 5) Establish a graph node scheduling thread to schedule built-in and user-defined graph nodes. 6) In the graph node scheduling thread, the built-in and user-defined graph node processing logic is executed sequentially using the round-robin scheduling algorithm. 7) When scheduling to different node types, built-in nodes will be processed according to the framework's fixed scheduling process, while user-defined nodes will be processed according to the user-written scheduling process.

2. A high-speed network packet processing framework based on graph node round-robin scheduling as described in claim 1, characterized in that: Step 1) The unified high-speed data packet sending and receiving interface includes: an interface for receiving data packets in batches and returning the number of packets received; and an interface for sending data packets in batches and returning the actual number of packets sent.

3. A high-speed network packet processing framework based on graph node round-robin scheduling as described in claim 1, characterized in that: Step 2) In the graph node repository, the graph node types include: INPUT, PROTOCOL, OUTPUT, PROCESS, and DROP, where the built-in node types include: INPUT, PROTOCOL, OUTPUT, and DROP; and the user-defined graph node type includes PROCESS.

4. A high-speed network packet processing framework based on graph node round-robin scheduling as described in claim 1, characterized in that: Step 3) The graph node information includes: graph node name, graph node type, graph node initialization function, and graph node logic processing function.

5. The graph node type according to claim 3 includes items, characterized in that: In the graph node type, the INPUT node is the packet receiving node, which is specifically used to obtain data packets from the network packet sending and receiving driver.

6. The graph node type according to claim 3 includes items, characterized in that: In the graph node type, the PROTOCOL node is a protocol node. Data packets received from the INPUT node will be automatically sent to the corresponding protocol node.

7. The graph node type according to claim 3 includes items, characterized in that: In the graph node type, the OUTPUT node is the packet sending node, which is specifically used to send the assembled data packets through the network packet sending and receiving driver.

8. The graph node type according to claim 3 includes items, characterized in that: In the graph node type, the PROCESS node is a user-defined processing node used to process user-written data packet logic. This type of node can inherit from a PROTOCOL node, thereby diverting data packets of a specific protocol to this node.

9. The graph node type according to claim 3 includes items, characterized in that: In the graph node type, the DROP node is a packet dropping node, specifically used to discard data packets that do not need to be processed.

10. A high-speed network packet processing framework based on graph node round-robin scheduling as described in claim 1, characterized in that: Step 6) describes the sequential execution of built-in and user-defined graph node processing logic using a round-robin scheduling algorithm. The round-robin scheduling algorithm is as follows: ① Traverse the nodes sequentially according to the order in which they were added; ② Obtain the type of the traversed nodes; ③ If it is an INPUT or OUTPUT type node, the processing logic will be executed directly; ④ If it is another node, then get the number of cached items in the context container of that node; ⑤ If the number of cached items is 0, skip the execution of the processing logic for that node; ⑥ If the number of cached items is not 0, then execute the processing logic for that node; ⑦ Clear the context container cache of the scheduled node.

11. A high-speed network packet processing framework based on graph node round-robin scheduling as described in claim 1, characterized in that: Step 7) The built-in nodes will be processed according to the framework's fixed scheduling process, while user-defined nodes will be processed according to the user-written scheduling process. The built-in INPUT node processing logic steps are as follows: ① Use the corresponding network packet transceiver driver to obtain data packets; ② If no data packet is received, stop receiving packets and wait for the next scheduling; ③ If a data packet is received, the network packet protocol parser is invoked to parse the data packet according to the protocol. ④ The parsed data packets are sent to the corresponding PROTOCOL node according to the protocol type. The PROTOCOL node processing logic steps are as follows: ① Determine whether the node is inherited by a certain PROCESS node; ② If it is inherited, it will be automatically sent to the corresponding PROCESS node; ③ If it is not inherited, determine whether it can still be sent to the next level PROTOCOL node; ④ If there is no next-level PROTOCOL node, then send to the DROP node; ⑤ If there is a next-level PROTOCOL node, then send it to the next PROTOCOL node. The built-in OUTPUT node processing logic steps are as follows: ① Use the corresponding network data packet transceiver driver to send data packets; ②If sending fails, it is sent to the DROP node. The built-in DROP node processing logic directly releases network data packets from memory. The user-defined PROCESS node will be executed according to the processing logic written by the user.