Data packet processing method, electronic equipment, readable medium and program product

Abstract

The invention provides a data packet processing method, which comprises the following steps: in a DPDK process, creating a memif interface and configuring the memif interface as a main mode interface; in the proxy service process, creating a memif interface and configuring the memif interface as a slave mode interface; establishing a connection between the memif interface of the DPDK process and the memif interface of the proxy service process; and after the connection is established, transmitting a data packet between the proxy service process and the DPDK process through the memif interface of the DPDK process and the memif interface of the proxy service process. The disclosure also provides an electronic device, a computer readable medium, and a computer program product. According to the invention, efficient processing of the data packet is realized, the data throughput of the system is significantly improved, the integrity of the TCP stack is retained, and the system is ensured to have good performance and stability when processing a simple protocol and a complex protocol.

Description

Technical Field

[0001] The present disclosure relates to the technical field of computer network communications, and in particular to a data packet processing method, an electronic device, a computer-readable medium, and a computer program product. Background Art

[0002] With the increasing demand for network communication, data traffic is growing rapidly, especially in high-speed network environments (such as 1G, 10G or higher bandwidth), which puts higher requirements on the data processing capabilities of network devices. Traditional network applications mostly rely on the Linux kernel protocol stack to process network data packets. However, when processing each data packet, the Linux kernel needs to frequently trigger interrupts and perform context switching between user mode and kernel mode, resulting in high CPU resource consumption and becoming a bottleneck for system performance. For connectionless and stateless protocols such as UDP, relying on the kernel protocol stack will generate unnecessary performance overhead.

[0003] To solve this problem, DPDK (Data Plane Development Kit) came into being. DPDK bypasses the kernel protocol stack and communicates directly with the network card to achieve high-speed zero-copy data transmission, significantly improving the throughput of packet processing. This makes DPDK an ideal tool for processing high-speed UDP traffic. However, DPDK itself is limited to packet forwarding and traffic monitoring, and does not provide complete application layer functions, and cannot directly process complex protocols (such as TCP or HTTP).

[0004] The current mainstream solution is to receive network card data packets through DPDK, and then connect to the application through a third-party user-mode protocol stack. Although this integration solution improves performance to a certain extent, it also has major disadvantages: large-scale code modifications are required for existing applications to adapt to the user-mode protocol stack interface; compared with the Linux kernel protocol stack, the third-party protocol stack is less stable and more prone to problems. Summary of the invention

[0005] The embodiments of the present disclosure provide a data packet processing method, an electronic device, a computer-readable medium, and a computer program product.

[0006] In a first aspect, an embodiment of the present disclosure provides a data packet processing method, including: in a DPDK process, creating a memif interface and configuring it as a master mode interface; in a proxy service process, creating a memif interface and configuring it as a slave mode interface; establishing a connection between the memif interface of the DPDK process and the memif interface of the proxy service process; after the connection is established, transmitting data packets between the proxy service process and the DPDK process through the memif interface of the DPDK process and the memif interface of the proxy service process.

[0007] In some embodiments, after establishing a connection, data packets are transmitted between the proxy service process and the DPDK process through the memif interface of the DPDK process and the memif interface of the proxy service process, including: after establishing a connection, the proxy service process creates shared memory and a circular queue, and sends its configuration information to the DPDK process; the DPDK process maps the shared memory to its own address space based on the received configuration information; when the proxy service process monitors an event notification, it queries the data packet index in the circular queue through the memif interface of the proxy service process, and reads the data packet from the shared memory.

[0008] In some embodiments, when the proxy service process monitors an event notification, the data packet index in the circular queue is queried through the memif interface of the proxy service process, and the data packet is read from the shared memory, including: creating an event notification descriptor; adding the event notification descriptor to the event monitoring mechanism of the proxy service process; when the event monitoring mechanism monitors the event notification of the event notification descriptor, the data packet index in the circular queue is queried through the memif interface of the proxy service process, and the data packet is read from the shared memory; parsing the quadruple information in the data packet; and routing the data packet to the corresponding monitoring service for processing based on the parsed quadruple information.

[0009] In some embodiments, after routing the data packet to the corresponding monitoring service for processing according to the parsed quad-tuple information, it includes: processing the routed data packet according to the standard data processing flow of the proxy service process, and writing the processed data packet into the shared memory; when the event monitoring mechanism monitors the event notification of the event notification descriptor, the DPDK process extracts the processed data packet from the shared memory; the DPDK process performs routing and addressing according to the network layer and transport layer information of the extracted data packet, and sends the data packet to the destination address.

[0010] In some embodiments, after the event notification descriptor is added to the event monitoring mechanism of the proxy service process, it includes: the DPDK process receives a data packet from the network card; the DPDK process writes the received data packet into the shared memory through the memif interface of the DPDK process, and writes the data packet index into the circular queue; the DPDK process sends an event notification to the proxy service process through the event notification descriptor.

[0011] In some embodiments, the data packet processing method also includes: the proxy service process reads the data packet from the kernel protocol stack through a socket; parses the five-tuple information in the data packet; and processes the data packet according to a standard data processing flow based on the parsed five-tuple information.

[0012] In some embodiments, the proxy service process is an Nginx process.

[0013] In some embodiments, the event notification is used to indicate a change in a data packet index in the ring queue, prompting the proxy service process or the DPDK process to read a new data packet.

[0014] In some embodiments, the data packets transmitted through the memif interface of the DPDK process and the memif interface of the proxy service process are UDP data packets, and the data packets read from the kernel protocol stack through the socket are TCP data packets.

[0015] In a second aspect, an embodiment of the present disclosure provides an electronic device, comprising: one or more processors; a memory on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the data packet processing method described in the first aspect of the embodiment of the present disclosure.

[0016] In a third aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for processing data packets described in the first aspect of the embodiment of the present disclosure is implemented.

[0017] In a fourth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the data packet processing method described in the first aspect of the embodiment of the present disclosure.

[0018] The present disclosure realizes efficient processing of UDP data packets, significantly improves the data throughput of the system, and retains the integrity of the TCP protocol stack, ensuring that the system has good performance and stability when processing simple protocols (such as UDP) and complex protocols (such as TCP). In addition, by reducing the context switching between user mode and kernel mode, the overall performance of the system is effectively improved, and it is highly reused based on the original processing flow of Nginx, reducing the difficulty of integration.

[0019] In the application scenario of a one-way network gate, this disclosure deploys DPDK and Nginx to achieve efficient transmission and processing of UDP data packets, significantly improving UDP performance. At the same time, the solution retains the ability of TCP data packets to be processed by the kernel protocol stack, ensuring the reliability and stability of complex protocols. It also has potential application value for network proxies, VPNs, and other high-performance network applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a data packet processing method according to an embodiment of the present disclosure;

[0021] Figure 2 It is an initialization flow chart of the communication between the DPDK process and the Nginx working process through the memif interface in the embodiment of the present disclosure;

[0022] Figure 3 It is an interactive flow chart of establishing a connection between a Master (DPDK process) and a Slave (Nginx process) through a memif interface according to an embodiment of the present disclosure;

[0023] Figure 4 is a data processing flow chart between Nginx, DPDK and kernel according to an embodiment of the present disclosure;

[0024] Figure 5 is a schematic diagram of the composition of an electronic device according to an embodiment of the present disclosure;

[0025] Figure 6 It is a block diagram of a data packet processing device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the technical solution of the present disclosure is described in detail below with reference to the accompanying drawings.

[0027] Example embodiments will be described more fully below with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete and to enable those skilled in the art to fully understand the scope of the present disclosure.

[0028] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0029] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0030] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded.

[0031] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0032] In this disclosure, unless otherwise specified, the following technical terms should be understood as follows:

[0033] UDP (User Datagram Protocol) is a connectionless, unreliable transport layer protocol that does not require a connection to be established before formal communication. UDP data packets are sent directly without caring whether the other party receives them successfully. Therefore, it is suitable for one-time small data transmission or scenarios with low reliability requirements.

[0034] Nginx is a high-performance HTTP and reverse proxy server that supports multiple asynchronous I / O mechanisms (such as epoll, poll, etc.) and has the characteristics of small memory usage and strong concurrent processing capabilities.

[0035] memif (Shared Memory Packet Interface) is a high-speed data transmission interface based on shared memory, which is used to transmit messages between processes on the same host. Its core mechanisms include shared memory and ring queues for data sharing and efficient management. Memif supports high-speed data transmission in Ethernet mode and is part of the DPDK ecosystem. In memif, the communicating parties interact in the roles of master and slave: the Master is responsible for creating sockets and listening for connection requests; the Slave is responsible for creating and initializing shared memory and actively initiating connections. A one-to-one connection is established between each master and slave, and the peer interface is identified by the ID parameter.

[0036] Unix Domain Socket (UDS) is an IPC (Inter-Process Communication) mechanism used for inter-process communication on the same host, supporting the transmission of control information and data through sockets.

[0037] The four-tuple describes the basic information of the network data packet, including source IP, destination IP, source port and destination port, which is used to match the connection of UDP data packets.

[0038] The quintuple adds a protocol type (such as TCP / UDP) to the quadruple to manage the connection and status of TCP data packets.

[0039] The kernel protocol stack refers to the network protocol processing module in the operating system kernel, which is responsible for managing data transmission and state maintenance of complex protocols (such as TCP).

[0040] A one-way optical switch is a physical isolation device used to ensure that data between different networks or security domains can only be transmitted in one direction. Its hardware architecture usually includes an external terminal, an internal terminal, and a one-way isolation component, supporting high availability (HA) functions to ensure business continuity.

[0041] AS mode (Active-Standby) is a dual-machine hot standby mode, where one device is in the active state (ACTIVE) to provide services, and the other is in the standby state (STANDBY). When the main device fails, the backup device quickly takes over the service.

[0042] DPDK (Data Plane Development Kit) is a set of packet processing tools that runs in user mode, bypassing the kernel protocol stack and communicating directly with the network card to achieve high-speed packet transmission and processing. The core features of DPDK include: zero-copy data transmission; huge page memory support; multi-queue processing and lock-free ring queues. DPDK is widely used in high-performance network scenarios such as network forwarding, load balancing and traffic monitoring.

[0043] A red-black tree is a balanced binary search tree used to efficiently manage and find active connections.

[0044] The data packet processing method disclosed in the present invention aims to solve the following problems:

[0045] 1. Optimize the problem of insufficient UDP throughput performance to meet the demand for high-performance network transmission;

[0046] 2. While optimizing UDP performance, retain the use of the Linux kernel protocol stack to ensure the processing stability of complex protocols such as TCP;

[0047] 3. Reduce changes to existing systems, achieve efficient collaboration between user-mode and kernel-mode network processing, and avoid affecting original business logic.

[0048] This technical solution integrates DPDK with Nginx to provide a solution that takes into account both performance and stability. In this solution, a communication connection is established between the DPDK process and the Nginx process through the memif interface to achieve high-speed transmission of UDP data packets, and the integrated solution is deployed in application scenarios such as one-way network gates. Through this architecture, the processing performance of UDP is greatly improved.

[0049] At the same time, the original business logic of Nginx remains unchanged: for complex protocols such as TCP, Nginx continues to use the kernel protocol stack to send, receive and process data packets to ensure the compatibility and stability of the system. The focus of this solution is not to directly optimize the proxy service logic of Nginx, but to receive UDP data packets in user mode by adding a new memif interface path, which coexists with the original kernel mode processing method to achieve efficient collaboration between user mode and kernel mode.

[0050] With the help of this architectural design, Nginx can flexibly select paths based on the data source (memif interface or kernel protocol stack) when processing data packets, thereby optimizing UDP data packet transmission performance and achieving reliable transmission of the TCP protocol without affecting the original business logic.

[0051] Figure 1 It is a flow chart of a data packet processing method according to an embodiment of the present disclosure.

[0052] First, refer to Figure 1 The present disclosure provides a method for processing a data packet, including:

[0053] S11. In the DPDK process, create a memif interface and configure it as a master mode interface.

[0054] S12. In the proxy service process, create a memif interface and configure it as a slave mode interface;

[0055] S13, establishing a connection between the memif interface of the DPDK process and the memif interface of the proxy service process;

[0056] S14: After the connection is established, data packets are transmitted between the proxy service process and the DPDK process through the memif interface of the DPDK process and the memif interface of the proxy service process.

[0057] In some embodiments, the DPDK process calls the provided interface function to create a memif interface, and sets it as a master mode interface, while listening to connection requests from a slave mode interface (proxy service process) through a domain socket; the proxy service process adds a memif interface creation mechanism, sets it as a slave mode interface, and sends a connection request to the master mode interface through a domain socket, and then waits for a response from the master mode interface. Through the above process, a connection is established between the memif interface of the DPDK process and the memif interface of the proxy service process.

[0058] In some embodiments, after the connection is established, transmitting a data packet between the proxy service process and the DPDK process through the memif interface of the DPDK process and the memif interface of the proxy service process includes:

[0059] After the connection is established, the proxy service process creates a shared memory and a circular queue, and sends its configuration information to the DPDK process;

[0060] The DPDK process maps the shared memory to its own address space according to the received configuration information;

[0061] When the proxy service process monitors the event notification, the data packet index in the ring queue is queried through the memif interface of the proxy service process, and the data packet is read from the shared memory.

[0062] In the disclosed embodiment, after the connection is established, the proxy service process creates shared memory and a circular queue for common access by the proxy service process and the DPDK process; the proxy service process sends the configuration information to the DPDK process through the memif interface; the DPDK process maps the shared memory to its own address space, and realizes efficient data exchange through the initialized circular queue.

[0063] In some embodiments, when the proxy service process monitors the event notification, querying the data packet index in the ring queue through the memif interface of the proxy service process and reading the data packet from the shared memory includes:

[0064] Create an event notification descriptor;

[0065] Adding the event notification descriptor to the event monitoring mechanism of the proxy service process;

[0066] When the event monitoring mechanism monitors the event notification of the event notification descriptor, the data packet index in the circular queue is queried through the memif interface of the proxy service process, and the data packet is read from the shared memory;

[0067] Parsing the four-tuple information in the data packet;

[0068] According to the parsed four-tuple information, the data packet is routed to the corresponding monitoring service for processing.

[0069] In some embodiments, after routing the data packet to the corresponding monitoring service for processing according to the parsed quad-tuple information, the method includes:

[0070] Processing the routed data packets according to the standard data processing flow of the proxy service process, and writing the processed data packets into the shared memory;

[0071] When the event monitoring mechanism monitors the event notification of the event notification descriptor, the DPDK process extracts the processed data packet from the shared memory;

[0072] The DPDK process performs routing and addressing according to the network layer and transport layer information of the extracted data packet, and sends the data packet to the destination address.

[0073] In some embodiments, after adding the event notification descriptor to the event monitoring mechanism of the proxy service process, the following steps are included:

[0074] The DPDK process receives a data packet from the network card;

[0075] The DPDK process writes the received data packet into the shared memory through the memif interface of the DPDK process, and writes the data packet index into the ring queue;

[0076] The DPDK process sends an event notification to the proxy service process through the event notification descriptor.

[0077] In some embodiments, the data packet processing method further includes:

[0078] The proxy service process reads data packets from the kernel protocol stack via a socket;

[0079] Parsing the five-tuple information in the data packet;

[0080] According to the parsed five-tuple information, the data packet is processed according to the standard data processing flow.

[0081] In some embodiments, the proxy service process is an Nginx process.

[0082] Those skilled in the art should understand that Nginx is chosen because its processing flow is highly compatible with the memif interface. When integrating, it is only necessary to adjust the data receiving method of the Nginx underlying layer without modifying the business logic. In addition to Nginx, other proxy frameworks can also transmit data through the memif interface. For example, the proxy service built using the libuv library requires a lot of framework changes and is difficult to integrate.

[0083] Those skilled in the art should understand that Nginx communicates with the DPDK process through the memif interface to efficiently process data packets. Nginx creates a memif slave and connects to the DPDK master. The memif interface creates an event_fd (event notification descriptor) to monitor whether new data enters the shared memory.

[0084] epoll is an efficient I / O event monitoring mechanism provided by Linux. Nginx adds event_fd to the epoll mechanism to monitor data receiving events on the memif interface. When there is data to read, epoll triggers to notify Nginx to read the data. Nginx will first query the data packet index in the ring queue through the memif interface and read the data packet from the shared memory based on these indexes.

[0085] If the data packet is received from the memif interface, Nginx will parse the four-tuple information (source IP, source port, destination IP, destination port) and process the data packet according to the standard process of Nginx. The DPDK process writes the data to the shared memory through the memif interface and writes the corresponding index to the ring queue, and then notifies Nginx to read the data through event_fd.

[0086] In some embodiments, the event notification is used to indicate a change in a data packet index in the ring queue, prompting the proxy service process or the DPDK process to read a new data packet.

[0087] In some embodiments, the data packets transmitted through the memif interface are UDP data packets, and the data packets read from the kernel protocol stack through the socket are TCP data packets.

[0088] Those skilled in the art should understand that the data packets transmitted through the memif interface are usually UDP data packets, because the UDP protocol does not require complex connection management and is suitable for efficient user-mode transmission through memif. TCP data packets rely on kernel protocol stack processing. Nginx reads TCP data packets from the kernel through a socket and processes them according to the parsed five-tuple information in accordance with the standard data processing flow.

[0089] Figure 2 It is an initialization flow chart of the communication between the DPDK process and the Nginx working process through the memif interface in the embodiment of the present disclosure.

[0090] Figure 2 This paper describes in detail how to establish a connection between two processes by creating a memif interface and complete the initialization of the packet transmission channel. The initialization process includes three parts: (1) DPDK process initialization, (2) Nginx worker process initialization, and (3) DPDK process establishes a connection with Nginx worker process. Figure 2 The focus is on the initialization of the two processes. The interaction process of the two processes will be described in Figure 3 Described in detail in.

[0091] Figure 2 The three partial steps of the initialization flow are shown as follows:

[0092] (1) DPDK process initialization

[0093] Create a memif interface and set it as the main mode interface;

[0094] Listening domain socket, waiting for connection request from slave mode interface.

[0095] (2) Nginx working process initialization

[0096] Create a memif interface for each worker process and set it as a slave mode interface;

[0097] Send a connection request to the master interface through the domain socket;

[0098] The memif interface completes master-slave authentication and binding, and returns a unique identifier ID;

[0099] After the connection is established, create and initialize shared memory and a ring queue (ring) to transfer data packets between the two processes;

[0100] Create an event notification descriptor event_fd and add it to the epoll event listening mechanism, waiting for the event to be triggered.

[0101] (3) The DPDK process establishes a connection with the Nginx worker process

[0102] Complete the master-slave authentication process to ensure that the connection between the two processes is successful (see Figure 3 );

[0103] In the Nginx process, memif acts as a slave mode interface and feeds back the shared memory information to the master mode interface;

[0104] The Nginx worker process adds the received event_fd to the epoll event monitoring mechanism and waits for the event (for example, the event that the memif interface receives a data packet) to be triggered;

[0105] After the connection is established, the data packet transmission channel is initialized successfully, preparing for subsequent efficient data transmission.

[0106] Figure 3 It is an interactive flow chart of establishing a connection between a Master (DPDK process) and a Slave (Nginx process) through a memif interface according to an embodiment of the present disclosure.

[0107] like Figure 3 As shown in the figure, it shows the process of establishing a connection between the DPDK process (Master) and the Nginx process (Slave) through the memif interface. This connection is initiated by the Slave end, and the exchange of control information and negotiation of parameters are completed through the domain socket. After completing the initialization and binding of shared resources, both parties enter the data transmission state.

[0108] 1) Initial connection request

[0109] Slave (Nginx process) initiates a connection request to Master (DPDK process) through the domain socket;

[0110] The master listens to the socket port and sends a Hello message after receiving a connection request. The Hello message contains boundary information that needs to be negotiated between the two parties, such as the interface version number, supported features, and configuration parameters.

[0111] The Slave adjusts its own parameters according to the boundary information in the Hello message to ensure consistency with the Master, and then generates an identifier ID for identification and binding of subsequent processes;

[0112] 2) Init message exchange

[0113] The Slave sends an init message to the Master, which carries the identifier ID and other initialization parameters;

[0114] After receiving the init message, the Master binds the memif interface to the identifier ID to confirm the connection status of both parties;

[0115] The Master replies with an Ack response message to confirm that the init step is completed;

[0116] 3) Add queue request

[0117] The slave creates a shared memory and a ring queue to transfer packets between the master and the slave.

[0118] At the same time, the slave generates an event notification descriptor (event_fd) to trigger events during subsequent data transmission;

[0119] The Slave sends an add request to the Master, which contains information about the shared memory and the ring queue, notifying the Master that these resources are ready;

[0120] After receiving the add request, the Master sends an Ack response message to confirm that it has received the information and is ready to proceed to the next step;

[0121] 4) Connection confirmation

[0122] After receiving the Master's Ack, the Slave sends a connection confirmation request;

[0123] After receiving the connection confirmation request, the Master maps the shared memory and circular queue created by the Slave to its own address space and initializes the circular queue;

[0124] The Master sends a "connected" message to the Slave, indicating that the connection of the memif interface is successfully established;

[0125] After receiving the connected message, the Slave adds event_fd to the epoll event monitoring mechanism, waits for the event to be triggered, and prepares for data transmission.

[0126] Figure 4 It is a data processing flow chart among Nginx, DPDK and kernel according to an embodiment of the present disclosure.

[0127] Figure 4 The figure shows the different data paths and interaction processes of Nginx, DPDK, and the kernel when processing TCP and UDP data. The black line represents the processing path of UDP data, and the gray line represents the processing path of TCP data. Figure 4 It also focuses on describing how to efficiently transmit UDP data in user mode through the memif interface, and process TCP data through sockets and the kernel protocol stack.

[0128] like Figure 4 As shown in the figure, the data interaction process is as follows:

[0129] This link shows the efficient user-mode communication between Nginx and DPDK processes, which implements data transmission through the memif interface, bypasses the kernel protocol stack, avoids unnecessary context switching, and greatly improves performance. In this mode, the memif interface uses shared memory and event notification descriptors (event_fd) for data transmission, and performs efficient data interaction directly in user mode.

[0130] This method is particularly suitable for UDP data processing scenarios, because it does not require complex connection management, so the memif interface can be used to directly exchange data in user mode. In this way, the system can achieve high-performance data transmission and is suitable for applications that require low latency and high throughput.

[0131] This link shows the traditional communication mode of Nginx interacting with the kernel protocol stack through socket. In this mode, Nginx transmits data with the kernel protocol stack through socket. The typical application scenario is to process TCP connections. Since the TCP protocol involves complex connection management and state maintenance (for example, three-way handshake, congestion control, etc.), leaving it to the kernel can simplify the implementation of the application layer while taking advantage of the kernel's mature protocol stack functions.

[0132] The way Nginx communicates with the kernel protocol stack ensures that the system can still rely on traditional, reliable kernel mechanisms when complex connection management is required (such as TCP data transmission).

[0133] based on Figure 4 , the data processing flow is as follows,

[0134] 1) Initialization and connection establishment of memif interface

[0135] When the DPDK process starts, the memif interface is created and configured as master mode.

[0136] After the Nginx process is started, a memif interface is created and set to slave mode, and the memif interface on the DPDK side is connected through a domain socket.

[0137] During the connection establishment process, shared memory, ring queue and event notification descriptor (event_fd) are created for subsequent data transmission and event notification;

[0138] 2) Receiving and processing UDP data

[0139] The DPDK process receives UDP packets through the network card and sends the data to the memif interface according to the policy;

[0140] memif uses shared memory to store received data and puts the data index into the ring queue;

[0141] After the data is stored, memif notifies Nginx through event_fd that new data has arrived;

[0142] Nginx listens to event_fd through epoll. When receiving event notification, it obtains the data index from the ring queue and reads the corresponding data packet from the shared memory.

[0143] Nginx distinguishes and routes according to the four-tuple of the data packet (source IP, source port, destination IP, destination port) to ensure that the data packet is sent to the corresponding listening service; (Since UDP is a connectionless protocol, even if multiple data packets share the same destination address and port, their source addresses and ports may be different. Therefore, Nginx needs to distinguish the four-tuple (source address, source port, destination address, destination port) of the received data packet to find the corresponding listening service)

[0144] After completing the routing of the data packet, Nginx processes the data packet using its standard data processing flow;

[0145] After the processing is completed, Nginx triggers an event callback to notify the upper-layer business for further processing;

[0146] 3) UDP data sending process

[0147] When Nginx needs to send UDP data through DPDK, it first constructs the data packet and fills in the L3 (network layer) and L4 (transport layer) protocol headers;

[0148] Nginx puts the constructed data packet into the ring queue of memif and notifies DPDK through event_fd that the data is ready;

[0149] After receiving the notification, DPDK extracts the data packet from the shared memory;

[0150] Perform routing and addressing based on the L3 / L4 information in the data packet and send the data packet to the destination address;

[0151] 4) TCP data processing

[0152] The TCP data remains unchanged and is handed over to the Linux kernel protocol stack for processing through the socket interface;

[0153] The kernel is responsible for complex operations such as TCP protocol connection management, three-way handshake, congestion control and retransmission;

[0154] Nginx only needs to rely on the kernel's mature protocol stack to handle TCP connections, without having to process complex protocol logic in user mode.

[0155] The design of the disclosed embodiment ensures that the complex TCP protocol is processed by the Linux kernel protocol stack, while the simple and efficient UDP data is processed in the user state through the memif interface. Through this architecture, the system avoids performance bottlenecks, minimizes the switching overhead between the kernel state and the user state, and greatly improves the throughput of UDP data.

[0156] In the standard Nginx model, UDP packets enter the application from the kernel protocol stack through the socket interface, and the kernel is responsible for data management and transmission. In the memif model, UDP packets are directly transmitted in user mode through shared memory and ring queues, bypassing the kernel, thereby greatly improving performance.

[0157] In the memif model, since the packets in the ring queue may come from different source IP and port combinations, Nginx must perform additional parsing and diversion of incoming UDP packets. This requires Nginx to distinguish packets according to the four-tuple (source IP, destination IP, source port, destination port) to ensure that each packet can be correctly routed to the corresponding listening service.

[0158] In contrast, when processing TCP packets, the Linux kernel protocol stack parses the five-tuple (source IP, destination IP, source port, destination port, protocol type) to support complex connection management and data transmission control, such as state tracking, retransmission, and flow control.

[0159] Through the memif interface, Nginx and DPDK can achieve efficient data exchange, and all data transmission and event notification are completed in user mode, avoiding frequent kernel mode switching. This greatly reduces system overhead and further improves the performance of UDP data processing.

[0160] Those skilled in the art should understand that the present disclosure aims to optimize the processing performance of UDP data by working in coordination through the memif interface between DPDK and Nginx. Specifically, UDP data packets are passed from DPDK to Nginx for processing through the memif interface; while TCP data packets still use Nginx's original processing method, that is, they are transmitted and processed through the Linux kernel protocol stack. This solution is applicable to general UDP proxy optimization scenarios, such as proxy servers and VPNs. However, the present disclosure focuses on the field of one-way firewalls, so non-one-way firewall scenarios are not described in detail.

[0161] When the system is running, DPDK is responsible for receiving all data on the network card, including TCP data packets, UDP data packets, and miscellaneous packets. Although the focus of this disclosure is on UDP performance optimization, the overall receiving and sending performance of the network card has also been improved due to the introduction of DPDK. Therefore, not only UDP data, but other types of data transmission can also benefit from it. For example, the use of DPDK+KNI (Kernel NIC Interface) can significantly improve the throughput of TCP data transmission. The design of this disclosure focuses on UDP data optimization and does not discuss in detail the optimization solutions for other data types such as TCP. However, the system architecture is scalable, and in theory Nginx can also receive other types of data through the memif interface.

[0162] The key technical points involved in the data packet processing method disclosed in the present invention are as follows:

[0163] (1) During Nginx initialization, the memif interface is created to allocate shared memory and ring queues, and the event notification mechanism is configured to achieve efficient data communication;

[0164] (2) Nginx parses the four-tuple information in the data packet and diverts the received data packets in the memif queue to the corresponding monitoring service or process for processing;

[0165] (3) After Nginx processes the data packet, it writes the data packet to the shared memory through the memif interface and notifies the DPDK process through the ring queue to perform subsequent sending operations.

[0166] Figure 5 It is a schematic diagram of the composition of an electronic device according to an embodiment of the present disclosure.

[0167] Second, refer to Figure 5 , an embodiment of the present disclosure provides an electronic device, comprising:

[0168] One or more processors 501;

[0169] A memory 502, on which one or more programs are stored. When the one or more programs are executed by one or more processors, the one or more processors implement the data packet processing method described in the first aspect of the embodiment of the present disclosure;

[0170] One or more I / O interfaces 503 are connected between the processor and the memory and are configured to implement information exchange between the processor and the memory.

[0171] Among them, the processor 501 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 502 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 503 is connected between the processor 501 and the memory 502, and can realize information interaction between the processor 501 and the memory 502, including but not limited to a data bus (Bus), etc.

[0172] In some embodiments, the processor 501 , the memory 502 , and the I / O interface 503 are connected to each other via a bus 504 , and further connected to other components of the computing device.

[0173] In a third aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, and when the program is executed by a processor, the method for processing data packets described in the first aspect of the embodiment of the present disclosure is implemented.

[0174] In a fourth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the data packet processing method described in the first aspect of the embodiment of the present disclosure.

[0175] Figure 6 It is a block diagram of a data packet processing device according to an embodiment of the present disclosure.

[0176] Fifth, refer to Figure 6 The present disclosure provides a data packet processing device, including:

[0177] DPDK process module 601 is configured to create a memif interface and configure it as a main mode interface;

[0178] The proxy service process module 602 is configured to create a memif interface and configure it as a slave mode interface;

[0179] The memif interface module 603 is configured to transmit data packets between the proxy service process module 602 and the DPDK process module 603 after the master mode interface establishes a connection with the slave mode interface.

[0180] In some embodiments, the proxy service process module 602 is further configured to: create a shared memory and a circular queue after the master mode interface establishes a connection with the slave mode interface, and send its configuration information to the DPDK process module 601 through the memif interface module 603;

[0181] The DPDK process module 601 is further configured to: map the shared memory to its own address space according to the received configuration information, and use the ring queue for data transmission.

[0182] In some embodiments, the proxy service process module 602 is also configured to: create an event notification descriptor and add the event notification descriptor to the event monitoring mechanism; when the event notification of the event notification descriptor is monitored through the event monitoring mechanism, query the data packet index in the circular queue through the memif interface module 603, and read the data packet from the shared memory; parse the quadruple information in the data packet, and route the data packet to the corresponding monitoring service for processing according to the parsing result.

[0183] In some embodiments, the proxy service process module 602 is further configured to: process the routed data packets according to a standard data processing process, and write the processed data packets into the shared memory;

[0184] The DPDK process module 601 is also configured to extract the processed data packet from the shared memory when the event monitoring mechanism monitors the event notification of the event notification descriptor; perform routing and addressing according to the network layer and transport layer information of the extracted data packet, and send the data packet to the destination address.

[0185] In some embodiments, the DPDK process module 601 is also configured to: receive data packets through a network card; write the received data packets into the shared memory through the memif interface 603, and write the data packet index into the circular queue; and send event notifications to the proxy service process module 602 through the event notification descriptor.

[0186] In some embodiments, the data packet processing device further includes: a socket module configured to transmit data packets between the proxy service process module 602 and the kernel protocol stack module;

[0187] The proxy service process module 602 is further configured to parse the five-tuple information in the data packet read from the kernel protocol stack module through the socket module; and process the data packet according to the parsed five-tuple information in accordance with the standard data processing flow.

[0188] In some embodiments, the proxy service process module 602 is an Nginx process module.

[0189] In some embodiments, the data packets transmitted through the memif interface 603 are UDP data packets, and the data packets read from the kernel protocol stack module through the socket module are TCP data packets.

[0190] In order to enable those skilled in the art to more clearly understand the technical solution provided by the embodiment of the present disclosure, the data packet processing method provided by the embodiment of the present disclosure is described in detail below through specific embodiments:

[0191] Embodiment 1

[0192] A one-way optical gate (also called a one-way network gate) is a physical isolation device that is mainly used between different networks or security domains to ensure that data can only be transmitted in one direction without feedback. This device is widely used in government and military networks to prevent the leakage of important internal information so that data in the external network can only be transmitted in one direction to the internal network.

[0193] The hardware architecture of a one-way optical gate is usually a 2+1 structure, that is, it consists of an external terminal, an internal terminal, and a one-way isolation component. The one-way isolation component is composed of two one-way isolation cards connected by a one-way optical fiber; the external terminal and the internal terminal provide services to the outside as a complete machine, respectively running agent software, and transmitting data through physical one-way communication.

[0194] In order to ensure the normal operation of the business, the one-way network gate must have high availability (HA) function and support dual-machine hot standby in AS mode (Active-Standby). In this mode, one device is in working state (ACTIVE) and provides services to the outside world; the other device is in standby state (STANDBY) and does not provide services to the outside world; when the working device fails, the standby device will quickly switch to working state and provide services to the outside world. During this switching process, the state switching of the external end machine and the internal end machine must be consistent.

[0195] The present disclosure can be applied to the design of a one-way network gate to meet the high performance requirements of UDP while supporting TCP business processing. By deploying DPDK and Nginx in the one-way network gate, this solution effectively improves UDP performance and retains the ability of TCP data packets to be transmitted through the kernel protocol stack.

[0196] Although the solution can be applied to other scenarios that require proxies in theory, this disclosure focuses on the implementation method in the field of one-way firewalls and does not expand the specific application of other scenarios. Based on the technical route of DPDK+Nginx, this paper specifically explains how to achieve high-performance packet processing in a one-way firewall through the disclosed solution.

[0197] According to the data packet processing method disclosed in the present invention, the specific process is as follows:

[0198] 1) The DPDK process starts and creates a memif interface. If DPDK uses multi-threaded or multi-process mode, an independent memif send and receive queue is created for each working thread / process. The number of memif send and receive queues is guaranteed to be consistent with the number of DPDK working threads / processes, and each DPDK process uses its own send and receive queue independently.

[0199] 2) Nginx starts, and after the Nginx master process completes configuration parsing and memory pool creation, it creates a memif interface and establishes a connection with DPDK. When the worker process is initialized, a corresponding memif sending and receiving queue is allocated to each worker, and the corresponding eventfd is added to the epoll monitor, waiting for epoll to trigger.

[0200] 3) After the DPDK process receives the data packet, it passes the data packet to the memif module according to the issued policy or other conditions. The data is queued to the shared memory through the ring queue and a notification is sent to Nginx through eventfd.

[0201] 4) When Nginx's epoll listens to the eventfd trigger, it enters the UDP callback function. The callback function obtains the corresponding mbuf data packet through the ring queue, which contains the L2, L3, and L4 layer protocol content. Nginx parses the five-tuple information (source IP, destination IP, source port, destination port, protocol type) and fills the parsing results into the corresponding structure.

[0202] 5) If a UDP packet is received for the first time, Nginx creates a connection and saves it in the red-black tree. If the packet comes from the memif interface, set the send callback function to memif send.

[0203] 6) When Nginx is ready to send data through memif, it needs to construct and fill in the L3 (network layer) and L4 (transport layer) headers to tell DPDK how to handle the routing and transmission of data packets.

[0204] 7) After Nginx completes the processing, when sending data, it directly calls the callback in step 6, sends the data packet to the memif interface, and sends a notification to DPDK through eventfd.

[0205] 8) DPDK polls the network card and each memif send and receive queue. If it finds data to be sent, it extracts the data packet from the memif and sends it to the destination address through routing.

[0206] 9) When a TCP data packet arrives, the data is processed by the Linux kernel protocol stack and passed to the corresponding socket. Nginx continues to listen to the fd (file descriptor) of the socket in the epoll mechanism. When new TCP data arrives, epoll triggers the corresponding event notification, and Nginx calls the registered callback function to read and process the data packet.

[0207] The data packet processing method disclosed in the present invention can bring the following beneficial effects:

[0208] 1) Transmit UDP data packets in user mode through DPDK and memif interface, effectively reducing kernel mode switching and improving data packet processing efficiency.

[0209] 2) Since DPDK already supports memif, Nginx only needs to perform a small amount of initialization configuration without changing its original data processing flow.

[0210] 3) Nginx can still process TCP packets through the kernel protocol stack, achieving flexible management and parallel processing of different protocols.

[0211] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0212] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A data packet processing method, comprising: In the DPDK process, create a memif interface and configure it as a master mode interface; In the proxy service process, create a memif interface and configure it as a slave mode interface; Establish a connection between the memif interface of the DPDK process and the memif interface of the proxy service process; After the connection is established, data packets are transmitted between the proxy service process and the DPDK process through the memif interface of the DPDK process and the memif interface of the proxy service process.

2. The data packet processing method according to claim 1, wherein: After the connection is established, transmitting data packets between the proxy service process and the DPDK process through the memif interface of the DPDK process and the memif interface of the proxy service process, including: After the connection is established, the proxy service process creates a shared memory and a circular queue, and sends its configuration information to the DPDK process; The DPDK process maps the shared memory to its own address space according to the received configuration information; When the proxy service process monitors the event notification, the data packet index in the ring queue is queried through the memif interface of the proxy service process, and the data packet is read from the shared memory.

3. The data packet processing method according to claim 2, wherein: When the proxy service process monitors the event notification, querying the data packet index in the ring queue through the memif interface of the proxy service process, and reading the data packet from the shared memory, including: Create an event notification descriptor; Adding the event notification descriptor to the event monitoring mechanism of the proxy service process; When the event monitoring mechanism monitors the event notification of the event notification descriptor, the data packet index in the circular queue is queried through the memif interface of the proxy service process, and the data packet is read from the shared memory; Parsing the four-tuple information in the data packet; According to the parsed four-tuple information, the data packet is routed to the corresponding monitoring service for processing.

4. The data packet processing method according to claim 3, wherein: After routing the data packet to the corresponding monitoring service for processing according to the parsed quadruple information, the method includes: Processing the routed data packets according to the standard data processing flow of the proxy service process, and writing the processed data packets into the shared memory; When the event monitoring mechanism monitors the event notification of the event notification descriptor, the DPDK process extracts the processed data packet from the shared memory; The DPDK process performs routing and addressing according to the network layer and transport layer information of the extracted data packet, and sends the data packet to the destination address.

5. The data packet processing method according to claim 4, wherein: After adding the event notification descriptor to the event monitoring mechanism of the proxy service process, the method includes: The DPDK process receives a data packet from the network card; The DPDK process writes the received data packet into the shared memory through the memif interface of the DPDK process, and writes the data packet index into the ring queue; The DPDK process sends an event notification to the proxy service process through the event notification descriptor.

6. The data packet processing method according to any one of claims 2 to 5, wherein: The data packet processing method further comprises: The proxy service process reads data packets from the kernel protocol stack via a socket; Parsing the five-tuple information in the data packet; According to the parsed five-tuple information, the data packet is processed according to the standard data processing flow.

7. The data packet processing method according to claim 6, wherein: The proxy service process is the Nginx process.

8. The data packet processing method according to claim 7, wherein: The event notification is used to indicate a change in a data packet index in the ring queue, prompting the proxy service process or the DPDK process to read a new data packet.

9. The data packet processing method according to claim 7, wherein: The data packet transmitted through the memif interface of the DPDK process and the memif interface of the proxy service process is a UDP data packet, and the data packet read from the kernel protocol stack through the socket is a TCP data packet.

10. An electronic device comprising: one or more processors; A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the data packet processing method according to any one of claims 1 to 9.

11. A computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the data packet processing method according to any one of claims 1 to 9.

12. A computer program product, comprising a computer program or instructions, wherein when the computer program or instructions are executed by a processor, the method for processing data packets according to any one of claims 1 to 9 is implemented.

Images