Method and system for optimizing network packet forwarding performance based on multi-core processor

Abstract

The application belongs to the technical field of network communication, and discloses a network data packet forwarding performance optimization method and system based on a multi-core processor, which comprises the following steps: a received data packet is stored in the form of a queue by using a hierarchical vectorization algorithm of dynamic flow classification hashing; a kernel thread allocates a data packet sending queue to different user states for processing, and establishes a communication mechanism between a kernel state and a user state, and transmits address information of the data packet sending queue to the corresponding user state; a new data queue is generated by modifying and processing the data packet, and the address information of the new data packet queue is fed back to the kernel thread through the communication mechanism established with the kernel state; and the new data packet is sent to the network by calling a network card driver. The kernel thread in the application receives the network data packet transmitted by the network card in the form of polling, which avoids context switching and interruption delay caused by interruption processing, thereby improving the efficiency of data packet reception.

Description

Technical Field

[0001] This invention relates to the field of network communication technology, and in particular to a method and system for optimizing network packet forwarding performance based on multi-core processors. Background Technology

[0002] In recent years, hardware technology has developed rapidly, and multi-core processors have been increasingly widely used in various computing scenarios. Their core count has also continued to rise, with CPUs boasting sixteen or even more cores becoming commonplace. Meanwhile, modern network interface cards (NICs) generally support multi-queue functionality. This feature aims to improve overall processing power by distributing data packets to different queues, allowing multiple CPU cores to process network packets in parallel and efficiently. However, with the continuous increase in the number of CPU cores, the mismatch between the number of NIC queues and the number of CPU cores has become increasingly prominent, which to some extent limits the performance of multi-core processors.

[0003] DPDK, a well-known packet forwarding acceleration solution, effectively improves packet processing speed. However, it has specific network card support requirements and, as a kernel bypass solution, interacts directly with the hardware, bypassing the operating system kernel. While this design reduces kernel overhead, it also prevents full utilization of the rich software ecosystem of the Linux system. Therefore, in practical applications, adopting the DPDK solution often requires extensive customization to adapt to different hardware environments and software requirements, undoubtedly increasing the complexity and cost of project implementation.

[0004] Therefore, how to provide a method and system for optimizing network packet forwarding performance based on multi-core processors is an urgent problem to be solved. Summary of the Invention

[0005] This invention provides a method and system for optimizing network packet forwarding performance based on multi-core processors to solve problems in the prior art.

[0006] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or to describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0007] According to a first aspect of the present invention, a method for optimizing network packet forwarding performance based on a multi-core processor is provided.

[0008] In one embodiment, a network packet forwarding performance optimization method based on a multi-core processor includes the following steps:

[0009] The kernel thread receives network data packets transmitted by the network card in a polling manner, and uses a hierarchical vectorization algorithm of dynamic stream classification hash to store the received data packets in the form of a queue, thus obtaining a data packet sending queue.

[0010] The kernel thread allocates the data packet sending queue to different user spaces for processing and establishes a communication mechanism between the kernel space and the user space to pass the address information of the data packet sending queue to the corresponding user space, forming a shared queue;

[0011] User space receives data packets allocated to it through a shared queue, modifies the data packets to generate a new data queue, and feeds back the address information of the new data packet queue to the kernel thread through a communication mechanism established with kernel space.

[0012] The kernel thread retrieves the new data packet queue from the shared queue based on the returned address information and calls the network card driver to send the new data packet to the network.

[0013] In one embodiment, the kernel thread receives network data packets transmitted by the network card in a polling manner, and uses a hierarchical vectorization algorithm of dynamic stream classification hashing to store the received data packets in the form of a queue to obtain a data packet sending queue, including the following steps:

[0014] A dynamic quintuple hash algorithm is used to analyze the quintuple characteristics of data packets, and the received network data packets are mapped to the initial queue according to flow classification based on the quintuple characteristics.

[0015] Based on real-time traffic characteristics, dynamic weight values ​​are assigned to each flow category in the initial queue using a spin graph structure.

[0016] The five-tuple characteristics of the data packets are combined with dynamic weight values ​​and mapped to sub-queues of different priorities through a multi-level hash function;

[0017] Data packets are sequentially scheduled to the sending buffer according to the priority order of the sub-queues, forming the final data packet sending queue.

[0018] In one embodiment, the step of analyzing the five-tuple characteristics of data packets using a dynamic five-tuple hash algorithm and mapping the received network data packets to the initial queue according to flow classification based on the five-tuple characteristics includes the following steps:

[0019] Extract field information from the header of network data packets, parse the field information, and obtain the five-tuple features of each data packet;

[0020] By using a combination of two hash functions, the hash value of the five-tuple feature is calculated, and a stream identifier is generated by combining it with the timestamp information of the data packet;

[0021] Based on the modulo operation between the flow identifier and the initial queue size, data packets are mapped to the corresponding initial queues;

[0022] Determine if the corresponding flow record already exists in the mapped initial queue. If it does, insert the data packet into the queue; otherwise, create a new flow record, initialize the queue, and then insert the data packet.

[0023] In one embodiment, the step of using a combination of two hash functions to calculate the hash value of the five-tuple feature and combining it with the timestamp information of the data packet to generate a stream identifier includes the following steps:

[0024] Based on the five-tuple characteristics, hash calculations are performed using the main hash function and the auxiliary hash function respectively to obtain the main hash value and the auxiliary hash value;

[0025] The main hash value and the auxiliary hash value are merged using the XOR operation to obtain the hash value with the characteristics of a quintuple;

[0026] Collect the timestamp information of received data packets, and mix the hash seed of the timestamp information with the hash value of the quintuple feature to obtain the stream identifier.

[0027] In one embodiment, assigning dynamic weight values ​​to each flow category in the initial queue based on real-time traffic characteristics using a spin graph structure includes the following steps:

[0028] Obtain real-time traffic information for each flow category, calculate real-time traffic characteristics based on the real-time traffic information, and analyze the correlation between the traffic characteristics using correlation analysis.

[0029] Based on the correlation between various flow characteristics, a spin graph structure for flow categories is constructed to determine the vertex and edge connection relationships of flow categories;

[0030] Based on flow characteristics and edge connectivity, dynamic weight values ​​are assigned to edges in the spin graph structure according to feature similarity.

[0031] In one embodiment, constructing a spin graph structure for flow categories based on the correlation between various flow characteristics and determining the vertex and edge connectivity relationships of flow categories includes the following steps:

[0032] Each flow category is mapped to a vertex in a spin graph structure, and an association matrix is ​​established based on the correlation between the flow characteristics.

[0033] Based on the correlation matrix, spin edges and adjacent edges are added to each vertex, and adjacent edges are filtered based on a preset correlation threshold;

[0034] Based on spin edges and adjacent edges, construct the spin graph structure for flow classes.

[0035] In one embodiment, the kernel thread allocates the data packet sending queue to different user spaces for processing and establishes a communication mechanism between the kernel space and user space to pass the address information of the data packet sending queue to the corresponding user space, forming a shared queue, including the following steps:

[0036] The kernel thread creates a shared memory region and configures a synchronization mechanism to obtain a shared memory identifier, maps the shared memory region to the kernel address space, and initializes the queue information of the data packet sending queue in the shared memory.

[0037] User-mode programs obtain shared memory identifiers through a preset identifier passing method and use the same shared memory identifier to map shared memory regions into the user-mode address space;

[0038] User-mode programs acquire synchronization mechanisms by reading queue information from shared memory and accessing the corresponding data packet sending queue in shared memory based on the queue information, thus forming a shared queue.

[0039] In one embodiment, the user-mode program acquires a synchronization mechanism, reads queue information through shared memory, and accesses the corresponding data packet sending queue in shared memory based on the queue information to form a shared queue, comprising the following steps:

[0040] User-mode programs read the identifier of the synchronization mechanism from shared memory and call the corresponding system function based on the identifier to obtain the handle of the synchronization mechanism;

[0041] The handle based on the synchronization mechanism locks the queue information area in the shared memory and reads the queue information from the locked shared memory.

[0042] The user-mode program accesses the corresponding data packet sending queue in shared memory through pointer operations based on the queue information it reads, and parses and processes the data packets to form a shared queue.

[0043] In one embodiment, the user space receives data packets allocated to it through a shared queue, modifies the data packets to generate a new data queue, and feeds back the address information of the new data packet queue to the kernel thread through a communication mechanism established with the kernel space, including the following steps:

[0044] The user space parses and processes the received data packets, and modifies the parsed data packets based on business requirements;

[0045] A new data packet queue is created in user-space memory, and the modified data packets are inserted into the queue in sequence to form a new data queue.

[0046] User space writes the address information of the new data packet queue to a memory area shared with kernel space and updates the queue status flag in the shared memory.

[0047] User space triggers a kernel thread through a communication mechanism established with kernel space, notifying it that there is a new data packet queue that needs to be processed, and waits for the kernel thread to confirm receipt or completion of processing.

[0048] According to a second aspect of the present invention, a network packet forwarding performance optimization system based on a multi-core processor is provided.

[0049] In one embodiment, the network packet forwarding performance optimization system based on a multi-core processor includes:

[0050] The queue generation module is used by the kernel thread to receive network data packets transmitted by the network card in a polling manner, and to store the received data packets in the form of a queue using a hierarchical vectorization algorithm of dynamic stream classification hashing, thus obtaining a data packet sending queue.

[0051] The data sharing module is used by kernel threads to allocate data packet sending queues to different user spaces for processing, and to establish a communication mechanism between kernel space and user space, passing the address information of the data packet sending queue to the corresponding user space to form a shared queue;

[0052] The information feedback module is used by the user space to receive data packets allocated to it through a shared queue, modify the data packets to generate a new data queue, and feed back the address information of the new data packet queue to the kernel thread through the communication mechanism established with the kernel space.

[0053] The data sending module is used by the kernel thread to retrieve the new data packet queue from the shared queue based on the feedback address information, and then call the network card driver to send the new data packet into the network.

[0054] According to a third aspect of the present invention, a computer device is provided.

[0055] In some embodiments, the computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0056] According to a fourth aspect of the present invention, a computer-readable storage medium is provided.

[0057] In one embodiment, a computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the above method.

[0058] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0059] 1. The kernel thread in this invention receives network data packets transmitted by the network card in a polling manner, avoiding context switching and interrupt delay caused by interrupt handling, thereby improving the efficiency of data packet reception. Furthermore, by using a hierarchical vectorization algorithm of dynamic flow classification hashing, data packets are mapped to different queues according to flow classification, and dynamic weight values ​​are allocated based on real-time traffic characteristics, so that data packets can be scheduled to the sending buffer according to priority, further improving the processing efficiency of data packets.

[0060] 2. This invention fully utilizes the parallel processing capabilities of multi-core processors by allocating data packet sending queues to different user modes for processing, enabling multiple CPU cores to process data packets simultaneously, thereby improving the overall performance of the system. Through the shared queue mechanism between kernel mode and user mode, efficient sharing and exchange of data packets are achieved, allowing user mode and kernel mode to perform data processing and sending operations independently.

[0061] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0062] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0063] Figure 1 This is a flowchart illustrating a network packet forwarding performance optimization method based on a multi-core processor, according to an exemplary embodiment.

[0064] Figure 2 This is a block diagram illustrating the principle of a network packet forwarding performance optimization system based on a multi-core processor, according to an exemplary embodiment.

[0065] Figure 3 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment. Detailed Implementation

[0066] The following description and accompanying drawings fully illustrate specific embodiments described herein to enable those skilled in the art to practice them. Some embodiments may include or substitute parts and features of other embodiments. The scope of the embodiments herein encompasses the entire scope of the claims and all available equivalents thereof. Throughout this document, the terms “first,” “second,” etc., are used only to distinguish one element from another without requiring or implying any actual relationship or order between the elements. Indeed, a first element can also be referred to as a second element, and vice versa. Furthermore, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a structure, apparatus, or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a structure, apparatus, or device. Without further limitation, an element defined by the phrase “comprising one…” does not exclude the presence of other identical elements in the structure, apparatus, or device that includes said element. The various embodiments described herein are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments; similar or identical parts between embodiments can be referred to interchangeably.

[0067] The terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer" used in this document to indicate orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings. They are used solely for the convenience of describing the document and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In the description herein, unless otherwise specified and limited, the terms "installed," "connected," and "linked" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two elements; they can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0068] In this document, unless otherwise stated, the term "multiple" means two or more.

[0069] In this article, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0070] In this article, the term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0071] It should be understood that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order constraint on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the diagram may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0072] The modules in the apparatus or system of this application can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0073] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0074] Figure 1 An embodiment of the network packet forwarding performance optimization method based on a multi-core processor according to the present invention is shown.

[0075] In this optional embodiment, the network packet forwarding performance optimization method based on multi-core processors includes the following steps:

[0076] In step S101, the kernel thread receives network data packets transmitted by the network card in a polling manner, and uses a hierarchical vectorization algorithm of dynamic stream classification hash to store the received data packets in the form of a queue to obtain a data packet sending queue.

[0077] It should be noted that polling is an active query technique, meaning that the kernel thread periodically or continuously checks whether new data packets have arrived at the network card. Polling does not require waiting for the network card to send an interrupt signal to notify the kernel that a data packet has arrived; instead, the kernel actively and periodically queries the status of the network card.

[0078] Specifically, active polling refers to the kernel thread periodically (or continuously in some implementations) checking the network card's receive buffer to see if any new data packets have arrived. Since it does not require waiting for or handling interrupts, polling can reduce context switching and interrupt latency caused by interrupt handling, thereby improving the overall performance of the system.

[0079] In addition, after the kernel starts, it configures the cores (core number M+1...) for interrupts, specifying which cores are used to handle interrupts and how to handle them. In order to minimize the interference of interrupts on user-mode message processing, the kernel disables all hardware interrupts and software interrupts except for the basic clock interrupt. The basic clock interrupt is necessary for the normal operation of the operating system to maintain the system's timing and scheduling functions. Other hardware interrupts and software interrupts may introduce additional latency and overhead, affecting the efficiency of user-mode message processing.

[0080] In step S102, the kernel thread allocates the data packet sending queue to different user spaces for processing, and establishes a communication mechanism between the kernel space and the user space, passing the address information of the data packet sending queue to the corresponding user space to form a shared queue.

[0081] In step S103, the user space receives the data packets allocated to it through the shared queue, modifies the data packets to generate a new data queue, and feeds back the address information of the new data packet queue to the kernel thread through the communication mechanism established with the kernel space.

[0082] In step S104, the kernel thread retrieves the new data packet queue from the shared queue based on the feedback address information and calls the network card driver to send the new data packet to the network.

[0083] In this optional embodiment, when the kernel thread receives network data packets transmitted by the network card in a polling manner and stores the received data packets in the form of queues using a hierarchical vectorization algorithm of dynamic flow classification hashing to obtain a data packet sending queue, a dynamic quintuple hashing algorithm can be used to analyze the quintuple characteristics of the data packets, and the received network data packets can be mapped to the initial queue according to flow classification based on the quintuple characteristics; based on real-time traffic characteristics, dynamic weight values ​​are assigned to each flow category in the initial queue using a spin graph structure; the quintuple characteristics of the data packets are combined with the dynamic weight values ​​and mapped to sub-queues of different priorities through a multi-level hash function; according to the priority order of the sub-queues, the data packets are scheduled to the sending buffer in sequence to form the final data packet sending queue.

[0084] In this optional embodiment, when using a dynamic quintuple hash algorithm to analyze the quintuple characteristics of data packets and mapping the received network data packets to the initial queue according to flow classification based on the quintuple characteristics, field information can be extracted from the header of the network data packets, parsed through the field information to obtain the quintuple characteristics of each data packet; the hash value of the quintuple characteristics is calculated using a combination of dual hash functions, and a flow identifier is generated by combining the timestamp information of the data packet; the data packet is mapped to the corresponding initial queue according to the modulo operation between the flow identifier and the number of initial queues; it is determined whether the corresponding flow record already exists in the mapped initial queue. If it does, the data packet is inserted into the queue; otherwise, a new flow record is created, the queue is initialized, and the data packet is inserted.

[0085] It should be noted that the dynamic quintuple hash algorithm is a technique used for network packet classification and identification. It performs hash calculations based on the quintuple characteristics of data packets, thereby achieving rapid classification and mapping of data packets.

[0086] The five-tuple characteristic refers to the five key fields in a network packet: source IP address, destination IP address, source port number, destination port number, and protocol type. These five fields uniquely identify a network connection or session. Using these five fields, packets can be categorized into a specific "flow." A flow refers to all network packets with the same five-tuple characteristic, typically belonging to the same session or connection (e.g., a TCP session with the same source and destination IP and port pair).

[0087] In this optional embodiment, when calculating the hash value of the five-tuple feature using a combination of dual hash functions and generating a stream identifier by combining the timestamp information of the data packet, the hash calculation can be performed using the main hash function and the auxiliary hash function respectively based on the five-tuple feature to obtain the main hash value and the auxiliary hash value; the main hash value and the auxiliary hash value are then fused using an XOR operation to obtain the hash value of the five-tuple feature; the timestamp information of the received data packet is collected, and the hash seed of the timestamp information is mixed with the hash value of the five-tuple feature to obtain the stream identifier.

[0088] Specifically, when processing network data packets, a dual hash function combination is used to more accurately identify and classify data streams. Two different hash functions (a primary hash function and an auxiliary hash function) are used to hash the five-tuple characteristics, thereby improving the stability and uniqueness of the hash value.

[0089] The primary hash value is obtained by hashing the five-tuple characteristics (source IP, destination IP, source port, destination port, protocol type) using a primary hash function (such as MD5, SHA-1, CRC32, MurmurHash, etc.). An auxiliary hash value is obtained by hashing the same five-tuple characteristics using an auxiliary hash function (such as the FNV-1a algorithm).

[0090] Furthermore, to combine the primary and secondary hash values ​​into a more stable hash value, the XOR operation can be used. XOR is a bitwise operation that compares corresponding bits of two binary numbers; if they are the same, the result is 0, and if they are different, the result is 1. Through the XOR operation, the information from the primary and secondary hash values ​​can be combined to generate a new hash value that contains information from both the primary and secondary hash functions.

[0091] Besides the 5-tuple feature, the timestamp information of the data packet is also an important basis for generating the stream identifier. The timestamp information records the time when the data packet was received or sent, which helps distinguish data packets arriving at different times under the same 5-tuple feature. The timestamp information is used as a hash seed and mixed with the hash value of the 5-tuple feature (i.e., the result of merging the main hash value and the auxiliary hash value) for calculation. Mixed calculation refers to combining the timestamp information with the hash value of the 5-tuple feature in some way (such as concatenation, weighting, etc.) and then performing another hash calculation, or directly using the timestamp information as one of the input parameters of the hash function. After mixed calculation, a new hash value is obtained, and this hash value is the stream identifier. The stream identifier uniquely identifies a data stream.

[0092] In this optional embodiment, when assigning dynamic weight values ​​to each flow category in the initial queue based on real-time flow characteristics using a spin graph structure, real-time flow information for each flow category can be obtained. Real-time flow characteristics are calculated based on the real-time flow information, and the correlation between each flow characteristic is analyzed using correlation analysis. Based on the correlation between each flow characteristic, a spin graph structure for the flow category is constructed, and the vertex and edge connection relationships of the flow category are determined. Based on the flow characteristics and edge connection relationships, dynamic weight values ​​are assigned to the edges in the spin graph structure based on feature similarity.

[0093] It's important to note that real-time traffic information refers to key metrics for each flow, such as current transmission rate, packet size, packet arrival interval, and latency. This information can be obtained by real-time monitoring of network interfaces or using traffic sampling methods (such as NetFlow and sFlow). Within network traffic, certain flow categories may exhibit similar traffic characteristics. For example, some flows may display similar arrival intervals, data volumes, or bandwidth utilization patterns. To achieve efficient traffic management, it's necessary to analyze the correlations between these traffic characteristics. This includes: extracting relevant features from the real-time traffic information of each flow, such as flow rate, packet size, and arrival interval; calculating the feature similarity between different flow categories using similarity calculation methods (such as Pearson correlation coefficient, cosine similarity, and covariance); and then constructing an association matrix based on the similarity calculation results. This matrix represents the relationships between various traffic features; each element in the matrix represents the similarity value between two flow categories, with larger values ​​indicating greater similarity.

[0094] In this optional embodiment, when constructing a spin graph structure for flow categories based on the correlation between various flow features and determining the vertex and edge connection relationships of flow categories, each flow category can be mapped to a vertex in the spin graph structure, and an association matrix can be established based on the correlation between various flow features; spin edges and adjacent edges are added to each vertex according to the association matrix, and adjacent edges are filtered based on a preset association threshold; and the spin graph structure for flow categories is constructed based on the spin edges and adjacent edges.

[0095] It's important to note that a spin graph is a graph structure containing two types of edges: spin edges and adjacent edges. In a spin graph, vertices represent different flow classes, and edges describe the relationships between these flow classes. Spin edges represent the correlation between a flow class and itself. Spin edges are used to capture the internal dynamic characteristics of flow classes, such as flow fluctuations and adaptive changes in flow. They can help identify the state changes of a flow class at different times. Adjacent edges represent the similarity or correlation between two flow classes, and these edges are established based on the correlation between flow characteristics.

[0096] To reduce redundancy and improve efficiency, a threshold is used to filter adjacent edges. Adjacent edges are only added to the spin graph when the correlation between flow categories is greater than this threshold. For example, if the similarity between flow A and flow B is 0.8, exceeding the set threshold (e.g., 0.7), an adjacent edge will be added between them. If the similarity is 0.5, below the set threshold, no adjacent edge will be added.

[0097] By constructing a spin graph, the correlation between different flow categories can be represented in the graph, allowing packets to be more accurately classified into the appropriate queue or processing unit.

[0098] In this optional embodiment, when the kernel thread allocates the data packet sending queue to different user spaces for processing and establishes a communication mechanism between the kernel space and user space to pass the address information of the data packet sending queue to the corresponding user space to form a shared queue, the kernel thread can create a shared memory region and configure a synchronization mechanism to obtain a shared memory identifier, map the shared memory region to the kernel address space, and initialize the queue information of the data packet sending queue in the shared memory. The user-space program obtains the shared memory identifier through a preset identifier passing method and uses the same shared memory identifier to map the shared memory region to the user-space address space. The user-space program obtains the synchronization mechanism, reads the queue information through the shared memory, and accesses the corresponding data packet sending queue in the shared memory according to the queue information to form a shared queue.

[0099] It's important to note that kernel threads create shared memory regions in kernel space. These shared memory regions are mapped into the address space of user-mode programs so that both can share data. The shared memory regions are used to store packet queues and related status information (such as the start position of the queue, the current processing position, etc.).

[0100] Synchronization mechanisms are used to ensure proper synchronization between kernel mode and user mode, avoiding data races. Commonly used synchronization mechanisms include semaphores, event identifiers, or futexes.

[0101] In this optional embodiment, when the user-mode program obtains the synchronization mechanism, reads queue information through shared memory, and accesses the corresponding data packet sending queue in shared memory to form a shared queue, the user-mode program can read the identifier of the synchronization mechanism from shared memory and call the corresponding system function to obtain the handle of the synchronization mechanism based on the identifier; based on the handle of the synchronization mechanism, it locks the queue information area in shared memory and reads the queue information from the locked shared memory; the user-mode program accesses the corresponding data packet sending queue in shared memory through pointer operations based on the read queue information and parses the data packets to form a shared queue.

[0102] It's important to note that once shared memory is successfully mapped into the user-space address space, the user-space program needs to acquire a handle to the synchronization mechanism so that it can perform synchronization operations when accessing data in the shared memory. The user-space program reads the identifier of the synchronization mechanism from the shared memory. This identifier is provided by the kernel when the shared memory is created and is used to identify the specific synchronization mechanism.

[0103] User-mode programs use this identifier to call corresponding system functions, such as `sem_init()`, `eventfd()`, or `futex()`, to obtain a handle to the synchronization mechanism. This handle represents an instance of the synchronization mechanism, allowing user-mode programs to synchronize with the kernel. Through the synchronization mechanism handle, user-mode programs can lock the queue information region in shared memory. After locking, user-mode programs can safely read relevant information about the packet queues from shared memory (such as the queue's starting position and current queue size).

[0104] In this optional embodiment, when user space receives data packets allocated to it through a shared queue, modifies the data packets to generate a new data queue, and feeds back the address information of the new data packet queue to the kernel thread through a communication mechanism established with kernel space, user space can parse the received data packets and modify the parsed data packets based on business requirements; create a new data packet queue in user space memory, insert the modified data packets into the queue in sequence to form a new data queue; user space writes the address information of the new data packet queue into the memory area shared with kernel space, and updates the queue status flag in shared memory; user space triggers the kernel thread through the communication mechanism established with kernel space, notifying it that there is a new data packet queue that needs to be processed, and waits for the kernel thread to confirm receipt or completion of processing.

[0105] Specifically, user-space programs receive data packets from the network or other processes via the socket mechanism. A socket is a programming interface for network communication, allowing data exchange between different processes or hosts. The user-space program parses the received data packets, extracting key information such as source IP, destination IP, port number, protocol type, and payload data. Then, based on business requirements, the user-space program modifies the parsed data packets. These modifications include:

[0106] Change the source or destination IP address, port number, TTL value, etc.

[0107] Convert a private IP address to a public IP address, or vice versa.

[0108] Analyze the contents of data packets to identify and filter out malicious traffic or specific types of data packets.

[0109] It should be noted that after processing the received messages, the user-space program does not send them directly through the socket, but instead places the processed messages into a shared queue. This separates the processing logic from the sending logic, improving the system's flexibility and scalability.

[0110] In addition, data packets can be retrieved in sequence and processed as necessary (e.g., parsing, modification, etc.); user-space programs will process the data packets according to business needs; for example, modifying the data packet header, performing Network Address Translation (NAT), performing Deep Packet Inspection (DPI), etc.; the processed data packets are inserted into a new data queue, forming a new shared queue, and finally fed back to the kernel thread for further processing or transmission.

[0111] Figure 2 An embodiment of the network packet forwarding performance optimization system based on a multi-core processor according to the present invention is shown.

[0112] In this optional embodiment, the network packet forwarding performance optimization system based on a multi-core processor includes:

[0113] The queue generation module 201 is used by the kernel thread to receive network data packets transmitted by the network card in a polling manner, and to store the received data packets in the form of a queue using a hierarchical vectorization algorithm of dynamic stream classification hashing to obtain a data packet sending queue.

[0114] The data sharing module 202 is used by the kernel thread to allocate the data packet sending queue to different user modes for processing, and to establish a communication mechanism between the kernel mode and the user mode, passing the address information of the data packet sending queue to the corresponding user mode to form a shared queue;

[0115] The information feedback module 203 is used to receive data packets allocated to itself in user space through a shared queue, modify the data packets to generate a new data queue, and feed back the address information of the new data packet queue to the kernel thread through a communication mechanism established with kernel space.

[0116] The data sending module 204 is used by the kernel thread to retrieve the new data packet queue from the shared queue based on the feedback address information, and call the network card driver to send the new data packet into the network.

[0117] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 3As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores static and dynamic information data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the above method embodiments.

[0118] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0119] In addition, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0120] In addition, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0122] This invention is not limited to the structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this invention is limited only by the appended claims.

Claims

1. A method for optimizing network packet forwarding performance based on multi-core processors, characterized in that, Includes the following steps: The kernel thread receives network data packets transmitted by the network card in a polling manner, and uses a hierarchical vectorization algorithm of dynamic stream classification hash to store the received data packets in the form of a queue, thus obtaining a data packet sending queue. The kernel thread allocates the data packet sending queue to different user spaces for processing and establishes a communication mechanism between the kernel space and the user space to pass the address information of the data packet sending queue to the corresponding user space, forming a shared queue; User space receives data packets allocated to it through a shared queue, modifies the data packets to generate a new data queue, and feeds back the address information of the new data packet queue to the kernel thread through a communication mechanism established with kernel space. The kernel thread retrieves the new data packet queue from the shared queue based on the returned address information and calls the network card driver to send the new data packet into the network; The kernel thread receives network data packets transmitted by the network card in a polling manner, and uses a hierarchical vectorization algorithm of dynamic stream classification hashing to store the received data packets in the form of a queue to obtain the data packet sending queue, including the following steps: Extract field information from the header of network data packets, parse the field information to obtain the five-tuple feature of each data packet; use a combination of double hash functions to calculate the hash value of the five-tuple feature, and generate a flow identifier by combining the timestamp information of the data packet; map the data packet to the corresponding initial queue according to the modulo operation of the flow identifier and the initial queue size; determine whether the corresponding flow record already exists in the mapped initial queue, if it exists, insert the data packet into the queue, otherwise create a new flow record, initialize the queue, and then insert the data packet; Based on real-time traffic characteristics, dynamic weight values ​​are assigned to each flow category in the initial queue using a spin graph structure. The five-tuple characteristics of the data packets are combined with dynamic weight values ​​and mapped to sub-queues of different priorities through a multi-level hash function; Data packets are sequentially scheduled to the sending buffer according to the priority order of the sub-queues, forming the final data packet sending queue.

2. The network packet forwarding performance optimization method based on a multi-core processor according to claim 1, characterized in that, The process of calculating the hash value of the five-tuple feature using a combination of two hash functions and generating a stream identifier by combining the timestamp information of the data packet includes the following steps: Based on the five-tuple characteristics, hash calculations are performed using the main hash function and the auxiliary hash function respectively to obtain the main hash value and the auxiliary hash value; The main hash value and the auxiliary hash value are merged using the XOR operation to obtain the hash value with the characteristics of a quintuple; Collect the timestamp information of received data packets, and mix the hash seed of the timestamp information with the hash value of the quintuple feature to obtain the stream identifier.

3. The network packet forwarding performance optimization method based on a multi-core processor according to claim 2, characterized in that, The process of assigning dynamic weight values ​​to each flow category in the initial queue based on real-time traffic characteristics using a spin graph structure includes the following steps: Obtain real-time traffic information for each flow category, calculate real-time traffic characteristics based on the real-time traffic information, and analyze the correlation between the traffic characteristics using correlation analysis. Based on the correlation between various flow characteristics, a spin graph structure for flow categories is constructed to determine the vertex and edge connection relationships of flow categories; Based on flow characteristics and edge connectivity, dynamic weight values ​​are assigned to edges in the spin graph structure according to feature similarity.

4. The network packet forwarding performance optimization method based on a multi-core processor according to claim 3, characterized in that, The process of constructing a spin graph structure for flow categories based on the correlation between various flow characteristics and determining the vertex and edge connectivity relationships of flow categories includes the following steps: Each flow category is mapped to a vertex in a spin graph structure, and an association matrix is ​​established based on the correlation between the flow characteristics. Based on the correlation matrix, spin edges and adjacent edges are added to each vertex, and adjacent edges are filtered based on a preset correlation threshold; Based on spin edges and adjacent edges, construct the spin graph structure for flow classes.

5. The network packet forwarding performance optimization method based on a multi-core processor according to claim 1, characterized in that, The kernel thread allocates the data packet sending queue to different user spaces for processing and establishes a communication mechanism between kernel space and user space. It passes the address information of the data packet sending queue to the corresponding user space to form a shared queue, including the following steps: The kernel thread creates a shared memory region and configures a synchronization mechanism to obtain a shared memory identifier, maps the shared memory region to the kernel address space, and initializes the queue information of the data packet sending queue in the shared memory. User-mode programs obtain shared memory identifiers through a preset identifier passing method and use the same shared memory identifier to map shared memory regions into the user-mode address space; User-mode programs acquire synchronization mechanisms by reading queue information from shared memory and accessing the corresponding data packet sending queue in shared memory based on the queue information, thus forming a shared queue.

6. The network packet forwarding performance optimization method based on a multi-core processor according to claim 5, characterized in that, The user-mode program acquires a synchronization mechanism by reading queue information from shared memory and accessing the corresponding data packet sending queue in shared memory based on the queue information, forming a shared queue through the following steps: User-mode programs read the identifier of the synchronization mechanism from shared memory and call the corresponding system function based on the identifier to obtain the handle of the synchronization mechanism; The handle based on the synchronization mechanism locks the queue information area in the shared memory and reads the queue information from the locked shared memory. The user-mode program accesses the corresponding data packet sending queue in shared memory through pointer operations based on the queue information it reads, and parses and processes the data packets to form a shared queue.

7. The network packet forwarding performance optimization method based on a multi-core processor according to claim 1, characterized in that, The user space receives data packets allocated to it through a shared queue, modifies the data packets to generate a new data queue, and feeds back the address information of the new data packet queue to the kernel thread through a communication mechanism established with the kernel space. This includes the following steps: The user space parses and processes the received data packets, and modifies the parsed data packets based on business requirements; A new data packet queue is created in user-space memory, and the modified data packets are inserted into the queue in sequence to form a new data queue. User space writes the address information of the new data packet queue to a memory area shared with kernel space and updates the queue status flag in the shared memory. User space triggers a kernel thread through a communication mechanism established with kernel space, notifying it that there is a new data packet queue that needs to be processed, and waits for the kernel thread to confirm receipt or completion of processing.

8. A network packet forwarding performance optimization system based on a multi-core processor, characterized in that, include: The queue generation module is used by the kernel thread to receive network data packets transmitted by the network card in a polling manner, and to store the received data packets in the form of a queue using a hierarchical vectorization algorithm of dynamic stream classification hashing, thus obtaining a data packet sending queue. The data sharing module is used by kernel threads to allocate data packet sending queues to different user spaces for processing, and to establish a communication mechanism between kernel space and user space, passing the address information of the data packet sending queue to the corresponding user space to form a shared queue; The information feedback module is used by the user space to receive data packets allocated to it through a shared queue, modify the data packets to generate a new data queue, and feed back the address information of the new data packet queue to the kernel thread through the communication mechanism established with the kernel space. The data sending module is used by the kernel thread to retrieve the new data packet queue from the shared queue based on the feedback address information, and call the network card driver to send the new data packet into the network; The kernel thread receives network data packets transmitted by the network card in a polling manner, and uses a hierarchical vectorization algorithm of dynamic stream classification hashing to store the received data packets in the form of a queue, resulting in a data packet sending queue including: Extract field information from the header of network data packets, parse the field information to obtain the five-tuple feature of each data packet; use a combination of double hash functions to calculate the hash value of the five-tuple feature, and generate a flow identifier by combining the timestamp information of the data packet; map the data packet to the corresponding initial queue according to the modulo operation of the flow identifier and the initial queue size; determine whether the corresponding flow record already exists in the mapped initial queue, if it exists, insert the data packet into the queue, otherwise create a new flow record, initialize the queue, and then insert the data packet; Based on real-time traffic characteristics, dynamic weight values ​​are assigned to each flow category in the initial queue using a spin graph structure. The five-tuple characteristics of the data packets are combined with dynamic weight values ​​and mapped to sub-queues of different priorities through a multi-level hash function; Data packets are sequentially scheduled to the sending buffer according to the priority order of the sub-queues, forming the final data packet sending queue.

Images