Network data forwarding method and device, equipment and storage medium
By binding the network interface card's port to different NUMA nodes and encryption cards in a NUMA architecture, the problems of performance degradation and high cost in traditional network data processing methods are solved, and efficient and secure data forwarding is achieved.
Patent Information
- Application Number
- CN202511056383.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-30
- Publication Date
- 2025-11-04
AI Technical Summary
In NUMA architecture, traditional network data processing methods are limited by the PCIe slots of network cards and encryption cards, resulting in a decrease in data forwarding and encryption/decryption performance. Furthermore, adding multiple network cards and encryption cards increases costs and fails to fully utilize the advantages of multiple NUMA nodes of the CPU.
By binding multiple network ports of the network card to different NUMA nodes and encryption cards, and mapping data to the memory of the corresponding NUMA nodes for forwarding, the advantages of multiple NUMA nodes are utilized to reduce the performance requirements of a single NUMA node.
It improves the performance of multiple encryption cards under a single network card, saves hardware costs, and improves the efficiency and security of network data forwarding.
Smart Images
Figure CN120896898A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of embedded technology, and in particular to a network data forwarding method, apparatus, device and storage medium. Background Technology
[0002] With the rapid development of cloud computing, big data, and SDN (Software Defined Network) technologies, network devices are facing increasingly massive amounts of data to process, and the performance requirements are also rising. Traditional network data processing methods are often limited by the operating system's kernel protocol stack, and suffer from bottlenecks such as interrupt handling, memory copying, and context switching, which cannot meet the demands of high-performance network data processing.
[0003] Currently, the emergence of NUMA (Non-Uniform Memory Access) architecture has provided new possibilities for high-performance network data processing. In NUMA architecture, the processor is divided into multiple nodes, and each node has its own local memory. Since accessing local memory is much faster than accessing remote memory, the number of cross-node memory accesses can be minimized when processing large amounts of data, thereby improving the performance of network data encryption, decryption and forwarding.
[0004] However, during network data forwarding, the bandwidth overhead for accessing local memory increases due to the stacking of devices used for forwarding and encryption / decryption operations, such as Network Interface Cards (NICs) and encryption cards (PCIe, a high-speed serial computer expansion bus standard). Furthermore, in practical use, since NICs are far more powerful than encryption cards, one NIC is often needed to support multiple encryption cards. However, a high-speed NIC (such as a 40G or 100G NIC) PCIe slot can only belong to one NUMA node. If other memory access devices, such as encryption cards, are added to the PCIe bus (where many devices can be inserted), the bandwidth for PCIe memory access will be limited (e.g., PCIe 4.0 x16 has a bandwidth of 31.5Gb / s). For example, when a 40G NIC and a 40G encryption card are simultaneously reading and writing to memory, the performance of a single NUMA node accessing memory will be less than that of the NIC and encryption card, resulting in a simultaneous decrease in data forwarding and encryption / decryption performance.
[0005] To improve network data forwarding performance, current methods typically involve adding multiple network interface cards (NICs) and encryption cards, mapping them to the same NUMA node. For example, NUMA0 handles encrypted forwarding of uplink traffic, while NUMA1 handles unencrypted forwarding of downlink traffic. This approach leverages multiple NUMA nodes on the CPU to achieve peak performance, but using multiple NICs and encryption cards increases costs. Furthermore, current NIC initialization processes often unify all network interfaces to a single NUMA node, allocating threads and memory resources within that single node, thus failing to fully utilize the advantages of multiple NUMA nodes on the CPU. Summary of the Invention
[0006] In view of this, the purpose of this application is to provide a network data forwarding method, apparatus, device, and storage medium, which can improve the performance of multiple encryption cards under a single network card, reduce the performance requirements of a single NUMA node, and improve the efficiency and security of network data forwarding. The specific solution is as follows:
[0007] In a first aspect, this application discloses a network data forwarding method applied to a target processor with a NUMA architecture. The target processor deploys multiple NUMA nodes, a target network interface card (NIC) with multiple network ports, and multiple encryption cards. A single network port of the target NIC is connected to a single encryption card and a single NUMA node via a preset bus. The method includes:
[0008] When the first thread corresponding to any NUMA node polls for a network data packet, it receives the network data packet through the first network port pre-bound to the NUMA node and maps the network data packet to the first receiving ring queue corresponding to the NUMA node.
[0009] The first queue address corresponding to the first receiving ring queue is sent to the first encryption card bound to any of the NUMA nodes, so as to obtain the network data packets in the first receiving ring queue based on the first queue address, encrypt the network data packets to obtain encrypted packets, and then copy the encrypted packets to the first receiving ring queue.
[0010] The encrypted message in the first receiving ring queue is mapped to the first sending queue corresponding to the first network interface, and the encrypted message in the first sending queue is sent out from the first network interface.
[0011] When the second thread corresponding to another NUMA node (excluding any of the NUMA nodes) polls for the encrypted message, it decrypts the encrypted message to obtain the network data packet and forwards the network data packet to the destination address.
[0012] Optionally, the network data forwarding method further includes:
[0013] The DPDK program is started and initialized to bind the corresponding encryption card and target network card to each NUMA node in the target processor, and to initialize the pre-created packet receiving ring queue on the large page memory of each NUMA node.
[0014] Optionally, the step of binding corresponding encryption cards and target network cards to each NUMA node in the target processor, and initializing a pre-created receiving ring queue on the large page memory of each NUMA node, includes:
[0015] The preset command-line parameters are parsed to bind corresponding encryption cards and target network cards to each NUMA node in the target processor; the preset command-line parameters include the kernel mask and memory configuration information of the target processor.
[0016] Read the relevant information of the target network card to obtain network card related information; the network card related information includes network card configuration information, information of the NUMA node where the target network card is located, and network port status information;
[0017] Create node threads corresponding to each NUMA node, and create target threads bound to the logical core of the target processor based on preset command line parameters, so as to prevent the scheduling of the kernel of the target processor through the target threads;
[0018] Based on the network interface card information, the corresponding large page memory is mounted for each NUMA node;
[0019] Based on the network card information and in the node thread, each network port of the target network card is bound to the corresponding NUMA node and the encryption card through a preset bus, and a pre-created packet receiving ring queue is initialized on the large page memory of the corresponding NUMA node.
[0020] Optionally, the step of binding each network port of the target network card to the corresponding NUMA node and the encryption card respectively through a preset bus based on the network card information and in the node thread includes:
[0021] Based on the network card information and in the node thread, each network port of the target network card is bound to the corresponding NUMA node and the encryption card via the PCIe bus;
[0022] The node thread is either the first thread or the second thread, and the network card related information also includes the network card port number.
[0023] Optionally, the step of decrypting the encrypted message to obtain the network data packet and forwarding the network data packet to the destination address includes:
[0024] The encrypted message is received through a second network port pre-bound to the other NUMA node, and the encrypted message is mapped to the second receiving ring queue corresponding to the other NUMA node.
[0025] Obtain the address of the second receiving ring queue, get the second queue address, and send the second queue address to the second encryption card bound to the other NUMA node;
[0026] The encrypted message in the second receiving ring queue is obtained through the second encryption card and based on the second queue address. The encrypted message is then decrypted to obtain the network data packet, and the network data packet is then copied to the second receiving ring queue.
[0027] The network data packets in the second receiving ring queue are mapped to the second sending queue corresponding to the second network interface, and the network data packets in the second sending queue are forwarded from the second network interface to the destination address.
[0028] Optionally, mapping the network data packets to the first receive circular queue corresponding to any NUMA node includes:
[0029] The network data packets are mapped to the first receiving circular queue corresponding to any NUMA node using direct memory access.
[0030] Accordingly, mapping the encrypted message in the first receiving ring queue to the first sending queue corresponding to the first network interface includes:
[0031] The encrypted packets in the first receiving ring queue are mapped to the first sending queue corresponding to the first network interface using direct memory access.
[0032] Optionally, encrypting the network data packet to obtain an encrypted packet includes:
[0033] The network data packets are encrypted using the AES-256 algorithm to obtain encrypted packets;
[0034] Accordingly, after forwarding the network data packet to the destination address, the process further includes:
[0035] The large page memory in the other NUMA node is reclaimed.
[0036] Secondly, this application discloses a network data forwarding device applied to a target processor with a NUMA architecture. The target processor deploys multiple NUMA nodes, a target network interface card (NIC) with multiple network ports, and multiple encryption cards. Each network port of the target NIC is connected to a single encryption card and a single NUMA node via a preset bus. The device includes:
[0037] The message receiving module is used to receive network data packets through a first network port pre-bound to any NUMA node when the first thread corresponding to any NUMA node polls for network data packets.
[0038] The first message mapping module is used to map the network data packets to the first receiving ring queue corresponding to any NUMA node;
[0039] The address sending module is used to send the first queue address corresponding to the first receiving ring queue to the first encryption card bound to any of the NUMA nodes, so as to obtain the network data packets in the first receiving ring queue based on the first queue address, encrypt the network data packets to obtain encrypted packets, and then copy the encrypted packets to the first receiving ring queue.
[0040] The second message mapping module is used to map the encrypted message in the first receiving ring queue to the first sending queue corresponding to the first network interface;
[0041] The message sending module is used to send the encrypted message in the first sending queue from the first network port;
[0042] The message forwarding module is used to decrypt the encrypted message when the second thread corresponding to another NUMA node (excluding any of the NUMA nodes) polls the encrypted message, obtain the network data packet, and forward the network data packet to the destination address.
[0043] Thirdly, this application discloses an electronic device, including a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the aforementioned network data forwarding method.
[0044] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned network data forwarding method.
[0045] As can be seen, this application is applied to a target processor with a NUMA architecture. The target processor deploys multiple NUMA nodes, a target network interface card (NIC) with multiple network ports, and multiple encryption cards. Each network port of the target NIC is connected to a single encryption card and a single NUMA node via a preset bus. The method includes: when a first thread corresponding to any NUMA node polls for a network data packet, receiving the network data packet through a first network port pre-bound to the NUMA node, and mapping the network data packet to a first receiving ring queue corresponding to the NUMA node; sending the first queue address corresponding to the first receiving ring queue to the address bound to the NUMA node. The first encryption card obtains the network data packets in the first receiving ring queue based on the first queue address, encrypts the network data packets to obtain encrypted packets, and then copies the encrypted packets to the first receiving ring queue. The encrypted packets in the first receiving ring queue are mapped to the first sending queue corresponding to the first network interface, and the encrypted packets in the first sending queue are sent from the first network interface. When the second thread corresponding to another NUMA node (excluding any NUMA node) polls for the encrypted packets, it decrypts the encrypted packets to obtain the network data packets and forwards them to the destination address. This application binds multiple network interfaces of the network card to different NUMA nodes and encryption cards, and maps the data received by the network interfaces to the memory of the corresponding NUMA nodes for data forwarding. This fully utilizes multiple NUMA nodes in the processor, thereby improving the performance of multiple encryption cards under a single network card. Since no additional network cards are needed, the hardware cost of multiple network cards is saved. In addition, the performance requirements of a single NUMA node are reduced, and the efficiency and security of network data forwarding are improved. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0047] Figure 1 This is a flowchart of a network data forwarding method disclosed in this application;
[0048] Figure 2 This is a schematic diagram of a specific processor hardware architecture disclosed in this application;
[0049] Figure 3 This is a flowchart of a specific network data forwarding method disclosed in this application;
[0050] Figure 4 This is a schematic diagram of the structure of a network data forwarding device disclosed in this application;
[0051] Figure 5 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation
[0052] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0053] This application discloses a network data forwarding method applied to a target processor with a NUMA architecture. The target processor deploys multiple NUMA nodes, a target network interface card (NIC) with multiple network ports, and multiple encryption cards. Each network port of the target NIC is connected to a single encryption card and a single NUMA node via a preset bus. (See also...) Figure 1 As shown, the method includes:
[0054] Step S11: When the first thread corresponding to any NUMA node polls for a network data packet, it receives the network data packet through the first network port pre-bound to the NUMA node and maps the network data packet to the first receiving ring queue corresponding to the NUMA node.
[0055] It should be noted that the network data forwarding scheme proposed in this application is specifically applied to NUMA architecture processors, such as CPUs (Central Processing Units). For details, see [link to relevant documentation]. Figure 2 As shown, the processor (i.e., CPU) deploys two NUMA nodes (NUMA0 and NUMA1), a network interface card (NIC) with two network ports (port0 and port1) (such as a 40G NIC, 100G NIC, etc.), and two encryption cards (encryption card 0 and encryption card 1). The two network ports of the NICs are connected to a single encryption card and a single NUMA node respectively via a pre-defined bus. Figure 2 Port 0 is connected to NUMA 0 and encryption card 0 via the PCIe bus, and port 1 is connected to NUMA 1 and encryption card 1 via the PCIe bus.
[0056] In this embodiment, see Figure 2As shown, when the first thread corresponding to any NUMA node (such as NUMA0) polls and receives a plaintext network data packet, it can first receive the network data packet through the first network port (i.e., port0) pre-bound to NUMA0, and then map the network data packet to the receiving ring queue (i.e., ring0) located in the Huge Pages0 memory of NUMA0. The network data packet can be a packet of different types of network data (such as files, emails, videos, images, etc.).
[0057] It should be noted that before performing network data forwarding, this application specifically includes: starting the DPDK program and initializing the DPDK program to bind corresponding encryption cards and target network cards to each NUMA node in the target processor, and initializing a pre-created receiving ring queue on the large page memory of each NUMA node. In this embodiment, it is necessary to first start the DPDK (Data Plane Development Kit) program, and then initialize the DPDK program to configure its runtime environment. Specifically, corresponding encryption cards and network cards can be bound to each NUMA node in the processor (such as CPU), and a pre-created receiving ring queue can be initialized on the large page memory of each NUMA node. For example, a pre-created receiving ring queue (i.e., ring0) can be initialized on the large page memory (Huge Pages0) of NUMA0, and a pre-created receiving ring queue (i.e., ring1) can be initialized on the large page memory (Huge Pages1) of NUMA1. Through the above steps, the runtime environment of the DPDK program can be configured, thereby greatly improving data processing performance and throughput, and thus improving the forwarding efficiency of network data.
[0058] Specifically, the step of binding corresponding encryption cards and target network cards to each NUMA node in the target processor, and initializing a pre-created receiving ring queue on the large page memory of each NUMA node, may include: parsing preset command-line parameters to bind corresponding encryption cards and target network cards to each NUMA node in the target processor; the preset command-line parameters include the core mask and memory configuration information of the target processor; reading the relevant information of the target network card to obtain network card related information; the network card related information includes network card configuration information, information of the NUMA node where the target network card is located, and network port status information; creating node threads corresponding to each NUMA node, and creating target threads bound to the logical core of the target processor based on the preset command-line parameters, so as to prevent the kernel of the target processor from being scheduled through the target threads; mounting corresponding large page memory to each NUMA node based on the network card related information; and binding each network port of the target network card to the corresponding NUMA node and the encryption card respectively through a preset bus based on the network card related information and in the node threads, and initializing a pre-created receiving ring queue on the large page memory of the corresponding NUMA node. In this embodiment, after the DPDK program starts, it can first allocate corresponding resources, such as storage resources and computing resources, and then parse the preset command line parameters (such as CPU core mask, large page memory configuration information, etc.) to bind the corresponding encryption card and network card to each NUMA node in the processor (i.e., CPU). For example, NUMA0 is bound to encryption card 0 and network card, and NUMA1 is bound to encryption card 1 and network card. Next, the relevant information of the network card is read, such as network card configuration information, NUMA node information where the network card is located, and port status information of the network card, to obtain the network card related information, and node threads corresponding to each NUMA node are created. For example, the first thread (thread0) corresponding to NUMA0 is created, and the second thread (thread1) corresponding to NUMA1 is created. Based on the preset command line parameters (such as CPU core mask, etc.), a target thread (lcore) bound to the logical core of the processor (CPU) is created, and the scheduling of the current processor (CPU) kernel is prohibited through the target thread (lcore), that is, kernel scheduling is disabled.Furthermore, based on network interface card (NIC) related information (such as large page memory configuration information), corresponding large pages are mounted for each NUMA node, thereby establishing a shared memory region between user space and kernel. Mounting large pages can also avoid physical memory fragmentation. Finally, based on NIC related information and in the node thread (such as thread0), the NIC's port0 and encryption card 0 are bound to NUMA0 through a preset bus, and a receiving ring queue (i.e., ring0) is initialized on the large page memory of NUMA0. Then, the above operations are performed on other ports of the NIC. For example, in the node thread (such as thread1), the NIC's port1 and encryption card 1 are bound to NUMA1 through a preset bus, and a receiving ring queue (i.e., ring1) is initialized on the large page memory of NUMA1.
[0059] It's important to note that a circular queue is a data structure that stores data in a fixed-size array and uses modulo operations to achieve circular access. When the tail of the queue reaches the end of the array, it automatically returns to the beginning of the array, thus forming a "circular" effect. Furthermore, a circular queue has two key indices: the front index and the rear index, used to indicate the head and tail positions of the queue, respectively. This data structure is particularly suitable for scenarios requiring frequent reads and writes with limited memory, enabling efficient data transmission and processing. Therefore, by creating and initializing a receiving circular queue on the large pages of memory on each NUMA node, the corresponding node threads, such as thread0 and thread1, can automatically receive packets (i.e., receive data packets) in polling mode.
[0060] Specifically, the step of binding each port of the target network card to the corresponding NUMA node and the encryption card via a preset bus based on the network card information and in the node thread can include: binding each port of the target network card to the corresponding NUMA node and the encryption card via a PCIe bus based on the network card information and in the node thread; wherein, the node thread is the first thread or the second thread, and the network card information also includes the network card port number. In this embodiment, see... Figure 2As shown, based on network card information (such as network card configuration information, NUMA node information where the network card is located, and port status information of the network card), and in the node thread (e.g., thread0), the network card's port0 and encryption card 0 are bound to NUMA0 respectively through a preset bus, and a receiving ring queue (i.e., ring0) is initialized on the large page memory of NUMA0. Then, in the node thread (e.g., thread1), the network card's port1 and encryption card 1 are bound to NUMA1 respectively through the PCIe bus, and a receiving ring queue (i.e., ring1) is initialized on the large page memory of NUMA1. It should be noted that the binding of different network ports to NUMA nodes can be implemented based on the port number of the network port and the identifier of the NUMA node, such as port0 being bound to NUMA0, i.e., the network card slot, such as the PCIe slot being on NUMA0.
[0061] Step S12: Send the first queue address corresponding to the first receiving ring queue to the first encryption card bound to any of the NUMA nodes, so as to obtain the network data packets in the first receiving ring queue based on the first queue address, encrypt the network data packets to obtain encrypted packets, and then copy the encrypted packets to the first receiving ring queue.
[0062] In this embodiment, after mapping the network data packets to the first receiving ring queue (ring0) corresponding to any NUMA node (e.g., NUMA0), the address of the first receiving ring queue (ring0) is obtained through route lookup and policy matching, resulting in a first queue address (address0). This first queue address (address0) is then sent to the first encryption card (encryption card 0) bound to any NUMA node (i.e., NUMA0). Based on this first queue address (address0), the network data packets in the first receiving ring queue (ring0) are retrieved. The retrieved network data packets are then encrypted to obtain encrypted packets, which are then copied back to the first receiving ring queue (ring0). The route lookup rules and matching policies can be pre-stored in a routing policy database for direct invocation. Only data packets (e.g., data packets) matched by the policy can undergo the relevant operations. Specifically, this can be managed through IP rules. For example, access control lists and routing policies can be configured to match packets (based on specific attributes of the packets, such as source IP, destination IP, protocol, etc.).
[0063] In one specific implementation, encrypting the network data packet to obtain an encrypted packet may include: encrypting the network data packet using the AES-256 algorithm to obtain an encrypted packet. That is, when encrypting plaintext network data packets, the encryption card uses the AES-256 (Advanced Encryption Standard-256, an advanced encryption standard using a 256-bit key length) algorithm. Of course, other encryption algorithms, such as DES (Data Encryption Standard, a block cipher algorithm using a key) or SM4 (a block cipher algorithm), can be selected depending on the specific application scenario.
[0064] Step S13: Map the encrypted message in the first receiving ring queue to the first sending queue corresponding to the first network interface, and send the encrypted message in the first sending queue from the first network interface.
[0065] In this embodiment, after copying the encrypted message to the first receiving ring queue (i.e., ring0), the encrypted message in the first receiving ring queue (i.e., ring0) can be mapped to the sending queue (send queue0) located at the first network port (i.e., port0), and then the encrypted message in the sending queue (send queue0) is sent out from the first network port (i.e., port0).
[0066] Step S14: When the second thread corresponding to another NUMA node (excluding any of the NUMA nodes) polls the encrypted message, it decrypts the encrypted message to obtain the network data packet and forwards the network data packet to the destination address.
[0067] In this embodiment, see Figure 2 As shown, when the second thread (i.e., thread1) of another NUMA node (such as NUMA1) other than any of the above NUMA nodes (i.e., NUMA0) polls the encrypted message, it can first decrypt the encrypted message to obtain a plaintext network data packet, and then forward the network data packet to the destination address, thereby realizing the encryption and decryption forwarding of the message.
[0068] Furthermore, after forwarding the network data packet to the destination address, the process may further include: reclaiming the large page memory in the other NUMA node. In this embodiment, in order to improve the utilization rate of memory resources and avoid wasting memory resources, after completing the forwarding of the network data packet, the large page memory in the NUMA node (i.e., NUMA1) where the decryption operation was performed can also be reclaimed for reuse.
[0069] Specifically, the step of decrypting the encrypted message to obtain the network data packet and forwarding the network data packet to the destination address may include: receiving the encrypted message through a second network port pre-bound to the other NUMA node, and mapping the encrypted message to a second receiving ring queue corresponding to the other NUMA node; obtaining the address of the second receiving ring queue, obtaining the second queue address, and sending the second queue address to a second encryption card bound to the other NUMA node; obtaining the encrypted message in the second receiving ring queue through the second encryption card and based on the second queue address, decrypting the encrypted message to obtain the network data packet, and then copying the network data packet to the second receiving ring queue; mapping the network data packet in the second receiving ring queue to a second sending queue corresponding to the second network port, and forwarding the network data packet in the second sending queue from the second network port to the destination address. In this embodiment, the process of decrypting the encrypted message using another NUMA node (such as NUMA1) and forwarding the decrypted plaintext message to the destination address is the same as the encryption process of NUMA0. For details, see [link to documentation]. Figure 2 As shown, when the thread (thread1) corresponding to NUMA1 polls for the encrypted message, it can first receive the encrypted message from port1, then map the encrypted message to the ring queue (ring1) of NUMA1, and send the queue address of ring1 to encryption card 1. When encryption card 1 receives the queue address, it retrieves the encrypted message in ring1 based on the queue address, decrypts it to obtain the plaintext network data packet, copies the network data packet to ring1, and finally maps the plaintext in ring1 to the send queue (send queue1) of port1, and sends the packet from port1 to the destination address.
[0070] It is understandable that an encryption card is an encryption card that uses specific encryption algorithms and security technologies, mainly used to protect the security of sensitive information and ensure that data is not obtained or tampered with by unauthorized third parties during transmission and storage. In this embodiment, in order to further improve the security of the data forwarding process, an integrity verification mechanism can be added. For example, before encrypting the message through the encryption card, a hash algorithm (such as MD5, SHA-1, SHA-256, etc.) can be used to map the file content into a fixed-length hash value. Then, the hash value used for message integrity verification and the encrypted message are sent together through port0. Furthermore, when the thread (such as thread1) corresponding to another NUMA node (such as NUMA1) receives the hash value and the encrypted message, it can decrypt the encrypted message and use the hash algorithm and the hash value to verify the integrity of the decrypted network data message, and obtain the corresponding verification result. If the verification result shows that the message is complete and has not been tampered with, the network data message is forwarded to the destination address through the network port (port1); otherwise, the message is incomplete, and message forwarding is prohibited, and corresponding alarm information is generated.
[0071] As can be seen, this embodiment of the application binds multiple network ports of the network card to different NUMA nodes and encryption cards respectively, and maps the data received by the network ports to the memory of the corresponding NUMA nodes for data forwarding. This can make full use of multiple NUMA nodes in the processor, thereby improving the performance of multiple encryption cards under a single network card. Since no additional network cards are needed, the hardware cost of multiple network cards is saved. In addition, the performance requirements of a single NUMA node are reduced, and the efficiency and security of network data forwarding are improved.
[0072] This application discloses a specific network data forwarding method applied to a target processor with a NUMA architecture. The target processor deploys multiple NUMA nodes, a target network interface card (NIC) with multiple network ports, and multiple encryption cards. Each network port of the target NIC is connected to a single encryption card and a single NUMA node via a preset bus. See [link to relevant documentation]. Figure 3 As shown, the method includes:
[0073] Step S21: When the first thread corresponding to any NUMA node polls for a network data packet, it receives the network data packet through the first network port pre-bound to the NUMA node, and maps the network data packet to the first receiving ring queue corresponding to the NUMA node through direct memory access.
[0074] In this embodiment, see Figure 2As shown, when the first thread (thread0) corresponding to any NUMA node (such as NUMA0) polls for a network data packet, it can first receive the network data packet through the first network port (port0) pre-bound to any NUMA node (i.e., NUMA0), and then map the network data packet to the first receiving ring queue (ring0) corresponding to the current NUMA node (such as NUMA0) through Direct Memory Access (DMA).
[0075] Step S22: Send the first queue address corresponding to the first receiving ring queue to the first encryption card bound to any of the NUMA nodes, so as to obtain the network data packets in the first receiving ring queue based on the first queue address, encrypt the network data packets to obtain encrypted packets, and then copy the encrypted packets to the first receiving ring queue.
[0076] Step S23: Map the encrypted message in the first receiving ring queue to the first sending queue corresponding to the first network interface through direct memory access, and send the encrypted message in the first sending queue from the first network interface.
[0077] In this embodiment, the encrypted packets in the first receiving ring queue (ring0) can be mapped to the first sending queue (sendqueue0) corresponding to the first network port (port0) through direct memory access (i.e., DMA), and the encrypted packets in the first sending queue (send queue0) can be sent from the first network port (i.e., port0).
[0078] Step S24: When the second thread corresponding to another NUMA node (excluding any of the NUMA nodes) polls the encrypted message, it decrypts the encrypted message to obtain the network data packet and forwards the network data packet to the destination address.
[0079] For more detailed processing procedures of steps S22 and S24, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be repeated here.
[0080] As can be seen, this embodiment of the application binds multiple network ports of the network card to different NUMA nodes and encryption cards respectively, and maps the data received by the network ports to the memory of the corresponding NUMA node for data forwarding through direct memory access (i.e., DMA). This improves the performance of multiple PCIe devices (such as encryption cards) under a single network card, with a performance improvement close to that of using multiple network cards. Therefore, it effectively solves the problem of high hardware costs caused by using multiple network cards. Furthermore, in DPDK polling mode, multiple PCIe devices (such as encryption cards) are allocated to different NUMA nodes, and the network ports of the network card are initialized on the same NUMA node as the PCIe devices (such as encryption cards). In this way, it is ensured that the network port memory is mapped to the remote memory, and the PCIe devices (such as encryption cards) access the near memory, thereby providing a safe and efficient address space for subsequent access to memory by PCIe devices (such as encryption cards). In addition, by mapping multiple network ports of a single network card to the large page memory of different NUMA nodes, and binding threads to the port numbers of the network cards, and polling packets according to the circular queue of the NUMA node where the thread is located, CPU resources can be fully utilized to achieve the effect that only the network card needs to access the remote memory, while the extended device (such as the PCIe encryption card) accesses the local memory, thereby improving the memory access efficiency of serialized PCIe devices (such as encryption cards).
[0081] Accordingly, this application also discloses a network data forwarding device applied to a target processor with a NUMA architecture. The target processor deploys multiple NUMA nodes, a target network interface card (NIC) with multiple network ports, and multiple encryption cards. Each network port of the target NIC is connected to a single encryption card and a single NUMA node via a preset bus. (See also...) Figure 4 As shown, the device includes:
[0082] The message receiving module 11 is used to receive network data packets through a first network port pre-bound to any NUMA node when the first thread corresponding to any NUMA node polls for network data packets.
[0083] The first message mapping module 12 is used to map the network data packets to the first receiving ring queue corresponding to any NUMA node;
[0084] Address sending module 13 is used to send the first queue address corresponding to the first receiving ring queue to the first encryption card bound to any of the NUMA nodes, so as to obtain the network data packets in the first receiving ring queue based on the first queue address, encrypt the network data packets to obtain encrypted packets, and then copy the encrypted packets to the first receiving ring queue.
[0085] The second message mapping module 14 is used to map the encrypted message in the first receiving ring queue to the first sending queue corresponding to the first network interface;
[0086] Message sending module 15 is used to send the encrypted message in the first sending queue from the first network port;
[0087] The message forwarding module 16 is used to decrypt the encrypted message to obtain the network data packet when the second thread corresponding to another NUMA node other than any of the NUMA nodes polls the encrypted message, and then forward the network data packet to the destination address.
[0088] The specific workflow of each of the above modules can be found in the relevant content disclosed in the foregoing embodiments, and will not be repeated here.
[0089] As can be seen, this embodiment of the application binds multiple network ports of the network card to different NUMA nodes and encryption cards respectively, and maps the data received by the network ports to the memory of the corresponding NUMA nodes for data forwarding. This can make full use of multiple NUMA nodes in the processor, thereby improving the performance of multiple encryption cards under a single network card. Since no additional network cards are needed, the hardware cost of multiple network cards is saved. In addition, the performance requirements of a single NUMA node are reduced, and the efficiency and security of network data forwarding are improved.
[0090] In some specific embodiments, the network data forwarding device may further include:
[0091] The program startup and initialization unit is used to start the DPDK program and perform initialization operations on the DPDK program to bind the corresponding encryption card and target network card to each NUMA node in the target processor, and to initialize the pre-created packet receiving ring queue on the large page memory of each NUMA node.
[0092] In some specific embodiments, the program startup and initialization unit may specifically include:
[0093] The parameter parsing unit is used to parse preset command-line parameters to bind corresponding encryption cards and target network cards to each NUMA node in the target processor; the preset command-line parameters include the core mask and memory configuration information of the target processor.
[0094] The information reading unit is used to read the relevant information of the target network card to obtain network card related information; the network card related information includes network card configuration information, information of the NUMA node where the target network card is located, and network port status information;
[0095] The thread creation unit is used to create node threads corresponding to each NUMA node, and to create target threads bound to the logical core of the target processor based on preset command line parameters, so as to prevent the scheduling of the kernel of the target processor through the target threads;
[0096] A memory mounting unit is used to mount corresponding large page memory for each NUMA node based on the network interface card information.
[0097] The first binding unit is used to bind each network port of the target network card to the corresponding NUMA node and the encryption card respectively through a preset bus based on the network card information and in the node thread;
[0098] The queue initialization unit is used to initialize a pre-created receiving circular queue on the large page memory of the corresponding NUMA node.
[0099] In some specific embodiments, the first binding unit may specifically include:
[0100] The second binding unit is used to bind each network port of the target network card to the corresponding NUMA node and the encryption card respectively through the PCIe bus, based on the network card information and in the node thread;
[0101] The node thread is either the first thread or the second thread, and the network card related information also includes the network card port number.
[0102] In some specific embodiments, the message forwarding module 16 may specifically include:
[0103] The message receiving unit is used to receive the encrypted message through a second network port pre-bound to the other NUMA node;
[0104] The first message mapping unit is used to map the encrypted message to the second receiving ring queue corresponding to the other NUMA node;
[0105] The address acquisition and sending unit is used to acquire the address of the second receiving ring queue, obtain the second queue address, and send the second queue address to the second encryption card bound to the other NUMA node;
[0106] The ciphertext acquisition unit is used to acquire the encrypted message in the second receiving ring queue through the second encryption card and based on the second queue address;
[0107] The ciphertext decryption unit is used to decrypt the encrypted message to obtain the network data packet;
[0108] A message copying unit is used to copy the network data packets to the second receive ring queue;
[0109] The second message mapping unit is used to map the network data packets in the second receive ring queue to the second send queue corresponding to the second network interface;
[0110] The message forwarding unit is used to forward the network data packets in the second sending queue from the second network interface to the destination address.
[0111] In some specific embodiments, the first message mapping module 12 may specifically include:
[0112] The third mapping unit is used to map the network data packets to the first receiving ring queue corresponding to any NUMA node through direct memory access.
[0113] Accordingly, the second message mapping module 14 may specifically include:
[0114] The fourth mapping unit is used to map the encrypted message in the first receiving ring queue to the first sending queue corresponding to the first network interface through direct memory access.
[0115] In some specific embodiments, the address sending module 13 may specifically include:
[0116] The message encryption unit is used to encrypt the network data packets using the AES-256 algorithm to obtain encrypted messages;
[0117] Accordingly, after the message forwarding module 16, it may further include:
[0118] A memory reclamation unit is used to reclaim large pages of memory in the other NUMA node.
[0119] Furthermore, embodiments of this application also disclose an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.
[0120] Figure 5This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the network data forwarding method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0121] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0122] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0123] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the network data forwarding method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.
[0124] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned network data forwarding method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.
[0125] Furthermore, embodiments of this application also disclose a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the network data forwarding method disclosed above.
[0126] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0127] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0128] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0129] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0130] The foregoing has provided a detailed description of a network data forwarding method, apparatus, device, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A network data forwarding method, characterized in that, A target processor applied to a NUMA architecture, wherein the target processor deploys multiple NUMA nodes, a target network interface card (NIC) with multiple network ports, and multiple encryption cards, and a single network port of the target NIC is connected to a single encryption card and a single NUMA node via a preset bus, the method comprising: When the first thread corresponding to any NUMA node polls for a network data packet, it receives the network data packet through the first network port pre-bound to the NUMA node and maps the network data packet to the first receiving ring queue corresponding to the NUMA node. The first queue address corresponding to the first receiving ring queue is sent to the first encryption card bound to any of the NUMA nodes, so as to obtain the network data packets in the first receiving ring queue based on the first queue address, encrypt the network data packets to obtain encrypted packets, and then copy the encrypted packets to the first receiving ring queue. The encrypted message in the first receiving ring queue is mapped to the first sending queue corresponding to the first network interface, and the encrypted message in the first sending queue is sent out from the first network interface. When the second thread corresponding to another NUMA node (excluding any of the NUMA nodes) polls for the encrypted message, it decrypts the encrypted message to obtain the network data packet and forwards the network data packet to the destination address.
2. The network data forwarding method according to claim 1, characterized in that, Also includes: The DPDK program is started and initialized to bind the corresponding encryption card and target network card to each NUMA node in the target processor, and to initialize the pre-created packet receiving ring queue on the large page memory of each NUMA node.
3. The network data forwarding method according to claim 2, characterized in that, The step of binding corresponding encryption cards and target network cards to each NUMA node in the target processor, and initializing a pre-created receiving ring queue on the large page memory of each NUMA node, includes: The preset command-line parameters are parsed to bind corresponding encryption cards and target network cards to each NUMA node in the target processor; the preset command-line parameters include the kernel mask and memory configuration information of the target processor. Read the relevant information of the target network card to obtain network card related information; the network card related information includes network card configuration information, information of the NUMA node where the target network card is located, and network port status information; Create node threads corresponding to each NUMA node, and create target threads bound to the logical core of the target processor based on preset command line parameters, so as to prevent the scheduling of the kernel of the target processor through the target threads; Based on the network interface card information, the corresponding large page memory is mounted for each NUMA node; Based on the network card information and in the node thread, each network port of the target network card is bound to the corresponding NUMA node and the encryption card through a preset bus, and a pre-created packet receiving ring queue is initialized on the large page memory of the corresponding NUMA node.
4. The network data forwarding method according to claim 3, characterized in that, The step of binding each network port of the target network card to the corresponding NUMA node and the encryption card via a preset bus based on the network card information and in the node thread includes: Based on the network card information and in the node thread, each network port of the target network card is bound to the corresponding NUMA node and the encryption card via the PCIe bus; The node thread is either the first thread or the second thread, and the network card related information also includes the network card port number.
5. The network data forwarding method according to claim 1, characterized in that, The step of decrypting the encrypted message to obtain the network data packet and forwarding the network data packet to the destination address includes: The encrypted message is received through a second network port pre-bound to the other NUMA node, and the encrypted message is mapped to the second receiving ring queue corresponding to the other NUMA node. Obtain the address of the second receiving ring queue, get the second queue address, and send the second queue address to the second encryption card bound to the other NUMA node; The encrypted message in the second receiving ring queue is obtained through the second encryption card and based on the second queue address. The encrypted message is then decrypted to obtain the network data packet, and the network data packet is then copied to the second receiving ring queue. The network data packets in the second receiving ring queue are mapped to the second sending queue corresponding to the second network interface, and the network data packets in the second sending queue are forwarded from the second network interface to the destination address.
6. The network data forwarding method according to claim 1, characterized in that, The step of mapping the network data packets to the first receive circular queue corresponding to any NUMA node includes: The network data packets are mapped to the first receiving circular queue corresponding to any NUMA node using direct memory access. Accordingly, mapping the encrypted message in the first receiving ring queue to the first sending queue corresponding to the first network interface includes: The encrypted packets in the first receiving ring queue are mapped to the first sending queue corresponding to the first network interface using direct memory access.
7. The network data forwarding method according to any one of claims 1 to 6, characterized in that, The process of encrypting the network data packets to obtain encrypted packets includes: The network data packets are encrypted using the AES-256 algorithm to obtain encrypted packets; Accordingly, after forwarding the network data packet to the destination address, the process further includes: The large page memory in the other NUMA node is reclaimed.
8. A network data forwarding device, characterized in that, A target processor applied to a NUMA architecture, wherein the target processor deploys multiple NUMA nodes, a target network interface card (NIC) with multiple network ports, and multiple encryption cards, and a single network port of the target NIC is connected to a single encryption card and a single NUMA node via a preset bus, the device comprising: The message receiving module is used to receive network data packets through a first network port pre-bound to any NUMA node when the first thread corresponding to any NUMA node polls for network data packets. The first message mapping module is used to map the network data packets to the first receiving ring queue corresponding to any NUMA node; The address sending module is used to send the first queue address corresponding to the first receiving ring queue to the first encryption card bound to any of the NUMA nodes, so as to obtain the network data packets in the first receiving ring queue based on the first queue address, encrypt the network data packets to obtain encrypted packets, and then copy the encrypted packets to the first receiving ring queue. The second message mapping module is used to map the encrypted message in the first receiving ring queue to the first sending queue corresponding to the first network interface; The message sending module is used to send the encrypted message in the first sending queue from the first network port; The message forwarding module is used to decrypt the encrypted message when the second thread corresponding to another NUMA node (excluding any of the NUMA nodes) polls the encrypted message, obtain the network data packet, and forward the network data packet to the destination address.
9. An electronic device, characterized in that, It includes a processor and a memory; wherein, when the processor executes a computer program stored in the memory, it implements the network data forwarding method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Used to store computer programs; wherein, when the computer programs are executed by a processor, they implement the network data forwarding method as described in any one of claims 1 to 7.