In-stack zero-copy transmission path method based on shared memory communication mode

Abstract

The present invention relates to the technical field of data communication. Disclosed is an in-stack zero-copy transmission path method based on a shared memory communication mode. The copying from a shared memory to a packet buffer is changed into writing a reference to data in the shared memory into the packet buffer, thereby eliminating data copies within packet buffer areas. While not changing an application side to maintain transparence and low coupling of a network protocol stack with respect to upper-layer applications, the original transmission path inside the network protocol stack is changed into a zero-copy transmission path, thereby eliminating copies inside the network protocol stack, so as to eliminate data duplication in LLC and save LLC, and improving the network performance of the network protocol stack for transmission of large files in high-load scenarios.

Description

A stack-based zero-copy send path method based on shared memory communication mode Technical Field

[0001] This invention relates to the field of data communication technology, and in particular to a stack-based zero-copy transmission path method based on shared memory communication mode. Background Technology

[0002] Today, people prefer to obtain information and share communication through short videos and streaming media. Consequently, video streaming has become a considerable portion of internet traffic. This massive traffic brings revenue to video platforms, but it also presents significant challenges. Platforms need sufficiently powerful hardware and software to support this enormous flow of traffic.

[0003] Thanks to the rapid development of hardware in recent years, network hardware performance is becoming increasingly powerful, with bandwidth constantly rising; 100G and 400G network cards have already emerged. However, it has been found that the processing speed of upper-layer software on the host side is gradually lagging behind the development of hardware, becoming a bottleneck for overall performance. The network protocol stack, as the key to communicating between upper-layer applications and lower-layer network hardware, has a significant impact on overall network performance.

[0004] A simplified video streaming scenario can be viewed as a client receiving a large file from a server via a persistent connection. In such scenarios involving large files, persistent connections, and massive overall traffic, traditional kernel-mode network protocol stacks are relatively inefficient. Because using a kernel-mode network protocol stack incurs context switching between user mode and kernel mode, and data copying between these modes incurs overhead, some researchers have begun exploring user-mode network protocol stacks to bypass kernel-mode stacks and improve network performance.

[0005] The user-space network protocol stack implements the network protocol stack in the user-space process, avoiding context switching between user-space and kernel-space. At the same time, user-space programs are more convenient to develop, test, and deploy than kernel programs, and can be updated quickly.

[0006] Common user-space protocol stacks can be divided into two categories: Library OS (LibOS) mode and microkernel mode.

[0007] The LibOS-style user-space protocol stack places the protocol stack and application in the same process, with the application implementing network communication through function calls. Function calls are more lightweight than traditional system calls, eliminating the overhead of context switching. However, this also results in tight coupling between application code and network protocol stack code, requiring additional development for different applications and leading to poor interoperability. Furthermore, this coupling means that updates and upgrades to the network protocol stack are tied to the application, requiring a complete process restart for updates and upgrades.

[0008] In a microkernel-based user-space protocol stack, the network protocol stack is deployed and runs in a separate process. It communicates with external applications via shared memory, providing them with network services. This design results in loose coupling between the application and the network protocol stack, making it suitable for a wider range of programs, offering good versatility, and allowing the network protocol stack to be upgraded independently of the application.

[0009] Because the user-space network protocol stack in a microkernel model belongs to a different process than the application, it needs to provide network services to the application through inter-process communication (IPC). A common IPC method is based on shared memory. However, shared memory IPC introduces multiple data copies.

[0010] Taking application data transmission as an example, the data to be transmitted resides in the application's private memory space. Therefore, the application copies the data to be transmitted to shared memory, making this data accessible to the network protocol stack. After obtaining this data, the network protocol stack needs to encapsulate it. This encapsulation operation occurs within the data packet buffer. Therefore, the network protocol stack copies the data from shared memory to the data packet buffer and then adds a header to the data for encapsulation. Finally, the network protocol stack hands the data packet over to the network interface card (NIC) for transmission.

[0011] In large file transfer scenarios, because the data blocks being transferred are generally very large, many CPU cycles are consumed for memory copying, making memory copying a significant overhead.

[0012] However, when analyzing the impact of memory copying from the perspective of processor microarchitecture, it is found that in large file transfer scenarios, memory copying not only consumes a large number of CPU cycles, but also consumes considerable LLC resources.

[0013] Figure 1 analyzes the original VPP network protocol stack's transmission process. Starting from the application side, the application copies data from private memory to shared memory and notifies the network protocol stack. In the Intel x86 architecture, when the application and network protocol stack run on different CPU cores of the same socket, the CPU caches the data in shared memory in LLC, allowing different cores to share it. Next, the network protocol stack copies the data from shared memory into the packet buffer, adds a packet header to the data, and then hands it over to the driver. The driver constructs a descriptor, writes it to the descriptor ring, and notifies the network card, allowing the network card to send the data packet out via Direct Memory Access (DMA). However, with the continuous increase in network bandwidth, the network card needs to complete packet processing in a shorter time, and the latency caused by accessing memory is too high. Therefore, some manufacturers have added Direct Cache Access (DCA) capabilities to network cards and other I / O devices, such as Intel's Data Direct I / O (DDIO). I / O devices can directly access LLC, reducing data read / write latency and improving throughput. Therefore, the CPU caches data packets in LLC for the network card to access.

[0014] Based on the above analysis, it was found that the data copying process between the application and the network protocol stack along the transmission path generates two copies of data: one copy of the application data stored in shared memory, and the other copy encapsulated into a data packet along with the packet header. Although the application data and the data packet have different data structures, their main content is the application's transmission data, and both copies occupy LLC space. However, LLC space is limited. On some common processors, such as the Intel Xeon series, the LLC on a single socket is only tens of MB, far less than the physical memory, and cannot be expanded. Therefore, when the amount of data sent by the application increases, this data will use more memory space, and the copies generated by the data copying will amplify this memory space, causing the LLC to be quickly exhausted. Once exhausted, more data and copies generated by the copying process can only be placed in memory, requiring more CPU cycles to complete this memory copying, and the network card also needs to spend more time waiting for data to be read from memory. The slower memory copying and slower data packet transmission by the network card will reduce the packet transmission rate of the network protocol stack, resulting in performance degradation.

[0015] Experiments were conducted to observe the LLC usage of the application and network protocol stacks, as well as the CPU LLC miss rate of the network protocol stack, under different numbers of network connections. Figures 2 and 3 illustrate this observation. In the experiments, when the number of network connections was less than 400, the application and network protocol stacks consumed a significant amount of LLC. After 400 connections, the application and network protocol stacks consumed almost all of the LLC, and the LLC store miss rate of the network protocol stack rapidly increased from 12% to 80%. The network protocol stack's LLC store operations primarily originate from memory copying from shared memory to the packet buffer. The increase in the LLC store miss rate leads to increased memory copy latency, reducing the packet processing rate of the network protocol stack and consequently, its network performance.

[0016] Therefore, those skilled in the art are dedicated to developing a zero-copy in-stack transmission path method based on shared memory communication. Without modifying the application side to maintain the transparency and low coupling of the network protocol stack to upper-layer applications, the original transmission path within the network protocol stack is transformed into a zero-copy transmission path. This eliminates copying within the network protocol stack, thereby eliminating data duplication within the LLC, saving LLC space, and improving the network performance of the network protocol stack when sending large files under high load scenarios.

[0017] Summary of the Invention

[0018] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is how to eliminate memory copying on the internal transmission path of the network protocol stack through zero copying, avoid LLC store miss caused by memory copying which reduces the data packet processing rate, and at the same time eliminate the duplication of data in LLC, so that the saved LLC can be used to cache more transmission data and improve the performance of the network protocol stack.

[0019] To achieve the above objectives, this invention provides a stack-based zero-copy transmission path method based on shared memory communication mode, comprising the following steps:

[0020] Step 1: The event polling module of the network protocol stack receives the application's I / O requests from the I / O event queue through polling, and then forwards the requests to the session module for processing.

[0021] Step 2: After receiving the send request, the session module finds the corresponding network session, locates the data send queue based on the data send queue information recorded in the network session, determines the position and length of the data to be sent, and then calls the memory connection module and passes the data send information to the memory connection module.

[0022] Step 3: After the memory connection module obtains the data information in the sending queue, it calculates the required data packet buffer resources, allocates them from the data packet buffer resource pool, and then saves the data reference to the data packet buffer, replacing the original operation of copying the data to the data packet buffer; during this process, an additional data packet buffer will be allocated.

[0023] Step 4: After the memory connection module writes the reference to the data packet cache, it hands the data packet cache over to the session module; the session module then hands the data packet cache over to the transport / network layer for packet encapsulation.

[0024] Step 5: The transport / network layer receives the session information and data packet buffer provided by the session module, finds the corresponding network connection, and the network protocol stack generates a data packet header. In order to quickly distinguish whether the data packet buffer stores a reference to shared memory data, the network protocol stack stores the data packet header and data reference in different data packet buffers, and adds a mark to the data packet buffer that stores the data reference. The network protocol stack pre-allocated an extra data packet buffer in step 3. The network protocol stack writes the generated data packet header into this reserved data packet buffer, and then links the data packet buffer that stores the corresponding data reference after the data packet buffer with the data packet header. This group of data packet buffers together represent a pre-encapsulated data packet with scattered data packet content.

[0025] Step 6: After the data packet encapsulation is completed, the session module notifies the event polling module that the data packet cache is ready, and the network protocol stack notifies the network card driver.

[0026] Step 7: The network card driver extracts the packet header or address and length information of the data in the packet buffer or shared memory, fills it into the descriptor, and places it in the descriptor ring. For linked packet buffers, the driver will traverse them one by one, extract information, fill the descriptor, and link them through the fields in the descriptor to obtain a string of descriptors, which is then written to the send descriptor ring. On network cards that support SGL features, the network card reads the packet header and data from the packet buffer and the data send queue respectively according to this string of descriptors, assembles them into a continuous data packet inside the network card, and then sends it out. After the transmission is completed, the network card marks this string of descriptors as complete.

[0027] Step 8: After receiving this series of completed descriptors, the network protocol stack hands over the data packet buffer to the memory separation module.

[0028] Step 9: The memory separation module checks the data packet cache one by one. For data packet caches that are marked and contain data references, the memory separation module deletes the reference information in the data packet cache, restores it to a normal data packet cache, and hands it over to the data packet cache resource pool for recycling.

[0029] Furthermore, development was based on the open-source User-Space Network Protocol Stack (VPP).

[0030] Furthermore, upper-layer applications communicate via shared memory and network protocol stacks to provide web services to the outside world.

[0031] Furthermore, the network protocol stack includes a memory connection module, a memory separation module, and a memory management module.

[0032] Furthermore, the memory connection module determines how many network packets need to be segmented for transmission; then, for each network packet, the memory connection module performs traversal, preparation, and connection steps to connect the data in the shared memory to the packet buffer.

[0033] Furthermore, the traversal counts how many blocks the data spans, and the data within each block is treated as a fragment, allowing the memory connection module to determine how much data packet cache resources should be allocated.

[0034] Furthermore, in the preparation process, the memory connection module allocates a set of data packet caches from the data packet cache resource pool, with one more data fragment than the number obtained through traversal. The module reserves the first data packet in this set of caches for subsequent transport / network layers to add packet headers.

[0035] Furthermore, in the connection, the memory connection module sequentially fills the address and length information of the data segments obtained through traversal into this group of data packet caches, completing the connection between the data in shared memory and the data packet cache. Then, the memory connection module will point the "next" of the previous one in this group of data packet caches to itself, thereby linking it with the previous data packet caches into a linked list.

[0036] Furthermore, in step 5, the network protocol stack generates data packet headers according to the network transmission protocol.

[0037] Furthermore, in step 5, the network protocol stack generates data packet header information including network connection and data length.

[0038] In a preferred embodiment of the present invention, during the transmission path, the data copying from shared memory to the packet buffer by the network protocol stack results in two identical copies of data in both shared memory and the packet buffer. Both copies occupy LLC (Limited Memory). When the amount of data transmitted is very large, LLC is exhausted, and numerous LLC store misses occur during the network protocol stack memory copy, leading to a degrade in network protocol stack performance. The present invention eliminates the memory copying from shared memory to the packet buffer using a zero-copy approach within the network protocol stack, thus eliminating the duplicate data in the packet buffer. Instead of copying from shared memory to the packet buffer, a reference to the data in shared memory is written into the packet buffer, eliminating the data duplicates within the packet buffer.

[0039] Data in shared memory belongs to the application layer and is incompatible with the data structures of the underlying transport / network layer. Achieving zero-copy requires solving the problem of encapsulating application-layer data packets and then handing them over to the network card for transmission. This invention adds support for data references. The original continuous data packets are changed to a discrete form, utilizing the characteristics of the network card to send discrete data packets. The data structure of the data packet buffer is expanded; the data packet buffer is only used to store references to the packet header or data. Multiple data packet buffers are chained together into a linked list according to the packet header-data reference order to represent a discrete data packet. The discrete data packets are then handed over to a network card that supports SGL features, which then aggregates the discrete data packets into a continuous data packet for transmission.

[0040] In the original network protocol stack, the packet buffer was immediately reclaimed by the packet buffer pool after transmission. Now, the packet buffer contains references to application layer data. Depending on the reliability design of different transport protocols, this data may need to be retained and eventually released by the management unit in shared memory. The problem of proper packet buffering and data reclamation needs to be addressed. This invention adds a memory separation module to ensure that the memory release of the application layer and the transport / network layer is independent and that both are correctly reclaimed. Before the packet buffer is handed over to the corresponding management unit for release and reclamation, the memory separation module deletes the references within the packet buffer, severing the connection between shared memory and the packet buffer. Afterward, the packet buffer management unit and the management unit in shared memory manage their respective memory separately.

[0041] Compared with the prior art, the present invention has the following obvious substantive features and significant advantages:

[0042] 1. This invention eliminates memory copying, avoiding the impact of LLC store misses caused by memory copying on the performance of the network protocol stack, and improving the network performance of the network protocol stack. It saves LLC resources previously occupied by data copies in the packet buffer, eliminates the problem of two identical copies of data in the LLC, reduces LLC usage, and allows the saved LLC to cache more transmitted data.

[0043] 2. This invention links shared memory and packet caching, encapsulating data into a complete data packet without modifying the application layer's data structure. Finally, the network interface card (NIC) directly reads the packet header from the transport layer and the data from the application layer before sending the packet out.

[0044] 3. This invention avoids the coupling between data packet buffer management logic and shared memory data management logic. It ensures the correctness of shared memory data release under zero-copy mode.

[0045] 4. This invention enables the network protocol stack to process network data packets faster; at the same time, the network protocol stack saves considerable CPU resources used for memory copying, which can be used to process more network data packets, thus improving the performance of the network protocol stack.

[0046] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0047] Figure 1 shows the VPP network protocol stack transmission process;

[0048] Figure 2 shows the LLC usage of the application and VPP protocol stack under different numbers of long connections;

[0049] Figure 3 shows the protocol stack CPU LLC miss rate under different numbers of long connections;

[0050] Figure 4 is a general process framework diagram;

[0051] Figure 5 is a flowchart of the memory connection module;

[0052] Figure 6 shows the transmission process when the transmission path within the protocol stack is in zero-copy mode. Detailed Implementation

[0053] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0054] In the accompanying drawings, components with the same structure are indicated by the same numerical designation, and components with similar structures or functions are indicated by similar numerical designations. The dimensions and thicknesses of each component shown in the drawings are arbitrary, and the present invention does not limit the dimensions and thicknesses of each component. To make the illustrations clearer, the thickness of some components has been appropriately exaggerated in the drawings.

[0055] Given that memory copying on the transmission path generates two copies of data, which can exhaust LLC when the amount of data being transmitted is large, and the resulting large number of LLC store misses in the network protocol stack can lead to a decrease in packet processing speed, the technical problem to be solved by this invention is how to eliminate memory copying on the transmission path within the network protocol stack through zero-copy, thereby avoiding the LLC store misses caused by memory copying that reduce the packet processing speed, and at the same time eliminating the duplication of data within LLC, so that the saved LLC can be used to buffer more transmitted data and improve the performance of the network protocol stack.

[0056] To address this, this invention provides an in-stack zero-copy transmission path based on a shared memory communication mode to achieve the above objectives. This invention is developed based on the open-source User-Space Network Protocol Stack (VPP). Upper-layer applications, such as Nginx, communicate with the network protocol stack via shared memory to provide web services. The developed network protocol stack includes a memory connection module, a memory separation module, and a memory management module, and its overall architecture is shown in Figure 4.

[0057] On the application side, when the application calls the socket's send interface, the VPP library loaded into the application via LD_PRELOAD will translate this send call into an inter-process communication based on shared memory. The communication mainly involves two steps, as shown in Figure 4:

[0058] [1]. The application writes a send event to the I / O event queue to notify the network protocol stack that there is a send request.

[0059] [2]. The application copies the data in its private memory to the data sending queue in the shared memory and hands the data over to the network protocol stack.

[0060] Implementing zero-copy on the application side means the network protocol stack needs to read data from the application's private memory and send it out, requiring the application's private memory to be visible to the network protocol stack. This allows malicious applications to pollute the network protocol stack's memory space through buffer overflows, potentially contaminating the memory of other applications and causing security issues. Improving security by reducing the amount of private memory exposed by the application means the application only shares the memory containing the data to be sent with the network protocol stack and promptly revokes the sharing after data transmission. This results in the network protocol stack needing to modify the page table twice for each transmission to establish and delete the shared memory. However, modifying the page table requires kernel intervention, and modifying the page table for each transmission incurs significant overhead. Therefore, to avoid the security risks associated with exposing the application's private memory and the substantial overhead of frequent page table modifications, the original memory-copy-based transmission logic on the application side is retained.

[0061] The overall process of data transmission on the network protocol stack side in Figure 4 is described as follows:

[0062] (1)(2) The event polling module of the network protocol stack receives the application's I / O request from the I / O event queue through polling, and then the event polling module forwards the request to the session module for processing.

[0063] (3) After receiving the request to send, the session module finds the corresponding network session, finds the data sending queue based on the data sending queue information recorded in the network session, determines the position and length of the data to be sent, and then calls the memory connection module and passes the data sending information to the memory connection module, which then processes the data in the data sending queue.

[0064] (4) After obtaining the data information in the sending queue, the memory connection module calculates the required data packet buffer resources, allocates them from the data packet buffer resource pool, and then saves the data reference into the data packet buffer, replacing the original operation of copying the data to the data packet buffer. During this process, the module will allocate an additional data packet buffer for step (7). The detailed process of the memory connection module will be further explained below.

[0065] (5)(6) After the memory connection module writes the references into the data packet cache, it will hand this set of data packet caches over to the session module. At this time, the data packet cache only contains references pointing to the data in the application layer. It is still necessary to add packet headers according to the corresponding network transmission protocol and encapsulate them. Therefore, the session module hands these data packet caches over to the lower transport / network layer for encapsulation.

[0066] (7) The transport / network layer receives the session information and data packet buffer provided by the upper-layer session module, finds the corresponding network connection, and the network protocol stack generates the data packet header according to the specific network transport protocol, based on information such as the network connection and data length. In order to quickly distinguish whether a data packet buffer stores a reference to shared memory data, the network protocol stack stores the data packet header and data reference in different data packet buffers, and adds a mark to the data packet buffer that stores the data reference. Therefore, the network protocol stack pre-allocates an extra data packet buffer in step (4). The network protocol stack writes the generated data packet header into this reserved data packet buffer, and then links the data packet buffer that stores the corresponding data reference after the data packet buffer with the data packet header. This group of data packet buffers together represents a pre-encapsulated data packet with scattered data packet content.

[0067] (8)(9)(10) After the data packet encapsulation is completed, the session module will notify the event polling module that the data packet cache is ready, and the network protocol stack will then notify the network card driver.

[0068] (11) The network card driver extracts information such as the packet header or the address and length of the data in the shared memory from the packet buffer, fills it into the descriptor and puts it into the descriptor ring; for the linked packet buffers, the driver will traverse them one by one, extract information, fill the descriptor, and link them through certain fields in the descriptor to get a string of descriptors, and write them into the send descriptor ring.

[0069] (a)(b) On a network interface card (NIC) that supports SGL features, the NIC reads the packet header and data from the packet buffer and data transmission queue respectively based on this string of descriptors, assembles them into a continuous data packet internally, and then sends it out. After transmission is complete, the NIC marks this string of descriptors as complete.

[0070] (12)(13) After receiving this series of completed descriptors, in the original design, the network protocol stack would directly instruct the packet buffer resource pool to reclaim the corresponding packet buffer. However, the current packet buffer may contain references to data in shared memory. It is necessary to ensure that the memory occupied by the data in shared memory is correctly reclaimed. However, the introduction of upper application layer data management into the packet buffer resource pool management logic has destroyed the independence between layers. In order to avoid the coupling of memory management logic between the application layer and the transport / network layer, the network protocol stack will first hand over the packet buffer to the memory separation module before handing it over to the packet buffer resource pool for reclamation.

[0071] (14) The memory separation module will check the data packet cache one by one. For data packet caches that are marked and contain data references, the memory separation module will delete the reference information in the data packet cache and restore it to a normal data packet cache. After that, the data packet cache will be handed over to the data packet cache resource pool for recycling.

[0072] The memory connection module is a crucial part of the network protocol stack, eliminating the memory copying from shared memory to packet buffers in the original network protocol stack. While eliminating this memory copying, the goal is to: 1. Maintain the original well-designed layered structure of the VPP network protocol stack, avoiding tight coupling between different layers; 2. Minimize the performance cost of modifying the data structures and function logic of any layer.

[0073] Based on the above considerations, the network protocol stack abandoned the approach of modifying the data structure of the data transmission queue in shared memory to be compatible with the packet buffer data structure, thus avoiding coupling between the application layer and the transport / network layer data structures. Similarly, sharing the transport / network layer's packet buffer resource pool with the application to store application layer data is also unacceptable. Therefore, the memory connection module needs to encapsulate data from both the transport / network layer and the application layer together without copying, and then hand it over to the network card for transmission. To this end, the data structure of the transport / network layer's packet buffer was extended, allowing the packet buffer to reference a memory space outside its own data structure; and a scatter-aggregate linked list data structure was used to encapsulate the packets.

[0074] The main tasks of the memory connection module are as follows: First, based on the length of the data from the application provided by the session module, the MSS of the data packet, and considering the window size for TCP protocols, determine how many network packets need to be segmented for transmission. Then, for each network packet, the memory connection module performs three steps: traversal, preparation, and connection, connecting the data in shared memory to the packet buffer. Specifically, the traversal, preparation, and connection workflow of the memory connection module is shown in Figure 5.

[0075] (1) Traversal: The data sending queue is organized into a set of blocks in the form of a linked list. The space where data is stored in each block is not contiguous, and the data written by the application may span multiple blocks. Based on the address and length information of the data, the first thing the module needs to do is to traverse and count how many blocks the data spans, based on the address, length and information of each block. The data in each block is treated as a fragment, so that the module knows how much data packet buffer resources should be allocated in the next step.

[0076] (2) Preparation: Next, the module will allocate a set of packet buffers from the packet buffer resource pool, the number of which is one more than the number of data fragments obtained in step (1). The module will reserve the first packet in this set of packet buffers for the subsequent transport / network layer to add packet headers.

[0077] (3) Connection: The module fills the address and length information of the data segments obtained in step (1) into this group of data packet buffers in sequence, completing the connection between the data in shared memory and the data packet buffers. Then the module will make the "next" of the previous one in this group of data packet buffers point to itself, thus linking it with the previous data packet buffers into a linked list.

[0078] These packet buffers, together with the packet buffer reserved for the packet header, form a scatter-aggregate linked list representing network packets that have not yet been fully encapsulated. Each element of the linked list points to a fragment of this network packet.

[0079] After the memory connection module completes the connection, the scattered aggregated linked list representing an incompletely encapsulated network data packet is handed over to the subsequent transport / network layer. The protocol processing node of the transport / network layer traverses the data packet buffer in the linked list, obtains the data length, fills in the protocol header with information such as network connection and data length, and encapsulates it into a complete data packet.

[0080] The subsequent driver traverses this set of packet cache linked lists, generates a descriptor for each packet cache, and makes the "next" of each descriptor point to the next one, resulting in a string of descriptors.

[0081] The next descriptors can be handed over to the network card. With the help of the network card's SGL feature, the network card obtains all the segments based on the address and length information, connects them into a continuous network packet inside the network card, and then sends it out.

[0082] Therefore, by extending the data structure of the packet buffer and utilizing a distributed aggregated linked list, multiple packet buffers can be used to represent a single, fully encapsulated network packet that is actually discrete in memory, thus eliminating the need for copying data from shared memory to the packet buffer. With the help of a network interface card (NIC) supporting SGL features, the network protocol stack can send data from the application layer without copying it, while ensuring the integrity of the network packets.

[0083] In the original design, after the network card sends out the network data packets in its packet buffer, the packet buffer can be recycled back to the packet buffer resource pool. If the sent network data packets are lost, reliable network connections like TCP will copy the data from the application layer for retransmission. However, the zero-copy design uses the data in the application layer's shared memory directly as the content of the network data packets. In addition to achieving the purpose of sending the data, the network protocol stack also needs to manage and recycle the space occupied by the data and the packet buffer.

[0084] When managing and reclaiming shared memory data and packet buffers, the following aspects must be ensured to be correct: 1. Packet buffers are correctly reclaimed after being sent by the network card and can still be used normally afterwards; 2. The timing of data reclamation in shared memory is correct. For reliable network transmission protocols such as TCP, data in shared memory should only be reclaimed after ensuring that the data has been delivered to the client; 3. Regardless of whether the transport / network layer uses a reliable network transmission protocol, the content of the data in shared memory should remain unchanged before reclamation to ensure that the retransmitted content is correct.

[0085] To ensure correctness and minimize the coupling between application layer and transport / network layer memory management logic, a memory separation module was designed to manage and reclaim data from the network interface card's (NIC) packet buffers and associated shared memory. Placing the memory separation module before the packet buffer resource pool reclaims the packet buffer, and handling the logic for processing data references in shared memory within the packet buffer, avoids coupling between the packet buffer resource pool management logic and the shared memory data management and reclamation logic. The workflow of the memory separation module is described below:

[0086] Once the network interface card (NIC) has finished sending data packets, the buffer is handed over to the memory separation module. This module inspects these buffers, identifying those marked as containing references to data in shared memory. The memory separation module then removes these references and markings, restoring the buffers to their original, unmarked state. Since these packets no longer have data references, they can be subsequently handed over to the data packet buffer resource pool for recycling. This not only ensures that data packets can be recycled and reused but also prevents other functions in the network protocol stack from modifying application data through references when reusing the data packet buffer.

[0087] To ensure the reliability of network transmission protocols and guarantee that data is reclaimed at the correct time, a callback function is added to the data packet buffer. During the connection phase of the memory connection module, different functions can be provided as callback functions for the data packet buffers of different network transmission protocols. These callback functions are called in the memory separation module, and they determine whether to notify the application layer to reclaim the data based on the corresponding network transmission protocol. Taking TCP as an example, the network card cannot guarantee that the client will receive the data after transmission; therefore, the corresponding callback function should retain the data and reclaim it only after receiving the client's ACK.

[0088] On the new transmission path, network packet data resides in shared memory. Therefore, the network protocol stack needs to extend its existing memory management module to add the functionality of managing DMA mappings for shared memory. Given the high frequency of transmission requests in high-load scenarios, creating a DMA mapping for each data packet and then deleting it after transmission would incur significant overhead. Therefore, the memory management module establishes DMA mappings for the entire shared memory block when the application starts; when the application closes, the module deletes the DMA mappings. If DMA mapping fails due to limited system resources, such as the number of pages in the DMA mapping exceeding system limits, this block of shared memory is marked. Subsequently, the session module will use the original memory copy method to copy the data from the shared memory to the packet buffer.

[0089] Technical Effects: After changing the sending path within the network protocol stack to zero-copy, the process of data being sent from the application to the network protocol stack and finally to the network card was re-analyzed, as shown in Figure 6. Compared to the original process in Figure 1, steps 1 and 2 executed on the application side are the same, and the data written to shared memory is cached in LLC. On the network protocol stack side, the memory connection module allocates a packet buffer and then writes the reference to the data in shared memory into the packet buffer. Next, the network protocol stack generates the corresponding packet header for this data and writes it into another packet buffer. Then, the driver generates multiple descriptors and writes them to the descriptor ring. After the network card DMA descriptors, it DMAs the packet header and data in sequence. Compared to Figure 1, although the network card needs more DMA operations to send a network packet, the hardware performance of network cards is now very powerful. The overhead introduced by collecting the scattered packet headers and data into the network card has little impact on the network card's performance in scenarios involving large file transfers, and will not cause the network card to become a bottleneck in overall performance.

[0090] During this process, we can see that shared memory and packet buffers still occupy LLC, but the load of packet buffers in Figure 1 is replaced by data references. The network protocol stack saves a lot of LLC resources that were originally used for the load, and only a very small portion of resources are needed to store data references (because the allocated packet buffer will only be used by one thread of the network protocol stack, so data references can even be cached in L1 and L2 caches), thus achieving the goal of saving LLC resources. Moreover, the network protocol stack no longer needs to write a large amount of data to LLC, and the large number of LLC store misses caused by memory copying are avoided, allowing the network protocol stack to process network packets faster. At the same time, the network protocol stack saves a considerable amount of CPU resources used for memory copying, which can be used to process more network packets, improving the performance of the network protocol stack.

[0091] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A stack-based zero-copy transmission path method based on shared memory communication mode, characterized in that, Includes the following steps: Step 1: The event polling module of the network protocol stack receives the application's I / O requests from the I / O event queue through polling, and then forwards the requests to the session module for processing. Step 2: After receiving the send request, the session module finds the corresponding network session, locates the data send queue based on the data send queue information recorded in the network session, determines the position and length of the data to be sent, and then calls the memory connection module and passes the data send information to the memory connection module. Step 3: After the memory connection module obtains the information of the data in the sending queue, it calculates the required data packet cache resources, allocates them from the data packet cache resource pool, and then saves the reference of the data into the data packet cache, replacing the original operation of copying the data to the data packet cache. During this process, an additional data packet buffer will be allocated; Step 4: After the memory connection module writes the reference to the data packet cache, it hands the data packet cache over to the session module; the session module then hands the data packet cache over to the transport / network layer for packet encapsulation. Step 5: The transport / network layer receives the session information and data packet buffer provided by the session module, finds the corresponding network connection, and the network protocol stack generates a data packet header. In order to quickly distinguish whether the data packet buffer stores a reference to shared memory data, the network protocol stack stores the data packet header and data reference in different data packet buffers, and adds a mark to the data packet buffer that stores the data reference. The network protocol stack pre-allocated an extra data packet buffer in step 3. The network protocol stack writes the generated data packet header into this reserved data packet buffer, and then links the data packet buffer that stores the corresponding data reference after the data packet buffer with the data packet header. This group of data packet buffers together represent a pre-encapsulated data packet with scattered data packet content. Step 6: After the data packet encapsulation is completed, the session module notifies the event polling module that the data packet cache is ready, and the network protocol stack notifies the network card driver. Step 7: The network card driver extracts the packet header or address and length information of the data in the packet buffer or shared memory, fills it into the descriptor and puts it into the descriptor ring. For linked packet buffers, the driver will traverse them one by one, extract information, fill the descriptor, and link them through the fields in the descriptor to get a string of descriptors, which is then written into the send descriptor ring. On network cards that support SGL features, the network card reads the packet header and data from the packet buffer and the data sending queue respectively according to this string of descriptors, assembles them into a continuous data packet inside the network card, and then sends it out. After the transmission is complete, the network card marks this string of descriptors as complete; Step 8: After receiving this series of completed descriptors, the network protocol stack hands over the data packet buffer to the memory separation module. Step 9: The memory separation module checks each data packet cache. For data packet caches marked with data references, the memory separation module deletes the reference information within the data packet cache and restores it to a normal data packet cache. Hand it over to the data packet cache resource pool for recycling.

2. The stack-based zero-copy transmission path method based on shared memory communication mode as described in claim 1, characterized in that, Developed based on the open-source user-space network protocol stack VPP.

3. The stack-based zero-copy transmission path method based on shared memory communication mode as described in claim 1, characterized in that, Upper-layer applications communicate with the outside world through shared memory and network protocol stacks to provide web services.

4. The stack-based zero-copy transmission path method based on shared memory communication mode as described in claim 1, characterized in that, The network protocol stack includes a memory connection module, a memory separation module, and a memory management module.

5. The stack-based zero-copy transmission path method based on shared memory communication mode as described in claim 1, characterized in that, The memory connection module determines how many network packets need to be segmented for transmission; then, for each network packet, the memory connection module performs traversal, preparation, and connection steps to connect the data in shared memory to the packet buffer.

6. The stack-based zero-copy transmission path method based on shared memory communication mode as described in claim 5, characterized in that, The traversal counts how many blocks the data spans, and the data within each block is treated as a fragment, allowing the memory connection module to determine how much data packet cache resources should be allocated.

7. The stack-based zero-copy transmission path method based on shared memory communication mode as described in claim 5, characterized in that, In the preparation process, the memory connection module allocates a set of data packet caches from the data packet cache resource pool. The number of data packets is one more than the number of data fragments obtained through traversal. The module reserves the first data packet in this set of caches for subsequent transport / network layers to add packet headers.

8. The stack-based zero-copy transmission path method based on shared memory communication mode as described in claim 5, characterized in that, In the connection process, the memory connection module sequentially fills the address and length information of the data segments obtained through traversal into this group of data packet caches, completing the connection between the data in shared memory and the data packet caches. Then, the memory connection module will point the "next" of the previous one in this group of data packet caches to itself, thus linking it with the previous data packet caches into a linked list.

9. The stack-based zero-copy transmission path method based on shared memory communication mode as described in claim 1, characterized in that, In step 5, the network protocol stack generates data packet headers according to the network transmission protocol.

10. The stack-based zero-copy transmission path method based on shared memory communication mode as described in claim 1, characterized in that, In step 5, the network protocol stack generates data packet header information including network connection and data length.

Images