Method for compatibility of VPP (Virtual Private Protocol) and DPDK (Data Platform Development Kit) sequence mechanism

Abstract

The invention provides a method for compatibility of a VPP (Virtual Private Protocol) and a DPDK (Data Platform Development Kit) session mechanism, and relates to the technical field of computer network communication, in an application process, a memory is mapped to a buffer memory allocated by the VPP, then a buffer is pre-allocated in the VPP, a buffer index is transmitted to the application process through ring of the DPDK, after the application process obtains the index, a buffer address is re-calculated, and the buffer address is sent to the VPP; in this way, the buffer in the VPP can be operated in the application process, the application process can efficiently access a data packet received by the VPP from a network port, the VPP can also efficiently forward data which the application process wants to send, the application process can be bound to different CPU cores according to needs, the purpose of flexible deployment is achieved, and the data can also be efficiently processed.

Description

Technical Field

[0001] This invention relates to the field of computer network communication technology, and in particular to a method compatible with VPP and DPDK secondary mechanisms. Background Technology

[0002] DPDK (Data Plane Development Kit) is a high-performance network driver component designed to significantly improve network processing performance by bypassing the kernel-mode limitations of the operating system's protocol stack through application-level processing of network packets. VPP (Vector Packet Processing) is a scalable framework that provides powerful application-layer IP protocol stack processing capabilities. Both are typically used in combination in high-performance IP data processing scenarios, such as high-end switches / routers, base stations, and core networks—network devices with high data forwarding requirements.

[0003] DPDK provides primary and secondary mechanisms for efficient data sharing between processes, a common data processing approach. When VPP integrates with DPDK, it takes over the allocation and release of DPDK's data memory, thus not supporting DPDK secondary mode. Furthermore, VPP uses a special memory management method, preventing application processes from sharing data within VPP via simple shared memory. Therefore, sharing network packets received in VPP within an application process using other inter-process communication mechanisms inevitably involves data copying, which is highly inefficient and significantly reduces system data processing capabilities. Integrating the process as a VPP plugin not only changes the original deployment method but also requires extensive adaptation modifications, increasing portability. Moreover, in multi-core systems, independent application processes can adjust their CPU core affinity to maximize multi-core system performance. Integrating the process as a plugin into VPP, making it a single process, will significantly degrade system performance. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a method compatible with VPP and DPDK secondary mechanisms, comprising:

[0005] During VPP initialization, large page memory is allocated according to the configuration and mapped to shared memory using the mmap method. The mapped file handle is saved to the corresponding data structure and recorded as vfd. VPP divides the allocated large page memory into N buffers of 2048 bytes each and records them as vbuf. The N buffers are placed into a buffer queue and recorded as vbuf_pool. VPP subtracts the starting address from the buffer address and shifts it to the right by 6 bits to use as the buffer index and records it as ibuf. The ibuf is stored in a 32-bit array in the buffer queue.

[0006] When VPP initializes DPDK, it registers DPDK's memory operation functions by calling the rte_mempool_register_ops function. This is used to call the DPDK rte_pktmbuf_alloc function to allocate a buffer and record it as mbuf. This allows VPP to obtain a usable buffer from its vbuf_pool and return the buffer to the vbuf_pool when calling DPDK rte_pktmbuf_free.

[0007] Modify the dpdk_config function to remove the use of the --in-memory parameter;

[0008] In VPP, a new node of type VLIB_NODE_TYPE_PROCESS is added and denoted as tnode. This node obtains the vfd and the size vsize and starting address vbase of the shared memory it points to, and saves them to a custom data structure. Then, a thread is started, and a domain socket server is started in the thread and listens for connections. When the application process starts, the client connects to this server through the domain socket. After a successful connection, the server sends the vfd and the size vsize and starting address vbase of the shared memory it points to to the client. In the client, mmap is called to remap the shared memory using the vfd and the size vsize and starting address vbase of the shared memory it points to, and the client obtains the mbuf.

[0009] In the tnode, the DPDK rte_ring_create function is called to create two circular queues, mring and fring. mring is used to store ibufs requested from vbuf_pool, and fring is used to store ibufs that the application process needs to release. During initialization, the vlib_buffer_pool_get function is called to request a certain number of ibufs and put them into the circular queue mring, while the circular queue fring is initialized to empty.

[0010] Add a new node of type VLIB_NODE_TYPE_INPUT in VPP and denot it as mnode to monitor the circular queues mring and fring. If the number of ibufs in the circular queue mring decreases by a certain threshold, then allocate the same number of ibufs in the circular queue mring. If the number of ibufs in the circular queue fring exceeds a certain threshold, then return the excess ibufs to the buffer queue.

[0011] In the application process, the DPDK interface is called to initialize it to secondary mode, a domain socket connection is created to the server, the values ​​of vfd, vsize and vbase are obtained, the mmap function is called to perform mapping, and after successful mapping, the starting address of the memory block that is the same as the VPP big page memory is obtained and recorded as abase.

[0012] When an application process sends data, it first obtains an ibuf from the circular queue mring, and then calculates the corresponding vbuf address using the following formula:

[0013] vbuf=ibuf<<6+abase

[0014] IP packets are assembled directly using vbuf and then handed over to VPP for transmission;

[0015] To manipulate the mbuf, the mbuf address is obtained through data conversion:

[0016] mbuf=(((struct rte_mbuf *)vbuf)-1)

[0017] When the application process releases the buffer, if it is operating on vbuf, ibuf is calculated using the following formula:

[0018] ibuf=(abase-vbuf)>>6

[0019] If you are operating on an mbuf, first convert the mbuf to a vbuf. The conversion method is as follows:

[0020] vbuf=((vlib_buffer_t *)(mbuf+1))

[0021] After obtaining ibuf, put it directly into fring to release the buffer.

[0022] The beneficial effects of the present invention are as follows:

[0023] This invention proposes a method compatible with VPP and DPDK secondary mechanisms. In the application process, memory is mapped to buffer memory allocated by VPP. Then, buffers are pre-allocated in VPP, and the buffer index is passed to the application process via DPDK's ring. After receiving the index, the application process recalculates the buffer address, allowing it to manipulate the buffers in VPP. This enables the application process to efficiently access data packets received from the network interface by VPP and allows VPP to efficiently forward data that the application process wants to send. Furthermore, the application process can be bound to different CPU cores as needed, achieving flexible deployment and efficient data processing.

[0024] It should be understood that the description in the foregoing summary section is not intended to limit the key or essential features of the embodiments of the present invention, nor is it intended to restrict the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 A schematic diagram of a method compatible with VPP and DPDK secondary mechanism is provided for an embodiment of the present invention;

[0027] Figure 2 The results of running 5G base station services and performing traffic packet injection on Linux based on an 8-core ARM Cortex-A72 processor (1.8GHz) using the embodiments of this invention.

[0028] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0031] like Figure 1 As shown, this disclosure provides a method for VPP to be compatible with the DPDK secondary mechanism, including the following steps:

[0032] During VPP initialization, large page memory is allocated according to the configuration and mapped to shared memory using the mmap method. The mapped file handle is also saved to the corresponding data structure (defined as vlib_physmem_map_t in VPP, and this file handle will be referred to as vfd below). Therefore, there is a feasible basis for sharing this data between processes. VPP divides this large page memory into N (large page memory / 2048) buffers (defined as vlib_buffer_t in VPP, hereinafter referred to as vbuf) of size 2048 bytes each, and puts these vbufs into a buffer queue (the corresponding data structure in VPP is vlib_buffer_pool_t, hereinafter referred to as vbuf_pool). Since the lower 6 bits of the vbuf address minus the starting address are all 0, VPP right-shifts the subtraction value by 6 bits as the buffer index. In this way, the buffer queue can use a 32-bit array to store these indices, reducing space usage. At the same time, when using vbuf, an available index (referred to as ibuf) is requested from vbuf_pool, and then the vbuf address can be obtained and the data in it can be accessed through simple calculations, without significant performance degradation.

[0033] When VPP initializes DPDK, it registers DPDK's memory operation functions by calling the `rte_mempool_register_ops` function. This is primarily to ensure that when the DPDK `rte_pktmbuf_alloc` function allocates a buffer (defined as `rte_mbuf` in DPDK, and referred to as `mbuf` hereafter), a single usable buffer can be obtained from VPP's `vbuf_pool`. When the DPDK `rte_pktmbuf_free` function is called, the buffer can be returned to the `vbuf_pool`, thus achieving unified buffer management. `vbuf` is compatible with `mbuf`; efficient conversion between `vbuf` and `mbuf` can be achieved using a fixed address offset.

[0034] Therefore, the existing mechanism cannot support directly using the DPDK secondary method to access buffers in VPP within the application process. Firstly, application processes cannot directly share buffers in VPP. Secondly, even if sharing were possible, the mbuf allocated via rte_pktmbuf_alloc cannot directly correspond to the vbuf. The following section will focus on how to solve this problem.

[0035] When VPP initializes DPDK, it passes the "--in-memory" parameter, which is why the application process cannot be initialized in secondary mode. To fix this, modify the dpdk_config function and simply remove the "--in-memory" parameter.

[0036] In VPP, a new node (referred to as tnode) of type VLIB_NODE_TYPE_PROCESS is created. This node primarily obtains the vfd, along with the size (referred to as vsize) and starting address (referred to as vbase) of the shared memory it points to, and saves these values ​​to a custom data structure. Then, a thread is started, within which a domain socket server is initiated and begins listening. When the application process starts, a domain socket client connects to this server. Upon successful connection, the server sends the vfd, shared memory size, and starting address to the client. The client can then use this information to call mmap to remap the shared memory. This allows the client to access data in VPP, but not directly. The client needs to obtain the mbuf (memory unit) to manipulate the data within it. These mbufs are actually managed by VPP and require allocation and release using VPP interfaces for proper access.

[0037] Within a tnode, the DPDK `rte_ring_create` function is called to create two circular queues (referred to as `mring` and `fring`). `mring` stores ibufs allocated from the `vbuf_pool`, while `fring` stores ibufs to be released by application processes. During initialization, the `vlib_buffer_pool_get` function is called to allocate 65536 ibufs (this value is empirical and needs to be adjusted based on performance on different systems), which are then placed into `mring` for use by application processes. The `fring` queue is initialized to empty.

[0038] A new node (referred to as mnode) is added to VPP, of type VLIB_NODE_TYPE_INPUT. This type of node will be continuously called in the subsequent VPP dispatch process, ensuring the real-time performance of node processing. In mnode, the main function is to monitor mring and fring. If the number of ibufs in mring decreases by a certain threshold (e.g., 512, this is an empirical value and can be adjusted according to system performance), 512 more ibufs are immediately allocated to ensure that mring always has enough ibufs for application processes. If the number of ibufs in fring exceeds a certain threshold (e.g., 512, this is an empirical value and can be adjusted according to system performance), these ibufs are immediately returned to vbuf_pool. VPP does not provide a direct interface; it is necessary to obtain vbuf_pool and, through locking, write the ibufs to be released into the buffers of vbuf_pool.

[0039] In the application process, the DPDK interface is called to initialize in secondary mode, ensuring access to the mring and fring created in the tnode. Then, a domain socket connection is created to the server to obtain the values ​​of vfd, vsize, and vbase. The mmap function is then called to perform mapping. After successful mapping, the starting address (denoted as abase) pointing to the same memory block as the VPP big page memory can be obtained.

[0040] When an application process wants to send data, it first obtains an ibuf from the mring, and then the corresponding vbuf address can be calculated using the following formula (which can be done by calling the VPP interface vlib_get_buffer_index):

[0041] vbuf=ibuf<<6+abase

[0042] IP packets can be assembled directly using vbuf and then handed over to VPP for transmission. If you want to manipulate mbuf, you can also convert it using the following formula (you can call the VPP macro rte_mbuf_from_vlib_buffer) to obtain the mbuf address:

[0043] mbuf=(((structrte_mbuf*)vbuf)-1)

[0044] Among them, structrte_mbuf* is a user-defined data structure;

[0045] When an application process wants to release the buffer, if it is operating on vbuf, it can calculate ibuf using the following formula (which can be done by calling the VPP interface vlib_get_buffer):

[0046] ibuf=(abase-vbuf)>>6

[0047] If you are operating on an mbuf, you can first convert the mbuf to a vbuf (you can call the VPP macro vlib_buffer_from_rte_mbuf), as shown in the following formula:

[0048] vbuf=((vlib_buffer_t*)(mbuf+1))

[0049] Among them, vlib_buffer_t* is a custom data structure;

[0050] Once you obtain the ibuf, simply put it into fring to release the buffer. The application process does not need to worry about the subsequent processes.

[0051] In this way, through the cooperation between mnode and application process, buffer allocation and release can be completed efficiently, thus providing a solid foundation for application process to process IP packets at high speed.

[0052] Applying this solution to 5G base station products required only about 200 lines of code modifications to the original system, enabling porting and adaptation with minimal cost. Running 5G base station services on Linux based on an 8-core ARM Cortex-A72 processor (1.8GHz) and performing traffic packet flooding, the downlink speed reached 5038.579 Mbits / s, meeting service requirements and approaching the performance limits of VPP and CPU. Figure 2 As shown.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method compatible with VPP and DPDK secondary mechanisms, characterized in that, include: During VPP initialization, large page memory is allocated according to the configuration and mapped to shared memory using the mmap method. The mapped file handle is saved to the corresponding data structure and recorded as vfd. VPP divides the allocated large page memory into N buffers of 2048 bytes each and records them as vbuf. The N buffers are placed into a buffer queue and recorded as vbuf_pool. VPP subtracts the starting address from the buffer address and shifts it to the right by 6 bits to use as the buffer index and records it as ibuf. The ibuf is stored in a 32-bit array in the buffer queue. When VPP initializes DPDK, it registers DPDK's memory operation functions by calling the rte_mempool_register_ops function. This is used to call the DPDK rte_pktmbuf_alloc function to allocate a buffer and record it as mbuf. This allows VPP to obtain a usable buffer from its vbuf_pool and return the buffer to the vbuf_pool when calling DPDK rte_pktmbuf_free. Modify the dpdk_config function to remove the use of the --in-memory parameter; In VPP, a new node of type VLIB_NODE_TYPE_PROCESS is added and denoted as tnode. This node obtains the vfd and the size vsize and starting address vbase of the shared memory it points to, and saves them to a custom data structure. Then, a thread is started, and a domain socket server is started in the thread and listens for connections. When the application process starts, the client connects to this server through the domain socket. After a successful connection, the server sends the vfd and the size vsize and starting address vbase of the shared memory it points to to the client. In the client, mmap is called to remap the shared memory using the vfd and the size vsize and starting address vbase of the shared memory it points to, and the client obtains the mbuf. In the tnode, the DPDK rte_ring_create function is called to create two circular queues, mring and fring. mring is used to store ibufs requested from vbuf_pool, and fring is used to store ibufs that the application process needs to release. During initialization, the vlib_buffer_pool_get function is called to request a certain number of ibufs and put them into the circular queue mring, while the circular queue fring is initialized to empty. Add a new node of type VLIB_NODE_TYPE_INPUT in VPP and denot it as mnode to monitor the circular queues mring and fring. If the number of ibufs in the circular queue mring decreases by a certain threshold, then allocate the same number of ibufs in the circular queue mring. If the number of ibufs in the circular queue fring exceeds a certain threshold, then return the excess ibufs to the buffer queue. In the application process, the DPDK interface is called to initialize it to secondary mode, a domain socket connection is created to the server, the values ​​of vfd, vsize and vbase are obtained, the mmap function is called to perform mapping, and after successful mapping, the starting address of the memory block that is the same as the VPP big page memory is obtained and recorded as abase. When an application process sends data, it first obtains an ibuf from the circular queue mring, and then calculates the corresponding vbuf address using the following formula: vbuf=ibuf<<6+abase IP packets are assembled directly using vbuf and then handed over to VPP for transmission; To manipulate the mbuf, the mbuf address is obtained through data conversion: mbuf=(((struct rte_mbuf *)vbuf)-1) When the application process releases the buffer, if it is operating on vbuf, ibuf is calculated using the following formula: ibuf=(abase-vbuf)>>6 If you are operating on an mbuf, first convert the mbuf to a vbuf. The conversion method is as follows: vbuf=((vlib_buffer_t *)(mbuf+1)) After obtaining ibuf, put it directly into fring to release the buffer.

Images