EBPF memory database acceleration method supporting stateful connection
By using the accumulation correction method and receiver scaling technology in the eBPF memory database, the problem that Memcached performance is limited by the kernel network stack is solved, and the TCP connection processing with high throughput and low latency is achieved, which significantly improves the performance scalability of Memcached and Throughput.
Patent Information
- Application Number
- CN202411142811.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-12-03
AI Technical Summary
The performance of Memcached is limited by the kernel network stack, especially the performance bottlenecks of the TCP protocol and the problem of packet retransmission caused by the inability to support stateful protocols, resulting in excessive CPU time consumption and poor performance scalability.
By modifying and maintaining the message sequence number using the accumulation correction method in the eBPF memory database, it supports stateful connections, accelerates TCP connection processing, and avoids concurrent access and locking through receiving end scaling technology and BPF MAP TYPE LRU HASH type mapping storage. contention overhead.
It significantly improves Memcached's throughput and request processing capabilities, reduces processing delays, and achieves better performance scalability and throughput improvements. Single-core CPUs can handle 1.05 million requests per second, and 4-core CPUs can handle 3M per second. Request, request processing delay is reduced by 56%.
Smart Images

Figure CN119066049A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data center performance optimization, and in particular to an eBPF memory database acceleration method supporting stateful connections. Background Art
[0002] Memcached is a high-performance in-memory key-value storage that is widely used by various large web service providers to cache high-frequency user requests to reduce the pressure on the backend database and respond to user requests faster. Memcached has been optimized from the beginning using various technologies to provide high performance. The recently developed eBPF technology can be used to build an additional in-kernel cache layer and process requests before the kernel network stack (i.e., pre-stack processing) to estimate overhead. Specifically, the query request is parsed before entering the stack, and if the requested key hits the cache, the response is returned directly and the request is discarded. Based on this paradigm, the latest work has built a Memcached acceleration system that supports the UDP protocol. It achieves throughput improvements in three aspects while avoiding the kernel bypass problem. First, the eBPF code is loaded after the kernel's built-in verifier check, which ensures security. Second, it is embedded in the existing protocol stack instead of occupying the CPU exclusively. Third, the scheme can achieve application-unawareness and avoid the extra work of redesigning the application.
[0003] However, Memcached, its performance is still limited by the kernel network stack, because the Linux network stack is designed to provide a standard unified interface for various applications; TCP is also affected by the performance bottleneck of the kernel network stack and is widely used by key-value stores in memory (such as Memcached, Redis); when using TCP connections to process query requests, memcached spends more than 50% of its CPU time on network-related system calls; more importantly, Redis, which has 89.2% of the market share, only supports the TCP protocol; unfortunately, existing work cannot support the stateful protocol TCP, because pre-stack processing will produce state deviations between the client and the server and put TCP into an unrecoverable packet retransmission state;
[0004] In view of the above technical defects, a solution is now proposed. Summary of the invention
[0005] The purpose of the present invention is to design an eBPF memory database acceleration method that supports stateful connections, which can use the "cumulative correction" method to modify and maintain the message sequence number, thereby achieving accelerated support for TCP connections.
[0006] The object of the present invention can be achieved by the following technical solutions: an eBPF memory database acceleration method supporting stateful connection, comprising step 1: a data packet received by a network card triggers the execution of an entry ma in the XDP program mount point, and filters data packets going to Memcached through the entry, and transmits the data to step 2;
[0007] Step 2: The response data packet generated by the Memcached application will trigger the execution of the out_main module located at the flow control TC mount point and pass the data to step 3;
[0008] Step 3: Use the receive-side scaling technology to always map the state processing of the same connection to the same CPU, avoiding potential concurrent access to the same entry in the state table. The data packet received by the network card will trigger the execution of an entry main in the program mount point, which will filter the data packets going to Memcached.
[0009] Preferably, the specific content of step 1 is: when the received data packet corresponds to a new connection, the connection state counter will be initialized here, and after using the value of the connection state counter to correct the seq and ack sequence number state repair and recalculate the checksum and update csum, the data length received and sent by the acceleration program will be accumulated and recorded, and the accumulated record follows the addition logic operation in mathematics. After receiving the data packet and before sending the data packet, the state correction module corrects the sequence number of the network packet according to the accumulated count, and the specific calculation process is as follows:
[0010] Suppose the sequence number of the data packet received by a connection at a certain time is SEQ in ,ACK in , the sequence number of the data packet to be sent is SEQ out ,ACK out , the length of the received data packet is length in , the length of the data packet to be sent is length out At this time, the cumulative length of the data packet processed by the acceleration program is L in ,L out ;
[0011] If the cache misses, the receiving direction sequence number should be corrected to SEQ in -L in ,ACK in -L out , the request is handed over to the upper-layer application for processing, and then the sequence number is changed to SEQ in the sending direction out +L out ,ACK out +L in , sent to the client:
[0012] If the cache hits, the sequence numbers of the receiving and sending directions are not modified, the request is handed over to the acceleration program for processing, and the cumulative count of the data packet is updated to L in +length in ,L out +length out ;
[0013] When the response packet generated by the upper-layer application enters the network card sending queue, it will also be processed by the program according to the addition and subtraction operations on its sequence number, so that it can finally be correctly received by the client. The GET data packet that hits the cache will exchange the prepared data packet in the data packet header and fill in the response content. It will be sent to the client after updating the checksum. In the write reply, the length of the incoming request and outgoing response will be accumulated in the connection status counter, and the counter will be updated.
[0014] Preferably, the specific content of step 2 is: the seq and ack sequence numbers of the data packet are corrected according to the value in the connection state counter through the state correction algorithm, the key value contained in the response, that is, the storage content of the database, is updated in the eBPF cache, and the data packet is sent after calculation and verification.
[0015] Preferably, the specific content of step three is: using BPF MAP TYPE LRU HASH type mapping storage to save the connection status, wherein the key is a structure containing a connection IP and port for identifying the connection, and the value is a flag containing an input and output length counter and a connection being established and closed; when multiple processes try to process the same user request at the same time, secure access to the state table is achieved by introducing lock contention overhead. In order to avoid additional overhead, the receiving end scaling RSS technology is used to always map the state processing of the same connection to the same CPU, thereby avoiding potential concurrent access to the same entry in the state table, thereby avoiding locking and unlocking overhead.
[0016] The beneficial effects of the present invention are as follows:
[0017] The throughput test results of the present invention are as follows Figure 4As shown in the figure, for Memcache with our method applied, part of the requests are processed by Memcached and the other part by Midd l eCache; when the number of CPU cores increases from 1 to 4, the request processing capability of Midd l eCache is significantly improved with the increase of the number of CPU cores. During this period, the CPU occupancy rate of the corresponding core of the server is 100%, which indicates that Midd l eCache has better performance scalability; for a single-core CPU, Memcached with our method applied can process about 1.05 million requests per second, which is 3.8 times the original throughput; for 4 cores, 3M requests can be processed per second, of which 2.8M are processed by Midd l eCache and the rest are processed by Memcached, which is 3.3 times the original performance.
[0018] The test results of request processing delay are as follows: Figure 5 As shown in the figure, the measurement of the processing delay of 10,000 requests shows that the median request processing delay of the Memcached with the application method is 7.9μs, and the median request processing delay of the original Memcached when the cache hits and misses are 17.8μs and 17.4μs respectively; this shows that the processing time of a single request can be reduced by 56%.
[0019] By further testing the effect of cache size on acceleration performance, the total throughput of Memcached program processing requests under different cache size configurations was tested. The experimental results are as follows: Figure 6 As shown in the figure, it is found that under all allocation ratios, Memcached that supports the eBPF memory database acceleration method with stateful connections has higher throughput than the original version; and only 1% additional cache is needed to achieve better performance than its original version; it can also be concluded from the figure that as the cache allocated to Midd l eCache gradually increases, the overall throughput also gradually increases; the reason why the performance does not improve after the cache percentage is greater than 25% is that the percentage of Memcached misses continues to increase, and most of the requests that can be processed have been processed by Mid d leCache. By increasing the total cache, the overall performance will be further improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The present invention will be further described below in conjunction with the accompanying drawings;
[0021] Figure 1 is a reference diagram of the method of the present invention;
[0022] Figure 2 It is a schematic diagram of the system architecture of this method.
[0023] Figure 3It is a processing flow chart of the acceleration system applying the method.
[0024] Figure 4 The figure shows the throughput improvement effect achieved after this method is applied to Memcached.
[0025] Figure 5 This is a test chart showing the decrease in request processing latency after this method is applied to Memcached.
[0026] Figure 6 This is a diagram of the total throughput effect under different cache size configurations. DETAILED DESCRIPTION
[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0028] Embodiment 1:
[0029] See also Figures 1 to 6 As shown, the present invention provides an eBPF memory database acceleration method supporting stateful connection, comprising the following steps: Step 1: The data packet received by the network card will trigger the execution of the entry main in the XDP program mount point, and filter the data packet to Memcached through the entry, and pass the data to step 2; Step 2: The response data packet generated by the Memcached application will trigger the execution of the out_ma in module located at the flow control TC mount point, and pass the data to step 3; Step 3: Use the receiving end scaling technology to always map the state processing of the same connection to the same CPU;
[0030] The eBPF memory database acceleration method that supports stateful connections consists of two parts: the eBPF program running in the kernel and the loader running in user space. During compilation, all eBPF programs will be packaged as a whole by the compiler. The loader will read the program package and extract each subroutine one by one to the kernel for verification and loading, and then bind it to a specific program mount point after loading. In addition, the loader is also responsible for creating and initializing the program mapping required for the tail call of the eBPF program, and establishing an association between the program's call address in memory and the program name for use during operation.
[0031] Figure 3 The processing flow chart of the acceleration system using this method is given; data packet reception: Figure 4As shown, the data packet received by the network card will trigger the XDP hook, that is, the execution of the entry ma in a program mount point, which will filter the data packets going to Memcached; if the received data packet corresponds to a new connection, the connection state counter will be initialized here; next, after the seq and ack sequence number state are corrected using the value of the connection state counter and the checksum is recalculated and updated, the SET request and GET request that do not hit the cache will be sent to the TCP stack; the GET packet that hits the cache will first exchange the source and destination in the packet header, and then fill in the response content to write the reply, and send it to the client after updating the checksum; in the write reply, the length of the incoming request and outgoing response will be accumulated in the connection state counter counter update;
[0032] Packet sending: The response packet generated by the Memcached application will trigger the execution of the out_main module located in the flow control TC hook; the seq and ack numbers of the packet are corrected here according to the values in the connection state counter through the state correction algorithm; then, the key value contained in the response, that is, the storage content of the database, is updated in the eBPF cache, and finally the packet is sent after calculation and verification;
[0033] Lock-free TCP state storage: We use BPF MAP TYPE LRU HASH type mapping storage to save the connection state. The key is a structure that contains the connection IP and port used to identify the connection, and the value is a flag containing the input and output length counters and the connection is being established and closed. Each entry only requires about 20 bytes. The maximum capacity of a single mapping of 4GB can store about 200M connections. This is completely sufficient for real-world applications. In our experiments, the capacity of the mapping storage is set to 8000. Since multiple CPUs may run their own eBPF programs at the same time, locks are required to ensure safe access to the state table. When multiple processes try to process the same user request at the same time, only one will succeed, and the rest need to wait in line, introducing lock contention overhead. In order to avoid additional overhead, the receive side scaling (RSS) technology is used to always map the state processing of the same connection to the same CPU, avoiding potential concurrent access to the same entry in the state table, thereby avoiding locking and unlocking overhead.
[0034] The above formulas are obtained by collecting a large amount of data for software simulation and selecting a formula that is close to the actual value. The coefficients in the formula are set by technical personnel in this field according to actual conditions. The above are only preferred specific implementations of the present invention, but the protection scope of the present invention is not limited to this. Any technical personnel familiar with the technical field within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, make equivalent replacement or change, which should be covered within the protection scope of the present invention.
Claims
1. An eBPF memory database acceleration method supporting stateful connections, characterized in that: The following steps are involved: Step 1: The data packet received by the network card will trigger the execution of the entry main in the XDP program mount point, and filter the data packets going to Memcached through this entry, and pass the data to step 2; Step 2: The response data packet generated by the Memcached application will trigger the execution of the out_main module located at the flow control TC mount point and pass the data to step 3; Step 3: Use receive-side scaling technology to always map the state processing of the same connection to the same CPU.
2. According to claim 1, a method for accelerating an eBPF memory database supporting stateful connections is characterized in that: The specific content of step one is: The received data packet corresponds to a new connection, so the connection state counter will be initialized here. After using the value of the connection state counter to correct the seq and ack sequence number status repair and recalculate the checksum and update csum, the data length received and sent by the acceleration program will be accumulated and recorded. The accumulated record follows the addition logic operation in mathematics. After receiving the data packet and before sending the data packet, the state correction module corrects the sequence number of the network packet according to the accumulated count. The specific calculation process is as follows: Suppose the sequence number of the data packet received by a connection at a certain time is SEQ in ,ACK in , the sequence number of the data packet to be sent is SEQ out ,ACK out , the length of the received data packet is length in , the length of the data packet to be sent is length out At this time, the cumulative length of the data packet processed by the acceleration program is L in ,L out : If the cache misses, the receiving direction sequence number should be corrected to SEQ in -L in ,ACK in -L out , the request is handed over to the upper layer application for processing, and then the sequence number is changed to SEQ in the sending direction out +L out ,ACK out +L in , sent to the client: If the cache hits, the sequence numbers of the receiving and sending directions are not modified, the request is handed over to the acceleration program for processing, and the cumulative count of the data packet is updated to L in +length in ,L out +length out ; When the response packet generated by the upper-layer application enters the network card sending queue, the processing program will also perform addition and subtraction operations on its sequence number so that it can be finally received correctly by the client.
3. The eBPF memory database acceleration method supporting stateful connections according to claim 1, characterized in that: The specific content of step 2 is: The packet's seq and ack sequence numbers are corrected by the state correction algorithm based on the values in the connection state counters, the key values included in the response, and updated in the eBPF cache.
4. The eBPF memory database acceleration method supporting stateful connections according to claim 1, characterized in that: The specific content of step three is: The connection status is saved using a BPF MAP TYPE LRU HASH type mapping storage, where the key is a structure containing the connection IP and port used to identify the connection, and the value is a flag containing the input and output length counters and the connection being established and closed; when multiple processes try to process the same user request at the same time, secure access to the status table is achieved by introducing lock contention overhead.
