A load balancing method and device based on pipeline forwarding model

By using a pipelined forwarding model and a symmetric hash algorithm, the problem of uneven distribution of packets from the same session across different CPU cores was solved, enabling the processing of data packets from the same source and destination, and improving the overall performance of the server.

CN111984415BActive Publication Date: 2024-11-29BEIJING ACT TECH DEV CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202010853485.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-08-24
Publication Date
2024-11-29
Estimated Expiration
2040-08-24

AI Technical Summary

Technical Problem

In existing technologies, bidirectional messages from the same session may be distributed to different CPU cores, requiring CPUs to exchange information, which affects performance. In particular, the failure to implement symmetric RSS when using DPDK leads to performance degradation.

Method used

A load balancing method based on a pipeline forwarding model is adopted. It utilizes an I/O packet receiving logic engine and a symmetric hash calculator to distribute packets of the same session to the same CPU core through double hashing. Lock-free queue storage is implemented through a ring buffer queue to ensure that data packets are processed independently on different CPU cores.

Benefits of technology

It enables the processing of data packets from the same source and destination without using a splitter, improving the overall performance of the server and avoiding the impact of information interaction between CPU cores.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111984415B_ABST
    Figure CN111984415B_ABST
Patent Text Reader

Abstract

A load balancing method and device based on an assembly line forwarding model relate to the field of information technology. The present invention is composed of an I / O packet receiving logic engine, a symmetric hash calculator, a ring area cache queue, and a first business logic engine to an N-1th business logic engine; the I / O packet receiving logic engine and the business logic engine of the present invention occupy different CPU cores respectively, and the data of packet receiving and business processing are connected through the ring area cache queue, so that packet receiving and business processing are independent of each other and do not affect each other. The same source and destination can be achieved without the need for a diversion device, and at the same time, business processing does not affect the reception or forwarding of the original data packet. It solves the problem that under different traffic models, the traffic distribution to each CPU core is not very uniform, affecting the processing performance of the overall server.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of information technology, in particular to the field of load balancing technology of computer computing cores in a multi-core state. Background Art

[0002] In the past 10 years, Ethernet interface technology has experienced rapid development. Companies such as Intel and Mellanox have also launched ultra-high-speed network interface technologies of 40Gbit / s, 100Gbit / s, and 200Gbit / s. However, the operating frequency of the CPU has basically remained at the level of 10 years ago, which has seriously hindered the improvement of speed. In order to meet the challenges of ultra-high-speed network technology, the data plane technology DPDK, which is mainly based on software optimization, has emerged. It provides an excellent platform for high-performance packet processing. DPDK is a data plane development tool set provided by Intel. It mainly uses the x86 general platform and provides library functions and driver support for efficient packet processing in user space. Unlike the Linux system, which is designed for universality, it focuses on high-performance processing of packets in network applications. DPDK has made the following improvements based on the problems of the existing Linux network model: the data layer is entirely handled by the application, reducing system scheduling, reducing system calls, reducing system interrupts and context switching; abandoning the Linux kernel protocol stack, transferring data packets to the user space customized protocol stack; using multi-core programming technology instead of multi-threading, binding the OS to a specified core to run; for SMP symmetric multi-processing systems, the CPU is made to use the memory of the NUMA non-uniform memory access system node as much as possible to reduce memory flushing; using large pages to reduce access; and using lock-free technology to solve multi-core or multi-threaded competition problems.

[0003] Due to the load balancing and routing asymmetry of the original data link, multiple messages of the same session may be load balanced to multiple links. The so-called load balancing is to distribute the load to multiple execution units for execution. For network traffic, it means distributing data packet processing to multiple links or multiple CPU cores. In this way, the messages of the same session will be distributed to different ports, and the server will receive the messages of the same session from different network card ports. In this way, there will be message interaction between different CPUs, which will inevitably affect performance. The industry usually uses diversion equipment to perfectly solve this problem by converging sessions for all links. This is the so-called same source and same destination, which means that all messages belonging to the same session are converged to the same port to complete data processing. In simple terms, the same source and same destination ensures that the traffic of the same session on different input lines can be output on the same interface.

[0004] The pipeline model is inspired by the industrial pipeline model, which decomposes a function into multiple independent stages, and delivers products between different stages through queues. In this way, for some CPU-intensive and I / O-intensive applications, the pipeline model can be used to execute CPU-intensive tasks on one CPU core and I / O-intensive tasks on another CPU core. By binding different CPU cores, different threads can be assigned to different tasks, and the queues connecting the two can match the processing speeds of the two, thereby achieving the best concurrent performance.

[0005] In the prior art, when the RSS function is enabled in DPDK, although the messages of the same session will be distributed to the same CPU, the bidirectional messages of the same session will be distributed to two different CPUs because the RSS key of DPDK itself is not symmetric. In this way, the two CPUs will interact with each other in this session information, which will inevitably affect the performance. It is necessary to distribute the bidirectional messages of the same session to the same CPU.

[0006] The present invention proposes a load balancing method and device based on a pipeline forwarding model, which uses simple information aggregation RSS technology to distribute to some CPU cores to receive data packets; and then distributes to other CPU cores to process services through secondary hashing. The pipeline model solves the problem of achieving the same source and the same destination without the need for diversion equipment, and at the same time, service processing does not affect the reception of data packets. It makes full use of CPU multi-cores to improve the overall performance of the server.

[0007] Description of the prior art

[0008] RSS is a network card driver technology that can efficiently distribute received messages between multiple CPUs in a multi-processor system. RSS is that the network card first parses the received message to obtain the five-tuple information of IP address, protocol and port. Then the hash value is calculated through the hash function according to the keyword and hash key, and then RETA, the index of the Redirection table, is obtained from the hash value. Finally, it is distributed to the corresponding network card queue according to the value stored in RETA. The network card will select different keyword types according to different data packet types. For example, the keyword of the IPV4 TCP packet is composed of four tuples, including: source IP address S-IP, destination IP address D-IP, source port number S-Port, and destination port number D-Port. Dynamic load balancing can be achieved by modifying RETA.

[0009] In network applications, if the bidirectional messages of the same connection are distributed to the same CPU for processing after RSS is enabled, this type of RSS is called symmetric RSS. For network applications that need to save some information for the connection, symmetric RSS is very helpful for performance improvement. If the bidirectional messages of the same connection are distributed to different CPUs, then sharing the information of this connection between the two CPUs will involve locks, which will obviously affect performance.

[0010] RSS generally uses the Toeplitz hash algorithm, which has two inputs: a default hashkey and a keyword extracted from the message. The default hash key used by DPDK is recommended by Microsoft. The hash values ​​calculated using this default hash key in different directions of the same connection are different. For example, the five-tuple information is src: 168.21.32.1, dst: 212.12.24.8, srcport: 45235, dstport: 80 and the five-tuple information is src: 212.12.24.8, dst: 168.21.32.1, srcport: 80, dstport: 45235. The calculated hash values ​​are different. Different hash values ​​will cause the messages in the two directions to be distributed to different receive queues and processed by different CPUs.

[0011] To generate symmetric RSS, the default hash key needs to be modified. The length of the RSS hash key is 40 bytes (320 bits), denoted as RSK

[40] . If you want to generate symmetric RSS, as long as the RSS hash key is a 32-bit cyclic method, then for the two-way messages of the same connection, whether it is an uplink message or a downlink message, the final calculated hash value is the same. Summary of the invention

[0012] In view of the shortcomings of the prior art, the present invention provides a load balancing method and device based on a pipeline forwarding model, which consists of an I / O packet receiving logic engine, a symmetric hash calculator, a ring cache queue, and a first business logic engine to an N-1th business logic engine; the ring cache queue consists of a first-in-first-out lock-free queue with the same number of CPU cores;

[0013] The I / O packet receiving logic engine is powered by the first CPU core. It uses symmetric RSS technology to obtain data packets from the queues of the first network card to the Nth network card, decode the data packets, and extract binary information. The binary information includes: source IP address and destination IP address.

[0014] The symmetric hash calculator is provided with computing power by the first CPU core, and uses the symmetric hash algorithm to perform secondary hash calculation on the binary information of the data packet received by the I / O packet receiving logic engine. According to the result of the secondary hash calculation, the data packet corresponding to the result of the secondary hash calculation is allocated and stored in the first-in-first-out lock-free queue in the ring area cache queue module; the data packets with the same result of the secondary hash calculation will be allocated to the same first-in-first-out lock-free queue, and the data packets with different results of the secondary hash calculation will be evenly allocated to different first-in-first-out lock-free queues for storage;

[0015] The secondary hash calculation uses a symmetric hash algorithm and supports both IPV4 and IPV6 dual-stack traffic. The pseudo code of the algorithm is as follows:

[0016] function second_hash(ethe_type, data, hash_seed)

[0017] if ether_type == ETHER_TYPE_IPv4 then

[0018] struct ipv4_hdr *k data

[0019] sip_hash_value jhash_1word(k->src_addr, hash_seed)

[0020] dip_hash_value jhash_1word(k->dst_addr, hash_seed)

[0021] return sip_hash_value + dip_hash_value

[0022] else if ether_type == ETHER_TYPE_IPv6 then

[0023] struct ipv6_hdr *k data

[0024] sip_hash_value jhash(k->src_addr, IPV6_ADDR_LEN, hash_seed)

[0025] dip_hash_value jhash(k->dst_addr, IPV6_ADDR_LEN, hash_seed)

[0026] return sip_hash_value + dip_hash_value

[0027] end if

[0028] end function;

[0029] The ring cache queue is a storage space composed of a first-in-first-out lock-free queue with the same number of CPU cores as the number of all CPU cores in the computer, which is generated by the first CPU core;

[0030] The first business logic engine is powered by the second CPU core. The binary information in the data packet involved in the business logic that the first business logic engine needs to execute is subjected to secondary hash calculation, which can be used to locate the first-in-first-out lock-free queue where the data packet required by the first business logic engine is stored. The first business logic engine reads data with the same secondary hash calculation value from the first-in-first-out lock-free queue where the data packet required by the first business logic engine is stored.

[0031] The N-1th business logic engine is provided with computing power by the Nth CPU core. The two-tuple information in the data packet involved in the business logic that the N-1th business logic engine needs to execute is subjected to secondary hash calculation, which can be used to locate the first-in-first-out lock-free queue where the data packet required by the N-1th business logic engine is stored. The N-1th business logic engine reads data with consistent secondary hash calculation values ​​from the first-in-first-out lock-free queue where the data packet required by it is stored.

[0032] Beneficial Effects

[0033] The I / O packet receiving logic engine and the business logic engine of the present invention occupy different CPU cores respectively, and the data of packet receiving and business processing are connected through the ring area cache queue, so that packet receiving and business processing are independent of each other and do not affect each other. The same source and the same destination can be achieved without the need for a diversion device, and the business processing does not affect the reception or forwarding of the original data packet. The problem that the traffic is not evenly distributed to each CPU core under different traffic models, affecting the processing performance of the overall server, is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a system structure diagram of the present invention. DETAILED DESCRIPTION

[0035] See also Figure 1, a load balancing method and device based on a pipeline forwarding model of the present invention is composed of an I / O packet receiving logic engine 10, a symmetric hash calculator 11, a ring area cache queue 2, and a first business logic engine 30 to an N-1th business logic engine 30; the ring area cache queue 2 is composed of a first-in-first-out lock-free queue with the same number of CPU cores;

[0036] The I / O packet receiving logic engine 10 is provided with computing power by the first CPU core 1, and uses the symmetric RSS technology to obtain the data packets in the queues from the first network card A to the Nth network card A, and decodes the data packets to extract the binary information, which includes: source IP address and destination IP address;

[0037] The symmetric hash calculator 11 is provided with computing power by the first CPU core 1, and uses the symmetric hash algorithm to perform secondary hash calculation on the binary information of the data packet received by the I / O packet receiving logic engine 10, and allocates and stores the data packet corresponding to the result of the secondary hash calculation in the first-in-first-out lock-free queue in the ring area cache queue 2 module according to the result of the secondary hash calculation; the data packets with the same result of the secondary hash calculation will be allocated to the same first-in-first-out lock-free queue, and the data packets with different results of the secondary hash calculation will be evenly allocated to different first-in-first-out lock-free queues for storage;

[0038] The secondary hash calculation uses a symmetric hash algorithm and supports both IPV4 and IPV6 dual-stack traffic. The pseudo code of the algorithm is as follows:

[0039] function second_hash(ethe_type, data, hash_seed)

[0040] if ether_type == ETHER_TYPE_IPv4 then

[0041] struct ipv4_hdr *k data

[0042] sip_hash_value jhash_1word(k->src_addr, hash_seed)

[0043] dip_hash_value jhash_1word(k->dst_addr, hash_seed)

[0044] return sip_hash_value + dip_hash_value

[0045] else if ether_type == ETHER_TYPE_IPv6 then

[0046] struct ipv6_hdr *k data

[0047] sip_hash_value jhash(k->src_addr, IPV6_ADDR_LEN, hash_seed)

[0048] dip_hash_value jhash(k->dst_addr, IPV6_ADDR_LEN, hash_seed)

[0049] return sip_hash_value + dip_hash_value

[0050] end if

[0051] end function;

[0052] The ring cache queue 2 is a storage space composed of a first-in-first-out lock-free queue generated by the first CPU core 1 according to the number of all CPU cores of the computer and the same number of CPU cores;

[0053] The first business logic engine 30 is provided with computing power by the second CPU core 3. The binary information in the data packet involved in the business logic to be executed by the first business logic engine 30 is subjected to secondary hash calculation, which can be used to locate the first-in-first-out lock-free queue where the data packet required by the first business logic engine is stored. The first business logic engine 30 reads data with consistent secondary hash calculation values ​​from the first-in-first-out lock-free queue where the data packet required by the first business logic engine is stored.

[0054] The N-1th business logic engine 30 is provided with computing power by the N-1th CPU core 3. The two-tuple information in the data packet involved in the business logic that the N-1th business logic engine 30 needs to execute is subjected to a secondary hash calculation, which can be used to locate the first-in-first-out lock-free queue where the data packet required by the N-1th business logic engine is stored. The N-1th business logic engine 30 reads data with consistent secondary hash calculation values ​​from the first-in-first-out lock-free queue where the data packet required by it is stored.

Claims

1. A load balancing device based on a pipeline forwarding model, characterized in that It consists of an I / O packet receiving logic engine, a symmetric hash calculator, a ring cache queue, and the first business logic engine to the N-1th business logic engine; the ring cache queue consists of a first-in-first-out lock-free queue with the same number of CPU cores; The I / O packet receiving logic engine is powered by the first CPU core. It uses symmetric RSS technology to obtain data packets from the queues of the first network card to the Nth network card, decode the data packets, and extract binary information. The binary information includes: source IP address and destination IP address. The symmetric hash calculator is provided with computing power by the first CPU core, and uses the symmetric hash algorithm to perform secondary hash calculation on the binary information of the data packet received by the I / O packet receiving logic engine. According to the result of the secondary hash calculation, the data packet corresponding to the result of the secondary hash calculation is allocated and stored in the first-in-first-out lock-free queue in the ring area cache queue module; the data packets with the same result of the secondary hash calculation will be allocated to the same first-in-first-out lock-free queue, and the data packets with different results of the secondary hash calculation will be evenly allocated to different first-in-first-out lock-free queues for storage; The secondary hash calculation uses a symmetric hash algorithm and supports both IPV4 and IPV6 dual-stack traffic; The ring cache queue is a storage space composed of a first-in-first-out lock-free queue with the same number of CPU cores as the number of all CPU cores in the computer, which is generated by the first CPU core; The first business logic engine is powered by the second CPU core. The second CPU core performs a secondary hash calculation on the binary information in the data packet involved in the business logic that the first business logic engine needs to execute. This can be used to locate the first-in-first-out lock-free queue where the data packet required by the first business logic engine is stored. The first business logic engine reads data with the same secondary hash calculation value from the first-in-first-out lock-free queue where the data packet required by the first business logic engine is stored. The N-1th business logic engine is provided with computing power by the N-th CPU core. The N-1th CPU core performs a secondary hash calculation on the binary information in the data packet involved in the business logic executed by the N-1th business logic engine, which can be used to locate the first-in-first-out lock-free queue where the data packet required by the N-1th business logic engine is stored. The N-1th business logic engine reads data with consistent secondary hash calculation values ​​from the first-in-first-out lock-free queue where the data packet it requires is stored.

Citation Information

Patent Citations

  • Method and system for improving data processing performance of multi-core CPU

    CN105159779A

  • Efficient load balancer and load balancing system

    CN107135278A

  • Processing method and device for network data packet and electronic equipment

    CN110022330A

  • TCP reinjection plugging method and device

    CN110572380A

  • System and method for load balancing

    KR1020060066603A