An eBPF-based network traffic governance method, system, device and medium

Abstract

The present application relates to a kind of network traffic governance methods, systems, equipment and medium based on eBPF, comprising: traffic interception, network traffic packet is intercepted from eBPF program in kernel mounting point, and the network traffic packet of access cluster Service is screened out, it is judged whether the network traffic packet destination address is Service address, if the network traffic packet of the cluster Service, then execute traffic forwarding scheme, if not, release the network traffic packet does not affect other network functions of host;Load balancing, in response to the execution traffic forwarding scheme, obtain the Endpoint information of multiple real service instances of access cluster Service back-end, and carry out load balancing algorithm selection Endpoint information;The present application proposes slot algorithm, can effectively reduce the update times of Pod meta information, optimizes the operation to storage space, improves system stability, applicable to a large number of, frequently updated Endpoint list Application scenarios.

Description

Technical Field

[0001] This disclosure relates to the field of cloud computing technology, and in particular to a network traffic management method, system, device and medium based on eBPF. Background Technology

[0002] In recent years, with the rapid development of the internet economy, cloud computing has become a key link for enterprises to integrate network, computing, and storage infrastructure, and an inevitable choice for enterprise digital transformation. Cloud vendors have established massive data centers to provide cloud services to a wide range of users, and more and more enterprises tend to choose on-demand, flexible, and convenient cloud platforms to run their business systems.

[0003] Container virtualization technology, as an important achievement of cloud computing development, uses Linux's Cgroups and Namespace technologies to implement resource restrictions and view isolation for processes. Through the sandbox mechanism, processes run in an independent environment, avoiding mutual interference.

[0004] Compared to traditional hypervisor virtualization technology, container virtualization technology is more lightweight, consumes fewer resources, and is more flexible in achieving application isolation. In 2013, the Docker project solved the problem of cloud platform application portability by packaging application configuration files and runtime environments into images, enabling "package once, run anywhere." This innovation led to technological changes in the field of cloud computing infrastructure, and container virtualization technology, represented by Docker, quickly gained dominance in cloud platforms.

[0005] With the rise of the Docker project, the concept of Container as a Service (CASS) has gradually gained attention. Many companies have launched container orchestration systems to compete, combining traditional cloud computing Infrastructure as a Service (IaaS) with Platform as a Service (PaaS) to provide a runtime environment for container applications. Ultimately, Google's Kubernetes (K8s), a container orchestration platform open-sourced in 2014, has gradually replaced projects such as Swarm, Mesos, and Cattle as the de facto standard for container orchestration systems, thanks to its powerful container orchestration capabilities, support for user-defined components, and excellent system architecture.

[0006] Kubernetes' default network traffic forwarding scheme for its Service network model suffers from high latency and resource consumption, making it unsuitable for large-scale application scenarios. While some existing network solutions address the issue of large-scale support, they neglect network performance improvements and ease of use. eBPF holds immense potential for network traffic optimization, overcoming the limitations of user-space system calls to enhance performance. However, due to the stringent security requirements of the operating system kernel, eBPF faces several limitations, including program size restrictions and lack of support for infinite loops.

[0007] Current industry solutions support basic functions such as traffic forwarding in the Service network model, but they lack comprehensive consideration of performance and effectiveness during implementation, and still suffer from problems such as high network latency and high resource consumption, failing to meet the high-performance requirements of cloud platform network traffic governance. While existing load balancing algorithms solve problems in specific scenarios, their implementation methods and dependent metrics are complex, failing to meet the lightweight requirements of Service network model traffic governance. Summary of the Invention

[0008] The purpose of this invention is to provide a network traffic management method, system, device, and medium based on eBPF. This method can be seamlessly embedded into a cloud platform, allowing traffic originally managed based on iptables rules to be managed by eBPF programs, thereby improving the efficiency of traffic forwarding.

[0009] To achieve the above objectives, a first aspect of this disclosure provides a network traffic governance method based on eBPF, the method comprising:

[0010] Traffic interception: The eBPF program intercepts network traffic packets at the kernel mount point, filters out the network traffic packets that access the cluster service, and determines whether the destination address of the network traffic packet is the service address. If it is the network traffic packet of the cluster service, the traffic forwarding scheme is executed; otherwise, the network traffic packet is released without affecting other network functions of the host.

[0011] Load balancing, in response to the execution traffic forwarding scheme, obtains Endpoint information of multiple real service instances accessing the backend of the cluster Service, and selects Endpoint information for load balancing algorithms;

[0012] Address translation: Based on the Endpoint information selected by the load balancing algorithm, address translation is performed on the Endpoint information to complete NAT translation;

[0013] Connection tracking is performed on the network traffic data packets based on address translation. When the eBPF program receives the response network traffic data packets, it modifies the data packet 5-tuple according to the connection tracking information and correctly returns the Endpoint information response information to the requesting client.

[0014] In one implementation, the interception of network traffic packets by the eBPF program at the kernel mount point specifically includes:

[0015] Depending on the network traffic path, the cluster service includes the ClusterIP Service network model and the NodePort Service network model;

[0016] Create a Socket process, deploy the microservice using a container approach, and assign a corresponding process control group;

[0017] A cgroup network packet filtering mechanism is constructed to implement network control over multiple processes within the process control group, and to intercept the connect and send system calls to forward network packets.

[0018] Based on the cgroup network packet filtering mechanism, the connect and send system calls of the socket are intercepted at the mount point to obtain the network packet information of the mount point and extract the destination IP and destination port.

[0019] Based on the ClusterIP Service network model of the cluster, Pod information is obtained in the eBPF mapping and DNAT is performed. After DNAT processing, the data packets enter the network protocol stack normally and are forwarded according to the host routing rules.

[0020] An XDP network packet filtering mechanism is constructed; based on the XDP network packet filtering mechanism being mounted to the network card, the network driver of the network card, and the kernel network protocol stack, three working modes with performance ranging from high to low are respectively Offloaded, Native, and Generic.

[0021] Based on the NodePort Service network model of the cluster, it receives access traffic from inside and outside the cloud platform. By opening ports on the host machine, it listens for request information, and the network traffic inside and outside the Kubernetes cluster reaches the network card hardware of the node.

[0022] In one implementation, the NodePort Service network model traffic management of the XDP network packet filtering mechanism includes traffic accessing the host port passing through Pods selected by the load balancing algorithm that are located on the same host, and traffic accessing the host port passing through Pods selected by the load balancing algorithm that are located on different nodes.

[0023] Traffic accessing the host port is filtered by the XDP network packet filtering mechanism at the network interface card (NIC) receiving layer. If the network traffic packet requests to establish a connection, the DNAT (Destination Address Translation) process is performed to replace the destination address of the network 5-tuple with the Pod address selected by the load balancing algorithm, and the header checksum is recalculated and the conntrack connection tracking information is maintained so that subsequent network traffic is forwarded to the same Pod. If a network traffic packet requests to disconnect is received, the conntrack connection tracking information is cleared.

[0024] The traffic accessing the host port is intercepted by the Pod selected by the load balancing algorithm on another node. The network traffic data packets are then subjected to DNAT and SNAT (Source Address Translation) processes. After the SNAT process, the forwarding information database is queried to update the MAC address of the data packets, and the network traffic data packets are forwarded to the output network card.

[0025] In one implementation, the load balancing algorithm is a priority load balancing algorithm based on entropy weighting. It uses a kernel-user space coordination mechanism to dynamically assess the priority of the current Pod. When making load balancing decisions, it relies on kernel-space information to complete the traffic management process, including:

[0026] Standardize the resource metrics for Pods, dividing them into positive and negative metrics;

[0027] The influence of indicators on scores is represented by the calculation formulas for positive and negative indicators, and the entropy of each indicator is calculated.

[0028] Calculate the redundancy of each metric, then calculate the weight of each metric based on the redundancy, and finally calculate the priority of each Pod based on the weight.

[0029] A second aspect of this disclosure provides a network traffic management system based on eBPF, applied to the method described herein, the system comprising:

[0030] The control plane, located in user space, is responsible for subscribing to Service and Endpoint resource objects of the Kubernetes cluster and synchronizing the endian and big-endian resource objects to the data plane after data processing.

[0031] The data plane is located in the kernel space. The data plane includes an eBPF program, an eBPF mapping, and a kernel mount point. The eBPF program uses the resource object information in the eBPF mapping as the basis for traffic governance and is used to implement packet forwarding function at the kernel mount point through which network traffic packets pass.

[0032] The traffic interception module is used to intercept network traffic packets from the eBPF program at the kernel mount point, filter out the network traffic packets that access the cluster service, determine whether the destination address of the network traffic packet is the service address, and if it is the network traffic packet of the cluster service, execute the traffic forwarding scheme; otherwise, release the network traffic packet.

[0033] Load balancing, in response to the execution traffic forwarding scheme, obtains Endpoint information of multiple real service instances accessing the backend of the cluster Service, and selects Endpoint information for load balancing algorithms;

[0034] The address translation module is used to perform address translation on the Endpoint information selected by the load balancing algorithm to complete NAT translation;

[0035] The connection tracking module is used to perform connection tracking based on the network traffic data packets with address translation. When the eBPF program receives the response network traffic data packets, it modifies the data packet 5-tuple according to the connection tracking information and correctly returns the Endpoint information response information to the requesting client.

[0036] In one implementation, the data plane includes a slot algorithm-optimized mapping storage method, the method comprising:

[0037] The Endpoints list is split into individual Pod elements and stored.

[0038] During the tuning of Pod elements, the update granularity of eBPF mapping is changed to a single Pod element, and the slot algorithm is used to maintain the one-to-many relationship between Service and Pod.

[0039] During traffic forwarding, a single Pod element is selected as the backend service based on the slot. By marking the elements of Endpoints as deleted, retained, or added, the elements in the Endpoints list are processed uniformly when the elements change dynamically.

[0040] In one implementation, the eBPF program is of type cgroup, and the kernel mount point is cgroupv2.

[0041] In one implementation, the NAT translation includes a DNAT translation process and an SNAT translation process.

[0042] A third aspect of this disclosure provides an electronic device, comprising:

[0043] microcontroller;

[0044] Memory used to store executable instructions for a microcontroller;

[0045] The microcontroller is configured to execute executable instructions stored in the memory to perform the method described in any one of the first aspects.

[0046] A fourth aspect of this disclosure provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, implement the steps of the method described in any of the first aspects.

[0047] This invention provides a network traffic management method, system, device, and medium based on eBPF. Compared with existing technologies, it has the following advantages:

[0048] 1) This invention is a method for traffic governance of microservices on cloud platforms. This method can adapt to dynamic traffic governance environments and meet the requirements of high performance and high throughput.

[0049] 2) This invention proposes a slot algorithm, which can effectively reduce the number of times Pod metadata is updated, optimize the operation of storage space, improve system stability, and is suitable for application scenarios with a large number of frequent Endpoints list updates.

[0050] 3) This invention proposes a priority load balancing algorithm based on the entropy weight method. In high-concurrency application scenarios, the priority algorithm based on the entropy weight method can avoid servers with high load and select servers with relatively sufficient resources for load distribution, thus solving the problem of load imbalance under complex requests.

[0051] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0052] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0053] Figure 1This is a flowchart illustrating a network traffic management method based on eBPF, according to an embodiment of the specification.

[0054] Figure 2 This is a flowchart of the traffic management process of the present invention;

[0055] Figure 3 This is a block diagram of a network traffic management system based on eBPF according to the present invention;

[0056] Figure 4 This is a flowchart illustrating the process of solving traffic management for ClusterIP type Services in this invention;

[0057] Figure 5 This is a schematic diagram illustrating the process of solving the traffic management of NodePort type Services in this invention;

[0058] Figure 6 This is a schematic diagram illustrating the implementation of the slot algorithm in this invention;

[0059] Figure 7 This is a schematic diagram illustrating the principle of load balancing using the entropy weight method as described in this invention.

[0060] Figure 8 This is a bar chart comparing network latency according to Embodiment 3 of the present invention;

[0061] Figure 9 This is a QPS comparison bar chart of Embodiment 3 of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0064] To achieve the above objectives, this disclosure provides a network traffic governance method based on eBPF. Currently, the default network traffic forwarding scheme of Kubernetes Service network model suffers from high access latency and high resource consumption, making it unable to effectively support large-scale traffic application scenarios. While some existing network solutions address the issue of large-scale support, they neglect network performance improvement and ease of use. eBPF has great potential for network traffic optimization, as it overcomes the limitations of user-space system calls to improve performance. However, due to the extremely high security requirements of the operating system kernel, eBPF has many limitations, such as program size restrictions and lack of support for infinite loops.

[0065] Example 1:

[0066] Figures 1-2 This is a flowchart illustrating a network traffic governance method based on eBPF according to an embodiment. Using kernel-mode eBPF to replace routing rules requires sequentially performing functions such as traffic interception, load balancing, address translation, and connection tracking. The method includes:

[0067] In step S11, traffic interception is performed. The eBPF program intercepts network traffic packets at the kernel mount point and filters out the network traffic packets that access the cluster service. It is determined whether the destination address of the network traffic packets is the service address. If it is the network traffic packet of the cluster service, the traffic forwarding scheme is executed. If not, the network traffic packets are released without affecting other network functions of the host.

[0068] Traffic interception: The eBPF program intercepts network traffic packets at the kernel mount point, filters out network traffic packets accessing the cluster service, and distinguishes whether the destination address of the packet is a service address. If it is network traffic to the cluster service, a traffic forwarding scheme is executed; otherwise, the traffic packet is released without affecting other network functions of the host. An eBPF program is written to mount to an appropriate location in the kernel (such as the tc (Traffic Control) layer or XDP (eXpressData Path) layer of the network stack) to capture and process network traffic packets.

[0069] The steps include:

[0070] Select mount point:

[0071] XDP: Suitable for very high-performance scenarios, it processes data packets directly at the network card driver level, which can reduce CPU usage and latency.

[0072] tc clsact: Located at a slightly higher level in the network stack, it provides more flexibility and access to packet content.

[0073] Write eBPF programs:

[0074] Write programs using the BPF programming language (based on the C language).

[0075] The program needs to be able to parse network traffic packets (usually using BPF Helper functions, such as bpf_skb_load_bytes, to read packet content).

[0076] Determine if the destination address of the data packet matches the IP address of the cluster service.

[0077] Mounting eBPF program:

[0078] Use the bpf() system call or the iproute2 tool (such as the tc command) to mount the eBPF program to the selected kernel point.

[0079] Implementation logic:

[0080] If the destination address of the packet is a Service address, then load balancing logic is executed (this may require additional eBPF programs or work in conjunction with other kernel modules / userspace programs).

[0081] If not, the packet is redirected back to the normal path via bpf_clone_redirect (for XDP) or the corresponding BPF action (such as bpf_redirect for tc clsact), or simply allowed to continue its normal flow.

[0082] Testing and debugging:

[0083] Use tools such as bpftrace or bpftool to debug and verify the behavior of eBPF programs.

[0084] Ensure network performance is not significantly affected and verify that traffic is being correctly blocked and redirected.

[0085] In step S12, the load balancer responds to the traffic forwarding scheme by obtaining the Endpoint information of multiple real service instances accessing the backend of the cluster Service and selecting the Endpoint information for the load balancing algorithm.

[0086] In step S13, address translation is performed on the Endpoint information selected by the load balancing algorithm to complete NAT translation.

[0087] When selecting an Endpoint based on the access policy, address translation should be performed based on the Endpoint to replace the original destination address (DNAT) of the Service network traffic data packet. If the backend Endpoint is not the host where the access entry point is located, source address translation (SNAT) is also required.

[0088] In step S14, connection tracking is performed on the network traffic data packets based on address translation. When the eBPF program receives the response network traffic data packets, it modifies the data packet 5-tuple according to the connection tracking information and correctly returns the Endpoint information response information to the requesting client.

[0089] In order to correctly return the Endpoint's response information to the requesting client after address translation, connection tracking should be implemented for the address translation data packets. When the eBPF program receives the response data packet, it can modify the data packet's five-tuple according to the connection tracking information and return the expected data packet to the client to prevent the data packet from being dropped.

[0090] Microservices deployed in containers on the cloud platform have corresponding process control groups. The two parties communicating over the network need to go through processes such as creating a Socket process, initiating a connect system call, establishing a connection, and sending information. ClusterIP Service enables network communication between microservices within the cloud platform, and each step of the communication process is carried out within the cloud platform.

[0091] Therefore, a cgroup network traffic packet filtering mechanism is designed to implement network control for multiple processes within a process control group, and to intercept system calls such as connect and send to forward network traffic packets.

[0092] The kernel mount point of a cgroup-type eBPF program is cgroupv2. At this point, network traffic packets have not yet entered the protocol layer. This has advantages such as not needing to maintain connection tracking information and not needing to recalculate checksums for NAT (Network Address Translation), resulting in relatively low overhead for traffic management.

[0093] In one implementation, see Figure 3 As shown, the interception of network traffic packets by the eBPF program at the kernel mount point specifically includes:

[0094] Depending on the network traffic path, the cluster service includes a ClusterIP Service network model and a NodePort Service network model; design the traffic forwarding path of the cloud platform service network model, and design the corresponding optimal mount point for ClusterIP and NodePort type services respectively;

[0095] Create a Socket process, deploy the microservice using a container approach, and assign a corresponding process control group;

[0096] A cgroup network packet filtering mechanism is constructed to implement network control over multiple processes within the process control group, and to intercept the connect and send system calls to forward network packets.

[0097] Based on the cgroup network packet filtering mechanism, the connect and send system calls of the socket are intercepted at the mount point to obtain the network packet information of the mount point and extract the destination IP and destination port.

[0098] Based on the ClusterIP Service network model of the cluster, Pod information is obtained in the eBPF mapping and DNAT is performed. After DNAT processing, the data packets enter the network protocol stack normally and are forwarded according to the host routing rules.

[0099] If it is a ClusterIP Service of the cluster, the Pod (equivalent to the Endpoint element in the Endpoints list) information is obtained in the eBPF mapping and DNAT (Destination Network Address Translation) is performed. The data packets processed by DNAT enter the network protocol stack normally and are forwarded according to the host routing rules.

[0100] The NodePort Service receives traffic from both inside and outside the cloud platform. By opening ports on the host machine, it listens for request information, and network traffic from within and outside the Kubernetes cluster reaches the node's network interface card (NIC). The NIC is the first point of entry for receiving network traffic packets, and processing packets at this point offers optimal performance.

[0101] An XDP network packet filtering mechanism is constructed; based on the XDP network packet filtering mechanism being mounted to the network card, the network driver of the network card, and the kernel network protocol stack, three working modes with performance ranging from high to low are respectively Offloaded, Native, and Generic.

[0102] Based on the NodePort Service network model of the cluster, it receives access traffic from inside and outside the cloud platform. By opening ports on the host machine, it listens for request information, and the network traffic inside and outside the Kubernetes cluster reaches the network card hardware of the node.

[0103] In one implementation, the NodePort Service network model traffic management of the XDP network packet filtering mechanism includes traffic accessing the host port that passes through Pods selected by the load balancing algorithm located on the same host, and traffic accessing the host port that passes through Pods selected by the load balancing algorithm located on different nodes. These two scenarios require separate management, and the corresponding processes are as follows: Figure 4 As shown:

[0104] Traffic accessing the host port is filtered by the XDP network packet filtering mechanism at the network interface card (NIC) receiving layer. If the network traffic packet requests to establish a connection, the DNAT (Destination Address Translation) process is performed to replace the destination address of the network 5-tuple with the Pod address selected by the load balancing algorithm, and the header checksum is recalculated and the conntrack connection tracking information is maintained so that subsequent network traffic is forwarded to the same Pod. If a network traffic packet requests to disconnect is received, the conntrack connection tracking information is cleared.

[0105] If the packet requests to establish a connection, the DNAT process is executed, replacing the destination address of the network 5-tuple with the Pod address selected by the load balancing algorithm, recalculating the header checksum to prevent the packet from being dropped, maintaining the conntrack connection tracking information, and forwarding subsequent network traffic to the same Pod; if a packet requests to close the connection is received, the conntrack connection tracking information is cleared.

[0106] The traffic accessing the host port is intercepted by the Pod selected by the load balancing algorithm on another node. The network traffic data packets are then subjected to DNAT and SNAT (Source Address Translation) processes. After the SNAT process, the forwarding information database is queried to update the MAC address of the data packets, and the network traffic data packets are forwarded to the output network card.

[0107] In addition to DNAT, SNAT (Source Network Address Translation) is also required for data packets. This is because conntrack connection tracking information between different hosts cannot be synchronized. When the host hosting the Pod receives a request and returns a response packet, it cannot forward the packet to the requesting client because it cannot find the connection tracking information corresponding to the destination address of the packet, and the response packet is discarded directly. After SNAT, the Forwarding Information Base (FIB) is queried to update the MAC address of the data packet, thus enabling the packet to be forwarded to the outgoing network interface card.

[0108] Example 2:

[0109] See Figure 5 As shown, a network traffic management system based on eBPF is applied to the method described above. The system includes:

[0110] The control plane, located in user space, is responsible for subscribing to Service and Endpoint resource objects in the Kubernetes cluster. After data processing, it synchronizes the endianened and size-adjusted resource objects to the data plane. It designs Service traffic paths, characterized by optimizing the performance of the Service network model through efficient kernel-level operation, requiring a refactoring of the traffic governance function for routing rules. For the ClusterIP and NodePort Service network models, the corresponding network traffic paths differ, necessitating targeted design of network traffic forwarding paths.

[0111] The data plane is located in the kernel space. The data plane includes an eBPF program, an eBPF mapping, and a kernel mount point. The eBPF program uses the resource object information in the eBPF mapping as the basis for traffic governance and is used to implement packet forwarding function at the kernel mount point through which network traffic packets pass.

[0112] When the data plane forwards traffic, it needs to rely on the resource object information synchronized by the control plane. The backend of the Service network model is a list of Endpoints composed of multiple Pods. If the key-value pairs composed of Service and Endpoints are directly stored in the eBPF mapping of the data plane, the Pod tuning process will cause the eBPF mapping to be updated frequently. The list of Endpoints in the kernel will be constantly created and deleted, which will cause performance issues. Moreover, this logical relationship is not convenient for load balancing algorithms to select backend services.

[0113] The packet forwarding function enables dynamic load balancing. The original network model uses a random algorithm to achieve load balancing, which relies solely on the number of connections to evenly distribute traffic to different Pods. This approach often results in load imbalance. Therefore, a new algorithm is proposed that comprehensively considers multiple main factors that cause load changes to ensure a relatively balanced load as much as possible.

[0114] Microservice applications on cloud platforms reside on nodes of a distributed cluster, each with varying resource conditions. Dynamic load balancing algorithms are more suitable for load balancing mechanisms. The time complexity of these algorithms directly translates to request-response latency, making complex algorithms unsuitable. The load balancing process in the Service network model is completed in kernel space. Due to eBPF limitations, too many metrics cannot be synchronized to the kernel. Therefore, this paper proposes a priority-based load balancing algorithm using Pod CPU utilization, memory utilization, CPU allocation, and memory allocation as load balancing decision metrics. A kernel-user space collaborative mechanism dynamically assesses the priority of the current Pod. When making load balancing decisions, kernel-space information is relied upon to complete the traffic management process.

[0115] The traffic interception module is used to intercept network traffic packets from the eBPF program at the kernel mount point, filter out the network traffic packets that access the cluster service, determine whether the destination address of the network traffic packet is the service address, and if it is the network traffic packet of the cluster service, execute the traffic forwarding scheme; otherwise, release the network traffic packet.

[0116] Load balancing, in response to the execution traffic forwarding scheme, obtains Endpoint information of multiple real service instances accessing the backend of the cluster Service, and selects Endpoint information for load balancing algorithms;

[0117] The address translation module is used to perform address translation on the Endpoint information selected by the load balancing algorithm to complete NAT translation;

[0118] The connection tracking module is used to perform connection tracking based on the network traffic data packets with address translation. When the eBPF program receives the response network traffic data packets, it modifies the data packet 5-tuple according to the connection tracking information and correctly returns the Endpoint information response information to the requesting client.

[0119] In one implementation, the data plane includes a slot algorithm-optimized mapping storage method, the method comprising:

[0120] The Endpoints list is split into individual Pod elements and stored.

[0121] During the tuning of Pod elements, the update granularity of eBPF mapping is changed to a single Pod element, and the slot algorithm is used to maintain the one-to-many relationship between Service and Pod.

[0122] During traffic forwarding, a single Pod element is selected as the backend service based on the slot. By marking the elements of Endpoints as deleted, retained, or added, the elements in the Endpoints list are uniformly processed when the elements change dynamically. Adopting the idea of ​​minimizing changes to the position of Pod elements, the uniform processing of the three states of elements when the elements of the Endpoints list change dynamically solves the problem caused by the large granularity and frequent updates of eBPF mapping data.

[0123] The mapping and storage relationship of the data plane needs to be optimized. When the data plane forwards traffic, it needs to rely on the resource object information synchronized by the control plane. The backend of the Service network model is a list of Endpoints composed of multiple Pods. If the key-value pairs composed of Service and Endpoint are directly stored in the eBPF mapping of the data plane, the Pod tuning process will cause the eBPF mapping to be updated frequently. The list of Endpoints in the kernel will be constantly created and deleted, which will cause performance issues. Moreover, this logical relationship is not convenient for load balancing algorithms to select backend services.

[0124] In one implementation, the eBPF program is of type cgroup, and the kernel mount point is cgroupv2.

[0125] In one implementation, the NAT translation includes a DNAT translation process and an SNAT translation process.

[0126] By combining traffic governance path design, slot algorithm, and dynamic load balancing algorithm, the performance of microservice traffic governance on cloud platforms can be improved.

[0127] Example 3:

[0128] A network traffic management method based on eBPF.

[0129] The implementation of this method involves the following steps:

[0130] 1) Design the traffic forwarding path for the cloud platform service network model, and design the corresponding optimal mount point for ClusterIP and NodePort type services respectively;

[0131] 2) Implement functions such as traffic interception, address translation, load balancing, and link tracing;

[0132] 3) Optimize the efficiency of controlling data synchronization on the data plane;

[0133] 4) Optimize the server-side load balancing strategy to achieve dynamic load balancing.

[0134] The overall architecture of the traffic path in step 1 includes a data plane and a control plane. When forwarding traffic, the data plane needs to rely on the resource object information synchronized by the control plane. The backend of the Service network model is a list of Endpoints composed of multiple Pods. If the key-value pairs composed of Service and Endpoints are directly stored in the eBPF mapping of the data plane, the Pod tuning process will cause the eBPF mapping to be updated frequently. The list of Endpoints in the kernel will be constantly created and deleted, which will cause performance problems. Moreover, this logical relationship is not convenient for the load balancing algorithm to select backend services.

[0135] The traffic path design in step 1 optimizes the performance of the Service network model through efficient kernel-mode operation, requiring a reconstruction of the traffic governance function of the routing rules. For the ClusterIP and NodePort Service network models, the corresponding network traffic paths differ, necessitating the design of targeted network traffic forwarding paths.

[0136] The traffic interception described in step 2 refers to the Service network model object of the cloud platform, which corresponds to data packets in the kernel. Interception is achieved based on the five-tuple of the data packet (source address, source port, destination address, destination port, protocol). Address translation includes source address translation, destination address translation, and all translations. Link tracing ensures that data packets from the same connection can reach the same destination normally based on the data packet establishment tracking information. Load balancing ensures that different backend services can receive network traffic evenly, improving network performance.

[0137] The data synchronization described in step 3 corresponds to a slot algorithm-optimized mapping storage method. This method splits the Endpoints list into individual Pod elements for storage. During continuous Pod tuning, the update granularity of the eBPF mapping becomes that of individual Pod elements, using slots to maintain a one-to-many relationship between Services and Pods. During traffic forwarding, a single Pod element is selected as the backend service based on its slot. The slot algorithm, by marking Endpoints elements as deleted, retained, or added, adopts the idea of ​​minimizing changes to Pod element positions. When elements in the Endpoints list dynamically change, it uniformly handles elements in all three states, solving the problem of large granularity and frequent updates in eBPF mapping data updates.

[0138] Example of Endpoints list status change, such as status 1:<A,B,C,D,E,F> State 2:<A,B,C,D,E,F> To state 3:<G,E,C,D> Each letter represents a Pod, and the number in parentheses represents the slot of the Pod. When Endpoints tunes from state 1 to state 2, Pod F is added to Endpoints and assigned a slot number equal to the length of the Endpoints list plus "1". When tuning from state 2 to state 3, both Pod deletion and Pod addition operations are involved. In this case, the slot algorithm prioritizes placing the newly added Pod into the smallest slot among the deleted Pods. If there is still a vacancy in the slot of a deleted Pod, and the vacant slot number is less than the slot number of a retained Pod, then the Pod with the largest slot number among the retained Pods is moved forward to the vacant slot, until the vacant slot is filled or there is no situation where the slot of a retained Pod is larger than the vacant slot. The slot algorithm then ends. The algorithm principle is as follows: Figure 6 As shown,<A,B,F> To delete the slot number,<E,C,D> To preserve the slot number, <g>Add a new slot number.

[0139] The slot algorithm changes the update granularity of eBPF mappings from a list of Endpoints to a single Pod element, reducing the number of eBPF mapping updates. Without the slot algorithm, transitioning from state 2 to state 3 requires deleting the old Endpoints containing 6 elements and creating new Endpoints containing 4 elements. With the slot algorithm, only 2 elements need to be deleted and 2 elements need to be updated, significantly improving algorithm efficiency.

[0140] Microservice applications on cloud platforms reside on nodes in a distributed cluster, each with varying resource conditions. Dynamic load balancing algorithms are more suitable for load balancing mechanisms. The time complexity of these algorithms directly translates to request-response latency, making complex algorithms unsuitable. Load balancing in the Service network model is performed in kernel space, and due to eBPF limitations, too many metrics cannot be synchronized to the kernel. Therefore, this paper proposes a priority-based load balancing algorithm using entropy weighting, employing Pod CPU utilization, memory utilization, CPU allocation, and memory allocation as load balancing decision metrics. A kernel-user space collaborative mechanism dynamically assesses the priority of the current Pod, and traffic management is performed based on kernel-space information during load balancing decisions.

[0141] Entropy weighting is a mathematical method that measures the impact of an indicator on a comprehensive evaluation based on the dispersion of data. The greater the difference between indicators, the greater their impact on the overall evaluation, and the higher their weight. Entropy weighting utilizes the property of entropy to objectively assign values ​​to the dispersion of data, achieving a comprehensive score for multiple indicators to distinguish the priority of samples. Assume there are n data points and m evaluation indicators, where the indicator values ​​are represented by x. ij (i = 1, ..., n, j = 1, ..., m) represents the Pod. First, the resource metrics of the Pod are standardized, dividing them into positive and negative metrics, with the corresponding formulas as follows:

[0142]

[0143] By using the formulas for positive and negative indicators, the influence of indicators on the scores can be effectively represented. Then, the entropy of each indicator is calculated using the following formula:

[0144]

[0145] Then, the redundancy of each metric is calculated, and the weight of each metric is calculated from the redundancy. Finally, the priority of each Pod is calculated from the weight. i The corresponding formula is as follows:

[0146]

[0147] The entropy weight method uses the CPU and memory allocation of a Pod as a positive indicator, meaning that the higher the allocation of CPU and memory, the better; and uses CPU utilization and memory utilization as negative indicators, meaning that the lower the CPU utilization and memory utilization, the better. It can objectively express the relative advantages and disadvantages of the Pod's resource allocation and utilization under various situations.

[0148] The process of the priority load balancing algorithm based on entropy weight is as follows: Figure 7 As shown, the control plane obtains four resource metrics of the cluster Pods from the monitoring module, organizes the metric data, and passes it to the entropy weight priority module. Based on the kernel-user space coordination mechanism, the control plane synchronizes the Pod resource objects and priority information to the kernel-space eBPF mapping. When traffic requesting a Service arrives and is captured by the kernel-space eBPF program, traffic forwarding is dynamically executed to ensure relatively balanced load on the Pods.

[0149] To verify the effectiveness of the eBPF-based Service network model proposed above, this paper constructs corresponding experiments. Since the Service network model involves communication between different nodes, a Kubernetes cluster environment needs to be built. The experiments were conducted on a 3-node Kubernetes cluster built on a cloud server, as shown in Table 1 below.

[0150] software Version CPU 4 cores Memory 16G operating system CentOS kernel 5.10.134-16.1.al8 Kubernetes 1.23.10

[0151] Table 1

[0152] Results analysis:

[0153] To test the performance of the eBPF solution when there are a large number of Services and Pods in a Kubernetes cluster, ClusterIP and NodePort type Services were created in quantities of 10, 20, 50, and 100 respectively, with each Service containing 10 backend Pods. Experiments were conducted on the eBPF and iptables solutions under the same concurrent load.

[0154] Depend on Figures 8-9 Experimental results show that as the number of Services and Pods in the cluster gradually increases, the network latency of the eBPF-based solution increases slowly, while the iptables solution shows a rapid increase. In terms of throughput, the eBPF solution can handle a higher QPS. Therefore, the eBPF-based solution implemented in this paper provides lower network latency and higher throughput when performing traffic governance tasks, improving the performance of cloud platform infrastructure for traffic governance.

[0155] In step 4, load balancing involves the control plane obtaining four resource metrics for cluster Pods from the monitoring module. After organizing the metric data, it passes it to the entropy weighting priority module. Based on a kernel-user space coordination mechanism, the control plane synchronizes Pod resource objects and priority information to the kernel-space eBPF mapping. When traffic requesting a Service arrives and is captured by the kernel-space eBPF program, dynamic traffic forwarding is executed to ensure relatively balanced load on the Pods.

[0156] Example 4

[0157] This disclosure also provides an electronic device, including:

[0158] microcontroller;

[0159] Memory used to store executable instructions for a microcontroller;

[0160] The microcontroller is configured to execute executable instructions stored in the memory to perform the method described in any one of the preceding embodiments.

[0161] This disclosure also provides a computer-readable storage medium storing computer program instructions thereon, which, when executed by a processor, implement the steps of the method described in any of the preceding embodiments.

[0162] The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings. However, the present disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of the present disclosure, various changes, modifications, substitutions and variations can be made to these embodiments, and all such changes, modifications, substitutions and variations fall within the protection scope of the present disclosure.

[0163] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction, and such combinations should also be considered as part of this disclosure. To avoid unnecessary repetition, this disclosure will not further describe the various possible combinations. The technical scope of this application is not limited to the contents of the specification, but must be determined according to the scope of the claims.< / g>

Claims

1. An eBPF-based network traffic governance method, characterized in that, The method comprises: Traffic interception, intercepting network traffic packets from eBPF programs at kernel mounting points, and screening network traffic packets accessing cluster services to determine whether the network traffic packet destination address is a service address. If it is a network traffic packet of the cluster service, execute traffic forwarding scheme, if not, release the network traffic packet; Load balancing, in response to the execution of the traffic forwarding scheme, obtaining Endpoint information of a plurality of real service instances accessing the cluster service backend, and selecting Endpoint information of the load balancing algorithm; Address conversion, according to the Endpoint information selected by the load balancing algorithm, address conversion is performed on the Endpoint information to complete NAT conversion; Connection tracking, according to the address converted network traffic packet, the eBPF program receives a response to the network traffic packet according to the connection tracking information to modify the packet quintuple, and correctly returns the response information of the Endpoint information to the request client; The load balancing algorithm is a priority load balancing algorithm based on entropy weight method, which dynamically evaluates the priority of the current Pod using the kernel / user mode cooperative mechanism. When load balancing makes decisions, it relies on kernel state information to complete the traffic management process, including: Standardize the resource indicators of the Pod, and divide the indicators into positive indicators and negative indicators; Through the positive indicator and negative indicator calculation formula, the influence between the index scores is represented, and the entropy of each indicator is calculated; Calculate the redundancy of each indicator, and then calculate the weight of each indicator from the redundancy. The priority of each Pod is calculated through the weight.

2. The method of claim 1, wherein, The eBPF program intercepts network traffic packets at the kernel mounting point, specifically including: According to the different network traffic paths, the cluster service includes ClusterIP Service network model and NodePort Service network model; Create a Socket process, deploy microservices in a container manner, and have a corresponding process control group; Build a cgroup network packet filtering mechanism to realize network control for multiple processes in the process control group, intercept connect and send system calls to realize network packet forwarding; According to the cgroup network packet filtering mechanism, intercept the connect and send system calls of the Socket at the mounting point to obtain network packet information of the mounting point, and extract the destination IP and destination Port; According to the ClusterIP Service network model of the cluster, obtain Pod information in eBPF mapping for DNAT. The packet processed by DNAT enters the network protocol stack normally, and realizes network packet forwarding according to the host routing rule; Build an XDP network packet filtering mechanism; according to the XDP network packet filtering mechanism mounted to the network card, the network driver of the network card and the kernel network protocol stack; According to the NodePort Service network model of the cluster, access traffic inside and outside the cloud platform is received, and a host port is opened to listen to the request information.

3. The method of claim 2, wherein, The NodePortService network model traffic management of the XDP network packet filtering mechanism includes that the traffic accessing the host port passes through the Pod selected by the load balancing algorithm located on the same host and the traffic accessing the host port passes through the Pod selected by the load balancing algorithm located on another node; The traffic accessing the host port passes through the Pod selected by the load balancing algorithm located on the same host, and the XDP network packet filtering mechanism intercepts network traffic packets at the network card receiving layer. If it is a network traffic packet requesting to establish a connection, a DNAT (Destination Address Translation) process is performed to replace the destination address of the network five-tuple with the address of the Pod selected by the load balancing algorithm, recalculate the header checksum, maintain the conntrack connection tracking information, and then forward the subsequent network traffic to the same Pod. If a network traffic packet requesting to disconnect is received, the conntrack connection tracking information is cleared; The traffic accessing the host port passes through the Pod selected by the load balancing algorithm located on another node, and the network traffic packet is intercepted. The DNAT process is performed on the network traffic packet, and the SNAT (Source Address Translation) process is performed. After the SNAT process, the forwarding information base is queried to update the MAC address of the packet, and the network traffic packet is forwarded to the output network card.

4. An eBPF-based network traffic governance system, applied to the method according to any one of claims 1-3, characterized in that, The system comprises: A control plane located in the user mode, wherein the control plane is configured to be responsible for subscribing to Service and Endpoints resource objects of a Kubernetes cluster, and after a data processing process, the resource objects after size end are synchronized to a data plane; The data plane is located in the kernel mode, wherein the data plane comprises an eBPF program, an eBPF map and a kernel mounting point, wherein the eBPF program takes resource object information in the eBPF map as a traffic management basis, and is configured to implement a packet forwarding function at a kernel mounting point through which network traffic packets pass; A traffic interception module configured to intercept network traffic packets at the kernel mounting point through the eBPF program, and screen out the network traffic packets accessing the cluster Service, and determine whether the destination address of the network traffic packets is a Service address. If the network traffic packets are the network traffic packets of the cluster Service, a traffic forwarding scheme is executed, and if not, the network traffic packets are released; Load balancing, in response to the execution of the traffic forwarding scheme, Endpoint information of a plurality of real service instances accessing the backend of the cluster Service is obtained, and the load balancing algorithm is selected to select the Endpoint information; An address conversion module configured to perform address conversion on the Endpoint information according to the Endpoint information selected by the load balancing algorithm, and complete NAT conversion. A connection tracking module is configured to perform connection tracking according to the network traffic data packet of address translation, the eBPF program modifies the packet quintuple according to the connection tracking information in response to the network traffic data packet, and response information of Endpoint information is correctly returned to the requesting client.

5. The system of claim 4, wherein, The data plane includes a slot algorithm optimization mapping storage method, the method comprising: Splitting the Endpoints list into single Pod elements for storage; In the process of tuning the Pod elements, the update granularity of the eBPF mapping is changed to a single Pod element, and the one-to-many relationship between the Service and the Pod is maintained by using the slot algorithm; In the traffic forwarding process, a single Pod element is selected as the backend service according to the slot, and the elements of the Endpoints list are marked as deleted, retained, and newly added, and the elements in the three states are uniformly processed when the elements of the Endpoints list dynamically change.

6. The system of claim 4, wherein, The eBPF program is of the cgroup type, and the kernel mounting point is cgroupv2.

7. The system of claim 4, wherein, The NAT conversion includes a DNAT conversion process and an SNAT conversion process.

8. An electronic device, comprising: Comprise: a microcontroller; a memory for storing executable instructions for the microcontroller; wherein the microcontroller is configured to execute the executable instructions stored in the memory to perform the method of any one of claims 1-3.

9. A computer-readable storage medium having stored thereon computer program instructions, wherein, The program instructions are executed by the processor to implement the steps of the method of any one of claims 1-3.

Images