Method and system for implementing transparent proxy based on eBPF technology

Abstract

The application relates to a method and system for realizing a transparent proxy based on an eBPF technology, which comprises the following steps: in the process of communication with a server through a tunnel, a tunnel network card of a load balancer encapsulates a data packet from a client based on the eBPF technology, the encapsulated data packet comprises tunnel outer layer address information and tunnel inner layer address information, wherein the source address of the tunnel inner layer address information is address information of the client, the destination address is address information of the server, the source address of the tunnel outer layer address information is address information of the load balancer, and the destination address is address information of the server; and the encapsulated data packet is sent to the server through the tunnel. According to the scheme, eBPF programs are built in the load balancer and the server, the eBPF programs are used to realize custom encapsulation and decapsulation of the tunnel, and the eBPF programs are used to realize storage and tracking of connection states, so that the transparent forwarding function is realized, and performance decline caused by the connection scale effect is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a method and system for realizing transparent proxy based on eBPF technology. BACKGROUND

[0002] In modern computer systems, load balancing is used to disperse a single high-load task into multiple low-load tasks through certain algorithms, and then hand over the tasks to multiple relatively low-performance processing units for processing, and finally return the processing results to the requester. The system using load balancing can realize the hot horizontal expansion of the system at a small cost, and at the same time reduce the impact of single point failure. It is the cornerstone of realizing high availability and elastic expansion of modern computer network architecture, and is an important basic component that almost all modern network architectures must have.

[0003] The typical application scenario of load balancing is shown in FIG. 1, a large number of clients send requests to the server, and the load balancer disperses these large number of requests to multiple backend servers for processing. Figure 1

[0004] Network layer load balancing is divided into two categories. The first category is only packet forwarding without TCP (Transmission Control Protocol) termination, that is, in essence, the TCP connection is actually established by the client directly with the server. The second category is TCP termination, that is, the client establishes a connection with the load balancing, and the load balancing establishes a connection with the server. In the first case, because the server reply packet does not need to pass through the load balancing, the load balancing can directly use the client IP (Internet Protocol) address as the source IP when forwarding to the server, and the present application does not discuss this. In the second case, because the connection with the client exists in the load balancing, the server packet must pass through the load balancing to reply to the client, that is, the server cannot directly forward the data packet using the client IP. How to solve the second problem is the focus of the present application.

[0005] The present application discusses the load balancing function with connection termination function of the transport layer TCP protocol in the load balancer. Here, connection termination refers to the fact that due to the functional requirements of TLS offloading, the load balancer cannot directly forward data packets to the backend server. For a TCP connection established by a client, two connections must be maintained on the load balancer, one between the load balancer and the client, and the other between the load balancer and the backend server. And this connection must be maintained in a one-to-one quantity relationship, and cannot be multiplexed. This type of load balancing implementation has many technical difficulties.

[0006] ​The transparent proxy is one of the basic functional requirements of the load balancer, that is, the load balancer is invisible to the request client and the backend server. In actual implementation, the load balancing cannot make any modification to the processing data packet content. To make the request initiator (client) and the request processing party (backend server) not aware of the existence of the proxy. Specifically, the TCP load balancing discussed in the present application has the following problems to be solved: how does the load balancer establish a connection with the backend server as a client, that is, the client IP, and how does the backend server send a request response to the client IP, and how does the response return to the load balancing.

[0007] In addition, the actual functional requirements are mainly based on the following three points:

[0008] 1. Network architecture is non-intrusive, and the client and the server do not need to be updated (for example, modified and updated after the software or application program) to use the numerous functions and features provided by the load balancing;

[0009] 2. The client and the server obtain the real network information of the opposite end, which is beneficial to the correct processing of network traffic by the application or security component;

[0010] 3. When the load balancer mainly processes network traffic based on long connection, it is necessary to allocate an IP address and a port for each connection to the server. Due to the limited number of IP address and port combinations, the number of connections processed by the load balancing is limited, which is lower than the load that the load balancing can handle. Therefore, the number of combinations will become a system load bottleneck in the normal case.

[0011] To achieve the above functions, the load balancer needs to retain the client source IP when forwarding the request. In the load balancing classification, the application layer load balancing can record the lower layer network layer source IP information through the application layer protocol field, and the network layer load balancing cannot use this method due to the lack of available fields in the protocol.

[0012] The existing network layer load balancing implementation scheme for retaining the source IP is as follows.

[0013] Proxy Protocol: By inserting an additional information frame (non-standard TCP protocol frame) after the TCP handshake is completed, the client IP information is transmitted. This method is equivalent to using a customized network layer protocol, and the server program must be adapted to the protocol to support the function implementation. The protocol is proposed by haproxy, and currently it has been implemented on a part of the load balancing, but the backend servers that can support this feature are limited. The main disadvantage is that it is highly invasive, and this method cannot solve the above-mentioned actual functional requirement 3.

[0014] The kernel tproxy module: by adding an extension field in the TCP handshake packet, the server program obtains the client IP when the client IP is returned, but the load balancing still uses its own IP instead of the client IP for communication in actual network communication. To implement this method, the server service programming language needs to use a specific system call to obtain the socket remote IP address. In some language implementations, exceptions occur, which is less universal. Moreover, this method cannot solve the actual functional requirement 3.

[0015] IP_TRANSPARENT+conntrack (iptables): the load balancing initiates a request to the server by binding the client IP through the IP_TRANSPARENT socket option, the server records the connection information through the conntrack function when receiving the packet, and reads the information record when returning the packet, so that the packet with the target address as the client can be correctly returned to the load balancing, and then returned to the client. Although this method can solve the actual functional requirement 3, there are still two problems. One problem is that the performance of conntrack decreases significantly when the number of tracked connections increases. The second problem is that this method directly uses the client IP for transmission in the server network, which affects the security of the server network. Some network environments prohibit external IP routing in the server network, which has certain incompatibility. SUMMARY

[0016] In order to achieve the load balancing at the network layer while preserving the source IP and meet the above actual functional requirements, the present application proposes a scheme for implementing transparent proxy based on eBPF technology.

[0017] According to a first aspect of the present application, a method for implementing transparent proxy based on eBPF technology is provided, applied to a load balancer, the method comprising:

[0018] In the process of communicating with the server through the tunnel, the tunnel network card of the load balancer encapsulates the data packet from the client based on eBPF technology, the encapsulated data packet comprising tunnel outer layer address information and tunnel inner layer address information, wherein the source address of the tunnel inner layer address information is the address information of the client, and the destination address is the address information of the server, and the source address of the tunnel outer layer address information is the address information of the load balancer, and the destination address is the address information of the server;

[0019] Send the encapsulated data packet to the server through the tunnel.

[0020] According to a second aspect of the present application, a method for implementing transparent proxy based on eBPF technology is provided, applied to a server, the method comprising:

[0021] In a case that a data packet is received from the load balancer through the tunnel, address information carried by the data packet is saved in a predetermined storage area based on an eBPF technology, wherein the saved address information includes address information of the client and address information of the load balancer;

[0022] The data packet is processed to obtain a response data packet;

[0023] The response data packet is forwarded to the tunnel network card of the server;

[0024] The tunnel network card of the server encapsulates the response data packet based on the eBPF technology according to the predetermined storage area, wherein the encapsulated response data packet includes tunnel outer layer address information and tunnel inner layer address information, wherein a source address of the tunnel inner layer address information is address information of the server, and a destination address is address information of the client, a source address of the tunnel outer layer address information is address information of the server, and a destination address is address information of the load balancer;

[0025] The encapsulated response data packet is sent to the load balancer through the tunnel.

[0026] According to a third aspect of the present application, a system for implementing transparent proxy based on an eBPF technology is provided, which includes a client, a load balancer and a server, wherein:

[0027] The client is configured to send a data packet to the client;

[0028] The load balancer is configured to perform the method according to the first aspect; and

[0029] The server is configured to perform the method according to the second aspect.

[0030] According to a fourth aspect of the present application, an electronic device is provided, which comprises:

[0031] A processor; and

[0032] A memory storing computer instructions, when the computer instructions are executed by the processor, the processor executes the method according to the first aspect and the second aspect.

[0033] According to a fifth aspect of the present application, a non-transitory computer storage medium is provided, which stores a computer program, when the computer program is executed by a plurality of processors, the processors execute the method according to the first aspect and the second aspect.

[0034] According to the scheme for implementing transparent proxy based on the eBPF technology provided in the application, the eBPF program is built in the load balancer and the server of the system, the custom packet encapsulation and decapsulation of the tunnel are implemented through the eBPF program, the performance is guaranteed to be consistent with the operating system kernel while the function is expanded, the connection state is stored and tracked through the eBPF program, the performance decline caused by the connection scale effect is avoided while the transparent forwarding function is implemented, and in addition, the traffic is distinguished based on the rule-based routing marking, and the transmission of non-target traffic of the host is not affected. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art based on these drawings without departing from the scope of the application.

[0036] Figure 1 is a schematic diagram of a load balancing application scenario.

[0037] Figure 2 is a processing flowchart of a system for implementing transparent proxy based on the eBPF technology according to an embodiment of the application.

[0038] Figure 3 is a flowchart of a method for implementing transparent proxy based on the eBPF technology implemented by a load balancer according to an embodiment of the application.

[0039] Figure 4 is a flowchart of a method for implementing transparent proxy based on the eBPF technology implemented by a server according to an embodiment of the application.

[0040] Figure 5 is a structural diagram of an electronic device provided in an embodiment of the application. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the application will be described clearly and completely in combination with the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all the embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the application.

[0042] The transparent proxy scheme based on the eBPF technology is provided, and in the load balancer, the tunnel network card encapsulates the data packet from the client based on the eBPF technology when performing tunnel transmission, the encapsulated data packet includes tunnel outer layer address information and tunnel inner layer address information, the source address of the tunnel inner layer address information is the address information of the client, the destination address is the address information of the server, the source address of the tunnel outer layer address information is the address information of the load balancer, and the destination address is the address information of the server; in the server, the tunnel network card saves the address information carried by the target data packet (for example, the data packet from the client) in a storage area based on the eBPF technology when receiving the data packet through the tunnel, the saved address information includes the address information of the client and the address information of the load balancer; and the server encapsulates the response data packet based on the eBPF technology according to the storage area when returning the response data packet, the encapsulated response data packet includes tunnel outer layer address information and tunnel inner layer address information, the source address of the tunnel inner layer address information is the address information of the server, the destination address is the address information of the client, the source address of the tunnel outer layer address information is the address information of the server, and the destination address is the address information of the load balancer. In this way, the eBPF program is used to realize custom encapsulation and decapsulation of the tunnel and storage and tracking of the connection state, first, the eBPF program is implemented in the host operating system, and the client and the server do not need to be updated, so the network architecture is not invasive; second, the load balancer retains the source IP of the client when forwarding the data packet, and the server can also know which load balancer to return the response data packet to. In addition, the rule-based routing marking is used to distinguish the traffic, which does not affect the transmission of non-target traffic of the host while realizing the processing of the target traffic (for example, the data packet from the client).

[0043] Figure 2 The processing flowchart of the system for realizing the transparent proxy based on the eBPF technology according to the embodiment of the application is shown in FIG. 1. Figure 2 As shown in the figure, the system includes a client, a load balancer and a server. A tunnel needs to be established between the load balancing host and the server host, and the established tunnel includes but is not limited to GRE, IPIP, VxLAN, Geneve and the like, Figure 2 The GRE tunnel is taken as an example in the figure. For the load balancer and the server, both of them include an application program, an operating system and a network card. The kernel of the operating system has a routing function, and can realize the forwarding of the target traffic through a routing table. The network card includes a main network card eth0 and a tunnel network card tun0. The IP address of the client, the load balancer and the server can be a wide area network WAN IP address or a local area network LAN address, which is not limited in the application. Figure 2In this case, the addresses of the client, the load balancer and the server are WAN IP 1.1.1.1, LAN IP 172.22.0.1 (WAN IP 2.2.2.2) and LAN IP 172.22.0.2, respectively.

[0044] As shown in FIG. 1, the client sends a data packet to the load balancer, and after receiving the data packet from the client, the load balancer and the server perform the following process. Figure 2

[0045] 1. The client initiates a new connection request message to the service WAN IP 2.2.2.2 using the WAN IP 1.1.1.1. The main network card of the load balancer forwards the received data packet to the application program, and the load balancer selects a reverse proxy server instance according to the balancing algorithm. After selecting the target server, the corresponding socket is started to initiate a new connection request message to the selected server, and the client IP is bound, for example, the load balancer uses the socket SO_IP_TRANSPARENT to bind the client IP as the source IP.

[0046] 2. At the same time, the load balancer marks the target traffic, for example, marks the data packet of the client. In an embodiment, the load balancer marks the traffic through the socket SO_MARK, for example, marks the target traffic as a specific mark value M, which is used in subsequent processing to distinguish the target traffic and non-target traffic on the operating system TC (Traffic Control) program, so as to ensure that they do not conflict with each other when coexisting.

[0047] 3. The data packet marked (for example, marked as a specific value M) is transferred to the tunnel network card tun0 for processing. There are two ways to achieve this step.

[0048] Method one: achieved through the rule-based routing (rule based routing) routing mechanism of the operating system kernel

[0049] Specifically, first, add a routing table RT1 in the operating system kernel, and only add a default routing rule to the tunnel network card tun0 in the table, and add a kernel routing rule through the ip rule tool to search the routing table RT1 for forwarding the traffic marked as M (fwmark M). Since there is only a default rule to tun0 in RT1, all traffic marked as M will be processed through the tunnel network card tun0.

[0050] Method two: forwarding through the eBPF program on the main network card eth0

[0051] ​In this mode, no modification of routing rules is needed as in mode one, and the data packet is sent out by default through the eth0 TC egress. Specifically, an eBPF program is designed to be mounted to the eth0 TC egress hook. The processing logic of the program is to first filter the non-marked traffic according to the mark field in the data packet (sk_buff), and redirect the marked traffic to the tunnel network card tun0 for processing.

[0052] 4. An eBPF program is designed and implemented to be mounted to the TC egress hook of the tunnel network card tun0 set in advance. The processing logic of the program is to set the tunnel-related packet properties and encapsulate the original data packet in the inner layer. Specifically, the tunnel network card tun0 encapsulates the data packet from the client based on the eBPF technology, and the encapsulated data packet includes tunnel outer layer address information and tunnel inner layer address information, wherein the source address of the tunnel inner layer address information is the address information of the client, and the destination address is the address information of the server, and the source address of the tunnel outer layer address information is the address information of the load balancer, and the destination address is the address information of the server. In the embodiment shown in Figure 2 , the source address of the tunnel inner layer address information is 1.1.1.1, and the destination address is 172.22.0.2. The source address of the tunnel outer layer address information is 172.22.0.1, and the destination address is 172.22.0.2. After encapsulation, the data packet is sent.

[0053] 5. The encapsulated data packet is sent to the server through the tunnel. As shown in Figure 2 , due to the existence of the tunnel, the switch in the LAN forwards the data packet according to the address (172.22.0.1 to 172.22.0.2). The tunnel network card tun0 of the server receives the data packet.

[0054] 6. After receiving the data packet from the load balancer, the server decapsulates the tunnel header and throws it to the kernel as a regular data packet. At this time, the data packet only has the inner layer address, i.e., from 1.1.1.1 to 172.22.0.2. At the same time, it can be known that the returned data packet will be sent from 172.22.0.2 to 1.1.1.1. 1.1.1.1 is the WAN IP, and due to TCP termination, it cannot be returned along the original path. If no intervention is made, the data packet will be directly returned to the client through the router WAN port, resulting in an error. In order to make the data packet return to the load balancing host along the original path, the path needs to be saved when the data packet is received.

[0055] According to some embodiments, the processing here distinguishes between different states of TCP links.

[0056] 1) For TCP SYN (Synchronize Sequence Numbers) handshake packet

[0057] When the first SYN packet (packet used by TCP three-way handshake to establish session) is received, save the inner source IP+Port (i.e. real client IP) as key, and the outer source IP+Port (i.e. load balancing IP) as value, as shown in Figure 2 , where the inner source IP+Port is 1.1.1.1:123, and the outer source IP+Port is 172.22.0.1:25. When the SYN packet is received, only the handshake is initiated, and in certain cases, such as TCP layer health detection, SYN flooding attack, etc., the handshake will not be truly completed. In these cases, data cleaning work needs to be considered to prevent storage leakage. In the eBPF program, a predetermined storage area, such as a BPF_MAP_TYPE_LRU_HASH type map (hereinafter referred to as a syn cache map), is used for storage. This type of map uses LRU algorithm to structure the data, and there is no need to worry about storage depletion. Although this type of map may have data overwritten, because this temporary storage path is only used when the server responds to the SYN packet with a SYN_ACK (accept handshake) or RST (reject handshake) (the response time is usually between tens to hundreds of milliseconds depending on the kernel parameters), as long as a certain LRU map length is guaranteed (according to the actual load requirements, the number of new connections per second), the data overwrite defect of the LRU map can be well avoided.

[0058] 2) For data packets after the TCP SYN handshake packet

[0059] As described in 1), the storage in 1) only guarantees the return cache during the TCP handshake phase. When the server receives the client ACK to complete the handshake, the TCP connection state on the server will change to ESTABLISHED. From this time on, until the TCP connection is disconnected (including any reason for disconnection), the connection return needs to be saved. According to some specific embodiments, the tracepoint hook is used to load the program into the "sock:inet_sock_set_state" tracepoint. When there is any change in the TCP connection state, the kernel will call back to execute the tracepoint.

[0060] According to one specific embodiment, the eBPF program processing logic pseudo code is

[0061]

[0062] When the connection state becomes ESTABLISHED, the cached data in 1) is queried first for screening target connections. When a data packet is received, it is determined whether the connection information (including client address information and load balancer address information) carried by the data packet exists in the syn cache map in 1). When it exists in the syn cache map in 1), it is determined as a target connection, otherwise it is a non-target connection. For non-target connections, the processing is returned; for target connections, the key and value information are stored in a predetermined storage area, for example, a map of type BPF_MAP_TYPE_HASH (hereinafter referred to as a conntrack map). As shown in Figure 2

[0063] According to other embodiments, different states of TCP links can not be distinguished. Specifically, for SYN packets and other data packets after SYN packets, the connection information (client IP and load balancer IP) of the data packets is saved in the same predetermined storage area, for example, in a map of type BPF_MAP_TYPE_HASH.

[0064] 7. The application on the server normally receives the data packet for processing (without awareness of the tunnel or lower layer network processing logic), at which time the data packet received by the application has a source IP of 1.1.1.1, i.e., the real client IP.

[0065] 8. The application on the server returns a response data packet to the main network card and sends it to the target IP 1.1.1.1.

[0066] 9. Since the target IP is the WAN IP 1.1.1.1, there will be no routing item matching on the server host, so it is sent to the outside through the default routing item via the main network card eth0. In this case, an eBPF program needs to be implemented on the TCegress HOOK on the server main network card to realize data packet forwarding. According to the data packet information, it is determined whether it is target traffic, and the target traffic is redirected to the tunnel network card tun0 for processing.

[0067] According to an embodiment, the processing logic pseudo code is as follows:

[0068]

[0069] ​10、Tunnel NIC tun0 prepares to send the data packet through the tunnel. At this time, the data packet processed by the tunnel NIC tun0 has a destination IP of 1.1.1.1, and the correct outer tunnel IP address needs to be set to enable the data packet to return to the original load balancing host. According to an embodiment, the eBPF program loaded on the tun0 TC egress processes the logic of taking the destination IP + port as the key to query the destination address to be used when returning.

[0070] According to an embodiment, for the case of two predetermined storage areas (for example, syn cache map and conntrack map), according to the different types of data packets, the information stored in the two predetermined storage areas is queried, specifically, for SYN ACK or RST data packets, the syn cache map is queried, and for other types of data packets, the conntrack map is queried. According to another embodiment, for the case of only one predetermined storage area, the destination IP + port is taken as the key to query the destination address to be used when returning in the predetermined storage area.

[0071] If the query result is empty, the data packet is not processed; if the query result is the load balancing host IP, the outer IP, that is, the tunnel destination IP, is set to this value. As shown in Figure 2 , for the key of destination IP + port "1.1.1.1: 123", the destination address to be used when returning is 172.22.0.1. Then, the outer address information of the tunnel has a destination address of 172.22.0.1.

[0072] In this way, the tunnel NIC tun0 encapsulates the response data packet based on the predetermined storage area based on eBPF technology, and the encapsulated response data packet includes tunnel outer address information and tunnel inner address information, wherein the source address of the tunnel inner address information is the address information of the server, and the destination address is the address information of the client, the source address of the tunnel outer address information is the address information of the server, and the destination address is the address information of the load balancer. As shown in Figure 2 , the source address of the tunnel inner address information is 172.22.0.2, and the destination address is 1.1.1.1. The source address of the tunnel outer address information is 172.22.0.2, and the destination address is 172.22.0.1. After encapsulation, it is sent.

[0073] 11、Send the encapsulated response data through the tunnel.

[0074] 12. After the load balancer's tunnel network interface tun0 receives a response packet from the server, it removes the external tunnel header normally, and the kernel receives the response packet. At this time, the destination IP of the response packet is 1.1.1.1. The kernel determines that this is not a packet destined for the local machine. If no processing is performed, it will forward the packet according to the kernel routing table, and the local application will not receive the packet. To solve this problem, according to one embodiment, an eBPF program is loaded into the TC ingress of the tunnel network interface tun0 to mark the packet before it is sent to the kernel for processing, for example, marking it as MM, so that specific kernel routing rules can be configured in the next step.

[0075] 13. Add a routing table RT2 in the kernel, and add a default routing rule so that all packets are received through the local loopback (LO) interface. Add kernel routing rules using the `ip rule` tool or a direct system call, and look up the routing table RT2 for traffic marked as MM (fwmarkMM) before forwarding it. Because RT2 only contains the default rule, all traffic marked as MM will be received through the local loopback interface.

[0076] 14. The load balancer received the response data packet normally.

[0077] 15. The load balancer sends the response packet to the client based on the destination IP 1.1.1.1 of the response packet.

[0078] Thus, through the above process, the load balancer forwards requests from clients to backend servers for processing and receives responses from the servers.

[0079] Based on the aforementioned system for implementing transparent proxy using eBPF technology, according to one aspect of this application, a method for implementing transparent proxy using eBPF technology implemented by a load balancer is provided. For example... Figure 3 As shown, the method includes the following steps.

[0080] Step S301: Mark the data packets from the client in the received data packets;

[0081] Step S302: Forward the tagged data packet to the tunnel network card of the load balancer.

[0082] In the embodiment shown in FIG. 2, the client initiates a new connection request message to the service WAN IP 2.2.2.2 using the WAN IP 1.1.1.1. The main network card of the load balancer forwards the received data packet to the application. The load balancer selects a reverse proxy server instance according to the balancing algorithm. After selecting the target server, the load balancer initiates a new connection request message to the selected server using the corresponding socket, binds the client IP, for example, the load balancer binds the client IP as the source IP using the socket SO_IP_TRANSPARENT.

[0083] At the same time, the load balancer marks the target traffic, for example, marks the data packet of the client. In one embodiment, the load balancer marks the traffic through the socket SO_MARK, for example, marks the target traffic as a specific mark value M. The mark is used in subsequent processing to distinguish between target traffic and non-target traffic by the program on the operating system TC, thereby ensuring that they do not conflict with each other when coexisting. The data packet to be marked (for example, marked as a specific value M) is transferred to the tunnel network card tun0 for processing. There are two ways to implement this step.

[0084] Method one: implementation through the rule-based routing mechanism of the operating system kernel

[0085] Specifically, first, add a routing table RT1 in the operating system kernel. In the table, only add a default routing rule to the tunnel network card tun0. At the same time, add a kernel routing rule through the ip rule tool. For traffic marked as M (fwmark M), search the routing table RT1 for forwarding. Since there is only a default rule to tun0 in RT1, all traffic marked as M will be processed through the tunnel network card tun0.

[0086] Method two: forwarding through the eBPF program on the main network card eth0

[0087] In this way, there is no need to modify the routing rule as in method one. The data packet is sent to the outside by default through the eth0 TC egress. Specifically, an eBPF program is designed to be mounted to the eth0 TC egress hook. The program processing logic is as follows: first, according to the mark field in the data packet (sk_buff), that is, the mark value, filter the non-marked traffic, and redirect the marked traffic to the tunnel network card tun0 for processing.

[0088] As can be seen, according to some embodiments, step S302 includes: forwarding the marked data packet to the tunnel network card of the load balancer according to the routing table in the operating system kernel of the load balancer; or forwarding the marked data packet to the tunnel network card of the load balancer through the main network card of the load balancer.

[0089] Step S303, in the process of communicating with the server through the tunnel, the tunnel network card of the load balancer encapsulates the data packet from the client based on the eBPF technology, and the encapsulated data packet includes tunnel outer layer address information and tunnel inner layer address information, wherein the source address of the tunnel inner layer address information is the address information of the client, and the destination address is the address information of the server, the source address of the tunnel outer layer address information is the address information of the load balancer, and the destination address is the address information of the server.

[0090] Step S304, the encapsulated data packet is sent to the server through the tunnel.

[0091] According to Figure 2 As shown in the embodiment, the eBPF program is designed and implemented to be mounted to the TC egress hook work of the tunnel network card tun0 set in advance. The program processing logic is to set the tunnel related packet attribute and encapsulate the original data packet in the inner layer. Specifically, the tunnel network card tun0 encapsulates the data packet from the client based on the eBPF technology, and the encapsulated data packet includes tunnel outer layer address information and tunnel inner layer address information, wherein the source address of the tunnel inner layer address information is the address information of the client, and the destination address is the address information of the server, the source address of the tunnel outer layer address information is the address information of the load balancer, and the destination address is the address information of the server. As shown in Figure 2 In the embodiment shown in the embodiment, the source address of the tunnel inner layer address information is 1.1.1.1, and the destination address is 172.22.0.2. The source address of the tunnel outer layer address information is 172.22.0.1, and the destination address is 172.22.0.2. After encapsulation, the data packet is sent.

[0092] According to Figure 2 As shown in the embodiment, the encapsulated data packet is sent to the server through the tunnel. As shown in Figure 2 Because of the existence of the tunnel, the switch in the LAN forwards the data packet according to the address (172.22.0.1 to 172.22.0.2). The tunnel network card tun0 of the server receives the data packet.

[0093] According to Figure 2 As shown in the embodiment, after the tunnel network card tun0 of the load balancer receives the response data packet from the server, the outer tunnel header is normally removed, and the response data packet is received by the kernel. At this time, the destination IP of the response data packet is 1.1.1.1, and the kernel judges that the data packet is not sent to the local machine. If no processing is performed, the data packet will be forwarded according to the kernel routing table, and then the local application program will not be able to receive the data packet. In order to solve this problem, according to an embodiment, the eBPF program is loaded on the TC ingress of the tunnel network card tun0, and the data packet is marked before being sent to the kernel for processing, for example, marked as MM, so as to configure a specific kernel routing rule in the next step.

[0094] Add a routing table RT2 in the kernel, add a default routing rule, all packets are received through the local loopback (LO) network interface. Add a kernel routing rule through the ip rule tool or a direct system call, and route the traffic marked as MM (fwmark MM) to the routing table RT2. Because there is only a default rule in RT2, that is, all traffic marked as MM will be received through the local loopback network interface. In this way, the load balancer normally receives the response packet. In the case of receiving a response packet from the server through a tunnel, the packet from the server is received through the loopback network interface of the load balancer.

[0095] In Figure 2 Based on the system for implementing transparent proxy based on eBPF technology as shown in Figure 4 According to another aspect of the present application, a method for implementing transparent proxy based on eBPF technology implemented by a server is provided. As shown in

[0096] Step S401, in the case of receiving a packet from the load balancer through a tunnel, the address information carried by the packet is saved in a predetermined storage area based on eBPF technology, wherein the saved address information includes the address information of the client and the address information of the load balancer.

[0097] According to the embodiment shown in Figure 2 After the server receives the packet from the load balancer, the tunnel header is unwrapped, and the packet is thrown to the kernel for normal packet reception. At this time, the packet only has the inner address, that is, it is sent from 1.1.1.1 to 172.22.0.2. At the same time, it is known that the returned packet will be sent from 172.22.0.2 to 1.1.1.1.1.1.1.1 is the WAN IP, and due to TCP termination, it cannot be returned along the logical original path. If no intervention is made, the packet will be directly returned to the client through the router WAN port, resulting in an error. In order to make the packet return to the load balancing host along the original path, the path needs to be saved when the packet is received.

[0098] According to some embodiments, the processing here distinguishes between different states of TCP links.

[0099] 1) For TCP SYN handshake packets

[0100] When the first SYN packet (a packet used in the three-way handshake of TCP session establishment) is received, the inner source IP+Port (that is, the real client IP) is saved as the key, and the outer source IP+Port (that is, the load balancing IP) is saved as the value, as shown in Figure 2As shown, the inner source IP+Port is 1.1.1.1:123, and the outer source IP+Port is 172.22.0.1:25. When receiving the SYN packet, only for initiating the handshake, in some specific cases, such as TCP layer health detection, SYN flooding attack, etc., the handshake will not be truly completed, and in this case, data cleaning work needs to be considered in the above cases to prevent storage leakage. In the eBPF program, a predetermined storage area is used, such as a BPF_MAP_TYPE_LRU_HASH type map (called a syn cache map) in the BPF file system dedicated to the eBPF program. This type of map uses the LRU algorithm to structure the data, and there is no need to worry about storage depletion. Although this type of map may have data overwritten, because this temporary storage path is only used when the server returns SYN_ACK (accept handshake) or RST (reject handshake) in response to the SYN (the response time is usually between tens of milliseconds and hundreds of milliseconds according to different kernel parameters), as long as a certain LRU map length is guaranteed (according to the actual load demand, the number of new connections per second), the defect of data overwriting of the LRU map can be well avoided.

[0101] 2) For the data packet after the TCP SYN handshake packet

[0102] As described in 1), the storage in 1) only guarantees the backhaul cache in the TCP handshake phase. When the server receives the client ACK to complete the handshake, the TCP connection state on the server will change to ESTABLISHED, and from this time on, until the TCP connection is disconnected (including any reason for disconnection), the connection backhaul needs to be saved. According to some specific embodiments, the tracepoint hook is used, and the program is loaded into the "sock:inet_sock_set_state" tracepoint. When there is any change in the TCP connection state, the kernel will call back to execute the tracepoint.

[0103] When the connection state changes to ESTABLISHED, the cached data in 1) is queried first for filtering target connections. When receiving the data packet, it is judged whether the connection information (including the client address information and the address information of the load balancer) carried by the data packet exists in the syn cache map in 1). When it exists in the syn cache map in 1), it is determined as a target connection, otherwise it is a non-target connection. For non-target connections, return processing; for target connections, store the key and value information to a predetermined storage area, such as a BPF_MAP_TYPE_HASH type map (called a conntrack map) in the BPF file system dedicated to the eBPF program. As shown in the following table, the key is the connection information (including the client address information and the address information of the load balancer), and the value is the connection information (including the server address information and the server port information) of the server. Figure 2As shown, the key in the conntrack map is 1.1.1.1:123 and the value is 172.22.0.1:25. When the connection state leaves ESTABLISHED, the stored information needs to be cleared.

[0104] In this way, step S401 comprises: in the case where the data packet is a SYN handshake packet of a TCP connection, saving the address information carried by the data packet in a first storage area; and in the case where the data packet is not a SYN handshake packet of a TCP connection, saving the address information carried by the data packet in a second storage area; wherein the first storage area and the first storage area have different types.

[0105] According to one specific embodiment, for the step of "in the case where the data packet is not a SYN handshake packet of a TCP connection, saving the address information carried by the data packet in a second storage area", the specific implementation manner comprises: in the case where it is determined that the TCP connection has been established, saving the address information carried by the data packet in the second storage area in response to determining that the address information carried by the received data packet exists in the first storage area.

[0106] According to some other embodiments, the TCP links in different states can not be distinguished. Specifically, for the SYN packet and other data packets after the SYN packet, the connection information (client IP and load balancer IP) of the data packet is saved in the same predetermined storage area, for example, in a map of type BPF_MAP_TYPE_HASH.

[0107] Step S402, processing the data packet to obtain a response data packet.

[0108] According to Figure 2 As shown in the embodiment, the application program on the server normally (without awareness of the tunnel or lower layer network processing logic) receives the data packet for processing, at which time the source IP of the data packet received by the application program is 1.1.1.1, i.e. the real client IP.

[0109] Step S403, forwarding the response data packet to the tunnel network card of the server.

[0110] According to Figure 2In the embodiment shown, the application on the server returns the response packet to the main network card, and sends it to the target IP 1.1.1.1. Since the target IP is the WAN IP 1.1.1.1, there will be no routing entry matching on the server host, and it will be sent out through the default routing entry via the main network card eth0. In this case, an eBPF program needs to be implemented on the TC egress HOOK on the server main network card to realize packet forwarding. According to the packet information, it is determined whether it is target traffic, and the target traffic is redirected to the tunnel network card tun0 for processing.

[0111] In this way, step S403 specifically includes: sending, by the application on the server, the response packet to the main network card of the server; and in the case where the target address of the response packet is the address information of the client, redirecting, by the main network card of the server, the response packet to the tunnel network card of the server.

[0112] In step S404, the tunnel network card of the server encapsulates the response packet based on the eBPF technology according to the predetermined storage area, wherein the encapsulated response packet includes tunnel outer layer address information and tunnel inner layer address information, wherein the source address of the tunnel inner layer address information is the address information of the server, and the destination address is the address information of the client, and the source address of the tunnel outer layer address information is the address information of the server, and the destination address is the address information of the load balancer.

[0113] In step S405, the encapsulated response packet is sent to the load balancer through the tunnel.

[0114] According to Figure 2 In the embodiment shown, the tunnel network card tun0 prepares to send the packet through the tunnel. At this time, the destination IP of the packet processed by the tunnel network card tun0 is 1.1.1.1, and the correct outer tunnel IP address needs to be set to make the packet return to the original load balancing host. According to an embodiment, the eBPF program loaded on the tun0 TC egress processing logic is to take the destination IP+port as the key, and query the destination address to be used when returning.

[0115] According to an embodiment, for the case of two predetermined storage areas (such as syn cache map and conntrack map), according to the different types of packets, the information stored in the two predetermined storage areas is queried, specifically, for SYN ACK or RST packets, syn cache map is queried, and for other types of packets, conntrack map is queried. According to another embodiment, for the case of only one predetermined storage area, the destination IP+port is taken as the key, and the destination address to be used when returning is queried in the predetermined storage area.

[0116] If the query result is empty, the data packet is not processed; if the query result is a load balancing host IP, the outer IP, i.e., the tunnel destination IP, is set to this value. As shown in Figure 2 , for the key with the destination IP + port "1.1.1.1:123", the destination address to be used when returning should be 172.22.0.1. Then, the outer address information of the tunnel is the destination address 172.22.0.1.

[0117] In this way, the tunnel card tun0 encapsulates the response data packet based on the predetermined storage area based on the eBPF technology, and the encapsulated response data packet includes the outer address information of the tunnel and the inner address information of the tunnel, wherein the source address of the inner address information of the tunnel is the address information of the server, and the destination address is the address information of the client, the source address of the outer address information of the tunnel is the address information of the server, and the destination address is the address information of the load balancer. In the embodiment shown in Figure 5 , the source address of the inner address information of the tunnel is 172.22.0.2, the destination address is 1.1.1.1, the source address of the outer address information of the tunnel is 172.22.0.2, and the destination address is 172.22.0.1, and the encapsulation is sent after completion.

[0118] Then, the encapsulated response data packet is sent through the tunnel.

[0119] According to the method and system for implementing transparent proxy based on the eBPF technology provided in the present application, the eBPF program is built in the load balancer and the server of the system, the custom encapsulation and decapsulation of the tunnel are realized through the eBPF program, the performance is consistent with the operating system kernel while the functions are expanded; and the connection state is tracked through the eBPF program, the transparent forwarding function is realized while the performance decline caused by the connection scale effect is avoided; in addition, the traffic is distinguished based on the rule-based routing marking, and the transmission of non-target traffic of the host is not affected.

[0120] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0121] Referring to Figure 5 , Figure 3 An electronic device is provided, including a processor; and a memory storing computer instructions, when the computer instructions are executed by the processor, the processor implements the method and detailed solutions as shown in Figure 4 and Figure 3 when the computer instructions are executed.

[0122] It should be understood that the above-mentioned apparatus embodiments are only illustrative, and the apparatus disclosed in the present application can also be implemented in other manners. For example, the division of the units / modules in the above-mentioned embodiments is only a logical function division, and actual implementation can be in another manner. For example, a plurality of units / modules or components can be combined, or can be integrated into another system, or some features can be omitted or not executed.

[0123] In addition, each functional unit / module in each embodiment of the present application can be integrated into one unit / module, or each unit / module can exist physically, or two or more units / modules can be integrated together, unless otherwise specified. The integrated unit / module can be implemented in the form of hardware or in the form of a software program module.

[0124] The integrated unit / module, if implemented in the form of hardware, can be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. Unless otherwise specified, the processor or chip can be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. Unless otherwise specified, the on-chip cache, off-chip memory, and storage can be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory (RRAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), an enhanced dynamic random access memory (EDRAM), a high-bandwidth memory (HBM), a hybrid memory cube (HMC), etc.

[0125] The integrated unit / module, if implemented in the form of a software program module and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present disclosure. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0126] The embodiments of the present application also provide a non-transitory computer storage medium, which stores a computer program. When the computer program is executed by one or more processors, the processors execute the method shown in the embodiments of the present application and the detailed solutions. Figure 4 and ​ The method and the detailed solutions.

[0127] The embodiments of the present application are described in detail above, and the principles and implementation manners of the present application are described by applying specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, the changes or modifications made by the person skilled in the art according to the idea of the present application, based on the specific implementation manners and application scope of the present application, all belong to the scope of protection of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for implementing transparent proxy based on eBPF technology, applied to a load balancer, the method comprising: receiving a data packet from a client; storing address information carried by the data packet in a predetermined storage area based on eBPF technology, the stored address information including address information of the client and address information of a target server; wherein the storing of the address information carried by the data packet in the predetermined storage area based on eBPF technology comprises: in a case where the data packet is a SYN handshake packet of a TCP connection, storing the address information carried by the data packet in a first storage area, the first storage area being a temporary storage area managed using an LRU algorithm to automatically clean up address information of a connection that is not successfully established; and in a case where the data packet is not a SYN handshake packet of a TCP connection, storing the address information carried by the data packet in a second storage area, the second storage area being a storage area that persistently stores address information during an entire establishment state of a TCP connection; wherein the storing of the address information carried by the data packet in the second storage area in the case where the data packet is not a SYN handshake packet of a TCP connection comprises, in a case where it is determined that a TCP connection has been established, storing the address information carried by the data packet in the second storage area in response to determining that the address information carried by the received data packet exists in the first storage area; in a process of communicating with a server through a tunnel, a tunnel network card of the load balancer encapsulates a data packet from a client based on eBPF technology, the encapsulated data packet including outer-layer address information of the tunnel and inner-layer address information of the tunnel, wherein a source address of the inner-layer address information of the tunnel is address information of the client, and a destination address is address information of the server, a source address of the outer-layer address information of the tunnel is address information of the load balancer, and a destination address is address information of the server; sending the encapsulated data packet to the server through the tunnel. 2.The method of claim 1, further comprising: labeling a data packet from the client in the received data packet; forwarding the labeled data packet to the tunnel network card according to a routing table in a kernel of an operating system of the load balancer; or forwarding the labeled data packet to the tunnel network card through a main network card of the load balancer. The tunnel includes a GRE, IPIP, VxLAN, or Geneve tunnel, and the outer-layer address information of the tunnel and the inner-layer address information of the tunnel each include IP address information or IP address information and port information.

3. The method of any one of claims 1 or 2, wherein, 4.A method for implementing transparent proxy based on eBPF technology, applied to a server, the method comprising: in a case where a data packet is received from a load balancer through a tunnel, storing address information carried by the data packet in a predetermined storage area based on eBPF technology, wherein the stored address information includes address information of a client and address information of the load balancer; processing the data packet to obtain a response data packet; and ​ a tunnel network card of the server; the tunnel network card of the server encapsulates the response data packet based on an eBPF technology according to the predetermined storage area, wherein the encapsulated response data packet comprises tunnel outer layer address information and tunnel inner layer address information, wherein a source address of the tunnel inner layer address information is address information of the server, and a destination address is address information of the client, a source address of the tunnel outer layer address information is address information of the server, and a destination address is address information of the load balancer; the encapsulated response data packet is sent to the load balancer through the tunnel; wherein the address information carried by the data packet is saved in a predetermined storage area based on the eBPF technology comprises: in a case where the data packet is a SYN handshake packet of a TCP connection, the address information carried by the data packet is saved in a first storage area; and in a case where the data packet is not a SYN handshake packet of a TCP connection, the address information carried by the data packet is saved in a second storage area; wherein the first storage area and the second storage area have different types; the first storage area is a temporary storage area managed using an LRU algorithm to automatically clean up address information that has not successfully established a connection; and the second storage area is a storage area that persistently saves address information during the entire establishment state of a TCP connection; wherein the address information carried by the data packet is saved in the second storage area in a case where the data packet is not a SYN handshake packet of a TCP connection comprises: in a case where it is determined that a TCP connection has been established, the address information carried by the data packet is saved in the second storage area in response to determining that the address information carried by the received data packet exists in the first storage area.

5. The method of claim 4, wherein, the tunnel network card of the server forwards the response data packet to the server comprises: the response data packet is sent to a host network card of the server by an application program of the server; in a case where a destination address of the response data packet is address information of the client, the response data packet is redirected to the tunnel network card of the server by the host network card of the server.

6. A system for implementing transparent proxy based on an eBPF technology, comprising a client, a load balancer, and a server, wherein: the client is configured to send a data packet to the client; the load balancer is configured to perform the method of any one of claims 1 to 3; and the server is configured to perform the method of any one of claims 4 to 5.

7. An electronic device, comprising: comprises: a processor; and a memory storing computer instructions that, when executed by the processor, cause the processor to perform the method of any one of claims 1 to 5.

8. A non-transitory computer storage medium storing a computer program, comprising: when the computer program is executed by multiple processors, the processors perform the method of any one of claims 1 to 5.

Images