K8s Service back-end fault active detection method based on eBPF

Abstract

The invention provides a K8s Service back-end fault active detection method based on an eBPF (evolved Binary Path Function). According to the K8s Service back-end fault active detection method based on the eBPF provided by the embodiment of the invention, the connection state of the K8s Service back-end Pod can be monitored in real time in a kernel mode, the flow forwarding of the fault Pod can be blocked in time, and the availability and stability of the service can be obviously improved.

Description

Technical Field

[0001] This invention relates to the field of container network service fault detection technology, and more particularly to a proactive fault detection method for the backend of a K8s Service based on eBPF. Background Technology

[0002] Kubernetes (K8s), as the core technology of container orchestration systems, is widely used in cloud computing, microservice architectures and large-scale distributed systems. Its Service component provides Pods with stable network access interfaces and load balancing capabilities.

[0003] In related technologies, a label-selector-based backend Pod discovery and traffic forwarding mechanism is built through the collaborative operation of iptables or ipvs with kube-proxy. Specifically, this mechanism covers the entire process from client request arrival, ServiceIP resolution, Pod selection to packet forwarding, including key aspects such as network rule generation, connection state maintenance, and traffic scheduling. However, existing Kubernetes Service implementations directly use static rules for traffic distribution and do not integrate proactive detection capabilities for backend Pod health status at the kernel level. This may lead to faulty Pods continuously receiving requests, causing connection timeouts, increased RST packets, or misjudgments due to network fluctuations, thus affecting service availability and system stability. Furthermore, although Kubernetes health check probes can detect Pod status, they are initiated from the control plane, have a long detection cycle, and cannot block traffic in real time at the data plane, making it difficult to meet the high real-time and high reliability requirements of business applications. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] Therefore, the first objective of this invention is to propose a proactive fault detection method for the backend of a K8s Service based on eBPF.

[0006] The second objective of this invention is to propose an active fault detection device for the backend of a K8s Service based on eBPF.

[0007] The third objective of this invention is to provide an electronic device.

[0008] The fourth objective of this invention is to provide a computer-readable storage medium.

[0009] The fifth objective of this invention is to provide a computer program product.

[0010] The present invention aims to at least partially solve one of the technical problems in the related art.

[0011] Therefore, the first objective of this invention is to propose a proactive fault detection method for the backend of a K8s Service based on eBPF.

[0012] The second objective of this invention is to propose an active fault detection device for the backend of a K8s Service based on eBPF.

[0013] The third objective of this invention is to provide an electronic device.

[0014] The fourth objective of this invention is to provide a computer-readable storage medium.

[0015] To achieve the above objectives, the first aspect of this invention proposes a method for proactively detecting backend faults in a K8s Service based on eBPF, comprising: S1, loading an eBPF program into the kernel of a K8s cluster node and attaching it to a critical position in the network protocol stack to monitor network packets sent to backend Pods in real time; S2, maintaining a forwarding mapping relationship between the Service and the backend Pods in the kernel based on the eBPF program, and counting the number of failures per unit time when a connection failure event is detected; S3, when the number of failures per unit time exceeds a preset threshold, determining that the corresponding backend Pod is in a fault state, and marking the fault flag bit of the Pod in the eBPF Map; S4, dynamically adjusting the forwarding logic according to the fault flag bit in the eBPF Map to prevent traffic from being forwarded to the faulty Pod, and automatically sending probe packets to verify the Pod status after the fault duration expires. If the verification is successful, the fault flag is cleared and forwarding is restored.

[0016] In one embodiment of the present invention, the step of loading the eBPF program into the kernel of the K8s cluster node and attaching it to a key position in the network protocol stack to monitor network packets sent to the backend Pod in real time further includes: S11, attaching the eBPF program to the XDP hook point to perform preliminary filtering and connection state detection before the data packets enter the network protocol stack; S12, attaching the eBPF program to the TC entry or exit hook point to perform connection state analysis and forwarding decisions when the data packets enter or leave the network interface.

[0017] In one embodiment of the present invention, the step of maintaining the forwarding mapping relationship between the Service and the backend Pod in the kernel based on the eBPF program, and counting the number of failures per unit time when a connection failure event is detected, further includes: S21, using an eBPF Map of type hash table, where the key is the five-tuple information of the Service (source IP, source port, destination IP, destination port, protocol type), and the value is the IP and port information of the backend Pod; S22, when a connection failure event is detected by the eBPF program, the failure event is recorded in the counter field of the eBPF Map, and it is determined whether it is within the preset fail_timeout period based on the current timestamp.

[0018] In one embodiment of the present invention, the step of determining the corresponding backend Pod as faulty when the number of failures per unit time exceeds a preset threshold and marking the fault flag of the Pod in the eBPF Map further includes: S31, when the number of failures per unit time reaches or exceeds a preset max_fails threshold, the eBPF program sets the fault flag of the Pod to 1 in the eBPF Map and records the timestamp of the failure; S32, based on the fault flag and timestamp information, the eBPF program skips the forwarding path of the Pod in subsequent forwarding processes until the fault duration expires.

[0019] In one embodiment of the present invention, the method further includes: S5, after the fault duration expires, the eBPF program automatically sends a TCP SYN probe packet to the Pod marked as faulty, and determines whether it has recovered based on whether a SYN-ACK response is received. If a SYN-ACK response is received three times consecutively, the fault flag is cleared and its forwarding path is restored.

[0020] To achieve the above objectives, a second aspect of this invention proposes an active fault detection device for a K8s Service backend based on eBPF, comprising: an eBPF program loading and mounting module, used to load the eBPF program into the kernel of the K8s cluster node and mount it to a key position in the network protocol stack to monitor network packets sent to the backend Pod in real time; a forwarding mapping and failure statistics module, used to maintain the forwarding mapping relationship between the Service and the backend Pod in the kernel based on the eBPF program, and to count the number of failures per unit time when a connection failure event is detected; a fault determination and flag marking module, used to determine that the corresponding backend Pod is in a fault state when the number of failures per unit time exceeds a preset threshold, and to mark the fault flag bit of the Pod in the eBPF Map; and a forwarding logic dynamic adjustment module, used to dynamically adjust the forwarding logic according to the fault flag bit in the eBPF Map, to prevent traffic from being forwarded to the faulty Pod, and to automatically send probe data packets to verify the Pod status after the fault duration expires. If the verification is successful, the fault flag is cleared and forwarding is restored.

[0021] In one embodiment of the present invention, the eBPF program loading and mounting module is further configured to: mount the eBPF program to an XDP hook point to perform preliminary filtering and connection status detection before data packets enter the network protocol stack; and mount the eBPF program to a TC ingress or egress hook point to perform connection status analysis and forwarding decisions when data packets enter or leave the network interface.

[0022] To achieve the above objectives, a third aspect of the present invention provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0023] The memory stores computer-executed instructions;

[0024] The processor executes computer execution instructions stored in the memory to implement the method as described in any one of the first aspects.

[0025] To achieve the above objectives, a fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of the first aspects.

[0026] To achieve the above objectives, a fifth aspect of the present invention provides a computer program product that, when executed by a processor, implements the method described in any one of the first aspects.

[0027] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects: it can monitor the connection status of the K8s Service backend Pod in real time in the kernel space, block the traffic forwarding of the faulty Pod in a timely manner, and significantly improve the availability and stability of the service.

[0028] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0029] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of an embodiment of the proactive fault detection method for K8s Service backend based on eBPF according to the present invention;

[0031] Figure 2 This is a schematic diagram of the structure of an active fault detection device for the backend of a K8s Service based on eBPF according to an embodiment of the present invention. Detailed Implementation

[0032] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0033] Figure 1 This is a flowchart of an embodiment of the present invention: a method for proactively detecting backend faults in a K8s Service based on eBPF.

[0034] like Figure 1 As shown, the proactive fault detection method for the Kubernetes Service backend based on eBPF includes the following steps:

[0035] S1 loads the eBPF program into the kernel of the K8s cluster node and mounts it to a key position in the network protocol stack to monitor network packets sent to the backend Pod in real time.

[0036] Specifically, S1 includes:

[0037] S11, mount the eBPF program to the XDP hook point to perform preliminary filtering and connection state detection before packets enter the network protocol stack.

[0038] Specifically, mounting the eBPF program to the XDP hook point to perform preliminary filtering and connection status detection before data packets enter the network protocol stack is one of the key technical steps in this invention for proactively detecting backend Pod faults. This step, by inserting the eBPF program into the network data path of the Linux kernel, enables real-time processing and status analysis of network data packets, thereby completing preliminary fault identification and flow control before the data packets enter the user-space protocol stack.

[0039] At the technical implementation level, the XDP hook point is located at the very front of the network device driver, typically triggered after the driver receives a data packet but before it enters the kernel protocol stack processing flow. The eBPF program is loaded using the xdp type program type and extracts the five-tuple information of the data packet, including source IP, destination IP, source port, destination port, and protocol type, through protocol header parsing functions such as ip or eth. After the program is mounted, the data packet can be classified and processed, such as dropped, redirected, or continued to be transmitted. In this invention, the eBPF program executes preliminary filtering logic at the XDP hook point to determine whether the target Pod is in a fault state. If it is in a fault state, the data packet is directly dropped or redirected to prevent it from entering the kernel protocol stack, thereby reducing kernel processing overhead and improving response speed.

[0040] At the parameter level, eBPF applications make decisions based on preset fault determination rules. For example, the maximum number of failures (max_fails) can be set to 3, the fault determination window (fail_timeout) to 5 seconds, and the fault duration (down_time) to 30 seconds. When the number of connection failures (such as not receiving ACKs, RST packets, connection timeouts, etc.) of a Pod reaches the max_fails threshold within the fail_timeout period, the eBPF application marks the Pod as faulty and will not forward any requests to that Pod for the down_time period. These parameters can be dynamically updated through user-space applications to adapt to network load and fault tolerance requirements under different business scenarios.

[0041] At the application level, this step is suitable for service load balancing architectures based on eBPF in Kubernetes clusters, especially in high-concurrency, low-latency network environments. By filtering at the XDP layer, the processing burden on the kernel protocol stack can be effectively reduced, improving overall forwarding efficiency. Simultaneously, this mechanism can work in conjunction with eBPF Maps to achieve dynamic removal and recovery detection of faulty Pods, ensuring that traffic is always directed to healthy backend instances.

[0042] The technical advantage of this step lies in significantly reducing fault response latency and improving the system's real-time performance and stability by filtering and performing state checks on data packets before they enter the protocol stack. Compared to traditional forwarding mechanisms based on iptables / ipvs, this solution offers higher performance and lower resource consumption, while avoiding resource waste and service avalanche effects caused by faulty Pods continuously receiving requests, thus providing K8sService with more efficient and reliable fault handling capabilities.

[0043] S12, mount the eBPF program to the TC (Traffic Control) ingress or egress hook point to perform connection state analysis and forwarding decisions when packets enter or leave the network interface.

[0044] Specifically, attaching the eBPF program to the TC (Traffic Control) ingress or egress hook point is one of the core technical steps of this invention to achieve proactive detection and forwarding control of backend Pod faults. This step, by deploying the eBPF program in the Linux kernel's traffic control subsystem, enables real-time analysis of the connection status when data packets enter or leave the network interface, and makes forwarding decisions based on preset policies, thereby achieving kernel-level fault isolation and traffic scheduling.

[0045] At the technical implementation level: In the Linux kernel, the TC hook point is located between the transport and network layers of the network protocol stack, supporting the mounting of eBPF programs in the ingress or egress direction of data packets. This invention mounts the eBPF program to the clsact (Classify and Act) hook point using the tc command. This hook point allows for packet classification and processing without relying on traditional QoS mechanisms. After mounting, the eBPF program can intercept data packets passing through the TC hook point in real time, parse its five-tuple (source IP, source port, destination IP, destination port, protocol type), and, combined with the mapping relationship between Services and backend Pods maintained in the eBPF Map, determine whether the target Pod is in a normal state. If the target Pod is marked as faulty, the eBPF program can directly drop the data packet or redirect it to other healthy Pods, achieving kernel-level fault isolation.

[0046] At the parameter level: During implementation, eBPF programs need to configure key parameters such as `max_fails` (maximum number of failures) and `fail_timeout` (failure timeout) to determine whether a backend Pod has failed. For example, `fail_timeout = 5s` and `max_fails = 3` can be set, meaning that if a Pod fails 3 times consecutively within 5 seconds, a failure flag is triggered. Additionally, the failure duration (`down_time`) can be set to 30 seconds to ensure that no traffic is received before the Pod recovers. The eBPF Map type is typically `BPF_MAP_TYPE_HASH`, with a 5-tuple as the key and the Pod's IP and port information as the value, supporting fast lookup and updates.

[0047] Application scenario: This step applies to Kubernetes clusters using CNI network plugins (such as Cilium) implemented with eBPF. In actual deployments, the eBPF program is attached to the TC hook point of the network interface (such as eth0) of the cluster node to process traffic entering and leaving the node in real time. When a Pod fails to connect due to application crashes, network interruptions, or resource exhaustion, the eBPF program can immediately identify and block subsequent traffic, preventing request backlog and fault propagation, and improving service availability and response efficiency.

[0048] In terms of technical effectiveness: By attaching the eBPF program to the TC hook point, this invention achieves proactive detection and real-time forwarding control of the backend Pod connection status in kernel mode. Compared with the traditional forwarding mechanism based on iptables / ipvs, this significantly reduces the latency of fault detection and response. Simultaneously, this approach avoids frequent context switching between user mode and kernel mode, improving overall processing performance and providing K8sService with more efficient and stable fault isolation capabilities.

[0049] S2, based on eBPF, maintains the forwarding mapping relationship between the Service and the backend Pod in the kernel, and counts the number of failures per unit time when a connection failure event is detected.

[0050] Specifically, this step involves dynamically maintaining the forwarding mapping relationship between Kubernetes (K8s) Services and backend Pods in the Linux kernel based on eBPF programs, and counting the number of failures per unit time when connection failure events are detected, in order to achieve real-time fault detection and response for backend Pods. This step is the core component of the entire technical solution for achieving proactive fault detection and rapid isolation.

[0051] Furthermore, S2 includes:

[0052] S21 uses a hash table-type eBPF Map, where the keys are the five-tuple information of the Service (source IP, source port, destination IP, destination port, protocol type), and the values ​​are the IP and port information of the backend Pod.

[0053] Specifically, this step involves building a hash table-type eBPF Map within the eBPF program to store the mapping relationship between the Kubernetes Service's five-tuple information (source IP, source port, destination IP, destination port, and protocol type) and the IP and port of the backend Pods. This Map serves as the core data structure for sharing state between the eBPF program and user-space control logic, and its design directly impacts the system's forwarding efficiency and fault handling capabilities in kernel space.

[0054] At the technical implementation level, this eBPF Map uses the BPF_MAP_TYPE_HASH type supported by the Linux kernel and is created in the kernel via the bpf_map_create system call. The key structure is a five-tuple, specifically including the source IP address (__be32 type), the source port (__be16 type), the destination IP address (__be32 type), the destination port (__be16 type), and the protocol type (__u8 type, such as TCP or UDP). The value structure is the backend Pod's IP address (__be32) and port number (__be16), used to guide the eBPF program in rewriting the destination address in the data plane. The size of this Map can be dynamically configured according to the cluster size, for example, set to max_entries = 10240 to support concurrent processing of large-scale services.

[0055] At the parameter level, the performance of the eBPF Map is closely related to memory usage. Key hash calculations must employ efficient hash algorithms (such as Jenkins Hash) to ensure fast lookups. Meanwhile, Map update operations are performed by the user-space controller through the `bpf_map_update_elem` interface, supporting atomic updates and concurrent access. To improve performance, `map_flags` can be set to `BPF_F_NO_PREALLOC` to avoid the overhead of pre-allocating memory. Furthermore, to prevent hash collisions, `key_size` and `value_size` can be set to fixed sizes to ensure the uniqueness and consistency of key-value pairs.

[0056] At the application level, this eBPF Map is mounted on the TC (Traffic Control) hook point or XDP (eXpress Data Path) entry point of the kernel network protocol stack to process network packets entering the Service in real time. When a packet arrives, the eBPF program extracts its five-tuple information, uses it as the key to query the Map, obtains the corresponding backend Pod address, and rewrites the destination address in kernel space, achieving seamless integration of load balancing and fault isolation.

[0057] The technical value of this step lies in the fact that, through the efficient hash table structure in kernel space, the mapping relationship between Service and Pod is maintained and queried in real time, providing basic data support for subsequent fault detection and forwarding control, and significantly improving the availability and stability of the system in high-concurrency, low-latency scenarios.

[0058] S22, when a connection failure event is detected by the eBPF program, the failure event is recorded in the counter field of the eBPF Map, and it is determined whether it is within the preset fail_timeout period based on the current timestamp.

[0059] Specifically, this step involves the eBPF program recording the connection failure event in a counter field of the eBPF Map when a connection failure event is detected, and determining whether it is within the preset fail_timeout period based on the current timestamp. This step is one of the core mechanisms for proactive detection and rapid response to backend Pod failures in this invention.

[0060] At the technical implementation level: In the Linux kernel, eBPF programs capture network connection events in real time through hooks attached to the network protocol stack, such as TC (TrafficControl) or XDP (eXpress Data Path). When an eBPF program detects a TCP connection failure event (such as no ACK confirmation received, connection establishment timeout, or receipt of an RST packet), it triggers the counter update logic. The eBPF Map, as a data structure shared between the kernel and user space, typically uses a hash table (BPF_MAP_TYPE_HASH) or an array (BPF_MAP_TYPE_ARRAY) to store the connection failure counter for each backend Pod. The eBPF program manipulates the counter fields in the Map using kernel APIs such as bpf_map_lookup_elem() and bpf_map_update_elem() to accumulate and store the failure count.

[0061] At the parameter level: Key parameters involved in this step include `fail_timeout` and `max_fails`. `fail_timeout` defines the time window for fault determination, typically set to 5 seconds (5000 milliseconds), while `max_fails` represents the maximum number of failures allowed within this time window, typically set to 3. Each time a failure event is detected, the eBPF program compares the current timestamp with the last failure time recorded in the Map. If the time difference is less than `fail_timeout`, the counter is updated; otherwise, it is reset. This mechanism conforms to the logic for handling HTTP connection failures in RFC 7230 and also references the health check parameter configurations recommended by the Kubernetes official documentation.

[0062] Application scenario: This step is applicable to service load balancing scenarios based on eBPF in Kubernetes clusters. When client requests are forwarded to backend Pods via eBPF, if consecutive connection failures occur, the system will quickly identify and isolate the faulty Pods, preventing them from receiving further requests. This mechanism is particularly suitable for microservice architectures with high concurrency and low latency requirements, such as financial trading systems and real-time communication services, effectively improving service availability and fault tolerance.

[0063] From a technical perspective: By recording connection failure events in real time in kernel mode and combining this with time window judgment, this step enables rapid identification and response to backend Pod failures, significantly reducing fault detection latency. Compared to traditional control plane-based health check mechanisms, this solution offers higher real-time performance and lower resource consumption, thereby improving the stability of K8sService and the overall fault tolerance of the system.

[0064] S3: When the number of failures per unit time exceeds a preset threshold, the corresponding backend Pod is determined to be in a fault state, and the fault flag of the Pod is marked in the eBPF Map.

[0065] Specifically, when the number of failures per unit time exceeds a preset threshold, the corresponding backend Pod is determined to be in a fault state, and the fault flag of the Pod is marked in the eBPF Map. This is one of the core steps in this invention to achieve proactive detection and isolation of backend Pod faults. This step achieves rapid fault identification and response for backend Pods by running an eBPF program in kernel space to collect and analyze network connection status in real time.

[0066] Furthermore, S3 includes:

[0067] S31, when the number of failures per unit time reaches or exceeds the preset max_fails threshold, the eBPF program sets the fault flag of the Pod to 1 in the eBPF Map and records the timestamp of the failure.

[0068] Specifically, when the number of failures per unit time reaches or exceeds the preset `max_fails` threshold, the eBPF program sets the fault flag of the Pod to 1 in the eBPF Map and records the timestamp of the failure. This step is one of the core mechanisms for proactive detection and isolation of backend Pod failures in this invention. At the technical implementation level, this step involves embedding status statistics logic into the eBPF program to monitor connection failure events of each backend Pod in real time. Specifically, when processing network packets, the eBPF program triggers a counter update operation based on the connection status (such as TCP connection establishment failure, RST packet reception, ACK timeout, etc.). The counter uses a sliding time window mechanism with a `fail_timeout` time window length (e.g., 5 seconds) to count the number of failures of the Pod within the window. When the counter value reaches or exceeds `max_fails` (e.g., 3 times), the program sets the status flag of the Pod in the eBPF Map to 1, indicating that it is in a fault state, and updates the corresponding entry in the Map through the `bpf_map_update_elem` interface, while recording the current timestamp for subsequent fault duration determination.

[0069] At the parameter level, `max_fails` and `fail_timeout` are two key configuration parameters, and their settings need to be balanced based on the fault tolerance of the business and network stability. `max_fails` is typically set to 3-5 failures, while `fail_timeout` is recommended to be 5-10 seconds to balance fault detection sensitivity and false positive rate. Furthermore, the update of the fault flag must follow the atomic operation specifications of eBPFMap to ensure data consistency during concurrent access. Timestamp recording typically uses the `bpf_ktime_get_ns()` function provided by the Linux kernel to obtain high-precision timestamps in nanoseconds to support accurate fault duration calculations.

[0070] At the application level, this step is suitable for microservice scenarios with high availability requirements in large-scale Kubernetes clusters. For example, in systems sensitive to service response time, such as finance and e-commerce, real-time fault detection in the kernel space can effectively reduce business interruptions caused by faulty Pods continuously receiving requests. eBPF programs run in kernel space without user-space intervention, thus possessing low latency and high throughput characteristics, making them suitable for high-concurrency, low-latency network processing scenarios.

[0071] The technical advantage of this step lies in its significant improvement in fault response speed by implementing fault determination logic within the kernel. Compared to the second-level latency of traditional Kubernetes health check probes (such as liveness probes), this solution can reduce fault isolation time to milliseconds. Simultaneously, by setting `max_fails` and `fail_timeout`, false positives caused by brief network fluctuations can be effectively filtered out, improving the accuracy of fault determination. Furthermore, the setting of fault flags provides data for subsequent traffic blocking and recovery detection, making it a crucial step in implementing a dynamic isolation and recovery mechanism for backend Pods.

[0072] S32, based on the fault flag and timestamp information, the eBPF program skips the forwarding path of the Pod during subsequent forwarding processes until the fault duration expires.

[0073] Specifically, in some implementations, based on fault flags and timestamp information, the eBPF program skips the forwarding path of the Pod during subsequent forwarding processes until the fault duration expires. This is one of the core mechanisms for fault isolation and automatic recovery in this invention. This step achieves real-time flow control in kernel mode by dynamically maintaining the health status information of backend Pods in the eBPF Map and combining timestamps and status flags.

[0074] At the technical implementation level, when an eBPF program detects consecutive connection failures in a backend Pod (such as not receiving an ACK, connection timeout, or receiving an RST packet), it sets a fault flag (fault_flag) in the corresponding eBPF Map entry and records the timestamp of the failure (fault_time). This Map entry typically uses the Service's 5-tuple (source IP, source port, destination IP, destination port, protocol type) as the key, the backend Pod's IP and port as the value, and appends a status field to store health information. When subsequent traffic arrives, the eBPF program first queries this Map. If it finds that the target Pod's fault_flag is 1 and the time difference between the current time and fault_time is less than the preset fault duration (fault_duration), it directly skips the Pod's forwarding path, redirects the request to other available backends, or discards it, thus achieving fault isolation.

[0075] At the parameter level, the fault duration (fault_duration) is typically set to 30 seconds, but this parameter can be dynamically configured according to business needs. The fault flag (fault_flag) is a boolean field used to indicate whether the Pod is in a faulty state. The timestamp (fault_time) uses a system timestamp (such as nanosecond-level time obtained by the Linux kmet_get_ns() function) to ensure the accuracy of time calculation. In addition, fault determination also relies on parameters such as fail_timeout (e.g., 5 seconds) and max_fails (e.g., 3 times) to control the sensitivity and stability of fault determination.

[0076] In terms of application scenarios, this mechanism is suitable for service load balancing scenarios in large-scale Kubernetes clusters, especially in systems with high service availability requirements such as finance, telecommunications, and cloud-native technologies. Through real-time kernel-level control, it avoids the latency issues of traditional health check probes, thus improving fault response speed.

[0077] From a technical perspective, this step effectively prevents faulty Pods from continuously receiving traffic, avoiding resource waste and fault propagation. Simultaneously, the timestamp mechanism ensures the timeliness of fault isolation, preventing recovery delays caused by permanent removal. Combined with an automatic recovery detection mechanism, the system can reassess the Pod status after the fault duration expires, achieving dynamic recovery and significantly improving the stability and fault tolerance of Kubernetes Services.

[0078] S4 dynamically adjusts the forwarding logic based on the fault flag in the eBPF Map to prevent traffic from being forwarded to the faulty Pod. After the fault duration expires, it automatically sends probe packets to verify the Pod status. If the verification is successful, the fault flag is cleared and forwarding is resumed.

[0079] Specifically, this step involves using eBPF technology in the kernel-level forwarding logic of a Kubernetes (K8s) Service to dynamically maintain the health status of backend Pods through an eBPF Map. When a fault is detected, traffic forwarding is blocked in real time. Simultaneously, after the fault duration expires, probe packets are automatically sent to verify the Pod status. If verification is successful, the fault flag is cleared and forwarding resumes. This step is one of the core mechanisms for proactively detecting and quickly recovering from backend Pod faults in a K8s Service.

[0080] At the technical implementation level, eBPF programs intercept and process network packets in real time by attaching to key points in the Linux kernel's network protocol stack (such as TC (Traffic Control) hooks or XDP (eXpress Data Path) entry points). When a Pod is detected to have experienced consecutive connection failure events (such as not receiving an ACK confirmation, connection establishment timeout, receiving an RST packet, etc.) within a unit of time (fail_timeout), the eBPF program sets a fault flag (e.g., the is_down field is set to 1) for the corresponding entry in the eBPF Map for that Pod. The eBPF Map typically uses a hash table structure, with the key being a 5-tuple of the Service (source IP, source port, destination IP, destination port, protocol type), and the values ​​being the IP and port information and status flags of the backend Pod. When subsequent traffic arrives, the eBPF program queries the Map; if it finds that the target Pod is in a faulty state, it directly discards or redirects the packet to prevent it from being forwarded to the faulty Pod.

[0081] At the parameter level, fault determination relies on two key parameters: `max_fails` (maximum number of failures) and `fail_timeout` (failure time window). For example, `max_fails = 3` and `fail_timeout = 5s` can be set, meaning that if a Pod fails 3 times consecutively within 5 seconds, it is considered a fault. The fault duration (`down_duration`) can be set to 30 seconds to control the duration of isolation for a faulty Pod after it is marked. After the fault duration expires, the eBPF program will automatically send probe packets (such as TCP SYN packets) to the Pod. If a SYN-ACK response is received within the preset probe timeout (such as `probe_timeout = 2s`), the Pod is considered to have recovered, its fault flag is cleared, and the status information in the eBPF Map is updated, restoring its availability in load balancing.

[0082] At the application level, this step is suitable for microservice architectures with high availability requirements in large-scale Kubernetes clusters. For example, in scenarios with extremely high service continuity requirements, such as finance and telecommunications, real-time fault isolation and recovery mechanisms in the kernel space can significantly reduce service interruption time and improve the overall fault tolerance of the system. Furthermore, this mechanism can be integrated with CNI plugins based on eBPF, such as Cilium, to achieve efficient network policy control without user-space intervention.

[0083] The technical advantage of this step lies in its ability to effectively shorten the response latency of traditional health check probes through real-time fault detection and isolation mechanisms in the kernel space, thereby improving the real-time performance and accuracy of fault handling. Simultaneously, by dynamically adjusting the forwarding logic, it avoids resource waste and fault propagation caused by faulty Pods continuing to receive traffic, thus enhancing the stability and availability of the Kubernetes Service.

[0084] S5, after the fault duration expires, the eBPF program automatically sends a TCP SYN probe packet to the Pod marked as faulty, and determines whether it has recovered based on whether a SYN-ACK response is received. If a SYN-ACK response is received three times in a row, the fault flag is cleared and its forwarding path is restored.

[0085] Specifically, this step involves a Kubernetes Service backend Pod fault recovery detection mechanism based on eBPF. Its technical implementation principle is based on a kernel-level proactive probing and status confirmation mechanism. After the fault duration expires, the eBPF program automatically sends TCP SYN probe packets to the Pod marked as faulty to verify whether its service availability has been restored. This process is implemented through the kernel-level eBPF program, without user-level intervention, thereby significantly reducing probing latency and improving system response efficiency.

[0086] At the technical implementation level, the eBPF program constructs and sends TCP SYN packets to the target Pod's IP address and port by calling kernel APIs such as bpf_send_skb() or bpf_clone_redirect(). The sending frequency, timeout, and number of retries of the SYN packets can all be dynamically adjusted through configuration parameters in the eBPF Map. For example, the probe interval can be set to 5 seconds, the timeout to 1 second, and the maximum number of retries to 3. When the eBPF program receives a SYN-ACK response from the target Pod, it indicates that the Pod has responsiveness and may have recovered. To ensure the accuracy of the recovery judgment, this invention adopts a mechanism of three consecutive successful SYN-ACK receptions; that is, only after receiving valid responses in three consecutive probes is the Pod confirmed to have recovered normally, and its status is updated through the eBPF Map, clearing the fault flag.

[0087] Regarding parameter metrics, this step involves several key parameters, including the probe packet sending interval (probe_interval), the SYN-ACK response timeout (ack_timeout), and the threshold for consecutive successful responses (recovery_threshold). These parameters can be configured based on service load, network environment, and service SLA requirements. For example, in a high-availability scenario, recovery_threshold can be set to 3 times, probe_interval to 5 seconds, and ack_timeout to 1 second to balance probe sensitivity with system overhead.

[0088] In application scenarios, this step is suitable for Service load balancing architectures based on eBPF in Kubernetes clusters, especially for production environments with high service availability requirements. For example, in a microservice architecture, when a Pod is marked as faulty due to temporary network jitter or resource contention, this mechanism can quickly restore its forwarding path after the fault is resolved, avoiding service interruption or request loss.

[0089] The technical advantage of this step lies in its ability to automatically assess the recovery of faulty Pods through a kernel-level proactive detection mechanism, effectively reducing resource waste and unnecessary service degradation caused by misjudgments. Simultaneously, this mechanism is tightly integrated with the eBPF Map to ensure the real-time nature and consistency of state updates, thereby improving the availability and stability of the entire Kubernetes Service.

[0090] The eBPF-based K8s Service backend fault proactive detection method of this invention achieves automatic recovery identification and dynamic reconstruction of the forwarding path for faulty backends by having the eBPF program actively send TCP SYN probe packets after the fault duration expires and judging the Pod recovery status based on the response, thereby further improving the system's self-healing capability and service availability.

[0091] To achieve the above embodiments, the present invention also proposes an active fault detection device for the backend of a K8s Service based on eBPF. Figure 2 This is a schematic diagram of a proactive fault detection device for a Kubernetes Service backend based on eBPF, provided as an embodiment of the present invention. Figure 2 As shown, the device includes:

[0092] The eBPF program loading and mounting module 100 is used to load the eBPF program into the kernel of the K8s cluster node and mount it to a key position in the network protocol stack to monitor network packets sent to the backend Pod in real time.

[0093] The forwarding mapping and failure statistics module 200 is used to maintain the forwarding mapping relationship between the Service and the backend Pod in the kernel based on the eBPF program, and to count the number of failures per unit time when a connection failure event is detected.

[0094] The fault determination and flag marking module 300 is used to determine that the corresponding backend Pod is in a fault state when the number of failures exceeds a preset threshold within a unit of time, and to mark the fault flag bit of the Pod in the eBPF Map.

[0095] The forwarding logic dynamic adjustment module 400 is used to dynamically adjust the forwarding logic according to the fault flag bit in the eBPF Map, prevent traffic from being forwarded to the faulty Pod, and automatically send probe data packets to verify the Pod status after the fault duration expires. If the verification is successful, the fault flag is cleared and forwarding is resumed.

[0096] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0097] To implement the above embodiments, the present invention also proposes an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0098] To implement the above embodiments, the present invention also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.

[0099] To implement the above embodiments, the present invention also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.

[0100] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0101] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0102] This invention is intended to provide implementation schemes for users to selectively prevent the use or access to personal information data. That is, this disclosure is intended to provide hardware and / or software to prevent or block access to such personal information data. Once personal information data is no longer needed, risks can be minimized by restricting data collection and deleting data. Furthermore, where applicable, such personal information can be de-identified to protect user privacy.

[0103] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0104] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0105] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of the invention pertain.

[0106] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0107] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0108] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0109] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0110] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0111] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0112] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A proactive fault detection method for Kubernetes Service backends based on eBPF, characterized in that, include: S1 loads the eBPF program into the kernel of the K8s cluster node and mounts it to a key position in the network protocol stack to monitor network packets sent to the backend Pod in real time. S2, based on eBPF, maintains the forwarding mapping relationship between the Service and the backend Pod in the kernel, and counts the number of failures per unit time when a connection failure event is detected; S3, when the number of failures per unit time exceeds the preset threshold, the corresponding backend Pod is determined to be in a fault state, and the fault flag bit of the Pod is marked in the eBPF Map; S4 dynamically adjusts the forwarding logic based on the fault flag in the eBPF Map to prevent traffic from being forwarded to the faulty Pod. After the fault duration expires, it automatically sends probe packets to verify the Pod status. If the verification is successful, the fault flag is cleared and forwarding is resumed.

2. The method as described in claim 1, characterized in that, The process of loading the eBPF program into the kernel of the K8s cluster node and attaching it to a critical location in the network protocol stack to monitor network packets sent to the backend Pod in real time also includes: S11, mount the eBPF program to the XDP hook point to perform preliminary filtering and connection state detection before packets enter the network protocol stack; S12, mount the eBPF program to the TC ingress or egress hook point to perform connection state analysis and forwarding decisions when packets enter or leave the network interface.

3. The method as described in claim 1, characterized in that, The eBPF-based program maintains the forwarding mapping relationship between the Service and the backend Pod in the kernel, and counts the number of failures per unit time when a connection failure event is detected. This also includes: S21 uses a hash table-type eBPF Map, where the key is the five-tuple information of the Service, including source IP, source port, destination IP, destination port, and protocol type, and the value is the IP and port information of the backend Pod. S22, when a connection failure event is detected by the eBPF program, the failure event is recorded in the counter field of the eBPFMap, and it is determined whether it is within the preset fail_timeout period based on the current timestamp.

4. The method as described in claim 1, characterized in that, The step of determining that the corresponding backend Pod is in a fault state when the number of failures exceeds a preset threshold within a unit of time, and marking the fault flag of the Pod in the eBPF Map, also includes: S31, When the number of failures per unit time reaches or exceeds the preset max_fails threshold, the eBPF program sets the fault flag of the Pod to 1 in the eBPFMap and records the timestamp of the failure. S32, based on the fault flag and timestamp information, the eBPF program skips the forwarding path of the Pod during subsequent forwarding processes until the fault duration expires.

5. The method as described in claim 1, characterized in that, Also includes: S5, after the fault duration expires, the eBPF program automatically sends a TCP SYN probe packet to the Pod marked as faulty, and determines whether it has recovered based on whether a SYN-ACK response is received. If a SYN-ACK response is received three times in a row, the fault flag is cleared and its forwarding path is restored.

6. A proactive fault detection device for Kubernetes Service backends based on eBPF, characterized in that, include: The eBPF program loading and mounting module is used to load eBPF programs into the kernel of the K8s cluster node and mount them to key positions in the network protocol stack to monitor network packets sent to the backend Pod in real time. The forwarding mapping and failure statistics module is used to maintain the forwarding mapping relationship between the Service and the backend Pod in the kernel based on eBPF programs, and to count the number of failures per unit time when a connection failure event is detected. The fault determination and flag marking module is used to determine that the corresponding backend Pod is in a fault state when the number of failures exceeds a preset threshold within a unit of time, and to mark the fault flag bit of the Pod in the eBPF Map; The forwarding logic dynamic adjustment module is used to dynamically adjust the forwarding logic according to the fault flag in the eBPF Map, prevent traffic from being forwarded to the faulty Pod, and automatically send probe packets to verify the Pod status after the fault duration expires. If the verification is successful, the fault flag is cleared and forwarding is resumed.

7. The apparatus as claimed in claim 6, characterized in that, The eBPF program loading and mounting module is also used for: The eBPF program is attached to the XDP hook point to perform preliminary filtering and connection state detection before packets enter the network protocol stack; The eBPF program is attached to the TC ingress or egress hook point to perform connection state analysis and forwarding decisions as packets enter or leave the network interface.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-5.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-5.

Images