An eBPF-based full-link network fault diagnosis method and system

Abstract

The application provides an eBPF-based full-link network fault diagnosis method and system. The application mounts an eBPF program on a kernel protocol stack key path, assigns a flow identifier to a newly created network flow at a first kernel hook point of packet stacking, writes the flow identifier and a flow survival state according to a bit field segmentation rule into an skb-mark field, realizes full-link transmission of the flow identifier by using an original skb-mark automatic inheritance feature of the kernel, extracts multiple kernel hook point events of the same packet by bit operation after collecting the flow identifier, constructs an analysis context, flips a survival bit when the network flow is terminated, and completes storage slot recovery in the kernel state, and finally completes cross-event correlation diagnosis based on the analysis context. Without modifying the kernel source code, efficient transmission of the flow identifier and autonomous recovery of the flow resource are realized, and correlation and fault diagnosis of packet full-link events are completed.

Description

Technical Field

[0001] This application belongs to the field of network fault diagnosis, and in particular relates to a method and system for diagnosing full-link network faults based on eBPF. Background Technology

[0002] Existing network fault diagnosis solutions based on eBPF (extended Berkeley Packet Filter) require performing network 5-tuple hash calculations and hash mapping queries for each kernel hook point of each data packet, resulting in continuous computational and storage overhead. After the network flow terminates, the storage resources corresponding to the flow identifier need to be periodically scanned and reclaimed by user-space programs, which can easily lead to storage resource leakage and slot exhaustion. At the same time, such solutions cannot uniformly collect events from multiple kernel hook points of the same data packet, making it difficult to form a complete data packet lifecycle view. They cannot achieve automated network fault root cause analysis and cannot simultaneously meet the requirements of low-overhead operation, automatic flow resource reclamation, and full-link event correlation. Summary of the Invention

[0003] To address the challenge of reducing the computational overhead of flow context access while simultaneously achieving kernel-mode autonomous reclamation of network flow resources and completing the full-link association of multiple kernel hook point events for the same data packet during the process of kernel-mode full-link event collection and fault diagnosis based on eBPF.

[0004] This application provides a method and system for end-to-end network fault diagnosis based on eBPF. This application mounts an eBPF program on the critical path of the kernel protocol stack, assigns a flow identifier to the newly created network flow at the first kernel hook point when the data packet enters the stack, writes the flow identifier and flow liveness status into the skb-mark field according to bit field segmentation rules, utilizes the kernel's native skb-mark automatic inheritance feature to achieve end-to-end transparent transmission of the flow identifier, extracts the flow identifier through bit operations, and then aggregates multiple kernel hook point events of the same data packet to construct an analysis context. When the network flow terminates, the liveness bit is flipped and the storage slot is reclaimed in kernel mode. Finally, cross-event correlation diagnosis is completed based on the analysis context. Without modifying the kernel source code, this application achieves efficient transparent transmission of the flow identifier and autonomous reclamation of flow resources, while simultaneously completing the correlation and fault diagnosis of end-to-end events of the data packet.

[0005] This application provides a full-link network fault diagnosis method based on eBPF, including: Mount the eBPF program on the critical path of the kernel protocol stack to assign a flow identifier to the newly created network flow and write it into the skb-mark field of the data packet according to the bit field segmentation rules; Based on the kernel's automatic inheritance feature of the skb-mark field, the flow identifier is extracted to aggregate the multi-kernel hook point events of the same data packet to build an analysis context. When the network flow terminates, the live bit of the skb-mark field is flipped and the storage slot corresponding to the flow identifier is reclaimed in kernel mode. Network fault diagnosis results are obtained by performing cross-event correlation analysis based on the analysis context. The flow identifier is an unsigned integer value used to uniquely identify a single network flow. The bit field segmentation rule is to use the high 31 bits of the skb-mark field to store the flow identifier and the low 1 bit to store the flow liveness status. The flow liveness status is a value used to identify whether the network flow is in an active or reclaimable state. The analysis context is a data set used to collect events generated by the same data packet at multiple kernel hook points.

[0006] Optionally, a flow identifier is assigned to the newly created network flow and written to the skb-mark field of the data packet according to the bit field segmentation rules, including: The system identifies newly created network flows at the first kernel hook point when a data packet is pushed onto the stack; it allocates free storage slots for the newly created network flows and uses the slot index as the flow identifier; it combines the flow identifier and flow liveness status according to the bit field segmentation rules to obtain the target value, and writes the target value into the skb-mark field of the current data packet; the first kernel hook point is the first interception node triggered when the data packet enters the kernel protocol stack, the free storage slot is the unoccupied storage location in the BPF array mapping, and the BPF array mapping is a kernel-mode storage structure that uses contiguous memory addressing and is indexed by integer values.

[0007] Optionally, the newly created network flow can be identified at the first kernel hook point when the data packet is pushed onto the stack, including: Extract the network 5-tuple information of the current data packet; query the flow state mapping table based on the network 5-tuple information; if no matching record is found, determine that the network flow to which the current data packet belongs is a newly created network flow; the network 5-tuple information is a combination of source IP address, destination IP address, source port, destination port and transport layer protocol information, and the flow state mapping table is a kernel-mode mapping structure used to store the correspondence between the allocated flow identifier and the network 5-tuple.

[0008] Optionally, the kernel's automatic inheritance feature based on the skb-mark field is used to extract flow identifiers to aggregate multi-kernel hook point events of the same data packet to build an analysis context, including: The algorithm reads the skb-mark field value inherited by the current data packet and extracts the flow identifier from the skb-mark field value through bitwise operations. It then accesses the BPF array mapping using the flow identifier as an index to obtain the corresponding flow context. Based on the flow context and the data packet identifier, it aggregates the multi-kernel hook point events of the same data packet and adds a lifecycle end identifier to the aggregated event set to obtain the analysis context. Bitwise operations are shift and bitwise logical operations that do not change the original attributes of the values. The flow context is structured data used to store the full lifecycle feature information of a single network flow. The data packet identifier is a kernel-state pointer value used to uniquely distinguish a single data packet. The lifecycle end identifier is a mark used to indicate that the data packet has completed the entire process of kernel protocol stack processing.

[0009] Optionally, the corresponding stream context can be obtained by accessing the BPF array mapping using the stream identifier as an index, including: The target storage location of the BPF array mapping is determined based on the flow identifier; the network quintuple, flow creation time, and transport layer state information are read from the target storage location; the read information is integrated into the flow context; the transport layer state information is the identifier information used to characterize the network flow transmission stage.

[0010] Optionally, when the network flow terminates, the live bits of the skb-mark field are toggled and the storage slot corresponding to the flow identifier is reclaimed in kernel mode, including: Identify the terminating data packet of the network flow; read the value of the skb-mark field of the terminating data packet and flip the lower 1 bit of the value; mark the corresponding BPF array mapping storage slot as idle based on the flipped flow identifier; the terminating data packet is a data packet carrying a transport layer close flag or that has reached the idle timeout condition.

[0011] Optionally, identifying terminating packets of a network flow includes: The transport layer identifier of the current data packet is detected; when the transport layer identifier is a closed identifier, the current data packet is determined to be a terminated data packet; when the duration during which the network flow has not generated data packets reaches the timeout threshold, the corresponding data packet is determined to be a terminated data packet; the timeout threshold is the preset duration of the network flow's idle state.

[0012] Optionally, cross-event correlation analysis can be performed based on the analysis context to obtain network fault diagnosis results, including: It iterates through all event information in the analysis context; matches event information with diagnostic rules; generates structured data of fault type, root cause information and inspection direction based on the matching results; and uses diagnostic rules as kernel-level judgment conditions to determine the type of network fault.

[0013] Optionally, match event information with diagnostic rules, including: Extract the kernel hook point type and feature fields from the event information; compare the feature fields with the judgment conditions of the diagnostic rules; when the feature fields meet the judgment conditions, complete the matching of the corresponding diagnostic rules.

[0014] This application also provides an end-to-end network fault diagnosis system based on eBPF, including: The mounting module mounts the eBPF program on the critical path of the kernel protocol stack, assigns a flow identifier to the newly created network flow and writes it into the skb-mark field of the data packet according to the bit field segmentation rules. The aggregation module extracts the flow identifier based on the kernel's automatic inheritance feature of the skb-mark field to aggregate the multi-kernel hook point events of the same data packet to build an analysis context. When the network flow terminates, it flips the live bit of the skb-mark field and reclaims the storage slot corresponding to the flow identifier in kernel mode. The analysis module performs cross-event correlation analysis based on the analysis context to obtain network fault diagnosis results. The flow identifier is an unsigned integer value used to uniquely identify a single network flow. The bit field segmentation rule is to use the high 31 bits of the skb-mark field to store the flow identifier and the low 1 bit to store the flow liveness status. The flow liveness status is a value used to identify whether the network flow is in an active or reclaimable state. The analysis context is a data set used to collect events generated by the same data packet at multiple kernel hook points.

[0015] As can be seen from the above technical solution, this application collects packet event information through the eBPF program mounted on the critical path of the kernel protocol stack. The allocation of the flow identifier and the writing of the bit field for the newly created network flow are only performed once in the first packet of the flow. Subsequent packets directly carry the flow identifier by relying on the kernel automatic inheritance feature of skb-mark. Bitwise operations to extract the flow identifier do not require hash calculation, reducing the computational overhead of flow context access. The flow liveness bit flipping can identify the flow state. Kernel-mode storage slot reclamation does not require user-mode participation, avoiding storage resource leakage. The analysis context formed by the aggregation of multiple kernel hook point events of the same packet provides complete packet lifecycle data. The various technical features work together in sequence, so that the diagnostic process does not require repeated hash lookups, flow resources can be reused, and packet full-link events can be completely correlated, realizing low-overhead and resource-leakage-free full-link network fault automated diagnosis. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1A schematic diagram of a full-link network fault diagnosis method based on eBPF is shown in an embodiment of this application; Figure 2 This illustration shows one of the sub-steps of the eBPF-based end-link network fault diagnosis method in an embodiment of this application; Figure 3 This illustration shows the second of the sub-step diagrams of the eBPF-based end-link network fault diagnosis method in an embodiment of this application; Figure 4 A schematic diagram of an end-to-end network fault diagnosis system based on eBPF is shown in an embodiment of this application. Detailed Implementation

[0018] Current network fault diagnosis methods typically rely on eBPF programs to collect network events in kernel space. Existing technologies deploy independent eBPF programs at multiple kernel protocol stack kernel hook points. When each eBPF program is triggered, it extracts the network 5-tuple information of the current data packet, calculates the hash value corresponding to the 5-tuple using a hash algorithm, and then uses the hash value as an index to query the kernel space hash mapping structure to obtain the corresponding flow context information. This operation is repeated every time a data packet passes through each kernel hook point. After the network flow is completed, relevant personnel need to periodically scan the hash mapping structure using user-space programs to identify and delete flow records that have not been updated for a long time, thereby freeing up storage resources. Moreover, this approach can only collect event information from a single kernel hook point and cannot aggregate events generated by the same data packet at different kernel hook points.

[0019] It should be noted that the implementation methods of the above-mentioned existing technologies will bring about a number of operational problems. Repeated hash calculations and hash mapping queries will consume kernel-mode computing resources. In high-concurrency network traffic scenarios, such overhead will affect the normal processing efficiency of the kernel protocol stack. The user-mode timed scanning recycling method cannot accurately match the termination time of the network flow, which may result in storage resources not being released in time or being released prematurely. Long-term operation will lead to the continuous occupation of kernel-mode storage slots, eventually resulting in the exhaustion of storage resources. Furthermore, the event collection method of a single kernel hook point cannot restore the complete life cycle of the data packet, and the results of fault diagnosis cannot reflect the abnormal nodes of the data packet in the entire kernel protocol stack processing flow.

[0020] Based on this, embodiments of this application provide a method for diagnosing end-to-end network faults based on eBPF, such as... Figure 1 As shown, it includes: S1: Mount the eBPF program on the critical path of the kernel protocol stack, assign a flow identifier to the new network flow and write it into the skb-mark field of the data packet according to the bit field segmentation rules; S2: Based on the kernel's automatic inheritance feature of the skb-mark field, the flow identifier is extracted to build an analysis context by aggregating the multi-kernel hook point events of the same data packet. When the network flow terminates, the live bit of the skb-mark field is flipped and the storage slot corresponding to the flow identifier is reclaimed in kernel mode. S3: Perform cross-event correlation analysis based on the analysis context to obtain network fault diagnosis results.

[0021] Specifically, the flow identifier is an unsigned integer value used to uniquely identify a single network flow. The bit field segmentation rule is to use the high 31 bits of the skb-mark field to store the flow identifier and the low 1 bit to store the flow liveness status. The flow liveness status is a value used to identify whether the network flow is in an active or reclaimable state. The analysis context is a data set used to aggregate the events generated by the same data packet at multiple kernel hook points.

[0022] This application collects packet event information through an eBPF program mounted on the critical path of the kernel protocol stack. The allocation of the flow identifier and the writing of bit fields for a newly created network flow are performed only once in the first packet of the flow. Subsequent packets directly carry the flow identifier by relying on the kernel's automatic inheritance feature of skb-mark. Bitwise operations are used to extract the flow identifier without hash calculations, reducing the computational overhead of flow context access. The flow liveness bit flipping can identify the flow state. Kernel-mode storage slot reclamation does not require user-mode participation, avoiding storage resource leakage. The analysis context formed by the aggregation of multiple kernel hook point events of the same packet provides complete packet lifecycle data. The various technical features work together in sequence, so that the diagnostic process does not require repeated hash lookups, flow resources can be reused, and packet full-link events can be completely correlated, realizing low-overhead and resource-leakage-free automated diagnosis of full-link network faults.

[0023] The embodiments of this application will be described in detail below.

[0024] In some embodiments, assigning a flow identifier to a new network flow and writing it into the skb-mark field of the data packet according to the bit field segmentation rules includes: The newly created network flow is identified at the first kernel hook point where the data packet is pushed onto the stack; Allocate free storage slots for newly created network flows and use the slot index as the flow identifier; The target value is obtained by combining the flow identifier and flow liveness status according to the bit field segmentation rules, and the target value is written into the skb-mark field of the current data packet; The first kernel hook point is the interception node that is first triggered when the data packet enters the kernel protocol stack. The free storage slot is the unoccupied storage location in the BPF array mapping. The BPF array mapping is a kernel-mode storage structure that uses contiguous memory addressing and is indexed by integer values.

[0025] In this embodiment, the critical path of the kernel protocol stack is the core processing nodes that a data packet passes through from entering the kernel to leaving the kernel. These nodes include data packet stacking nodes, network filtering nodes, transport layer processing nodes, and data packet destruction nodes. By mounting the eBPF program on the above nodes, there is no need to modify the kernel's native processing logic. The processing information of the data packet is collected only through probes. The operation of the eBPF program will not block the normal transmission process of the data packet.

[0026] Specifically, the flow identifier uses an unsigned integer value to avoid indexing anomalies caused by negative numbers. Each network flow corresponds to a unique flow identifier, and flow identifiers for different network flows are not duplicated, ensuring the accuracy of flow context access. The skb-mark field is a kernel-native packet marking field. During packet cloning, copying, and forwarding, the kernel automatically transfers the value without requiring additional program instructions. The bit-field segmentation rule divides the 32-bit skb-mark field into a high 31 bit and a low 1 bit. The high 31 bits can meet the identification needs of large-scale concurrent network flows, while the low 1 bit has only two states, accurately representing the active and reclaimable states of a network flow without requiring additional storage resources to record flow states. Analysis context is aggregated on a packet-by-packet basis. All kernel hook events for each packet are integrated into the same dataset, allowing for a complete reconstruction of the packet processing flow within the kernel protocol stack.

[0027] For example, when a data packet enters the kernel protocol stack, the eBPF program first identifies whether the network flow to which the data packet belongs is a new flow. If it is a new flow, it assigns a corresponding flow identifier and combines the value according to the rule of storing the flow identifier in the high 31 bits and marking the active state in the low 1 bit. This value is written into the skb-mark field. In the subsequent kernel processing flow, the value of the skb-mark field will be automatically inherited by the kernel. The eBPF program at the subsequent kernel hook point can directly read this value to extract the flow identifier without re-performing the flow identifier allocation operation. After the network flow has completed the transmission of all data packets, the eBPF program will flip the value of the low 1 bit of the skb-mark field to mark the network flow as reclaimable. The kernel program will directly reclaim the storage slot corresponding to the flow identifier without the participation of the user-space program. Finally, based on the collected analysis context, all event information is traversed to complete the fault diagnosis.

[0028] Those skilled in the art will understand that the kernel's automatic inheritance feature is a native design of the Linux kernel network stack. All derivative operations on data packets retain the value of the skb-mark field. This feature takes effect without additional kernel configuration, is applicable to multiple kernel versions of Linux systems, and ensures the compatibility of the solution. The construction of the analysis context is based on the data packet identifier, which is a unique pointer value assigned by the kernel. Different data packets have different pointer values, ensuring the accuracy of event aggregation and avoiding confusion between events from different data packets.

[0029] In this embodiment, the process of mounting the eBPF program is completed by a user-mode loader through the standard kernel library. The loader reads a preset configuration file, which defines the type of kernel hook point to be mounted, the type of eBPF program, and the range of fields for event collection. After the loader loads the bytecode of the eBPF program into the kernel, the kernel verifies the legality of the eBPF program's instructions. If the verification is successful, the program is mounted to the corresponding kernel hook point. After mounting, the eBPF program enters a listening state, waiting for data packets to trigger kernel hook point events. The number of instructions in the eBPF program is controlled within the kernel's limits, and all instructions are safe kernel-mode instructions, with no illegal memory accesses or system calls, ensuring the stability of kernel operation.

[0030] Specifically, when identifying a new network flow, the network 5-tuple information is standardized. IP addresses are converted to integer values, while port numbers and transport layer protocols use native integer values. The standardized 5-tuple information can be directly used as the lookup key in the flow state mapping table without additional format conversion. The allocation of free storage slots uses a lock-free allocation method, implemented using kernel per-CPU variables. Each CPU core corresponds to an independent sub-list of free slots, and allocation operations are performed only in the current CPU core's sub-list. There is no lock contention between CPU cores, further improving allocation efficiency.

[0031] During the execution of the bit field segmentation rule, the eBPF program first verifies the value range of the stream identifier to ensure that the value of the stream identifier does not exceed the maximum value of a 31-bit unsigned integer. Only after the verification is passed will the shift and bitwise OR operations be performed to avoid errors in writing the skb-mark field due to numerical overflow.

[0032] Specifically, when accessing the flow context, the validity of the flow identifier is first verified, checking whether the flow identifier is within the index range of the BPF array mapping. Simultaneously, the network 5-tuple of the flow context is verified to be consistent with the 5-tuple of the current data packet, thus avoiding flow context access anomalies caused by incorrect flow identifiers. Event aggregation operations are implemented using per-CPU event buffers, with each CPU core corresponding to an independent event buffer. Aggregation operations are executed only within the buffer of the current CPU core, eliminating data contention between CPU cores and improving aggregation efficiency.

[0033] Specifically, when flipping the live bit, the current value of the stream's live state is first checked. The flip operation is only performed if the value is 1, avoiding stream state errors caused by repeated flips. After a storage slot is reclaimed, the free slot list is updated in real time, and new network streams can be directly allocated to the reclaimed slots without waiting for user-space program cleanup. The timeout threshold is calculated using the kernel's monotonic clock. The value of the monotonic clock does not change with system time adjustments, ensuring the accuracy of idle time statistics.

[0034] Specifically, during the traversal and analysis of the context, the kernel hook point location and occurrence time of each event are recorded, and the processing latency between adjacent events is calculated. This latency data serves as the basis for judging latency anomaly diagnostic rules. The matching of diagnostic rules adopts a sequential matching method, first matching rules with higher severity levels, then matching rules with lower severity levels, thus prioritizing the identification of serious faults affecting business operations. The structured data of the diagnostic results is stored in binary format, resulting in small storage size and fast transmission speed. User-space programs can directly parse and display the binary data.

[0035] In this embodiment, the first kernel hook point where the data packet enters the stack is usually set as the TC entry node. This node is the first interception point for the data packet from the network card into the kernel protocol stack. The identification of the newly created network flow and the allocation of the flow identifier are performed at this node, which can ensure that the flow identifier can be used in all subsequent kernel hook points, avoiding the problem that some kernel hook points cannot obtain the flow identifier due to the lag in flow identifier allocation.

[0036] In addition, the allocation of free storage slots adopts a sequential allocation method. When the system starts, it initializes all storage slots mapped by the BPF array and marks all slots as unoccupied. During allocation, the first unoccupied slot is selected from the free slot list, and the index value of the slot is directly used as the stream identifier. There is no need to perform additional conversion calculations on the stream identifier, which simplifies the generation logic of the stream identifier.

[0037] It should be noted that BPF array mapping uses contiguous physical memory for storage. Compared with hash mapping, this storage structure does not require calculating hash values ​​or handling hash collisions. Data access can be completed solely through integer indexes. The access speed and stability can meet the processing requirements of high-concurrency network traffic. The contiguous memory addressing method avoids kernel-mode memory fragmentation, improving the efficiency of kernel storage resource utilization. The combination of target values ​​is achieved through bitwise operations. The stream identifier value is shifted one bit to the left, making the stream identifier value occupy the high 31 bits of the skb-mark field. Then, a bitwise OR operation is performed between the stream liveness status value and the shifted stream identifier value, making the stream liveness status value occupy the low 1 bit of the skb-mark field. This bitwise operation process requires only two kernel instructions and does not incur additional computational overhead.

[0038] For example, when a packet of a newly created network flow arrives at the TC entry kernel hook point, the eBPF program queries the free slot linked list mapped by the BPF array, selects the free slot with index 1024, uses 1024 as the flow identifier of the network flow, and the flow liveness status is active, with a corresponding value of 1. 1024 is shifted one bit to the left to get the value 2048, and then 2048 and 1 are bitwise ORed to get the target value 2049. 2049 is written to the skb-mark field of the current packet, completing the writing operation of the flow identifier and the flow liveness status.

[0039] Furthermore, the maintenance of the free slot linked list is handled independently by the kernel-mode eBPF program, without the need for user-mode program involvement. Slots are selected from the head of the linked list during allocation and added to the tail of the list during recycling, forming a circular utilization mechanism. This ensures efficient use of storage slots and avoids waste of storage resources. The selection of the first kernel hook point can be adjusted according to the actual network environment. In addition to the TC entry node, the XDP receiving node can also be selected as the first kernel hook point. The triggering timing of the XDP node is earlier than that of the TC entry node, which can further advance the allocation timing of the flow identifier, making it suitable for network scenarios with high processing timeliness requirements.

[0040] In this embodiment, the size of the BPF array mapping can be adjusted according to the actual number of concurrent flows, supporting a maximum of 2,147,483,647 concurrent flows, which can meet the network diagnostic needs of ultra-large-scale data centers. The memory usage of the BPF array mapping is proportional to the mapping size and can be configured according to the actual server memory resources to avoid excessive consumption of kernel memory resources. For example, for network scenarios with no more than 1 million concurrent flows, the size of the BPF array mapping can be set to 1,048,576, with the corresponding flow identifier range from 0 to 1,048,575, which can meet the needs of 1 million concurrent flows, and the memory usage is only tens of megabytes, which will not put pressure on the server's memory resources.

[0041] Furthermore, the flow identifier allocation logic supports priority settings, allowing different flow identifier allocation priorities to be set for network flows of different service types. Network flows of high-priority services can be allocated storage slots first, avoiding situations where high-priority services cannot be allocated flow identifiers in high-concurrency scenarios, thus meeting the diagnostic needs of core services. Priority settings can be implemented through the matching rules of the network 5-tuple, without modifying the core logic of flow identifier allocation, offering flexible configuration. For example, high priority can be set for network flows of database services, and normal priority can be set for network flows of ordinary services. When the number of free storage slots is insufficient, flow identifiers are allocated first to network flows of database services, ensuring that fault diagnosis of core services is not affected.

[0042] In this embodiment, anti-collision verification of flow identifiers is also supported. When allocating flow identifiers, it is checked whether there is a record with the same flow identifier in the flow state mapping table. If there is a conflict, a new flow identifier is reassigned to ensure the uniqueness of the flow identifier and avoid flow context access errors caused by duplicate flow identifiers of different network flows. The anti-collision verification process only requires one hash mapping query, which has extremely low overhead and will not affect the allocation efficiency of flow identifiers. This application does not impose any restrictions on this.

[0043] In some embodiments, identifying a new network flow at the first kernel hook point where a data packet is pushed onto the stack includes: Extract the network 5-tuple information of the current data packet; query the flow state mapping table based on the network 5-tuple information; If no matching record is found, the current data packet is determined to belong to a newly created network flow. The network 5-tuple information is a combination of source IP address, destination IP address, source port, destination port and transport layer protocol. The flow state mapping table is a kernel-mode mapping structure used to store the correspondence between the allocated flow identifier and the network 5-tuple.

[0044] In this embodiment, network 5-tuple information is the core basis for distinguishing different network flows. All data packets of the same network flow have the same network 5-tuple information, while the network 5-tuple information of different network flows differs. When extracting network 5-tuple information, the eBPF program directly parses the corresponding fields from the protocol header of the data packet, without parsing the payload content of the data packet, which ensures parsing efficiency and avoids privacy issues caused by payload parsing. The flow state mapping table adopts a lightweight kernel-mode mapping structure, which only stores the correspondence between the allocated flow identifier and the network 5-tuple, without storing additional flow feature information, thus reducing the storage overhead of the mapping table. The query operation adopts an exact matching method, matching the network 5-tuple information of the current data packet with the records stored in the mapping table one by one. The matching process only needs to compare whether the values ​​are consistent, without complex calculation logic.

[0045] Specifically, when the eBPF program extracts network 5-tuple information, it sequentially parses the source and destination IP addresses from the IP header of the data packet, and the source, destination, and transport layer protocols from the transport layer header. It then combines the values ​​of these five fields into a unique identifier, which serves as the lookup key for the flow state mapping table. The flow state mapping table lookup operation is executed directly in kernel space, without requiring data interaction with user space, ensuring real-time query performance. If a matching record is found, it indicates that the current data packet belongs to a network flow with an already assigned flow identifier, and no reallocation of the flow identifier is needed. If no matching record is found, it indicates that the current data packet belongs to a newly created network flow without an assigned flow identifier, requiring subsequent flow identifier allocation operations.

[0046] For example, the source IP address of the current data packet is [192.168.1.10] (192.168.1.10), the destination IP address is [10.0.0.5] (10.0.0.5), the source port is 50000, the destination port is 80, and the transport layer protocol is TCP. The eBPF program combines the above information into a query key. No corresponding record is found in the flow state mapping table. Therefore, it determines that the network flow to which the data packet belongs is a newly created network flow and starts the flow identifier allocation process.

[0047] Those skilled in the art will understand that the storage capacity of the flow state mapping table can be adjusted according to the number of concurrent network flows. The number of entries in the mapping table is consistent with the number of storage slots in the BPF array mapping, ensuring that each allocated flow identifier has a corresponding mapping record and avoiding mismatches between flow identifiers and network 5-tuples. For stateless transport layer protocols, such as UDP, there is no explicit connection establishment process for network flows. The eBPF program will determine the network flow corresponding to the first occurrence of UDP 5-tuple information as a newly established network flow, and subsequent packets with the same 5-tuple will be assigned to this network flow, ensuring the accuracy of flow identifier allocation for stateless protocols.

[0048] The size of the flow state mapping table is consistent with the size of the BPF array mapping, ensuring that each allocated flow identifier has a corresponding mapping record and avoiding mismatches between flow identifiers and network 5-tuples. The flow state mapping table is implemented using BPF hash mapping, with constant-time complexity for query and insertion operations, meeting performance requirements in high-concurrency scenarios. This application's technical solution is scalable to support IPv6 network environments. The IP address in the network 5-tuple information can be replaced with an IPv6 address. The allocation of flow identifiers, writing of the skb-mark field, access to flow context, and recycling of storage slots do not require modification. Only the parsing logic of the network 5-tuple information needs adjustment to achieve end-to-end network fault diagnosis in IPv6 network environments, and it is compatible with both IPv4 and IPv6 dual-stack network environments.

[0049] In some embodiments, the kernel's automatic inheritance feature based on the skb-mark field extracts the flow identifier to aggregate multiple kernel hook point events of the same data packet to construct an analysis context, such as... Figure 2 As shown, it includes: S001: Read the value of the skb-mark field inherited by the current data packet, and extract the stream identifier from the skb-mark field value through bitwise operations; S002: Access the BPF array mapping using the stream identifier as the index to obtain the corresponding stream context; S003: Based on the flow context and packet identifier, the multi-kernel hook point events of the same packet are aggregated. The life cycle end identifier is added to the aggregated event set to obtain the analysis context; bit operations are shift and bitwise logical operations that do not change the original attributes of the values; the flow context is structured data used to store the full life cycle feature information of a single network flow; the packet identifier is a kernel state pointer value used to uniquely distinguish a single packet; and the life cycle end identifier is a mark used to indicate that the packet has completed the full process of kernel protocol stack processing.

[0050] In this embodiment, the kernel automatic inheritance feature of the skb-mark field is applicable to all kernel processing scenarios for data packets, including packet cloning, cross-network device forwarding, and virtual bridge transmission. Regardless of the kernel processing flow of the data packet, the value of its skb-mark field will be completely preserved. The skb-mark value read by the eBPF program at subsequent kernel hook points is completely consistent with the value written at the first kernel hook point, ensuring the stability of flow identifier extraction. The bitwise operation to separate the flow identifier uses a right shift of 1 bit to remove the lower 1 bit of the skb-mark field. The remaining 31 bits are the original value of the flow identifier. This operation does not change the numerical attributes of the flow identifier, ensuring its accuracy.

[0051] It's important to note that when accessing the BPF array mapping using the stream identifier as an index, the kernel directly locates the corresponding storage location in contiguous memory based on the stream identifier value. This eliminates the need for hash calculations, bucket lookups, or other operations, resulting in access latency controlled at the nanosecond level. This makes it suitable for high-concurrency network scenarios with millions of PPS. The stream context is stored in a structured data format with fixed fields, including network 5-tuples, stream creation time, and transport layer status. Each field's storage location is fixed, allowing eBPF programs to directly locate and read the corresponding field without parsing variable-length data structures, thus improving the efficiency of stream context reading.

[0052] For example, the skb-mark field value inherited by the current data packet is 2049. The eBPF program shifts this value one bit to the right to get the value 1024, which is the stream identifier. Using 1024 as the index, the BPF array mapping is accessed to directly read the stream context information at the corresponding storage location. The entire access process only requires three kernel instructions, with no additional calculation or query overhead.

[0053] Furthermore, the bitwise operation logic is fixed and applicable to all Linux kernel versions supporting eBPF technology. No adjustment of the operation logic is required based on the kernel version, ensuring cross-version compatibility. The storage content of the flow context can be expanded according to diagnostic needs, adding diagnostic feature information such as queue status and congestion status. The expanded flow context still uses structured storage, without affecting access efficiency. The packet identifier is a unique skb pointer value assigned by the kernel to each packet. This value remains unchanged throughout the packet's lifecycle, and different packets have identical skb pointer values, serving as the sole basis for aggregating events for the same packet. Events collected from multiple kernel hook points include packet reception events, network filtering events, transport layer processing events, and packet release events. Each event contains information such as the kernel hook point type, event occurrence time, and packet identifier. When aggregating events based on packet identifiers, the eBPF program integrates all events carrying the same packet identifier into the same dataset, forming a complete packet lifecycle event chain. The lifecycle end marker is added after the data packet has completed all processing steps of the kernel protocol stack. Typically, this marker is added at the point where the kernel hook is released. The data set after the marker is added constitutes the complete analysis context, indicating that all kernel processing events of the data packet have been collected and can be used for subsequent fault diagnosis and analysis.

[0054] In some embodiments, the event aggregation operation supports cross-network namespace event aggregation. For data packets forwarded by containers across network namespaces, cross-namespace event aggregation can be achieved through data packet identifiers, fully reconstructing the transmission path of data packets between the host and the container, which can meet the network fault diagnosis needs in containerized environments. During the cross-network namespace event aggregation process, the value of the skb-mark field is automatically retained during cross-namespace forwarding, and the logic for extracting flow identifiers and accessing flow contexts does not need to be modified, ensuring the consistency of the diagnostic process.

[0055] In some embodiments, such as Figure 3 As shown, accessing the BPF array mapping using the stream identifier as an index yields the corresponding stream context, including: S010: Determine the target storage location for the BPF array mapping based on the stream identifier; S020: Read the network quintuple, stream creation time, and transport layer status information from the target storage location; S030: Integrate the read information into a stream context; the transport layer state information is identification information used to characterize the network stream transmission stage.

[0056] In this embodiment, the target storage location of the BPF array mapping is directly determined by the value of the flow identifier. The value of the flow identifier is completely consistent with the index value of the storage location. The kernel-mode program can directly calculate the memory address of the target storage location through the flow identifier value, without traversing the mapping structure to find the storage location. The network 5-tuple information is a basic field of the flow context, used to verify the matching of the flow identifier. The flow creation time is used to record the start time of the network flow and can be used to determine the lifespan of the network flow. The transport layer state information is used to characterize the transmission stage of the network flow, including connection establishment, data transmission, connection closure, etc., which can assist the fault diagnosis program in judging the running status of the network flow.

[0057] Specifically, when an eBPF program reads information from the target storage location, it reads the fields sequentially according to a preset order: first, the network 5-tuple information; then, the stream creation time; and finally, the transport layer status information. After reading, the above information is integrated into structured data with a fixed format to form a stream context. The stream context can be directly used by subsequent event-related programs without data format conversion. The transport layer status information uses integer values, with different values ​​corresponding to different transmission stages. The numerical values ​​occupy little storage space and are read quickly.

[0058] For example, in the target storage location corresponding to flow identifier 1024, the stored network 5-tuple information is [192.168.1.10:50000](192.168.1.10:50000)-[10.0.0.5:80](10.0.0.5:80)-TCP. The flow creation time is a second-level timestamp after kernel startup, and the transport layer state information is 1, representing the data transmission stage. The eBPF program integrates the above information into a flow context for use by the subsequent event collection program.

[0059] Those skilled in the art will understand that the flow context information reading operation is performed in full only on the first access. Subsequent accesses can read only the updated field information, such as transport layer status information, which can further reduce the data reading overhead. The information of the target storage location is updated in real time as the network flow is processed. For example, the transport layer status information is updated synchronously during connection establishment, data transmission, and connection closure, which can ensure that the flow context information is consistent with the actual state of the network flow.

[0060] The storage of the stream context supports incremental updates. The corresponding fields in the stream context are only updated when the state of the network stream changes, eliminating the need to rewrite the entire stream context on every access. This reduces kernel-level memory write overhead and improves system efficiency. During incremental updates, the validity of the stream identifier and the integrity of the stream context are verified to prevent data corruption and ensure the accuracy of stream context access.

[0061] In some embodiments, when a network flow terminates, the live bit of the skb-mark field is flipped and the storage slot corresponding to the flow identifier is reclaimed in kernel mode, including: identifying the terminating data packet of the network flow; reading the value of the skb-mark field of the terminating data packet and flipping the lower 1 bit value; marking the corresponding BPF array-mapped storage slot as idle based on the flipped flow identifier; the terminating data packet is a data packet carrying a transport layer shutdown flag or reaching an idle timeout condition.

[0062] In this embodiment, the identification of termination packets is a core prerequisite for flow resource reclamation. Transport layer closure identifiers include the TCP protocol's FIN and RST identifiers. Packets carrying these identifiers indicate active closure of the network flow. The idle timeout condition, where the network flow does not generate any packets within a preset time, indicates passive closure. These two identification methods cover all network flow termination scenarios, preventing unrecognized network flow termination. The flipping operation of the live bits in the skb-mark field is implemented using a bitwise AND operation. Performing a bitwise AND operation between the skb-mark field value and the hexadecimal value FFFFFFFE flips the lower 1 bit from 1 to 0, while the higher 31 bits of the flow identifier remain unchanged, ensuring the accuracy of the flow identifier.

[0063] Specifically, when a storage slot is marked as free, the kernel-mode eBPF program adds the slot to the end of the free slot list and clears the flow context information in the storage slot to prevent residual information from affecting the use of subsequent new network flows. Both the marking and clearing operations are performed directly in kernel mode without the need for instructions or intervention from user-mode programs. The timing of the reclamation action is perfectly matched with the timing of the termination of the network flow, enabling the instant reclamation of storage slots.

[0064] For example, when a network flow terminates, it carries a TCPFIN flag. The eBPF program recognizes this packet as a terminated packet, reads the skb-mark field value as 2049, performs a bitwise AND operation between 2049 and FFFFFFFE to obtain the value 2048, flips the live bit, marks the corresponding BPF array-mapped storage slot as idle based on the flow flag 1024, clears the flow context information in the slot, and adds the slot to the idle slot list.

[0065] Those skilled in the art will understand that the live bit flipping operation only modifies the lower 1 bit of the skb-mark field and does not affect the higher 31 bits of the stream identifier value, thus ensuring that the correspondence between the stream identifier and the storage slot is not disrupted. The storage slot reclamation operation will not affect the storage and access of other network streams, and there is no lock contention during the reclamation process, which can ensure the stability of reclamation in high-concurrency scenarios.

[0066] The kernel-mode recycling module's slot recycling logic supports batch recycling. When a large number of network flows terminate simultaneously, it can identify terminated data packets in batches, flip live bits in batches, and reclaim the corresponding storage slots in batches. This improves recycling efficiency in high-concurrency scenarios and avoids situations where a large number of terminated flows are not reclaimed in time. During batch recycling, the allocation of flow identifiers and access to flow contexts for active network flows will not be affected, ensuring stable system operation.

[0067] In some embodiments, identifying the terminating packet of a network flow includes: Detect the transport layer identifier of the current data packet; When the transport layer flag is set to the off flag, the current data packet is determined to be a terminated data packet; when the duration during which the network flow has not generated data packets reaches the timeout threshold, the corresponding data packet is determined to be a terminated data packet; the timeout threshold is the preset duration of the network flow's idle state.

[0068] In this embodiment, the detection of transport layer identifiers targets stateful transport layer protocols. The eBPF program directly parses the transport layer header field of the data packet to detect the presence of a FIN or RST identifier. These identifiers are standard closing identifiers defined by the transport layer protocol, and the detection logic does not require custom rules, ensuring accurate identification. Idle time statistics are based on the flow creation time and the last data packet time of the flow context. The eBPF program updates the last data packet time of the network flow in real time and periodically calculates the difference between the current time and the last data packet time. When the difference reaches a timeout threshold, the network flow is determined to be in an idle termination state, and the latest data packet is identified as the termination data packet.

[0069] It should be noted that the timeout threshold can be preset according to the needs of the network scenario. For network scenarios with more short connections, a shorter timeout threshold can be set, and for network scenarios with more long connections, a longer timeout threshold can be set. Adjusting the threshold does not require modifying the kernel source code; only the configuration parameters of the eBPF program need to be adjusted. The two methods of identifying terminated data packets complement each other. Actively closed network flows are identified through transport layer identifiers, while passively closed network flows are identified through idle timeouts. This covers all network flow termination scenarios without any omissions.

[0070] For example, if a TCP network stream packet carries a FIN flag, the eBPF program will detect this closing flag and directly determine that the packet is a terminated packet; if a UDP network stream has not generated any packets within 60 seconds, and the timeout threshold is set to 60 seconds, the eBPF program will determine that the latest packet in the network stream is a terminated packet.

[0071] Furthermore, the idle time statistics are executed directly in kernel mode, without the need for user-mode programs to participate in the statistics and judgment, ensuring real-time statistics while avoiding frequent data interactions between the kernel and user mode. The transport layer identifier detection operation is only performed on transport layer packets; for non-transport layer packets, the detection step is skipped, reducing unnecessary computational overhead.

[0072] For stateless UDP network flows, the allocation and recycling logic for flow identifiers is consistent with that of stateful TCP, differing only in the logic for identifying terminated packets. Since UDP lacks an explicit connection closure identifier, it primarily identifies terminated packets through idle timeout conditions. The timeout threshold can be adjusted according to the characteristics of the UDP service. For UDP services with high real-time requirements, a shorter timeout threshold can be set; for long-cycle UDP services, a longer timeout threshold can be set. This ensures timely recycling of UDP flow resources while preventing abnormal flow context access caused by premature recycling of flow identifiers. For ICMP network packets, the same processing logic as UDP can be used. A unique identifier is constructed based on the source IP address, destination IP address, ICMP type, and ICMP code. A corresponding flow identifier is then assigned, enabling end-to-end event collection and fault diagnosis of ICMP packets, covering network layer fault diagnosis requirements.

[0073] In some embodiments, network fault diagnosis results are obtained by performing cross-event correlation analysis based on the analysis context, including: Iterate through and analyze all event information in the context; Match event information with diagnostic rules, and generate structured data of fault type, root cause information and inspection direction based on the matching results; the diagnostic rules are kernel-level judgment conditions used to determine the type of network fault.

[0074] In this embodiment, the context traversal operation is executed sequentially according to the occurrence time of events, checking the kernel hook point type, event status, and characteristic information of each event in turn. The traversal process does not require skipping or repeating checks, ensuring analysis efficiency. The diagnostic rules are preset kernel-mode judgment conditions, including packet loss event judgment rules, retransmission event judgment rules, latency anomaly judgment rules, and policy discard judgment rules, etc. Each rule corresponds to a network fault type, and the judgment conditions of the rules are based on the standard characteristics of kernel events, eliminating the need for custom non-standard rules.

[0075] Specifically, when matching event information with diagnostic rules, the eBPF program compares the event's characteristic information with the rule's judgment conditions. If the conditions are met, the corresponding fault type is determined to have occurred. At the same time, it combines other event information in the analysis context to determine the root cause of the fault and the corresponding inspection direction. The generated structured data contains three fixed fields: fault type, root cause information, and inspection direction. The data format is uniform and can be directly parsed and used by the operation and maintenance system without the need for format conversion.

[0076] For example, the analysis context includes packet reception events and packet release events. The characteristic information of the packet release event is kernel drop. By matching the packet loss event diagnostic rules and combining the intermediate network filtering event information, the fault type is determined to be policy drop. The root cause is the matching of network filtering rules. The inspection direction is to check the network filtering rule configuration, and the corresponding structured diagnostic data is generated.

[0077] Those skilled in the art will understand that diagnostic rules can be extended according to the needs of fault diagnosis, adding new fault type judgment rules. The extended rules still use kernel event characteristics as the judgment basis, without modifying the core analysis logic. Cross-event correlation analysis combines event information from multiple kernel hook points for comprehensive judgment, rather than judging a single event, which can avoid misjudgments caused by a single event and improve the accuracy of diagnostic results. This application is not limited to this.

[0078] In some embodiments, matching event information with diagnostic rules includes: Extract the kernel hook point type and feature fields from the event information; The feature fields are compared with the judgment conditions of the diagnostic rules; When the feature field meets the judgment condition, the matching of the corresponding diagnostic rule is completed.

[0079] In this embodiment, the kernel hook point type in the event information is used to determine the kernel processing stage where the event occurred. Different kernel hook point types correspond to different diagnostic rules. The feature field is the kernel processing status information carried by the event, including data packet processing results, kernel return values, queue status, transport layer status, etc., which is the core basis for matching diagnostic rules. The comparison between the feature field and the judgment conditions of the diagnostic rules adopts a numerical comparison method. It only needs to compare whether the values ​​meet the preset range or fixed value, without complex calculation logic, which can ensure matching efficiency.

[0080] Specifically, after extracting the kernel hook point type and feature fields from the event information, the eBPF program first filters the corresponding diagnostic rules based on the kernel hook point type. It only compares the feature fields with the filtered diagnostic rules, eliminating the need to iterate through all diagnostic rules, thus further reducing matching overhead. When the value of the feature field meets the judgment condition of the diagnostic rule, the diagnostic rule is considered to have matched successfully, the corresponding fault type is recorded, and root cause information and inspection directions are supplemented by combining other event information in the analysis context.

[0081] For example, the kernel hook point type of the event information is a network filtering processing node, the feature field is rule matching and discarding, the diagnostic rule corresponding to policy discarding is filtered out, the feature field is compared with the judgment condition of the rule, if the feature field meets the judgment condition, the matching of the diagnostic rule is completed, and the corresponding fault type is recorded as policy discarding.

[0082] Furthermore, the judgment conditions of diagnostic rules can be adjusted according to the actual network environment. This adjustment process does not require modifying the core logic of the eBPF program; only the judgment condition values ​​of the rules need to be updated, making it adaptable to different network scenarios. Feature field extraction is performed only on the fields required by the diagnostic rules, eliminating the need to extract all fields of the event, thus reducing unnecessary computational overhead.

[0083] The kernel-mode acquisition module supports filtering for event collection. Filtering rules can be configured via configuration files to collect event information only from packets that match the filtering rules. These rules can be set based on network 5-tuples, protocol type, packet size, and other information, reducing unnecessary event collection and further lowering system overhead. Matching of filtering rules is performed within the eBPF program. Subsequent flow identifier allocation, event collection, and context construction operations are only executed when a packet matches the filtering rules. Packets that do not match the filtering rules are skipped, incurring no additional computational overhead. For example, filtering rules can be configured to collect event information only from packets with source IP address ranges of [10.0.0.0 / 24] (10.0.0.0 / 24). Packets from other IP address ranges are skipped, significantly reducing the number of events collected and lowering system overhead.

[0084] This application further provides an end-to-end network fault diagnosis system based on eBPF, such as... Figure 4As shown, the system includes: Mounting module 1, which mounts the eBPF program on the critical path of the kernel protocol stack, assigns a flow identifier to the newly created network flow, and writes it into the skb-mark field of the data packet according to the bit field segmentation rule; Aggregation module 2, which extracts the flow identifier based on the kernel automatic inheritance feature of the skb-mark field to aggregate multiple kernel hook point events of the same data packet to build an analysis context, and flips the live bit of the skb-mark field when the network flow terminates and reclaims the storage slot corresponding to the flow identifier in kernel mode; Analysis module 3, which performs cross-event correlation analysis based on the analysis context to obtain network fault diagnosis results, wherein the flow identifier is an unsigned integer value used to uniquely identify a single network flow, the bit field segmentation rule is to use the high 31 bits of the skb-mark field to store the flow identifier and the low 1 bit to store the flow liveness status, the flow liveness status is a value used to identify whether the network flow is in an active or reclaimable state, and the analysis context is a data set used to aggregate events generated by the same data packet at multiple kernel hook points.

[0085] Those skilled in the art will understand that the technical solution of this application can be directly deployed on existing Linux server environments without replacing hardware devices, modifying kernel source code, or restarting the system. It takes effect immediately after deployment and is applicable to various scenarios such as carrier networks, data center networks, and cloud platform networks. It can stably handle network diagnostic needs with high concurrency and large traffic. At the same time, the autonomous reclamation of streaming resources can ensure the long-term stable operation of the system without the risk of exhausting storage resources. The full-link event correlation can improve the accuracy of fault diagnosis and meet the needs of fault root cause location in complex network environments.

[0086] In this embodiment, the loading process of the eBPF program supports dynamic updates without requiring a system or service process restart. The mounting location of kernel hook points, the range of event collection fields, and diagnostic rules can be adjusted during service operation without affecting normal service operation. During dynamic updates, the flow identifier management module retains the allocated flow identifiers and flow context information, preventing flow identifier loss or abnormal flow context access and ensuring the continuity of the diagnostic process.

[0087] The technical solution of this application supports multi-instance deployment. Multiple independent diagnostic instances can be deployed on the same server. Each instance corresponds to a different business network segment, a different protocol type, or a different fault diagnosis requirement. Each instance is independent of the others. Flow identifiers, storage slots, and diagnostic rules are all isolated from each other, so there will be no data conflicts or mutual interference. This can meet the differentiated diagnostic needs of multiple business scenarios.

[0088] The core logic of this technical solution is implemented entirely in kernel mode. Only the display of diagnostic results and the updating of configurations require the participation of user-mode programs. This minimizes data interaction between the kernel and user modes, reduces system overhead, and improves the real-time performance of diagnostics. Data interaction between kernel and user modes is implemented using a circular buffer, which ensures high data transmission efficiency, eliminates lock contention, and meets the data transmission requirements of high-concurrency scenarios.

[0089] Bitwise operations utilize standard bitwise instructions natively supported by the kernel, ensuring compatibility with all Linux kernel versions supporting eBPF technology. No adaptation modifications are required for different kernel architectures, supporting various processor architectures such as x86, ARM, and RISC-V, adapting to diverse hardware environments. Writing to the skb-mark field does not affect native kernel functions such as flow control, policy routing, and firewalls. The kernel's native functions read the complete skb-mark field value; the bit-field segmentation rules in this application do not compromise the integrity of the skb-mark field, ensuring normal operation of native functions without functional conflicts. If some bit fields of the skb-mark field are already used in the existing network environment, the bit-field segmentation rules can be adjusted to allocate unused bit fields to flow identifiers and flow liveness status without modifying core logic, thus adapting to network environments already using the skb-mark field. For example, if the lower 4 bits of the skb-mark field are already used for flow control in the existing network environment, the bit field segmentation rule can be adjusted so that the upper 28 bits are used to store the flow identifier, the highest bit of the lower 4 bits is used to store the flow liveness status, and the remaining 3 bits are reserved for the native flow control function. This can ensure that the native function and the diagnostic function of this application can operate normally at the same time without functional conflict.

[0090] The technical solution presented in this application supports high-availability deployment. When an eBPF program encounters an anomaly, the kernel automatically unloads the abnormal eBPF program, without affecting the normal processing flow of the kernel protocol stack or causing network interruption, thus ensuring the stable operation of the business system. After the abnormal unloading, the user-space program automatically reloads the eBPF program, restoring diagnostic functionality without manual intervention, ensuring the continuity of the diagnostic process. The system's operational status can be monitored through kernel-level statistical mappings. These mappings record statistical information such as the number of flow identifier allocations, the number of free storage slots, the number of events collected, and the number of diagnostic results. User-space programs can read the information in the statistical mappings to monitor the system's operational status and resource usage in real time, enabling timely detection of anomalies during system operation and ensuring stable system operation.

[0091] The runtime overhead of an eBPF program can be controlled by adjusting the scope of event collection and the number of diagnostic rules. For low-configuration server environments, unnecessary kernel hook points can be reduced, and the number of diagnostic rules can be simplified, further reducing the system's CPU and memory usage. This adapts to server environments with different configurations. For example, for low-configuration edge server environments, only two core kernel hook points—TC entry and packet release—can be mounted, and only two core diagnostic rules—packet loss and retransmission—can be enabled. The system's CPU utilization can be controlled to within 0.5%, without affecting the normal operation of edge services.

[0092] In practical network diagnostic scenarios, taking short-connection HTTP traffic in a data center as an example, each client's HTTP request creates an independent TCP network flow. After the data packet is sent from the client's network card, it is transmitted through the physical switch to the server's network card and enters the TC entry point of the server's kernel protocol stack kernel hook point. The eBPF program of the kernel-mode acquisition module is triggered, and the flow identifier management module identifies the network flow as a new flow, allocates an idle storage slot, generates a flow identifier, and writes the combined values ​​according to the bit field segmentation rules into the skb-mark field. When the data packet subsequently passes through the IP layer and the transport layer to receive the kernel hook point, the flow identifier management module extracts the flow identifier through the skb-mark field, accesses the BPF array mapping to obtain the flow context, and the context construction module aggregates the events of the two kernel hook points into the analysis context of the corresponding data packet. After the HTTP request is processed, the server sends a data packet carrying the FIN flag. When this data packet reaches the kernel hook point where the data packet is released, the kernel-mode recycling module recognizes it as a terminated data packet, flips the live bit of the skb-mark field, and reclaims the corresponding storage slot. The fault diagnosis module traverses and analyzes the context, matches the diagnostic rules, and generates a diagnostic result. The entire process involves no duplicate hash lookups, no user-mode recycling intervention, and no storage resource leaks. The entire link event of the data packet is completely collected, and the fault diagnosis result can reflect the running status of the network flow.

[0093] Those skilled in the art will understand that the technical solution of this application can be integrated with existing network monitoring systems and operation and maintenance management platforms. Diagnostic results can be output to the monitoring system and operation and maintenance platform through standardized interfaces, enabling automated alarm and closed-loop processing of network faults. It can be integrated into existing operation and maintenance systems to improve operational efficiency. The technical solution of this application solves the technical problems of high flow context access overhead, inability to automatically reclaim flow resources, and inability to correlate data packet events in existing technologies through the coordinated use of features such as bit field writing of flow identifiers, automatic kernel inheritance of skb-mark fields, lock-free access to flow context, kernel-mode autonomous reclamation of flow resources, and collection and diagnosis of end-to-end events. It achieves low-overhead, resource-leakage-free, and automated end-to-end network fault diagnosis, meeting the fault diagnosis needs in complex network environments.

Claims

1. A method for diagnosing end-to-end network faults based on eBPF, characterized in that, include: Mount the eBPF program on the critical path of the kernel protocol stack to assign a flow identifier to the newly created network flow and write it into the skb-mark field of the data packet according to the bit field segmentation rules; Based on the kernel's automatic inheritance feature of the skb-mark field, the flow identifier is extracted to aggregate the multi-kernel hook point events of the same data packet to build an analysis context. When the network flow terminates, the live bit of the skb-mark field is flipped and the storage slot corresponding to the flow identifier is reclaimed in kernel mode. Network fault diagnosis results are obtained by performing cross-event correlation analysis based on the analysis context. The flow identifier is an unsigned integer value used to uniquely identify a single network flow. The bit field segmentation rule is to use the high 31 bits of the skb-mark field to store the flow identifier and the low 1 bit to store the flow liveness status. The flow liveness status is a value used to identify whether the network flow is in an active or reclaimable state. The analysis context is a data set used to collect events generated by the same data packet at multiple kernel hook points. Assign a flow identifier to the newly created network flow and write it to the skb-mark field of the data packet according to the bit field segmentation rules, including: The first kernel hook point when the data packet is pushed onto the stack identifies the newly created network flow; an idle storage slot is allocated for the newly created network flow and the slot index is used as the flow identifier; the target value is obtained by combining the flow identifier and the flow liveness status according to the bit field segmentation rules, and the target value is written into the skb-mark field of the current data packet; the first kernel hook point is the first interception node triggered when the data packet enters the kernel protocol stack, the idle storage slot is the unoccupied storage location in the BPF array mapping, and the BPF array mapping is a kernel-mode storage structure that uses contiguous memory addressing and is indexed by integer values; Based on the kernel's automatic inheritance feature of the skb-mark field, flow identifiers are extracted to aggregate multi-kernel hook point events of the same data packet to construct an analysis context, including: The algorithm reads the skb-mark field value inherited by the current data packet and extracts the flow identifier from the skb-mark field value through bitwise operations. It then accesses the BPF array mapping using the flow identifier as an index to obtain the corresponding flow context. Based on the flow context and the data packet identifier, it aggregates the multi-kernel hook point events of the same data packet and adds a lifecycle end identifier to the aggregated event set to obtain the analysis context. Bitwise operations are shift and bitwise logical operations that do not change the original attributes of the values. The flow context is structured data used to store the full lifecycle feature information of a single network flow. The data packet identifier is a kernel-state pointer value used to uniquely distinguish a single data packet. The lifecycle end identifier is a mark used to indicate that the data packet has completed the entire process of kernel protocol stack processing.

2. The method according to claim 1, characterized in that, The newly created network flow is identified at the first kernel hook point where the data packet is pushed onto the stack, including: Extract the network 5-tuple information of the current data packet; query the flow state mapping table based on the network 5-tuple information; if no matching record is found, determine that the network flow to which the current data packet belongs is a newly created network flow; the network 5-tuple information is a combination of source IP address, destination IP address, source port, destination port and transport layer protocol information, and the flow state mapping table is a kernel-mode mapping structure used to store the correspondence between the allocated flow identifier and the network 5-tuple.

3. The method according to claim 1, characterized in that, Accessing the BPF array map using the stream identifier as an index yields the corresponding stream context, including: The target storage location of the BPF array mapping is determined based on the flow identifier; the network quintuple, flow creation time, and transport layer state information are read from the target storage location; the read information is integrated into the flow context; the transport layer state information is the identifier information used to characterize the network flow transmission stage.

4. The method according to claim 1, characterized in that, When a network flow terminates, the live bits of the skb-mark field are toggled, and the storage slot corresponding to the flow identifier is reclaimed in kernel mode, including: Identify the terminating data packet of the network flow; read the value of the skb-mark field of the terminating data packet and flip the lower 1 bit of the value; mark the corresponding BPF array mapping storage slot as idle based on the flipped flow identifier; the terminating data packet is a data packet carrying a transport layer close flag or that has reached the idle timeout condition.

5. The method according to claim 4, characterized in that, Identify terminating packets in a network flow, including: The transport layer identifier of the current data packet is detected; when the transport layer identifier is a closed identifier, the current data packet is determined to be a terminated data packet; when the duration during which the network flow has not generated data packets reaches the timeout threshold, the corresponding data packet is determined to be a terminated data packet; the timeout threshold is the preset duration of the network flow's idle state.

6. The method according to claim 1, characterized in that, Network fault diagnosis results are obtained by performing cross-event correlation analysis based on the analysis context, including: It iterates through all event information in the analysis context; matches event information with diagnostic rules; generates structured data of fault type, root cause information and inspection direction based on the matching results; and uses diagnostic rules as kernel-level judgment conditions to determine the type of network fault.

7. The method according to claim 6, characterized in that, Matching event information with diagnostic rules, including: Extract the kernel hook point type and feature fields from the event information; compare the feature fields with the judgment conditions of the diagnostic rules; when the feature fields meet the judgment conditions, complete the matching of the corresponding diagnostic rules.

8. A full-link network fault diagnosis system based on eBPF, characterized in that, include: The mounting module mounts the eBPF program on the critical path of the kernel protocol stack, assigns a flow identifier to the newly created network flow and writes it into the skb-mark field of the data packet according to the bit field segmentation rules. The aggregation module extracts the flow identifier based on the kernel's automatic inheritance feature of the skb-mark field to aggregate the multi-kernel hook point events of the same data packet to build an analysis context. When the network flow terminates, it flips the live bit of the skb-mark field and reclaims the storage slot corresponding to the flow identifier in kernel mode. The analysis module performs cross-event correlation analysis based on the analysis context to obtain network fault diagnosis results. The flow identifier is an unsigned integer value used to uniquely identify a single network flow. The bit field segmentation rule is to use the high 31 bits of the skb-mark field to store the flow identifier and the low 1 bit to store the flow liveness status. The flow liveness status is a value used to identify whether the network flow is in an active or reclaimable state. The analysis context is a data set used to collect events generated by the same data packet at multiple kernel hook points. Assign a flow identifier to the newly created network flow and write it to the skb-mark field of the data packet according to the bit field segmentation rules, including: The first kernel hook point when the data packet is pushed onto the stack identifies the newly created network flow; an idle storage slot is allocated for the newly created network flow and the slot index is used as the flow identifier; the target value is obtained by combining the flow identifier and the flow liveness status according to the bit field segmentation rules, and the target value is written into the skb-mark field of the current data packet; the first kernel hook point is the first interception node triggered when the data packet enters the kernel protocol stack, the idle storage slot is the unoccupied storage location in the BPF array mapping, and the BPF array mapping is a kernel-mode storage structure that uses contiguous memory addressing and is indexed by integer values; Based on the kernel's automatic inheritance feature of the skb-mark field, flow identifiers are extracted to aggregate multi-kernel hook point events of the same data packet to construct an analysis context, including: The algorithm reads the skb-mark field value inherited by the current data packet and extracts the flow identifier from the skb-mark field value through bitwise operations. It then accesses the BPF array mapping using the flow identifier as an index to obtain the corresponding flow context. Based on the flow context and the data packet identifier, it aggregates the multi-kernel hook point events of the same data packet and adds a lifecycle end identifier to the aggregated event set to obtain the analysis context. Bitwise operations are shift and bitwise logical operations that do not change the original attributes of the values. The flow context is structured data used to store the full lifecycle feature information of a single network flow. The data packet identifier is a kernel-state pointer value used to uniquely distinguish a single data packet. The lifecycle end identifier is a mark used to indicate that the data packet has completed the entire process of kernel protocol stack processing.

Images