A kernel-state on-demand telemetry system and method for digital twin networks

Abstract

This invention provides a kernel-mode on-demand telemetry system and method for digital twin networks, relating to the field of network telemetry technology. This invention uses a central scheduling server to construct trigger messages carrying multi-field fingerprints and on-demand probe control field bitmaps, which are then sent to designated nodes at configurable frequencies. The bitmap flexibly specifies the subset of data dimensions each node needs to transmit back. After the monitored node's eBPF probe intercepts the trigger message in kernel mode through multi-stage fingerprint authentication, it parses the on-demand probe control field bitmap and performs selective state harvesting only on the requested dimensions. The aggregated statistical summary is then written into the trigger message after two-step buffer adjustment encoding, generating a telemetry response message for reflection and transmission. All processing in this invention is completed in kernel mode, without transmitting copies of the original messages. A single response is only tens to hundreds of bytes, solving the problems of high bandwidth overhead, inflexible on-demand probe, and lack of deep indicator collection capabilities in existing solutions.

Description

Technical Field

[0001] This invention relates to the field of network telemetry technology, and in particular to a full kernel-mode on-demand telemetry system and method for digital twin networks. Background Technology

[0002] Digital twin networks achieve real-time perception and intelligent decision-making by constructing a virtual mapping model of the physical network. The accuracy of this modeling depends on the timeliness, dimensionality coverage, and flexible controllability of the synchronized virtual and physical data. Existing network data acquisition and telemetry schemes have the following shortcomings:

[0003] First, traditional telemetry transmission schemes adopt a "kernel-mode acquisition - user-mode agent packet transmission" architecture. Data needs to cross the kernel-user-mode boundary before being sent by the agent via sockets. This architecture introduces two context switches. Under high host load, the agent faces CPU preemption and time slice rotation, resulting in unpredictable fluctuations in telemetry transmission intervals and severe latency jitter, which cannot provide a stable guarantee of timeliness for digital twin networks.

[0004] Second, while some eBPF-based kernel-mode traffic collection schemes sink the collection and forwarding process to kernel space to avoid user-mode copy overhead, they essentially clone the raw data packets passing through the monitored interface packet by packet and then encapsulate and forward them to remote storage devices via tunnels. Although such schemes support truncating cloned packets to retain only a few bytes of the packet header, each data packet matching the collection criteria still requires a complete cloning, encapsulation, and forwarding operation. The bandwidth overhead is still proportional to the number of packets collected, causing significant bandwidth congestion and CPU resource consumption on the service network in high packet rate scenarios. These schemes also have two inherent limitations:

[0005] First, the collection mode is a passive and continuous mirroring, lacking on-demand detection capabilities. It cannot flexibly specify the target host, collection frequency, and specific data dimensions to be collected from the control plane. When the administrator only needs to obtain the latency indicators of a specific host rather than the full traffic, he still has to clone and forward each passing packet.

[0006] Secondly, this type of solution can only obtain a copy of the original traffic at the message level, and cannot provide deep network status indicators such as active flow latency, queue queuing delay, and implicit packet loss rate, which can only be obtained through active detection or kernel event tracking. These unconventional statistics are precisely the key inputs that digital twin networks cannot do without for link quality modeling and anomaly reasoning.

[0007] Furthermore, directly building telemetry transmission channels in kernel space faces strict constraints from the eBPF security model: eBPF probe programs cannot create sockets, allocate heap memory, actively generate network packets, and have a stack space of only 512 bytes. These constraints make the traditional "create connection—allocate buffer—fill and send" transmission paradigm completely infeasible in the eBPF environment.

[0008] Therefore, there is an urgent need for a kernel-mode on-demand telemetry system and method for digital twin networks to solve the above problems. Summary of the Invention

[0009] To address the aforementioned technical problems in related technologies, this invention provides a kernel-mode on-demand telemetry system and method for digital twin networks.

[0010] In a first aspect, the present invention provides a kernel-mode on-demand telemetry system for digital twin networks, including a central scheduling server and multiple monitored nodes;

[0011] The central scheduling server is used to construct trigger messages carrying multi-field fingerprints and on-demand detection control fields according to the configuration table of monitored nodes and send them to each monitored node. It also obtains multi-dimensional network status data based on the telemetry response messages reflected back by each monitored node and provides it to the digital twin network modeling engine.

[0012] The monitored node includes a perception aggregation module and a trigger reflection module residing in its kernel space;

[0013] The perception aggregation module is used to deploy eBPF probe programs at multiple layers of the kernel protocol stack of the monitored node.

[0014] The trigger reflection module is used to perform multi-stage cascaded filtering on inbound messages through the eBPF probe program to intercept the trigger message, extract the on-demand detection control field bitmap for selective state harvesting, and then write the aggregated statistical summary encoding into the trigger message through a two-step buffer adjustment and generate a telemetry response message to be reflected back to the central dispatch server.

[0015] Specifically, the central scheduling server includes a monitored node configuration table, a telemetry scheduling engine, and a telemetry response parsing engine;

[0016] The monitored node configuration table is used to record the network address, detection period, and data requirement description of each monitored node;

[0017] The telemetry scheduling engine copies the constructed trigger message template multiple times according to the detection cycle of each monitored node in the monitored node configuration table. It fills each copy with the destination address of the corresponding monitored node and the on-demand detection control field specific to that node, and writes the multi-field fingerprint into the trigger message payload area and sends it to each monitored node.

[0018] The telemetry response parsing engine is used to receive telemetry response messages reflected back from each monitored node, and then parse the response header and data items to obtain multi-dimensional network status data, which is provided to the digital twin network modeling engine.

[0019] Specifically, the on-demand detection control field adopts a bitmap encoding method, where each bit corresponds to a collectable data dimension. The control plane sets the value of each bit in the bitmap of the on-demand detection control field to specify the subset of data dimensions that each monitored node needs to transmit back in this period. When a bit is set to 1, it indicates that the control plane requests data for that dimension, and when it is set to 0, it indicates that data for that dimension is not needed in this period. The collectable data dimensions include inbound traffic statistics bits, outbound traffic statistics bits, round-trip delay detection bits, and implicit packet loss statistics bits. The trigger message is a UDP format message.

[0020] The telemetry scheduling engine generates an on-demand detection control field based on the data requirement description of each monitored node in the monitored node configuration table; the telemetry scheduling engine writes a multi-field fingerprint structure into the UDP payload area; the multi-field fingerprint includes a protocol magic word, a monotonically increasing session sequence number, a session token pre-shared with the monitored node, and a sending timestamp from the central scheduling server.

[0021] Specifically, the perception aggregation module also includes a global flow state table, an inbound active flow index stack, an outbound active flow index stack, a session configuration mapping table storing pre-shared session tokens, a timestamp mapping table for RTT probing, and a Tracepoint probe program deployed on Tracepoint for packet loss compensation. The global flow state table adopts a partitioned field design, with each entry divided into multiple logical partitions, including an inbound statistics partition, an outbound statistics partition, a latency measurement partition, and a packet loss compensation partition. Each level of eBPF probe program only performs atomic updates on the partition fields it is responsible for.

[0022] Specifically, the monitored node also obtains deep network status through kernel-resident round-trip latency detection and packet loss compensation based on kernel trace points;

[0023] The kernel-resident round-trip latency detection specifically includes: After the TC layer eBPF probe program intercepts TCP SYN probe packets carrying predefined port identifiers sent by the central scheduling server, the TC layer eBPF probe program obtains the next-hop MAC address of the monitored node at the peer end of each active flow through the eBPF FIB query helper function. For the TCP SYN probe packets, it sequentially performs source IP replacement with the local IP, destination IP replacement with the monitored node's IP, and destination MAC replacement with the next-hop MAC. Then, it calls the L3 layer incremental checksum helper function and the L4 layer incremental checksum helper function to perform incremental correction on the IP checksum and TCP checksum based on the old and new values ​​of each address field, generating a copy of the probe packet. Then, it sends the modified probe packet copy to the monitored node through the clone redirection helper function, simultaneously recording a nanosecond-level sending timestamp in the timestamp mapping table using the monitored node's IP as the key. The TC layer eBPF probe program residing in the kernel of the monitored node sequentially sends clone probe packets to multiple other monitored nodes, and then... SYN probe packets are discarded; after the XDP layer eBPF probe program captures the TCP RST response packets returned by the other monitored nodes, it retrieves the corresponding sending timestamp from the timestamp mapping table using the source IP address as the key, calculates the difference between the current receiving timestamp and the sending timestamp, shifts it to the right by one bit and takes half of the value as the one-way delay, writes it to the delay measurement partition of the corresponding entry in the global flow state table, and deletes the key corresponding to the source IP address from the timestamp mapping table.

[0024] Specifically, the packet loss compensation based on kernel tracepoints includes: deploying a Tracepoint probe program at the kernel packet release tracepoint. When the kernel packet release tracepoint is triggered, the Tracepoint probe program reads the discarded socket buffer pointer from the context, parses its IP header to obtain the source IP address through a kernel read helper function, checks the packet loss cause enumeration value to exclude normal release cases, and then searches the global flow state table using the source IP address as the key. If a match is found, an atomic increment operation is performed on the count value of the packet loss statistics partition of that entry to record the abnormal packet loss event. At the same time, if the inbound packet count of that entry is greater than zero, an atomic decrement operation is performed on the inbound packet count field to deduct the packets discarded inside the kernel protocol stack from the counted inbound packets, so that the packet count value reflects the actual number of valid packets that successfully passed through the kernel protocol stack.

[0025] Specifically, the multi-stage cascaded screening includes:

[0026] 1) Protocol layer pre-filtering: Check if the IP header protocol field of the inbound packet is UDP. If it does not match, it is immediately allowed to enter the normal protocol stack; if it matches, it enters the port number double verification.

[0027] 2) Dual port number verification: Check whether the UDP source port of the inbound packet is equal to the first predefined telemetry port and whether the UDP destination port is equal to the second predefined telemetry port. If not, allow the packet to pass; if so, proceed to payload fingerprint depth verification.

[0028] 3) Payload fingerprint depth verification: After port matching is successful, read the multi-field fingerprint at the fixed offset of the UDP payload of the inbound packet, and perform the protocol magic word fast filtering, pre-shared token authentication and sequence number anti-replay triple verification in sequence. If all three verifications are successful, the payload fingerprint depth verification is considered to be successful, and the on-demand probe control field extraction is entered.

[0029] 4) On-demand probe control field extraction: After the payload fingerprint depth verification is passed, the bitmap value of the on-demand probe control field is extracted from the trigger message payload and temporarily stored in the local variable of the eBPF probe program for use in subsequent state harvesting steps; the processed sequence number is updated, the inbound message is removed from the protocol stack path and its original kernel buffer is reused.

[0030] Specifically, the selective state harvesting by extracting the on-demand probe control field bitmap includes:

[0031] Pop the flow identifier key one by one from the outbound and inbound active flow index stacks. For each active entry in the global flow status table, read only the partition field corresponding to the dimension set to 1 in the bitmap, including: if the inbound traffic statistics position is 1, read the inbound statistics partition; if the outbound traffic statistics position is 1, read the outbound statistics partition; if the round-trip delay detection position is 1, read the delay measurement partition; if the implicit packet loss statistics position is 1, read the packet loss compensation partition. Perform an atomic swap operation on the read field to zero and delete the entry. The partition fields of unrequested dimensions remain unchanged and are not reset after reading. Their data continues to accumulate in subsequent cycles.

[0032] Specifically, the step of encoding and writing the aggregated statistical digest into the trigger message and generating a telemetry response message to be reflected back to the central scheduling server after two-step buffer adjustment includes:

[0033] The first step involves expanding the buffer according to the expected payload using a kernel buffer tail adjustment auxiliary function. First, a telemetry response header is written at the beginning of the payload. Then, each active flow statistical value is serialized and written to the payload area using a fixed-width compact packing structure format. Simultaneously, for each data item, a checksum contribution value is accumulated using a kernel differential checksum auxiliary function to obtain an aggregated statistical summary. The telemetry response header includes the monitored node identifier, acquisition timestamp, number of active flow entries, overflow flag, actual returned dimension bitmap, and session sequence number and sent timestamp backfill value extracted from the trigger message.

[0034] The second step involves writing all data items in the aggregated statistical summary and then shrinking the buffer to the exact length of the actual data. Finally, the length and checksum fields in the IP and UDP headers of the message are updated. The source and destination MAC addresses and IP addresses of the original trigger message are directly extracted and swapped, and the source and destination port numbers are set to predefined telemetry ports. Based on the swapped address information, the protocol headers of each layer of the message are rewritten to achieve route reversal and generate a telemetry response message. The telemetry response message is then sent back to the central scheduling server through the eBPF message redirection helper function.

[0035] Secondly, the present invention provides a kernel-mode on-demand telemetry method for digital twin networks, based on the kernel-mode on-demand telemetry system for digital twin networks described in the first aspect above, comprising the following steps:

[0036] S1. The central dispatch server constructs a trigger message carrying a multi-field fingerprint and an on-demand detection control field based on the configuration table of the monitored nodes and sends it to each monitored node.

[0037] S2. Deploy eBPF probe programs at multiple levels of the kernel protocol stack of the monitored node;

[0038] S3. The inbound message is filtered in multiple stages by the eBPF probe program to intercept the trigger message, and the bitmap of the on-demand probe control field is extracted for selective state harvesting. Then, the aggregated statistical summary is written into the trigger message after two-step buffer adjustment and a telemetry response message is generated and reflected back to the central dispatch server.

[0039] S4. The central dispatch server obtains multi-dimensional network status data based on the telemetry response messages reflected back by each monitored node and provides it to the digital twin network modeling engine.

[0040] This invention proposes a kernel-mode on-demand telemetry system for digital twin networks. It employs a "control plane triggering—data plane reflection" telemetry architecture, cooperating between a central scheduling server and distributed monitored nodes. The central scheduling server constructs UDP trigger messages carrying multi-field fingerprints and on-demand probe control fields according to a configuration strategy, and sends them to designated monitored nodes at a configurable frequency. The eBPF probe program on the monitored nodes intercepts the trigger messages in kernel mode, parses the on-demand probe control fields to determine the set of data dimensions to be transmitted back in the current cycle, harvests and encodes only the specific subset of indicators requested by the control plane, writes the aggregated statistical summary into the trigger message payload, and then reflects it back. The entire process does not transmit any copies of the original messages; a single telemetry response carries only tens to hundreds of bytes of aggregated statistical data, resulting in negligible bandwidth overhead. Simultaneously, it supports flexible on-demand control of the collection targets, collection frequency, and data dimensions by the control plane.

[0041] This invention provides a kernel-mode on-demand telemetry system for digital twin networks, which also has the following advantages:

[0042] (1) This system adopts an on-demand telemetry architecture that coordinates the central scheduling server and the distributed monitored nodes. The control plane flexibly specifies the collection frequency and data dimension requirements of each node through the on-demand detection control field bitmap. The monitored nodes only harvest and send back the specific subset of indicators requested by the control plane, avoiding the packet-by-packet cloning and tunnel forwarding of all original packets by the traffic mirroring scheme. This fundamentally solves the inherent defects of the mirroring scheme, which is proportional to the number of collected packets and cannot flexibly detect on demand. The telemetry response carries an aggregated statistical summary rather than a packet copy. The single response load is only tens to hundreds of bytes, and the bandwidth overhead is negligible.

[0043] (2) This system has two deep indicator detection mechanisms built into the kernel space of the monitored node: round-trip time delay detection and implicit packet loss compensation. RTT detection is completed in the kernel space by combining TCP SYN one-to-many cloning with FIB query and target-by-target checksum and incremental correction. Packet loss compensation is corrected by performing atomic subtraction operation on packet count through Tracepoint probe. These two types of deep indicators are key network status information that cannot be obtained by traffic mirroring schemes by copying the original packets. They provide irreplaceable data support for link quality modeling and anomaly reasoning of digital twin networks.

[0044] (3) This system designs a “trigger-reflection” telemetry protocol and introduces a multi-stage fingerprint authentication mechanism. It ensures the security of the telemetry channel through triple verification of protocol magic word fast filtering, pre-shared token authentication and serial number anti-replay. It uses the kernel buffer of the trigger message as the telemetry response carrier to avoid the constraint that eBPF cannot actively create messages, realizes full kernel-mode telemetry transmission with zero sockets and zero dynamic memory allocation. The telemetry latency is completely determined by the eBPF instruction execution time, and the jitter is significantly reduced.

[0045] (4) The selective state harvesting mechanism allows data in unrequested dimensions to continue to accumulate in subsequent cycles instead of being cleared and discarded. The stack type mapping table accurately enumerates the active stream to avoid the traversal limitations of the eBPF validator. The two-step buffer adjustment achieves precise control of the payload space. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1This is a schematic diagram of a full kernel-mode on-demand telemetry system for digital twin networks provided in an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of a full kernel-mode on-demand telemetry method for digital twin networks provided in an embodiment of the present invention; Detailed Implementation

[0049] The invention will be explained in detail through the following embodiments. The purpose of this invention is to protect all technical improvements within its scope. In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0050] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0051] Example 1

[0052] refer to Figure 1 This embodiment provides a kernel-mode on-demand telemetry system for digital twin networks, including a central dispatch server and multiple monitored nodes, and adopts an on-demand telemetry architecture in which the central dispatch server and distributed monitored nodes work together.

[0053] The central scheduling server is used to construct trigger messages carrying multi-field fingerprints and on-demand detection control fields according to the configuration table of monitored nodes and send them to each monitored node. It also obtains multi-dimensional network status data based on the telemetry response messages reflected back by each monitored node and provides it to the digital twin network modeling engine.

[0054] The central dispatch server includes a monitored node configuration table, a telemetry dispatch engine, and a telemetry response parsing engine.

[0055] The central dispatch server maintains the configuration table of the monitored nodes, which is used to record the network address, detection period and data requirement description of each monitored node.

[0056] The telemetry scheduling engine is used to construct a trigger message carrying a bitmap of multi-field fingerprints and on-demand detection control fields according to the configuration table of the monitored nodes and send it in a targeted manner according to the detection cycle of each node.

[0057] The telemetry scheduling engine constructs a trigger message carrying a multi-field fingerprint and an on-demand probe control field bitmap according to the configuration table of the monitored nodes. The trigger message template is copied multiple times and each copy is filled with the destination address and specific control field (the on-demand probe control field specific to that node) of the corresponding node. Then, it is sent in a targeted manner according to the probe cycle configured for each node. At the same time, a TCP SYN probe message is constructed and sent to each node to trigger kernel-resident RTT probe.

[0058] The monitored node's kernel space incorporates two deep indicator detection mechanisms: round-trip time (RTT) detection and implicit packet loss compensation. RTT detection is performed in kernel space by combining TCP SYN one-to-many cloning with FIB query and target-by-target checksum incremental correction. Packet loss compensation corrects statistical deviations by performing atomic decrement operations on packet counts using the Tracepoint probe.

[0059] The trigger message is a UDP format message; the telemetry scheduling engine constructs and sends the trigger message according to the following process:

[0060] The telemetry scheduling engine generates an on-demand detection control field based on the data requirement description of each monitored node in the monitored node configuration table. The on-demand detection control field adopts a bitmap encoding method, and each bit in the on-demand detection control field corresponds to a data dimension that can be collected, including but not limited to: inbound traffic statistics bit, outbound traffic statistics bit, round-trip delay detection bit, and implicit packet loss statistics bit. When a bit is set to 1, it indicates that the control plane requests data for that dimension, and when it is set to 0, it indicates that data for that dimension is not needed in this period. The control plane specifies the subset of data dimensions that each monitored node needs to return in this period by setting the value of each bit in the bitmap.

[0061] The telemetry scheduling engine writes a multi-field fingerprint structure into the UDP payload area. The multi-field fingerprint structure includes: a protocol magic word, a monotonically increasing session sequence number, a session token pre-shared with the monitored node, and a sending timestamp from the central scheduling server.

[0062] The telemetry scheduling engine copies the constructed trigger message template multiple times at the bottom layer according to the detection cycle of each node in the configuration table of the monitored nodes. It fills each copy with the destination address of the corresponding monitored node and the on-demand detection control field specific to that node, and writes the multi-field fingerprint into the trigger message payload area and sends it to each monitored node.

[0063] The telemetry response parsing engine is used to receive telemetry response messages reflected back from each monitored node, and parse the response header and data items to obtain multi-dimensional network status data, which is then provided to the digital twin network modeling engine.

[0064] The multidimensional network status data specifically includes network flow characteristics harvested on demand by each monitored node at the kernel level, mainly covering three dimensions: first, the traffic statistics dimension, including the number of inbound data packets, the number of outbound data packets, and the corresponding number of bytes for each active flow; second, the latency characteristic dimension, namely the precise latency obtained based on kernel-resident probing; and third, the abnormal state dimension, namely the underlying implicit packet loss records captured through kernel buffer tracking points.

[0065] The digital twin network modeling engine refers to cutting-edge network modeling algorithms in academia, such as RouteNet and xNet. These engines can infer the future state of the network based on real-time multi-dimensional network status data to facilitate network operation and maintenance (such as link throughput, utilization, and network latency), thereby achieving accurate prediction and early warning of potential congestion or failures. By deeply integrating real-time sensing data with digital twin models, the system can dynamically optimize routing strategies and resource allocation, significantly improving the intelligence and response efficiency of network operation and maintenance, ensuring business continuity, reducing operating costs, and ultimately building a next-generation intelligent network architecture with self-sensing and self-decision-making capabilities.

[0066] The monitored node includes a perception aggregation module and a trigger reflection module;

[0067] The perception aggregation module resides in the kernel space of each monitored node and includes eBPF probe programs deployed at different levels of the kernel protocol stack, a global flow state table with partitioned field design, an active flow index stack in different directions, a session configuration mapping table storing pre-shared session tokens, a timestamp mapping table for RTT detection, and a Tracepoint probe program deployed at Tracepoint for packet loss compensation.

[0068] The monitored node deploys eBPF probe programs at at least two different levels of the kernel protocol stack. Each level of eBPF probe program writes network state data into the same entry of a global flow state table shared in kernel space through atomic operations. The global flow state table adopts a partitioned field design and uses the flow identifier key as the index key. At the same time, a stack-type eBPF mapping table is used to record active flow identifier keys.

[0069] In this embodiment, the IP address (flow identifier key) of each monitored node is used as the index key (flow identifier key, such as __be32 key in the code) of the kernel eBPF mapping table.

[0070] Specifically, the global flow status table adopts a partitioned field design, with each entry divided into multiple logical partitions, including inbound statistics partition, outbound statistics partition, latency measurement partition, and packet loss compensation partition. Each level of eBPF probe program only performs atomic updates on the partition fields it is responsible for, and write operations on different partitions do not interfere with each other.

[0071] Simultaneously, two stack types of eBPF mapping tables are set up: the inbound active flow index stack and the outbound active flow index stack. When a flow identifier key triggers the creation of the corresponding statistical table entry for the first time, the eBPF probe program pushes the flow identifier key onto the active flow index stack of the corresponding direction.

[0072] In this embodiment, the different layers include, but are not limited to, the XDP layer and the TC layer;

[0073] Understandably, each network layer's mount point (such as the TC Ingress hook, TC Egress hook, and XDP hook) deploys its own main eBPF probe program. These independently running probe programs achieve cross-layer data aggregation and collaboration by sharing the same underlying kernel mapping table (such as the global_flow_map in the code).

[0074] The monitored node also obtains deep network status that cannot be provided by the traffic mirroring scheme through two kernel-state deep indicator detection mechanisms;

[0075] Specifically, the two kernel-mode deep indicator detection mechanisms are automatically triggered and executed by the eBPF probe program residing in the kernel of the monitored node. They require no user-mode process intervention. When the eBPF probe program intercepts a specific packet or the system triggers a packet loss tracing point, it automatically invokes modification instructions in the very low-level kernel context and notifies the kernel to physically discard the packet by returning a specific opcode (such as TC_ACT_SHOT).

[0076] The two kernel-mode deep indicator detection mechanisms include kernel-resident round-trip latency detection and packet loss compensation based on kernel tracking points:

[0077] 1) Kernel-resident round-trip latency detection: After the TC layer eBPF probe program intercepts the TCP SYN probe packet carrying the predefined port identifier sent by the central scheduling server, the TC layer eBPF probe program obtains the next-hop MAC address of the monitored node at the other end of each active flow through the eBPF FIB query helper function. After performing source IP replacement with local IP, destination IP replacement with monitored node IP, and destination MAC replacement with next-hop MAC on the TCP SYN probe packet in sequence, the L3 layer incremental checksum helper function is called. The L4 layer incremental checksum helper function performs incremental correction on the IP checksum and TCP checksum based on the old and new values ​​of each address field to generate a copy of the probe packet. Then, the modified probe packet copy is sent to the monitored node through the clone redirection helper function. At the same time, the nanosecond-level sending timestamp is recorded in the timestamp mapping table with the monitored node IP as the key.

[0078] The active flow refers to the network communication connection that actually sends and receives data packets with the monitored node during the current telemetry detection period, while the monitored node at the other end of each active flow refers to other internal nodes in the network that are exchanging network data with the currently monitored node.

[0079] The TC layer eBPF probe program residing in the kernel of the monitored node sends clone probe packets to multiple other monitored nodes in the same manner, and then discards the original TCP SYN probe packets. After the XDP layer eBPF probe program captures the TCP RST response packets returned by the other monitored nodes, it retrieves the corresponding sending timestamp from the timestamp mapping table using the source IP address as the key, calculates the difference between the current receiving timestamp and the sending timestamp, shifts it to the right by one bit and takes half of the value as the one-way delay, writes it to the delay measurement partition of the corresponding entry in the global flow state table, and deletes the key (the key corresponding to the source IP address) from the timestamp mapping table.

[0080] After cloning and sending all probe packet copies, the TC layer eBPF probe program will automatically take over the processing flow of the original packets. It does not require the participation of user-space processes, but directly returns a specific drop opcode (such as TC_ACT_SHOT) to the underlying kernel protocol stack. After the operating system kernel receives the code instruction, it will directly release the original TCPSYN probe packets at the network card level.

[0081] The eBPF FIB lookup helper function is a built-in interface function `bpf_fib_lookup` provided by the Linux kernel to the eBPF probe program. This function allows the eBPF probe program to directly query the kernel's forwarding information database (FIB) at the network card driver layer. The probe only needs to pass in the destination IP address to be reflected, and this interface can accurately return the corresponding outgoing interface index (ifindex), next-hop MAC address, and local source MAC address within a very short period of time, enabling the probe to directly reconstruct the underlying Ethernet frame header locally.

[0082] The L3 layer incremental checksum auxiliary function is specifically the built-in eBPF functions bpf_l3_csum_replace and bpf_csum_diff; the L4 layer incremental checksum auxiliary function is specifically the built-in eBPF functions bpf_l4_csum_replace and bpf_csum_diff.

[0083] The L3 and L4 layers use the same function bpf_csum_diff;

[0084] bpf_l3_csum_replace is an L3 layer incremental checksum helper function, specifically used to incrementally correct the IPv4 header checksum in place after modifying network layer fields such as IP address;

[0085] bpf_l4_csum_replace is an L4 layer incremental checksum helper function, specifically used to incrementally correct the checksum of the TCP / UDP transport layer in place after modifying the port number or pseudo header;

[0086] bpf_csum_diff is a general memory differential checksum helper function in eBPF. It is used to calculate the checksum contribution difference of the large block of newly added payload memory when the packet is dynamically expanded and written to the aggregate statistical digest, so as to complete the accurate correction of the final checksum field.

[0087] When the eBPF probe program directly modifies the IP address, total length, or payload content of a data packet, in order to avoid the high CPU overhead of recalculating the checksum by re-traversing the entire message, the above function uses the algebraic operation characteristics of one's complement summation to directly calculate the difference between the old binary value and the updated new binary value of the overwritten field. It accurately corrects the IP header checksum field and the transport layer checksum field in O(1) time complexity, ensuring that the reflected packet is not discarded by the network card's hardware check interception mechanism.

[0088] The clone redirection helper function is a mature low-level technology of the Linux kernel eBPF subsystem, corresponding to the bpf_redirect or bpf_clone_redirect functions in the Linux eBPF API. Its essence is to directly redirect the current socket buffer pointer to the specified network interface card (NIC) transmit queue. This scheme utilizes this mechanism to perform retransmission of processed data packets at the TC layer, either along the original path or across NICs, thereby achieving a complete bypass of the host TCP / IP protocol stack.

[0089] 2) Kernel tracepoint-based packet loss compensation: A tracepoint probe program is deployed at the kernel packet release tracepoint. When the kernel packet release tracepoint is triggered, the tracepoint probe program reads the discarded socket buffer pointer from the context, parses its IP header to obtain the source IP address through the kernel read helper function, checks the packet loss cause enumeration value to exclude normal release cases, and then looks up the global flow state table with the source IP address as the key. If a match is found, the count value of the packet loss statistics partition of that table entry is atomically incremented to record the abnormal packet loss event. At the same time, if the inbound packet count of that table entry is greater than zero, the inbound packet count field is atomically decremented to deduct the packets dropped inside the kernel protocol stack from the counted inbound packets, so that the packet count value reflects the number of valid packets that actually successfully passed through the kernel protocol stack.

[0090] Understandably, the Tracepoint probe is one type of eBPF probe.

[0091] The kernel read helper function refers to the basic underlying mechanism of using eBPF Tracepoint to listen to kernel packet release points, parse skb pointers to obtain IP headers, and read packet loss reasons. This is an existing technology (specifically, the kfree_skb Tracepoint hook provided by Linux). The innovation of this invention is to deeply integrate it with the eBPF full kernel-state telemetry system: when the probe captures abnormal packet loss, it no longer reports logs to user space, but directly retrieves the global flow state table at the kernel level with the source IP as the key, and performs an atomic reverse deduction operation on the inbound packet count, thereby realizing full kernel-state flow state closed-loop compensation.

[0092] The trigger reflection module resides in the kernel space of each monitored node. It is used to intercept legitimate trigger messages after performing multi-stage cascading filtering on inbound messages, extract the bitmap of the on-demand detection control field, and then perform selective state harvesting only on the request dimension. After a two-step buffer adjustment, the aggregated statistical summary is encoded and written into the trigger message and a telemetry response message is generated and reflected back to the central scheduling server. The entire process does not involve user-space participation and does not transmit copies of the original messages.

[0093] The eBPF probe program of the monitored node performs multi-stage cascaded filtering on inbound packets in kernel mode to intercept the triggering packets. The multi-stage cascaded filtering includes protocol layer pre-filtering, port number dual verification, payload fingerprint depth verification, and on-demand probe control field extraction. After successful interception, the on-demand probe control field bitmap is extracted. Only the global flow state table partition field corresponding to the dimension set to 1 in the bitmap is subjected to atomic read-after-reset operation and selective state harvesting. The partition field of the unrequested dimension set to 0 in the bitmap is not reset.

[0094] The multi-stage cascaded screening specifically includes:

[0095] 1) Protocol layer pre-filtering: Check if the IP header protocol field of the inbound packet is UDP. If it does not match, it is immediately allowed to enter the normal protocol stack; if it matches, it enters the port number double verification.

[0096] 2) Dual port number verification: Check whether the UDP source port of the inbound packet is equal to the first predefined telemetry port and whether the UDP destination port is equal to the second predefined telemetry port. If not, allow the packet to pass; if so, proceed to payload fingerprint depth verification.

[0097] The first predefined telemetry port and the second predefined telemetry port are two predefined port numbers (such as the source port 43210 of the central server and the receiving port 12113 of the monitored node in the code). They are service-specific ports that are pre-allocated during the initialization of the telemetry system and dynamically issued through the BPF Map.

[0098] 3) Payload fingerprint depth verification: After port matching is successful, read the multi-field fingerprint at the fixed offset of the UDP payload of the inbound packet, and perform the protocol magic word fast filtering, pre-shared token authentication and sequence number anti-replay triple verification in sequence. If all three verifications are successful, the payload fingerprint depth verification is considered to be successful, and the on-demand probe control field extraction is entered.

[0099] The triple verification includes:

[0100] a) Fast filtering of protocol magic words: The protocol magic word of the multi-field fingerprint is strictly compared with the predefined protocol identifier fixed by the eBPF probe program at compile time. If the two values ​​match, the verification passes; otherwise, the verification fails, indicating that the packet does not come from a legitimate trigger command from the central scheduling server (it may be background noise in the network or malicious port scanning). At this time, the eBPF probe program will immediately return the drop opcode (TC_ACT_SHOT) to the kernel protocol stack. After receiving the command, the underlying network card driver will silently drop the illegal packet.

[0101] b) Pre-shared token authentication: The pre-shared session token is retrieved from the session configuration mapping table and compared byte-by-byte with the session token field of the multi-field fingerprint; if all bytes match, the verification passes; otherwise, the verification fails, and the eBPF probe program immediately returns a drop opcode (TC_ACT_SHOT) to the kernel protocol stack. Upon receiving this instruction, the underlying network card driver will silently discard the illegal packet.

[0102] The pre-shared session token is an encrypted credential that is pre-shared between the central scheduling server and the monitored nodes and can be rotated periodically. Even if an attacker forges the correct port and magic word, as long as he does not obtain the current legitimate session token through the security control plane, his forged message will still be discarded.

[0103] c) Sequence number replay prevention: Check whether the session sequence number is strictly greater than the most recently processed sequence number maintained in kernel mode. If it is greater, the verification is considered to have passed. If it is not greater, it is determined to be an expired or replayed message and is discarded, indicating that the verification has failed.

[0104] 4) On-demand probe control field extraction: After the payload fingerprint depth verification is passed, the bitmap value of the on-demand probe control field is extracted from the trigger message payload and temporarily stored in the local variable of the eBPF probe program for use in subsequent state harvesting steps; the processed sequence number is updated, the inbound message is removed from the protocol stack path and its original kernel buffer is reused.

[0105] The protocol stack path refers to the standard kernel network protocol processing flow of the Linux system (NIC driver -> TC layer -> IP layer -> UDP / TCP layer -> user space). Removing the packet from the protocol stack path means that after the eBPF probe program processes the trigger instruction at the lower level (TC layer), it directly sends it back to the network cable through redirection or discards it, preventing the telemetry packet from continuing to consume the local CPU protocol stack resources.

[0106] The above steps intercepted the trigger message from multiple dimensions, including protocol type and port number verification, protocol magic word comparison, pre-shared token verification, and sequence number anti-replay check.

[0107] After extracting the on-demand probe control field bitmap, selective state harvesting is performed only on the requested dimension, specifically including:

[0108] One by one, pop the flow identifier key from the outbound active flow index stack and the inbound active flow index stack. For each active entry in the global flow status table, only read the partition field corresponding to the dimension set to 1 in the bitmap, including: if the inbound traffic statistics position is 1, read the inbound statistics partition; if the outbound traffic statistics position is 1, read the outbound statistics partition; if the round-trip delay detection position is 1, read the delay measurement partition; if the implicit packet loss statistics position is 1, read the packet loss compensation partition. Perform an atomic swap operation on the read field to zero and delete the entry. The partition fields of unrequested dimensions remain unchanged and are not reset after reading. Their data continues to accumulate in subsequent cycles. The above is the harvesting process.

[0109] The TC probe program maintains an active flow index stack. When the probe first captures a new network flow and creates a record in the global state table, it pushes the flow's identifier key onto the active flow index stack. When a telemetry command is received, these identifier keys are popped from the active flow index stack in sequence to accurately locate and harvest the corresponding active flow data (active entries).

[0110] The harvest statistics are temporarily stored in the global flow status table. When the total number of temporary records exceeds the single message payload capacity limit, the records are truncated according to the priority of delay records, followed by packet loss records, and finally traffic statistics, and an overflow flag is set.

[0111] Specifically, this includes: using a multi-round conditional traversal mechanism. During the harvesting phase, the probe will traverse the global flow state table in three rounds in sequence: the first round specifically extracts high-priority flow entries with round-trip delay records, the second round extracts medium-priority entries with hidden packet loss records, and the last round processes basic traffic statistics entries that only contain regular packet sending and receiving quantities.

[0112] During these three rounds of top-down sequential loading, the system will check in real time whether the amount of data currently loaded has reached the physical capacity limit of a single message payload (determined by the network's maximum transmission unit MTU). Once the capacity threshold is reached in any round of traversal, the probe will immediately terminate the entire harvesting cycle and set the overflow flag, thereby naturally truncating the low-priority statistics that have not yet been loaded by utilizing the physical memory boundary.

[0113] The harvested statistics refer to the data obtained by passively monitoring real business network traffic through the TC layer eBPF probe program, XDP layer eBPF probe program, and Tracepoint probe program mentioned above, extracting features such as the number of data packets and bytes in real time, and continuously accumulating them. Normally, these data are silently stored in the kernel's global flow state table, and will only be accurately extracted and packaged and transmitted back when a telemetry trigger command is received.

[0114] The two-step buffer adjustment process involves encoding the aggregated statistical summary and writing it into the trigger message, then generating a telemetry response message based on FIB routing queries and reflecting it back. Specifically, this includes:

[0115] The first step involves expanding the buffer according to the expected payload using a kernel buffer tail adjustment auxiliary function. First, a telemetry response header is written at the beginning of the payload. Then, each active flow statistical value is serialized and written to the payload area using a fixed-width packed structure. Simultaneously, the checksum contribution value is accumulated for each data item using a kernel differential checksum auxiliary function to obtain an aggregated statistical summary. The telemetry response header includes the monitored node identifier, acquisition timestamp, number of active flow entries, overflow flag, actual returned dimension bitmap, and session sequence number and sent timestamp backfill value extracted from the trigger message.

[0116] The specific method of accumulating the checksum contribution value through the kernel differential checksum auxiliary function is as follows: the kernel differential checksum contribution value (i.e., the difference) generated by the newly added data block is calculated separately and quickly using the underlying function bpf_csum_diff provided by eBPF. Then, this difference is directly added to the original TCP / UDP checksum, which can maintain the legitimacy of network packets with extremely low performance loss.

[0117] The kernel buffer tail adjustment helper function corresponds to bpf_skb_change_tail() in the Linux eBPF API. It allows the payload memory of network packets to be dynamically stretched or shrunk directly in kernel mode, which is used to forcibly expand the physical memory space for aggregated data.

[0118] In the above text, `packed` means compact packing, which is a compiler directive in the C language that tells the compiler to cancel automatic alignment padding for structure members.

[0119] The second step involves writing all data items in the aggregated statistical summary and then shrinking the buffer to the exact length of the actual data. Finally, the length and checksum fields in the IP and UDP headers of the message are updated. The source and destination MAC addresses and IP addresses of the original trigger message are directly extracted and swapped, and the source and destination port numbers are set to predefined telemetry ports. Based on the swapped address information, the protocol headers of each layer of the message are rewritten to achieve route reversal and generate a telemetry response message. The telemetry response message is then sent back to the central scheduling server through the eBPF message redirection helper function.

[0120] It is understandable that the aggregated statistical digest is not a data structure generated independently of the message. Instead, it is a continuous data block dynamically assembled in-situ within the payload memory of the trigger message by the probe program using kernel helper functions. After the probe program expands the buffer, the telemetry response header written at the beginning, along with the subsequent continuous writing of the statistics of each active stream in a compact structure format, constitutes the aggregated statistical digest by tightly concatenating these two parts of data in physical memory. The first step of serialization writing is the process of generating the aggregated statistical digest. Since the first step expands the buffer to the maximum expected amount, the second step of shrinking the buffer to the precise length of the actual data essentially defines the precise physical boundaries of the newly generated aggregated statistical digest, removing excess blank memory at the end, making it a compact and complete payload that is transmitted back with the message.

[0121] All processing on the monitored node side is completed in kernel mode, without the involvement of user-mode processes, without creating sockets, and without allocating dynamic memory; the telemetry response carries an aggregated statistical summary rather than a copy of the original message.

[0122] This embodiment provides a kernel-mode on-demand telemetry system for digital twin networks. It employs a "control plane triggering—data plane reflection" telemetry architecture, where a central scheduling server collaborates with distributed monitored nodes. The central scheduling server constructs UDP trigger messages carrying multi-field fingerprints and on-demand probe control fields according to a configuration strategy, and sends them to designated monitored nodes at a configurable frequency. The eBPF probe program on the monitored nodes intercepts the trigger messages in kernel mode, parses the on-demand probe control fields to determine the set of data dimensions to be transmitted back in the current cycle, harvests and encodes only the specific subset of indicators requested by the control plane, writes the aggregated statistical summary into the trigger message payload, and then reflects it back. The entire process does not transmit any copies of the original messages; a single telemetry response carries only tens to hundreds of bytes of aggregated statistical data, resulting in negligible bandwidth overhead. Simultaneously, it supports flexible on-demand control of the collection targets, collection frequency, and data dimensions by the control plane.

[0123] This embodiment provides a kernel-mode on-demand telemetry system for digital twin networks, which also has the following beneficial effects:

[0124] (1) This system adopts an on-demand telemetry architecture that coordinates the central scheduling server and the distributed monitored nodes. The control plane flexibly specifies the collection frequency and data dimension requirements of each node through the on-demand detection control field bitmap. The monitored nodes only harvest and send back the specific subset of indicators requested by the control plane, avoiding the packet-by-packet cloning and tunnel forwarding of all original packets by the traffic mirroring scheme. This fundamentally solves the inherent defects of the mirroring scheme, which is proportional to the number of collected packets and cannot flexibly detect on demand. The telemetry response carries an aggregated statistical summary rather than a packet copy. The single response load is only tens to hundreds of bytes, and the bandwidth overhead is negligible.

[0125] (2) This system has two deep indicator detection mechanisms built into the kernel space of the monitored node: round-trip time delay detection and implicit packet loss compensation. RTT detection is completed in the kernel space by combining TCP SYN one-to-many cloning with FIB query and target-by-target checksum and incremental correction. Packet loss compensation is corrected by performing atomic subtraction operation on packet count through Tracepoint probe. These two types of deep indicators are key network status information that cannot be obtained by traffic mirroring schemes by copying the original packets. They provide irreplaceable data support for link quality modeling and anomaly reasoning of digital twin networks.

[0126] (3) This system designs a “trigger-reflection” telemetry protocol and introduces a multi-stage fingerprint authentication mechanism. It ensures the security of the telemetry channel through triple verification of protocol magic word fast filtering, pre-shared token authentication and serial number anti-replay. It uses the kernel buffer of the trigger message as the telemetry response carrier to avoid the constraint that eBPF cannot actively create messages, realizes full kernel-mode telemetry transmission with zero sockets and zero dynamic memory allocation. The telemetry latency is completely determined by the eBPF instruction execution time, and the jitter is significantly reduced.

[0127] (4) The selective state harvesting mechanism allows data in unrequested dimensions to continue to accumulate in subsequent cycles instead of being cleared and discarded. The stack type mapping table accurately enumerates the active stream to avoid the traversal limitations of the eBPF validator. The two-step buffer adjustment achieves precise control of the payload space.

[0128] Example 2

[0129] refer to Figure 2 This embodiment provides a full kernel-mode on-demand telemetry method for digital twin networks, based on the full kernel-mode on-demand telemetry system for digital twin networks described in Embodiment 1, including the following steps:

[0130] S1. The central dispatch server constructs a trigger message carrying a multi-field fingerprint and an on-demand detection control field based on the configuration table of the monitored nodes and sends it to each monitored node.

[0131] S11. Generate on-demand detection control fields based on the data requirement descriptions of each monitored node in the monitored node configuration table;

[0132] The on-demand detection control field adopts a bitmap encoding method, where each bit corresponds to a data dimension that can be collected, including but not limited to: inbound traffic statistics bit, outbound traffic statistics bit, round-trip delay detection bit, and implicit packet loss statistics bit; when a bit is set to 1, it indicates that the control plane requests data for that dimension, and when it is set to 0, it indicates that data for that dimension is not needed in this period.

[0133] S12. Construct a multi-field fingerprint for the trigger message payload area; the multi-field fingerprint includes a protocol magic word, a monotonically increasing session sequence number, a session token pre-shared with the monitored node, and a sending timestamp from the central scheduling server;

[0134] S13. The telemetry scheduling engine copies the constructed trigger message template multiple times at the bottom layer according to the detection cycle of each node in the configuration table of the monitored nodes. It fills each copy with the destination address of the corresponding monitored node and the on-demand detection control field specific to that node, and writes the multi-field fingerprint into the trigger message payload area and sends it to each monitored node.

[0135] Different monitored nodes can be configured with different detection cycles and data dimension requirements. Different data dimension requirements correspond to different data requirement descriptions, enabling differentiated on-demand telemetry.

[0136] S2. Deploy eBPF probe programs at multiple levels of the kernel protocol stack of the monitored node;

[0137] S3. The inbound message is filtered in multiple stages by the eBPF probe program to intercept the trigger message, and the bitmap of the on-demand probe control field is extracted for selective state harvesting. Then, the aggregated statistical summary is written into the trigger message after two-step buffer adjustment and a telemetry response message is generated and reflected back to the central dispatch server.

[0138] The eBPF probe program of the monitored node performs multi-stage cascaded filtering on inbound packets in kernel mode to intercept the triggering packets. After successful interception, it extracts the bitmap of the on-demand probe control field and performs atomic read-after-reset operation only on the global flow state table partition field corresponding to the dimension set to 1 in the bitmap to selectively harvest the state. The partition field of the unrequested dimension is not reset.

[0139] The multi-stage cascaded screening includes protocol layer pre-filtering, dual port number verification, payload fingerprint depth verification, and on-demand detection control field extraction.

[0140] Each level of eBPF probe program writes network state data into the same entry of the kernel-mode shared global flow state table through atomic operations, and at the same time uses a stack-type eBPF mapping table to record active flow identifier keys; the global flow state table adopts a partitioned field design, and the global flow state table uses the flow identifier key as the index key;

[0141] The multi-stage cascaded screening specifically includes:

[0142] 1) Protocol layer pre-filtering: Check if the IP header protocol field of the inbound packet is UDP. If it does not match, it is immediately allowed to enter the normal protocol stack; if it matches, it enters the port number double verification.

[0143] 2) Dual port number verification: Check whether the UDP source port of the inbound packet is equal to the first predefined telemetry port and whether the UDP destination port is equal to the second predefined telemetry port. If not, allow the packet to pass; if so, proceed to payload fingerprint depth verification.

[0144] 3) Payload fingerprint depth verification: After port matching is successful, read the multi-field fingerprint at the fixed offset of the UDP payload of the inbound packet, and perform triple verification in sequence. If all three verifications are successful, the payload fingerprint depth verification is considered successful, and the on-demand probe control field extraction is initiated.

[0145] 4) On-demand probe control field extraction: After the payload fingerprint depth verification is passed, the bitmap value of the on-demand probe control field is extracted from the trigger message payload and temporarily stored in the local variable of the eBPF probe program for use in subsequent state harvesting steps; the processed sequence number is updated, the inbound message is removed from the protocol stack path and its original kernel buffer is reused.

[0146] S4. The central dispatch server obtains multi-dimensional network status data based on the telemetry response messages reflected back by each monitored node and provides it to the digital twin network modeling engine.

[0147] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 A process, multiple processes, and / or boxes Figure 1 Devices that specify the functions in one or more boxes.

[0148] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction device, which is implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0149] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0150] The parts of this invention not described in detail are prior art. It will be apparent to those skilled in the art that this invention is not limited to the details of the above exemplary embodiments, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects, and are intended to encompass all changes falling within the meaning and scope of equivalents within this invention.

Claims

1. A kernel-state on-demand telemetry system for digital twin networks, characterized in that, This includes a central dispatch server and multiple monitored nodes; The central scheduling server is used to construct trigger messages carrying multi-field fingerprints and on-demand detection control fields according to the configuration table of monitored nodes and send them to each monitored node. It also obtains multi-dimensional network status data based on the telemetry response messages reflected back by each monitored node and provides it to the digital twin network modeling engine. The monitored node includes a perception aggregation module and a trigger reflection module residing in its kernel space; The perception aggregation module is used to deploy eBPF probe programs at multiple layers of the kernel protocol stack of the monitored node. The trigger reflection module is used to perform multi-stage cascaded filtering on inbound messages through the eBPF probe program to intercept the trigger message, extract the on-demand detection control field bitmap for selective state harvesting, and then write the aggregated statistical summary encoding into the trigger message through a two-step buffer adjustment and generate a telemetry response message to be reflected back to the central dispatch server.

2. The system according to claim 1, characterized in that, The central dispatch server includes a monitored node configuration table, a telemetry dispatch engine, and a telemetry response parsing engine. The monitored node configuration table is used to record the network address, detection period, and data requirement description of each monitored node; The telemetry scheduling engine copies the constructed trigger message template multiple times according to the detection cycle of each monitored node in the monitored node configuration table. It fills each copy with the destination address of the corresponding monitored node and the on-demand detection control field specific to that node, and writes the multi-field fingerprint into the trigger message payload area and sends it to each monitored node. The telemetry response parsing engine is used to receive telemetry response messages reflected back from each monitored node, and then parse the response header and data items to obtain multi-dimensional network status data, which is provided to the digital twin network modeling engine.

3. The system according to claim 2, characterized in that, The on-demand detection control field uses a bitmap encoding method, where each bit corresponds to a collectable data dimension. The control plane sets the value of each bit in the bitmap of the on-demand detection control field to specify the subset of data dimensions that each monitored node needs to transmit back in this period. When a bit is set to 1, it indicates that the control plane requests data for that dimension, and when it is set to 0, it indicates that data for that dimension is not needed in this period. The collectable data dimensions include inbound traffic statistics bits, outbound traffic statistics bits, round-trip delay detection bits, and implicit packet loss statistics bits. The trigger message is a UDP format message. The telemetry scheduling engine generates an on-demand detection control field based on the data requirement description of each monitored node in the monitored node configuration table; the telemetry scheduling engine writes a multi-field fingerprint structure into the UDP payload area; the multi-field fingerprint includes a protocol magic word, a monotonically increasing session sequence number, a session token pre-shared with the monitored node, and a sending timestamp from the central scheduling server.

4. The system according to claim 3, characterized in that, The perception aggregation module also includes a global flow status table, an inbound active flow index stack, an outbound active flow index stack, a session configuration mapping table storing pre-shared session tokens, a timestamp mapping table for RTT probing, and a Tracepoint probe program deployed on Tracepoint for packet loss compensation. The global flow status table adopts a partitioned field design, with each entry divided into multiple logical partitions, including an inbound statistics partition, an outbound statistics partition, a latency measurement partition, and a packet loss compensation partition. Each level of eBPF probe program only performs atomic updates on the partition fields it is responsible for.

5. The system according to claim 4, characterized in that, The monitored node also obtains deep network status through kernel-resident round-trip latency detection and packet loss compensation based on kernel trace points; The kernel-resident round-trip latency detection specifically includes: After the TC layer eBPF probe program intercepts TCP SYN probe packets carrying predefined port identifiers sent by the central scheduling server, the TC layer eBPF probe program obtains the next-hop MAC address of the monitored node at the peer end of each active flow through the eBPF FIB query helper function. For the TCP SYN probe packets, it sequentially performs source IP replacement with the local IP, destination IP replacement with the monitored node's IP, and destination MAC replacement with the next-hop MAC. Then, it calls the L3 layer incremental checksum helper function and the L4 layer incremental checksum helper function to perform incremental correction on the IP checksum and TCP checksum based on the old and new values ​​of each address field, generating a copy of the probe packet. Then, it sends the modified probe packet copy to the monitored node through the clone redirection helper function, simultaneously recording a nanosecond-level sending timestamp in the timestamp mapping table using the monitored node's IP as the key. The TC layer eBPF probe program residing in the kernel of the monitored node sequentially sends clone probe packets to multiple other monitored nodes, and then... SYN probe packets are discarded; after the XDP layer eBPF probe program captures the TCP RST response packets returned by the other monitored nodes, it retrieves the corresponding sending timestamp from the timestamp mapping table using the source IP address as the key, calculates the difference between the current receiving timestamp and the sending timestamp, shifts it to the right by one bit and takes half of the value as the one-way delay, writes it to the delay measurement partition of the corresponding entry in the global flow state table, and deletes the key corresponding to the source IP address from the timestamp mapping table.

6. The system according to claim 5, characterized in that, The packet loss compensation based on kernel tracepoints specifically includes: deploying a Tracepoint probe program at the kernel packet release tracepoint. When the kernel packet release tracepoint is triggered, the Tracepoint probe program reads the discarded socket buffer pointer from the context, parses its IP header to obtain the source IP address through a kernel read helper function, checks the packet loss cause enumeration value to exclude normal release cases, and then searches the global flow state table using the source IP address as the key. If a match is found, an atomic increment operation is performed on the count value of the packet loss statistics partition of that table entry to record the abnormal packet loss event. At the same time, if the inbound packet count of that table entry is greater than zero, an atomic decrement operation is performed on the inbound packet count field to deduct the packets discarded inside the kernel protocol stack from the already counted inbound packets, so that the packet count value reflects the actual number of valid packets that successfully passed through the kernel protocol stack.

7. The system according to claim 6, characterized in that, The multi-stage cascaded screening specifically includes: 1) Protocol layer pre-filtering: Check if the IP header protocol field of the inbound packet is UDP. If it does not match, it is immediately allowed to enter the normal protocol stack; if it matches, it enters the port number double verification. 2) Dual port number verification: Check whether the UDP source port of the inbound packet is equal to the first predefined telemetry port and whether the UDP destination port is equal to the second predefined telemetry port. If not, allow the packet to pass; if so, proceed to payload fingerprint depth verification. 3) Payload fingerprint depth verification: After port matching is successful, read the multi-field fingerprint at the fixed offset of the UDP payload of the inbound packet, and perform the protocol magic word fast filtering, pre-shared token authentication and sequence number anti-replay triple verification in sequence. If all three verifications are successful, the payload fingerprint depth verification is considered to be successful, and the on-demand probe control field extraction is entered. 4) On-demand probe control field extraction: After the payload fingerprint depth verification is passed, the bitmap value of the on-demand probe control field is extracted from the trigger message payload and temporarily stored in the local variable of the eBPF probe program for use in subsequent state harvesting steps; the processed sequence number is updated, the inbound message is removed from the protocol stack path and its original kernel buffer is reused.

8. The system according to claim 7, characterized in that, The selective state harvesting process by extracting the on-demand probe control field bitmap specifically includes: Pop the flow identifier key one by one from the outbound and inbound active flow index stacks. For each active entry in the global flow status table, read only the partition field corresponding to the dimension set to 1 in the bitmap, including: if the inbound traffic statistics position is 1, read the inbound statistics partition; if the outbound traffic statistics position is 1, read the outbound statistics partition; if the round-trip delay detection position is 1, read the delay measurement partition; if the implicit packet loss statistics position is 1, read the packet loss compensation partition. Perform an atomic swap operation on the read field to zero and delete the entry. The partition fields of unrequested dimensions remain unchanged and are not reset after reading. Their data continues to accumulate in subsequent cycles.

9. The system according to claim 1, characterized in that, The process of adjusting the buffer in two steps, encoding the aggregated statistical summary, writing it into the trigger message, and generating a telemetry response message to be reflected back to the central dispatch server specifically includes: The first step involves expanding the buffer according to the expected payload using a kernel buffer tail adjustment auxiliary function. First, a telemetry response header is written at the beginning of the payload. Then, each active flow statistical value is serialized and written to the payload area using a fixed-width compact packing structure format. Simultaneously, for each data item, a checksum contribution value is accumulated using a kernel differential checksum auxiliary function to obtain an aggregated statistical summary. The telemetry response header includes the monitored node identifier, acquisition timestamp, number of active flow entries, overflow flag, actual returned dimension bitmap, and session sequence number and sent timestamp backfill value extracted from the trigger message. The second step involves writing all data items in the aggregated statistical summary and then shrinking the buffer to the exact length of the actual data. Finally, the length and checksum fields in the IP and UDP headers of the message are updated. The source and destination MAC addresses and IP addresses of the original trigger message are directly extracted and swapped, and the source and destination port numbers are set to predefined telemetry ports. Based on the swapped address information, the protocol headers of each layer of the message are rewritten to achieve route reversal and generate a telemetry response message. The telemetry response message is then sent back to the central scheduling server through the eBPF message redirection helper function.

10. A full kernel-mode on-demand telemetry method for digital twin networks, characterized in that, The kernel-mode on-demand telemetry system for digital twin networks based on any one of claims 1-9 includes the following steps: S1. The central dispatch server constructs a trigger message carrying a multi-field fingerprint and an on-demand detection control field based on the configuration table of the monitored nodes and sends it to each monitored node. S2. Deploy eBPF probe programs at multiple levels of the kernel protocol stack of the monitored node; S3. The inbound message is filtered in multiple stages by the eBPF probe program to intercept the trigger message, and the bitmap of the on-demand probe control field is extracted for selective state harvesting. Then, the aggregated statistical summary is written into the trigger message after two-step buffer adjustment and a telemetry response message is generated and reflected back to the central dispatch server. S4. The central dispatch server obtains multi-dimensional network status data based on the telemetry response messages reflected back by each monitored node and provides it to the digital twin network modeling engine.

Images