An active network telemetry method driven by in-network cache

Abstract

This paper discloses an active network telemetry solution driven by in-network caching. The solution includes: using a designed caching strategy to store and reuse network information collected by telemetry probes; using incremental transmission technology to reduce the amount of data transmitted from the data plane to the controller; and using deep reinforcement learning based on an attention mechanism to plan paths for in-network cache probes. The proposed method significantly reduces the amount of data transmitted from the data plane to the controller while also significantly reducing control overhead.

Description

Technical Field

[0001] The present invention belongs to the technical field of network traffic remote sensing, and in particular relates to an active network telemetry method driven by an in-network cache. Background Art

[0002] With the rapid development of network technology, modern networks are becoming increasingly large and complex, making network management particularly important. Network management systems need to monitor network status in real time to promptly identify and resolve potential problems and ensure stable and efficient network operation. Network telemetry, as a key technology, collects network status information by embedding probes in the network, providing network managers with in-depth insights into network behavior. However, with the expansion of network scale and the surge in traffic, traditional network telemetry methods face significant challenges. In particular, the amount of telemetry data transmitted from the data plane to the control plane has increased dramatically. This not only increases network bandwidth consumption but can also increase the processing burden on the control plane, thereby affecting the performance and responsiveness of the entire network.

[0003] In particular, in-band network telemetry (INT)-based technologies have attracted significant attention within network telemetry systems due to their ability to embed telemetry information directly within data packets. INT allows nodes along the way (e.g., routers) to add information about their processing state (e.g., latency, packet loss rate, etc.) to the packet header during packet transmission, enabling comprehensive monitoring of network status without requiring additional network resources. However, this approach also faces the challenges of data redundancy and unnecessary information transmission, as multiple packets may contain duplicate or similar telemetry information, which can be redundant for the controller's decision-making.

[0004] Furthermore, effectively planning detection paths to maximize telemetry data collection efficiency while minimizing the impact on normal network traffic is a key issue that network telemetry systems must address. Traditional path planning methods are often based on static network topologies and pre-defined rules, making them difficult to adapt to dynamically changing network environments.

[0005] Given these challenges, designing a network telemetry system that effectively reduces redundant and unnecessary telemetry data while improving the efficiency of probe path planning has become a research hotspot in the field of network management. This paper addresses this issue with its innovative approach. By introducing a caching mechanism and incremental transmission technology, along with a probe path planning algorithm based on deep reinforcement learning (DRL), it aims to significantly improve the efficiency and performance of network telemetry systems. Summary of the Invention

[0006] The purpose of the present invention is to reduce the amount of redundant and unnecessary telemetry data from the data plane to the controller, while improving detection efficiency and adapting to dynamic network environments, and to provide an active network telemetry method driven by in-network cache;

[0007] To achieve the above objectives, the present invention adopts the following technical solution: an active network telemetry method driven by an in-network cache, comprising:

[0008] Step S1: The controller generates a planned telemetry path according to a detection path planning algorithm based on deep reinforcement learning and attention mechanism, and then sends probes to the detection network along the planned telemetry path;

[0009] Step S2: In step S1, the probe records the relevant information to be detected at each node, and then the probe caches the collected information in the cache node, and the cache node stores, updates and retrieves the cached information through the preset cache strategy;

[0010] Step S3: The cache node uses incremental transmission technology to transmit the information stored in the cache node back to the controller.

[0011] Furthermore, the detection path planning algorithm based on deep reinforcement learning and attention mechanism in step S1 specifically includes:

[0012] Step S11: Set a specific detection path planning goal, which is expressed as a constrained optimization problem, and its mathematical expression is:

[0013] min C

[0014]

[0015]

[0016] L=E,

[0017] T≤T max ,

[0018] |p k |≤l th ,k=1,…,K.

[0019] The optimization goal is to minimize the control cost C of the telemetry system, and the constraints are After collecting all network information, the probe must store the information on the cache node. The kth telemetry path passes through the Nth k nodes; is the cache capacity limit of the i-th router, S k is the cache space required by the k-th probe, is an indicator function, when The value is 1 when L=E, otherwise it is 0; L=E means that the probe needs to detect all physical paths in the network, L is the set of physical links of all detection paths, and E is the set of direct physical links of network nodes; T≤T max is the telemetry delay constraint, T is the actual maximum telemetry delay of the telemetry system, and T max is the maximum telemetry delay allowed for the control plane; |p k |<l th Indicates that the length of the port information has an upper limit, |p k | is the length of the kth telemetry path, l th represents the length threshold of the auxiliary probe path;

[0020] Step S12: Map the high-load network graph to the vector space, input in It is a tuple containing the connection status and delay information of the i-th node and other nodes at time slot t, and is processed by one-dimensional convolution. Embed it and get

[0021] Step S13: Use content-based attention mechanism to obtain h from the decoder t , combined with step S12 to obtain Compute variable-length alignment vectors Variable-length alignment vector The calculation formula is: in h t ∈R D Represents the memory state of the RNN unit at time t during decoding. The symbol “;” represents the connection of two vectors. The variable and is a trainable variable;

[0022] Step S14: Using step S12 to obtain Combined with step S13, the variable length alignment vector is obtained Calculate the context vector c t , context vector c t The calculation formula is: Where M is the input length, and the input The length is equal, and the softmax function is used to normalize the conditional probability Conditional probability for: in variable v c and W c is a trainable variable;

[0023] Step S15: Use the greedy decoder to get the conditional probability according to step S14 To select an action, and update the telemetry path planning scheme according to the selected action, and at the same time Update and get the updated input and telemetry path until the specific detection path planning goal in step S11 is met, where Where f is the state transition function.

[0024] Furthermore, the training steps of the deep reinforcement learning neural network in the detection path planning algorithm based on deep reinforcement learning and attention mechanism specifically include:

[0025] Step S101: Input In each It is a tuple containing the connection status and delay information of the i-th node and other nodes at time slot t, which is embedded using one-dimensional convolution. After embedding, it is represented as Get the dataset;

[0026] Step S102: Initialize the action network and the evaluation network with random weights θ and δ, and then set the number of iteration cycles. In each iteration cycle, first reset the gradients dθ←0, dδ←0, and then select the sample set M for training from the data set obtained in step S101. The number of samples is recorded as batch. Then, for each sample in M, start from time slot t=0 and calculate the conditional probability Get new status Then update t to t+1, repeatedly calculate the conditional probability and get the new state until the specific detection path planning goal is met. After the calculation is completed, calculate the reward R according to the generated strategy m , and after the strategy generation and reward calculation are completed for each sample in M, the gradients dθ and dδ are calculated according to the rewards:

[0027]

[0028]

[0029] Then, θ and δ are updated according to the calculated gradient to complete one iteration cycle of training. When the set number of iteration cycles is completed, the deep reinforcement learning neural network training is completed.

[0030] Furthermore, the cache strategy preset in step S2 specifically includes:

[0031] Step S21: Cache object classification: classify cached information into time-varying data and static data;

[0032] Step S22: Cache key design: Each cache item corresponds to the same cached information bound to a specific timestamp, and the cache key of each cache item is formulated based on the data type and its associated timestamp;

[0033] Step S23: Cache update strategy design: For time-varying data, cache contents and keys are updated when new probes are received, ensuring that the cache always reflects the latest network status. When the capacity limit is reached, the smallest fresh items of the same type are cleared first. For static data, only the latest cached information is cached and directly updated when new probes are received, thereby minimizing unnecessary cache operations.

[0034] Step S24: Cache expiration mechanism design: Set a global cache expiration time, which is dynamically adjusted according to the network environment. For time-varying data, cache items that exceed the global cache expiration time will be automatically deleted to ensure the timeliness and accuracy of cached data. For static data, it is set to never fail to reduce unnecessary cache operations.

[0035] Step S25: Cache access mechanism design: Based on the cache key search method, when querying the cache, the system quickly searches for the corresponding cache item according to the cache key. If a matching cache item is found, the cached network information is returned. Otherwise, the "cache lost" status is returned.

[0036] Furthermore, the incremental transmission technology in step S3 specifically includes:

[0037] Step S31: Divide the information stored in the cache node into multiple data blocks according to the probe;

[0038] Step S32: Use a hash function to calculate unique hash values ​​for all data blocks and systematically store them in a dedicated hash library for subsequent comparison and analysis. Whenever the system receives a new probe, the network telemetry system calculates a new hash value based on the new data block and compares it with the existing hash value in the hash library.

[0039] Step S33: Update the hash library accordingly. For static data, the network telemetry system will immediately generate an incremental file whenever its hash value changes. For time-varying data, after checking whether the hash value has changed, it is necessary to evaluate whether the data change exceeds a preset threshold. Only when the data change exceeds this threshold will the network telemetry system generate an incremental file.

[0040] Step S34: After the incremental file is generated, it is immediately sent to the controller. After receiving the file, the controller calculates and compares the hash value. If the comparison result is the correct hash, it indicates that the data has not been tampered with. The information stored in the cache node received by the controller is used for the local database and network management application. Finally, the controller updates the local hash database to facilitate subsequent data synchronization and verification.

[0041] Beneficial Effects: This invention successfully addresses the challenges of data redundancy and excessive transmission volume in modern network management by integrating efficient caching strategies, incremental transmission technology, and an intelligent detection path planning algorithm based on deep reinforcement learning. The system intelligently stores and reuses critical network information, transmitting only necessary changes, significantly reducing network bandwidth consumption and the processing burden on the control plane. Furthermore, its flexible cache router upgrade solution and intelligent path planning capabilities ensure comprehensive and efficient network telemetry, bringing unprecedented convenience and performance improvements to modern network management. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a framework diagram of an active network telemetry solution driven by an in-network cache provided in an embodiment.

[0043] Figure 2 This is the flow chart of incremental transmission technology.

[0044] Figure 3 Schematic diagram of deep reinforcement learning. DETAILED DESCRIPTION

[0045] The present invention will be further explained below with reference to the accompanying drawings.

[0046] like Figure 1 As shown, the present invention provides an active network telemetry method driven by an in-network cache, comprising:

[0047] Step S1: The controller generates a planned telemetry path according to the detection path planning algorithm based on deep reinforcement learning and attention mechanism, and then sends probes to the detection network according to the planned telemetry path.

[0048] Step S2: In step S1, the probe records the relevant information to be detected at each node, and then the probe caches the collected information in the cache node, and the cache node stores, updates and retrieves the cached information through a preset cache strategy.

[0049] Step S3: The cache node uses incremental transmission technology to transmit the information stored in the cache node back to the controller.

[0050] In step S1, the network topology is initialized according to the original data, specifically including: defining the detection network as an undirected physical graph, represented by G = (V, E); where V = {v i |i=1,…,N} is the set of physical nodes, v i ∈V is the physical node with serial number i in the monitoring network; E={e ij =(v i ,v j )|v i ,v j ∈V} is the set of direct physical links of nodes, where e ij ∈E is the physical link status between node i and node j. If there is a connected physical link between node i and node j, then e ij exists; otherwise e ij Does not exist. The set of cache nodes is represented as U = {j∈V|x j =1}, x i Is a binary variable indicating whether node i is upgraded. i =1, then node i has been upgraded; if x i = 0, node i has not been upgraded. The total number of cache nodes is Q, that is

[0051] Assume there are K telemetry paths, where the kth path is represented as: where v k,i is the i-th node that the k-th telemetry path passes through. N k is the total number of nodes passed through in the kth path. Since the Fwd stack is a fixed-length stack, the length of the port information cannot exceed the length of the SR stack, which can be expressed as |p k |≤l th Among them, |p k | represents the length of the kth path, l th Indicates the length threshold of the auxiliary probe path.

[0052] The kth physical link in the kth probe is routed by (v k,i ,v k,i+1 ) is represented by the set of physical links that the k-th probe passes through is represented by L k ={(v k,i ,v k,i+1 )|i=1,…,N i -1}, After obtaining the set of links covered by each probe, we can express the set of physical links L of all probe paths as Since the probe needs to detect all physical paths in the network, the set L should satisfy L=E.

[0053] Telemetry latency is very important for real-time network services. The latency of each link can be obtained through the network telemetry of the previous cycle. Define the latency function t:E→T. The latency of forwarding a packet from node i to node j is expressed as t(i,j). If there is no physical link between the nodes, then t(i,j) is infinite. The telemetry latency of the Kth detection path can be expressed as Where t(v k,i ,v k,i+1 ) is the delay of the ith physical link on the kth detection path. Considering the frequency consistency, the telemetry delay T of the network telemetry system is the maximum detection delay, which is expressed as T = max{T k}, k=1,2,…,K. Assume that the maximum telemetry delay allowed by the control plane is Tmax. To ensure that the control plane obtains network information in time, the telemetry delay constraint can be expressed as T≤T max .

[0054] Control overhead is a key metric for network telemetry systems. In active network telemetry, control overhead is primarily caused by the generation and collection of probes. Therefore, control overhead is related to the number of probes in the network and can be expressed as a linear function of the number of probe paths. This can be expressed as follows:

[0055] C = h·K, where h is a scaling factor used to quantify the overhead of each detection path.

[0056] In order to efficiently utilize the in-network cache, we need to store all the network information collected by the probe in the cache router. This not only maximizes the use of differential transmission technology to reduce the amount of transmitted data, but also uses the same set of cached data to respond to control plane requests due to the integrity of the network information. To ensure that all network information can be stored in the cache router, the probe must store the information on the cache node after collecting all the network information, which can be expressed as

[0057] In addition, the total amount of network information stored in each cache router cannot exceed the maximum cache limit. The number of bytes that each probe needs to store in the cache switch is determined by the number of INT tags in the INT stack. We can define the cache space required by the kth probe as s k , can be calculated as s k =a·|p k |, a is a scaling factor determined by the number of bytes per label. Therefore, the cache capacity limit of the i-th router can be expressed as in is an indicator function, when (i.e., node i is the last hop of the k-th telemetry path), and 0 otherwise.

[0058] Since both the number and frequency of probes increase the network load for network telemetry, the number of probes should be minimized when the probe frequency is fixed, and all switches should be within the telemetry range. In addition, to ensure that all network information can be stored by cache-enabled routers, all probes should be forwarded to cache-enabled switches. Therefore, based on the above analysis, the probe path planning problem can be summarized as a constrained optimization problem, whose mathematical expression is:

[0059] minC

[0060]

[0061]

[0062] L=E,

[0063] T≤T max ,

[0064] |p k |≤l th ,k=1,…,K.

[0065] The optimization goal is to minimize the telemetry system control overhead C, and the constraints are After collecting all network information, the probe must store the information on the cache node. The kth telemetry path passes through the Nth k nodes; is the cache capacity limit of the i-th router, S k is the cache space required by the k-th probe, is an indicator function, when The value is 1 when L=E, otherwise it is 0; L=E means that the probe needs to detect all physical paths in the network, L is the set of physical links of all detection paths, and E is the set of direct physical links of network nodes; T≤T max is the telemetry delay constraint, T is the maximum detection delay, T max is the maximum telemetry delay allowed for the control plane; |p k |<l th Indicates that the length of the port information has an upper limit, |p k | represents the length of the kth telemetry path, l th Indicates the length threshold of the auxiliary probe path.

[0066] Then we use a network model based on deep reinforcement learning to solve the above optimization problem. The main implementation idea is: first map the high-load network graph to the vector space, input In each It is a tuple containing the connection status and delay information of the i-th node and other nodes at time slot t;

[0067] We first arbitrarily choose At time slot t,t∈[0,T], we select the available input Select y t Until the constraints in constraint formula (1) are met, the node sequence generated by this process can be expressed as T″ is the sequence length, and the sequence is decomposed using the probability chain rule Probability of generation Expressed as:

[0068]

[0069] in Calculated by the attention layer of the deep reinforcement learning network. If an affine function g is used to represent the transformation from input to output vector, the state of the recurrent neural network (RNN) decoder is represented by h t , which summarizes the previous steps y0,…,y t Information, probability It can be expressed as:

[0070]

[0071] Based on this, the input Recursively update to:

[0072]

[0073] The role of the state transition function f is to recursively update the input

[0074] like Figure 3 As shown in the figure, during the actual algorithm operation, the input of the model is the information of a group of nodes. Their input order does not affect the output probability information, so input embedding is used directly.

[0075] So through deep reinforcement learning network computing The specific steps are as follows: First, the input is mapped into a vector space, which may have multiple embeddings corresponding to different input elements. The embedding operation is performed using a one-dimensional convolutional layer.

[0076] A content-based attention mechanism is used to extract relevant information h from the decoder input. t Variable length alignment vector For computing this mechanism, input After embedding, it is represented as Among them, h t ∈R D Represents the memory state of the RNN unit when decoding t. Variable length alignment vector It can be calculated as:

[0077]

[0078] and:

[0079]

[0080] The relevance of each input data point to the upcoming decoding step t is determined, and the symbol “;” represents the concatenation of two vectors. and is a trainable variable. To calculate the conditional probability, we first calculate the context vector c using the embedded input t , defined as:

[0081]

[0082] These values ​​are then normalized using the softmax function to obtain the conditional probabilities as follows:

[0083]

[0084] and:

[0085]

[0086] Among them, the variable v c and W c It is also a trainable variable.

[0087] Finally, a greedy decoder is used to calculate the conditional probability To select an action, and update the telemetry path planning scheme according to the selected action, and at the same time Update to get new input and the planned telemetry path until the telemetry path planning goal is met.

[0088] This example also trains the deep reinforcement learning neural network in the detection path planning algorithm based on deep reinforcement learning and attention mechanism, thereby implementing the detection path planning algorithm based on deep reinforcement learning and attention mechanism. The training steps include:

[0089] Step S101: Input In each It is a tuple containing the connection status and delay information of the i-th node and other nodes at time slot t, which is embedded using one-dimensional convolution. After embedding, it is represented as Get the dataset.

[0090] Step S102: Initialize the action network and the evaluation network with random weights θ and δ, and then set the number of iteration cycles. In each iteration cycle, first reset the gradients dθ←0, dδ←0, and then select the sample set M for training from the data set obtained in step S101. The number of samples is recorded as batch. Then, for each sample in M, start from time slot t=0 and calculate the conditional probability Get new status Then update the time slot t to t+1, repeatedly calculate the conditional probability and get the new state until the specific detection path planning goal is met. After the calculation is completed, calculate the reward R according to the generated strategy m , and after the strategy generation and reward calculation are completed for each sample in M, the gradients dθ and dδ are calculated according to the rewards:

[0091]

[0092]

[0093] Then, θ and δ are updated according to the calculated gradient to complete one iteration cycle of training. When the set number of iteration cycles is completed, the training of the detection path planning algorithm based on deep reinforcement learning and attention mechanism is completed.

[0094] In step S2, the preset cache strategy specifically includes: cache object selection, cache key design, cache update strategy, cache invalidation mechanism and cache access strategy.

[0095] Cached Object Classification: Network information can be divided into two types: time-varying data and static data. Time-varying data refers to dynamic network information that needs to be updated in real time to reflect the latest network status, such as router load and latency. Static data is network information that rarely changes but is equally important for understanding the overall network, such as port connection status.

[0096] Cache key design: Each cache item corresponds to the same network information tied to a specific timestamp. The cache key for each item is determined based on the data type and its associated timestamp. This approach ensures the cache maintains a high level of accuracy and timeliness while simplifying retrieval and subsequent updates.

[0097] Cache update strategy, as follows:

[0098] We employ different cache update strategies to handle time-varying and static data. For time-varying data, we use a freshness-based update strategy that updates cache contents and keys upon receiving new probes, ensuring the cache always reflects the latest network status. When capacity is reached, the least fresh items of the same type are prioritized for eviction.

[0099] For static data, only the latest network information is cached and updated directly when new probes are received, minimizing unnecessary cache operations.

[0100] To maintain the timeliness and accuracy of time-varying data, we set a global cache expiration timer, which is dynamically adjusted based on the network environment. Cache items that exceed this timer are automatically deleted to ensure the timeliness and accuracy of cached data. For static data, which changes less frequently, we can set it to never fail, reducing unnecessary caching operations.

[0101] For cache access, we use a cache key-based lookup method. When querying the cache, the system quickly searches for the corresponding cache item based on the cache key. If a matching cache item is found, the cached network information is returned. Otherwise, a "cache miss" status is returned.

[0102] like Figure 2 As shown in Figure 3, the details of the incremental transmission technique include: First, network information is divided into multiple data blocks based on the probe. Then, a hash function is used to calculate unique hash values ​​for all data blocks, which are systematically stored in a dedicated hash library for subsequent comparison and analysis. Whenever the system receives a new probe, the network telemetry system calculates a new hash value based on the new data block and compares it with the existing hash value in the hash library. The hash library is then updated accordingly. For static data, the network telemetry system immediately generates an incremental file whenever its hash value changes. For time-varying data, after checking whether the hash value has changed, it also assesses whether the data change exceeds a preset threshold, which can be flexibly adjusted based on the network environment and user requirements. The network telemetry system only generates an incremental file when the data change exceeds this threshold. Once generated, the incremental file is immediately sent to the controller. Upon receiving the file, the controller calculates and compares the hash value to ensure data integrity and security. A correct hash value indicates that the data has not been tampered with. The controller's control plane uses the received new network information for its local database and network management applications. Finally, to ensure the consistency of information between the data plane and the controller’s control plane, the controller updates the local hash database to facilitate subsequent data synchronization and verification.

[0103] In summary, the present invention proposes an active network telemetry solution driven by in-network cache. This method can effectively reduce the overall number of probes and data transmission volume based on cache strategy, incremental transmission and deep reinforcement learning, and improve detection efficiency.

[0104] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. An active network telemetry method driven by in-network cache, characterized in that: include: Step S1: The controller generates a planned telemetry path according to a detection path planning algorithm based on deep reinforcement learning and attention mechanism, and then sends probes to the detection network along the planned telemetry path; Step S2: In step S1, the probe records the relevant information to be detected at each node, and then the probe caches the collected information in the cache node, and the cache node stores, updates and retrieves the cached information through the preset cache strategy; Step S3: The cache node uses incremental transmission technology to transmit the information stored in the cache node back to the controller; The detection path planning algorithm based on deep reinforcement learning and attention mechanism in step S1 specifically includes: Step S11: Set a specific detection path planning goal, which is expressed as a constrained optimization problem, and its mathematical expression is: min C L=E, T≤T max , |p k |≤l th ,k=1,…,K. The optimization goal is to minimize the control cost C of the telemetry system, and the constraints are After collecting all network information, the probe must store the information on the cache node. The kth telemetry path passes through the Nth k nodes; is the cache capacity limit of the i-th router, S k is the cache space required by the k-th probe, is an indicator function, when The value is 1 when L=E, otherwise it is 0; L=E means that the probe needs to detect all physical paths in the network, L is the set of physical links of all detection paths, and E is the set of direct physical links of network nodes; T≤T max is the telemetry delay constraint, T is the actual maximum telemetry delay of the telemetry system, and T max is the maximum telemetry delay allowed for the control plane; |p k |<l th Indicates that the length of the port information has an upper limit, |p k | is the length of the kth telemetry path, l th represents the length threshold of the auxiliary probe path; Step S12: Map the high-load network graph to the vector space, input in It is a tuple containing the connection status and delay information of the i-th node and other nodes at time slot t, and is processed by one-dimensional convolution. Embed it and get Step S13: Use content-based attention mechanism to obtain h from the decoder t , combined with step S12 to obtain Compute variable-length alignment vectors Variable-length alignment vector The calculation formula is: in, Represents the memory state of the RNN unit at time t during decoding. The symbol ";" represents the connection of two vectors. The variable and is a trainable variable; Step S14: Using step S12 to obtain Combined with step S13, the variable length alignment vector is obtained Calculate the context vector c t , context vector c t The calculation formula is: Where M is the input length, and the input The length is equal, and the softmax function is used to normalize the conditional probability Conditional probability for: in variable v c and W c is a trainable variable; Step S15: Use the greedy decoder to get the conditional probability according to step S14 To select an action, and update the telemetry path planning scheme according to the selected action, and at the same time Update and get the updated input and telemetry path until the specific detection path planning goal in step S11 is met, where Where f is the state transition function.

2. The active network telemetry method driven by in-network cache according to claim 1, characterized in that: The training steps of the deep reinforcement learning neural network in the detection path planning algorithm based on deep reinforcement learning and attention mechanism specifically include: Step S101: Input In each It is a tuple containing the connection status and delay information of the i-th node and other nodes at time slot t, which is embedded using one-dimensional convolution. After embedding, it is represented as Get the dataset; Step S102: Initialize the action network and the evaluation network with random weights θ and δ, and then set the number of iteration cycles. In each iteration cycle, first reset the gradients dθ←0, dδ←0, and then select the sample set M for training from the data set obtained in step S101. The number of samples is recorded as batch. Then, for each sample in M, start from time slot t=0 and calculate the conditional probability Get new status Then update t to t+1, repeatedly calculate the conditional probability and get the new state until the specific detection path planning goal is met. After the calculation is completed, calculate the reward R according to the generated strategy m , and when each sample in M ​​completes the strategy generation and reward calculation, the gradients dθ and dδ are calculated according to the rewards, and θ and δ are updated according to the calculated gradients to complete one iteration cycle of training. When the set number of iteration cycles is completed, the deep reinforcement learning neural network training is completed.

3. The active network telemetry method driven by in-network cache according to claim 1, characterized in that: The cache strategy preset in step S2 specifically includes: Step S21: Cache object classification: classify cached information into time-varying data and static data; Step S22: Cache key design: Each cache item corresponds to the same cached information bound to a specific timestamp, and the cache key of each cache item is formulated based on the data type and its associated timestamp; Step S23: Cache update strategy design: For time-varying data, cache contents and keys are updated when new probes are received, ensuring that the cache always reflects the latest network status. When the capacity limit is reached, the smallest fresh items of the same type are cleared first. For static data, only the latest cached information is cached and directly updated when new probes are received, thereby minimizing unnecessary cache operations. Step S24: Cache expiration mechanism design: Set a global cache expiration time, which is dynamically adjusted according to the network environment. For time-varying data, cache items that exceed the global cache expiration time will be automatically deleted to ensure the timeliness and accuracy of cached data. For static data, it is set to never fail to reduce unnecessary cache operations. Step S25: Cache access mechanism design: Based on the cache key search method, when querying the cache, the system quickly searches for the corresponding cache item according to the cache key. If a matching cache item is found, the cached network information is returned. Otherwise, the "cache lost" status is returned.

4. The active network telemetry method driven by in-network cache according to claim 1, characterized in that: The incremental transmission technology in step S3 specifically includes: Step S31: Divide the information stored in the cache node into multiple data blocks according to the probe; Step S32: Use a hash function to calculate unique hash values ​​for all data blocks and systematically store them in a dedicated hash library for subsequent comparison and analysis. Whenever the system receives a new probe, the network telemetry system calculates a new hash value based on the new data block and compares it with the existing hash value in the hash library. Step S33: Update the hash library accordingly. For static data, the network telemetry system will immediately generate an incremental file whenever its hash value changes. For time-varying data, after checking whether the hash value has changed, it is necessary to evaluate whether the data change exceeds a preset threshold. Only when the data change exceeds this threshold will the network telemetry system generate an incremental file. Step S34: After the incremental file is generated, it is immediately sent to the controller. After receiving the file, the controller calculates and compares the hash value. If the comparison result is the correct hash, it indicates that the data has not been tampered with. The information stored in the cache node received by the controller is used for the local database and network management application. Finally, the controller updates the local hash database to facilitate subsequent data synchronization and verification.

Images