Active telemetering scene-oriented important node screening method

Abstract

The invention discloses an important node screening method for an active telemetering scene, and provides a comprehensive evaluation and optimization framework aiming at the problems of probe resource waste and overhigh system load caused by improper node selection in the existing telemetering technology. According to the method, network static topology features and dynamic state features are combined, a node importance scoring model is constructed through feature extraction, dimension reduction fusion and multi-task feature comprehensive analysis, and accurate selection of important network nodes is achieved. According to the method, topological information is analyzed through a weighted TOPSIS method to obtain spatial features, key dimensions of state features in stage historical telemetry data are extracted through principal component analysis (PCA), redundant data are remarkably reduced, and efficient state feature representation is generated. On the basis, a screening method combining spatial features and node state features is designed, and a screening strategy is dynamically adjusted through feature analysis of staged telemetering so as to optimize node priority ranking. The method is suitable for an active telemetering scene in a complex network environment, the capturing capacity of important events can be remarkably improved, system resource consumption is reduced, and the overall performance of an active telemetering system is comprehensively optimized.

Description

Technical Field

[0001] The present invention relates to the field of cyberspace security, and in particular to an important node screening method for active telemetry scenarios. Background Art

[0002] In recent years, with the diversification of business applications and the continuous growth of user scale, the network has gradually shown the characteristics of "high speed, large scale, multi-access, and unpredictable", and the demand for efficient network measurement has become more urgent. As the core technology of network perception and management, network measurement is directly related to the realization of key tasks such as network anomaly detection, performance optimization, and resource scheduling. In-band Network Telemetry (INT) is a network measurement method developed in recent years based on programmable data plane technology. It dynamically attaches telemetry metadata to data packets at switching nodes in the path to achieve real-time collection and hop-by-hop monitoring of network status. Compared with traditional network measurement solutions, in-band telemetry changes the traditional network measurement idea of ​​treating network switching equipment as an intermediate black box, and uses the data plane to directly drive the network measurement process. It has stronger real-time and more fine-grained performance monitoring capabilities, and has gradually become an important direction for modern network measurement.

[0003] From the perspective of implementation, network telemetry is mainly divided into two types: passive telemetry and active telemetry. Passive network telemetry carries telemetry metadata through existing business flows, and can efficiently collect various network status information on the target path. Since passive telemetry relies on business traffic to cover the path, it is difficult to achieve flexible and comprehensive monitoring for some paths that are not traversed by traffic, which limits its applicability across the entire network. In contrast, in in-band telemetry technology, active telemetry has gradually attracted widespread attention due to its advantages in flexibility and coverage. Unlike passive telemetry that relies on business traffic to cover the path, in recent years, with the maturity of technologies such as Segment Routing (SR), active network telemetry based on probes can easily cover the entire network without being constrained by business flows, and has gradually entered the public eye. Active telemetry can cover any target path by constructing specific probe packets, and can achieve precise customization of the path through source routing labels. This feature enables active telemetry to flexibly adapt to different network structures and dynamically changing business scenarios, providing unprecedented flexibility and adaptability for network status monitoring.

[0004] Although active telemetry has shown great potential, its development in practical applications is still constrained by data redundancy and resource waste. The information collected by many probes is relatively stable, lacking abnormal events that upper-layer applications pay attention to. These invalid probes not only occupy valuable bandwidth resources, but also significantly increase the load of north-south communication in the network, and further increase the communication pressure between the data plane and the telemetry server. This waste of resources directly leads to a decline in system performance and has a negative impact on telemetry efficiency.

[0005] To solve the above problems, existing research mainly focuses on the optimization of passive telemetry systems, improving the effectiveness of telemetry data by reducing system overhead. For example, Flexible Sampling introduces a dynamic sampling mechanism to dynamically adjust the probe embedding frequency according to changes in network traffic, thereby reducing the amount of meaningless telemetry data; Selective In-band Network Telemetry optimizes the use and allocation of probe resources by reducing the telemetry frequency when the network status is stable. These methods perform well in reducing the load on passive telemetry systems and significantly improve their efficiency. However, since passive telemetry relies on existing business traffic coverage paths, its optimization strategy is difficult to adapt to the needs of active telemetry. Especially in active telemetry scenarios, the problem of how to reasonably allocate probe resources for important nodes has not been effectively solved.

[0006] In addition, active telemetry is also limited by the basic framework, which leads to obvious technical bottlenecks in practical applications. A large amount of information in the telemetry data is smooth and lacks value, and only a small part of the data contains abnormal events or critical states of interest to upper-layer applications. This redundant data not only wastes a lot of computing time and storage space of the telemetry server, but also has a serious impact on the overall performance of the system. Compared with passive telemetry, the additional INT probes introduced by active telemetry further increase the network load, especially when facing invalid probes, the occupation of network bandwidth and the communication pressure between the data plane and the telemetry server are more significant.

[0007] In summary, existing research on reducing telemetry overhead has achieved certain results, but in active telemetry scenarios, the following problems exist, which make it impossible to accurately select important telemetry nodes according to specific network conditions: (1) Existing research results are mostly focused on passive telemetry systems and lack targeted optimization of active telemetry scenarios. Passive telemetry relies on existing business traffic for path coverage, while active telemetry requires active construction of probes to complete network monitoring. This essential difference makes it difficult for the optimization strategy of passive telemetry to be directly applied to active telemetry scenarios, especially in the allocation of probes and resource scheduling of important nodes, which cannot meet the flexibility requirements of active telemetry. (2) Current methods are usually based on a single network feature (such as degree centrality) and fail to comprehensively consider static topological features and dynamic state features, resulting in inaccurate node importance assessment and lack of comprehensiveness and reliability in the judgment of node importance. (3) Existing probe allocation strategies do not fully focus on important nodes. The data collected by some probes remain stable for a long time, which cannot effectively capture abnormal events that upper-level applications are concerned about, resulting in a waste of probe resources. (4) The network state has the characteristics of dynamic change, but the existing methods are too static in evaluating the importance of nodes. The node screening mechanism lacks real-time adjustment capabilities and is difficult to adapt to the needs of dynamic network changes, thus affecting the effectiveness of node selection.

[0008] The important node selection method for active telemetry provided by the present invention effectively solves the technical problems caused by excessive additional load and unreasonable path planning of active telemetry in the current network security field. Summary of the invention

[0009] In response to the above problems, this paper proposes a highly versatile probe frequency planning method for active telemetry, aiming to achieve load optimization of active telemetry across the entire network. The innovation of this method is that the concept of telemetry node importance is introduced into probe frequency planning for the first time. Specifically, within the entire network, we believe that the importance of each node in the telemetry process is different. Some nodes are relatively more important and need to be assigned a higher detection frequency to ensure the real-time nature of their status information. For relatively unimportant nodes, the detection frequency can be appropriately reduced, thereby effectively reducing the additional load brought by active telemetry.

[0010] In order to achieve the purpose of the present invention, the specific technical steps of this scheme are as follows: A method for screening important nodes for active telemetry scenarios, the method comprising the following steps:

[0011] Step (1) In the telemetry system, the network data plane data is collected through distributed probes. The data includes the delay information, bandwidth occupancy, throughput information and other states of the network nodes. To ensure the data quality, the noise filtering and missing value filling algorithms are used to clean the data. The cleaned data is stored in the form of a time series matrix for subsequent feature extraction and analysis;

[0012] Step (2) extracts the topological information of the nodes based on the basic data collected in step (1), analyzes the spatial factors affecting the nodes, and uses a weighted TOPSIS method to integrate multiple centrality measurements to obtain a spatial factor that is one of the important node evaluation indicators;

[0013] Step (3) extracts important features that affect the node state based on the basic data collected in step (1), including dynamic features such as significant change frequency and state entropy. The significant change frequency is obtained by counting the number of significant changes in the node's periodic historical telemetry data, and the state entropy is calculated by calculating the state fluctuations in the node's periodic historical telemetry data. A node state feature matrix is ​​formed based on the calculation results, and then high-dimensional state features are extracted. In order to eliminate feature redundancy and multicollinearity, principal component analysis (PCA) is used for dimensionality reduction processing;

[0014] In the actual deployment of step (4), many telemetry tasks are carried out simultaneously, that is, the probe message needs to collect multiple network status data at each hop node. In the case of multiple tasks, each task data can reflect the status characteristics of the telemetry node. However, the possible correlation between different telemetry tasks will cause the final generated state factor to be inaccurate. Therefore, based on the dimensionality reduction results of step (3), the present invention processes the multi-source data features generated by multiple telemetry tasks, fuses the different task features after standardization, and uses the fusion result as a unified comprehensive state factor for the importance scoring model;

[0015] Step (5) Given a network topology with n nodes, construct the spatial importance and comprehensive state importance vector of the entire network according to the spatial factor and state factor of each node obtained in step (2) and step (4);

[0016] Step (6) Based on the spatial importance vector and the optimal state importance vector obtained in step (5), the importance scoring model of the telemetry node is constructed using the Hadamard product. By calculating the score, the importance of the nodes is arranged in descending order, and the high priority nodes are marked;

[0017] Step (7) combines the historical telemetry data collected in step (1) and the node importance score result calculated in step (6) to set a score threshold σ. Nodes with node importance scores exceeding σ are screened as priority telemetry objects and serve as key targets for probe resource allocation. After the initial threshold is set, during the periodic operation of the telemetry system, the latest information on network topology and traffic status is collected in real time, and the scoring model is continuously optimized and the threshold σ is updated through a dynamic feedback mechanism to ensure the real-time and accuracy of the screening results;

[0018] Step (8) To ensure the accuracy and reliability of the important node screening, the nodes screened in step (7) are applied to the anomaly detection and active telemetry probe allocation optimization experiment. Specifically, it includes verifying the effectiveness of the screening results, evaluating its performance by calculating the node coverage rate and the improvement ratio of anomaly detection ability, and comparing the high-scoring node coverage rate, anomaly detection ability and other indicators with the existing methods. Figure 2 The working framework for verification and application of important nodes in active telemetry systems is demonstrated.

[0019] Furthermore, in step (1), the steps of collecting and preprocessing basic data are as follows:

[0020] (1.1) In the network telemetry system, the control plane sends the probe configuration policy to the telemetry source node of the data plane through the north-south communication interface. The configuration includes the probe path, probe identification header information and probe collection frequency;

[0021] (1.2) The telemetry probe is injected into the network according to the configuration strategy in step (1.1). The data plane node processes the data packet matching the probe identification header hop by hop, adds node status information (such as latency, bandwidth occupancy, throughput information, etc.) hop by hop, and transmits it to the telemetry tail node along the path;

[0022] (1.3) The telemetry tail node packages the hop-by-hop data set from the starting node to the terminal node and uploads it to the telemetry server. The telemetry server performs noise filtering, missing value filling and normalization on the collected raw data to ensure data quality and consistency. The cleaned data is stored in a structured format for subsequent feature extraction and analysis.

[0023] Furthermore, in step (2), based on the basic data collected in step (1), the topological information of the nodes is extracted, and the spatial factors affecting the nodes are analyzed. The calculation method uses the weighted TOPSIS method to integrate multiple centrality measurement values ​​to obtain the spatial factor of one of the important node evaluation indicators:

[0024]

[0025] Factor spatial =R = TOPSIS(M,W)

[0026] in:

[0027] DC i Degree Centrality (DC) is the number of nodes directly connected to node i;

[0028] CC iCloseness Centrality (CC), which is the average path length between node i and other nodes;

[0029] EC i Eigenvector Centrality (EC) is the linear combination of the importance of neighboring nodes of node i;

[0030] Factor spatial It is a spatial factor, one of the important indicators for evaluating important nodes;

[0031] Furthermore, in step (3), the steps of node state feature extraction and feature dimension reduction are as follows:

[0032] (3.1) By performing time series analysis on the node traffic and connection status data collected in step (1), the number of significant changes in the node's periodic (e.g., within a fixed time window of 10 minutes) historical telemetry data is counted. This indicator is the significant change frequency, which reflects the degree of drastic changes in the node status. The larger the value, the more frequent the drastic changes in the node status.

[0033] (3.2) Based on the state fluctuations in the periodic historical telemetry data, the node state entropy is calculated to evaluate the stability of its state. Among them, a lower state entropy indicates that the node state is relatively stable, and a higher state entropy indicates that the surrounding traffic fluctuates greatly.

[0034] (3.3) In order to eliminate the differences between different feature dimensions, the present invention performs standardization on the state feature data. After standardization, the mean of all feature data is 0 and the variance is 1, ensuring that the weights of different features are balanced during the dimensionality reduction process;

[0035] (3.4) Calculate the covariance matrix C for the feature data matrix X after standardization in step (3.3) to evaluate the correlation between the features. The covariance matrix used in the present invention is a symmetric matrix, which reveals the strength of the correlation between the features;

[0036] (3.5) In order to ensure that the key information of high-dimensional feature data is retained as much as possible during the dimensionality reduction process, the present invention performs eigenvalue decomposition on the covariance matrix in step (3.4), arranges the eigenvalues ​​in descending order, calculates the cumulative contribution rate and selects the top k principal components with a cumulative contribution rate of 95%, and constructs the eigenvector after dimensionality reduction.

[0037] Furthermore, in step (4), the steps of normalizing telemetry task features and calculating task feature weights are as follows:

[0038] (4.1) Multi-task telemetry requires collecting n sets of indicators (delay, bandwidth, etc.), and the telemetry result set of each set of indicators is Xi ={x′ i1 ,x′ i2 ,···,x′ ik Considering that the values ​​of various indicators may not be of the same order of magnitude, and the positive and negative attributes of various telemetry tasks as network health evaluation indicators are different: for positive indicators, the larger the corresponding collected information, the better the network condition (such as the available bandwidth of the port), and for negative indicators, the smaller the corresponding information, the better the network condition (such as the number of packet losses). Therefore, for telemetry result sets of different projects, it is necessary to pre-calculate x i To do standardized processing:

[0039] standardization: Where k is the number of probe telemetry information collected periodically, x′ ik is a single telemetry result;

[0040] (4.2) After standardizing the indicators, the task characteristics are weighted:

[0041] Weight w i calculate:

[0042] where x′ i ={x′ i1 ,x′ i2 ,···,x′ ik}, pc ij Represents the Pearson correlation coefficient between telemetry indicators;

[0043] Among them CS i Represents the strength of association and is used to assign weights to different telemetry tasks;

[0044] where w i Represents the weight of the telemetry task.

[0045] (4.3)The formula for calculating the state factor is as follows:

[0046] Factor state =w1×F1+w2×F2+…+w n ×F n , where Factor state Represents the comprehensive state factor of the node, w i represents the weight of the telemetry task, F i Represents the state features after dimensionality reduction obtained under the telemetry task.

[0047] Step (5) Given a network topology with n nodes, construct the spatial importance and comprehensive state importance vectors of the entire network according to the spatial factor and comprehensive state factor of each node obtained in step (2) and step (4);

[0048] In one embodiment of the present invention, the specific implementation steps of the important node scoring model are as follows:

[0049] (5.1) First, based on the spatial factor of each telemetry node, the spatial importance vector of the entire network is obtained:

[0050] Where V spatial represents the spatial importance vector, is the spatial factor found in step (2).

[0051] (5.2) According to the telemetry state cycle, the highest value among the comprehensive state factors generated by the node ports is selected as the optimal comprehensive state factor, and is used to calculate the state importance vector of the network:

[0052] in represents the periodic comprehensive state factor of port j of node i calculated using the calculation formula in step (4.3), is the optimal comprehensive state factor state of the node;

[0053] Where V state Represents the state importance vector of the node, is the optimal comprehensive state factor obtained in step (5.2).

[0054] Furthermore, in step (6), based on the spatial importance vector and the optimal state importance vector obtained in step (5), the importance scoring model of the telemetry node is constructed using the Hadamard product. By calculating the score, the importance of the nodes is arranged in descending order, and the high priority nodes are marked:

[0055] Among them, ⊙ represents Hadamard product, Tele imp Represents the telemetry node importance vector, V spatial Represents the spatial importance vector, V state Represents the state importance vector of the node, and are the spatial factor and comprehensive state factor obtained in steps (2) and (5.2) respectively.

[0056] Furthermore, in step (7), the specific implementation steps of the dynamic adjustment mechanism and node screening are as follows:

[0057] (7.1) According to the telemetry feedback data and a large amount of prior data, the scoring threshold σ is set;

[0058] (7.2) After multiple rounds of telemetry, all nodes are re-evaluated with new spatial factors and state factors, and the list of important nodes is updated to ensure that the screening results match the real-time network status.

[0059] Furthermore, in step (8), the specific implementation steps for verifying the important node screening results are as follows:

[0060] Step (8.1) simulates abnormal events in the test network environment, including but not limited to traffic surge, packet loss rate increase, delay jitter, etc. Compare these events with the selected important node locations, and record the proportion of abnormal events captured by important nodes (capture rate);

[0061] Step (8.2) calculates the coverage based on the records obtained in step (8.1) to evaluate the effectiveness of the screening nodes in anomaly detection compared to the probe configuration with full network coverage.

[0062] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for screening important nodes for active telemetry scenarios is implemented.

[0063] Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0064] (1) The present invention proposes a method for screening important nodes suitable for active telemetry scenarios, focusing on improving screening efficiency and accuracy through collaborative analysis of key dimensions of static topology information and state characteristics in historical telemetry information. The method is based on the structural position of the node in the network, comprehensively considers its static position contribution in the topology, and combines characteristics such as significant change frequency and state entropy to comprehensively capture potential anomalies and network events in real-time traffic. Compared with traditional methods, the present invention can flexibly adapt to the diversity of complex network environments, and achieve accurate and efficient node selection by dynamically adjusting screening criteria, providing reliable support for efficient allocation of active telemetry probes and network anomaly detection.

[0065] (2) In the process of processing node state features, the present invention introduces the principal component analysis (PCA) method to reduce the dimensionality of multiple features, aiming to reduce redundant information and extract key features. In view of the possible correlation between node state features such as significant change frequency and state entropy, the present invention optimizes through linear combination, which not only retains the effective information in the original features, but also effectively eliminates the redundant interference between features. This dimensionality reduction method can fully reflect the real state changes of nodes, significantly improve the accuracy of important node screening, and optimize the utilization efficiency of computing resources.

[0066] (3) The present invention is aimed at multi-telemetry task scenarios, where each task data may reflect different state characteristics of the node. However, since there may be certain correlations between tasks, direct merging may lead to information redundancy or result bias. By analyzing the correlations between telemetry tasks, a reasonable weight is assigned to each task to reduce information overlap and highlight the contribution of important tasks. When allocating weights, indicators such as the Pearson correlation coefficient are used to quantify the correlation between tasks to ensure that the comprehensive state characteristics can objectively reflect the importance of the node. This method can flexibly adapt to different telemetry needs in a multi-task environment and provides strong support for the identification of key nodes in complex networks.

[0067] (4) The present invention proposes an importance assessment model that systematically evaluates the importance of network nodes by integrating static network topology features with dynamic traffic state features. The model uses a weighted fusion approach to combine static features in the topology structure (such as degree centrality, closeness centrality, and eigenvector centrality) with dynamic features of the node state (such as significant change frequency and state entropy), and generates the final importance score vector through the Hadamard product. This model can not only dynamically adjust the importance score threshold of nodes in different telemetry tasks, but also highlight the role of high-priority nodes in resource scheduling, ensuring the pertinence and effectiveness of the screening results in subsequent telemetry tasks, and overall optimizing the utilization efficiency of probe resources and improving the overall performance of the telemetry system. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 This is a framework diagram of the important node screening method for active telemetry scenarios;

[0069] Figure 2 Assist active telemetry probe optimization process diagram for important node telemetry information feedback; DETAILED DESCRIPTION

[0070] The technical solution provided by the present invention will be described in detail below in conjunction with specific embodiments. It should be understood that the following specific implementation methods are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0071] Embodiment: The present invention provides an important node screening method for active telemetry scenarios, and its overall system structure is as follows: Figure 1 As shown, the following steps are included:

[0072] Step (1) In the telemetry system, the network data plane data is collected through distributed probes. The data includes the delay information, bandwidth occupancy, throughput information and other states of the network nodes. To ensure the data quality, the noise filtering and missing value filling algorithms are used to clean the data. The cleaned data is stored in the form of a time series matrix for subsequent feature extraction and analysis;

[0073] In one embodiment of the present invention, the steps of collecting and preprocessing basic data are as follows:

[0074] (1.1) In the network telemetry system, the control plane sends the probe configuration policy to the telemetry source node of the data plane through the north-south communication interface. The configuration includes the probe path, probe identification header information and probe collection frequency;

[0075] (1.2) The telemetry probe is injected into the network according to the configuration strategy in step (1.1). The data plane node processes the data packet matching the probe identification header hop by hop, adds node status information (such as latency, bandwidth occupancy, throughput information, etc.) hop by hop, and transmits it to the telemetry tail node along the path;

[0076] (1.3) The telemetry tail node packages the hop-by-hop data set from the starting node to the terminal node and uploads it to the telemetry server. The telemetry server performs noise filtering, missing value filling and normalization on the collected raw data to ensure data quality and consistency. The cleaned data is stored in a structured format for subsequent feature extraction and analysis.

[0077] Step (2) extracts the topological information of the nodes based on the basic data collected in step (1) and analyzes the spatial factors that affect the nodes. The calculation method uses the weighted TOPSIS method to integrate multiple centrality measurements to obtain the spatial factor, which is one of the important node evaluation indicators:

[0078]

[0079] Factor spatial =R = TOPSIS(M,W)

[0080] in:

[0081] DC i Degree Centrality (DC) is the number of nodes directly connected to node i;

[0082] CC iCloseness Centrality (CC), which is the average path length between node i and other nodes;

[0083] EC i Eigenvector Centrality (EC) is the linear combination of the importance of neighboring nodes of node i;

[0084] Factor spatial It is a spatial factor, one of the important indicators for evaluating important nodes;

[0085] Step (3) extracts important features that affect the node state based on the basic data collected in step (1), including dynamic features such as significant change frequency and state entropy. The significant change frequency is obtained by counting the number of significant changes in the node's periodic historical telemetry data, and the state entropy is calculated by calculating the state fluctuations in the node's periodic historical telemetry data. A node state feature matrix is ​​formed based on the calculation results, and then high-dimensional state features are extracted. In order to eliminate feature redundancy and multicollinearity, principal component analysis (PCA) is used for dimensionality reduction processing;

[0086] In one embodiment of the present invention, the steps of dimension reduction and optimization between features are as follows:

[0087] (3.1) By performing time series analysis on the node traffic and connection status data collected in step (1), the number of significant changes in the node's periodic (e.g., within a fixed time window of 10 minutes) historical telemetry data is counted. This indicator is the significant change frequency, which reflects the degree of drastic changes in the node status. The larger the value, the more frequent the drastic changes in the node status.

[0088] (3.2) Based on the state fluctuations in the periodic historical telemetry data, the node state entropy is calculated to evaluate the stability of its state.

[0089] (3.3) In order to eliminate the differences between different feature dimensions, the present invention performs standardization on the state feature data. After standardization, the mean of all feature data is 0 and the variance is 1, ensuring that the weights of different features are balanced during the dimensionality reduction process;

[0090] (3.4) Calculate the covariance matrix C for the feature data matrix X after standardization in step (3.3) to evaluate the correlation between the features. The covariance matrix used in the present invention is a symmetric matrix, which reveals the strength of the correlation between the features;

[0091] (3.5) In order to ensure that the key information of high-dimensional feature data is retained as much as possible during the dimensionality reduction process, the present invention performs eigenvalue decomposition on the covariance matrix in step (3.4), arranges the eigenvalues ​​in descending order, calculates the cumulative contribution rate and selects the top k principal components with a cumulative contribution rate of 95%, and constructs the eigenvector after dimensionality reduction.

[0092] In the actual deployment of step (4), many telemetry tasks are carried out simultaneously, that is, the probe message needs to collect multiple network status data at each hop node. In the case of multiple tasks, each task data can reflect the status characteristics of the telemetry node. However, the possible correlation between different telemetry tasks will cause the final generated state factor to be inaccurate. Therefore, based on the dimensionality reduction results of step (3), the present invention processes the multi-source data features generated by multiple telemetry tasks, fuses the different task features after standardization, and uses the fusion result as a unified comprehensive state factor for the importance scoring model;

[0093] In one embodiment of the present invention, the steps of performing telemetry task feature standardization and task feature weight calculation are as follows:

[0094] (4.1) Multi-task telemetry requires the collection of n sets of indicators (delay, bandwidth, etc.), and the telemetry result set of each set of indicators is i = {x′ i1 ,x′ i2 ,···,x′ ik Considering that the values ​​of various indicators may not be of the same order of magnitude, and the positive and negative attributes of various telemetry tasks as network health evaluation indicators are different: for positive indicators, the larger the corresponding collected information, the better the network condition (such as the available bandwidth of the port), and for negative indicators, the smaller the corresponding information, the better the network condition (such as the number of packet losses). Therefore, for telemetry result sets of different projects, it is necessary to pre-calculate x i To do standardized processing:

[0095] standardization: Where k is the number of probe telemetry information collected periodically, x′ ik is a single telemetry result;

[0096] (4.2) After standardizing the indicators, the task characteristics are weighted:

[0097] Weight w i calculate:

[0098] where X′ i ={x′ i1 ,x′ i2 ,···,x′ ik}, pcij Represents the Pearson correlation coefficient between telemetry indicators;

[0099] Among them CS i Represents the strength of association and is used to assign weights to different telemetry tasks;

[0100] where w i Represents the weight of the telemetry task.

[0101] (4.3)The formula for calculating the state factor is as follows:

[0102] Factor state =w1×F1+w2×F2+…+w n ×F n , where Factor state Represents the comprehensive state factor of the node, w i represents the weight of the telemetry task, F i Represents the state features after dimensionality reduction obtained under the telemetry task.

[0103] Step (5) Given a network topology with n nodes, construct the spatial importance and comprehensive state importance vectors of the entire network according to the spatial factor and comprehensive state factor of each node obtained in step (2) and step (4);

[0104] In one embodiment of the present invention, the specific implementation steps of the important node scoring model are as follows:

[0105] (5.1) First, based on the spatial factor of each telemetry node, the spatial importance vector of the entire network is obtained:

[0106] Where V spatial represents the spatial importance vector, is the spatial factor found in step (2).

[0107] (5.2) According to the telemetry state cycle, the highest value among the comprehensive state factors generated by the node ports is selected as the optimal comprehensive state factor, and is used to calculate the state importance vector of the network:

[0108] in represents the periodic comprehensive state factor of port j of node i calculated using the calculation formula in step (4.3), is the optimal comprehensive state factor state of the node;

[0109] Where V state Represents the state importance vector of the node, is the optimal comprehensive state factor obtained in step (5.2).

[0110] Step (6) Based on the spatial importance vector and the optimal state importance vector obtained in step (5), the importance scoring model of the telemetry node is constructed using the Hadamard product. By calculating the score, the importance of the nodes is arranged in descending order, and the high priority nodes are marked;

[0111] In one embodiment of the present invention, the comprehensive eigenvalue is used as input, and the corresponding elements of two matrices of the same size, namely, the space vector and the state vector, are multiplied one by one by using the Hadamard product operation, so as to construct an importance scoring model. The scoring formula is as follows:

[0112] Among them, ⊙ represents Hadamard product, Tele imp Represents the telemetry node importance vector, V spatial Represents the spatial importance vector, V state Represents the state importance vector of the node, and are the spatial factor and comprehensive state factor obtained in steps (2) and (5.2) respectively.

[0113] Step (7) combines the historical telemetry data collected in step (1) and the node importance score result calculated in step (6) to set a score threshold σ. Nodes with node importance scores exceeding σ are screened as priority telemetry objects and serve as key targets for probe resource allocation. After the initial threshold is set, during the periodic operation of the telemetry system, the latest information on network topology and traffic status is collected in real time, and the scoring model is continuously optimized and the threshold σ is updated through a dynamic feedback mechanism to ensure the real-time and accuracy of the screening results;

[0114] In one embodiment of the present invention, the specific implementation steps of the dynamic adjustment mechanism and node screening are as follows:

[0115] (7.1) According to the telemetry feedback data and a large amount of prior data, the scoring threshold σ is set;

[0116] (7.2) After multiple rounds of telemetry, all nodes are re-evaluated with new spatial factors and state factors, and the list of important nodes is updated to ensure that the screening results match the real-time network status.

[0117] Step (8) To ensure the accuracy and reliability of the important node screening, the nodes screened in step (7) are applied to the anomaly detection and active telemetry probe allocation optimization experiment. Specifically, it includes verifying the effectiveness of the screening results, evaluating its performance by calculating the node coverage rate and the improvement ratio of anomaly detection ability, and comparing the high-scoring node coverage rate, anomaly detection ability and other indicators with the existing methods. Figure 2 The working framework for verification and application of important nodes in active telemetry systems is demonstrated.

[0118] In one embodiment of the present invention, the specific implementation steps of the important node screening result verification are as follows:

[0119] Step (8.1) simulates abnormal events in the test network environment, including but not limited to traffic surge, packet loss rate increase, delay jitter, etc. Compare these events with the selected important node locations, and record the proportion of abnormal events captured by important nodes (capture rate);

[0120] Step (8.2) calculates the coverage based on the records obtained in step (8.1) to evaluate the effectiveness of the screening nodes in anomaly detection compared to the probe configuration with full network coverage.

[0121] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A method for selecting important nodes for active telemetry scenarios, characterized in that: The method comprises the following steps: Step (1) In the telemetry system, network data plane data is collected through distributed probes. The data includes delay information, bandwidth occupancy, throughput information and other states of the network nodes. To ensure data quality, noise filtering and missing value filling algorithms are used to clean the data. The cleaned data is stored in the form of a time series matrix for subsequent feature extraction and analysis. Step (2) extracts the topological information of the nodes based on the basic data collected in step (1), analyzes the spatial factors affecting the nodes, and uses a weighted TOPSIS method to integrate multiple centrality measurements to obtain a spatial factor that is one of the important node evaluation indicators; Step (3) extracts important features that affect the node state based on the basic data collected in step (1), including dynamic features such as significant change frequency and state entropy. The significant change frequency is obtained by counting the number of significant changes in the node's periodic historical telemetry data. The state entropy is calculated by calculating the state fluctuation in the node's periodic historical telemetry data. A node state feature matrix is ​​formed based on the calculation results, and then high-dimensional state features are extracted. In order to eliminate feature redundancy and multicollinearity, principal component analysis (PCA) is used for dimensionality reduction processing; Step (4) In the actual deployment of telemetry, many tasks are carried out simultaneously, that is, the probe message needs to collect multiple network status data at each hop node. In the case of multiple tasks, each task data can reflect the status characteristics of the telemetry node. Based on the dimensionality reduction results in step (3), the multi-source data features generated by multiple telemetry tasks are processed, and the different task features are fused after standardization. The fusion result is used as a unified comprehensive status factor for the importance scoring model; Step (5) Given a network topology with n nodes, construct the spatial importance and comprehensive state importance vectors of the entire network according to the spatial factor and comprehensive state factor of each node obtained in step (2) and step (4); Step (6) Based on the spatial importance vector and the optimal comprehensive state importance vector obtained in step (5), a Hadamard product is used to construct an importance scoring model for the telemetry node, and the importance of the nodes is arranged in descending order by calculating the scores, and high priority nodes are marked; Step (7) combines the historical telemetry data collected in step (1) and the node importance score result calculated in step (6) to set a score threshold σ. Nodes with a node importance score exceeding σ are screened as priority telemetry objects as the key target of probe resource allocation. After the initial threshold is set, the latest information on network topology and traffic status is collected in real time during the periodic operation of the telemetry system. The scoring model is continuously optimized and the threshold σ is updated through a dynamic feedback mechanism to ensure the real-time and accuracy of the screening results. Step (8) To ensure the accuracy and reliability of the screening of important nodes, the nodes screened out in step (7) are applied to the allocation optimization experiment of anomaly detection and active telemetry probes, which specifically includes verifying the effectiveness of the screening results, evaluating its performance by calculating the improvement ratio of node coverage and anomaly detection capability, and comparing the high-scoring node coverage and anomaly detection capability indicators with existing methods.

2. According to claim 1, a method for selecting important nodes for active telemetry scenarios is characterized in that: The step (1) specifically comprises the following sub-steps: (1.1) In the network telemetry system, the control plane sends the probe configuration policy to the telemetry source node of the data plane through the north-south communication interface. The configuration includes the probe path, probe identification header information and probe collection frequency; (1.2) The telemetry probe is injected into the network according to the configuration strategy in step (1.1). The data plane node processes the data packet matching the probe identification header hop by hop, adds node status information hop by hop, and transmits it to the telemetry tail node along the path; (1.3) The telemetry tail node packages the hop-by-hop data set from the starting node to the terminal node and uploads it to the telemetry server. The telemetry server performs noise filtering, missing value filling and normalization on the collected raw data to ensure data quality and consistency. The cleaned data is stored in a structured format for subsequent feature extraction and analysis.

3. The important node screening method for active telemetry scenarios according to claim 1 is characterized in that: Based on the basic data collected in step (1), the topological information of the nodes is extracted and the spatial factors affecting the nodes are analyzed. The calculation method uses the weighted TOPSIS method to integrate multiple centrality measurements to obtain the spatial factor, which is one of the important node evaluation indicators: Factor spatial =R=TOPSIS(M,W) in: DC i Degree Centrality (DC) is the number of nodes directly connected to node i; CC i Closeness Centrality (CC), which is the average path length between node i and other nodes; EC i Eigenvector Centrality (EC) is the linear combination of the importance of neighboring nodes of node i; Factor spatial It is a spatial factor and one of the important indicators for evaluating important nodes.

4. The important node screening method for active telemetry scenarios according to claim 1 is characterized in that: The step (3) specifically comprises the following sub-steps: (3.1) By performing time series analysis on the node traffic and connection status data collected in step (1), the number of significant changes in the node's periodic historical telemetry data is counted; (3.2) Based on the state fluctuations in the periodic historical telemetry data, the node state entropy is calculated to evaluate the stability of its state; (3.3) The state feature data is standardized. After standardization, the mean of all feature data is 0 and the variance is 1, ensuring that the weights of different features are balanced during the dimensionality reduction process; (3.4) Calculate the covariance matrix C for the feature data matrix X after standardization in step (3.3) to evaluate the correlation between the features. The covariance matrix used is a symmetric matrix, which reveals the strength of the correlation between the features; (3.5) Perform eigenvalue decomposition on the covariance matrix in step (3.4), arrange it in descending order according to the size of the eigenvalues, calculate the cumulative contribution rate and select the first k principal components with a cumulative contribution rate of 95%, and construct the eigenvector after dimensionality reduction.

5. The important node screening method for active telemetry scenarios according to claim 1 is characterized in that: The step (4) specifically comprises the following sub-steps: (4.1) Multi-task telemetry requires collecting n sets of indicators (delay, bandwidth, etc.), and the telemetry result set of each set of indicators is X i ={x′ i1 ,x′ i2 ,···,x′ ik }, considering that the values ​​of various indicators may not belong to the same order of magnitude, and the positive and negative attributes of various telemetry tasks as network health evaluation indicators are different: for positive indicators, the larger the corresponding collected information, the better the network condition (such as the available bandwidth of the port), and for negative indicators, the smaller the corresponding information, the better the network condition (such as the number of packet losses). Therefore, for the telemetry result sets of different projects, it is necessary to pre-calculate x i To do standardized processing: standardization: Where k is the number of probe telemetry information collected periodically, x′ ik is a single telemetry result; (4.2) After standardizing the indicators, the task characteristics are weighted: Weight w i calculate: where X′ i ={x′ i1 ,x′ i2 ,···,x′ ik }, pc ij Represents the Pearson correlation coefficient between telemetry indicators; Among them CS i Represents the strength of association and is used to assign weights to different telemetry tasks; where w i Represents the weight of the telemetry task, (4.3)The formula for calculating the state factor is as follows: Factor state =w1×F1+w2×F2+…+w n ×F n , where Factor state Represents the comprehensive state factor of the node, w i represents the weight of the telemetry task, F i Represents the state features after dimensionality reduction obtained under the telemetry task.

6. The important node screening method for active telemetry scenarios according to claim 1 is characterized in that: The step (5) specifically comprises the following sub-steps: Step (5) Given a network topology with n nodes, construct the spatial importance and comprehensive state importance vectors of the entire network according to the spatial factor and comprehensive state factor of each node obtained in step (2) and step (4); The specific implementation steps of the important node scoring model are as follows: (5.1) First, based on the spatial factor of each telemetry node, the spatial importance vector of the entire network is obtained: Where V spatial represents the spatial importance vector, is the spatial factor found in step (2), (5.2) According to the telemetry state cycle, the highest value among the comprehensive state factors generated by the node ports is selected as the optimal comprehensive state factor, and is used to calculate the state importance vector of the network: in represents the periodic comprehensive state factor of port j of node i calculated using the calculation formula in step (4.3), is the optimal comprehensive state factor state of the node; Where V state Represents the state importance vector of the node, is the optimal comprehensive state factor obtained in step (5.2).

7. The important node screening method for active telemetry scenarios according to claim 1 is characterized in that: In step (6), the Hadamard product is used to construct an importance scoring model for telemetry nodes. By calculating the scores, the importance of the nodes is arranged in descending order, and high-priority nodes are marked. The scoring formula is as follows: Among them, ⊙ represents Hadamard product, Tele imp Represents the telemetry node importance vector, V spatial Represents the spatial importance vector, V state Represents the state importance vector of the node, and are the spatial factor and comprehensive state factor obtained in steps (2) and (5.2) respectively.

8. The important node screening method for active telemetry scenarios according to claim 1 is characterized in that: The step (7) specifically comprises the following sub-steps: (7.1) According to the telemetry feedback data and a large amount of prior data, the scoring threshold σ is set; (7.2) After multiple rounds of telemetry, all nodes are re-evaluated with new spatial factors and state factors, and the list of important nodes is updated to ensure that the screening results match the real-time network status.

9. The important node screening method for active telemetry scenarios according to claim 1 is characterized in that: The step (8) specifically comprises the following sub-steps: Step (8.1) simulates abnormal events in the test network environment, including traffic surge, packet loss rate increase, and delay jitter, compares these events with the selected important node locations, and records the proportion of abnormal events captured by important nodes (capture rate); Step (8.2) calculates the coverage based on the records obtained in step (8.1) to evaluate the effectiveness of the screening nodes in anomaly detection compared to the probe configuration with full network coverage.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, an important node screening method for active telemetry scenarios as described in any one of claims 1 to 9 is implemented.

Images