A packet loss root cause positioning method based on a heterogeneous graph

Abstract

The application provides a packet loss root cause positioning method based on a heterogeneous graph, and belongs to the technical field of network operation and maintenance. The method collects multi-source measurement data of network interface level, queue level and global quality, calculates feature increments, and constructs a heterogeneous network topology graph containing host and switch nodes; uses a heterogeneous graph attention network to aggregate differentiated neighbor features, and generates node high-order embedding vectors; uses a multi-task learning framework to perform output fault distribution, link anomaly scoring and performance prediction in parallel, and generates a root cause report based on a counterfactual explanation mechanism of gradient backpropagation. The application effectively solves the problem of difficulty in distinguishing end-to-end congestion, and realizes high-precision and interpretable root cause positioning.

Description

Technical Field

[0001] This invention belongs to the field of network operation and maintenance technology, and in particular relates to a method for locating the root cause of packet loss based on heterogeneous graphs. Background Technology

[0002] In existing network operation and maintenance and fault diagnosis systems, the root cause of packet loss is usually located by actively probing tools such as SNMP polling or Ping to obtain network status, which results in coarse sampling granularity and delayed location. Although some diagnostic methods based on graph convolutional networks have attempted to introduce artificial intelligence for automated analysis, these methods typically model the network topology as a homogeneous graph, treating end-side host nodes and network-side switch nodes as entities with the same attributes. This ignores the essential differences in traffic behavior and fault propagation patterns between end-side network interface card queues and intermediate forwarding devices, making it difficult for the model to effectively distinguish between end-side congestion and network-side congestion, and easily leading to misjudgments in location.

[0003] Furthermore, existing data-driven models often directly use the cumulative count statistics of device interfaces as input features without performing incremental data cleaning and preprocessing for fine-grained time windows. This results in the introduction of a large amount of historical data noise into the model, masking the true network behavior in the short term. This coarse-grained feature construction method makes it difficult for the model to effectively capture packet loss features that occur at the microsecond level and are instantaneous.

[0004] Finally, current deep learning models generally suffer from "black box" decision-making defects, only able to output packet loss categories without interpretable feature attribution and counterfactual inference evidence. Operation and maintenance personnel find it difficult to quickly verify the source of the fault and formulate accurate repair strategies based on the probability results output by the model, resulting in low efficiency in troubleshooting network faults and failing to meet the automated operation and maintenance needs of high-reliability production environments. Summary of the Invention

[0005] Purpose of the invention: To address the aforementioned problems, this invention proposes a heterogeneous graph-based root cause localization method for packet loss. By distinguishing the heterogeneous characteristics of hosts and switches through heterogeneous graphs, it utilizes incremental features to capture instantaneous network changes, and employs a heterogeneous graph attention network and a multi-task decoder, combined with a counterfactual explanation mechanism, to achieve end-to-end network collaborative root cause localization and interpretable analysis.

[0006] This invention proposes a method for locating the root cause of packet loss based on heterogeneous graphs, specifically including the following steps:

[0007] S1. Constructing a heterogeneous network topology: Based on the collected interface-level statistics, queue-level statistics, and end-to-end connectivity measurement data of host nodes and switch nodes, calculate the feature increments of measurement data within adjacent time windows and standardize them. Construct a heterogeneous network topology map containing host nodes, switch nodes, and various edge types based on physical connections:

[0008] S1.1 Extract the current sampling time from the acquired measurement data sequence. Compared with the previous sampling time Data records, calculation time window The feature increment of the measurement data is used to parse the queue backlog status string into byte values. The feature increment includes the received byte increment, the sent byte increment, the received error increment, the sent error increment, and the queue packet loss increment.

[0009] S1.2 Constructing a heterogeneous network topology diagram , where the set of nodes The set of edges includes host nodes and switch nodes with device type labels. This includes the uplink connection from the host node to the switch node, the downlink connection from the switch node to the host node, and the forwarding connection between switch nodes. The feature increment calculated in step S1.1 is standardized and used as the initial feature matrix of the corresponding node. .

[0010] S2. Generate high-order node embedding vectors: Input the heterogeneous network topology graph obtained in step S1 into the pre-trained heterogeneous graph attention network, calculate attention weights independently based on edge relationship types, aggregate multi-type neighbor features through meta-path mechanism, capture the propagation patterns of end-side congestion and network-side faults, and generate high-order node embedding vectors that integrate local and global semantics.

[0011] S3. Multi-task parallel decoding output: The high-order node embedding vectors obtained in step S2 are processed using a multi-task decoder to obtain the fault category probability distribution, link anomaly score, and predicted values ​​of key performance indicators; wherein, the multi-task decoder is obtained through supervised training:

[0012] S3.1 Map the node high-order embedding vectors output in step S2 to the common feature space;

[0013] S3.2 Constructs a global classification, root cause localization, and auxiliary regression decoding head to perform global pooling of the node features of the entire graph and output the probability distribution of fault categories including normal, congested, and link error through a multilayer perceptron. It concatenates the embedding vectors of the source node and target node connected to each link and outputs the link's anomaly score. It also uses the global feature vector to output the predicted values ​​of key performance indicators, including average latency and packet loss rate.

[0014] S4. Counterfactual Root Cause Localization: When the fault category probability distribution indicates an anomaly, fix the parameters of the heterogeneous graph attention network and the multi-task decoder, and find the minimum feature perturbation that flips the predicted value of the key performance index obtained in step S3 back to normal by gradient optimization of the feature mask matrix. Based on this, screen the feature increment that contributes the most and its corresponding root cause device and link:

[0015] S4.1 When the fault category probability distribution indicates an anomaly, i.e., indicates either congestion or link error, obtain the initial feature matrix of the node mentioned in step S1.2. , building and Feature mask matrices of the same dimension By and The perturbation feature matrix is ​​obtained by element-wise multiplication. ;

[0016] S4.2 Fix the parameters of the heterogeneous graph attention network and the multi-task decoder to obtain the perturbation feature matrix. The prediction result after input; constructing an optimization objective function, which includes the cross-entropy loss between the perturbed prediction result and the normal class, and the mask matrix. The sparse regularization term; the mask matrix is ​​iteratively updated using the gradient descent algorithm. This continues until either the preset convergence condition is met or the maximum number of iterations is reached.

[0017] S4.3 Extract the mask matrix The report generates a counterfactual explanation containing key root cause indicators by taking the top-K feature dimensions with the highest weights and their corresponding node identifiers or link identifiers.

[0018] Beneficial effects:

[0019] 1. This invention constructs a heterogeneous network topology graph that includes host nodes, switch nodes and various connection relationships, and uses a heterogeneous graph attention network to aggregate different types of neighbor features, thereby capturing the essential differences in traffic behavior between end-side devices and core forwarding devices.

[0020] 2. This invention abandons the single classification task architecture and adopts a shared encoder to drive three tasks—fault classification, link localization, and performance regression—in parallel. By introducing physical performance indicators as auxiliary constraints, the generalization ability of the model is enhanced by utilizing shared information among multiple tasks. At the same time, the specially designed root cause localization decoding head can directly output the anomaly score of each physical link, realizing fine-grained packet loss localization from the "network element level" to the "link level," meeting the needs of refined operation and maintenance.

[0021] 3. This invention innovatively introduces a counterfactual explanation mechanism based on gradient backpropagation. This mechanism not only outputs packet loss classification results but also automatically generates a report containing key root cause indicators by calculating the minimum feature perturbation. This enables operations and maintenance personnel to "know not only what happened but also why," completing root cause localization and repair based on the report, greatly reducing the threshold and time cost of fault diagnosis, and significantly improving the transparency and interpretability of intelligent diagnosis. Attached Figure Description

[0022] Figure 1 This is a schematic diagram of a heterogeneous graph-based method for locating the root cause of packet loss.

[0023] Figure 2 This is a schematic diagram illustrating the construction logic and structure of a heterogeneous network topology.

[0024] Figure 3 This is an architecture diagram of a multi-task parallel decoding network.

[0025] Figure 4 This is a flowchart for root cause localization and report generation based on counterfactual interpretation. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the embodiments of this invention are described in detail below with reference to the accompanying drawings. These embodiments are implemented based on the technical solutions of this invention, providing detailed implementation methods and specific operating procedures. It should be understood that the specific examples described herein are merely illustrative of this invention, but the scope of protection of this invention is not limited to the following embodiments.

[0027] This invention proposes a heterogeneous graph-based root cause localization method for packet loss within the existing network operation and maintenance and fault diagnosis system. The overall processing flow is shown in the attached figure. Figure 1 As shown, network interface-level, queue-level, and global quality data are collected and processed, and a heterogeneous network topology graph is constructed using unified encoding. A heterogeneous graph attention network is used to model relationships and aggregate features of different types of nodes and edges, extracting high-order semantic features representing network states. Based on the learned graph embeddings, fault classification, root cause localization, and performance regression are performed in parallel. Finally, counterfactual reasoning is combined to generate packet loss analysis results and an explanation report, providing a basis for network operation and maintenance decisions.

[0028] As attached Figure 2 To be continued Figure 4 As shown, the present invention provides a method for locating the root cause of packet loss based on heterogeneous graphs, which specifically includes the following steps:

[0029] S1. Construct a heterogeneous network topology diagram:

[0030] This method first preprocesses the collected raw time-series measurement data and constructs a heterogeneous network topology.

[0031] The measurement data includes:

[0032] Interface-level statistics: For each host node and switch node in the entire network, interface-level statistics are obtained by reading the kernel interface. Specific fields include: received bytes (rx_bytes), sent bytes (tx_bytes), received error packets (rx_errors), and sent error packets (tx_errors).

[0033] Queue-level statistics: For each network interface, queue-level statistics are read using the flow control tool tc. Specifically, the output of the command `tc -s qdisc show` is analyzed to extract the packet drop count `tc_drops` and the queue backlog status string `tc_backlog`.

[0034] Global quality data: Collect end-to-end global measurement data, including average round-trip latency (avg_latency) and packet loss rate (packet_loss). If a probe times out or the packet loss rate reaches 100%, the latency is marked as a sentinel value (e.g., 9999) for subsequent processing. The specific processing flow is as follows:

[0035] Feature increment calculation: The program reads the current sampling time. Compared with the previous sampling time Data records, calculation time window The feature increment within. For example, the formula for calculating the interface receive rate rx_rate_bps is: ,in, Indicates the current sampling time Number of bytes received Indicates the previous sampling time The number of bytes received; the formula for calculating the queue packet loss increment drops_delta is: ,in, Indicates the current sampling time The number of packets lost in the flow control queue. Indicates the previous sampling time The number of packet losses in the flow control queue. At the same time, the obtained tc_backlog string is parsed into a uniform byte value, such as parsing "10Kb" into a 10240-byte value.

[0036] Numerical standardization: All incremental features are subjected to Z-Score standardization, i.e. ,in The mean of the training set, Standard deviation, This represents the standardized data. This represents the original measurement data, which aims to eliminate numerical differences between indicators of different dimensions (such as rate and number of packet losses); the specific measurement and calculation data indicators are listed in Table 1.

[0037] Table 1 Measurement and Calculation Data Indicators

[0038] Heterogeneous graph instantiation: as attached Figure 2 As shown, a heterogeneous network topology diagram is constructed. Specifically, it includes:

[0039] Node set Nodes are classified as host nodes based on their device type labels. and switch nodes For the host node, its initial characteristic matrix Key features include network interface card (NIC) queue backlog, transmit rate, and receive rate; for switch nodes, their initial feature matrix... Key features include port queue packet loss and forwarding error counts.

[0040] edge set Define three types of directed connection relationships: , ,as well as ,in , Representing physical links The source node and the target node, This represents the set of uplink connections that a host points to on the switch. This represents the set of downlink connections from the switch to the host. This represents the set of core forwarding connections that the switch points to.

[0041] S2. Generate higher-order node embedding vectors:

[0042] The constructed heterogeneous network topology graph is input into a pre-trained heterogeneous graph attention network model, specifically, the model contains multiple stacked HeteroGAT layers. Within each layer, attention is applied to each type of edge relationship. (e.g., uplink, downlink, forwarding), initialize independent linear transformation matrices respectively. and attention vector Next, for the target node... The model first calculates its relationship with its neighboring nodes. Attention coefficient between ,in Represents the normalized exponential function, Denotes the modified linear unit activation function with leakage, superscript Represents the transpose of a vector. and Representing nodes respectively Feature vectors and nodes Feature vectors; attention coefficients This dynamically reflects the weighted impact of neighboring node failure states on the target node at a specific moment. For example, when a switch experiences congestion, the attention weight of its connected hosts increases significantly; subsequently, the calculated attention coefficients are used to weight and aggregate the features of neighboring nodes to update the node. Feature representation : ,in Indicates a relationship Next node The set of neighboring nodes; finally, through multi-layer network stacking and non-linear activation functions. Generate high-order embedding vectors that contain multi-hop neighborhood information. .

[0043] S3, Multi-task parallel decoding output:

[0044] As attached Figure 3 As shown, a multi-task learning framework is used to map high-order embedding vectors to nodes in the common feature space. Parallel decoding processing includes three types of tasks:

[0045] Fault Classification: Constructing a global classification decoding head. First, global average pooling is performed on the host node features and switch node features of the entire graph, and then the two are concatenated to obtain a graph-level feature vector. .Will Input a multilayer perceptron and output the network's current fault category probability distribution. The categories include normal, congested, and link error.

[0046] Root cause localization: Construct the root cause localization decoding head. For each physical link in the diagram... , its source node Embedded vector Target Node Embedded vector and edge features The data is then concatenated and input into the fully connected layer to calculate the anomaly score for the link. The score is mapped to using the Sigmoid function. The higher the score within a given range, the greater the likelihood that the link is the root cause of packet loss.

[0047] Performance Regression: Constructing an Auxiliary Regression Decoding Head. Utilizing Graph-Level Feature Vectors Predict the average latency and packet loss rate of the network. This task serves as a regularization constraint, forcing the model to learn feature representations strongly correlated with the network's physical performance, thereby improving generalization ability. Finally, a weighted loss function is used during model training. The parameters of each module are jointly optimized, among which... Represents classification loss. Indicates location loss. Indicates regression loss; and This represents the balancing hyperparameter.

[0048] S4. Counterfactual Root Cause Locating:

[0049] As attached Figure 4 As shown, when the fault classification result in step S3 is an abnormal state (such as congestion), the counterfactual interpretation process is initiated:

[0050] (1) Constructing a feature mask: Fix the parameters of the heterogeneous graph attention network and the multi-task decoder, and instantiate an initial feature matrix of the input nodes. Learnable mask matrix of the same dimension (Initial values ​​are all 1).

[0051] (2) Gradient optimization loop: Calculate the perturbed feature matrix Define an optimization objective function that makes the model perform optimally on input. The prediction results at that time approach the "normal state", while constraining the mask matrix. The sparsity of the mask matrix is ​​addressed by using gradient descent backpropagation to iteratively update the mask matrix. .

[0052] (3) Extract key root causes and generate a report: When the optimization converges or reaches the maximum number of iterations, extract the mask matrix. The system identifies the Top-K feature dimensions with the largest weight changes (i.e., those contributing most to packet loss detection) and their corresponding node or link IDs. Based on this, the system generates a counterfactual explanation report to accurately pinpoint the root cause.

Claims

1. A method for locating the root cause of packet loss based on heterogeneous graphs, characterized in that, The method includes the following steps: S1. Construct a heterogeneous network topology: Based on the collected interface-level statistics, queue-level statistics, and end-to-end connectivity measurement data of host nodes and switch nodes, calculate the feature increments of measurement data within adjacent time windows and standardize them. Construct a heterogeneous network topology containing host nodes, switch nodes, and various edge types based on physical connections. S2. Generate high-order node embedding vectors: Input the heterogeneous network topology graph obtained in step S1 into the pre-trained heterogeneous graph attention network, calculate attention weights independently based on edge relationship types, aggregate multi-type neighbor features through meta-path mechanism, capture the propagation patterns of end-side congestion and network-side faults, and generate high-order node embedding vectors that integrate local and global semantics. S3. Multi-task parallel decoding output: The node high-order embedding vector obtained in step S2 is processed by the multi-task decoder to obtain the fault category probability distribution, link anomaly score and key performance index prediction values; wherein, the multi-task decoder is obtained through supervised training; S4. Counterfactual Root Cause Localization: When the probability distribution of the fault category indicates an anomaly, fix the parameters of the heterogeneous graph attention network and the multi-task decoder, optimize the feature mask matrix through gradient, find the minimum feature perturbation that makes the predicted value of the key performance index obtained in step S3 flip to normal, and select the feature increment that contributes the most and its corresponding root cause device and link.

2. The method for locating the root cause of packet loss based on heterogeneous graphs according to claim 1, characterized in that, The specific implementation steps of step S1 include: S1.1 Extract the current sampling time from the acquired measurement data sequence. Compared with the previous sampling time Data records, calculation time window The feature increment of the measurement data is used to parse the queue backlog status string into byte values. The feature increment includes the received byte increment, the sent byte increment, the received error increment, the sent error increment, and the queue packet loss increment. S1.2 Constructing a heterogeneous network topology diagram , where the set of nodes The set of edges includes host nodes and switch nodes with device type labels. This includes the uplink connection from the host node to the switch node, the downlink connection from the switch node to the host node, and the forwarding connection between switch nodes. The feature increment calculated in step S1.1 is standardized and used as the initial feature matrix of the corresponding node. .

3. The method for locating the root cause of packet loss based on heterogeneous graphs according to claim 1, characterized in that, The specific implementation steps of step S3 include: S3.1 Map the node high-order embedding vectors output in step S2 to the common feature space; S3.2 Constructs a global classification, root cause localization, and auxiliary regression decoding head to perform global pooling of the node features of the entire graph and output the probability distribution of fault categories including normal, congested, and link error through a multilayer perceptron. It concatenates the embedding vectors of the source node and target node connected to each link and outputs the link's anomaly score. It also uses the global feature vector to output the predicted values ​​of key performance indicators, including average latency and packet loss rate.

4. The method for locating the root cause of packet loss based on heterogeneous graphs according to claim 2, characterized in that, The specific implementation steps of step S4 include: S4.1 When the fault category probability distribution indicates an anomaly, i.e., indicates either congestion or link error, obtain the initial feature matrix of the node mentioned in step S1.

2. , building and Feature mask matrices of the same dimension By and The perturbation feature matrix is ​​obtained by element-wise multiplication. ; S4.2 Fix the parameters of the heterogeneous graph attention network and the multi-task decoder to obtain the perturbation feature matrix. The prediction result after input; constructing an optimization objective function, which includes the cross-entropy loss between the perturbed prediction result and the normal class, and the mask matrix. The sparse regularization term; the mask matrix is ​​iteratively updated using the gradient descent algorithm. This continues until either the preset convergence condition is met or the maximum number of iterations is reached. S4.3 Extract the mask matrix The report generates a counterfactual explanation containing key root cause indicators by taking the top-K feature dimensions with the highest weights and their corresponding node identifiers or link identifiers.

Images