A method and system for realizing network topology status monitoring based on IP probe

Abstract

The present invention discloses a method and system for realizing network topology status monitoring based on IP probe, which belongs to the field of communication technology. The method mainly comprises the following steps: obtaining characteristic data of each node in the current network topology diagram according to the IP probe, determining a first abnormal node from each node according to the characteristic data and a first preset threshold value; constructing a first characteristic matrix according to the characteristic data, and obtaining a first adjacency matrix according to the current network topology diagram; inputting the first characteristic matrix and the first adjacency matrix into a preset model to obtain an abnormality score corresponding to the first abnormal node, wherein the preset model includes a graph convolutional neural network and a long short-term memory network; determining a second abnormal node in the first abnormal node according to the abnormality score and the second preset threshold value, and determining an abnormal source node according to the second abnormal node. The present invention captures the correlation characteristics of the device through the graph neural network, and obtains the fluctuation law of the device status in combination with the long short-term memory network, so as to realize the identification and positioning of unknown abnormal status.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and more particularly to a method and system for realizing network topology status monitoring based on IP probes. Background Art

[0002] With the acceleration of enterprise digital transformation, the scale of local area networks has continued to expand, the types and number of network devices have increased dramatically, and the complexity of network architecture has also increased, making network status monitoring increasingly difficult. However, traditional monitoring methods only monitor individual devices and are unable to accurately and comprehensively monitor cable connection status, link transmission quality, and network topology. Therefore, there are many shortcomings. Summary of the Invention

[0003] In response to the shortcomings of the existing technology, the purpose of the present invention is to provide a method and system for network topology status monitoring based on IP probes. Based on the real-time acquired network topology diagram, a graph neural network is used to capture device association characteristics, and a timing model is used to analyze the device status fluctuation pattern. Finally, combined with a dynamic threshold adaptive mechanism, the cable connection status, link transmission quality and unknown abnormal device status can be identified and located.

[0004] To achieve the above object, the present invention provides the following technical solutions:

[0005] The present invention provides a method for realizing network topology status monitoring based on IP probe, comprising:

[0006] Acquiring characteristic data of each node in a current network topology graph according to an IP probe, and determining a first abnormal node from each node according to the characteristic data and a first preset threshold, wherein the network topology graph is composed of a plurality of nodes and a plurality of edges, each node corresponding to a network device, and each edge representing a connection relationship between two network devices;

[0007] constructing a first characteristic matrix according to the characteristic data, and obtaining a first adjacency matrix according to the current network topology graph;

[0008] Inputting the first feature matrix and the first adjacency matrix into a preset model to obtain an anomaly score corresponding to the first abnormal node, wherein the preset model is trained based on the historical operation data of each node, and the preset model includes a graph convolutional neural network and a long short-term memory network;

[0009] According to the abnormality score and a second preset threshold, a second abnormal node is determined among the first abnormal nodes, and an abnormal source node is determined according to the second abnormal node.

[0010] As a further improvement of the present invention, the preset model is trained based on the historical operation data of each node, including:

[0011] Dividing the historical operation data into a training set and a validation set, and dividing the training set into multiple training subsets, each training subset corresponding to a moment;

[0012] For each training subset, performing a first iterative operation, the first iterative operation including: obtaining a current training model, generating a second feature matrix and a second adjacency matrix based on the training subset, inputting the second feature matrix and the second adjacency matrix into the graph convolutional neural network of the current training model to obtain a multi-scale neighborhood fusion feature, obtaining a weighted hidden state based on the training subset and the long short-term memory network in the current training model, fusing the multi-scale neighborhood fusion feature and the weighted hidden state through a gating mechanism to obtain a fused feature, calculating a loss function based on the fused feature and a fully connected layer, and updating the current training model according to the loss function until a preset termination condition is met to obtain a training model;

[0013] The preset model is obtained according to the training model and the verification set.

[0014] As a further improvement of the present invention, the second feature matrix and the second adjacency matrix are input into the graph convolutional neural network of the training model to obtain multi-scale neighborhood fusion features, including:

[0015] Determining the static weight of each node according to the number of edges and weight corresponding to each node in the training subset;

[0016] Determining a first neighboring node corresponding to each node according to the second adjacency matrix, and calculating features corresponding to the first neighboring node according to a preset time window;

[0017] Obtaining a timing weight corresponding to each of the first neighborhood nodes according to the features corresponding to the first neighborhood nodes;

[0018] The multi-scale neighborhood fusion feature is obtained according to the static weight, the temporal weight, and the feature vector of each node, and the feature vector is obtained according to the second feature matrix.

[0019] As a further improvement of the present invention, the multi-scale neighborhood fusion feature is obtained according to the static weight, the temporal weight and the feature vector of each node, including:

[0020] Determining a second neighboring node corresponding to the first neighboring node according to the second adjacency matrix, and calculating a feature corresponding to the second neighboring node according to the preset time window;

[0021] Obtaining a timing weight corresponding to each of the second neighborhood nodes according to the features corresponding to the second neighborhood nodes;

[0022] Obtaining a first fused feature according to the static weight, the temporal weight corresponding to the first neighborhood node, and the feature vector of each node;

[0023] Obtaining a second fused feature according to the static weight, the temporal weight corresponding to the second neighborhood node, and the feature vector of each node;

[0024] The multi-scale neighborhood fusion feature is obtained according to the first fusion feature and the second fusion feature.

[0025] As a further improvement of the present invention, obtaining a weighted hidden state based on the training subset and the long short-term memory network in the training model includes:

[0026] Obtaining an input feature vector corresponding to each time step in the long short-term memory network based on the training subset;

[0027] Obtain calculation results corresponding to the forget gate, input gate, output gate, and candidate calculation unit according to the input feature vector;

[0028] Update the memory unit according to the calculation results of the forget gate and the input gate;

[0029] Obtaining a hidden state according to the calculation result of the output gate and the updated memory unit;

[0030] The weighted hidden state is obtained according to the hidden state and the attention mechanism.

[0031] As a further improvement of the present invention, the multi-scale neighborhood fusion feature and the weighted hidden state are fused through a gating mechanism to obtain a fusion feature, including:

[0032] performing a linear transformation on the multi-scale neighborhood fusion features and the weighted hidden state according to a weight matrix corresponding to the gating mechanism;

[0033] According to the result of linear transformation and activation function, the gate value is obtained;

[0034] According to the gate value, the multi-scale neighborhood fusion feature and the weighted hidden state are fused to obtain the fusion feature.

[0035] As a further improvement of the present invention, the loss function is calculated based on the fusion features and the fully connected layer, including:

[0036] Inputting the fused features into the fully connected layer to obtain an output value;

[0037] Converting the output value into a probability value according to an activation function;

[0038] The loss function is calculated based on the probability value.

[0039] As a further improvement of the present invention, determining the abnormal source node according to the second abnormal node includes:

[0040] Clustering the feature data according to a clustering algorithm to obtain a plurality of clusters, each cluster corresponding to a node set;

[0041] Calculating the distance between each of the second abnormal nodes and its corresponding cluster center;

[0042] The abnormal source node is determined according to the distance.

[0043] As a further improvement of the present invention, determining the abnormal source node according to the distance includes:

[0044] If the distance between a second abnormal node and its corresponding cluster center exceeds a preset distance, the second abnormal node is regarded as the abnormal source node, and the preset distance is determined according to the average distance of the clusters corresponding to the cluster centers;

[0045] If the distance between the second abnormal node and its corresponding cluster center does not exceed the preset distance, obtaining its corresponding first adjacent node according to the first adjacency matrix;

[0046] The abnormal source node is determined according to the distance between the first adjacent node and its corresponding cluster center.

[0047] The present invention provides a system for realizing network topology status monitoring based on IP probes, comprising:

[0048] a collection module, configured to obtain characteristic data of each node in a current network topology graph based on an IP probe, and determine a first abnormal node from each node based on the characteristic data and a first preset threshold, wherein the network topology graph is composed of a plurality of nodes and a plurality of edges, each node corresponding to a network device, and each edge representing a connection relationship between two network devices;

[0049] a computing module, configured to construct a first feature matrix based on the feature data, and obtain a first adjacency matrix based on the current network topology; and input the first feature matrix and the first adjacency matrix into a preset model to obtain an anomaly score corresponding to the first abnormal node, wherein the preset model is trained based on the historical operating data of each node, and the preset model includes a graph convolutional neural network and a long short-term memory network;

[0050] A positioning module is configured to determine a second abnormal node among the first abnormal nodes according to the abnormal score and a second preset threshold, and determine an abnormal source node according to the second abnormal node.

[0051] The present invention collects network device connection events through IP probes and establishes a network topology graph. The parameters contained in the edges in the network topology graph can be used to monitor cable connection status and link transmission quality. Then, device information is acquired in real time through the network topology graph to obtain a first feature matrix and a first adjacency matrix. These are input into a preset model. Multi-scale neighborhood fusion features are obtained through GNN (graph neural network), and the weighted hidden state of the device's historical operation data is obtained through LSTM (long short-term memory network). The multi-scale neighborhood fusion features and the weighted hidden state are then fused through a gating mechanism to obtain a fused feature. Finally, the first abnormal node is determined based on the fused feature, and the second abnormal node and the abnormal source node are further determined based on the first abnormal node. The present invention fuses and analyzes the multi-scale neighborhood fusion features and the weighted hidden state to capture abnormal features from multiple angles, improving the accuracy of anomaly identification. At the same time, the dynamic threshold mechanism ensures that abnormal device status can be effectively detected under different network conditions. The combination of GNN, LSTM, and dynamic threshold forms a complete system that can accurately identify and locate unknown abnormal state devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the steps of the present invention;

[0053] Figure 2 Schematic diagram of graph convolutional neural network;

[0054] Figure 3 Schematic diagram of long short-term memory network;

[0055] Figure 4 A schematic diagram of a node cluster. DETAILED DESCRIPTION

[0056] The technical solution of the present invention is described in detail below through the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations of the technical solution of the present invention.

[0057] Identical components are denoted by the same reference numerals. It should be noted that the terms "front," "rear," "left," "right," "upper," and "lower" used in the following description refer to directions in the accompanying drawings, and the terms "bottom," "top," "inner," and "outer" refer to directions toward or away from the geometric center of a particular component, respectively.

[0058] The term "and / or" in the following text simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Additionally, the character " / " generally indicates an "or" relationship between the related objects.

[0059] like Figure 1 As shown, the embodiment of the present application provides a method for implementing network topology status monitoring based on IP probes, including:

[0060] Acquire characteristic data of each node in a current network topology graph according to the IP probe, and determine a first abnormal node from each node according to the characteristic data and a first preset threshold value, wherein the network topology graph is composed of a plurality of nodes and a plurality of edges, each node corresponds to a network device, and each edge represents a connection relationship between two network devices;

[0061] Constructing a first characteristic matrix according to the characteristic data, and obtaining a first adjacency matrix according to the current network topology graph;

[0062] Inputting the first feature matrix and the first adjacency matrix into a preset model to obtain an anomaly score corresponding to the first abnormal node, the preset model is trained based on the historical operation data of each node, and the preset model includes a graph convolutional neural network and a long short-term memory network;

[0063] According to the abnormality score and the second preset threshold, a second abnormal node is determined in the first abnormal node, and the abnormal source node is determined according to the second abnormal node.

[0064] Among them, the current network topology map needs to be obtained based on the IP probe and incremental graph structure update mechanism. Specifically, the network device connection events (such as port plugging and unplugging, VLAN configuration changes, and routing table updates) are first collected in real time based on the IP probe, and then the network topology map is updated through incremental learning algorithms (such as GraphSAGE's dynamic node embedding method). For example, suppose a server (Server-1) is connected to port GE0 / 1 of an access layer switch (Switch-A). The network topology needs to be updated based on this event. First, an IP probe is performed to obtain the port status of Switch-A via SNMP. The port status of GE0 / 1 changes from "DOWN" to "UP" and the MAC address of the peer is read (via LLDP). If Server-1 is newly connected, a node is created in the current network topology. A directed edge is then created between Switch-A and Server-1. The GraphSAGE algorithm is then used to update the neighboring node embeddings and recalculate the vector representations of Switch-A and its downstream devices. Nodes contain basic device information and real-time status (such as CPU utilization and memory usage), while edges contain connection parameters (bandwidth, latency, and MTU) and historical anomaly records (such as the number of link outages in the past 24 hours. Links can be physical, such as cables, or virtual, such as virtual forwarding paths). Furthermore, this mechanism uses ICMP ping to scan all nodes every 10 minutes. Devices that fail to respond three times in a row are marked as "offline," triggering edge deletion and node archiving. Based on the above mechanism, IP probes can be used to update the network topology map in real time during the monitoring of network device status, thereby obtaining the current network topology map and obtaining the characteristic data of each node based on the information stored in each node and edge of the network topology map, such as CPU utilization, memory usage, and bandwidth.

[0065] Furthermore, the first preset threshold is determined based on the device type and load period. Taking the bandwidth utilization of a network device as an example, we first obtain the bandwidth utilization of the network device during the past 30 days when no anomalies occur (the sampling period is determined to be 1 minute or 1 hour, depending on the actual situation). This data is then categorized by load period, where load periods are divided into high-load periods (9:00-18:00 on weekdays), medium-load periods (18:00-24:00 on weekdays and 9:00-18:00 on weekends), and low-load periods (0:00-9:00 every day and 18:00-24:00 on weekends). For each period, the first preset threshold for the device's bandwidth utilization is determined based on the corresponding sample data and the 3σ principle. Based on this, the first preset threshold for each feature of each device in each period is determined. Each feature data point of each node is then compared with the corresponding first preset threshold. If a feature data point for a node contains a value greater than the first preset threshold, the node is considered a suspicious node with a potential anomaly and is designated as the first abnormal node. There can be multiple first abnormal nodes, and the first preset threshold value can change over time.

[0066] Then, the corresponding eigenvector is constructed according to the characteristic data of each node, and the eigenvectors are combined to obtain the first characteristic matrix, where each row in the first characteristic matrix represents the eigenvector of a node, that is, the first characteristic matrix is Nodes dimensional vector expansion, for example, the first characteristic matrix is The matrix of represents the number of nodes, Represents the dimension of the eigenvector, that is, the number of elements contained in the eigenvector, and then obtains the first adjacency matrix based on the current network topology , the first adjacency matrix is Matrix, if the node With node are connected by an edge, then ,on the contrary, , for example, the rows and columns of the first adjacency matrix are The node number of each node, for example, there are four nodes abcd. When the row is a and the column is b, the corresponding matrix value is 1, which means that a and b have an adjacency relationship; when the row is b and the column is c, the corresponding matrix value is 0, which means that b and c have no adjacency relationship. Based on the first adjacency matrix, we can determine Which nodes in the nodes have a connection relationship? This application does not go into details about the first characteristic matrix and the first adjacency matrix.

[0067] Then, the first feature matrix and the first adjacency matrix are input into the preset model to obtain the anomaly score corresponding to the first abnormal node. If the anomaly score of a first abnormal node is greater than the second preset threshold, it is considered that the first abnormal node is indeed abnormal and is used as the second abnormal node. The abnormal source node is determined based on the second abnormal node.

[0068] This embodiment collects network device connection events through IP probes and establishes a network topology map. The parameters contained in the edges in the network topology map can be used to monitor the cable connection status and link transmission quality. Then, device information is obtained in real time through the network topology map, providing a global perspective for anomaly detection. The operating status of different nodes in different time periods is then monitored based on a dynamic first preset threshold. The fusion analysis of GNN (graph neural network) and LSTM (long short-term memory network) is then used to capture abnormal features from multiple angles, thereby improving the accuracy of anomaly identification. The combination of these three forms a complete system that can accurately identify and locate unknown abnormal status devices.

[0069] Furthermore, this embodiment provides a step of training a preset model based on the historical operation data of each node, including:

[0070] The historical running data is divided into a training set and a validation set, and the training set is divided into multiple training subsets, each training subset corresponds to a moment;

[0071] For each training subset, a first iterative operation is performed, which includes obtaining the current training model, generating a second feature matrix and a second adjacency matrix based on the training subset, inputting the second feature matrix and the second adjacency matrix into the graph convolutional neural network of the current training model to obtain multi-scale neighborhood fusion features, obtaining a weighted hidden state based on the long short-term memory network in the training subset and the current training model, fusing the multi-scale neighborhood fusion features and the weighted hidden state through a gating mechanism to obtain a fusion feature, calculating a loss function based on the fusion feature and the fully connected layer, and updating the current training model according to the loss function until a preset termination condition is met to obtain a training model.

[0072] According to the training model and validation set, the preset model is obtained.

[0073] Specifically, the historical operation data contains the characteristic data of multiple nodes and the connection relationship between the nodes. Based on the characteristic data and the connection relationship, the second characteristic matrix and the second adjacency matrix corresponding to each training subset can be generated. The generation method is the same as the first characteristic matrix and the first adjacency matrix, and this embodiment will not be repeated here. Then the historical operation data is divided into a training set and a verification set, and the training set and the verification set are divided into multiple training subsets and multiple verification subsets respectively. Each training subset and verification subset corresponds to a sampling time, and the nodes included in each training subset and verification subset are the same. The preset termination condition is that the above steps are completed for each training subset, that is, the number of iterations is equal to the number of training subsets.

[0074] After obtaining the training model, use each validation subset to validate the training model in turn, and calculate the value of the preset indicator based on the output results of the training model. The preset indicator can be accuracy or precision, etc. If the preset indicators corresponding to each validation subset can meet the preset standards, for example, the accuracy or precision reaches 80%, it means that the model training results are good, and the training model is used as the preset model. Otherwise, re-training is required to obtain the preset model.

[0075] This embodiment uses a graph convolutional neural network to effectively capture the spatial structural information of nodes, that is, the relationships and dependencies between nodes, and then uses a long short-term memory network to capture the long-term dependencies in the time series. The two are fused for feature, which can simultaneously utilize the spatial and temporal information of the node data to more comprehensively describe the characteristics of the data, enabling the model to learn more comprehensive and general feature patterns, reducing the risk of overfitting, thereby improving the generalization ability of the model and improving the accuracy of the model output results.

[0076] Furthermore, this embodiment provides a step of inputting the second feature matrix and the second adjacency matrix into the graph convolutional neural network of the training model to obtain multi-scale neighborhood fusion features, including:

[0077] Determine the static weight of each node based on the number of edges and weights corresponding to each node in the training subset;

[0078] Determine the first neighboring node corresponding to each node according to the second adjacency matrix, and calculate the features corresponding to the first neighboring node according to a preset time window;

[0079] Obtaining a time series weight corresponding to each first neighborhood node according to the features corresponding to the first neighborhood node;

[0080] According to the static weight, temporal weight and the feature vector of each node, the multi-scale neighborhood fusion feature is obtained, and the feature vector is obtained according to the second feature matrix.

[0081] Furthermore, this embodiment provides a step of obtaining a multi-scale neighborhood fusion feature based on a static weight, a temporal weight, and a feature vector of each node, including:

[0082] Determine a second neighboring node corresponding to the first neighboring node according to the second adjacency matrix, and calculate a feature corresponding to the second neighboring node according to a preset time window;

[0083] Obtaining a time series weight corresponding to each second neighborhood node according to the features corresponding to the second neighborhood node;

[0084] Obtain a first fusion feature based on the static weight, the temporal weight corresponding to the first neighborhood node, and the feature vector of each node;

[0085] Obtain a second fusion feature according to the static weight, the temporal weight corresponding to the second neighborhood node, and the feature vector of each node;

[0086] A multi-scale neighborhood fusion feature is obtained according to the first fusion feature and the second fusion feature.

[0087] Specifically, first determine the static weight of each node according to the number of edges and weights corresponding to each node in the training subset. For example, for node , first obtain its corresponding degree centrality, that is, the node The number of connected edges, then according to the node The type of node gets its corresponding weight, for example, if the node If the corresponding network device is a server, its corresponding weight is 0.5. If it is a convergence layer switch, the weight is 0.8. If it is a common terminal, the weight is 0.3. Then get the corresponding node Nodes connected by an edge are treated as nodes For each first neighbor node, the corresponding degree centrality and weight are obtained according to the above method, and the node is calculated. The static weight of :

[0088]

[0089] in, Representation node The degree centrality of Representation node The weight of Represented by the node The set of all first neighboring nodes is called the neighborhood set. Represents any first neighbor node in the set.

[0090] Then, for each first neighboring node, calculate its Features within Indicates the current moment, Indicates the length of the time window, which can be determined according to the actual situation. Preferably, the length should include three sampling moments. Since the current training subset only includes the data of the current moment, it is necessary to combine the data of other training subsets during calculation. For example, assuming the time window The three sampling moments are included, and the characteristic data included in the historical operation data are memory usage and bandwidth utilization. For each first neighborhood node, the memory usage and bandwidth utilization at these three sampling moments can be obtained, so the standard deviation of the memory usage and bandwidth utilization of each first neighborhood node can be calculated. The standard deviation is the feature corresponding to each first neighborhood node in the preset time window.

[0091] Then we need to obtain multi-scale neighborhood fusion features based on graph convolutional neural network. Specifically, Figure 2 As shown, the graph convolutional neural network includes an input layer, multiple graph convolutional layers, and an output layer, wherein the input layer is used to input the second feature matrix and the second adjacency matrix, and the multiple graph convolutional layers are used to obtain multi-scale neighborhood fusion features and output them by the output layer. For example, for any graph convolutional layer, it is denoted as Layer, you need to perform the following operations, first get the node and nodes In the The feature vector of the layer and , where the nodes For nodes One of the corresponding first neighbor nodes, if ,but is the node in the first characteristic matrix and nodes The corresponding eigenvector is then calculated to obtain the node The timing weight of for:

[0092]

[0093] in, and is a trainable weight matrix, || represents vector concatenation, Representation node The standard deviation of memory usage, Representation node Repeat the above steps to get the timing weight of each first neighbor node.

[0094] Then combine the nodes The static weight calculation of the node Aggregation features:

[0095]

[0096] in, Representation node The static weight of the node is calculated in the same way as same, Representation node The timing weight of the node is calculated in the same way as same, Representation node In the The feature vector of the layer, For the The trainable weight matrix of the layer.

[0097] Then through the activation function Get the node In the The feature vector of the layer , that is, the first fusion feature:

[0098]

[0099] Furthermore, in order to capture richer neighborhood features, a second fusion feature can be obtained based on the second neighborhood node. The second neighborhood node is a node connected to the first neighborhood node by an edge, and the second neighborhood node does not include the node The steps of obtaining the second fusion feature based on the features of the second neighborhood node are the same as those of obtaining the first fusion feature, which will not be described in detail in this application. After that, the first fusion feature and the second fusion feature can be integrated by weighted fusion to obtain the node Multi-scale neighborhood fusion features :

[0100]

[0101] in, and is the trainable fusion weight, at this time the node Multi-scale neighborhood fusion features That is the node In the The feature vector of the layer serves as the basis for calculations in the next layer. Based on the parallel computing principle of graph convolutional neural networks, the multi-scale neighborhood fusion features corresponding to each node can be obtained simultaneously. Ultimately, the output layer outputs the multi-scale neighborhood fusion features of each node obtained in the last graph convolution layer to participate in subsequent calculation steps.

[0102] This embodiment takes into account that in an actual network environment, the relationship between nodes is not static, but will change over time. Therefore, this embodiment introduces temporal weights and static weights. Compared with the graph convolutional network in the prior art that simply averages or weighted averages the characteristics of neighborhood nodes when aggregating neighborhood information, static weights can help the model learn the characteristics of the node itself, and temporal data can reflect the changes in node status over time, which can help the model capture the dynamic characteristics of the network, so that the model can better adapt to the dynamic changes of the network environment and improve the real-time detection capability of abnormal situations.

[0103] Furthermore, this embodiment provides a step of obtaining a weighted hidden state based on the training subset and the long short-term memory network in the training model, including:

[0104] According to the training subset, the input feature vector corresponding to each time step in the long short-term memory network is obtained;

[0105] Obtain the calculation results corresponding to the forget gate, input gate, output gate and candidate calculation unit according to the input feature vector;

[0106] Update the memory unit according to the calculation results of the forget gate and the input gate;

[0107] According to the calculation results of the output gate and the updated memory unit, the hidden state is obtained;

[0108] According to the hidden state and attention mechanism, the weighted hidden state is obtained.

[0109] Specifically, since each training subset contains data at only one sampling moment, in order to obtain the input feature vector corresponding to each time step in the long short-term memory network, it is necessary to arrange each training subset in chronological order and obtain multiple training subsets located after the current training subset, so as to obtain a total of training subsets, That is the number of time steps, preferably, , and then the input feature vector corresponding to each time step is obtained according to the second feature matrix corresponding to each training subset.

[0110] For example, Figure 3 As shown, for the node , get the nodes in each second characteristic matrix above The corresponding feature vector is used as the input feature vector for each time step according to the time sequence. Then repeat the following steps in each time step. As an example, let’s first calculate the forget gate:

[0111]

[0112] in, is the hidden state obtained in the previous time step, is the input feature vector corresponding to the current time step, is the weight matrix in the forget gate, is the bias vector in the forget gate, Represents the Sigmoid function.

[0113] Simultaneously calculate the input gate and candidate memory cells ,in and are the weight matrices in the input gate and candidate memory unit, respectively. and are the bias vectors in the input gate and candidate memory unit, respectively.

[0114] Then, based on the calculation results of the forget gate and the input gate, the memory unit is updated ,in is the memory unit updated in the previous time step. Then calculate the output gate ,in is the weight matrix in the output gate, is the bias vector in the output gate.

[0115] Then use the result of the output gate and the updated memory unit to calculate the hidden state , and calculate the attention weight based on the hidden state , and then get the weighted hidden state ,in is the weight matrix used to calculate the attention weight. Based on the parallel computing principle of the long short-term memory network, the weighted hidden state corresponding to each node can be obtained simultaneously.

[0116] Furthermore, this embodiment provides a step of fusing the multi-scale neighborhood fusion feature and the weighted hidden state through a gating mechanism to obtain a fusion feature, including:

[0117] According to the weight matrix corresponding to the gating mechanism, the multi-scale neighborhood fusion features and weighted hidden states are linearly transformed;

[0118] According to the result of linear transformation and activation function, the gate value is obtained;

[0119] According to the gate value, the multi-scale neighborhood fusion features and the weighted hidden state are fused to obtain the fusion features.

[0120] Specifically, the node For example, the gating formula is:

[0121]

[0122] in is the weight matrix of the gating mechanism, Indicates that and Splicing, is the bias vector of the gating mechanism, Nodes output by the output layer Multi-scale neighborhood fusion features, is the weighted hidden state obtained at the last time step, is the gate value.

[0123] Get nodes based on gating value Fusion Features , repeat the above steps to obtain the fusion features of each node.

[0124] This embodiment takes into account that in practical applications, data often has both spatial and temporal complexity. For example, there are connection relationships between nodes, and the memory usage of nodes also has the characteristic of dynamic change in time. Therefore, the fusion of multi-scale neighborhood fusion features and weighted hidden states can better adapt to complex data structures and make full use of various information in the data, thereby improving the model's ability to handle complex problems.

[0125] Furthermore, this embodiment provides a step of calculating a loss function based on fusion features and a fully connected layer, including:

[0126] Input the fused features into the fully connected layer to obtain the output value;

[0127] Convert the output value into a probability value according to the activation function;

[0128] According to the probability value, the loss function is calculated.

[0129] Specifically, for the node , input the fusion features into the classifier (such as the fully connected layer) for abnormal classification. Assume that the weight matrix of the fully connected layer is , the bias vector is , then the output of the classifier is:

[0130]

[0131] The softmax function is used to convert the output value into a probability value. is a vector, each element in the vector represents a node The probability of being normal or abnormal. Repeat the above steps for each node to get the output corresponding to each node. Historical operation data acquisition node The true label at the current moment, and according to the true label and Calculate the node The cross entropy loss of , takes the average of the cross entropy loss corresponding to each node as the loss function.

[0132] Afterwards, the parameters in all models (graph convolutional neural network, long short-term memory network and classifier) ​​(all trainable weight matrices and weight coefficients in graph convolutional neural network, all weight matrices and bias vectors in long short-term memory network and weight matrices and bias vectors in classifier) ​​are updated according to the loss function and backpropagation algorithm. The method of updating parameters according to the loss function is a prior art and will not be described in detail in this application.

[0133] Furthermore, this embodiment provides a step of determining an abnormal source node based on the second abnormal node, including:

[0134] Cluster the feature data according to the clustering algorithm to obtain multiple clusters, each cluster corresponds to a node set;

[0135] Calculate the distance between each second abnormal node and its corresponding cluster center;

[0136] Determine the abnormal source node based on the distance.

[0137] Furthermore, this embodiment provides a step of determining an abnormal source node based on distance, including:

[0138] If the distance between a second abnormal node and its corresponding cluster center exceeds a preset distance, the second abnormal node is regarded as the abnormal source node, and the preset distance is determined according to the average distance of the clusters corresponding to the cluster centers;

[0139] If the distance between a second abnormal node and its corresponding cluster center does not exceed the preset distance, obtaining its corresponding first adjacent node according to the first adjacency matrix;

[0140] The abnormal source node is determined based on the distance between the first adjacent node and its corresponding cluster center.

[0141] Specifically, after obtaining the second abnormal node, clustering is performed according to the characteristic data corresponding to each node to obtain multiple cluster clusters. Then, the distance between each second abnormal node and its corresponding cluster center is calculated. For each second abnormal node, if the distance to its corresponding cluster center exceeds the preset distance, the second abnormal node is regarded as the abnormal source node. Otherwise, it means that although the second abnormal node is abnormal, its abnormal state does not deviate significantly from the normal state, which usually means that the abnormal state is caused by the abnormality of other nodes. At this time, the first adjacent node corresponding to the second abnormal node is obtained, and the judgment step is repeated until the abnormal source node appears.

[0142] For example, Figure 4 As shown, it is assumed that node 1 is the second abnormal node, nodes 2-7 are nodes that belong to the same cluster as node 1, the cluster center in the node cluster is exactly the position of node 5, and the first adjacent node corresponding to node 1 is node 2-4. First, the distance between node 1 and the cluster center is calculated. Then the average distance of the cluster is calculated, that is, the mean of the distance from each node in the cluster to the cluster center. Preferably, twice the average distance is used as the preset distance. Since the distance between node 1 and the cluster center does not exceed the preset distance, the distance between nodes 2-4 and the cluster center needs to be calculated. Among them, the distance between node 4 and the cluster center exceeds the preset distance, so node 4 is regarded as the abnormal source node that causes the abnormality of node 1.

[0143] Since this embodiment performs clustering based on data features rather than location features, the distance between the second abnormal node and the cluster center actually refers to the difference between the feature data corresponding to the second abnormal node and the feature data corresponding to the cluster center (which can be calculated using the Euclidean distance), and is not the actual distance between the node positions. Figure 4 The distance between the positions actually represents the difference between the data. The farther the distance, the greater the data difference.

[0144] This embodiment identifies the abnormal source node through clustering. Clustering is performed based on the data features of the nodes. Nodes in the same cluster have similar data features. If the distance between a node and the cluster center is too large, it means that its data features are significantly different from those of other nodes in the cluster. In a network environment, this reflects that certain indicators of the node (such as CPU utilization, interface traffic, etc.) are significantly different from those of other normal nodes, and these unique features are related to abnormal situations.

[0145] This embodiment provides a method for network topology status monitoring based on IP probes. It obtains device information in real time through a network topology map, providing a global perspective for anomaly detection. It then uses a fusion analysis of a graph neural network and an LSTM long short-term memory network to capture anomaly features from multiple angles, thereby improving the accuracy of anomaly identification.

[0146] Furthermore, an embodiment of the present application provides a system for monitoring network topology status based on an IP probe, comprising:

[0147] a collection module, configured to obtain characteristic data of each node in a current network topology graph based on an IP probe, and determine a first abnormal node from each node based on the characteristic data and a first preset threshold, wherein the network topology graph is composed of a plurality of nodes and a plurality of edges, each node corresponding to a network device, and each edge representing a connection relationship between two network devices;

[0148] A computing module is configured to construct a first feature matrix based on the feature data and obtain a first adjacency matrix based on the current network topology; and input the first feature matrix and the first adjacency matrix into a preset model to obtain an anomaly score corresponding to the first abnormal node. The preset model is trained based on the historical operating data of each node and includes a graph convolutional neural network and a long short-term memory network.

[0149] The positioning module is configured to determine a second abnormal node in the first abnormal node according to the abnormal score and a second preset threshold, and determine an abnormal source node according to the second abnormal node.

[0150] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0152] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0153] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A method for monitoring network topology status based on IP probes, characterized in that: include: Acquiring characteristic data of each node in a current network topology graph according to an IP probe, and determining a first abnormal node from each node according to the characteristic data and a first preset threshold, wherein the network topology graph is composed of a plurality of nodes and a plurality of edges, each node corresponding to a network device, and each edge representing a connection relationship between two network devices; constructing a first characteristic matrix according to the characteristic data, and obtaining a first adjacency matrix according to the current network topology graph; Inputting the first feature matrix and the first adjacency matrix into a preset model to obtain an anomaly score corresponding to the first abnormal node, wherein the preset model is trained based on the historical operation data of each node, and the preset model includes a graph convolutional neural network and a long short-term memory network; Determine a second abnormal node among the first abnormal nodes according to the abnormal score and a second preset threshold, and determine an abnormal source node according to the second abnormal node; The preset model is trained based on the historical operation data of each node, and includes: Dividing the historical operation data into a training set and a validation set, and dividing the training set into multiple training subsets, each training subset corresponding to a moment; For each training subset, performing a first iterative operation, the first iterative operation including: obtaining a current training model, generating a second feature matrix and a second adjacency matrix based on the training subset, inputting the second feature matrix and the second adjacency matrix into the graph convolutional neural network of the current training model to obtain a multi-scale neighborhood fusion feature, obtaining a weighted hidden state based on the training subset and the long short-term memory network in the current training model, fusing the multi-scale neighborhood fusion feature and the weighted hidden state through a gating mechanism to obtain a fused feature, calculating a loss function based on the fused feature and a fully connected layer, and updating the current training model according to the loss function until a preset termination condition is met to obtain a training model; Obtaining the preset model according to the training model and the verification set; The second feature matrix and the second adjacency matrix are input into the graph convolutional neural network of the training model to obtain multi-scale neighborhood fusion features, including: Determining the static weight of each node according to the number of edges and weight corresponding to each node in the training subset; Determining a first neighboring node corresponding to each node according to the second adjacency matrix, and calculating features corresponding to the first neighboring node according to a preset time window; Obtaining a timing weight corresponding to each of the first neighborhood nodes according to the features corresponding to the first neighborhood nodes; The multi-scale neighborhood fusion feature is obtained according to the static weight, the temporal weight, and the feature vector of each node, and the feature vector is obtained according to the second feature matrix.

2. The method for realizing network topology status monitoring based on IP probe according to claim 1, characterized in that: Obtaining the multi-scale neighborhood fusion feature according to the static weight, the temporal weight, and the feature vector of each node includes: Determining a second neighboring node corresponding to the first neighboring node according to the second adjacency matrix, and calculating a feature corresponding to the second neighboring node according to the preset time window; Obtaining a timing weight corresponding to each of the second neighborhood nodes according to the features corresponding to the second neighborhood nodes; Obtaining a first fused feature according to the static weight, the temporal weight corresponding to the first neighborhood node, and the feature vector of each node; Obtaining a second fused feature according to the static weight, the temporal weight corresponding to the second neighborhood node, and the feature vector of each node; The multi-scale neighborhood fusion feature is obtained according to the first fusion feature and the second fusion feature.

3. The method for realizing network topology status monitoring based on IP probe according to claim 1, characterized in that: The step of obtaining a weighted hidden state based on the training subset and the long short-term memory network in the training model includes: Obtaining an input feature vector corresponding to each time step in the long short-term memory network based on the training subset; Obtain calculation results corresponding to the forget gate, input gate, output gate, and candidate calculation unit according to the input feature vector; Update the memory unit according to the calculation results of the forget gate and the input gate; Obtaining a hidden state according to the calculation result of the output gate and the updated memory unit; The weighted hidden state is obtained according to the hidden state and the attention mechanism.

4. The method for realizing network topology status monitoring based on IP probe according to claim 1, characterized in that: The multi-scale neighborhood fusion feature and the weighted hidden state are fused through a gating mechanism to obtain a fusion feature, including: performing a linear transformation on the multi-scale neighborhood fusion features and the weighted hidden state according to a weight matrix corresponding to the gating mechanism; According to the result of linear transformation and activation function, the gate value is obtained; According to the gate value, the multi-scale neighborhood fusion feature and the weighted hidden state are fused to obtain the fusion feature.

5. The method for realizing network topology status monitoring based on IP probe according to claim 1, characterized in that: The loss function is calculated based on the fusion features and the fully connected layer, including: Inputting the fused features into the fully connected layer to obtain an output value; Converting the output value into a probability value according to an activation function; The loss function is calculated based on the probability value.

6. The method for realizing network topology status monitoring based on IP probe according to claim 1, characterized in that: Determining the abnormal source node according to the second abnormal node includes: Clustering the feature data according to a clustering algorithm to obtain a plurality of clusters, each cluster corresponding to a node set; Calculating the distance between each of the second abnormal nodes and its corresponding cluster center; The abnormal source node is determined according to the distance.

7. The method for realizing network topology status monitoring based on IP probe according to claim 6, characterized in that: Determining the abnormal source node according to the distance includes: If the distance between a second abnormal node and its corresponding cluster center exceeds a preset distance, the second abnormal node is regarded as the abnormal source node, and the preset distance is determined according to the average distance of the clusters corresponding to the cluster centers; If the distance between the second abnormal node and its corresponding cluster center does not exceed the preset distance, obtaining its corresponding first adjacent node according to the first adjacency matrix; The abnormal source node is determined according to the distance between the first adjacent node and its corresponding cluster center.

8. A system for monitoring network topology status based on an IP probe, for implementing a method for monitoring network topology status based on an IP probe according to any one of claims 1 to 7, characterized in that: include: a collection module, configured to obtain characteristic data of each node in a current network topology graph based on an IP probe, and determine a first abnormal node from each node based on the characteristic data and a first preset threshold, wherein the network topology graph is composed of a plurality of nodes and a plurality of edges, each node corresponding to a network device, and each edge representing a connection relationship between two network devices; a computing module, configured to construct a first feature matrix based on the feature data, and obtain a first adjacency matrix based on the current network topology; and input the first feature matrix and the first adjacency matrix into a preset model to obtain an anomaly score corresponding to the first abnormal node, wherein the preset model is trained based on the historical operating data of each node, and the preset model includes a graph convolutional neural network and a long short-term memory network; A positioning module is configured to determine a second abnormal node among the first abnormal nodes according to the abnormal score and a second preset threshold, and determine an abnormal source node according to the second abnormal node.

Images